Passive Optical Networks (PONs) are widely adopted as the last-mile technology due to the large bandwidth capacity it provides to end users. In addition, PON is viewed as a green access technology since it reduces energy consumption compared to other access technologies (e.g. Fiber to the Node). However, there is still room for enhancing the energy efficiency of PON further, and we can find many attempts along those lines in academia and industry. A widely used approach to save energy in Time Division Multiplexing (TDM)-PON is to keep the Optical Network Units (ONUs) in sleep mode when they do not have anything to receive or transmit. However, sleep intervals have a direct negative impact on increasing traffic delay. Therefore, energy efficiency in a TDM-PON presents a clear trade-off: the longer an ONU sleeps, the less energy it consumes, but the higher the delay experienced by the downlink traffic, and vice versa. In this paper, we propose an Adaptive Delay-Aware Energy Efficient (ADAEE) TDM-PON solution. The ADAEE aims at saving as much energy as possible while meeting the PON access delay restrictions imposed by the operator. We evaluate our solution in terms of energy consumption and delay performance using real traffic traces. The results demonstrate that the proposed solution can meet delay requirements while being more energy efficient solution compared to the existing solutions.