The processing of temporal intervals is essential to create causal maps and to predict future events so as to best adapt one's behavior. In this review, we explore the different brain activity patterns associated with processing durations and expressing temporally-adapted behavior in animals. We begin by describing succinctly some of the current models of the internal clock that can orient us in what to look for in brain activity. We then outline how durations can be decoded by single cell activity and which activity patterns could be associated with interval timing. We further point to similar patterns that have been observed at a more global level within brain areas (e.g. local field potentials) or, even, between these areas, that could represent another way of encoding duration or could constitute a necessary part for more complex temporal processing. Finally, we discuss to what extent neural data fit with internal clock models, and highlight improvements for experiments to obtain a more in-depth understanding of the brain's temporal encoding and processing.