In this paper, we propose a solution for detecting changes in the behaviour of the elderly person based on the monitoring of activities of daily living (ADL). The elderly person's daily routine is characterized by the following five indexes: 1) percentage of time lying down, 2) percentage of time sitting, 3) percentage of time standing, 4) percentage of time absent from home, and 5) number of falls during the day. In our framework, these indexes are computed using characteristics extracted from depth and thermal data. We hypothesize that elderly persons have a well-defined, regular life routine, organized around their environment, habits, and social relations. Then, given the indexes values, a day is defined as routine or non-routine day. Thus, looking for changes of day type allows to detect changes in a person's routine. The method has been tested on a database of depth and thermal images recorded in a nursing home over an 85 days period. These tests proved the reliability of the proposed method. Clinical Relevance-Monitoring the daily routine provides indicators for detecting changes in the behaviour of an elderly person.