The use of grey-box models for short-time forecasting of buildings’ thermal behavior requires the determination of the models’ order since this order could influence the grey-box models’ performance. This paper presents an analysis of the optimal order of these models for different thermal conditions. The novelty of this work consists of considering the influence of the heating conditions on the determination of the performances of grey-box models. The analysis is based on experimental tests that were conducted in a room with different thermal conditions, related to the variation of the heating power. Experimental results were used for the determination of the optimal grey-box models’ order that minimizes the gap between the experimental results and the grey-box forecasting. Results show that the optimal grey-box models’ order depends on the buildings’ thermal conditions, but generally lies between two and three with an error less than 0.2 °C and a fit percent greater than 90%.