The simultaneous heat and moisture transfer in the building envelope has an important influence on the indoor environment and the overall performance of buildings. In this paper, a model for predicting whole building heat and moisture transfer was presented. Both heat and moisture transfer in the building envelope and indoor air were simultaneously considered; their interactions were modeled. The coupled model takes into account most of the main hygrothermal effects in buildings. The coupled system model was implemented in MATLAB-Simulink, and validated by using a series of published testing tools. The new program was applied to investigate the moisture transfer effect on indoor air humidity and building energy consumption under different climates. The results show that the use of more detailed simulation routines can result in improvements to the building's design for energy optimisation through the choice of proper hygroscopic materials, which would not be indicated by simpler calculation techniques.