The reliability of mathematical models for heat and mass transfer in building porous material is of capital importance. A reliable model permits to carry predictions of the physical phenomenon with sufficient confidence in the results. Among the physical phenomena, the hysteresis effects on moisture sorption and moisture capacity need to be integrated in the mathematical model of transfer. This article proposes to explore the use of an smooth Bang-Bang model to simulate the hysteresis effects coupled with heat and mass transfer in porous material. This model adds two supplementary differential equations to the two classical ones for heat and mass transfer. The solution of these equations ensures smooth transitions between the main sorption and desorption curves. Two parameters are required to control the speed of transition through the intermediary curves. After the mathematical description of the model, an efficient numerical model is proposed to compute the fields with accuracy and reduced computational efforts. It is based on the Du Fort-Frankel scheme for the heat and mass balance equations. For the hysteresis numerical model, an innovative implicit-explicit approach is proposed. Then, the predictions of the numerical model are compared with experimental observations from literature for two case studies. The first one corresponds to a slow cycle of adsorption and desorption while the second is based on a fast cycling case with alternative increase and decrease of moisture content. The comparisons highlight a very satisfactory agreement between the numerical predictions and the observations. In the last Section, the reliability and efficiency of the proposed model is investigated for long term simulation cases. The importance of considering hysteresis effects in the reliability of the predictions are enhanced by comparison with classical approaches from literature.