Bio-based materials, such as hemp concrete, have been widely recommended to reduce carbon emissions and energy consumption of buildings as a means of addressing current environmental problems. These materials may possess very interesting hygrothermal and acoustic performance. Hemp concrete is composed of hemp particles embedded in a binder that forms a very heterogeneous and porous component. The aim of this paper is to examine the influence of hemp concrete heterogeneity on the mechanisms of heat and mass transfer. The originality of this paper is to consider the real 3D structure of hemp concrete for modelling coupled heat and moisture transfers within the material. These 3D microstructures were obtained using 3D X-ray tomography reconstruction with a voxel size of 31.8 µm. Then, a specific finite element mesh was generated from the real, heterogeneous geometry of hemp shives, binder and interparticle air. A mesoscopic model was developed to simulate coupled heat and mass transfer phenomena within the material. The 3D temperature and relative humidity fields showed high heterogeneity and complex distributions that are governed by the hemp concrete morphology. In addition, the anisotropy of the material led to different effective thermal conductivities in each transfer direction. Numerical comparison with simulations performed on the fictive elementary representative volume of hemp concrete showed that consideration of the real geometry allows a better understanding of coupled heat and moisture transfer phenomena modelling.