Nowadays, the digital documentation of architectural heritage necessarily requires the integration of different types of representation and the organization of information on different levels, in order to plan adequate restoration and conservation operations [1]. Semantic segmentation techniques relying on Artificial Intelligence are emerging in this field as privileged tools to appropriately organize, structure and classify the complex system of analytical and survey data related to an architectural object or site [2]–[4]. In this contribution, semi-automatic classification methods are exploited in order to associate the different semantic and descriptive information to the raw outputs of the three-dimensional survey and H-BIM based representations are finally created to display the results. The case study on which the methodological approach is tested is a church located in Pisa (Italy), Chiesa del Carmine: the classification is performed on the liturgical decorative apparatus of the church, and the textured meshes of the altars are analyzed in order to characterize the state of preservation, and in particular the material and decay mapping of these objects. Taking as input data the raw 3D models derived from laser scanning or photogrammetry, a supervised Machine Learning (ML) algorithm is applied in order to read, classify and return different degrees of degradation and/or types of materials of the altars. In detail, starting from the 3D survey, orthophotos and UV maps are generated. On these, the classes of materials or the levels of the degradation are identified and annotated over a reduced portion, and this constitutes a set of samples on which the ML model is trained. The training data are also supported by the so-called features, i.e. radiometric or geometric characteristics that allow to distinguish one class from another [5]. A predictive model (Random Forest) is trained on these data so as to foresee and map the classification of the entire dataset. Once this supervised classification is performed on the orthophoto or UV map, the results are projected onto the 3D by exploiting the projective relationships between images and model, and this allows to obtain an overall mesh model in which different colors correspond to different degrees of degradation or different types of material. This distinction into classes is preserved even at a final stage, when the classified mesh models derived from the segmentation process are inserted within specific H-BIM platforms, in a Scan-to-BIM application. To this end, portions of mesh having different material and degradation characteristics are imported into BIM platforms thanks to visual programming algorithms implemented in Rhino’s Grasshopper. This step ensures the autonomous management and informatization of each class derived from segmentation, and the semantic data can be more easily shared, retrieved, visualized and stored, also in view of the use of heritage models for augmented reality applications. The results obtained in terms of description and semantic mapping of the model and of traceability and retrieval of information in H-BIM environment suggest the extension of the proposed methodological approach to the study of ornamental apparatuses related to other churches of the Carmelite order.