Silk is a particularly important material in European history. Its trade has facilitated the growth of new economic centres, stimulated the development of innovative weaving techniques, and allowed the creation of exceptional objects. While many institutions specialise in its conservation, this fragile Cultural Heritage is under threat. In this article, we explain the methodology used by the Horizon 2020 SILKNOW project to better promote these Cultural Heritage collections. We present the CIDOC CRM-based data model that we designed for the creation of a Knowledge Graph. We also present the methodology used to integrate, in the data model, the annotations generated by the analysis of texts and images. Finally, we present the SIlKNOW extension, designed to model the complex semantics describing the production process of silk fabrics.