In the Ontology Learning research domain, despite recent advancements, the performance of current non or semi-supervised approaches for concept formation remains sub-optimal, particularly from a single, small-sized corpus for a specialized domain. In order to answer the performance drawback, this paper introduces a novel pipeline, called CO-ISSC (Core Ontology-based Incremental Semi-Supervised Clustering), for concept formation towards ontology learning. This pipeline uses a PLM (Pre-trained Language Model) and combines in an incremental manner a semi-supervised dimension reduction technique and a clustering technique, guided by core concepts as prior knowledge to align results with the ontology domain. Its incremental nature enhances prior knowledge and boosts its performance. The CO-ISSC pipeline’s performance is evaluated on the recent and significant AI Act text established by the European Union, which aims to ensure the safety, transparency, and non-discrimination of AI systems. To this end, we manually built a benchmark terminology for the AI Act domain given that no reference model exists yet. The results demonstrate promising performance of the CO-ISSC pipeline, outperforming baseline non-supervised or semi-supervised approaches such as DBSCAN, similarity measure based approaches, SVM and ANN.