Large Language Models have shown high performances in a large number of tasks, being recently applied also to support Knowledge Graphs construction. An important step for data modeling consists in the definition of a set of competency questions, which are often used as a guide for the development of an ontology and as a mean to evaluate the resulting schema. In this work, we investigate the suitability of LLMs for the automatic generation of competency questions given an existing ontology. We compare different large language models under various set-tings in order to give a comprehensive overview of what LLMs can do to support the knowledge engineer.