Intrusion Detection Systems (IDSs) are essential cybersecurity components. Previous cyberattack detection methods relied more on signatures and rules to detect cyberattacks, although there has been a change in paradigm in the last decade, with Machine Learning (ML) enabling more efficient and flexible statistical methods. However, ML is currently unable to integrate cybersecurity information into its inner workings. This paper introduces Cyber Informedness, a new metric taking into account cybersecurity information to give a more informed representation of performance, influenced by the severity of the attacks encountered. This metric uses a de facto standard in cybersecurity: the Common Vulnerability Scoring System (CVSS). Results on two public datasets show that this new metric validates results obtained with generic metrics. Furthermore, this new metric highlights ML-based IDSs that prioritize high performance on severe attacks, which is not visible with generic metrics. Consequently, this new metric nicely completes generic metrics by bridging the gap between ML and cybersecurity.