Communication Dans Un Congrès Année : 2025

How Good are Multilingual Small Language Models at Explaining Medical Knowledge?

Résumé

Large Language Models (LLMs) became easily accessible to the general public through chatbots like ChatGPT (OpenAI, 2022). The general public use language models for many tasks, including for health related questions (Singhal et al., 2023). Nevertheless, medical terms, which are specialized lexical units (Condamines and Rebeyrolle, 1997) can be difficult to understand for laypeople (LeBlanc et al., 2014; Tavakoly Sany et al., 2020). Thus, medical concepts have to be simplified through paraphrases or definitions, such as "arthrosis of the thumb" for rhizarthrosis. In this sense, we tested an open-access multilingual medical LLM, BioMistral (Labrak et al., 2024), to assess the quality of its answers in a downstream task: people asking medical-related questions to a language model. We evaluated the zero-shot performance of BioMistral in two languages: French and Romanian, a low resource-language. French results show that BioMistral’s answers in French are 94% correct using an English prompt, while in Romanian only 70% are correct, as the language might be rare in the corpus of the language model. However, when using a Romanian prompt, the model’s performance rises to 94%.

Fichier non déposé

Dates et versions

hal-05152584 , version 1 (09-07-2025)

Identifiants

  • HAL Id : hal-05152584 , version 1

Citer

Ioana Buhnila. How Good are Multilingual Small Language Models at Explaining Medical Knowledge?. NéALA 2025 Conference, "The Natural and the Artificial in Applied Linguistics: a time of paradoxes", Jul 2025, Nancy, France. ⟨hal-05152584⟩
45 Consultations
0 Téléchargements

Partager

  • More