Communication Dans Un Congrès Année : 2025

Evaluating Multilingual Small Language Models in Medical and Educational Settings

Résumé

Large Language Models are becoming more performant in different tasks, such as summarizing, chatbot dialogue or question answering. Moreover, LLMs are still not reliable enough to be used as plug-and-play tools in medical or educational contexts, especially in a multilingual context (which reflects the most a real world usage of these tools). Multilingual LLMs are not equally performant on all languages, their performance being lower on languages other than English. Furthermore, LLMs are not adapted to understand the complexity of the medical knowledge, nor to be learning assistants for young students. We evaluated the multilingual Small Language Models (less than 3 billion parameters) performance on QA tasks in three languages, English, French, and Romanian, a less resourced language. We present results on the model’s capabilities in answering medical research questions of different levels of complexity in terms of technical knowledge and reasoning, and assisting 11 year-old schoolchildren with their homework.

Fichier non déposé

Dates et versions

hal-05069660 , version 1 (15-05-2025)

Identifiants

  • HAL Id : hal-05069660 , version 1

Citer

Ioana Buhnila. Evaluating Multilingual Small Language Models in Medical and Educational Settings. Invited talk at the Research Seminar in Language Technology – Spring 2025, University of Helsinki (online), May 2025, Helsinki, Finland. ⟨hal-05069660⟩
70 Consultations
0 Téléchargements

Partager

  • More