Deploying Open-Source Large Language Models: A Performance Analysis - Archive ouverte HAL
Pré-Publication, Document De Travail (Preprint/Prepublication) Année : 2024

Deploying Open-Source Large Language Models: A Performance Analysis

Déploiement de grands modèles de langage open-source : Une analyse des performances

Xavier Hinaut

Résumé

Since the release of ChatGPT in November 2022, large language models (LLMs) have seen considerable success, including in the open-source community, with many open-weight models available. However, the requirements to deploy such a service are often unknown and difficult to evaluate in advance. To facilitate this process, we conducted numerous tests at the Centre Inria de l'Université de Bordeaux. In this article, we propose a comparison of the performance of several models of different sizes (mainly Mistral and LLaMa) depending on the available GPUs, using vLLM, a Python library designed to optimize the inference of these models. Our results provide valuable information for private and public groups wishing to deploy LLMs, allowing them to evaluate the performance of different models based on their available hardware. This study thus contributes to facilitating the adoption and use of these large language models in various application domains.
Fichier principal
Vignette du fichier
LLM_Deployment.pdf (234.39 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-04777469 , version 1 (12-11-2024)

Licence

Identifiants

  • HAL Id : hal-04777469 , version 1

Citer

Yannis Bendi-Ouis, Dan Dutartre, Xavier Hinaut. Deploying Open-Source Large Language Models: A Performance Analysis. 2024. ⟨hal-04777469⟩
17 Consultations
39 Téléchargements

Partager

More