Communication Dans Un Congrès Année : 2025

WaterLLMarks: In-data User Tracing for Distributed LLMaaS Environments

Résumé

Due to their size and cost of operation, Large Language Models (LLMs) are often deployed in the cloud on Large Language Model as a Service (LLMaaS) platforms. Following the trends in cloud-based services and multi-tenancy, these platforms are becoming increasingly distributed. Monitoring such environments is challenging, especially without access to the underlying infrastructure. Moreover, applications may rely on multiple services from different providers, further complicating end-to-end observability. In this paper, we propose a novel type of telemetry, termed in-data telemetry, where services are outsourced across multiple LLMaaS providers. We illustrate its application in tracing user interactions with LLMs by embedding user-specific watermarks in the text. These watermarks propagate transparently through multiple services with minimal impact on the semantics of the generated content. We evaluate our approach in a Retrieval-Assisted Generation (RAG) scenario and showcase its potential in a demonstrator.

Fichier principal
Vignette du fichier
WaterLLMarks__Demo_ICDCS_2025_.pdf (851.51 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05042411 , version 1 (22-04-2025)

Licence

Identifiants

  • HAL Id : hal-05042411 , version 1

Citer

Léo Lavaur, Jérôme François. WaterLLMarks: In-data User Tracing for Distributed LLMaaS Environments. 2025 IEEE International Conference on Distributed computing Systems (ICDCS), Jul 2025, Glasgow, United Kingdom. ⟨hal-05042411⟩
112 Consultations
105 Téléchargements

Partager

  • More