In space, no one (but AI) can hear you scream - Laboratoire Interdisciplinaire des Sciences du Numérique
Conference Poster Year : 2024

In space, no one (but AI) can hear you scream

Alexander Bock
Tobias Isenberg
Lonni Besançon

Abstract

We present our initial work on integrating a conversational agent using a large-language model (LLM) in OpenSpace, to provide conversational-based navigation for astrophysics visualization software. We focus on applications of visualization for education and outreach, where the versatility and intuitiveness of conversational agents can be leveraged to provide engaging and meaningful learning experiences. Visualization benefits from the development of LLMs by leveraging its capability to understand requests in natural language, allowing users to express complex tasks efficiently. Natural Language Interfaces can be combined with more traditional visualization interaction techniques, streamlining real-time interaction and facilitating free data exploration. We thus instructed a voice-controlled GPT-4o LLM to send commands to an OpenSpace instance, effectively providing the LLM with the the ability to steer the visualization software as a museum facilitator would for educational shows. We present our implementation and discuss future possibilities.
Fichier principal
Vignette du fichier
Brossier_2024_SNO.pdf (1.42 Mo) Télécharger le fichier
Vignette du fichier
Brossier_2024_SNO.jpg (9.62 Ko) Télécharger le fichier
Origin Files produced by the author(s)
Format Figure, Image

Dates and versions

hal-04684718 , version 1 (03-09-2024)

Licence

Copyright

Identifiers

  • HAL Id : hal-04684718 , version 1

Cite

Mathis Brossier, Alexander Bock, Konrad Schönborn, Tobias Isenberg, Anders Ynnerman, et al.. In space, no one (but AI) can hear you scream. VIS 2024 - IEEE Visualization, Oct 2024, St. Pete Beach, Florida, United States. ⟨hal-04684718⟩
39 View
39 Download

Share

More