Multimodal Interaction for Cobot Using MQTT - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Multimodal Technologies and Interaction Année : 2023

Multimodal Interaction for Cobot Using MQTT

Résumé

For greater efficiency, human-machine and human-robot interactions must be designed with the idea of multimodality in mind. To allow the use of several interaction modalities, such as the use of voice, touch, gaze tracking, on several different devices (computer, smartphone, tablets, etc.) and to integrate possible connected objects, it is necessary to have an effective and secure means of communication between the different parts of the system. This is even more important with the use of a collaborative robot (cobot) sharing the same space and very close to the human during their tasks. This study present research work in the field of multimodal interaction for a cobot using the MQTT protocol, in virtual (Webots) and real worlds (ESP microcontrollers, Arduino, IOT2040). We show how MQTT can be used efficiently, with a common publish/subscribe mechanism for several entities of the system, in order to interact with connected objects (like LEDs and conveyor belts), robotic arms (like the Ned Niryo), or mobile robots. We compare the use of MQTT with that of the Firebase Realtime Database used in several of our previous research works. We show how a "pick-wait-choose-and place" task can be carried out jointly by a cobot and a human, and what this implies in terms of communication and ergonomic rules, via health or industrial concerns (people with disabilities, and teleoperation).
Fichier principal
Vignette du fichier
version_finale_mti-07-00078.pdf (16.16 Mo) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte

Dates et versions

hal-04180200 , version 1 (11-08-2023)

Identifiants

Citer

José Rouillard, Jean-Marc Vannobel. Multimodal Interaction for Cobot Using MQTT. Multimodal Technologies and Interaction, 2023, 7 (8), pp.78. ⟨10.3390/mti7080078⟩. ⟨hal-04180200⟩
60 Consultations
4 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More