Now you see me: finding the right observation space to learn diverse behaviours by reinforcement in games - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Now you see me: finding the right observation space to learn diverse behaviours by reinforcement in games

Abstract

Training virtual agents to play a game using reinforcement learning (RL) has gained a lot of traction in recent years. Indeed, RL has delivered agents with superhuman performances on multiple gameplays. Yet, from a human-machine interaction standpoint, raw performance is not the only dimension of a "good" game AI. Exhibiting diversified behaviours is key to generate novelty, one of the core components of player engagement. In the RL framework, teaching agents to discover multiple strategies to achieve the same task is often framed as skill discovery. However, we observe that the current RL literature defines diversity as the exploration of different states, i.e. the incentive of the agent to "see" new observations. In this work, we argue that this definition does not make sense from a gameplay point of view. Instead, diversity should be defined as a distance on observations from an observer, external to the agent. We illustrate how DIAYN/SMERL, state of the art RL algorithms for skill discovery, fail to discover meaningful behaviours in a simple tag game. We propose an easy fix by introducing the notion of diversity spaces, defined as the observations gathered by a third-party external to the agent.
Fichier principal
Vignette du fichier
CAp2022_paper_0257.pdf (1.41 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03678280 , version 1 (25-05-2022)

Identifiers

  • HAL Id : hal-03678280 , version 1

Cite

Raphaël Boige, Nicolas Audebert, Clément Rambour, Guillaume Levieux. Now you see me: finding the right observation space to learn diverse behaviours by reinforcement in games. Conférence sur l'Apprentissage automatique (CAp), Jul 2022, Vannes, France. ⟨hal-03678280⟩
155 View
71 Download

Share

Gmail Facebook X LinkedIn More