Nebula: A discourse aware Minecraft Builder
Abstract
We show how incorporating the prior discourse and nonlinguistic context of a conversation situated in a nonlinguistic environment can improve the "language to action" component of such interactions. We finetune an LLM to predict actions based on prior context; our model, Nebula, doubles the net-action F1 score over the baseline on this task of Jayannavar et al. (2020). We also investigate our model's ability to construct shapes and understand location descriptions using a synthetic dataset.
Origin | Files produced by the author(s) |
---|---|
Licence |