Bottlenecks of Vision Language Models in Image to PDDL State Extraction
Résumé
The advent of Large, Pre-trained, Vision or Language Models (LLM, VLM, etc.) has lead to their wide use in multiple robotic applications. To better understand how they can be leveraged in robotics applications in generating a precise description of an environment from an image, we evaluate the generation of Natural Language (NL) descriptions from images by comparing different prompts, image-to-PDDL data and state-of-the-art Vision-Language Models (VLMs). Results reveal specific strengths and weaknesses of individual VLMs and a strong reliance on image complexity in terms of object shape, cardinality and position, when generating NL descriptions.
Domaines
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |