PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking

Résumé

Estimating the relative pose of a new object without prior knowledge is a hard problem, while it is an ability very much needed in robotics and Augmented Reality. We present a method for tracking the 6D motion of objects in RGB video sequences when neither the training images nor the 3D geometry of the objects are available. In contrast to previous works, our method can therefore consider unknown objects in open world instantly, without requiring any prior information or a specific training phase. We consider two architectures, one based on two frames, and the other relying on a Transformer Encoder, which can exploit an arbitrary number of past frames. We train our architectures using only synthetic renderings with domain randomization. Our results on challenging datasets are on par with previous works that require much more information (training images of the target objects, 3D models, and/or depth data). Our source code is available at https://github.com/nv-nguyen/pizza

Dates et versions

hal-03793789 , version 1 (02-10-2022)

Identifiants

Citer

van Nguyen Nguyen, Yuming Du, Yang Xiao, Michael Ramamonjisoa, Vincent Lepetit. PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking. International Conference on 3D Vision (3DV), Sep 2022, Prague, Czech Republic. ⟨hal-03793789⟩
68 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More