Breaking the 3D Dataset Bottleneck: Fast Scalable Generation of Aligned 3D Assets from Scratch for 6D Pose Estimation and Robotic Grasping
Résumé
We introduce a scalable framework for generating category-level 6D pose datasets from text prompts automatically. Our approach addresses three key bottlenecks in the creation of 3D datasets: (1) automated asset generation through a controlled text-to-image-to-3D pipeline; (2) built-in canonical alignment through depth-conditioned generation; and (3) large-scale 6D annotation using mixed reality rendering. GenNOCS produces high-quality aligned 3D meshes in under 3min per object, achieving a 5--20 times speedup over traditional scanning---with a 96 percent mesh generation success rate including consistent pose alignment. We demonstrate the applicability of our pipeline by integrating these meshes in state-of-the-art sim2real 6D pose generation. We demonstrate sim2real transfer on the NOCS benchmark for 6D pose estimation with competitive results in a zero-shot manner. Finally, we confirm the practical utility of our generated assets in real-world robotic grasping scenarios. By eliminating dependencies on existing 3D assets, our method enables rapid creation of custom 6D datasets, achieving 6000 aligned instances over 6 categories of the NOCS benchmark with only 20min of human effort. This work provides a critical step toward foundation models for 3D understanding, offering new possibilities for research in 6D perception and manipulation for custom datasets. The code and dataset from prompt to 6D pose dataset generation are publicly available at https://huggingface.co/datasets/Guillaume0477/GenNOCS.
Domaines
| Origine | Fichiers produits par l'(les) auteur(s) |
|---|---|
| Licence |