Pré-Publication, Document De Travail Année : 2025

Breaking the 3D Dataset Bottleneck: Fast Scalable Generation of Aligned 3D Assets from Scratch for 6D Pose Estimation and Robotic Grasping

Résumé

We introduce a scalable framework for generating category-level 6D pose datasets from text prompts automatically. Our approach addresses three key bottlenecks in the creation of 3D datasets: (1) automated asset generation through a controlled text-to-image-to-3D pipeline; (2) built-in canonical alignment through depth-conditioned generation; and (3) large-scale 6D annotation using mixed reality rendering. GenNOCS produces high-quality aligned 3D meshes in under 3min per object, achieving a 5--20 times speedup over traditional scanning---with a 96 percent mesh generation success rate including consistent pose alignment. We demonstrate the applicability of our pipeline by integrating these meshes in state-of-the-art sim2real 6D pose generation. We demonstrate sim2real transfer on the NOCS benchmark for 6D pose estimation with competitive results in a zero-shot manner. Finally, we confirm the practical utility of our generated assets in real-world robotic grasping scenarios. By eliminating dependencies on existing 3D assets, our method enables rapid creation of custom 6D datasets, achieving 6000 aligned instances over 6 categories of the NOCS benchmark with only 20min of human effort. This work provides a critical step toward foundation models for 3D understanding, offering new possibilities for research in 6D perception and manipulation for custom datasets. The code and dataset from prompt to 6D pose dataset generation are publicly available at https://huggingface.co/datasets/Guillaume0477/GenNOCS.

Fichier principal
Vignette du fichier
Formatting_Instructions_For_NeurIPS_2025 (2).pdf (114.51 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Licence

Dates et versions

hal-05220360 , version 1 (23-08-2025)
hal-05220360 , version 2 (24-08-2025)
hal-05220360 , version 3 (26-02-2026)

Licence

Identifiants

  • HAL Id : hal-05220360 , version 1

Citer

Guillaume Duret, Danylo Mazurak, Florence Zara, Jan Peters, Liming Chen. Breaking the 3D Dataset Bottleneck: Fast Scalable Generation of Aligned 3D Assets from Scratch for 6D Pose Estimation and Robotic Grasping. 2025. ⟨hal-05220360v1⟩
981 Consultations
138 Téléchargements

Partager

  • More