Franka Panda Three-Cube Stacking: 1,000 MimicGen Demonstrations plus Cosmos Transfer1 Photorealistic Augmentation (NVIDIA Isaac Lab)
- Provider
- NVIDIA Corporation
- License
- CC-BY-4.0 (source)
- Size
- 104 files
- Format
- HDF5, JSON (in sampled files)
- Stored at
- Hugging Face (https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Augmented)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Robotics
- Source
- https://arxiv.org/abs/2503.14492
About this dataset
Synthetic robot manipulation demonstrations from NVIDIA in which a single Franka Panda arm stacks three cubes in a fixed order (blue bottom, red middle, green top) in Isaac Lab. Ten human teleoperation demonstrations were expanded to 1,000 trajectories with Isaac Lab's Mimic tool (MimicGen), and the table-camera RGB of every trajectory was then re-rendered with the Cosmos Transfer1 world model for photorealistic domain adaptation, so the same motions exist with plain simulator images and with augmented, realistic-looking images. Each demonstration holds a 7-D action (6-D relative end-effector motion plus gripper), joint positions and velocities, end-effector pose, the three cube poses, and 200x200 table-camera RGB, depth, segmentation and surface-normal images plus wrist-camera RGB. The two sets are single HDF5 files, mimic_dataset_1k.hdf5 and cosmos_dataset_1k.hdf5, readable with h5py and replayable with Isaac Lab's replay_demos.py; a helper script exports MP4 video, and robomimic BC-RNN visuomotor checkpoints are included. About 78 GB (measured 2026-11-21). Use it for behavior cloning, sim-to-real visual transfer and data augmentation studies. NVIDIA releases it under CC-BY-4.0 and states it is ready for commercial use; cite MimicGen (Mandlekar et al., CoRL 2023) and Cosmos-Transfer1 (arXiv:2503.14492).
How to load this dataset in Python
from huggingface_hub import hf_hub_download
path = hf_hub_download(repo_id="nvidia/PhysicalAI-Robotics-Manipulation-Augmented", filename="cosmos_dataset_1k.hdf5", repo_type="dataset")
print(path)Data structure
File types in sampled files: HDF5, JSON
Files
Example file: cosmos_dataset_1k.hdf5
Analyze this dataset without downloading it
On Scigantic, Franka Panda Three-Cube Stacking: 1,000 MimicGen Demonstrations plus Cosmos Transfer1 Photorealistic Augmentation (NVIDIA Isaac Lab) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Robotics datasets
- PhysicalAI Robotics GR 00 T Teleop Sim
- DMC-VB: DeepMind Control Vision Benchmark, Locomotion and Ant Maze Episodes with Visual Distractors for Offline RL (Google DeepMind)
- IsaacLab-Arena GR1 Humanoid Microwave-Opening Task: 10 Teleoperated and 50 MimicGen Demonstrations (NVIDIA, v1)
- PhysicalAI Robotics Mindmap GR 1 Drill In Box
- MolmoAct Dataset: Robot Manipulation Demonstrations (Ai2)
- LeRobot Community Datasets v3
- PhysicalAI Robotics GR 00 T X Embodiment Sim
- GR00T N1.7 AppleToPlate: 402 Real-Robot Teleoperated Demonstrations on a Unitree G1 Humanoid (NVIDIA)