Franka Panda Three-Cube Stacking: 1,000 MimicGen Demonstrations plus Cosmos Transfer1 Photorealistic Augmentation (NVIDIA Isaac Lab)

Provider
NVIDIA Corporation
License
CC-BY-4.0 (source)
Size
104 files
Format
HDF5, JSON (in sampled files)
Stored at
Hugging Face (https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Augmented)
Access
Free and public. No account with the data provider is needed.
Category
Robotics
Source
https://arxiv.org/abs/2503.14492

About this dataset

Synthetic robot manipulation demonstrations from NVIDIA in which a single Franka Panda arm stacks three cubes in a fixed order (blue bottom, red middle, green top) in Isaac Lab. Ten human teleoperation demonstrations were expanded to 1,000 trajectories with Isaac Lab's Mimic tool (MimicGen), and the table-camera RGB of every trajectory was then re-rendered with the Cosmos Transfer1 world model for photorealistic domain adaptation, so the same motions exist with plain simulator images and with augmented, realistic-looking images. Each demonstration holds a 7-D action (6-D relative end-effector motion plus gripper), joint positions and velocities, end-effector pose, the three cube poses, and 200x200 table-camera RGB, depth, segmentation and surface-normal images plus wrist-camera RGB. The two sets are single HDF5 files, mimic_dataset_1k.hdf5 and cosmos_dataset_1k.hdf5, readable with h5py and replayable with Isaac Lab's replay_demos.py; a helper script exports MP4 video, and robomimic BC-RNN visuomotor checkpoints are included. About 78 GB (measured 2026-11-21). Use it for behavior cloning, sim-to-real visual transfer and data augmentation studies. NVIDIA releases it under CC-BY-4.0 and states it is ready for commercial use; cite MimicGen (Mandlekar et al., CoRL 2023) and Cosmos-Transfer1 (arXiv:2503.14492).

How to load this dataset in Python

from huggingface_hub import hf_hub_download
path = hf_hub_download(repo_id="nvidia/PhysicalAI-Robotics-Manipulation-Augmented", filename="cosmos_dataset_1k.hdf5", repo_type="dataset")
print(path)

Data structure

File types in sampled files: HDF5, JSON

Files

Example file: cosmos_dataset_1k.hdf5

Analyze this dataset without downloading it

On Scigantic, Franka Panda Three-Cube Stacking: 1,000 MimicGen Demonstrations plus Cosmos Transfer1 Photorealistic Augmentation (NVIDIA Isaac Lab) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Robotics datasets