# Franka Panda Three-Cube Stacking: 1,000 MimicGen Demonstrations plus Cosmos Transfer1 Photorealistic Augmentation (NVIDIA Isaac Lab)

## Facts

- Provider: NVIDIA Corporation
- License: CC-BY-4.0 (https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Augmented)
- Size: 104 files
- Format: HDF5, JSON (in sampled files)
- Stored at: Hugging Face, https://huggingface.co/datasets/nvidia/PhysicalAI-Robotics-Manipulation-Augmented
- Access: public, readable without an account
- Category: Robotics
- Source: https://arxiv.org/abs/2503.14492
- Page: https://scigantic.com/archive/059e3d01-06e9-5dba-a9f5-b19c3519c48d
- Keywords: Robotics, image segmentation, robot learning, HDF5, JSON, Hugging Face

## About

Synthetic robot manipulation demonstrations from NVIDIA in which a single Franka Panda arm stacks three cubes in a fixed order (blue bottom, red middle, green top) in Isaac Lab. Ten human teleoperation demonstrations were expanded to 1,000 trajectories with Isaac Lab's Mimic tool (MimicGen), and the table-camera RGB of every trajectory was then re-rendered with the Cosmos Transfer1 world model for photorealistic domain adaptation, so the same motions exist with plain simulator images and with augmented, realistic-looking images. Each demonstration holds a 7-D action (6-D relative end-effector motion plus gripper), joint positions and velocities, end-effector pose, the three cube poses, and 200x200 table-camera RGB, depth, segmentation and surface-normal images plus wrist-camera RGB. The two sets are single HDF5 files, mimic_dataset_1k.hdf5 and cosmos_dataset_1k.hdf5, readable with h5py and replayable with Isaac Lab's replay_demos.py; a helper script exports MP4 video, and robomimic BC-RNN visuomotor checkpoints are included. About 78 GB (measured 2026-11-21). Use it for behavior cloning, sim-to-real visual transfer and data augmentation studies. NVIDIA releases it under CC-BY-4.0 and states it is ready for commercial use; cite MimicGen (Mandlekar et al., CoRL 2023) and Cosmos-Transfer1 (arXiv:2503.14492).

## Load the data

```python
from huggingface_hub import hf_hub_download
path = hf_hub_download(repo_id="nvidia/PhysicalAI-Robotics-Manipulation-Augmented", filename="cosmos_dataset_1k.hdf5", repo_type="dataset")
print(path)
```

## Data structure

File types in sampled files: HDF5, JSON

Example file: `cosmos_dataset_1k.hdf5`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/059e3d01-06e9-5dba-a9f5-b19c3519c48d
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/059e3d01-06e9-5dba-a9f5-b19c3519c48d
- Over MCP: `get_schema_card` with id `059e3d01-06e9-5dba-a9f5-b19c3519c48d` at https://api.scigantic.com/mcp

## Related topics

- [JSON datasets](https://scigantic.com/topics/json)
- [Image segmentation datasets](https://scigantic.com/topics/image-segmentation)
- [HDF5 datasets](https://scigantic.com/topics/hdf5)
- [Robotics datasets](https://scigantic.com/topics/robotics)
- [Robot learning datasets](https://scigantic.com/topics/robot-learning)
