# RxR (Room-Across-Room): Multilingual Vision-and-Language Navigation Instructions in English, Hindi and Telugu with Pose Traces for Matterport3D (Google Research)

## Facts

- Provider: Google Research (Ku, Anderson, Patel, Ie, Baldridge)
- License: CC-BY-4.0 (https://github.com/google-research-datasets/RxR)
- Format: NumPy (in sampled files)
- Stored at: Google Cloud Storage, gs://rxr-data (us)
- Access: public, readable without an account
- Category: Machine Learning & AI
- Source: https://github.com/google-research-datasets/RxR
- Page: https://scigantic.com/archive/39c3f2a7-28fb-5d93-8c31-62b61effceaa
- Keywords: Machine Learning & AI, NumPy, Google Cloud Storage

## About

Room-Across-Room (RxR) is a multilingual vision-and-language navigation (VLN) dataset: people describe paths through indoor environments of the Matterport3D building scans, and an agent must follow the spoken-then-transcribed instruction to the goal. RxR is about ten times larger than Room-to-Room (R2R), covers English (en-IN and en-US), Hindi and Telugu, and has longer, more varied paths. The bucket (about 161 GB) holds guide annotations (gzipped JSON Lines with path, scan id, viewpoints, initial heading, instruction text, word-level timestamps from the recording and annotator ids, for train, val_seen, val_unseen and test splits), follower annotations, pose_traces/ per split (NumPy archives recording where each annotator looked in the panorama as they spoke, which grounds each word to pixels and surfaces), text_features/ (precomputed multilingual text encodings), grounded landmark annotations, model-generated instructions and Gibson navigation graphs. The Matterport3D imagery itself is not included and is governed by the Matterport3D Terms of Use. Used for embodied AI agents, instruction following, multilingual grounding and instruction generation. Download with gsutil -m cp -R gs://rxr-data. License: the RxR annotations are released under CC BY 4.0. Cite Ku et al., Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Grounding (EMNLP 2020).

## Load the data

```python
import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('rxr-data/', detail=False)[:10]
```

Lists the files with anonymous access. Runs on any machine, with no cloud account.

## Data structure

File types in sampled files: NumPy

Example file: `pose_traces/rxr_train/000000_follower_pose_trace.npz`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/39c3f2a7-28fb-5d93-8c31-62b61effceaa
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/39c3f2a7-28fb-5d93-8c31-62b61effceaa
- Over MCP: `get_schema_card` with id `39c3f2a7-28fb-5d93-8c31-62b61effceaa` at https://api.scigantic.com/mcp

## Related topics

- [Machine Learning & AI datasets](https://scigantic.com/topics/machine-learning-and-ai)
- [NumPy datasets](https://scigantic.com/topics/numpy)
