SciVid CalMS21: Resident-Intruder Mouse Social Interaction Video Clips for Behavior Classification (Google DeepMind)

Provider
Google DeepMind (SciVid); source data Sun, Karigo et al., Caltech and Northwestern University
License
CC-BY-4.0 (source)
Stored at
Google Cloud Storage, gs://scivid/full/calms21_slim (us)
Access
Free and public. No account with the data provider is needed.
Category
Neuroscience
Source
https://arxiv.org/abs/2104.02710

About this dataset

Preprocessed CalMS21 (Caltech Mouse Social Interactions 2021) videos of freely behaving mice in the standard resident-intruder assay, for frame-level classification of social behaviors, as packaged in the SciVid benchmark. The original CalMS21 release has over 1 million frames with frame-level behavior annotations and 6 million unlabeled frames of tracked poses. SciVid filtered and spatially downsampled the videos (scale 0.5) and held out part of the original train split for validation, giving about 27,000 training samples with train, validation and test shards (calms21_test_split_ids_shuffle_scale05.array_record-00000-of-01024 and onward). Used to benchmark video models for automated rodent behavior annotation in behavioral neuroscience. SciVid (Hasson et al., SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications, ICCV 2025, arXiv:2507.03578) is a five-task benchmark for video foundation models in science; this folder is one task. Files are ArrayRecord shards, read with the array_record Python package or the loaders in the google-deepmind/scivid repository. License: SciVid materials CC BY 4.0 (Google DeepMind); CalMS21 is CC BY 4.0 at CaltechDATA (DOI 10.22002/D1.1991). Cite SciVid and Sun et al., The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions, arXiv:2104.02710.

How to access this dataset in Python

import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('scivid/full/calms21_slim/', detail=False)[:10]

Lists the files with anonymous access. Runs on any machine, with no cloud account.

Data structure

Files

Example file: full/calms21_slim/

Analyze this dataset without downloading it

On Scigantic, SciVid CalMS21: Resident-Intruder Mouse Social Interaction Video Clips for Behavior Classification (Google DeepMind) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Neuroscience datasets