Virtual Cell Modeling: sample notebook
A worked notebook for Virtual Cell Modeling, with 15 code cells and the output each one printed. On Scigantic it opens in a hosted JupyterLab with the dataset already mounted, so nothing is downloaded first.
- Provider
- Arc Institute, Tahoe Therapeutics, and independent research groups (see schema card for per-dataset attribution)
- License
- CC0-1.0
- Size
- 4,427 files
- Format
- Parquet (in sampled files)
- Stored at
- Hugging Face (https://huggingface.co/datasets/tahoebio/Tahoe-100M)
- Access
- Free and public. No account with the data provider is needed.
The notebook
Public training data for virtual cell modeling
Nine independently-published, independently-licensed single-cell datasets that Arc Institute's Virtual Cell Challenge points participants at as suggested training data (see arcinstitute.org/tools/virtualcellatlas). None of this is the Challenge's own held-back competition data. That stays on virtualcellchallenge.org, restricted under the Challenge's own Terms of Use. Everything here lives independently, with its own license.
Tahoe-100M is mounted at /mnt/archive (via Hugging Face). Three more sources (the Arc Virtual Cell Atlas umbrella, scBaseCount, and the 2025 Virtual Cell Challenge dataset) are also real, read-only mounts, but through a *companion* archive, "Arc Virtual Cell Atlas (GCS, requester-pays)", which you attach when you launch this notebook (the archive picker's "Mount additional archives" panel). That bucket is requester-pays, so rather than hand this notebook a GCP credential of its own (readable by any code you run here), Scigantic's own mount-manager authenticates and bills the read on your behalf. The remaining five sources (GEO/Figshare/Zenodo papers) are plain public downloads, pulled directly in the cells below; each one loads a small real slice rather than the full multi-gigabyte release, so this notebook runs end to end in a few minutes.
| # | Dataset | Host | License |
| 1 | Tahoe-100M | Hugging Face (this archive's primary mount) | CC0-1.0 |
| 2 | Arc Virtual Cell Atlas (umbrella) | GCS, via the companion archive | CC0-1.0 |
| 3 | scBaseCount | GCS, via the companion archive | CC0-1.0 |
| 4 | 2025 Virtual Cell Challenge dataset | GCS, via the companion archive | CC0-1.0 |
| 5 | Replogle et al. 2022 | Figshare+ | CC BY 4.0 |
| 6 | Nadig et al. 2025 | GEO GSE264667 | not stated |
| 7 | Jiang et al. 2025 | Zenodo | CC BY 4.0 |
| 8 | Srivatsan et al. 2020 (sci-Plex) | GEO GSE139944 | not stated |
| 9 | McFaline-Figueroa et al. 2024 (sci-Plex-GxE) | GEO GSE225775 | not stated |
Where a source states no license, treat it as public via NCBI's general-access norms, not as a formal open-reuse grant.
This notebook's recommended "Genomics & Bioinformatics" image ships scanpy/anndata/scipy/pandas but not Hugging Face's datasets client, which section 1 needs for Tahoe-100M, installed once below.
%pip install -q datasets huggingface_hub
Output:
Note: you may need to restart the kernel to use updated packages.
1. Tahoe-100M: the mounted primary
100.6M cells from ~60,000 drug-perturbation experiments across 50 cancer models and 1,100+ compounds. Already mounted at /mnt/archive, but it's also directly streamable through the datasets library without touching the mount. Useful if you want to run this notebook somewhere other than Scigantic.
The drug_metadata and cell_line_metadata tables carry real biological annotation (mechanism of action per drug, driver-gene mutations per cell line), not just IDs.
from datasets import load_dataset
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
drug_metadata = load_dataset("tahoebio/Tahoe-100M", "drug_metadata", split="train").to_pandas()
cell_line_metadata = load_dataset("tahoebio/Tahoe-100M", "cell_line_metadata", split="train").to_pandas()
print(f"{len(drug_metadata):,} drugs, {len(cell_line_metadata):,} cell-line driver-mutation rows")
drug_metadata[["drug", "targets", "moa-broad", "human-approved"]].head()
Output:
379 drugs, 1,000 cell-line driver-mutation rows
drug targets moa-broad human-approved
0 Talc None unclear yes
1 Bortezomib PSMB5 inhibitor/antagonist yes
2 Ixazomib PSMB5 inhibitor/antagonist yes
3 Ixazomib citrate PSMB1, PSMB2, PSMB5 inhibitor/antagonist yes
4 Lactate (calcium) None unclear yes
A real differential-expression result, not just metadata. Tahoe-100M ships precomputed pseudobulk DESeq2-style tables (log2FoldChange, padj) per drug × cell-line × concentration: 4.09 billion rows total. Streaming and filtering directly for one drug/cell-line pair (no need to download the full table) returns real results in well under a minute.
The plot below marks the padj 1 thresholds used to call a gene up- or down-regulated, and labels the five strongest hits by name.
# Bortezomib is an approved proteasome inhibitor (see drug_metadata above).
# A549 is a lung adenocarcinoma line. Cap at 500 genes: verified fast
# (~45s) and plenty for a real volcano plot.
ds = load_dataset("tahoebio/Tahoe-100M", "pseudobulk_differential_expression", streaming=True, split="train")
filtered = ds.filter(lambda r: r["drug"] == "Bortezomib" and r["Cell_Name_Vevo"] == "A549")
rows = []
for row in filtered:
rows.append(row)
if len(rows) >= 500:
break
de = pd.DataFrame(rows).dropna(subset=["padj", "log2FoldChange", "gene_name"])
de["neg_log10_padj"] = -np.log10(de["padj"].clip(lower=1e-300))
PADJ_CUTOFF, LFC_CUTOFF = 0.05, 1.0
up = (de["padj"] < PADJ_CUTOFF) & (de["log2FoldChange"] > LFC_CUTOFF)
down = (de["padj"] < PADJ_CUTOFF) & (de["log2FoldChange"] < -LFC_CUTOFF)
ns = ~(up | down)
plt.figure(figsize=(6.5, 4.5))
plt.scatter(de.loc[ns, "log2FoldChange"], de.loc[ns, "neg_log10_padj"], s=10, alpha=0.35, color="#94a3b8", label="not significant", edgecolors="none")
plt.scatter(de.loc[down, "log2FoldChange"], de.loc[down, "neg_log10_padj"], s=14, alpha=0.8, color="#f59e0b", label="down (padj<0.05, log2FC<-1)", edgecolors="none")
plt.scatter(de.loc[up, "log2FoldChange"], de.loc[up, "neg_log10_padj"], s=14, alpha=0.8, color="#2563eb", label="up (padj<0.05, log2FC>1)", edgecolors="none")
plt.axhline(-np.log10(PADJ_CUTOFF), color="#64748b", linestyle="--", linewidth=1)
plt.axvline(LFC_CUTOFF, color="#64748b", linestyle="--", linewidth=1)
plt.axvline(-LFC_CUTOFF, color="#64748b", linestyle="--", linewidth=1)
top_hits = de.loc[up | down].nlargest(5, "neg_log10_padj")
for _, r in top_hits.iterrows():
plt.annotate(r["gene_name"], (r["log2FoldChange"], r["neg_log10_padj"]), fontsize=8, xytext=(4, 4), textcoords="offset points")
plt.xlabel("log2 fold change (Bortezomib vs. DMSO)")
plt.ylabel("-log10(padj)")
plt.title(f"Bortezomib vs. DMSO in A549 (Tahoe-100M): {up.sum()} up, {down.sum()} down of {len(de)} genes")
plt.legend(loc="upper left", frameon=False, fontsize=8)
plt.tight_layout()
plt.show()
2 to 4. The Arc Virtual Cell Atlas: umbrella, scBaseCount, 2025 VCC dataset
All three live in one requester-pays GCS bucket, gs://arc-institute-virtual-cell-atlas. Attach the companion archive "Arc Virtual Cell Atlas (GCS, requester-pays)" when you launch this notebook (or add it afterward via the archive picker). Scigantic's mount-manager sidecar authenticates and bills the read with its own service account, so no GCP credential of your own is needed and nothing sensitive is ever exposed inside this notebook's environment. If you skip that step, the cell below reports no companion attached rather than silently pretending the data is there.
- scBaseCount (scBaseCount/ prefix): 502M+ cells, 27 organisms, 75 tissues, sharded as h5ad (AnnData) per organism/tissue. Far too large to browse in full: list a subdirectory before assuming a filename.
- 2025 Virtual Cell Challenge dataset (virtual-cell-challenge/2025/ prefix): last year's Challenge data, now openly released. h5ad + parquet/csv, ~300K cells, H1 hESC, 300 target genes. Small enough to load in full.
import os
from pathlib import Path
ADDITIONAL = Path.home() / "additional-datasets"
companions = os.listdir(ADDITIONAL) if ADDITIONAL.exists() else []
if companions:
print("attached companions:", companions)
else:
print("no companion archive attached in this session. Attach 'Arc Virtual Cell Atlas (GCS, requester-pays)' "
"from the archive picker to run this section for real.")
# The exact subdirectory name is sanitized from the bucket name at mount
# time. List ADDITIONAL above to confirm it, then browse underneath it,
# e.g. <that dir>/scBaseCount/ and <that dir>/virtual-cell-challenge/.
import anndata as ad
# ATLAS_MOUNT is whichever subdirectory the listing above showed. Set it
# once you've confirmed the name.
ATLAS_MOUNT = ADDITIONAL / companions[0] if companions else None
if ATLAS_MOUNT is not None:
# Arc's own tutorial (github.com/ArcInstitute/arc-virtual-cell-atlas,
# virtual-cell-challenge/tutorial-py.ipynb) nests the 2025 splits one
# level deeper than a flat listing might suggest. List before trusting
# this exact path: Arc has migrated this bucket's layout before.
vcc_train_dir = ATLAS_MOUNT / "virtual-cell-challenge" / "2025" / "train"
print("files:", os.listdir(vcc_train_dir))
vcc_train = ad.read_h5ad(vcc_train_dir / "adata_Training.h5ad")
print(vcc_train)
vcc_train.obs.head()
else:
print("attach the companion archive first (see the cell above)")
if ATLAS_MOUNT is not None:
# Arc's public materials describe obs["target_gene"] as the CRISPRi
# target per cell. If this key errors, list vcc_train.obs.columns and
# adjust: the schema is not hand-verified against a live mount here.
if "target_gene" in vcc_train.obs.columns:
pert_counts = vcc_train.obs["target_gene"].value_counts()
top15 = pert_counts.head(15)
plt.figure(figsize=(6.5, 4.5))
top15.sort_values().plot.barh(color="#2563eb")
plt.xlabel("cells")
plt.ylabel("")
plt.title(f"2025 VCC training set: {pert_counts.shape[0]} target genes, top 15 by cell count")
plt.tight_layout()
plt.show()
else:
print("obs has no 'target_gene' column. Actual columns:", list(vcc_train.obs.columns))
else:
print("attach the companion archive first, then re-run this cell")
5. Replogle et al. 2022: genome-scale Perturb-seq (K562, RPE1)
CRISPRi Perturb-seq across >2.5M cells, hosted on Figshare+ (CC BY 4.0, attribute the paper if you use this). Arc cites the full single-cell files by size (K562 genome-wide 61.3GB, K562 essential 9.9GB, RPE1 8.1GB); the same deposit also has much smaller pseudobulk aggregates of the same data, which is what this cell loads for a fast, real demo. Each row is one CRISPRi knockdown, with real QC metrics: pct_expr (how much the target gene itself dropped) and energy_test_p_value (whether the knockdown produced a detectable transcriptome-wide effect at all).
import anndata as ad
import fsspec
# K562-essential pseudobulk (79.8MB). See the schema card for the full-size
# single-cell file URLs if you need per-cell resolution instead.
url = "https://ndownloader.figshare.com/files/35780870"
with fsspec.open(url) as f:
replogle_k562_bulk = ad.read_h5ad(f)
replogle_k562_bulk
qc = replogle_k562_bulk.obs[["pct_expr", "energy_test_p_value"]].copy()
qc["target_gene"] = [name.split("_")[1] if "_" in name else name for name in replogle_k562_bulk.obs_names]
qc = qc.dropna(subset=["pct_expr", "energy_test_p_value"])
qc["neg_log10_p"] = -np.log10(qc["energy_test_p_value"].clip(lower=1e-300))
sig = qc["energy_test_p_value"] < 0.01
plt.figure(figsize=(6.5, 4.5))
plt.scatter(qc.loc[~sig, "pct_expr"], qc.loc[~sig, "neg_log10_p"], s=10, alpha=0.35, color="#94a3b8", label="no detectable effect", edgecolors="none")
plt.scatter(qc.loc[sig, "pct_expr"], qc.loc[sig, "neg_log10_p"], s=10, alpha=0.6, color="#2563eb", label="significant (p<0.01)", edgecolors="none")
plt.axhline(-np.log10(0.01), color="#64748b", linestyle="--", linewidth=1)
plt.xlabel("target knockdown (pct_expr, negative means reduced expression)")
plt.ylabel("-log10(energy test p-value)")
plt.title(f"K562-essential: {sig.sum()} of {len(qc)} knockdowns show a significant effect")
plt.legend(loc="upper right", frameon=False, fontsize=8)
plt.tight_layout()
plt.show()
6. Nadig et al. 2025: DepMap essential-gene screens (HepG2, Jurkat)
Single-cell CRISPR screens of DepMap Common Essential Genes, GEO GSE264667. No license is stated on the GEO page: public via NCBI's norms, not a formal open license.
The full HepG2 file is 5.2GB, too much to load wholesale here. h5py can read just the tiny per-cell metadata (obs) over HTTP range requests without ever touching the expression matrix, which is enough to see how many cells landed on each CRISPRi target.
import h5py
import fsspec
url = "https://ftp.ncbi.nlm.nih.gov/geo/series/GSE264nnn/GSE264667/suppl/GSE264667_hepg2_raw_singlecell_01.h5ad"
with fsspec.open(url) as f:
hf = h5py.File(f, "r")
categories = [c.decode() if isinstance(c, bytes) else c for c in hf["obs"]["__categories"]["gene"][:]]
codes = hf["obs"]["gene"][:]
targets = pd.Series(codes).map(lambda i: categories[i])
top10 = targets.value_counts().head(10)
print(f"{len(categories):,} distinct CRISPRi targets across {len(targets):,} HepG2 cells")
top10
Output:
2,394 distinct CRISPRi targets across 145,473 HepG2 cells non-targeting 4976 TFAM 1213 GFM1 749 SLC1A5 631 GTF3C4 536 GTF2E2 476 ZDHHC7 466 PPP6C 459 SMN2 452 EIF3CL 435 Name: count, dtype: int64
plt.figure(figsize=(6, 4))
top10.sort_values().plot.barh(color="#2563eb")
plt.xlabel("cells")
plt.title("Nadig et al. 2025 (HepG2): top 10 CRISPRi targets (incl. non-targeting controls)")
plt.tight_layout()
plt.show()
7. Jiang et al. 2025 (six-cell-line Perturb-seq)
More than 1,500 perturbations across six cell lines (A549, MCF7, HT29, HAP1, BxPC3, K562) and five signaling pathways (interferon-beta, interferon-gamma, TNF-alpha, TGF-beta1, insulin), hosted on Zenodo (record 10520190, CC BY 4.0). Each pathway's six-cell-line screen ships as one Seurat object, not six separate per-cell-line files.
This one is not yet Python-loadable as-is. It ships only as Seurat .rds objects (2.6 to 5.6GB each), and pyreadr can only parse plain R data.frames/lists, not the S4 Seurat class. Converting requires R:
library(Seurat)
library(SeuratDisk)
obj <- readRDS("Seurat_object_IFNB_Perturb_seq.rds")
SaveH5Seurat(obj, filename = "IFNB_Perturb_seq.h5Seurat")
Convert("IFNB_Perturb_seq.h5Seurat", dest = "h5ad")
The cell below queries Zenodo's own records API for the five pathway files' real sizes (a small JSON response, not a multi-gigabyte download), so you can see the actual scale before choosing which one to convert in R.
import re
import requests
record = requests.get("https://zenodo.org/api/records/10520190", timeout=30).json()
pathway_files = {f["key"]: f["size"] / 1e9 for f in record["files"] if f["key"].startswith("Seurat_object_")}
labels = {k: re.sub(r"^Seurat_object_|_Perturb_seq\.rds$", "", k) for k in pathway_files}
sizes = pd.Series({labels[k]: v for k, v in pathway_files.items()}).sort_values()
plt.figure(figsize=(6.5, 4))
sizes.plot.barh(color="#2563eb")
plt.xlabel("file size (GB)")
plt.title("Jiang et al. 2025: five pathway screens, six cell lines each")
plt.tight_layout()
plt.show()
8. Srivatsan et al. 2020 (sci-Plex)
Introduces "sci-Plex": nuclear hashing to screen thousands of chemical perturbations at single-cell resolution. GEO GSE139944. No license is stated on the GEO page.
The full series ships as one 9.2GB RAW.tar, but each sample inside is also individually downloadable and far smaller. sciPlex1 is the pilot experiment: a human (HEK293T) plus mouse (NIH3T3) species-mixing control used to validate droplet purity before the real screens. Its three files together are under 30MB. The count matrix ships as a plain sparse triplet (gene_row\tcell_col\tcount), one line per non-zero entry.
from scipy.sparse import coo_matrix
base = "https://ftp.ncbi.nlm.nih.gov/geo/samples/GSM4150nnn/GSM4150376/suppl/"
genes = pd.read_csv(base + "GSM4150376_sciPlex1_HEK293T_NIH3T3_gene.annotations.txt.gz", sep="\t", header=None, names=["gene_id", "gene_name"])
cells = pd.read_csv(base + "GSM4150376_sciPlex1_HEK293T_NIH3T3_cell.annotations.txt.gz", sep="\t", header=None, names=["cell_barcode", "batch"])
triplets = pd.read_csv(base + "GSM4150376_sciPlex1_HEK293T_NIH3T3_UMI.count.matrix.gz", sep="\t", header=None, names=["gene_idx", "cell_idx", "count"])
mat = coo_matrix((triplets["count"], (triplets["gene_idx"] - 1, triplets["cell_idx"] - 1)), shape=(len(genes), len(cells))).tocsr()
umi_per_cell = np.asarray(mat.sum(axis=0)).ravel()
print(f"{len(cells):,} cells, {len(genes):,} genes, median {int(np.median(umi_per_cell)):,} UMI/cell")
# Human gene symbols are conventionally ALL CAPS (TSPAN6); mouse symbols are
# Title/lower case (Tspan6, mt-Rnr2). Classify every gene by that convention
# and sum UMI per cell within each species. This is the actual test a
# species-mixing pilot exists to run: how cleanly do droplets separate?
def is_human(name):
letters = [c for c in name if c.isalpha()]
return len(letters) > 0 and all(c.isupper() for c in letters)
human_mask = genes["gene_name"].apply(is_human).values
human_umi = np.asarray(mat[human_mask].sum(axis=0)).ravel()
mouse_umi = np.asarray(mat[~human_mask].sum(axis=0)).ravel()
frac_human = human_umi / (human_umi + mouse_umi + 1e-9)
species = np.where(frac_human > 0.9, "human (HEK293T)", np.where(frac_human < 0.1, "mouse (NIH3T3)", "mixed / doublet"))
print(pd.Series(species).value_counts())
colors = {"human (HEK293T)": "#2563eb", "mouse (NIH3T3)": "#f59e0b", "mixed / doublet": "#94a3b8"}
plt.figure(figsize=(6, 4))
for label in ["mixed / doublet", "human (HEK293T)", "mouse (NIH3T3)"]:
m = species == label
plt.scatter(human_umi[m] + 1, mouse_umi[m] + 1, s=10, alpha=0.6, color=colors[label], label=label, edgecolors="none")
plt.xscale("log")
plt.yscale("log")
plt.xlabel("human UMI per cell")
plt.ylabel("mouse UMI per cell")
plt.title("sci-Plex1 pilot: species-mixing separates human, mouse, and doublet droplets")
plt.legend(loc="upper left", frameon=False, fontsize=9)
plt.tight_layout()
plt.show()
9. McFaline-Figueroa et al. 2024 (sci-Plex-GxE)
Combined genetic + chemical screening in glioblastoma lines, GEO GSE225775. No license is stated on the GEO page. The full series is 12.5GB; the CRISPRi guide library design for one sample is a few hundred bytes and shows exactly which DNA-damage-repair genes (MGMT, MLH1, ...) were targeted alongside the chemical perturbations.
base = "https://ftp.ncbi.nlm.nih.gov/geo/samples/GSM7056nnn/GSM7056148/suppl/"
sgrna = pd.read_csv(base + "GSM7056148_sciPlexGxE_1_sgRNA_sequences.txt.gz", sep="\t")
guides_per_gene = sgrna.groupby("gene").size().sort_values(ascending=False)
print(f"{sgrna['gene'].nunique()} genes targeted, {len(sgrna)} guides total (incl. non-targeting controls)")
guides_per_gene.head(15)
Output:
8 genes targeted, 30 guides total (incl. non-targeting controls) gene HPRT1 6 negative_control 6 MLH1 3 MGMT 3 MSH2 3 MSH3 3 MSH6 3 PMS2 3 dtype: int64
plt.figure(figsize=(6, 3.5))
guides_per_gene.sort_values().plot.barh(color="#2563eb")
plt.xlabel("guides")
plt.title("McFaline-Figueroa et al. 2024: guide library for sample sciPlexGxE_1")
plt.tight_layout()
plt.show()
Next steps
- Mind the license per source: CC0-1.0 (Arc's own resources), CC BY 4.0 (Replogle, Jiang; attribute the paper), or unstated (the other three GEO series).
- scBaseCount and the Atlas umbrella are far larger than what's shown here. This notebook loads representative slices, not the full corpora. List a subdirectory before assuming a specific shard exists.
- Jiang et al. needs the R conversion above before it's usable from anndata/scanpy directly.
- Forgot to attach the companion archive? Add "Arc Virtual Cell Atlas (GCS, requester-pays)" from the archive picker, then re-run section 2 to 4's cells. No need to relaunch.
Ask the scigantic-ai assistant (right panel) to help adapt any of these loaders, or to point you at the specific perturbation/gene/cell-line slice you actually need for training.
Run it
Open this notebook to run it, see the dataset page for the full description, or browse the full catalog.