Dataset guides
20 short guides to opening public scientific datasets in a Jupyter notebook. Each one gives code that was run against the real files, explains how the files are laid out, and links to a hosted notebook where the data is already mounted.
- How to open NEXRAD Level II radar data in a Jupyter notebook without downloading itRead a Level II volume file with Py-ART:
pyart.io.read_nexrad_archive(path)returns a Radar object holding every sweep and field. - How to read NEXRAD Level III products in Python in a Jupyter notebookRead a Level III product file with Py-ART:
pyart.io.read_nexrad_level3(path)returns a Radar object with one field, such as reflectivity or rain rate. - How to query 1000 Genomes VCF files on S3 from a Jupyter notebookOpen the VCFs with pysam:
pysam.VariantFile(path)reads a bgzipped, tabix-indexed VCF, andfetch(chrom, start, end)pulls one region without scanning the whole file. - How to open the OpenBind Zika virus blind challenge data in a Jupyter notebookThe 400 blinded compounds are in
parquet/compounds.parquet, which opens with pyarrow, pandas or DuckDB. - How to open the OpenADMET CYP inhibition challenge data in a Jupyter notebookThe dataset is five CSV files, and pandas reads each one directly:
pd.read_csv(mount / "cyp-challenge-TRAIN_inhibition.csv")gives the 4,905-compound training table. - How to open the Alchemistry 2027 dengue binding free energy challenge data in a Jupyter notebookThe whole challenge input is one 54-row CSV of compound IDs and CXSMILES plus one PDB file, read with pandas and RDKit.
- How to open Tahoe-100M single-cell perturbation data in a Jupyter notebookTahoe-100M is 3,388 Parquet files of about 28,000 cells each, and pandas reads one with
pd.read_parquet. - How to open the Genomes to Fields (G2F) maize genotype by environment data in a Jupyter notebookThe trial data are CSV files that pandas reads directly, starting with
Training_data/1_Training_Trait_Data_2014_2023.csvfor plot-level yield. - How to read prebuilt MSA databases for structure prediction (Thal-Kak) without unpacking themEach database is one
.tar.zstarchive, and you can stream alignments from the front of it withzstandardandtarfilewithout extracting anything. - How to load CMMD mammography DICOM files and clinical data in a Jupyter notebookRead the collection index with pandas (
pd.read_parquetonseries.parquet) and the images with pydicom (pydicom.dcmread). - How to open IDC cervical cancer whole-slide DICOM images in a Jupyter notebookRead the collection index with pandas (
pd.read_parquetonseries.parquet) and a slide with pydicom (pydicom.dcmread). - How to open the Connectome 2.0 high b-value diffusion MRI dataset in a Jupyter notebookLoad the NIfTI volumes with nibabel:
nib.load(path)reads the header, and slicingimg.dataobjreads only the volumes you ask for. - How to open an OpenNeuro fMRI dataset from S3 without DataLad (ds000115)An OpenNeuro dataset is a BIDS folder tree, so
pathlibfinds the subjects and Nilearn plots a NIfTI image straight from its path. - How to open ECMWF IFS S2S reforecasts stored as Icechunk in a Jupyter notebookOpen the Icechunk repository, take a read-only session on
main, and pass its store toxr.open_zarrwith a group name. - How to query ChEMBL bioactivity data as Parquet without installing the databaseQuery the Parquet copy of ChEMBL with DuckDB:
read_parquetonchembl_37/derived/activities_enriched.parquetreturns every curated activity for a target in one SQL statement. - How to open CAMELYON lymph node whole-slide images from S3 in a Jupyter notebookThe slides and masks are tiled BigTIFF pyramids, and
tifffile.TiffFile(path)opens one and reports its levels without loading pixels. - How to browse CZ CryoET Data Portal datasets in a Jupyter notebookEvery dataset is a numbered folder with a
dataset_metadata.jsondescribing the sample, and its runs hold tomograms as multi-scale OME-Zarr. - How to open OME-Zarr volumes in a Jupyter notebookRead an OME-Zarr store with ngff-zarr:
nz.from_ngff_zarr(path)returns the multiscale image, andimages[0].datais the full-resolution array. - How to query 122 million PubChem compounds with ADMET predictions in a Jupyter notebookThe corpus is 21 Parquet shards, so DuckDB or pyarrow can query all 122 million rows by reading only the columns a query needs.
- How to open Sid Sijbrandij's osteosarcoma multi-omic dataset in a Jupyter notebookStart with
manifest.txtat the bucket root, which lists all 397,036 objects, then read individual tables with pandas.
Looking for a dataset that is not here? Browse datasets by topic or search the catalog.