Brain Image Library (BIL): BICCN whole-brain microscopy: sample notebook
A worked notebook for Brain Image Library (BIL): BICCN whole-brain microscopy, with 7 code cells and the output each one printed. On Scigantic it opens in a hosted JupyterLab with the dataset already mounted, so nothing is downloaded first.
- Provider
- Brain Image Library (Pittsburgh Supercomputing Center / University of Pittsburgh)
- License
- CC BY-SA 4.0 (some datasets additionally under Allen Institute Terms of Use)
- Stored at
- download.brainimagelibrary.org
- Access
- Free and public. No account with the data provider is needed.
- DOI
- 10.1038/s41597-024-03761-8
The notebook
Brain Image Library: whole-brain microscopy, read in place
The Brain Image Library (BIL) is the BRAIN Initiative's repository for whole-brain microscopy: about 14,000 datasets and 6 PB from BICCN, BICAN and ASAP labs, the largest public collection of cleared-tissue light-sheet brains, plus fMOST and serial two-photon tomography volumes, spatial transcriptomics and Patch-seq morphologies.
Nothing here downloads a dataset. scigantic_bil (preinstalled in this image) loads BIL's own daily inventory into a catalog, finds datasets by technique or free text, and reads one slice, one thumbnail or one OME-Zarr level over HTTP. The same tree is also mounted read-only at $SCIGANTIC_MOUNT_PATH if you prefer paths.
Data license: CC BY-SA 4.0. Cite each dataset's DOI (retrieve(id).doi).
import time
import numpy as np
import scigantic_bil as bil
import matplotlib.pyplot as plt
%config InlineBackend.figure_formats = ["jpeg"]
%config InlineBackend.print_figure_kwargs = {"pil_kwargs": {"quality": 80}}
plt.rcParams["figure.dpi"] = 72
cat = bil.BilCatalog.load() # one 5 MB GET, cached on disk afterwards
s = cat.summary()
print(f"inventory {s['inventory_date']}: {s['datasets']:,} datasets, {s['files']:,} files, {s['terabytes']:,} TB")
print("technique:", ", ".join(f"{k} ({v})" for k, v in list(s["technique"].items())[:8]))
print("species: ", ", ".join(f"{k} ({v})" for k, v in list(s["species"].items())[:6]))
Output:
inventory 20260731: 14,224 datasets, 144,335,214 files, 6,034.6 TB technique: other (6074), Patch-seq (2959), fMOST (985), retrograde transsynaptic tracing (914), STPT (757), light sheet microscopy (347), transsynaptic (341), EM (301) species: mouse (11350), human (1177), marmoset (308), macaca nemestrina (291), rat (231), homo sapiens (206)
1. Find the light-sheet data
The inventory's technique field says "light sheet microscopy" for a few hundred datasets. Many more are tagged other and only say light sheet in their abstract or instrument record. light_sheet() unions the structured field with BIL's fulltext index, which is the honest count.
by_field = cat.filter(light_sheet=True)
ls = cat.light_sheet()
tb = sum(d.size_bytes or 0 for d in ls) / 1e12
print(f"{len(by_field)} datasets by technique field, {len(ls)} with the fulltext union, {tb:.0f} TB")
df = cat.to_dataframe(ls)
top = (df.groupby(["contributor", "affiliation"])
.agg(datasets=("bildid", "size"), TB=("size_gb", lambda g: g.sum() / 1000))
.sort_values("datasets", ascending=False)
.head(8))
top
Output:
348 datasets by technique field, 808 with the fulltext union, 278 TB
datasets \
contributor affiliation
Zhang, Jiangyang New York University 193
Kim, Yongsoo Pennsylvania State University 149
Betty, Rebecca Pennsylvania State University 116
Svoboda, Karel Allen Institute for Neural Dynamics 82
Yongsoo Kim Lab Pennsylvania State University 53
Hampel, Stefanie University of Puerto Rico, Medical Sciences Campus 51
Kronman, Fae Pennsylvania State University 46
Tasic, Bosiljka Allen Institute for Brain Science 40
TB
contributor affiliation
Zhang, Jiangyang New York University 15.220843
...
fig, ax = plt.subplots(figsize=(7, 3.2))
sizes = df["size_gb"].dropna()
cap = sizes.quantile(0.98) # a few multi-TB outliers would flatten the plot
ax.hist(sizes.clip(upper=cap), bins=40, color="#3b6fb6")
ax.set_xlabel(f"dataset size (GB, clipped at the 98th percentile = {cap:.0f} GB)")
ax.set_ylabel("light-sheet datasets")
ax.set_title(f"median light-sheet dataset: {sizes.median():.0f} GB, largest {sizes.max() / 1000:.0f} TB")
plt.tight_layout()
2. Read one slice, not the stack
A light-sheet dataset here is usually one folder of a few thousand single-slice TIFFs, one per z-plane and channel, about 16 MB each. Listing the dataset is one request; reading a slice is one more. thumbnail() reads the middle slice and stride-downsamples it.
stack = bil.slices("ace-bin-run") # Kim lab (Penn State) LSFM, nothing fetched yet
d = cat["ace-bin-run"]
print(f"{d.bildid}: {len(stack)} slices, {d.size_gb:.0f} GB, {d.technique}, {d.contributor}")
print("first / middle / last:", stack[0].name, stack[len(stack) // 2].name, stack[-1].name)
fig, axes = plt.subplots(1, 4, figsize=(11, 3.4))
for ax, z in zip(axes, (300, 700, 1100, 1500)):
img = bil.downsample(bil.read_tiff(stack[z]), 384) # one HTTP read per panel
lo, hi = np.percentile(img, (1, 99.7))
ax.imshow(img, cmap="gray", vmin=lo, vmax=hi)
ax.set_title(f"z = {z} ({img.shape[1]}x{img.shape[0]} preview)", fontsize=9)
ax.axis("off")
plt.suptitle("E13.5 mouse embryo, NeuroTrace, four of 1,923 planes", fontsize=10)
plt.tight_layout()
Output:
ace-bin-run: 1923 slices, 29 GB, light sheet microscopy, Kim, Yongsoo first / middle / last: Z00001_ch02.tif Z00962_ch02.tif Z01923_ch02.tif
3. OME-Zarr stores, when a dataset ships one
A zarr store is a directory, so it never shows up in the inventory's file-extension histogram. find_zarr() checks a dataset's manifest; open_zarr() opens it lazily and zarr_levels() reports the pyramid levels the server actually has (one BIL store declares eight and serves seven).
stores = bil.find_zarr("ace-cup-eel") # UC Irvine iDISCO+ light sheet, technique tagged "other"
print(stores)
g = bil.open_zarr("ace-cup-eel")
levels = bil.zarr_levels(g)
print("levels:", levels, " full-resolution shape:", tuple(g[levels[0]].shape), "(t, c, z, y, x)")
plane = bil.zarr_thumbnail(g, max_size=512)
ys, xs = np.nonzero(plane > np.percentile(plane, 50)) # the array is padded; crop to the tissue
plane = plane[ys.min():ys.max() + 1, xs.min():xs.max() + 1]
plt.figure(figsize=(6, 4))
plt.imshow(plane, cmap="magma", vmax=np.percentile(plane, 99.5))
plt.title(f"coarsest level, middle z plane ({plane.shape[1]}x{plane.shape[0]} px)"); plt.axis("off")
Output:
['https://download.brainimagelibrary.org/42/e4/42e4274e0579397f/subject_5/subject_5.zarr/'] levels: ['0', '1', '2', '3', '4', '5', '6'] full-resolution shape: (1, 1, 848, 6300, 9600) (t, c, z, y, x) (np.float64(-0.5), np.float64(149.5), np.float64(97.5), np.float64(-0.5))
4. Two more layouts: JPEG 2000 sections and fMOST trees
Most STPT sections and nearly all viral tracing brains are JPEG 2000 (5,787 datasets). They decode in place through imagecodecs' OpenJPEG. fMOST brains are TeraFly trees with a full resolution pyramid; thumbnail() reads the coarsest level, about 25 small blocks, and never lists the full-resolution folder (1.6 million files on this one).
sec = bil.slices("ace-bit-wig") # STPT, 267 JPEG 2000 coronal sections
t0 = time.time(); stpt = bil.read_image(sec[137]); t_stpt = time.time() - t0
t0 = time.time(); fmost = bil.thumbnail("ace-cap-cop", max_size=512); t_fmost = time.time() - t0
print(f"STPT section {stpt.shape} {stpt.dtype} in {t_stpt:.1f}s; fMOST coarsest level {fmost.shape} in {t_fmost:.1f}s")
def tissue(img, pct=60, pad=0.03):
# crop to the bounding box of the brighter pixels, so the panel is tissue, not black frame
ys, xs = np.nonzero(img > np.percentile(img, pct))
dy, dx = int((ys.max() - ys.min()) * pad), int((xs.max() - xs.min()) * pad)
return img[max(0, ys.min() - dy):ys.max() + dy, max(0, xs.min() - dx):xs.max() + dx]
fig, axes = plt.subplots(1, 2, figsize=(11, 4.2))
for ax, img, title in ((axes[0], tissue(bil.downsample(stpt, 512), 70), "STPT, one JPEG 2000 section"), (axes[1], tissue(fmost, 70), "fMOST, coarsest TeraFly level")):
lo, hi = np.percentile(img[img > 0], (2, 97)) # labelled cells are far brighter than tissue; do not let them set the range
ax.imshow(img, cmap="gray", vmin=lo, vmax=hi); ax.set_title(title, fontsize=10); ax.axis("off")
plt.tight_layout()
Output:
STPT section (11377, 8557) uint16 in 1.9s; fMOST coarsest level (265, 423) in 4.1s
5. Provenance: the full record
The inventory row is what BIL indexes. The full record (title, abstract, instrument, specimen, rights, DOI, publications) lives on BIL's metadata API and is one retrieve() away.
r = bil.retrieve("ace-cup-eel")
print(r.title)
print("microscope:", r.instrument.get("microscopetype"), "|", r.instrument.get("microscopemanufacturerandmodel"))
print("specimen: ", r.species, r.specimen.get("genotype") or "", "| technique:", r.technique, f"({r.technique_other})")
print("rights: ", r.rights_identifier, "|", r.rights_uri)
for p in r.publications:
print("cites: ", p.doi or p.citation[:120])
print()
print(r.abstract[:500])
Output:
Light-sheet imaged brain follwing T.Gondii injection: Subject 5 microscope: Light-sheet | Zeiss Z.1 specimen: mouse | technique: other (iDISCO+ light sheet) rights: CC-BY-SA-4.0 | https://creativecommons.org/licenses/by-sa/4.0/, https://alleninstitute.org/legal/terms-use/ cites: https://doi.org/10.1073/pnas.1915778116 This dataset includes light-sheet imaged brain follwing T.Gondii injection. We analyzed monocyte recruitment to the brain during infection with Toxoplasma gondii, a global foodborne parasite that establishes chronic infection in the CNS. By using 2-photon imaging through cranial windows, we dynamically tracked fluorescent monocytes at the blood–brain barrier from the acute to chronic stage of infection in mice. Whole-brain imaging revealed distinct localization patterns of monocytes in specific
Next
- cat.filter(technique="fMOST"), cat.filter(extension=".swc"), cat.search("MERFISH"): the same catalog covers every modality in BIL.
- bil.read_stack(id, start, stop, step) builds a (z, y, x) array from a strided range of slices; keep the range small, a full stack is 30 GB.
- bil.read_jp2(entry, reduce=3) decodes a large JPEG 2000 section at 1/8 resolution; bil.preview_plane(entry) bounds the read for a TIFF of any size.
- SWC morphologies open with navis.read_swc; neuroglancer is available for interactive 3D.
- The same files are on the FUSE mount: os.environ["SCIGANTIC_MOUNT_PATH"] + d.bildirectory.replace("/bil/data", "").
Package docs and source: https://github.com/Scigantic/scigantic-bil
Run it
Open this notebook to run it, see the dataset page for the full description, or browse the full catalog.