TARGET-ALL-P1 Acute Lymphoblastic Leukemia - Phase I (GDC open-access tables, 24 cases): sample notebook
A worked notebook for TARGET-ALL-P1 Acute Lymphoblastic Leukemia - Phase I (GDC open-access tables, 24 cases), with 3 code cells and the output each one printed. On Scigantic it opens in a hosted JupyterLab with the dataset already mounted, so nothing is downloaded first.
- Provider
- NCI Genomic Data Commons; Therapeutically Applicable Research to Generate Effective Treatments; tables assembled by Scigantic
- License
- GDC open-access tier: no access restrictions; NIH Genomic Data Sharing Policy applies to the source. Cite the TARGET initiative (https://www.cancer.gov/ccg/research/genome-sequencing/target) and the disease-specific marker paper; acknowledge the TARGET initiative in publications.
- Size
- 13 parquet tables, 4.3 MB total, plus README.md and BUILD_REPORT.json
- Format
- Parquet (in sampled files)
- Stored at
- Amazon S3, s3://scigantic-gdc-open/TARGET-ALL-P1 (us-east-1)
- Access
- Free and public. No account with the data provider is needed.
The notebook
TARGET-ALL-P1: Acute Lymphoblastic Leukemia - Phase I, ready to analyze
This mount is Scigantic's per-project re-arrangement of the NCI Genomic Data Commons open-access files for TARGET-ALL-P1 (24 cases) into parquet tables you can read directly: clinical, copy_number, rna_seq. The GDC's own bucket stores the same data as 18 files in one flat UUID namespace; here every matrix is already stacked (one column per aliquot) and every table is documented in README.md and BUILD_REPORT.json at the mount root.
What is not here: aligned reads and germline calls (controlled access via dbGaP), and the whole-slide images (listed in MANIFEST.parquet with their S3 location; the same slides are on the NCI Imaging Data Commons in DICOM).
Built from Data Release 46.0 - August 10, 2026. Values are unchanged from the GDC source files; only the arrangement is new.
The code below reads every table with the scigantic-nci package (pip install scigantic-nci, already installed in this image): from scigantic_nci import gdc, then one call per table, such as gdc.clinical("TARGET-ALL-P1"). Run anywhere else, the same calls read s3://scigantic-gdc-open/TARGET-ALL-P1/ anonymously.
import os, json
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from scigantic_nci import gdc
plt.rcParams['figure.dpi'] = 72 # keeps the saved figures small enough for the notebook preview
try:
from matplotlib_inline.backend_inline import set_matplotlib_formats
set_matplotlib_formats('jpeg') # JPEG keeps every inline figure well under the preview size limit
except ImportError:
pass
P = 'TARGET-ALL-P1'
# scigantic_nci reads the mounted archive when its BUILD_REPORT.json names this project, else s3://scigantic-gdc-open/<project>/ anonymously.
M = os.environ.get('SCIGANTIC_MOUNT_PATH', '/mnt/archive')
on_mount = os.path.exists(f'{M}/BUILD_REPORT.json') and json.load(open(f'{M}/BUILD_REPORT.json')).get('project') == P
print('reading', P, 'from', M if on_mount else f's3://scigantic-gdc-open/{P}/')
report = gdc.report(P)
print(report['project'], '|', report['gdc_data_release'], '| built', report['built_at'][:10])
print(f"{report['files_read']:,} source files read and md5-verified, {report['bytes_read']/1e9:.1f} GB\n")
for t in gdc.tables(P).itertuples():
print(f"{t.table + '.parquet':58s} {t.rows:>9,} rows x {t.columns:>5,} cols {t.bytes/1e6:8.1f} MB")
Output:
reading TARGET-ALL-P1 from /mnt/archive TARGET-ALL-P1 | Data Release 46.0 - August 10, 2026 | built 2026-09-13 13 source files read and md5-verified, 0.1 GB MANIFEST.parquet 18 rows x 19 cols 0.0 MB clinical/aliquots.parquet 184 rows x 7 cols 0.0 MB clinical/cases.parquet 24 rows x 23 cols 0.0 MB clinical/diagnoses.parquet 24 rows x 18 cols 0.0 MB clinical/follow_ups.parquet 333 rows x 14 cols 0.0 MB clinical/samples.parquet 121 rows x 13 cols 0.0 MB clinical/treatments.parquet 23 rows x 10 cols 0.0 MB copy_number/segments_gatk4_cnv.parquet 1,185 rows x 8 cols 0.0 MB rna_seq/counts_unstranded.parquet 60,660 rows x 13 cols 1.1 MB rna_seq/genes.parquet 60,660 rows x 3 cols 0.7 MB rna_seq/samples.parquet 12 rows x 11 cols 0.0 MB ...
1. Who is in the cohort
gdc.clinical(project) returns one row per case: clinical/cases.parquet (demographics flattened in as demographic_*) joined to the case's primary row of clinical/diagnoses.parquet. The diagnoses table (gdc.diagnoses) has more rows than cases because metastasis, recurrence and prior-primary entries each get their own row; the join keeps the row flagged diagnosis_is_primary_disease, which is the one you want for cohort tables and survival. GDC stores ages in days; the package adds age_at_diagnosis_years, os_time_days, os_event and stage.
cohort = gdc.clinical(P) # cases joined to the primary diagnosis, one row per case
dx_all = gdc.diagnoses(P)
n_primary = int(cohort['submitter_id_dx'].notna().sum()) if 'submitter_id_dx' in cohort else len(cohort) # diagnosis columns that clash with case columns carry _dx
print(f'{len(cohort)} cases, {len(dx_all)} diagnosis rows, {n_primary} primary diagnoses')
sex_col = next((c for c in ('demographic_sex_at_birth', 'demographic_gender') if c in cohort), None) # GDC renamed gender -> sex_at_birth in 2026
if sex_col: print(f'\n{sex_col}:'); print(cohort[sex_col].value_counts(dropna=False).to_string())
yrs = cohort['age_at_diagnosis_years']
if yrs.notna().any():
print(f'\nAge at diagnosis: median {yrs.median():.0f} y (IQR {yrs.quantile(.25):.0f}-{yrs.quantile(.75):.0f})')
print('\nVital status:'); print(cohort['demographic_vital_status'].value_counts(dropna=False).to_string())
print('\nPrimary diagnosis (top 5):'); print(cohort['primary_diagnosis'].value_counts().head(5).to_string())
print('\nNo stage column is populated for this project; gdc.diagnoses(P) shows what is.')
Output:
24 cases, 24 diagnosis rows, 24 primary diagnoses demographic_sex_at_birth: demographic_sex_at_birth male 14 female 9 unknown 1 Age at diagnosis: median 13 y (IQR 7-14) Vital status: demographic_vital_status Dead 14 Alive 9 Not Reported 1 ...
2. Expression: structure of the cohort
rna_seq/tpm_unstranded.parquet is genes x aliquots (60,660 GENCODE v36 genes, 12 aliquots). gdc.expression(project, protein_coding=True) returns it indexed by gene name, with the _PAR_Y duplicate rows dropped and the first row of any repeated name kept; gdc.expression_samples(project) says what each column is. The matrix is 2 MB on disk and reads into a 4 GiB pod in seconds; restrict to protein-coding genes and log2(TPM+1) before any distance-based method.
from sklearn.decomposition import PCA
samples = gdc.expression_samples(P).set_index('column') # matrix column (aliquot) -> case_submitter_id, sample_type
tpm = gdc.expression(P, protein_coding=True) # genes x aliquots TPM, indexed by gene name
logtpm = np.log2(tpm + 1)
print(f'{logtpm.shape[0]:,} protein-coding genes x {logtpm.shape[1]:,} aliquots')
print(samples.sample_type.value_counts().to_string())
# 2,000 most variable genes, centred, PCA
var = logtpm.var(axis=1).sort_values(ascending=False)
X = logtpm.loc[var.index[:2000]].T
X = X - X.mean()
pcs = PCA(n_components=5).fit(X)
Z = pd.DataFrame(pcs.transform(X)[:, :2], index=X.index, columns=['PC1', 'PC2']).join(samples[['sample_type', 'case_submitter_id']])
fig, ax = plt.subplots(figsize=(6.5, 5))
for label, g in Z.groupby('sample_type'):
ax.scatter(g.PC1, g.PC2, s=14, alpha=.7, label=f'{label} ({len(g)})')
ax.set_xlabel(f'PC1 ({pcs.explained_variance_ratio_[0]:.0%})'); ax.set_ylabel(f'PC2 ({pcs.explained_variance_ratio_[1]:.0%})')
ax.set_title('TARGET-ALL-P1: PCA of log2 TPM, top 2,000 variable genes'); ax.legend(fontsize=8); plt.tight_layout()
loadings = pd.Series(pcs.components_[0], index=var.index[:2000]).sort_values()
print('\nPC1 is driven by:', ', '.join(loadings.index[:8]), '... vs ...', ', '.join(loadings.index[-8:]))
Output:
19,938 protein-coding genes x 12 aliquots sample_type Primary Blood Derived Cancer - Peripheral Blood 6 Primary Blood Derived Cancer - Bone Marrow 6 PC1 is driven by: HLA-DQA2, TAMALIN, GP6, CRLF2, XIRP1, ATF3, FOSL1, SH3D19 ... vs ... HBA2, HBB, CR2, ADGRF1, RPS4Y1, RPS4Y2, MSTN, HBA1
Try this in the AI assistant
The chat panel is prefilled with the first of these; the rest are good next steps.
- *Cluster the TARGET-ALL-P1 tumors on expression and tell me what distinguishes the clusters.*
- *List the raw files in MANIFEST.parquet for one case and read its STAR counts file straight from the GDC bucket.*
Everything above reads the project's tables through scigantic_nci.gdc: the mount here, the public bucket anywhere else. For the raw per-sample files, gdc.manifest("TARGET-ALL-P1") has the S3 bucket and key of all 18 open files of TARGET-ALL-P1, and gdc.fetch_raw(row) downloads one and checks its md5; newer GDC objects are KMS-encrypted, so it uses a signed boto3 client and falls back to the GDC HTTPS URL. Cite the generating program and the GDC (Grossman et al., NEJM 2016) in anything you publish.
Using R instead?
The same parquet tables read directly with arrow, which is preinstalled on the R + Bioconductor image along with survival, glmnet, survminer, timeROC, limma and DESeq2. Pick that image in the image menu before launching. Then open a new R notebook for your own code and keep this one as the reference for what the tables mean: these cells are Python, and switching this notebook's kernel to R makes every one of them fail.
library(arrow)
root <- "/mnt/archive" # outside Scigantic: s3://scigantic-gdc-open/TARGET-ALL-P1/
cases <- read_parquet(file.path(root, "clinical/cases.parquet"))
diagnoses <- read_parquet(file.path(root, "clinical/diagnoses.parquet"))
diagnoses <- diagnoses[diagnoses$diagnosis_is_primary_disease %in% TRUE, ] # one row per case
tpm <- read_parquet(file.path(root, "rna_seq/tpm_unstranded.parquet")) # genes x aliquots
samples <- read_parquet(file.path(root, "rna_seq/samples.parquet")) # aliquot column -> case, sample_type
Ages are in days (age_at_diagnosis / 365.25), survival time is days_to_death for the dead and days_to_last_follow_up otherwise, and the expression and mutation tables join through the case id, not the aliquot barcode. install.packages() on that image persists in your home folder across relaunches.
Run it
Open this notebook to run it, see the dataset page for the full description, or browse the full catalog.