CPTAC-3 CPTAC-Brain, Head and Neck, Kidney, Lung, Pancreas, Uterus (GDC open-access tables, 1,866 cases): sample notebook

A worked notebook for CPTAC-3 CPTAC-Brain, Head and Neck, Kidney, Lung, Pancreas, Uterus (GDC open-access tables, 1,866 cases), with 7 code cells and the output each one printed. On Scigantic it opens in a hosted JupyterLab with the dataset already mounted, so nothing is downloaded first.

Provider
NCI Genomic Data Commons; Clinical Proteomic Tumor Analysis Consortium; tables assembled by Scigantic
License
GDC open-access tier: no access restrictions; NIH Genomic Data Sharing Policy applies to the source. Cite CPTAC (https://proteomics.cancer.gov/programs/cptac) and the cohort paper (ccRCC Clark 2019, LUAD Gillette 2020, LSCC Satpathy 2021, HNSCC Huang 2021, GBM Wang 2021, UCEC Dou 2020, PDAC Cao 2021, and others).
Size
22 parquet tables, 1270.1 MB total, plus README.md and BUILD_REPORT.json
Format
Parquet (in sampled files)
Stored at
Amazon S3, s3://scigantic-gdc-open/CPTAC-3 (us-east-1)
Access
Free and public. No account with the data provider is needed.

The notebook

CPTAC-3: CPTAC-Brain, Head and Neck, Kidney, Lung, Pancreas, Uterus, ready to analyze

This mount is Scigantic's per-project re-arrangement of the NCI Genomic Data Commons open-access files for CPTAC-3 (1,866 cases) into parquet tables you can read directly: clinical, copy_number, mirna, rna_seq, somatic_mutations. The GDC's own bucket stores the same data as 24,171 files in one flat UUID namespace; here every matrix is already stacked (one column per aliquot) and every table is documented in README.md and BUILD_REPORT.json at the mount root.

What is not here: aligned reads and germline calls (controlled access via dbGaP), and the whole-slide images (listed in MANIFEST.parquet with their S3 location; the same slides are on the NCI Imaging Data Commons in DICOM).

Built from Data Release 46.0 - August 10, 2026. Values are unchanged from the GDC source files; only the arrangement is new.

The code below reads every table with the scigantic-nci package (pip install scigantic-nci, already installed in this image): from scigantic_nci import gdc, then one call per table, such as gdc.clinical("CPTAC-3"). Run anywhere else, the same calls read s3://scigantic-gdc-open/CPTAC-3/ anonymously.

import os, json
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from scigantic_nci import gdc
plt.rcParams['figure.dpi'] = 72  # keeps the saved figures small enough for the notebook preview
try:
    from matplotlib_inline.backend_inline import set_matplotlib_formats
    set_matplotlib_formats('jpeg')  # JPEG keeps every inline figure well under the preview size limit
except ImportError:
    pass

P = 'CPTAC-3'
# scigantic_nci reads the mounted archive when its BUILD_REPORT.json names this project, else s3://scigantic-gdc-open/<project>/ anonymously.
M = os.environ.get('SCIGANTIC_MOUNT_PATH', '/mnt/archive')
on_mount = os.path.exists(f'{M}/BUILD_REPORT.json') and json.load(open(f'{M}/BUILD_REPORT.json')).get('project') == P
print('reading', P, 'from', M if on_mount else f's3://scigantic-gdc-open/{P}/')

report = gdc.report(P)
print(report['project'], '|', report['gdc_data_release'], '| built', report['built_at'][:10])
print(f"{report['files_read']:,} source files read and md5-verified, {report['bytes_read']/1e9:.1f} GB\n")
for t in gdc.tables(P).itertuples():
    print(f"{t.table + '.parquet':58s} {t.rows:>9,} rows x {t.columns:>5,} cols  {t.bytes/1e6:8.1f} MB")

Output:

reading CPTAC-3 from /mnt/archive
CPTAC-3 | Data Release 46.0 - August 10, 2026 | built 2026-09-13
16,233 source files read and md5-verified, 21.7 GB

MANIFEST.parquet                                              24,171 rows x    19 cols       3.5 MB
clinical/aliquots.parquet                                      9,497 rows x     7 cols       0.3 MB
clinical/cases.parquet                                         1,866 rows x    30 cols       0.2 MB
clinical/diagnoses.parquet                                     1,857 rows x    35 cols       0.2 MB
clinical/exposures.parquet                                     1,807 rows x    19 cols       0.1 MB
clinical/follow_ups.parquet                                    7,028 rows x    14 cols       0.3 MB
clinical/samples.parquet                                       7,463 rows x    24 cols       0.3 MB
clinical/treatments.parquet                                    1,688 rows x    13 cols       0.1 MB
copy_number/allele_specific_segments_ascatngs.parquet        511,529 rows x     9 cols       5.9 MB
copy_number/gene_level_ascatngs.parquet                       60,623 rows x 1,904 cols       8.2 MB
copy_number/gene_level_ascatngs_samples.parquet                1,899 rows x    11 cols       0.1 MB
...

1. Who is in the cohort

gdc.clinical(project) returns one row per case: clinical/cases.parquet (demographics flattened in as demographic_*) joined to the case's primary row of clinical/diagnoses.parquet. The diagnoses table (gdc.diagnoses) has more rows than cases because metastasis, recurrence and prior-primary entries each get their own row; the join keeps the row flagged diagnosis_is_primary_disease, which is the one you want for cohort tables and survival. GDC stores ages in days; the package adds age_at_diagnosis_years, os_time_days, os_event and stage.

cohort = gdc.clinical(P)  # cases joined to the primary diagnosis, one row per case
dx_all = gdc.diagnoses(P)
n_primary = int(cohort['submitter_id_dx'].notna().sum()) if 'submitter_id_dx' in cohort else len(cohort)  # diagnosis columns that clash with case columns carry _dx
print(f'{len(cohort)} cases, {len(dx_all)} diagnosis rows, {n_primary} primary diagnoses')
sex_col = next((c for c in ('demographic_sex_at_birth', 'demographic_gender') if c in cohort), None)  # GDC renamed gender -> sex_at_birth in 2026
if sex_col: print(f'\n{sex_col}:'); print(cohort[sex_col].value_counts(dropna=False).to_string())
yrs = cohort['age_at_diagnosis_years']
if yrs.notna().any():
    print(f'\nAge at diagnosis: median {yrs.median():.0f} y (IQR {yrs.quantile(.25):.0f}-{yrs.quantile(.75):.0f})')
print('\nVital status:'); print(cohort['demographic_vital_status'].value_counts(dropna=False).to_string())
print('\nPrimary diagnosis (top 5):'); print(cohort['primary_diagnosis'].value_counts().head(5).to_string())
stage = cohort['stage'].fillna('Not reported')  # gdc.clinical copies the best-covered stage column; its name is in cohort.attrs['stage_column']
print(f"\nStage ({cohort.attrs['stage_column']}):"); print(stage.value_counts().to_string())
ax = stage.value_counts().sort_index().plot.bar(figsize=(7, 3), color='#4a6fa5', title='CPTAC-3: ajcc_pathologic_stage')
ax.set_ylabel('cases'); plt.tight_layout()

Output:

1866 cases, 1857 diagnosis rows, 1842 primary diagnoses

demographic_sex_at_birth:
demographic_sex_at_birth
male      1003
female     863

Age at diagnosis: median 65 y (IQR 57-71)

Vital status:
demographic_vital_status
Alive           1087
Dead             700
Not Reported      79

...

2. Overall survival by stage

gdc.clinical already carries the survival columns: os_time_days is demographic_days_to_death for cases whose vital status is Dead, otherwise the primary diagnosis's days_to_last_follow_up, and os_event is death. The package has no survival estimator, so the Kaplan-Meier curve below is ten lines of numpy (lifelines is not installed in this image).

def kaplan_meier(t, e):
    t, e = np.asarray(t, float), np.asarray(e, bool)
    order = np.argsort(t); t, e = t[order], e[order]
    s, out = 1.0, [(0.0, 1.0)]
    for ti in np.unique(t):
        at_risk = (t >= ti).sum(); d = ((t == ti) & e).sum()
        s *= 1 - d / at_risk; out.append((ti, s))
    return pd.DataFrame(out, columns=['days', 'survival'])

# scigantic_nci has no Kaplan-Meier estimator, so the numpy function above does it from os_time_days and os_event
surv = cohort.dropna(subset=['os_time_days', 'stage'])
surv = surv[surv['os_time_days'] >= 0]
groups = [(s, g) for s, g in surv.groupby('stage') if len(g) >= 10]
fig, ax = plt.subplots(figsize=(7, 4.5))
for stage, g in groups:
    curve = kaplan_meier(g['os_time_days'] / 365.25, g['os_event'])
    ax.step(curve['days'], curve['survival'], where='post', label=f'{stage} (n={len(g)}, {int(g.os_event.sum())} deaths)')
ax.set_xlabel('years since diagnosis'); ax.set_ylabel('overall survival'); ax.set_ylim(0, 1.02); ax.set_xlim(0, 10)
ax.set_title('CPTAC-3: Kaplan-Meier by ajcc_pathologic_stage'); ax.legend(fontsize=8); plt.tight_layout()
print(f'{len(surv)} cases with a survival time; {int(surv.os_event.sum())} deaths; median follow-up of survivors {surv.loc[~surv.os_event, "os_time_days"].median()/365.25:.1f} y')

Output:

1721 cases with a survival time; 655 deaths; median follow-up of survivors 2.7 y

3. The mutation landscape

somatic_mutations/masked_somatic_mutations.parquet is every aliquot-level masked MAF of the project stacked (648,554 rows, 2,086 tumors, all 140 GDC MAF columns). gdc.mutations(project, genes=..., non_silent=True, columns=...) reads it with the gene and class filters pushed down to parquet, and gdc.mutation_frequency(project) counts per unique tumor barcode. "Non-silent" is the maftools set in gdc.NON_SILENT: missense, nonsense, nonstop, frameshift and in-frame indels, splice site and translation start. Long genes such as TTN and MUC16 rank high in nearly every cohort because of their length, not because they matter; the driver you should look at first here is TP53.

freq = gdc.mutation_frequency(P)  # fraction of tumors with a non-silent call, per gene
n_tumors = freq.attrs['n_samples']  # every tumor barcode in the MAF, silent calls included
ns = gdc.mutations(P, non_silent=True, columns=['Hugo_Symbol', 'Variant_Classification', 'Tumor_Sample_Barcode', 'HGVSp_Short'])
n_calls = report['tables']['somatic_mutations/masked_somatic_mutations']['rows']
per_tumor = ns.groupby('Tumor_Sample_Barcode').size()
print(f'{n_calls:,} calls, {len(ns):,} non-silent, {n_tumors} tumors; median {per_tumor.median():.0f} non-silent mutations per tumor (max {per_tumor.max()})')
top = freq.head(20)
fig, axes = plt.subplots(1, 2, figsize=(12, 4.5), gridspec_kw={'width_ratios': [1, 1.4]})
top.iloc[::-1].plot.barh(ax=axes[0], color='#4a6fa5'); axes[0].set_xlabel('fraction of tumors with a non-silent mutation'); axes[0].set_title('most frequently mutated genes')
# oncoprint-style presence matrix for the top 12 genes
genes12 = top.head(12).index
pres = ns[ns.Hugo_Symbol.isin(genes12)].groupby(['Hugo_Symbol', 'Tumor_Sample_Barcode']).size().unstack(fill_value=0).reindex(genes12).gt(0)
pres = pres.loc[:, pres.sum().sort_values(ascending=False).index]
axes[1].imshow(pres.values, aspect='auto', cmap='Blues', interpolation='nearest')
axes[1].set_yticks(range(len(genes12))); axes[1].set_yticklabels(genes12, fontsize=8); axes[1].set_xticks([]); axes[1].set_xlabel(f'{pres.shape[1]} tumors with at least one of these')
axes[1].set_title('who has what'); plt.tight_layout()
gene_calls = gdc.mutations(P, genes='TP53', non_silent=True)  # one gene: the parquet row filter reads only its calls
print(f'\nTP53: {gene_calls.Tumor_Sample_Barcode.nunique()} tumors; commonest protein changes:')
print(gene_calls.HGVSp_Short.value_counts().head(6).to_string())

Output:

648,554 calls, 434,485 non-silent, 2086 tumors; median 57 non-silent mutations per tumor (max 15044)

TP53: 747 tumors; commonest protein changes:
HGVSp_Short
p.R175H    29
p.R282W    22
p.R273H    19
p.R248W    16
p.R248Q    16
p.R213*    15

4. Expression: tumor versus normal

rna_seq/tpm_unstranded.parquet is genes x aliquots (60,660 GENCODE v36 genes, 2,968 aliquots, of which 761 are normal tissue). gdc.expression(project, protein_coding=True) returns it indexed by gene name, with the _PAR_Y duplicate rows dropped and the first row of any repeated name kept; gdc.expression_samples(project) says what each column is. The matrix is 603 MB on disk and reads into a 4 GiB pod in seconds; restrict to protein-coding genes and log2(TPM+1) before any distance-based method.

from sklearn.decomposition import PCA

samples = gdc.expression_samples(P).set_index('column')  # matrix column (aliquot) -> case_submitter_id, sample_type
tpm = gdc.expression(P, protein_coding=True)  # genes x aliquots TPM, indexed by gene name
logtpm = np.log2(tpm + 1)
print(f'{logtpm.shape[0]:,} protein-coding genes x {logtpm.shape[1]:,} aliquots')
print(samples.sample_type.value_counts().to_string())

# 2,000 most variable genes, centred, PCA
var = logtpm.var(axis=1).sort_values(ascending=False)
X = logtpm.loc[var.index[:2000]].T
X = X - X.mean()
pcs = PCA(n_components=5).fit(X)
Z = pd.DataFrame(pcs.transform(X)[:, :2], index=X.index, columns=['PC1', 'PC2']).join(samples[['sample_type', 'case_submitter_id']])

fig, ax = plt.subplots(figsize=(6.5, 5))
for label, g in Z.groupby('sample_type'):
    ax.scatter(g.PC1, g.PC2, s=14, alpha=.7, label=f'{label} ({len(g)})')
ax.set_xlabel(f'PC1 ({pcs.explained_variance_ratio_[0]:.0%})'); ax.set_ylabel(f'PC2 ({pcs.explained_variance_ratio_[1]:.0%})')
ax.set_title('CPTAC-3: PCA of log2 TPM, top 2,000 variable genes'); ax.legend(fontsize=8); plt.tight_layout()
loadings = pd.Series(pcs.components_[0], index=var.index[:2000]).sort_values()
print('\nPC1 is driven by:', ', '.join(loadings.index[:8]), '... vs ...', ', '.join(loadings.index[-8:]))

Output:

19,938 protein-coding genes x 2,968 aliquots
sample_type
Primary Tumor                                      2006
Solid Tissue Normal                                 761
Primary Blood Derived Cancer - Bone Marrow          101
Primary Blood Derived Cancer - Peripheral Blood      51
Recurrent Tumor                                      21
                                                     17
Metastatic                                           11

PC1 is driven by: SPP1, OLFM4, LBP, RPTN, KRT19, SYT13, CP, PAEP ... vs ... S100A14, TACSTD2, APOD, RAB25, KRT5, PIGR, KRT14, CEACAM6
from scipy import stats
tumor_cols = samples.index[samples.sample_type == 'Primary Tumor']
normal_cols = samples.index[samples.sample_type.str.contains('Normal', na=False)]
lt, ln = logtpm[tumor_cols], logtpm[normal_cols]
t, p = stats.ttest_ind(lt, ln, axis=1, equal_var=False)
p = np.maximum(p, 1e-300)  # underflows to 0 for the strongest genes, which would vanish from a -log10 plot
de = pd.DataFrame({'log2fc': lt.mean(1) - ln.mean(1), 'p': p}, index=logtpm.index).dropna()
order = np.argsort(de.p.values); ranked = de.p.values[order] * len(de) / (np.arange(len(de)) + 1)
de['padj'] = np.nan; de.iloc[order, de.columns.get_loc('padj')] = np.minimum(1, np.minimum.accumulate(ranked[::-1])[::-1])  # Benjamini-Hochberg
sig = de[(de.padj < 0.01) & (de.log2fc.abs() > 2)].sort_values('log2fc')
print(f'{len(tumor_cols)} tumors vs {len(normal_cols)} normals: {len(sig)} genes with |log2FC| > 2 and BH-adjusted p < 0.01')
print('\nmost down in tumor:', ', '.join(sig.index[:10])); print('most up in tumor:  ', ', '.join(sig.index[-10:][::-1]))
fig, ax = plt.subplots(figsize=(6.5, 4.5))
ax.scatter(de.log2fc, -np.log10(de.p), s=4, alpha=.4, color='#9aa5b1'); ax.scatter(sig.log2fc, -np.log10(sig.p), s=6, color='#c0392b')
for g in list(sig.index[:5]) + list(sig.index[-5:]): ax.annotate(g, (de.loc[g, 'log2fc'], -np.log10(de.loc[g, 'p'])), fontsize=7)
ax.set_xlabel('log2 fold change (tumor / normal)'); ax.set_ylabel('-log10 p'); ax.set_title('CPTAC-3: tumor vs normal, protein-coding TPM'); plt.tight_layout()

Output:

2006 tumors vs 761 normals: 130 genes with |log2FC| > 2 and BH-adjusted p < 0.01

most down in tumor: SFTPC, ADH1B, C7, SFTPA1, SFTPA2, CYP4B1, CHRDL1, CLEC3B, SFTPB, GPC3
most up in tumor:   SPP1, TOP2A, MMP11, UBE2C, PRAME, TPX2, CA9, H2BC17, CENPF, ANLN

5. Join the layers: does a TP53 mutation show in expression?

The mutation table is keyed by tumor aliquot barcode and the expression matrix by RNA aliquot barcode; they are different aliquots of the same sample, so the join goes through case: gdc.expression_samples gives case_submitter_id for every expression column, and gdc.aliquots(project) maps the MAF's Tumor_Sample_Barcode to its case (for TCGA and TARGET barcodes gdc.case_for_barcode truncates to the same id). Here: TP53 expression in primary tumors that carry a non-silent TP53 mutation versus the rest.

from scipy import stats
case_of = gdc.aliquots(P).drop_duplicates('aliquot_submitter_id').set_index('aliquot_submitter_id')['case_submitter_id']
def to_cases(barcodes):
    b = pd.Series(barcodes).drop_duplicates()
    return set(b.map(case_of).fillna(b.map(gdc.case_for_barcode)))  # aliquot table first, positional TCGA/TARGET barcode as the fallback
mut_cases = to_cases(ns.loc[ns.Hugo_Symbol == 'TP53', 'Tumor_Sample_Barcode'])
maf_cases = to_cases(gdc.mutations(P, columns=['Tumor_Sample_Barcode']).Tumor_Sample_Barcode)  # every sequenced tumor, silent calls included
tum = samples[samples.sample_type.str.startswith('Primary', na=False)].copy()  # Primary Tumor, or Primary Blood Derived Cancer in the leukemias
tum = tum[tum.case_submitter_id.isin(maf_cases)]  # only cases that were sequenced for mutations
tum['TP53_mutated'] = tum.case_submitter_id.isin(mut_cases)
if 'TP53' not in logtpm.index:
    raise SystemExit('TP53 is not a protein-coding row of the expression matrix; pick another gene from the frequency table above.')
expr = logtpm.loc['TP53', tum.index]
a, b = expr[tum['TP53_mutated']], expr[~tum['TP53_mutated']]
u, p = stats.mannwhitneyu(a, b) if len(a) > 2 and len(b) > 2 else (np.nan, np.nan)
print(f'TP53 mutated: {len(a)} tumors, median log2 TPM {a.median():.2f}; wild-type: {len(b)}, median {b.median():.2f}; Mann-Whitney p = {p:.2g}')
fig, ax = plt.subplots(figsize=(4.5, 4))
ax.boxplot([b, a], tick_labels=[f'wild-type (n={len(b)})', f'mutated (n={len(a)})'], widths=.5)
ax.set_ylabel('TP53 log2(TPM+1)'); ax.set_title('CPTAC-3: TP53 expression by mutation status'); plt.tight_layout()

Output:

TP53 mutated: 766 tumors, median log2 TPM 4.18; wild-type: 1345, median 4.55; Mann-Whitney p = 2.6e-17

Try this in the AI assistant

The chat panel is prefilled with the first of these; the rest are good next steps.

Everything above reads the project's tables through scigantic_nci.gdc: the mount here, the public bucket anywhere else. For the raw per-sample files, gdc.manifest("CPTAC-3") has the S3 bucket and key of all 24,171 open files of CPTAC-3, and gdc.fetch_raw(row) downloads one and checks its md5; newer GDC objects are KMS-encrypted, so it uses a signed boto3 client and falls back to the GDC HTTPS URL. Cite the generating program and the GDC (Grossman et al., NEJM 2016) in anything you publish.

Using R instead?

The same parquet tables read directly with arrow, which is preinstalled on the R + Bioconductor image along with survival, glmnet, survminer, timeROC, limma and DESeq2. Pick that image in the image menu before launching. Then open a new R notebook for your own code and keep this one as the reference for what the tables mean: these cells are Python, and switching this notebook's kernel to R makes every one of them fail.

library(arrow)
root <- "/mnt/archive"   # outside Scigantic: s3://scigantic-gdc-open/CPTAC-3/
cases     <- read_parquet(file.path(root, "clinical/cases.parquet"))
diagnoses <- read_parquet(file.path(root, "clinical/diagnoses.parquet"))
diagnoses <- diagnoses[diagnoses$diagnosis_is_primary_disease %in% TRUE, ]  # one row per case
tpm       <- read_parquet(file.path(root, "rna_seq/tpm_unstranded.parquet"))  # genes x aliquots
samples   <- read_parquet(file.path(root, "rna_seq/samples.parquet"))         # aliquot column -> case, sample_type
maf       <- read_parquet(file.path(root, "somatic_mutations/masked_somatic_mutations.parquet"))

Ages are in days (age_at_diagnosis / 365.25), survival time is days_to_death for the dead and days_to_last_follow_up otherwise, and the expression and mutation tables join through the case id, not the aliquot barcode. install.packages() on that image persists in your home folder across relaunches.

Run it

Open this notebook to run it, see the dataset page for the full description, or browse the full catalog.