Every measured drug-target pair

Medicinal chemistry sample notebook on Scigantic.

Pull a potency table for any target out of ChEMBL's 24.5 million measurements, with the filters that separate real numbers from noise already applied.

Figure: Two-dimensional structures of twenty-four approved drugs drawn from ChEMBL, white skeletons on a dark field

The notebook

2 code cells, shown with the output each one printed.

ChEMBL 37, and what it does to your training set

Two things are mounted read-only:

EBI ships ChEMBL as a 30 GB SQLite file. Scigantic pre-processed it so you don't have to. That file is the wrong shape for a mounted filesystem and for an agent: answering one question means dragging the whole database across the network. We expanded it once, re-encoded all 72 tables to Parquet (30.5 GB becomes 1.66 GB), and published derived tables next to it that answer the common questions directly.

filewhat it answerscost
activities_enriched.parquet"what is measured against target X"0.9s, no joins
fingerprints.parquet"what looks like this molecule"0.5s over 1.68M compounds
chembl_cyp_training.parquet"give me public CYP data I can train on"one read

The five-table join behind that first file takes 9 seconds if you write it yourself against the raw tables, and does not finish at all against the SQLite copy. The assistant panel on the right reads a schema card describing all of it, including the traps, so you can ask in English instead of learning the schema. Everything here is also queryable from anywhere with no account, straight off s3://scigantic-chembl.

import os
from pathlib import Path
import duckdb

ARCHIVE = Path(os.environ.get("SCIGANTIC_MOUNT_PATH", Path.home() / "archive"))
DERIVED = ARCHIVE / "chembl_37" / "derived"
enriched = DERIVED / "activities_enriched.parquet"

# The five-table join is already done in this file, and the filters that
# separate real measurements from noise are already applied.
df = duckdb.connect().execute(f"""
    SELECT compound_chembl_id, compound_pref_name, standard_type,
           standard_value, standard_units, pchembl_value, canonical_smiles
    FROM read_parquet('{enriched}')
    WHERE target_chembl_id = 'CHEMBL203'      -- EGFR. Swap in any target.
      AND target_type = 'SINGLE PROTEIN'
      AND confidence_score >= 8
    ORDER BY pchembl_value DESC
""").df()

print(f"{len(df):,} activities, {df.compound_chembl_id.nunique():,} distinct compounds")
df.head(5)

Output:

18,998 activities, 11,202 distinct compounds
  compound_chembl_id compound_pref_name standard_type  standard_value  \
0      CHEMBL4650319       MOBOCERTINIB          IC50           0.010   
1      CHEMBL5270693               None          IC50           0.010   
2       CHEMBL176582               None          IC50           0.010   
3      CHEMBL5790648               None          IC50           0.016   
4      CHEMBL5746224               None          IC50           0.017   

  standard_units  pchembl_value  \
0             nM          11.00   
1             nM          11.00   
2             nM          11.00   
3             nM          10.80   
4             nM          10.77   

...

Now the part that costs people models

The challenge training set is small, so the obvious move is to pad it with public data. ChEMBL has roughly six times as many compounds on the same endpoint, on the same scale (pchembl_value is the same quantity as the challenge's pIC50_direct_inhibition).

Before concatenating, plot the labels.

import os
from pathlib import Path
import duckdb, pandas as pd

ARCHIVE = Path(os.environ.get("SCIGANTIC_MOUNT_PATH", Path.home() / "archive"))
DERIVED = ARCHIVE / "chembl_37" / "derived"

# Companion archives mount under /mnt/archives/<repo>. Live-added mounts land
# under ~/additional-datasets instead, so check both rather than assuming which
# way this notebook was started. followlinks because the second one is a symlink.
roots = [Path("/mnt/archives"), Path.home() / "additional-datasets"]
hits = [os.path.join(d, f)
        for root in roots if root.exists()
        for d, _sub, files in os.walk(root, followlinks=True)
        for f in files if f.endswith("TRAIN_inhibition.csv")]
if not hits:
    seen = {str(r): sorted(p.name for p in r.iterdir()) for r in roots if r.exists()}
    raise FileNotFoundError(f"No *TRAIN_inhibition.csv under {roots}. Found: {seen or 'nothing mounted'}")

challenge = pd.read_csv(hits[0])["CYP3A4_pIC50_direct_inhibition"].dropna()

cyp_file = DERIVED / "chembl_cyp_training.parquet"
chembl = duckdb.connect().execute(
    f"SELECT CYP3A4_pIC50 FROM read_parquet('{cyp_file}') WHERE CYP3A4_pIC50 IS NOT NULL"
).df()["CYP3A4_pIC50"]

for name, s in (("challenge", challenge), ("ChEMBL", chembl)):
    print(f"{name:10} n={len(s):>6,}   below pIC50 4.0: {100 * (s < 4).mean():5.1f}%")

ax = challenge.plot(kind="hist", bins=40, density=True, alpha=0.55,
                    label="challenge", figsize=(9, 4))
chembl.plot(kind="hist", bins=40, density=True, alpha=0.55, label="ChEMBL", ax=ax)
ax.set_xlabel("CYP3A4 pIC50")
ax.set_ylabel("density")
ax.legend()
ax.set_title("Public data has no weak compounds in it")

Output:

challenge  n= 2,335   below pIC50 4.0:  40.4%
ChEMBL     n=16,477   below pIC50 4.0:   0.0%
Text(0.5, 1.0, 'Public data has no weak compounds in it')

Nothing gets published without a dose-response curve, and weak compounds never produce one. So ChEMBL is an excellent source of actives and a poor source of inactives, and the union of the two is not one sample.

Pretrain on it and fine-tune on the challenge set, or use it as extra actives, or reweight. chembl_cyp_training.parquet already excludes the one blinded-test compound that appears in ChEMBL, and flags the 155 that overlap the training split.

Ask the assistant panel for a different target, a structure search, or the rest of the traps in this release.

Run it yourself

This sample opens as a hosted JupyterLab notebook with its datasets already mounted, so nothing has to be downloaded first. The notebook arrives with its outputs already rendered, and an AI assistant in the notebook can explain each step or change the analysis. Open the sample or browse the full catalog.