PubChem: compound identifier, name and mass registry

PubChem is NCBI's open chemistry database, with over 124.6 million compounds. This archive is a narrow, weekly-refreshed parquet mirror of PubChem's compound identifier registry, not a mirror of the full PubChem Compound database: smiles.parquet (isomeric SMILES), titles.parquet (compound summary page title), iupac_names.parquet (computed IUPAC name), inchi_keys.parquet (full InChI and InChIKey, one row per protonation state), mass.parquet (molecular formula, monoisotopic and exact mass), parent.parquet (CID-to-parent-CID mapping), cid_sid.parquet (CID-to-Substance-ID provenance links) and synonyms.parquet (all known names per CID, multiple rows per CID). Together these answer the identifier/name/mass point-lookups (resolve an InChIKey, name or CID to the others) that a parquet file sorted by CID cannot serve efficiently at 100M+ rows if the query filters on anything else, which is why this mirror exists as a companion to the scigantic-pubchem PyPI package rather than a replacement for it: BioAssay results, gene/protein data, 3D conformers, fingerprint similarity search and substructure search all stay served live through that package's PUG REST wrapper, since PubChem has no release versioning (every file here is regenerated in place continuously) and those larger, less identifier-shaped datasets don't fit a static snapshot the way this narrow registry does. This mirror itself is refreshed weekly by a scheduled job that only re-downloads when NCBI's source files actually changed. Useful for bulk identifier resolution and cross-referencing (e.g. joining against BindingDB's measurements.parquet via its pubchem_cid column) without per-lookup API round trips. License is not a simple public-domain/CC0 claim: per NCBI's own site policy, NCBI asserts no restriction on this data's use or distribution, but does not warrant that individual depositor-submitted records are free of third-party patent/copyright claims, since rights are never transferred from depositors to NCBI. See _MIRROR.txt in the archive for the full attribution text.

Category
Drug Discovery
Provider
National Center for Biotechnology Information (NCBI)
License
No restriction asserted by NCBI; not warranted free of third-party depositor claims (see NCBI site policy)
Hosted on
Amazon S3, bucket scigantic-pubchem, us-east-1
Access
Free and public. No account with the data provider is needed.
Source
https://pubchem.ncbi.nlm.nih.gov/
Data structure and loading code

Files

Example file: parquet/cid_sid.parquet

Load the data

import os
import pyarrow.parquet as pq
pf = pq.ParquetFile(os.path.join(os.environ['SCIGANTIC_MOUNT_PATH'], 'parquet/cid_sid.parquet'))
print(pf.metadata)
df = pf.read_row_group(0).to_pandas()
df.head()

Run in a Scigantic notebook on 2026-11-02 against parquet/cid_sid.parquet: Parquet 391,123,662 rows, 3 columns, 3191 row groups; columns ['cid:int64', 'sid:int64', 'link_type:int8'].

Analyze this dataset without downloading it

On Scigantic, PubChem: compound identifier, name and mass registry opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

Related Drug Discovery datasets