UR100P: AMPLIFY protein language model training and validation sets (UniProt, OAS paired antibodies, SCOP 2)

Provider
Chandar Lab (Fournier et al.)
License
CC-BY-4.0 (source)
Size
45 files
Format
CSV (in sampled files)
Stored at
Hugging Face (https://huggingface.co/datasets/chandar-lab/UR100P)
Access
Free and public. No account with the data provider is needed.
Category
Proteomics
Source
https://doi.org/10.1101/2024.09.23.614603

About this dataset

The curated protein sequence corpus used to train and evaluate AMPLIFY, the efficient protein language model from Chandar Lab. It combines UniProt (UniRef100) sequences, paired heavy and light antibody chains from the Observed Antibody Space (OAS), and domains from the Structural Classification of Proteins version 2 (SCOP 2), all collected in December 2023. Sequences with ambiguous residues (B, J, O, U, X, Z) were removed. OAS pairs appear in both heavy|light and light|heavy order separated by a chain-break token, and MMseqs2 was used to drop training sequences above 90% identity to the validation sets so that perplexity and downstream evaluations are not inflated by leakage. The repository (about 186 GB) has one folder per source, UniProt/, OAS/ and SCOP/, each with train and test splits; load a subset with Hugging Face datasets, for example load_dataset("chandar-lab/UR100P", data_dir="UniProt", split="test"). Useful for protein language model pretraining, antibody language modeling and leakage-controlled evaluation. The card states that UniProt, OAS and SCOP are licensed CC BY 4.0; cite Fournier et al., "Protein Language Models: Is Scaling Necessary?" (bioRxiv 2024) and the UniProt, OAS and SCOP papers.

How to load this dataset in Python

import pandas as pd
df = pd.read_csv("hf://datasets/chandar-lab/UR100P/OAS/test-00000-of-00001.csv")
df.head()

Data structure

File types in sampled files: CSV

Files

Example file: OAS/test-00000-of-00001.csv

Analyze this dataset without downloading it

On Scigantic, UR100P: AMPLIFY protein language model training and validation sets (UniProt, OAS paired antibodies, SCOP 2) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Proteomics datasets