ESM-2 650M protein language model weights (esm2_t33_650M_UR50D)

Provider
Meta AI (FAIR)
License
MIT (source)
Size
9 files
Format
JSON, HDF5 (in sampled files)
Stored at
Hugging Face (https://huggingface.co/facebook/esm2_t33_650M_UR50D)
Access
Free and public. No account with the data provider is needed.
Category
Biology
Source
https://www.science.org/doi/10.1126/science.ade2574

About ESM-2 650M protein language model weights (esm2_t33_650M_UR50D)

Open weights for ESM-2 650M, the 33-layer, 650 million parameter protein language model from Meta's ESM project (Lin et al., Science 2023), trained with masked language modelling on UniRef50 protein sequences. It produces per-residue and per-sequence embeddings used for fitness prediction, contact prediction, function annotation and as the base model for many fine-tuned protein predictors. The mounted repo holds model.safetensors, pytorch_model.bin and tf_model.h5 (about 2.6 GB each), config.json, vocab.txt and tokenizer files, and loads with transformers (EsmModel, EsmForMaskedLM) from the mount path. Pair it with the UniRef, ProteinGym or PDB archives to embed sequences next to the data. MIT licence.

How to load ESM-2 650M protein language model weights (esm2_t33_650M_UR50D) in Python

import os, json
base = "/mnt/archive"   # this model repo, mounted read-only
files = sorted(os.listdir(base)); print(files)
print(json.dumps(json.load(open(os.path.join(base, "config.json"))), indent=1)[:800])
import struct
with open(os.path.join(base, "model.safetensors"), "rb") as f:
    hdr = json.loads(f.read(struct.unpack("<Q", f.read(8))[0]))
tensors = {k: v for k, v in hdr.items() if k != "__metadata__"}
print(len(tensors), "tensors, e.g.", list(tensors.items())[:3])
# To load them: pip install safetensors torch, then safetensors.torch.load_file(path)

Written for a Scigantic notebook, where /mnt/archive is the mounted dataset.

Data structure

File types in sampled files: JSON, HDF5

Files

Example file: model.safetensors

Analyze this dataset without downloading it

On Scigantic, ESM-2 650M protein language model weights (esm2_t33_650M_UR50D) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Biology datasets