DNABERT-2 117M multi-species genome foundation model weights

Provider
Zhihan Zhou, Han Liu et al. (MAGICS Lab)
License
Apache-2.0 (source)
Size
12 files
Format
JSON (in sampled files)
Stored at
Hugging Face (https://huggingface.co/zhihan1996/DNABERT-2-117M)
Access
Free and public. No account with the data provider is needed.
Category
Genomics & Bioinformatics
Source
https://arxiv.org/abs/2306.15006

About DNABERT-2 117M multi-species genome foundation model weights

Open weights for DNABERT-2, the official 117 million parameter genome foundation model from Zhou et al. (ICLR 2024), trained on genomes from many species. It replaces k-mer tokens with byte pair encoding over DNA and uses ALiBi attention, so it accepts sequences of varying length and produces 768-dimensional embeddings by mean or max pooling over hidden states. The mounted repo holds pytorch_model.bin (468 MB), config.json, the BPE tokenizer and the custom MosaicBERT modelling code, loaded with transformers and trust_remote_code from the mount path. Pair it with the reference genome, 1000 Genomes or HPRC archives to embed sequence next to the data. Apache 2.0.

How to load DNABERT-2 117M multi-species genome foundation model weights in Python

import os, json
base = "/mnt/archive"   # this model repo, mounted read-only
files = sorted(os.listdir(base)); print(files)
print(json.dumps(json.load(open(os.path.join(base, "config.json"))), indent=1)[:800])
print("pytorch_model.bin", os.path.getsize(os.path.join(base, "pytorch_model.bin")) // 2**20, "MB")

Written for a Scigantic notebook, where /mnt/archive is the mounted dataset.

Data structure

File types in sampled files: JSON

Files

Example file: pytorch_model.bin

Analyze this dataset without downloading it

On Scigantic, DNABERT-2 117M multi-species genome foundation model weights opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Genomics & Bioinformatics datasets