DNABERT-2 117M multi-species genome foundation model weights
- Provider
- Zhihan Zhou, Han Liu et al. (MAGICS Lab)
- License
- Apache-2.0 (source)
- Size
- 12 files
- Format
- JSON (in sampled files)
- Stored at
- Hugging Face (https://huggingface.co/zhihan1996/DNABERT-2-117M)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Genomics & Bioinformatics
- Source
- https://arxiv.org/abs/2306.15006
About DNABERT-2 117M multi-species genome foundation model weights
Open weights for DNABERT-2, the official 117 million parameter genome foundation model from Zhou et al. (ICLR 2024), trained on genomes from many species. It replaces k-mer tokens with byte pair encoding over DNA and uses ALiBi attention, so it accepts sequences of varying length and produces 768-dimensional embeddings by mean or max pooling over hidden states. The mounted repo holds pytorch_model.bin (468 MB), config.json, the BPE tokenizer and the custom MosaicBERT modelling code, loaded with transformers and trust_remote_code from the mount path. Pair it with the reference genome, 1000 Genomes or HPRC archives to embed sequence next to the data. Apache 2.0.
How to load DNABERT-2 117M multi-species genome foundation model weights in Python
import os, json
base = "/mnt/archive" # this model repo, mounted read-only
files = sorted(os.listdir(base)); print(files)
print(json.dumps(json.load(open(os.path.join(base, "config.json"))), indent=1)[:800])
print("pytorch_model.bin", os.path.getsize(os.path.join(base, "pytorch_model.bin")) // 2**20, "MB")Written for a Scigantic notebook, where /mnt/archive is the mounted dataset.
Data structure
File types in sampled files: JSON
Files
Example file: pytorch_model.bin
Analyze this dataset without downloading it
On Scigantic, DNABERT-2 117M multi-species genome foundation model weights opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Genomics & Bioinformatics datasets
- TCGA-UCS Uterine Carcinosarcoma (GDC open-access tables, 57 cases)
- dbSNP (NCBI Single Nucleotide Polymorphism Database)
- Conformational Space of Short Peptides
- T2T HG002 Q100 Complete Diploid Genome Assembly (hg002v1.2)
- Tumor cell villages define the co-dependency of tumor and microenvironment in liver cancer
- ORGANOID-PANCREATIC Pancreas Cancer Organoid Profiling (GDC open-access tables, 70 cases)
- Cell-Free DNA Nucleosome Footprint (Plasma cfDNA Whole-Genome Sequencing)
- Cancer Genome Characterization Initiatives - Burkitt Lymphoma, HIV+ Cervical Cancer