# DNABERT-2 117M multi-species genome foundation model weights

## Facts

- Provider: Zhihan Zhou, Han Liu et al. (MAGICS Lab)
- License: Apache-2.0 (https://github.com/MAGICS-LAB/DNABERT_2/blob/main/LICENSE)
- Size: 12 files
- Format: JSON (in sampled files)
- Stored at: Hugging Face, https://huggingface.co/zhihan1996/DNABERT-2-117M
- Access: public, readable without an account
- Category: Genomics & Bioinformatics
- Source: https://arxiv.org/abs/2306.15006
- Page: https://scigantic.com/archive/96532320-8a01-5447-8e00-0dc5afcb7640
- Keywords: Genomics & Bioinformatics, JSON, Hugging Face

## About

Open weights for DNABERT-2, the official 117 million parameter genome foundation model from Zhou et al. (ICLR 2024), trained on genomes from many species. It replaces k-mer tokens with byte pair encoding over DNA and uses ALiBi attention, so it accepts sequences of varying length and produces 768-dimensional embeddings by mean or max pooling over hidden states. The mounted repo holds pytorch_model.bin (468 MB), config.json, the BPE tokenizer and the custom MosaicBERT modelling code, loaded with transformers and trust_remote_code from the mount path. Pair it with the reference genome, 1000 Genomes or HPRC archives to embed sequence next to the data. Apache 2.0.

## Load the data

```python
import os, json
base = "/mnt/archive"   # this model repo, mounted read-only
files = sorted(os.listdir(base)); print(files)
print(json.dumps(json.load(open(os.path.join(base, "config.json"))), indent=1)[:800])
print("pytorch_model.bin", os.path.getsize(os.path.join(base, "pytorch_model.bin")) // 2**20, "MB")
```

Written for a Scigantic notebook, where /mnt/archive (SCIGANTIC_MOUNT_PATH) is the mounted dataset.

## Data structure

File types in sampled files: JSON

Example file: `pytorch_model.bin`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/96532320-8a01-5447-8e00-0dc5afcb7640
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/96532320-8a01-5447-8e00-0dc5afcb7640
- Over MCP: `get_schema_card` with id `96532320-8a01-5447-8e00-0dc5afcb7640` at https://api.scigantic.com/mcp

## Related topics

- [JSON datasets](https://scigantic.com/topics/json)
- [Genomics & Bioinformatics datasets](https://scigantic.com/topics/genomics-and-bioinformatics)
