Genomic-NIAH: needle-in-a-haystack long-context retrieval benchmark for DNA language models

Provider
Hugging Face (HuggingFaceBio, Carbon team)
License
Apache-2.0 (source)
Size
28 files
Format
PNG, Parquet (in sampled files)
Stored at
Hugging Face (https://huggingface.co/datasets/HuggingFaceBio/genomic-niah)
Access
Free and public. No account with the data provider is needed.
Category
Genomics & Bioinformatics
Source
https://huggingface.co/datasets/HuggingFaceBio/genomic-niah

About this dataset

A long-context retrieval benchmark for genomic language models from Hugging Face, built for the Carbon DNA models. Each example plants a random 24 bp KEY and 24 bp VALUE pair inside a real eukaryotic genome haystack and asks the model to return VALUE given the haystack followed by KEY, so success requires actual long-range retrieval. Four tasks (plain retrieval and three near-duplicate variants with eight distractor keys differing by 4, 2 or 1 bp, inspired by RULER multi-key) cross six context lengths from 4k to 128k 6-mer tokens (about 24 kbp to 786 kbp), giving 24 configurations of 500 examples each (12,000 in total), stratified over five needle depths (10 to 90%) and four kingdoms (animalia, fungi, plantae, protista). Haystacks come from the OpenGenome2 NCBI eukaryotic test and validation splits, using contiguous ACGT runs up to 16k tokens and stitched within-record runs beyond. Data are Parquet under data/, one config per task and length (e.g. niah_16k, niah_neardup_d1_64k), loadable with Hugging Face datasets; about 4 GB. Released under Apache 2.0; cite the Carbon paper (Ben Allal et al., 2026).

How to load this dataset in Python

from datasets import load_dataset
ds = load_dataset("HuggingFaceBio/genomic-niah")
print(ds)

Data structure

File types in sampled files: PNG, Parquet

Files

Example file: data/niah_128k/test-00000-of-00001.parquet

Analyze this dataset without downloading it

On Scigantic, Genomic-NIAH: needle-in-a-haystack long-context retrieval benchmark for DNA language models opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Genomics & Bioinformatics datasets