PromoterZoo: 13.6 Million Multi-Species Promoter Sequences for Training the LOL-EVE Variant Effect Model (Harvard Marks Lab)
- Provider
- Marks Lab, Harvard Medical School
- License
- MIT (source)
- Size
- 13 files
- Format
- Parquet, Arrow (in sampled files)
- Stored at
- Hugging Face (https://huggingface.co/datasets/Marks-lab/PromoterZoo)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Genomics & Bioinformatics
- Source
- https://huggingface.co/datasets/Marks-lab/PromoterZoo
About this dataset
Training data for LOL-EVE (Language of Life, Evolutionary Variant Effects), a genomic language model from the Marks Lab at Harvard Medical School that scores the effect of indels and other variants in gene promoters. The dataset holds 13,611,373 promoter DNA sequences of 1,000 base pairs taken around transcription start sites (TSS) across many species grouped into evolutionary clades, giving the model evolutionary context for each gene's regulatory region. Each record has the sequence, gene symbol, species, clade, chromosome, strand, genomic start and stop, sequence length and direction (towards or away from the TSS), and both strands are included. Stored as one 6.4 GB data.parquet plus Arrow shards under dataset/, it loads with the datasets library or pandas and can be filtered by gene or species. Suited to pretraining DNA language models, comparative promoter analysis and cis-regulatory evolution studies. Released by the Marks Lab under the MIT License.
How to load this dataset in Python
from datasets import load_dataset
ds = load_dataset("Marks-lab/PromoterZoo")
print(ds)Data structure
File types in sampled files: Parquet, Arrow
Files
Example file: data.parquet
Analyze this dataset without downloading it
On Scigantic, PromoterZoo: 13.6 Million Multi-Species Promoter Sequences for Training the LOL-EVE Variant Effect Model (Harvard Marks Lab) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Genomics & Bioinformatics datasets
- COVID-19 Genome Sequence Dataset
- Rye Reference NLRome
- TCGA-TGCT Testicular Germ Cell Tumors (GDC open-access tables, 263 cases)
- Reactome (curated human pathway database)
- Aging Gene Expression Single Cell Mouse
- Global Biodiversity Information Facility (GBIF) Species Occurrences
- TenK10K multiome: single-cell chromatin accessibility QTLs across 28 immune cell types in 922 donors
- LongBench - cross-platform reference dataset profiling cancer cell lines with bulk and single-cell approaches