# WikiDomains: 22,561 Wikipedia Term Definitions across 13 Academic Domains with Annotated Difficult Concepts for Targeted Concept Simplification (Google DeepMind)

## Facts

- Provider: Google DeepMind (Asthana, Rashkin, Clark, Huot, Lapata)
- License: CC-BY-SA-4.0 (https://github.com/google-deepmind/wikidomains)
- Format: TSV (in sampled files)
- Stored at: Google Cloud Storage, gs://wikidomains (us)
- Access: public, readable without an account
- Category: Natural Language Processing
- Source: https://github.com/google-deepmind/wikidomains
- Page: https://scigantic.com/archive/d7b6699a-2658-5d40-a428-1403df8e8974
- Keywords: Natural Language Processing, TSV, Google Cloud Storage

## About

WikiDomains is an English dataset for targeted concept simplification: helping readers understand a hard concept inside a domain-specific definition without simplifying away the rest. Each row is an English Wikipedia article in one of thirteen domains (Food and Drink, Performing Arts, Business and Economics, Politics and Government, Biology, Chemistry, Computing, Earth and Environment, Mathematics, Medicine and Health, Physics, Engineering, Technology) with the term (article title), its definition (the first sentence of the lead), the difficult concept inside that definition, the domain, the Wikipedia page id and Wikidata id, the article's PageRank score, in-domain and out-of-domain term frequencies and the full lead section. The bucket holds three tab-separated files: wikidomains_train.tsv (15,873 rows), wikidomains_dev.tsv (3,384) and wikidomains_test.tsv (3,304), about 28 MB together. Used to evaluate large language models on explaining jargon, text simplification, definition generation and readability for lay readers. Read with pandas.read_csv(sep='\t'). License: CC BY-SA 4.0, covering both the Wikipedia-derived text and the other materials, stated in the repository. Cite Asthana et al., Evaluating LLMs for Targeted Concept Simplification for Domain-Specific Texts (EMNLP 2024, arXiv:2410.20763).

## Load the data

```python
import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('wikidomains/', detail=False)[:10]
```

Lists the files with anonymous access. Runs on any machine, with no cloud account.

## Data structure

File types in sampled files: TSV

Example file: `wikidomains_dev.tsv`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/d7b6699a-2658-5d40-a428-1403df8e8974
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/d7b6699a-2658-5d40-a428-1403df8e8974
- Over MCP: `get_schema_card` with id `d7b6699a-2658-5d40-a428-1403df8e8974` at https://api.scigantic.com/mcp

## Related topics

- [TSV datasets](https://scigantic.com/topics/tsv)
- [Natural Language Processing datasets](https://scigantic.com/topics/natural-language-processing)
