# StreamingQA: Time-Stamped Questions over WMT English News for Testing How QA Models Adapt to New Knowledge (Google DeepMind)

## Facts

- Provider: Google DeepMind (Liska, Kocisky, Gribovskaya, Terzi et al.)
- License: CC-BY-4.0 (https://github.com/google-deepmind/streamingqa)
- Format: JSON Lines (in sampled files)
- Stored at: Google Cloud Storage, gs://dm-streamingqa (us)
- Access: public, readable without an account
- Category: Natural Language Processing
- Source: https://github.com/google-deepmind/streamingqa
- Page: https://scigantic.com/archive/16a6de8c-37e5-5098-afe6-1bca972f4043
- Keywords: Natural Language Processing, benchmark, JSON Lines, Google Cloud Storage

## About

StreamingQA is a question answering benchmark for adaptation to new knowledge over time. Questions are tied to articles in the English WMT News Crawl corpus, and each carries the date it was asked (question_ts) and the publication date of its evidence article (evidence_ts), so a model can be evaluated on questions about events that happened after its training data. The bucket holds streaminqa_train.jsonl.gz (99,402 questions), streaminqa_valid.jsonl.gz (9,939) and streaminqa_eval.jsonl.gz (36,378), plus wmt_sorting_key_ids.txt.gz, a list of 11,393,471 SHA256-based document keys that reproduce the deduplicated WMT subset used in the paper with the repository's extraction script. Each record has the question, answers (one for train and valid, three references plus an additional fourth for eval), the evidence document key, a recent or past label, whether the question was written by annotators or generated, and Perspective API toxicity scores. The news articles themselves are not redistributed; they come from WMT News Crawl. Used to study temporal knowledge drift, continual learning, retrieval-augmented generation and knowledge updating in language models. License: CC BY 4.0 for the data (software Apache 2.0), stated in the repository. Cite Liska et al., StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models (ICML 2022).

## Load the data

```python
import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('dm-streamingqa/', detail=False)[:10]
```

Lists the files with anonymous access. Runs on any machine, with no cloud account.

## Data structure

File types in sampled files: JSON Lines

Example file: `streaminqa_eval.jsonl.gz`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/16a6de8c-37e5-5098-afe6-1bca972f4043
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/16a6de8c-37e5-5098-afe6-1bca972f4043
- Over MCP: `get_schema_card` with id `16a6de8c-37e5-5098-afe6-1bca972f4043` at https://api.scigantic.com/mcp

## Related topics

- [Benchmark datasets](https://scigantic.com/topics/benchmark)
- [JSON Lines datasets](https://scigantic.com/topics/json-lines)
- [Natural Language Processing datasets](https://scigantic.com/topics/natural-language-processing)
