# Natural Questions (NQ): 307,373 Real Google Search Questions with Long and Short Answers Annotated on Full Wikipedia Pages (Google Research)

## Facts

- Provider: Google Research (Kwiatkowski et al.)
- License: CC-BY-SA-3.0 (https://huggingface.co/datasets/google-research-datasets/natural_questions)
- Size: 303 files
- Format: Parquet (in sampled files)
- Stored at: Hugging Face, https://huggingface.co/datasets/google-research-datasets/natural_questions
- Access: public, readable without an account
- Category: Natural Language Processing
- Source: https://doi.org/10.1162/tacl_a_00276
- Page: https://scigantic.com/archive/126007a3-cf44-5b7a-bf0a-63d86de870f0
- Keywords: Natural Language Processing, benchmark, Parquet, Hugging Face

## About

Natural Questions (NQ) is an open-domain question answering benchmark built from real, anonymized queries that users issued to Google Search. For each question an annotator was shown an entire English Wikipedia article and marked a long answer (usually a paragraph or table) and, where one exists, a short answer span or a yes/no answer; some questions have no answer on the page. Because systems must read the whole page rather than a selected passage, NQ is a standard test for reading comprehension, passage retrieval and long-document machine reading. This Hugging Face release has 307,373 training examples and 7,830 development examples, each with the full article HTML, tokens, the question, and the long and short answer annotations, stored as about 300 Parquet shards (about 58 GB) under default/ and dev/. Load it with the Hugging Face datasets library (load_dataset('google-research-datasets/natural_questions')) or read the Parquet files with pandas or pyarrow. Released by Google Research under Creative Commons Attribution-ShareAlike 3.0 (CC BY-SA 3.0); cite Kwiatkowski et al., Natural Questions: a Benchmark for Question Answering Research, TACL 2019.

## Load the data

```python
from datasets import load_dataset
ds = load_dataset("google-research-datasets/natural_questions")
print(ds)
```

## Data structure

File types in sampled files: Parquet

Example file: `default/train-00000-of-00287.parquet`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/126007a3-cf44-5b7a-bf0a-63d86de870f0
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/126007a3-cf44-5b7a-bf0a-63d86de870f0
- Over MCP: `get_schema_card` with id `126007a3-cf44-5b7a-bf0a-63d86de870f0` at https://api.scigantic.com/mcp

## Related topics

- [Parquet datasets](https://scigantic.com/topics/parquet)
- [Benchmark datasets](https://scigantic.com/topics/benchmark)
- [Natural Language Processing datasets](https://scigantic.com/topics/natural-language-processing)
