IndicGenBench XorQA-IN: Cross-lingual Open-Retrieval Question Answering in 29 Indic Languages (Google Research)
- Provider
- Google Research (Singh et al.)
- License
- MIT (source)
- Size
- 89 files
- Format
- JSON (in sampled files)
- Stored at
- Hugging Face (https://huggingface.co/datasets/google/IndicGenBench_xorqa_in)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Natural Language Processing
- Source
- https://arxiv.org/abs/2404.16816
About this dataset
XorQA-IN is IndicGenBench's cross-lingual question answering task, built on XOR QA (cross-lingual open-retrieval question answering): a question is posed in an Indic language, the supporting evidence passage is in English, and the model must produce the answer, testing whether a model can bridge languages between the question and the evidence. IndicGenBench is a multilingual, multi-way parallel benchmark from Google Research for measuring language generation by large language models in 29 Indic languages spanning 13 writing scripts and 4 language families, including many low-resource languages such as Assamese, Bhojpuri, Awadhi, Maithili, Santali and Konkani alongside Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada and Malayalam. Google extended existing English benchmarks by collecting human translations of their English examples into each target language. The repository holds 87 JSON files, one train, dev and test file per language, named like xorqa_bho_test.json (about 45 MB in total). Load with the Hugging Face datasets library or plain JSON readers. Distributed under the MIT License, following the XOR QA source license; cite Singh et al., IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages, ACL 2024.
How to load this dataset in Python
from huggingface_hub import hf_hub_download
path = hf_hub_download(repo_id="google/IndicGenBench_xorqa_in", filename="xorqa_as_dev.json", repo_type="dataset")
print(path)Data structure
File types in sampled files: JSON
Files
Example file: xorqa_as_dev.json
Analyze this dataset without downloading it
On Scigantic, IndicGenBench XorQA-IN: Cross-lingual Open-Retrieval Question Answering in 29 Indic Languages (Google Research) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Natural Language Processing datasets
- ExploreToM
- Shopping Humor Generation
- Common Screens
- Phrase Clustering Dataset (PCD)
- Natural Questions (NQ): 307,373 Real Google Search Questions with Long and Short Answers Annotated on Full Wikipedia Pages (Google Research)
- Nemotron Instruction Following Chat V 1
- Low Context Name Entity Recognition (NER) Datasets with Gazetteer
- StreamingQA: Time-Stamped Questions over WMT English News for Testing How QA Models Adapt to New Knowledge (Google DeepMind)