Scholarly Article Citations in Wikipedia: DOI, PubMed and PMC Identifiers Cited in English Wikipedia (Wikimedia Research)

Provider
Wikimedia Foundation Research (Halfaker, Mansurov, Redi, Taraborelli)
License
CC0-1.0 (source)
Size
6 files
Format
Parquet, TSV (in sampled files)
Stored at
Hugging Face (https://huggingface.co/datasets/wikimedia-community/scholarly-article-citations-in-wikipedia)
Access
Free and public. No account with the data provider is needed.
DOI
10.6084/m9.figshare.1299540
Category
Open Data
Source
https://github.com/halfak/Extract-scholarly-article-citations-from-Wikipedia

About this dataset

Every citation to a scholarly article, identified by DOI, PubMed ID (pmid) or PubMed Central ID (pmcid), found in English Wikipedia as of January 2015. That is about 1.84 million (Wikipedia article, scholarly article) pairs. Each row gives the Wikipedia page id and title, the revision id and timestamp at which the citation was first added, the identifier type, and the identifier. Spot checks by the authors found that about 98% of the identifiers resolve. Altmetrics and science-communication researchers use it to measure how research reaches the public encyclopedia and how quickly new papers get cited there. The repository holds the raw TSV (doi-and-pubmed-citations-enwiki-20150112.tsv) and a Parquet copy. A larger, multilingual version is on figshare. The data was released by Aaron Halfaker, Bahodir Mansurov, Miriam Redi and Dario Taraborelli (Wikimedia Foundation) under CC0. Cite Halfaker et al. 2015, Citations with identifiers in Wikipedia, DOI 10.6084/m9.figshare.1299540.

How to load this dataset in Python

from datasets import load_dataset
ds = load_dataset("wikimedia-community/scholarly-article-citations-in-wikipedia")
print(ds)

Data structure

File types in sampled files: Parquet, TSV

Files

Example file: data/train-00000-of-00001.parquet

Analyze this dataset without downloading it

On Scigantic, Scholarly Article Citations in Wikipedia: DOI, PubMed and PMC Identifiers Cited in English Wikipedia (Wikimedia Research) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Open Data datasets