S2AND: Author Name Disambiguation Benchmark, Eight Bibliographic Datasets plus Semantic Scholar Production Models (Allen Institute for AI)
- Provider
- Allen Institute for AI, Semantic Scholar (Shivashankar Subramanian, Daniel King, Doug Downey, Sergey Feldman)
- License
- ODC-By-1.0 (source)
- Format
- JSON (in sampled files)
- Stored at
- Amazon S3, s3://ai2-s2-research-public/s2and-release (us-west-2)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Natural Language Processing
- Source
- https://api.semanticscholar.org/CorpusID:232233421
About this dataset
S2AND is a benchmark for author name disambiguation (AND): deciding which author mentions (signatures) on scholarly papers belong to the same real person. It unifies eight existing labeled AND datasets into one format, each in its own folder: aminer, arnetminer, inspire, kisti, medline, pubmed, qian and zbmath, each with signatures, papers and gold cluster files, organized into name blocks (for example all 'j smith' signatures). The release also holds full_union_seed_1-5 training pickles, name_counts.pickle (first and last name frequency counts), a fastText language-ID model (lid.176.bin), Semantic Scholar production clustering models (production_model_v1.0 to v1.2), and s2and_and_big_blocks_linker_dataset_20260513/. A newer Arrow layout is at s2and-release-arrow/ in the same bucket. Load with the s2and Python package (ANDData). Some affiliation data comes from the Microsoft Academic Graph. Used for bibliometrics, scholarly knowledge graphs and entity resolution research. License: dataset under ODC-BY. Cite Subramanian, King, Downey and Feldman, S2AND: A Benchmark and Evaluation System for Author Name Disambiguation, JCDL 2021.
How to access this dataset in Python
import boto3
from botocore import UNSIGNED
from botocore.client import Config
s3 = boto3.client('s3', region_name='us-west-2', config=Config(signature_version=UNSIGNED))
s3.list_objects_v2(Bucket='ai2-s2-research-public', Prefix='s2and-release/', Delimiter='/')Lists the files with anonymous access. Runs on any machine, with no cloud account.
Data structure
File types in sampled files: JSON
Files
Example file: s2and-release/.feature_cache/aminer/51/all_features.json
Analyze this dataset without downloading it
On Scigantic, S2AND: Author Name Disambiguation Benchmark, Eight Bibliographic Datasets plus Semantic Scholar Production Models (Allen Institute for AI) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Natural Language Processing datasets
- IndicGenBench XQuAD-IN: Extractive Question Answering Translated into Indic Languages (Google Research)
- BFCL Hi
- BLiMP: Benchmark of Linguistic Minimal Pairs for English Grammar (67 Phenomena, NYU)
- PASS: Perturb-and-Select Summarizer for Product Reviews
- Nemotron Safety Guard Dataset V 3
- Asset
- MultiCoNER Datasets
- Content Rephrasing