ARIES: Corpus of Scientific Paper Edits Made in Response to Peer Review Comments (OpenReview, Allen Institute for AI)
- Provider
- Allen Institute for AI (Mike D'Arcy et al.)
- License
- ODC-By-1.0 (source)
- Format
- JSON Lines (in sampled files)
- Stored at
- Amazon S3, s3://ai2-s2-research-public/aries (us-west-2)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Natural Language Processing
- Source
- https://arxiv.org/abs/2306.12587
About this dataset
ARIES pairs peer review comments with the edits authors made to their papers in response, built from OpenReview submissions with an original and a revised PDF. Files: paper_edits.jsonl (paragraph-level edits between source and target versions, keyed by OpenReview PDF ids), review_comments.jsonl (review comments keyed by doc_id and comment_id), edit_labels_train/dev/test.jsonl (comment-to-edit alignment labels; train and dev are synthetic labels derived from author responses, test is manually annotated), alignment_human_eval.jsonl, review_replies.jsonl (reviews and author responses used to build the synthetic data), raw_split_ids.json, generated_edits.jsonl (GPT-generated edits for test comments, with annotations), gpt3_cache.sqlite, and s2orc.tar.gz with S2ORC full-text parses of the papers. Tasks: aligning review feedback to revisions and generating edits that address a reviewer comment, for research on scientific writing assistance and peer review. Load with any JSON-lines reader; code is in allenai/aries. License: ODC-BY 1.0. Cite D'Arcy, Ross, Bransom, Kuehl, Bragg, Hope and Downey, ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews, arXiv:2306.12587.
How to access this dataset in Python
import boto3
from botocore import UNSIGNED
from botocore.client import Config
s3 = boto3.client('s3', region_name='us-west-2', config=Config(signature_version=UNSIGNED))
s3.list_objects_v2(Bucket='ai2-s2-research-public', Prefix='aries/', Delimiter='/')Lists the files with anonymous access. Runs on any machine, with no cloud account.
Data structure
File types in sampled files: JSON Lines
Files
Example file: aries/LICENSE
Analyze this dataset without downloading it
On Scigantic, ARIES: Corpus of Scientific Paper Edits Made in Response to Peer Review Comments (OpenReview, Allen Institute for AI) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Natural Language Processing datasets
- Flores
- AdvancedIF
- Civil Comments: 2 Million Public News-Site Comments with Crowd-Rated Toxicity and Identity-Mention Labels (Jigsaw, Google)
- Community Alignment Dataset
- Nemotron RL Instruction Following MultiTurnChat V 1
- WikiDomains: 22,561 Wikipedia Term Definitions across 13 Academic Domains with Annotated Difficult Concepts for Targeted Concept Simplification (Google DeepMind)
- Common Crawl
- Japanese Tokenizer Dictionaries