ARIES: Corpus of Scientific Paper Edits Made in Response to Peer Review Comments (OpenReview, Allen Institute for AI)

Provider
Allen Institute for AI (Mike D'Arcy et al.)
License
ODC-By-1.0 (source)
Format
JSON Lines (in sampled files)
Stored at
Amazon S3, s3://ai2-s2-research-public/aries (us-west-2)
Access
Free and public. No account with the data provider is needed.
Category
Natural Language Processing
Source
https://arxiv.org/abs/2306.12587

About this dataset

ARIES pairs peer review comments with the edits authors made to their papers in response, built from OpenReview submissions with an original and a revised PDF. Files: paper_edits.jsonl (paragraph-level edits between source and target versions, keyed by OpenReview PDF ids), review_comments.jsonl (review comments keyed by doc_id and comment_id), edit_labels_train/dev/test.jsonl (comment-to-edit alignment labels; train and dev are synthetic labels derived from author responses, test is manually annotated), alignment_human_eval.jsonl, review_replies.jsonl (reviews and author responses used to build the synthetic data), raw_split_ids.json, generated_edits.jsonl (GPT-generated edits for test comments, with annotations), gpt3_cache.sqlite, and s2orc.tar.gz with S2ORC full-text parses of the papers. Tasks: aligning review feedback to revisions and generating edits that address a reviewer comment, for research on scientific writing assistance and peer review. Load with any JSON-lines reader; code is in allenai/aries. License: ODC-BY 1.0. Cite D'Arcy, Ross, Bransom, Kuehl, Bragg, Hope and Downey, ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews, arXiv:2306.12587.

How to access this dataset in Python

import boto3
from botocore import UNSIGNED
from botocore.client import Config
s3 = boto3.client('s3', region_name='us-west-2', config=Config(signature_version=UNSIGNED))
s3.list_objects_v2(Bucket='ai2-s2-research-public', Prefix='aries/', Delimiter='/')

Lists the files with anonymous access. Runs on any machine, with no cloud account.

Data structure

File types in sampled files: JSON Lines

Files

Example file: aries/LICENSE

Analyze this dataset without downloading it

On Scigantic, ARIES: Corpus of Scientific Paper Edits Made in Response to Peer Review Comments (OpenReview, Allen Institute for AI) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Natural Language Processing datasets