Civil Comments: 2 Million Public News-Site Comments with Crowd-Rated Toxicity and Identity-Mention Labels (Jigsaw, Google)

Provider
Jigsaw and Google (Borkan et al.); Civil Comments platform
License
CC0-1.0 (source)
Size
6 files
Format
Parquet (in sampled files)
Stored at
Hugging Face (https://huggingface.co/datasets/google/civil_comments)
Access
Free and public. No account with the data provider is needed.
Category
Natural Language Processing
Source
https://arxiv.org/abs/1903.04561

About this dataset

Civil Comments holds about two million public comments posted between 2015 and 2017 on roughly 50 English-language news sites that used the Civil Comments commenting plugin. When the platform shut down in 2017 it released the comment text as an open archive for research, and Jigsaw (Google) added crowd-sourced labels: a toxicity score for each comment (the fraction of raters who judged it toxic) plus sub-attributes such as severe toxicity, obscene, threat, insult, identity attack and sexually explicit. This is the data released for the Jigsaw Unintended Bias in Toxicity Classification challenge, widely used to study hate speech and abusive language detection, online harassment, content moderation and bias against identity groups in text classifiers. Splits: 1,804,874 training, 97,320 validation and 97,320 test comments, stored as four Parquet files (about 420 MB) under data/. Load with the Hugging Face datasets library (load_dataset('google/civil_comments')) or pandas. Released under CC0 1.0, as is the underlying comment text; cite Borkan et al., Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification, WWW 2019 Companion.

How to load this dataset in Python

from datasets import load_dataset
ds = load_dataset("google/civil_comments")
print(ds)

Data structure

File types in sampled files: Parquet

Files

Example file: data/test-00000-of-00001.parquet

Analyze this dataset without downloading it

On Scigantic, Civil Comments: 2 Million Public News-Site Comments with Crowd-Rated Toxicity and Identity-Mention Labels (Jigsaw, Google) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Natural Language Processing datasets