ChessBench: 10 Million Chess Games with Stockfish 16 Action-Value, State-Value and Best-Move Annotations (Google DeepMind)

Provider
Google DeepMind (Anian Ruoss, Gregoire Deletang et al.)
License
CC-BY-4.0 (source)
Stored at
Google Cloud Storage, gs://searchless_chess/data (us)
Access
Free and public. No account with the data provider is needed.
Category
Machine Learning & AI
Source
https://arxiv.org/abs/2402.04494

About this dataset

ChessBench is a benchmark of 10 million chess games with legal moves and value annotations from the Stockfish 16 engine, about 15 billion data points, released with the NeurIPS 2024 paper on amortized planning with large-scale transformers (searchless chess). Board states come from Lichess games. The data/ folder has train/ and test/ splits in .bag files (a simple record format read by the repository's bagz reader): action_value (win probability for every legal move of a position; the training set is sharded into 2,148 files, about 1.1 TB), behavioral_cloning (the best move per position, 34 GB train) and state_value (win probability per position, 36 GB train), plus puzzles.csv (Lichess puzzles with Elo ratings used for evaluation) and eco_openings.pgn. Positions are FEN strings and moves are UCI. Used to train transformer chess policies without search (a 270M-parameter model reached Lichess blitz Elo 2895), for planning and distillation research. License: CC BY 4.0, with the Lichess-derived portions CC0. Cite Ruoss, Deletang, Medapati et al., Amortized Planning with Large-Scale Transformers: A Case Study on Chess, NeurIPS 2024 (arXiv:2402.04494).

How to access this dataset in Python

import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('searchless_chess/data/', detail=False)[:10]

Lists the files with anonymous access. Runs on any machine, with no cloud account.

Data structure

Files

Example file: data/test/action_value_data.bag

Analyze this dataset without downloading it

On Scigantic, ChessBench: 10 Million Chess Games with Stockfish 16 Action-Value, State-Value and Best-Move Annotations (Google DeepMind) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Machine Learning & AI datasets