AstroPT Galaxies: 8.5 Million DESI Legacy Survey DR8 Galaxy Cutouts

Provider
Michael J. Smith et al. (AstroPT)
License
CC-BY-SA-4.0 (source)
Size
442 files
Format
Parquet (in sampled files)
Stored at
Hugging Face (https://huggingface.co/datasets/Smith42/galaxies)
Access
Free and public. No account with the data provider is needed.
Category
Astronomy & Space
Source
https://arxiv.org/abs/2405.14930

About AstroPT Galaxies: 8.5 Million DESI Legacy Survey DR8 Galaxy Cutouts

About 8.5 million galaxy cutouts from the DESI Legacy Imaging Surveys DR8, each a 512 by 512 pixel JPEG centred on a galaxy from the Galaxy Zoo DESI catalogue (Walmsley et al. 2023), assembled to train AstroPT, a family of large observation models for astronomy. 98 percent of images form the training set, with 1 percent each for validation and test. Galaxy properties are provided as metadata linked by dr8_id, and pre-computed AstroPT v2 embeddings are available in a companion repository; revision v2.0 joins images and metadata directly. Listing the repository (measured 2026-11-13) shows data/ with 196 Parquet shards (982.5 GB, train and test), with_crops/ with 243 Parquet shards (1,206 GB) and a root metadata.parquet (2.16 GB). Open with Hugging Face datasets (streaming=True is advisable), pyarrow or pandas, decoding the image bytes with PIL. Distributed under CC-BY-SA-4.0 by the uploader over CC-BY-4.0 Legacy Surveys imagery and Galaxy Zoo DESI catalogue; use the Legacy Surveys acknowledgment and cite Dey et al. 2019, Walmsley et al. 2023 and Smith et al. 2024.

How to load AstroPT Galaxies: 8.5 Million DESI Legacy Survey DR8 Galaxy Cutouts in Python

from datasets import load_dataset
ds = load_dataset("Smith42/galaxies")
print(ds)

Data structure

File types in sampled files: Parquet

Files

Example file: data/test-00000-of-00002.parquet

Analyze this dataset without downloading it

On Scigantic, AstroPT Galaxies: 8.5 Million DESI Legacy Survey DR8 Galaxy Cutouts opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Astronomy & Space datasets