AstroPT Galaxies: 8.5 Million DESI Legacy Survey DR8 Galaxy Cutouts
- Provider
- Michael J. Smith et al. (AstroPT)
- License
- CC-BY-SA-4.0 (source)
- Size
- 442 files
- Format
- Parquet (in sampled files)
- Stored at
- Hugging Face (https://huggingface.co/datasets/Smith42/galaxies)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Astronomy & Space
- Source
- https://arxiv.org/abs/2405.14930
About AstroPT Galaxies: 8.5 Million DESI Legacy Survey DR8 Galaxy Cutouts
About 8.5 million galaxy cutouts from the DESI Legacy Imaging Surveys DR8, each a 512 by 512 pixel JPEG centred on a galaxy from the Galaxy Zoo DESI catalogue (Walmsley et al. 2023), assembled to train AstroPT, a family of large observation models for astronomy. 98 percent of images form the training set, with 1 percent each for validation and test. Galaxy properties are provided as metadata linked by dr8_id, and pre-computed AstroPT v2 embeddings are available in a companion repository; revision v2.0 joins images and metadata directly. Listing the repository (measured 2026-11-13) shows data/ with 196 Parquet shards (982.5 GB, train and test), with_crops/ with 243 Parquet shards (1,206 GB) and a root metadata.parquet (2.16 GB). Open with Hugging Face datasets (streaming=True is advisable), pyarrow or pandas, decoding the image bytes with PIL. Distributed under CC-BY-SA-4.0 by the uploader over CC-BY-4.0 Legacy Surveys imagery and Galaxy Zoo DESI catalogue; use the Legacy Surveys acknowledgment and cite Dey et al. 2019, Walmsley et al. 2023 and Smith et al. 2024.
How to load AstroPT Galaxies: 8.5 Million DESI Legacy Survey DR8 Galaxy Cutouts in Python
from datasets import load_dataset
ds = load_dataset("Smith42/galaxies")
print(ds)Data structure
File types in sampled files: Parquet
Files
Example file: data/test-00000-of-00002.parquet
Analyze this dataset without downloading it
On Scigantic, AstroPT Galaxies: 8.5 Million DESI Legacy Survey DR8 Galaxy Cutouts opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Astronomy & Space datasets
- Sup3rCC
- Mars-Bench MER Pancam Multi-Label Surface Classification (Spirit and Opportunity, 25 labels)
- Mars-Bench Boulder Detection, MRO HiRISE (YOLO, COCO and Pascal VOC boxes)
- SuperDARN Super Dual Auroral Radar Network: HF Radar Ionospheric Convection Velocity Key Parameters, Seven Radars, 1992-2001 (CDAWeb on HelioCloud)
- Voyager 1 Magnetic Field, Plasma, Energetic Particle and Plasma Wave Data (CDAWeb on HelioCloud)
- MSL Curiosity RAD Radiation Assessment Detector: Mars Surface Dose Rate, LET Spectra and Neutral Particles, Level 1 and 2, 2011-present (CDAWeb on HelioCloud)
- IMAGE Mission Magnetospheric Imaging: EUV, FUV, RPI and Neutral Atom Imagers, 2000-2005 (CDAWeb on HelioCloud)
- ISEE-1 International Sun-Earth Explorer 1: Fluxgate Magnetometer, LANL FPE Plasma, Iowa Plasma Waves and Energetic Particles, 1977-1987 (CDAWeb on HelioCloud)