CommonCatalog CC-BY: Flickr Creative Commons images with synthetic captions
- Provider
- MosaicML / Databricks (Aaron Gokaslan et al.)
- License
- CC-BY-4.0 (source)
- Size
- 5,575 files
- Format
- Parquet (in sampled files)
- Stored at
- Hugging Face (https://huggingface.co/datasets/common-canvas/commoncatalog-cc-by)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Computer Vision
- Source
- https://arxiv.org/abs/2310.16825
About CommonCatalog CC-BY: Flickr Creative Commons images with synthetic captions
CommonCatalog is the image-caption dataset assembled by MosaicML to train CommonCanvas, a family of latent diffusion models built to match Stable Diffusion 2 using only openly licensed images of known provenance (Gokaslan et al., arXiv:2310.16825). The full catalogue covers roughly 100 million high-resolution photographs from Yahoo Flickr Creative Commons 100M (YFCC100M), collected from Flickr users around 2014, each given a synthetic caption generated with BLIP-2. The collection is split by licence into separate repositories; this one holds the images whose Flickr licence is CC BY. Each Parquet row carries the JPEG bytes, the BLIP-2 caption, the original Flickr title, user tags and machine tags, capture device, date taken, latitude and longitude where present, image size, the Flickr page and download URLs, and the licensename and licenseurl of the individual photo, so attribution can be generated per image. Images go up to 4K resolution. Measured 2026-11-14: ten top-level subsets (0 to 9), each partitioned by least_dim_range (256-512 up to 4096+) and aspect_ratio_bucket folders of Parquet files; the Hub reports about 21.5 TB of storage. Read with pyarrow, pandas or the datasets library in streaming mode. Uses: text-to-image and image-to-text training, captioning, data attribution research. Each image is CC BY under its Flickr licence; credit the photographer.
How to load this dataset in Python
from datasets import load_dataset
ds = load_dataset("common-canvas/commoncatalog-cc-by")
print(ds)Data structure
File types in sampled files: Parquet
Files
Example file: 0/least_dim_range=1024-2048/aspect_ratio_bucket=1-1/part-00000-tid-6217004709699050820-27b7ba6e-e051-4949-a7d0-9573b234b3fd-52879-1-c000.parquet
Analyze this dataset without downloading it
On Scigantic, CommonCatalog CC-BY: Flickr Creative Commons images with synthetic captions opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.