CommonCatalog CC-BY-SA: Flickr ShareAlike images with synthetic captions

Provider
MosaicML / Databricks (Aaron Gokaslan et al.)
License
CC-BY-SA-4.0 (source)
Size
4,131 files
Format
Parquet (in sampled files)
Stored at
Hugging Face (https://huggingface.co/datasets/common-canvas/commoncatalog-cc-by-sa)
Access
Free and public. No account with the data provider is needed.
Category
Computer Vision
Source
https://arxiv.org/abs/2310.16825

About CommonCatalog CC-BY-SA: Flickr ShareAlike images with synthetic captions

This is the CC BY-SA partition of CommonCatalog, the openly licensed image-caption collection MosaicML built to train the CommonCanvas latent diffusion models (Gokaslan et al., arXiv:2310.16825). Like the CC BY partition, it draws on Yahoo Flickr Creative Commons 100M (YFCC100M) photographs uploaded by Flickr users around 2014, keeps only images under a ShareAlike Creative Commons licence that allows commercial use, and pairs each with a synthetic BLIP-2 caption. Parquet rows contain the JPEG bytes, the BLIP-2 caption, the Flickr title, user and machine tags, capture device, date taken, geolocation where present, original and stored image dimensions, Flickr page and download URLs, and the per-photo licensename and licenseurl, so derived works can carry correct attribution and share-alike notices. Images go up to 4K resolution. Measured 2026-11-14: ten top-level subsets (0 to 9), each partitioned by least_dim_range and aspect_ratio_bucket folders of Parquet files; the Hub reports about 13.6 TB of storage. Read with pyarrow, pandas or the datasets library in streaming mode. Uses: text-to-image diffusion training, image captioning, provenance and attribution studies. Every image is under a CC BY-SA licence from its Flickr author; attribute the photographer and share adaptations alike.

How to load this dataset in Python

from datasets import load_dataset
ds = load_dataset("common-canvas/commoncatalog-cc-by-sa")
print(ds)

Data structure

File types in sampled files: Parquet

Files

Example file: 0/least_dim_range=1024-2048/aspect_ratio_bucket=1-1/part-00000-tid-7761264679195244086-4c6ec9dc-fb3d-4bd3-98ec-f987466013dd-3488916-1-c000.parquet

Analyze this dataset without downloading it

On Scigantic, CommonCatalog CC-BY-SA: Flickr ShareAlike images with synthetic captions opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Computer Vision datasets