# CommonCatalog CC-BY: Flickr Creative Commons images with synthetic captions

## Facts

- Provider: MosaicML / Databricks (Aaron Gokaslan et al.)
- License: CC-BY-4.0 (https://huggingface.co/datasets/common-canvas/commoncatalog-cc-by)
- Size: 5,575 files
- Format: Parquet (in sampled files)
- Stored at: Hugging Face, https://huggingface.co/datasets/common-canvas/commoncatalog-cc-by
- Access: public, readable without an account
- Category: Computer Vision
- Source: https://arxiv.org/abs/2310.16825
- Page: https://scigantic.com/archive/b5472514-e3a9-50e4-ace2-b18ebc804a13
- Keywords: Computer Vision, Parquet, Hugging Face

## About

CommonCatalog is the image-caption dataset assembled by MosaicML to train CommonCanvas, a family of latent diffusion models built to match Stable Diffusion 2 using only openly licensed images of known provenance (Gokaslan et al., arXiv:2310.16825). The full catalogue covers roughly 100 million high-resolution photographs from Yahoo Flickr Creative Commons 100M (YFCC100M), collected from Flickr users around 2014, each given a synthetic caption generated with BLIP-2. The collection is split by licence into separate repositories; this one holds the images whose Flickr licence is CC BY. Each Parquet row carries the JPEG bytes, the BLIP-2 caption, the original Flickr title, user tags and machine tags, capture device, date taken, latitude and longitude where present, image size, the Flickr page and download URLs, and the licensename and licenseurl of the individual photo, so attribution can be generated per image. Images go up to 4K resolution. Measured 2026-11-14: ten top-level subsets (0 to 9), each partitioned by least_dim_range (256-512 up to 4096+) and aspect_ratio_bucket folders of Parquet files; the Hub reports about 21.5 TB of storage. Read with pyarrow, pandas or the datasets library in streaming mode. Uses: text-to-image and image-to-text training, captioning, data attribution research. Each image is CC BY under its Flickr licence; credit the photographer.

## Load the data

```python
from datasets import load_dataset
ds = load_dataset("common-canvas/commoncatalog-cc-by")
print(ds)
```

## Data structure

File types in sampled files: Parquet

Example file: `0/least_dim_range=1024-2048/aspect_ratio_bucket=1-1/part-00000-tid-6217004709699050820-27b7ba6e-e051-4949-a7d0-9573b234b3fd-52879-1-c000.parquet`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/b5472514-e3a9-50e4-ace2-b18ebc804a13
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/b5472514-e3a9-50e4-ace2-b18ebc804a13
- Over MCP: `get_schema_card` with id `b5472514-e3a9-50e4-ace2-b18ebc804a13` at https://api.scigantic.com/mcp

## Related topics

- [Parquet datasets](https://scigantic.com/topics/parquet)
- [Computer Vision datasets](https://scigantic.com/topics/computer-vision)
