WIT Base: Wikimedia release of Wikipedia-based Image Text images
- Provider
- Wikimedia Research (images); Google Research (WIT dataset)
- License
- CC-BY-SA-4.0 (source)
- Size
- 335 files
- Format
- Parquet (in sampled files)
- Stored at
- Hugging Face (https://huggingface.co/datasets/wikimedia/wit_base)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Computer Vision
- Source
- https://arxiv.org/abs/2103.01913
About WIT Base: Wikimedia release of Wikipedia-based Image Text images
WIT Base is the Wikimedia Research team's release of the image files behind the Wikipedia-based Image Text (WIT) dataset, the multilingual image-text corpus of more than 37 million image-text associations from Wikipedia articles in 108 languages published by Google Research (SIGIR 2021). Google's WIT gives only image names and URLs; Wikimedia released the pixels so that models can be trained directly. This copy contains more than six million images from Wikipedia articles in 100+ languages, resized to 300 px, covering nearly all captioned images in WIT that have a non-null reference description. Images where a person's face covers more than 10% of the frame were excluded for privacy (detected with RetinaFace), as were images that were candidates for deletion on Wikimedia Commons. Each row pairs an image with its reference and attribution descriptions, page and section text and URLs in its language, and was released for the Wikipedia image caption matching Kaggle challenge. Uses: multilingual image captioning, image-text retrieval, contrastive vision-language pretraining. Measured 2026-11-14: 330 Parquet shards in data/, about 308 GB; open with the datasets library (streaming recommended), pandas or pyarrow. Wikimedia releases it under CC BY-SA 4.0; credit Wikimedia and the WIT paper (Srinivasan et al. 2021).
How to load WIT Base: Wikimedia release of Wikipedia-based Image Text images in Python
from datasets import load_dataset
ds = load_dataset("wikimedia/wit_base")
print(ds)Data structure
File types in sampled files: Parquet
Files
Example file: data/train-00000-of-00330.parquet
Analyze this dataset without downloading it
On Scigantic, WIT Base: Wikimedia release of Wikipedia-based Image Text images opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Computer Vision datasets
- PE Video
- Stillwater Lift Bridge Drone Photogrammetry: 2,223 Geotagged Images, Camera Poses and OpenDroneMap Solve of a Steel Truss Bridge (Minnesota)
- AVIRIS-1: San Diego airport hyperspectral target detection scene
- TartanAir: a dataset to push the limits of visual SLAM
- Vipe Web 360
- SKZ 3D Printing Defect Detection, Nozzle Camera: YOLO-Labeled FFF Print Errors
- Nemotron Personas Korea
- The Metropolitan Museum of Art Open Access: collection metadata and images