Machine Learning & AI datasets
147 public Machine Learning & AI datasets in the Scigantic catalog, from providers including NVIDIA, Amazon and Meta AI. 145 can be read without an account, and 89 carry an open license (19 are non-commercial or no-derivatives). Common formats: Parquet, JSON Lines and JSON. 8 have loading code that was run against a real file. Each opens in a hosted Jupyter notebook with the data mounted, so nothing has to be downloaded first.
| Dataset | Provider | License | Access |
|---|---|---|---|
| BodyM Dataset PNGloading code checked | Amazon | Creative Commons Attribution-Non Commercial 4.0… | public |
| Humor Detection from Product Question Answering Systems CSVloading code checked | Amazon | [CDLA-Sharing](https://cdla.io/sharing-1-0/) | public |
| NAIP data embedded with Clay v1.5 Parquetloading code checked | clay via Source Cooperative | CC-BY-4.0 | public |
| Trazo Field Boundary Annotations TIFFloading code checked | wri-data-lab via Source Cooperative | CC-BY-4.0 | public |
| Clay Model v0 Embeddings loading code checked | brunosan via Source Cooperative | not stated | public |
| Clay v1 Embeddings for California NAIP Parquetloading code checked | clay via Source Cooperative | not stated | public |
| KITTI Vision Benchmark Suite JPEGloading code checked | Max Planck Campus Tübingen | Creative Commons Attribution-NonCommercial-Share… non-commercial or no-derivatives | public |
| Sentinel 2 Embeddings Parquetloading code checked | earthgenome via Source Cooperative | not stated | public |
| AI2 Reasoning Challenge (ARC) 2018 | Allen Institute for AI | [CC BY-SA](https://creativecommons.org/licenses/… | public |
| AI2 Tablestore (November 2015 Snapshot) | Allen Institute for AI | [CC BY-SA](https://creativecommons.org/licenses/… | public |
| AI2 TabMCQ: Multiple Choice Questions aligned with the Aristo Tablestore | Allen Institute for AI | [CC BY-SA](https://creativecommons.org/licenses/… | public |
| Airborne Object Tracking Dataset CSVJSON | Amazon | Community Data License Agreement – Permissive… | public |
| Amazon Seller Contact Intent Sequence CSV | Amazon | https://cdla.dev/permissive-1-0/ | public |
| Amazon-PQA JSON | Amazon | https://cdla.dev/permissive-1-0/ | public |
| Answer Reformulation CSV | Amazon | [cc-by-sa 4.0](https://creativecommons.org/licen… | public |
| Anthropic Economic Index CSV | Anthropic Economic Research | CC-BY-4.0 | public |
| Aristo Tuple KB | Allen Institute for AI | [CC BY-SA](https://creativecommons.org/licenses/… | public |
| arXiv Metadata Snapshot (weekly mirror of the Cornell arXiv OAI dataset) Parquet | Cornell University arXiv; mirrored by Hugging Face librarian | CC0-1.0 | public |
| Automatic Speech Recognition (ASR) Error Robustness | Amazon | See https://github.com/anjiefang/asr-error-robus… | public |
| ChessBench: 10 Million Chess Games with Stockfish 16 Action-Value, State-Value and Best-Move Annotations (Google DeepMind) | Google DeepMind (Anian Ruoss, Gregoire Deletang et al.) | CC-BY-4.0 | public |
| CLIP Embeddings of 562,842 Library of Congress Historical Map Sheets (Sanborn maps, atlases, city views) JSON | Library of Congress Computing Cultural Heritage (Huang and L | CC-BY-4.0 | public |
| COCO - Common Objects in Context - fast.ai datasets | fast.ai | Creative Commons http://cocodataset.org/#termsof… | public |
| CoTracker 3 Kubric | Meta AI | apache-2.0 | public |
| CSU Synthetic Attribution Benchmark for Explainable AI in Geoscience JSON | Mamalakis, Ebert-Uphoff and Barnes (Colorado State Universit | CC-BY-4.0 | public |
| Daring Anteater JSON Lines | NVIDIA | cc-by-4.0 | public |
| Deutsche Bahn Train Delay and Cancellation Data Parquet | Deutsche Bahn (source data); collected by piebro | CC-BY-4.0 | public |
| DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue JSON | Amazon | [CDLA-Sharing](https://cdla.io/sharing-1-0/) | public |
| DigiData JSON Lines | Meta AI | cc-by-4.0 | public |
| DiTEC-WDN: Large-Scale Water Distribution Network Simulation Dataset Parquet | DiTEC project, University of Groningen data science group (r | CC-BY-4.0 | public |
| Enriched Topical-Chat Dataset for Knowledge-Grounded Dialogue Systems JSON | Amazon | [CDLA-Sharing](https://cdla.io/sharing-1-0/) | public |
| EOTA-OCR Parquet | govscape via Source Cooperative | CC0-1.0 | public |
| Fashion-MNIST Parquet | Zalando Research (Han Xiao, Kashif Rasul, Roland Vollgraf) | MIT | public |
| FreshRetailNet-50K: Fresh Retail Demand with Stockout Annotations Parquet | Dingdong Inc. | CC-BY-4.0 | public |
| Google Cloud AI/ML Sample Datasets | Google Cloud | Apache 2.0 | public |
| Habermas Machine Dataset: Human Deliberation Opinions, Group Statements and Preference Rankings from AI-Mediated Deliberation (Google DeepMind, Science 2024) Parquet | Google DeepMind (Michael Henry Tessler, Michiel A. Bakker, C | CC-BY-4.0 | public |
| HelpSteer PNGJSON Lines | NVIDIA | cc-by-4.0 | public |
| HelpSteer 2 JSON Lines | NVIDIA | cc-by-4.0 | public |
| HelpSteer 3 JSON Lines | NVIDIA | cc-by-4.0 | public |
| Judges Verdict JSON | NVIDIA | cc-by-sa-4.0 | public |
| Kimodo Motion Gen Benchmark JSON | NVIDIA | cc-by-4.0 | public |
| Learning to Rank and Filter - community question answering CSV | Amazon | [CDLA-Permissive](https://cdla.dev/permissive-1-… | public |
| LGND Embeddings PNG | clay via Source Cooperative | CC-BY-4.0 | public |
| Lichess Chess Position Evaluations (Stockfish) Parquet | Lichess (lichess.org) | CC0-1.0 | public |
| Lichess Chess Puzzles Database Parquet | Lichess (lichess.org) | CC0-1.0 | public |
| Lichess Rated Standard Chess Games Database Parquet | Lichess (lichess.org) | CC0-1.0 | public |
| Llamafirewall Alignmentcheck Evals JSON | Meta AI | mit | public |
| Los Alamos National Laboratory Cyber-Security Event Datasets (2015 and 2017) | Los Alamos National Laboratory (Alexander D. Kent and others | CC0-1.0 | public |
| Milling Tool Wear Time Series across Three Machine Tools (Flank Wear VB, Forces, Spindle Torque) Parquet | Stiehl et al. (original); Parquet conversion by alpha-by-bet | CC-BY-4.0 | public |
| Monash Time Series Forecasting Repository | Rakshitha Godahewa, Christoph Bergmeir, Geoffrey Webb, Rob H | CC-BY-4.0 | public |
| Multiview Extended Video with Activities (MEVA) | Kitware | http://mevadata.org/resources/MEVA-data-license.… | public |
| Nemotron Agentic V 1 JSON Lines | NVIDIA | cc-by-4.0 | public |
| Nemotron Cascade RL RLHF Parquet | NVIDIA | cc-by-4.0 | public |
| Nemotron Cascade RM Training JSON Lines | NVIDIA | cc-by-4.0 | public |
| Nemotron Personas Brazil Parquet | NVIDIA | cc-by-4.0 | public |
| Nemotron Personas Singapore ParquetPNG | NVIDIA | cc-by-4.0 | public |
| Nemotron Personas USA Parquet | NVIDIA | cc-by-4.0 | public |
| Nemotron RL Agent Calendar Scheduling JSON Lines | NVIDIA | cc-by-4.0 | public |
| Nemotron RL Agent Workplace Assistant JSON Lines | NVIDIA | cc-by-4.0 | public |
| Nemotron RL Agentic Conversational Tool Use Pivot V 1 JSON Lines | NVIDIA | cc-by-4.0 | public |
| Nemotron RL Agentic Function Calling Pivot V 1 JSON Lines | NVIDIA | cc-by-4.0 | public |
| Nemotron RL Bixbench Hypothesis TSVCSV | NVIDIA | cc-by-4.0 | public |
| Nemotron RL Identity Following V 1 JSON Lines | NVIDIA | cc-by-4.0 | public |
| Nemotron SFT Agentic V 2 JSON Lines | NVIDIA | cc-by-4.0 | public |
| Nemotron SFT OpenCode V 1 JSON Lines | NVIDIA | cc-by-4.0 | public |
| NeuralGCM Pre-trained Model Checkpoints (v1) | Google Research (NeuralGCM team) | CC-BY-SA-4.0 | public |
| OGB ogbg-code2: Code Summarization as Graph Prediction (Open Graph Benchmark) JSON Lines | Open Graph Benchmark team (Hu et al., Stanford) | MIT | public |
| OGB ogbg-ppa: Protein Association Graph Classification (Open Graph Benchmark) JSON Lines | Open Graph Benchmark team (Hu et al., Stanford) | CC0-1.0 | public |
| Open Food Facts Images JPEGJSON | Open Food Facts | All data contained in this dataset is licenced… | public |
| OpenScience JSON Lines | NVIDIA | cc-by-4.0 | public |
| OpenScienceReasoning 2 Parquet | NVIDIA | cc-by-4.0 | public |
| PD12M JPEG | Spawning | https://cdla.dev/permissive-2-0/ | public |
| Pre- and post-purchase product questions CSV | Amazon | [Apache-2.0](https://www.apache.org/licenses/LIC… | public |
| Product Comparison Dataset for Online Shopping TSV | Amazon | [CC-BY-SA 4.0](https://creativecommons.org/licen… | public |
| Retrieval Synthetic NVDocs V 1 ParquetJSON | NVIDIA | cc-by-4.0 | public |
| RxR (Room-Across-Room): Multilingual Vision-and-Language Navigation Instructions in English, Hindi and Telugu with Pose Traces for Matterport3D (Google Research) NumPy | Google Research (Ku, Anderson, Patel, Ie, Baldridge) | CC-BY-4.0 | public |
| Sft Datablend V 1 JSON Lines | NVIDIA | cc-by-4.0 | public |
| Sophos/ReversingLabs 20 Million malware detection dataset | Sophos AI | See the [Terms of Use](https://github.com/sophos… | public |
| Surya Bench NetCDF | NASA IMPACT | Creative Commons Attribution 4.0 International. | public |
| TechQA RAG Eval JSON | NVIDIA | apache-2.0 | public |
| The Massively Multilingual Image Dataset (MMID) | Penn NLP | See citation instructions at… | public |
| TSBench JSONNumPy | Amazon Web Services | [CC BY](https://creativecommons.org/licenses/by/… | public |
| TwitterFollowGraph: Temporal User-Author Follow Graph | Twitter Cortex (El-Kishky et al.) | CC-BY-4.0 | public |
| VoiSeR JSON | Amazon | [cc-by-sa 4.0](https://creativecommons.org/licen… | public |
| When 2 Call JSON Lines | NVIDIA | cc-by-4.0 | public |
| Wind AI Bench HDF5 | National Laboratory of the Rockies | Creative Commons Attribution 4.0 United States… | public |
| Wind AI Bench HDF5 | National Laboratory of the Rockies | Creative Commons Attribution 4.0 United States… | public |
| Wind AI Bench HDF5 | National Laboratory of the Rockies | Creative Commons Attribution 4.0 United States… | public |
| Wizard of Tasks JSON | Amazon | [cc-by-sa 4.0](https://creativecommons.org/licen… | public |
| Y NQ CSV | Meta AI | apache-2.0 | public |
| ZEST: ZEroShot learning from Task descriptions | Allen Institute for AI | [CC BY](https://creativecommons.org/licenses/by/… | public |
| MNIST Handwritten Digits | Yann LeCun, Corinna Cortes, Christopher J.C. Burges | CC BY-SA 3.0 | public |
| AbstentionBench JSONCSV | Meta AI | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| AEF-v1 embeddings Rasteret Collection Parquet | terrafloww via Source Cooperative | not stated | public |
| AlphaEarth Foundations GeoZarr Mosaic JSON | tge-labs via Source Cooperative | not stated | public |
| AlphaEarth Foundations Satellite Embedding Dataset - ZARR | firststreet via Source Cooperative | not stated | public |
| Amazon Berkeley Objects Dataset CSV | Amazon | Creative Commons Attribution-NonCommercial 4.0… non-commercial or no-derivatives | public |
| Amazon Bin Image Dataset JPEG | Amazon | Creative Commons Attribution-NonCommercial-Share… non-commercial or no-derivatives | public |
| Astrophysics Division Galaxy Segmentation Benchmark Dataset FITS | NASA | There are no restrictions on the use of this… | public |
| CIMemories JSONCSV | Meta AI | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| Community coral reef image classification training data PNG | MERMAID | [Creative Commons Attribution-NonCommercial-Shar… non-commercial or no-derivatives | public |
| Consented Activities of People | Visym Labs | Custom (see documentation) | public |
| DigitalCorpora | Simson Garfinkel | There are no restrictions on the use of this… | public |
| Earth Index Embeddings Parquet | earthgenome via Source Cooperative | not stated | public |
| Embeddings Parquet | geovibes via Source Cooperative | not stated | public |
| ESA WorldCover 2021 | major-tom via Source Cooperative | not stated | public |
| Essential-Web v1.0: 24T tokens of organized web data Parquet | EssentialAI | Custom (see documentation) | public |
| FACTORY JSON Lines | Meta AI | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| FashionLocalTriplets JPEG | Amazon | Creative Commons Non-Commercial 4.0 non-commercial or no-derivatives | public |
| FTW Foundation Model Features | mvrl via Source Cooperative | not stated | public |
| FTW Inference Inputs TIFF | ftw via Source Cooperative | not stated | public |
| FTW Inference output TIFFJSON | ftw via Source Cooperative | not stated | public |
| Google Satellite Embeddings v1 Parquet | cr458 via Source Cooperative | not stated | public |
| Humor patterns used for querying Alexa traffic JSON | Amazon | CDLA-Sharing license | public |
| Image classification - fast.ai datasets | fast.ai | Varies by dataset - see documentation link | public |
| Image localization - fast.ai datasets | fast.ai | Varies by dataset - see documentation link | public |
| LAFC-Evict: counterfactual labels for learning-augmented cache eviction | Soroush Vahidi | Mixed per trace family (see LICENSE): wiki2018… | public |
| LCFO Parquet | Meta AI | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| Major TOM Elliot-Pretrain TIFF | major-tom via Source Cooperative | not stated | public |
| Major-TOM Elliot-Pretrain TIFF | major-tom via Source Cooperative | not stated | public |
| Mars Spectrometry 2: Gas Chromatography for the Sample Analysis at Mars Data (SAM) Instrument CSV | NASA | There are no restrictions on the use, access… | public |
| Mars Spectrometry: Detect Evidence for Past Habitability CSV | NASA | There are no restrictions on the use, access… | public |
| MegaScenes | Cornell University https://www.cs.cornell.edu/~snavely/ | Creative Commons Attribution 4.0 International… | public |
| MMRB 2 Image | Meta AI | cc-by-nc-sa-4.0 non-commercial or no-derivatives | public |
| Multi Token Completion CSV | Amazon | Datasets are published under… non-commercial or no-derivatives | public |
| Multimedia Commons JPEG | Multimedia Commons | Custom (see documentation) | public |
| NASA Lunar FM Bench (SomBench): multimodal LRO tiles, benchmarks and Lunar FM weights | NASA IMPACT and IBM Research | CC-BY-4.0 (data and bucket README); model… | public |
| NASA SOHO/LASCO2 comet challenge on AWS | NASA | There are no restrictions on the use of this… | public |
| Nemotron ClimbLab PNGParquet | NVIDIA | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| Nemotron ClimbMix PNGParquet | NVIDIA | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| NLP - fast.ai datasets | fast.ai | Varies by dataset - see documentation link | public |
| OpenSurfaces | Cornell University | The annotations are licensed under a Creative… | public |
| PersonPath22 JSON | Amazon Web Services | Creative Commons Attribution-NonCommercial 4.0… non-commercial or no-derivatives | public |
| Principia Bench PNGJSON Lines | Meta AI | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| PyEnvs and CallArgs | Amazon | Custom (see documentation) | public |
| PyEnvs and CallArgs | Amazon | Custom (see documentation) | public |
| Recycling The Web PNG | Meta AI | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| Search Parquet | geovibes via Source Cooperative | not stated | public |
| Solar Dynamics Observatory (SDO) Machine Learning Dataset | NASA | There are no restrictions on the use of this… | public |
| Voices Obscured in Complex Environmental Settings (VOiCES) | In-Q-Tel | Creative Commons BY 4.0 (see… | public |
| WikiSum: Coherent Summarization Dataset for Efficient Human-Evaluation | Amazon | Dataset is published under… non-commercial or no-derivatives | public |
| Wildchat RIP Filtered By 70 B Llama JSON Lines | Meta AI | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| Wildchat RIP Filtered By 8 B Llama JSON Lines | Meta AI | cc-by-nc-4.0 non-commercial or no-derivatives | public |
| Xiph.Org Test Media | Xiph.org | Various. Most are under the CC-BY license.… | public |
| ImageNet | not stated | public | |
| ISBI Challenge | not stated | public | |
| ASL 1000 | NVIDIA Corporation | Please see the [NVIDIA Dataset… | credentialed |
| YouTube 8 Million - Data Lakehouse Ready | Amazon Web Services | https://github.com/aws-samples/data-lake-as-code… | credentialed |
Related topics
- Parquet datasets (903)
- Benchmark datasets (371)
- JSON Lines datasets (116)
- JSON datasets (679)
- CSV datasets (651)
- PNG datasets (188)
- Image segmentation datasets (357)
- Time series datasets (222)
- JPEG datasets (163)
- TIFF datasets (434)
- Graph learning datasets (22)
- Satellite imagery datasets (436)