# MBPP (Mostly Basic Python Problems): 974 Crowd-Sourced Python Programming Tasks with Reference Solutions and Unit Tests (Google Research)

## Facts

- Provider: Google Research (Austin et al.)
- License: CC-BY-4.0 (https://huggingface.co/datasets/google-research-datasets/mbpp)
- Size: 10 files
- Format: Parquet (in sampled files)
- Stored at: Hugging Face, https://huggingface.co/datasets/google-research-datasets/mbpp
- Access: public, readable without an account
- Category: Computer Science
- Source: https://arxiv.org/abs/2108.07732
- Page: https://scigantic.com/archive/bc986b33-48e5-5d3b-9d23-c0bf4e3f0060
- Keywords: Computer Science, benchmark, Parquet, Hugging Face

## About

MBPP (Mostly Basic Python Problems) is a program synthesis benchmark of 974 crowd-sourced Python programming problems designed to be solvable by entry-level programmers, covering programming fundamentals and standard library functionality. Each problem has a short natural language task description, a reference code solution and three automated assert test cases, so generated code can be scored by execution (pass@k). A hand-verified subset of 427 problems is provided as the sanitized configuration. The authors' split uses task IDs 11 to 510 for testing, 1 to 10 for few-shot prompts, 511 to 600 for validation and 601 to 974 for training. The repository holds two folders of Parquet files, full/ and sanitized/, each with train, validation, test and prompt splits (well under 1 MB). It is one of the standard benchmarks for code generation by large language models, alongside HumanEval. Load with the Hugging Face datasets library (load_dataset('google-research-datasets/mbpp')). Licensed CC BY 4.0 by Google Research; cite Austin et al., Program Synthesis with Large Language Models, 2021.

## Load the data

```python
from datasets import load_dataset
ds = load_dataset("google-research-datasets/mbpp")
print(ds)
```

## Data structure

File types in sampled files: Parquet

Example file: `full/prompt-00000-of-00001.parquet`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/bc986b33-48e5-5d3b-9d23-c0bf4e3f0060
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/bc986b33-48e5-5d3b-9d23-c0bf4e3f0060
- Over MCP: `get_schema_card` with id `bc986b33-48e5-5d3b-9d23-c0bf4e3f0060` at https://api.scigantic.com/mcp

## Related topics

- [Parquet datasets](https://scigantic.com/topics/parquet)
- [Benchmark datasets](https://scigantic.com/topics/benchmark)
