# MADIS METAR Surface Station Observations 2001-2023 (WeatherBench 2, Parquet)

## Facts

- Provider: NOAA MADIS; converted by Google Research (WeatherBench 2)
- License: CC-BY-4.0 (https://storage.googleapis.com/weatherbench2/datasets/metar/LICENSE)
- Format: Parquet (in sampled files)
- Stored at: Google Cloud Storage, gs://weatherbench2/datasets/metar (us)
- Access: public, readable without an account
- DOI: https://doi.org/10.1029/2023MS004019
- Category: Climate & Weather
- Source: https://doi.org/10.1029/2023MS004019
- Page: https://scigantic.com/archive/0982537f-0952-5ee7-8fee-fc3376d42243
- Keywords: Climate & Weather, weather forecasting, Parquet, Google Cloud Storage

## About

Global METAR and SPECI weather station reports from the NOAA Meteorological Assimilation Data Ingest System (MADIS), downloaded from the MADIS FTP server in NetCDF and converted by the WeatherBench 2 team to Parquet for verifying forecasts against point observations rather than reanalysis. Each row is one report with station name, location name, latitude, longitude, elevation, observation, nominal and receipt times, report type, visibility, present weather, sea level pressure, temperature, dewpoint, wind direction, speed and gust, altimeter, 24-hour min and max temperature, 1, 3, 6 and 24-hour precipitation, 3-hour pressure change and snow cover, with MADIS data-descriptor (DD) quality flags. The same data is laid out four ways, Hive-partitioned by year and month: by nominal time per month, per day and per hour, and by observation time per hour from 2001-07-01 to 2024-01-01. Measured 2026-11-14: the monthly layout holds 270 Parquet files totalling 26.8 GB. Read with pandas.read_parquet or pyarrow.dataset. The WeatherBench 2 LICENSE file states CC BY 4.0 and notes MADIS METAR data is publicly accessible without restrictions; cite NOAA MADIS and Rasp et al. 2024.

## Load the data

```python
import os
import pyarrow.parquet as pq
pf = pq.ParquetFile(os.path.join(os.environ['SCIGANTIC_MOUNT_PATH'], 'metar-timeNominal-by-day/year=2001/month=10/day=1/2001-10-01.parquet'))
print(pf.metadata)
df = pf.read_row_group(0).to_pandas()
df.head()
```

Run in a Scigantic notebook on 2026-11-15 against `metar-timeNominal-by-day/year=2001/month=10/day=1/2001-10-01.parquet`: Parquet 53,323 rows, 44 columns, 1 row groups; columns ['stationName:string', 'locationName:string', 'latitude:float', 'longitude:float', 'elevation:float', 'timeObs:timestamp[ns]', 'timeNominal:timestamp[ns]', 'timeReceived:timestamp[ns]', 'reportType:string', 'autoStationType:string'].

From any machine:

```python
import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('weatherbench2/datasets/metar/', detail=False)[:10]
```

Lists the files with anonymous access. Runs on any machine, with no cloud account.

## Data structure

File types in sampled files: Parquet

Example file: `datasets/metar/metar-timeNominal-by-day/year=2001/month=10/day=1/2001-10-01.parquet`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/0982537f-0952-5ee7-8fee-fc3376d42243
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/0982537f-0952-5ee7-8fee-fc3376d42243
- Over MCP: `get_schema_card` with id `0982537f-0952-5ee7-8fee-fc3376d42243` at https://api.scigantic.com/mcp

## Related topics

- [Parquet datasets](https://scigantic.com/topics/parquet)
- [Climate & Weather datasets](https://scigantic.com/topics/climate-and-weather)
- [Weather forecasting datasets](https://scigantic.com/topics/weather-forecasting)
