NNJA-AI: AI-Ready Parquet Reprocessing of the NOAA-NASA Joint Archive of Weather Observations (Brightband)

Provider
Brightband, in partnership with NOAA
License
CC-BY-4.0 (source)
Format
Parquet (in sampled files)
Stored at
Google Cloud Storage, gs://nnja-ai/data (us)
Access
Free and public. No account with the data provider is needed.
DOI
10.5281/zenodo.14633508
Category
Climate & Weather
Source
https://doi.org/10.5281/zenodo.14633508

About this dataset

Brightband's reprocessing of the NOAA-NASA Joint Archive (NNJA) of Earth system observations from the original BUFR files into Parquet, so that observation-based AI weather models can read them without BUFR decoders. Version 1 holds about 50 TiB of observations with a JSON data catalog. Measured 2026-11-17 from data/v1/catalog.json: 14 datasets, including AMSU-A, ATMS and MHS Level 1B microwave brightness temperatures (NOAA-15 to -19, METOP, NPP, NOAA-20), CrIS full-spectral radiances (431-channel subset), IASI radiances, Himawari AHI clear-sky radiances, GOES ABI all-sky and clear-sky radiances, SEVIRI all-sky radiances, and conventional ADP surface (SYNOP fixed and mobile land, METAR/SPECI) and upper-air rawinsonde reports. The bucket has data/v1-preview/, data/v1/ and data/v2/; v2 adds atmospheric motion vectors (amv/), GPS radio occultation (gps/), convbufr and prepbufr conventional data, and _sidecars/ metadata. Open with the nnja-ai Python SDK (DataCatalog) or any Parquet reader such as pandas, polars or DuckDB. The data carry the same CC BY 4.0 license as the original NNJA; cite the nnja-ai Zenodo DOI 10.5281/zenodo.14633508 and follow the NNJA citation guidance from NOAA PSL.

How to access this dataset in Python

import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('nnja-ai/data/', detail=False)[:10]

Lists the files with anonymous access. Runs on any machine, with no cloud account.

Data structure

File types in sampled files: Parquet

Files

Example file: data/v1-preview/amsua/1bamua/NC021023/.pmetadata

Analyze this dataset without downloading it

On Scigantic, NNJA-AI: AI-Ready Parquet Reprocessing of the NOAA-NASA Joint Archive of Weather Observations (Brightband) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Climate & Weather datasets