# NNJA-AI: AI-Ready Parquet Reprocessing of the NOAA-NASA Joint Archive of Weather Observations (Brightband)

## Facts

- Provider: Brightband, in partnership with NOAA
- License: CC-BY-4.0 (https://nnja-ai.readthedocs.io/en/stable/)
- Format: Parquet (in sampled files)
- Stored at: Google Cloud Storage, gs://nnja-ai/data (us)
- Access: public, readable without an account
- DOI: https://doi.org/10.5281/zenodo.14633508
- Category: Climate & Weather
- Source: https://doi.org/10.5281/zenodo.14633508
- Page: https://scigantic.com/archive/6ae11274-0280-5716-9084-09d83e5b01aa
- Keywords: Climate & Weather, Parquet, Google Cloud Storage

## About

Brightband's reprocessing of the NOAA-NASA Joint Archive (NNJA) of Earth system observations from the original BUFR files into Parquet, so that observation-based AI weather models can read them without BUFR decoders. Version 1 holds about 50 TiB of observations with a JSON data catalog. Measured 2026-11-17 from data/v1/catalog.json: 14 datasets, including AMSU-A, ATMS and MHS Level 1B microwave brightness temperatures (NOAA-15 to -19, METOP, NPP, NOAA-20), CrIS full-spectral radiances (431-channel subset), IASI radiances, Himawari AHI clear-sky radiances, GOES ABI all-sky and clear-sky radiances, SEVIRI all-sky radiances, and conventional ADP surface (SYNOP fixed and mobile land, METAR/SPECI) and upper-air rawinsonde reports. The bucket has data/v1-preview/, data/v1/ and data/v2/; v2 adds atmospheric motion vectors (amv/), GPS radio occultation (gps/), convbufr and prepbufr conventional data, and _sidecars/ metadata. Open with the nnja-ai Python SDK (DataCatalog) or any Parquet reader such as pandas, polars or DuckDB. The data carry the same CC BY 4.0 license as the original NNJA; cite the nnja-ai Zenodo DOI 10.5281/zenodo.14633508 and follow the NNJA citation guidance from NOAA PSL.

## Load the data

```python
import gcsfs
fs = gcsfs.GCSFileSystem(token='anon')
fs.ls('nnja-ai/data/', detail=False)[:10]
```

Lists the files with anonymous access. Runs on any machine, with no cloud account.

## Data structure

File types in sampled files: Parquet

Example file: `data/v1-preview/amsua/1bamua/NC021023/.pmetadata`

## Open it

- In a hosted notebook with the data mounted: https://scigantic.com/archive/6ae11274-0280-5716-9084-09d83e5b01aa
- Full record as JSON (schema card included): https://api.scigantic.com/api/archives/6ae11274-0280-5716-9084-09d83e5b01aa
- Over MCP: `get_schema_card` with id `6ae11274-0280-5716-9084-09d83e5b01aa` at https://api.scigantic.com/mcp

## Related topics

- [Parquet datasets](https://scigantic.com/topics/parquet)
- [Climate & Weather datasets](https://scigantic.com/topics/climate-and-weather)
