National Reef Monitoring Network - Survey metadata
- Provider
- AODN
- License
- http://creativecommons.org/licenses/by/4.0/
- Size
- 439 objects, 4,866,001 bytes (4.87 MB)
- Records
- 36,974 rows = 36,974 unique survey_id (sum of all 438 footers, 2026-10-04); 2025 partition 1,234 rows, 2026 partition 605 rows (to 2026-07-19)
- Format
- Parquet (in sampled files)
- Stored at
- Amazon S3, s3://aodn-cloud-optimised/diver_survey_metadata_qc.parquet (ap-southeast-2)
- Access
- Free and public. No account with the data provider is needed.
- Category
- Life Sciences
- Source
- https://catalogue-imos.aodn.org.au/geonetwork/srv/eng/catalog.search#/metadata/997c2e15-b345-438f-afac-49a4ac19be38
About National Reef Monitoring Network - Survey metadata
The IMOS National Reef Monitoring Network (NRMN) sub-facility brings together shallow reef surveys conducted around Australia into a centralised database, collating, cleaning and publishing data from Reef Life Survey (RLS), the Australian Temperate Reef Collaboration (ATRC), Parks Australia, DBCA (WA), DEW (SA), DPI (NSW), Tasmanian Parks and Wildlife and Parks Victoria. RLS and ATRC coordinate surveys of rocky and coral reefs using standard underwater visual census methods, carried out by marine scientists and trained recreational divers, to establish national marine baselines and support the management of Commonwealth and State marine reserves. This dataset provides the metadata associated with individual 50 m transects surveyed by RLS and ATRC divers and partners, all undertaken using standard methods. ATRC surveys are predominantly collected as a group of 4 x 50 m transects per site; RLS surveys commonly have a minimum of 2 x 50 m transects per site on a date (typically at different depths), but this number varies. It supersedes the RLS-specific 'Survey Locations' collection. This store is the quality-controlled survey list as hive-partitioned parquet: one row per transect survey, 36,974 rows with unique survey_id, 4,892 sites in 59 countries, 1992-03-10 to 2026-07-19, with the methods done, depth, visibility, photoquadrat links and flags; it is the master key for the four NRMN observation tables and the only place the 2026 South Australian surveys currently appear.
How to load National Reef Monitoring Network - Survey metadata in Python
import os
import pyarrow.parquet as pq
pf = pq.ParquetFile(os.path.join(os.environ['SCIGANTIC_MOUNT_PATH'], '_common_metadata'))
print(pf.metadata)
df = pf.read_row_group(0).to_pandas()
df.head()Run in a Scigantic notebook on 2026-10-05 against _common_metadata: Parquet 0 rows, 31 columns, 0 row groups; columns ['TIME:timestamp[ns]', 'area:string', 'geom:string', 'hour:string', 'depth:float', 'country:string', 'methods:string', 'polygon:string', 'program:string', 'surface:string'].
From any machine
# Task: same SA subset over anonymous S3 from outside the pod
import pyarrow.dataset as ds, pyarrow.fs as pafs
s3 = pafs.S3FileSystem(anonymous=True, region='ap-southeast-2')
dset = ds.dataset('aodn-cloud-optimised/diver_survey_metadata_qc.parquet', filesystem=s3, format='parquet', partitioning='hive')
m = dset.to_table(filter=(ds.field('timestamp') >= 1735689600) & (ds.field('area') == 'South Australia') & (ds.field('survey_date') >= '2025-03-01'),
columns=['survey_id','site_code','site_name','location','survey_date','methods']).to_pandas()
print(len(m), m['site_code'].nunique(), m['survey_date'].min(), m['survey_date'].max())
print(m.groupby(m['survey_date'].str[:7]).size())Lists the files with anonymous access. Runs on any machine, with no cloud account.
Data structure
File types in sampled files: Parquet
Fields
TIME | timestamp[ns]. timestamp[ns], naive (no tz). Equals survey_date + hour where hour is present, survey_date at 00:00… |
area | string. string. State/territory or sub-national region, e.g. "South Australia", "Tasmania", "Catalonia". 153 distinct… |
geom | string. string. WKB hex POINT SRID 4326 in (longitude, latitude) order here (the observation tables swap the axes).… |
hour | string. string "HH:MM:SS", documented local time; null on 16,529 rows (45%). 8% of non-null Australian values are… |
depth | float. float32, metres, transect depth 0 to 42. |
country | string. string. 59 distinct; Australia 31,859 surveys (86%). Null on 3. |
methods | string. string. Comma-separated RLS/ATRC method codes run on this transect, e.g. "1, 2" (9,734 surveys), "1, 2, 3"… |
polygon | string. string, hive partition key (not stored in the files). WKB hex of the 20 x 20 degree spatial cell the row is… |
program | string. string. RLS 20,660, ATRC 13,720, Parks Vic 2,430, FRDC 128, RRH 36. |
surface | string. string. "Coral" 1,059, "Rock" 419, null on 35,496 (96%). |
filename | string. string. Constant "ep_survey_list_public_data.csv". |
latitude | float. float32, degrees_north. Site latitude. |
location | string. string. Named region within the area, e.g. "Encounter", "Yorke Peninsula", "Adelaide", "Kangaroo Island". 221… |
direction | string. string. Free-text transect heading: N/S/E/W/NE/SW ... but also lowercase, "N ", "West", "S.W.", "Sam G"… |
longitude | float. float32, degrees_east. Site longitude. |
site_code | string. string. Site identifier (e.g. "GSV135"); 4,892 distinct. |
site_name | string. string. 4,864 distinct names. |
survey_id | int32. int32. UNIQUE per row (36,974 distinct = row count). Primary key; join target for the four observation tables. |
timestamp | int64. int64 per _common_metadata, read back as int32 by the hive partitioning. Partition key = epoch seconds of 1… |
pq_zip_url | string. string. URL of the photoquadrat zip for the survey, e.g. http://rls.tpac.org.au/pq/923406716/zip/ ; null on… |
and 11 more
Analyze this dataset without downloading it
On Scigantic, National Reef Monitoring Network - Survey metadata opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.
This page is also available as Markdown and as JSON.
Related Life Sciences datasets
- Pacific Ocean Sound Recordings
- Pacific Ocean Sound Recordings
- OpenOrganelle jrc_fly-acc-calyx-1: Drosophila Mushroom Body Accessory Calyx, FIB-SEM
- Pacific Ocean Sound Recordings
- Biogeochemical Database - National Reference Station and Continuous Plankton Recorder plankton, pigments and chemistry (CSV)
- Replay Protected Monotonic Counters (RPMC) data
- National Reef Monitoring Network - Global cryptobenthic fish abundance
- Cell Painting Gallery cpg0016: JUMP Cell Painting Production Dataset, 116,000 Compounds and 15,000 Genes