National Reef Monitoring Network - Survey metadata

Provider
AODN
License
http://creativecommons.org/licenses/by/4.0/
Size
439 objects, 4,866,001 bytes (4.87 MB)
Records
36,974 rows = 36,974 unique survey_id (sum of all 438 footers, 2026-10-04); 2025 partition 1,234 rows, 2026 partition 605 rows (to 2026-07-19)
Format
Parquet (in sampled files)
Stored at
Amazon S3, s3://aodn-cloud-optimised/diver_survey_metadata_qc.parquet (ap-southeast-2)
Access
Free and public. No account with the data provider is needed.
Category
Life Sciences
Source
https://catalogue-imos.aodn.org.au/geonetwork/srv/eng/catalog.search#/metadata/997c2e15-b345-438f-afac-49a4ac19be38

About National Reef Monitoring Network - Survey metadata

The IMOS National Reef Monitoring Network (NRMN) sub-facility brings together shallow reef surveys conducted around Australia into a centralised database, collating, cleaning and publishing data from Reef Life Survey (RLS), the Australian Temperate Reef Collaboration (ATRC), Parks Australia, DBCA (WA), DEW (SA), DPI (NSW), Tasmanian Parks and Wildlife and Parks Victoria. RLS and ATRC coordinate surveys of rocky and coral reefs using standard underwater visual census methods, carried out by marine scientists and trained recreational divers, to establish national marine baselines and support the management of Commonwealth and State marine reserves. This dataset provides the metadata associated with individual 50 m transects surveyed by RLS and ATRC divers and partners, all undertaken using standard methods. ATRC surveys are predominantly collected as a group of 4 x 50 m transects per site; RLS surveys commonly have a minimum of 2 x 50 m transects per site on a date (typically at different depths), but this number varies. It supersedes the RLS-specific 'Survey Locations' collection. This store is the quality-controlled survey list as hive-partitioned parquet: one row per transect survey, 36,974 rows with unique survey_id, 4,892 sites in 59 countries, 1992-03-10 to 2026-07-19, with the methods done, depth, visibility, photoquadrat links and flags; it is the master key for the four NRMN observation tables and the only place the 2026 South Australian surveys currently appear.

How to load National Reef Monitoring Network - Survey metadata in Python

import os
import pyarrow.parquet as pq
pf = pq.ParquetFile(os.path.join(os.environ['SCIGANTIC_MOUNT_PATH'], '_common_metadata'))
print(pf.metadata)
df = pf.read_row_group(0).to_pandas()
df.head()

Run in a Scigantic notebook on 2026-10-05 against _common_metadata: Parquet 0 rows, 31 columns, 0 row groups; columns ['TIME:timestamp[ns]', 'area:string', 'geom:string', 'hour:string', 'depth:float', 'country:string', 'methods:string', 'polygon:string', 'program:string', 'surface:string'].

From any machine

# Task: same SA subset over anonymous S3 from outside the pod
import pyarrow.dataset as ds, pyarrow.fs as pafs
s3 = pafs.S3FileSystem(anonymous=True, region='ap-southeast-2')
dset = ds.dataset('aodn-cloud-optimised/diver_survey_metadata_qc.parquet', filesystem=s3, format='parquet', partitioning='hive')
m = dset.to_table(filter=(ds.field('timestamp') >= 1735689600) & (ds.field('area') == 'South Australia') & (ds.field('survey_date') >= '2025-03-01'),
                  columns=['survey_id','site_code','site_name','location','survey_date','methods']).to_pandas()
print(len(m), m['site_code'].nunique(), m['survey_date'].min(), m['survey_date'].max())
print(m.groupby(m['survey_date'].str[:7]).size())

Lists the files with anonymous access. Runs on any machine, with no cloud account.

Data structure

File types in sampled files: Parquet

Fields

TIMEtimestamp[ns]. timestamp[ns], naive (no tz). Equals survey_date + hour where hour is present, survey_date at 00:00…
areastring. string. State/territory or sub-national region, e.g. "South Australia", "Tasmania", "Catalonia". 153 distinct…
geomstring. string. WKB hex POINT SRID 4326 in (longitude, latitude) order here (the observation tables swap the axes).…
hourstring. string "HH:MM:SS", documented local time; null on 16,529 rows (45%). 8% of non-null Australian values are…
depthfloat. float32, metres, transect depth 0 to 42.
countrystring. string. 59 distinct; Australia 31,859 surveys (86%). Null on 3.
methodsstring. string. Comma-separated RLS/ATRC method codes run on this transect, e.g. "1, 2" (9,734 surveys), "1, 2, 3"…
polygonstring. string, hive partition key (not stored in the files). WKB hex of the 20 x 20 degree spatial cell the row is…
programstring. string. RLS 20,660, ATRC 13,720, Parks Vic 2,430, FRDC 128, RRH 36.
surfacestring. string. "Coral" 1,059, "Rock" 419, null on 35,496 (96%).
filenamestring. string. Constant "ep_survey_list_public_data.csv".
latitudefloat. float32, degrees_north. Site latitude.
locationstring. string. Named region within the area, e.g. "Encounter", "Yorke Peninsula", "Adelaide", "Kangaroo Island". 221…
directionstring. string. Free-text transect heading: N/S/E/W/NE/SW ... but also lowercase, "N ", "West", "S.W.", "Sam G"…
longitudefloat. float32, degrees_east. Site longitude.
site_codestring. string. Site identifier (e.g. "GSV135"); 4,892 distinct.
site_namestring. string. 4,864 distinct names.
survey_idint32. int32. UNIQUE per row (36,974 distinct = row count). Primary key; join target for the four observation tables.
timestampint64. int64 per _common_metadata, read back as int32 by the hive partitioning. Partition key = epoch seconds of 1…
pq_zip_urlstring. string. URL of the photoquadrat zip for the survey, e.g. http://rls.tpac.org.au/pq/923406716/zip/ ; null on…

and 11 more

Analyze this dataset without downloading it

On Scigantic, National Reef Monitoring Network - Survey metadata opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

This page is also available as Markdown and as JSON.

Related Life Sciences datasets