National Reef Monitoring Network - Benthic cover data (in situ surveys)

The National Reef Monitoring Network (NRMN) brings together shallow reef surveys conducted around Australia into a centralised database. The IMOS NRMN Sub-Facility collates, cleans, stores and makes this data rapidly available from contributors including Reef Life Survey, Parks Australia, DBCA (Western Australia), DEW (South Australia), DPI (New South Wales), Tasmanian Parks and Wildlife Service and Parks Victoria. This dataset contains the cover of macroalgae and sessile invertebrates scored in situ at Australian Temperate Reef Collaboration (ATRC) sites: quadrats are placed at 10 m spacing along each transect (5 per 50 m transect), and the canopy, mid-storey and substrate layers are assessed in sequence in each single 50-point quadrat by divers skilled in macroalgal identification. This store is the quality-controlled table as hive-partitioned parquet: 489,046 point-count rows (one per species per quadrat) from 14,419 transects at 720 sites in TAS, WA, VIC, SA and NSW, 1992-03 to 2025-03, partitioned by survey year and 20-degree cell.

Category
Life Sciences
Provider
AODN
License
http://creativecommons.org/licenses/by/4.0/
Hosted on
Amazon S3, bucket aodn-cloud-optimised/diver_benthic_cover_in_situ_qc.parquet, ap-southeast-2
Access
Free and public. No account with the data provider is needed.
Source
https://catalogue-imos.aodn.org.au/geonetwork/srv/eng/catalog.search#/metadata/ec424e4f-0f55-41a5-a3f2-726bc4541947
Data structure and loading code

Fields

TIMEtimestamp[ns]; 'Derived timestamp' = sur
areastring; state/territory: Tasmania 293,49
geomstring; EWKB hex of the site point with
hourstring HH:mm:ss local time; null in 78.2
classstring; taxonomic class of the scored el
depthfloat32 m; transect depth 1.1 to 15.0 m,
orderstring; taxonomic order; null 19.6%
realmstring; MEOW realm, single value 'Temper
totalint32; number of quadrat points (of 50)
familystring; taxonomic family; null 26.8%
phylumstring; Rhodophyta, Heterokontophyta, Po
countrystring; single value 'Australia'
polygonstring (hive partition key); WKB hex of
programstring; ATRC 437,775 | Parks Vic 43,145
quadratstring; Q1..Q5 (5 x 50-point quadrats pe
filenamestring; single value 'ep_m3_isq_public_d
latitudefloat32 degrees_north; SITE coordinate r
locationstring; 36 regions, e.g. 'Encounter', 'Y
ecoregionstring; MEOW ecoregion: Bassian, Cape Ho
longitudefloat32 degrees_east; site coordinate, 1

and 12 more

Load the data

import time, pandas as pd, pyarrow.dataset as ds, s3fs
# NRMN in-situ benthic cover (ATRC method 3): point counts per species per 50-point quadrat, 5 quadrats per transect
fs = s3fs.S3FileSystem(anon=True, client_kwargs={'region_name': 'ap-southeast-2'})
root = 'aodn-cloud-optimised/diver_benthic_cover_in_situ_qc.parquet'   # in the pod: '/mnt/archive' with no filesystem=
SA_CELL = ('010300000001000000050000000000000000005E4000000000000044C0000000000080614000000000000044C0'
           '000000000080614000000000000034C00000000000005E4000000000000034C00000000000005E4000000000000044C0')
t0 = time.time()
tbl = ds.dataset(root, filesystem=fs, format='parquet', partitioning='hive').to_table(
    filter=(ds.field('polygon') == SA_CELL) & (ds.field('timestamp') >= 1672531200))   # 2023-01-01 onward
sa = tbl.to_pandas().query("area == 'South Australia'").copy()
print(f"SA rows since 2023: {len(sa)} in {time.time()-t0:.1f}s; latest survey {sa['TIME'].max()}; rows since 2025-03-01: {int((sa['TIME'] >= '2025-03-01').sum())}")
# `total` is points scored, layers are counted separately, so normalise by the survey's own point total, not by 50
sa['share_pct'] = sa['total'] / sa.groupby('survey_id')['total'].transform('sum') * 100
top = sa.groupby(['site_name', 'species_name'])['share_pct'].sum().groupby(level=0, group_keys=False).nlargest(3)
print(top.round(1).to_string())

Written for a Scigantic notebook, where /mnt/archive is the mounted dataset.

Analyze this dataset without downloading it

On Scigantic, National Reef Monitoring Network - Benthic cover data (in situ surveys) opens in a hosted JupyterLab notebook with the data already mounted, so you can read files where they are stored instead of copying them first. Each notebook includes an AI assistant that can see the mounted files and write and run analysis code. Open the dataset page to launch a notebook, or browse the full catalog.

Related Life Sciences datasets