Nature intelligence deserves its own playground.
Existing benchmarksThe landscape today
Existing benchmarks
Nature intelligence is measured in many places, one discipline at a time.
Bioacoustics
-
BEANS
Classification and detection across 12 datasets of birds, land and marine mammals, anurans and insects. By Earth Species Project.
github.com -
BEANS-Zero
Extends BEANS with call types, life stages, captioning and counting to test zero-shot transfer. The reference eval for audio-language models.
arxiv.org -
BirdSet
6,800+ training hours over ~10,000 classes, plus eight strongly labeled evaluation sets. ICLR 2025.
huggingface.co -
BIRB
Retrieve bird calls from passive recordings after training on citizen-science audio. A hard test of distribution shift.
github.com -
BirdCLEF+
Annual competition to detect and classify animal calls in soundscapes from autonomous recorders.
imageclef.org
Earth observation
Species distribution
Vision & multimodal
-
iWildCam
Camera-trap photos of 182 species, where each trap is its own domain. Tests generalization to new deployments.
wilds.stanford.edu -
BioCLIP Rare Species
~12K images of 400 IUCN-threatened species held out from training to test zero-shot recognition.
huggingface.co -
INQUIRE
250 expert retrieval queries over 5M iNaturalist images. The best models stay below 50 mAP@50. NeurIPS 2024.
inquire-benchmark.github.io