EarthRecord: From Pixels to Land Change with Multimodal Agents over Four Decades of Satellite Imagery
Abstract
While multimodal agents now carry out complex data analysis with tools, their ability to turn decades of satellite imagery into reliable land-change analysis, crucial for Earth observation science, remains unmeasured. Existing benchmarks fall short: they pose questions on curated images or give agents preprocessed inputs, leaving out the raw multi-sensor archives where such analysis begins. We introduce EarthRecord, a benchmark built on the raw record of 56 sites: four decades of Landsat, MODIS, VIIRS, and Sentinel-2 data, with cloud masking and cross-sensor harmonization left to the agent. It poses 238 land-change questions across ten analysis tasks, scored against withheld references from global products, agreement among methods, and visual interpretation. Two controls test what answers rest on: the closed book removes the record, and instrument pairs alter one satellite's data while nothing changes on the ground. Evaluations of twelve multimodal agents reveal that leading agents complete most analyses end to end but falter when an answer requires mapping from imagery. Further analysis shows that the record moves answers from prior knowledge to the pixels but not yet to the real change on the ground: when one satellite's data are altered, half of the right answers follow the alteration. EarthRecord thus provides a principled platform for measuring and advancing agents from pixels to land change.
Community
EarthRecord asks whether multimodal agents can turn four decades of raw satellite imagery into reliable land-change analysis. For each question, the agent works in an offline sandbox (Python, shell, image viewer) on the record of one site as distributed by its providers: Landsat, MODIS, VIIRS and Sentinel-2 data, with cloud masking and cross-sensor harmonization left to the agent.
- 56 sites, 1984–2026; 238 land-change questions over ten analysis tasks, scored against withheld references
- Two controls: a closed book that removes the record, and 22 instrument pairs in which one satellite's data are altered while nothing changes on the ground
- Twelve agents evaluated; the best reaches 85.4% pass³. Leading agents complete most analyses end to end but falter when an answer requires mapping from imagery, and when one satellite's data are altered, half of the right answers follow the alteration
Data: https://huggingface.co/datasets/haoranranranran/EarthRecord
Code and leaderboard: https://github.com/ranranrannervous/EarthRecord
Get this paper in your agent:
hf papers read 2611.12934 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 1
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper