About

The increasing frequency and severity of extreme weather events, including droughts, heatwaves and floods, are placing growing pressure on agricultural production. Climate change is altering growing seasons and increasing the variability of environmental conditions, making timely information on crop development, condition increasingly important for sustainable and climate-resilient agriculture. Effective monitoring therefore requires not only knowing where crops are grown, but also understanding how they develop and how they respond to changing conditions. Recent advances in Earth Observation (EO) and Artificial Intelligence (AI), including Geospatial Foundation Models (Geo-FMs), offer new opportunities for large-scale and information-rich agricultural monitoring. However, their development and application remain constrained by the limited availability of high-quality, spatially representative reference data. Geo- and time-tagged photos offer a promising alternative to sharply speed up data collection based on manual annotation while providing detailed insights that are difficult to obtain through traditional field surveys. Due to the rise in crowdsourced data availability, there has been a growing interest in understanding how to use field photos to create reference data for different applications. As of today, no operational solutions were established since translating these images into usable information was unclear, along with challenges related to image quality and inconsistencies in field photo datasets. This is a crucial aspect as field photos can reveal crop conditions, offering real-time insights into crop health (e.g., affected by climatic events like droughts, floods, or pest attacks). This detailed information can complement satellite-derived data, leading to a comprehensive agricultural reference dataset.

This project aims to contribute to the development of cutting-edge AI4EO solutions, providing the EO research community solutions to integrate the ground information provided by field photos and the temporal information provided by SITS to generate comprehensive agricultural reference data. AgroVision will enable the automatic labelling of field photos from agricultural areas, moving beyond traditional crop-type data collected in the field, by translating the visual information present in the field photos into a crop condition description. This multimodal integration enables AgroVision to learn not only what is growing, but also how crops develop and what is happening to them. Indeed, Visual Language Models (VLMs) can be used to automatically identify crop types, but also to extract objective descriptions of crop condition, including phenological stages and farming practices such as intercropping. These detailed ground observations provide complementary semantic information to satellite-derived representations and can help models better interpret satellite signals, particularly where spectral responses are influenced by environmental, climatic, seasonal, soil, or acquisition conditions. By linking satellite-based predictions to interpretable ground-level observations, this multimodal approach also provides a pathway towards more transparent, explainable, and trustworthy agricultural AI.

The automatic generation of reference information from field photographs and Geo-FMs embeddings provides a mechanism to extend agricultural monitoring beyond data-rich regions towards data-scarce areas, where conventional field surveys and annotated reference datasets are limited. This creates opportunities for adapting agricultural AI across regions with substantially different data availability, including challenging contexts in Africa and Asia, characterized by complex agricultural systems. To move beyond direct fusion, AgroVision further investigates knowledge distillation and contrastive learning to transfer fine-grained ground-level information to satellite-based representations, enabling scalable crop type mapping also where photos are not available. With the increasing frequency and severity of extreme weather events, timely and scalable agricultural monitoring is becoming imperative to better address their impacts on crop production. By combining detailed field photos with SITS, AgroVision supports the monitoring of agricultural conditions and climate-driven impacts such as drought, heat, and flooding, while enabling adaptation across crops, regions, and seasons without extensive retraining. Automated interpretation of field photographs further reduces manual annotation effort, making field-data collection faster, more consistent, and scalable.