About

European agriculture is affected by extreme weather events, such as droughts, hail, and storms, which are expected to intensify in frequency and severity. Accurately mapping crop locations and their growth stages is crucial for mitigating these impacts. Substantial efforts have focused on using EO data, particularly the free and open-access Sentinel-1 and Sentinel-2 data, for monitoring agricultural areas. However, collecting field data is both expensive and time-consuming. Geo- and time-tagged photos offer a promising alternative to sharply speed up data collection based on manual annotation while providing detailed insights that are difficult to obtain through traditional field surveys. Due to the rise in crowdsourced data availability, there has been a growing interest in understanding how to use field photos to create reference data for different applications. As of today, no operational solutions were established since translating these images into usable information was unclear, along with challenges related to image quality and inconsistencies in field photo datasets. This is a crucial aspect as field photos can reveal crop conditions, offering real-time insights into crop health (e.g., affected by climatic events like droughts, floods, or pest attacks). This detailed information can complement satellite-derived data, leading to a comprehensive agricultural reference dataset.

This project aims to contribute to the development of cutting-edge AI4EO solutions, providing the EO research community with a new Geo-FM able to integrate the ground information provided by field photos and the temporal information provided by SITS to generate comprehensive agricultural reference data. AgroVision will enable the automatic labelling of field photos from agricultural areas, moving beyond traditional crop-type data collected in the field, by translating the visual information present in the field photos into a crop condition description. To meet these goals, the project will design and develop an innovative multimodal Geo-FM that integrates field photos with SITS focusing on two main objectives: (1) automatic identification of crop types present in the geo-tagged field photos, leveraging the complementary information provided by SITS and field photos; (2) effectively translate field photos into a description of crop phenology (developmental stages), crop status (whether it is affected by climatic events such as droughts, floods, or pest attacks), as well as crop management practices (e.g., intercropping), providing comprehensive reference data to support precision agriculture.