ESA title

SENTINELS AnyTIME Development of a continuous Sentinel-1/2 fusion-timeseries service through AI based Deep Bayesian Neural Networks Reconstruction

Data Segment
  • Data Processing & Visualisation
  • Data Analytics, Insights & Applications
SENTINELS AnyTIME is developing a revolutionary real-time Sentinel-1/2 fusion-based timeseries reconstruction API service through AI based Deep Bayesian Neural Networks. The ultimate goal is to remove the harmful effects of cloud cover in the EO imagery and be able to provide continuous (daily) monitoring services required by AgroFood (served by AgriTech) and the land monitoring EO sector as a whole.
Objectives of the Product

Persistent cloud coverage significantly impairs the frequency of EO observations required by the end users in the Potato sector, and thus harming confidence by the broader customer pool in the AgroFood.

The ultimate goal is to remove the harmful effects of cloud cover in the EO imagery and be able to provide continuous (daily) monitoring services required by AgroFood (served by AgriTech) and the land monitoring EO sector as a whole.

Customers and their Needs

Our customer are users in every parts of the EO domain which are in need of continuous monitoring. The highest demand is given in the agriculture domain.

Specifically, the most important agricultural seasons in Europe are, on the one hand, the growing period in spring, and on the other, the harvesting months in fall. This reality, impacting the EO monitoring capacities in the Potato sector, manifests itself with all major crops, especially since early season crop detection and vegetation monitoring is one of the most valuable services in the industry. For example, the identification of emergence which is needed an input for yield modelling, such as maize and soybeans among many others, which have similar low precision and recall values in early season crop type predictions. Those regions located mid-to-high latitude in Europe suffer from consistent cloud coverage in these 2 important periods, which is statistically evident in the Sentinel-2 data archive records and further substantiated by the continuing failure of the Copernicus Third Party mission suppliers to deliver cloud-free VHR coverages within one year of tasking.

Target customers/users countries

Nearly all sectors in the EO Value chain can benefit from cloud-free imagery, the target customer are mostly EO Value Adders with existing value chains that aim to improve their services.

Product description

SENTINELSAnyTIME is a service that allows the user to retrieve cloud-free, radar corrected optical Sentinel-2 data through a subscription model. The product is divided into two separate services:

  • 1. Sentinel-2 cloud-free bands
  • 2. Sentinel-2 cloud-free (pre-defined and custom) indices and biophysical variables.

Both services will make use of advanced AI technologies in both the temporal and spatial domain to allow for an efficient cloud-detection, removal, and ground reflectance reconstruction. This is achieved by temporal S-2 time-series reconstruction (Bayesian approaches, gaussian processes and recurrent NN models) and integration of auxiliary data (Sentinel-1) in a spatial context (Convolutional NN) for the detection of optically hidden sudden events. These include for example landcover changes such as harvesting events or natural disasters that cannot be predicted through interpolation techniques.

Both services are presented to the customer as an easily accessible and user-friendly Graphical User Interface. Furthermore, all service elements can be accessed through an API which facilitates tasks that need to be carried out automatically by the end user or to allow for derived products/services that are built on top of SENTINELSAnyTIME (e.g., agricultural monitoring or environmental services).

SENTINELSAnyTIME Services: Cloud-free band data (left) and cloud-free (custom) Indices

Added value

The added value is manifold and very central for most EO applications.
A continuous (gap free) timeseries of bio-physical indices, time span & time step selectable will lead

  1. an overall increase in the timeseries quality, increases the accuracy of the added value output service.
  2. Increase in temporal continuity (gap free), leads to a more granular and better decision-making points.
    A spatial reconstruction time series of bio-physical indices
  3. Availability of timeseries where there would be no data otherwise, increases the spatial consistency of the added value output service
  4. Increase in spatial continuity (gap free), leads to a higher data completeness, and avoids blind spots in decision-making process.

Allow for continuous monitoring, despite cloud condition. The future is hybrid!

Current Status

A prototype version of Sentinels AnyTIME is available and goes through various testing stages and is adapted to our latest requirement specification. A public demonstrator will be available in 2023.

Prime Contractor Company
Austria Flag Austria
Contractor Project Manager
Dr. David Kolitzus
Sparkassenplatz 2, 6020 Innsbruck, Austria
ESA Technical Officer

Current activities