This document discusses work package 2 of the EUXDAT project, which focuses on defining requirements and e-infrastructure for extreme data analytics. It provides an overview of the objectives, tasks, and deliverables of WP2. Several pilot scenarios are described in detail to validate the e-infrastructure developments, including enriching an open land use map with morphological characteristics and monitoring crop status using earth observation data. The progress on gathering requirements and defining the e-infrastructure is also summarized.

1. WP 2 – Requirements and e-Infrastructure
Definition
General Assembly
(17-18th September, Thessaloniki)
Karel Jedlička (WirelessInfo),
This project has received funding from the European Union’s
Horizon 2020 research and innovation programme under
grant agreement No 777549
www.EUXDAT.eu
European e-Infrastructure
for Extreme Data Analytics
in Sustainable Development

2. 2www.euxdat.eu
WP 2 – Requirements and e-Infrastructure
Definition
1. Status of the WP2
2. Revision Plan M23-M24

3. 3www.euxdat.eu
WP 2 – Objectives
Objective 2.1: Describe the pilots proposed as the validators of the e-
Infrastructure developments to be done in the rest of WPs.
Objective 2.2: Identify and describe those end users communities that might
benefit from the proposed e-Infrastructure, which may have new data
management-related needs.
Objective 2.3: Gather requirements from the pilots and from other
stakeholders, in order to understand the current and future needs for
computation and data management.
Objective 2.4: Define the main features to be fulfilled by the
e-Infrastructure, identifying the key components to be modified and the
potential bottlenecks for scaling up to extremely large data analysis.

4. 4www.euxdat.eu
WP 2 – Tasks
Task 2.1: Pilots and Future Problems Description (WRLS)
Task 2.2: Platform-Driven e-Infrastructure Requirements (WRLS)
Task 2.3: High Level Definition of the e-Infrastructure (ATOSES)

5. 5www.euxdat.eu
WP 2 – Deliverables
• D2.1 Description of Proposed Pilots and Requirements
M4 (WRLS)
• D2.2 EUXDAT e-Infrastructure Definition v1
M6 (ATOSES)
• D2.3 Updated Report on e-Infrastructure Requirements v1
M12 (WRLS)
• D2.4 EUXDAT e-Infrastructure Definition v2
M15 (ATOSES)
• D2.5 Updated Report on e-Infrastructure Requirements v2
M20 (WRLS)
• D2.6 EUXDAT e-Infrastructure Definition v3
M24 (ATOSES)

6. 6www.euxdat.eu
T2.1 Pilots description T2.2 e-Infrastructure requirements
T2.3 e-Infrastructure definition
WP 2 – Objectives – Tasks - Deliverables
O2.1:
describe
pilots
Agriculture
experts
group
O2.3: pilots &
users
requirements
O2.4:
e-Infra-
structure
features
D2.1, .3, .5: (Pilot) and e-Infrastructure requirements
O2.2:
describe
users
D2.2, .4, .6:
e-Infra
definition

7. 7www.euxdat.eu
WP 2 – Progress
Objective 2.1: Describe the pilots proposed as the validators of the e-
Infrastructure developments to be done in the rest of WPs.
Scenarios description
• Morphometry characteristics for OpenLandUse
• Monitoring of crop status
• Agroclimatic zones
• Looking for climatic patterns changes
• Information support for field use recommendations
• Effective utilization of natural resources
• Determination of crop and growth stage
• Calculate long time climate and geographic position effects on the mass changes in
Insects

8. 8www.euxdat.eu
Enrichment of OLU by geo-
morphological characteristics
Scenario 1: Open Land Use Map improvement
• Example for a particular field
min_elevation: 186.64203
max_elevation: 196.92177
mean_elevation: 190.70232
median_elevation: 190.44339
min_slope: 0.75737
max_slope: 1.56950
mean_slope: 1.23466
median_slope: 1.25559
min_azimuth: -179.98296
max_azimuth: 179.62314
mean_azimuth: -13.07384
median_azimuth: -30.85966

9. 9www.euxdat.eu
Enrichment of OLU by geo-
morphological characteristics
Data generation
 On the fly
 The data on user request is generated onfly on ATOS cloud. The
code used for generation on backend is in Python. The API is
written in Flask microframework and deployed as container.
 Pregenerated:
 The data for the whole Europe are generated at once at HPC in
Stuttgart and then dumped and sent to ATOS cloud. This
approach generates all data at once and allows some analysis
(selection by complex criteria) and visualization of the whole
database

10. 10www.euxdat.eu
Enrichment of OLU by geo-
morphological characteristics
Simple web-app example

11. 11www.euxdat.eu
Enrichment of OLU by geo-
morphological characteristics
Orchestrator

12. 12www.euxdat.eu
Enrichment of OLU by geo-
morphological characteristics
WMS from pregenerated data
 https://mapserver.test.euxdat.eu/cgi-
bin/mapserv?map=/maps/elevation/olu.map

13. 13www.euxdat.eu
Current results
Use Earth Observation (EO) data to
define field boundaries...

14. 14www.euxdat.eu
 Stress detection in olive trees
 Enhance the ability to differentiate between biotic and
abiotic crop stress
 Pathogen identification between the surveyed ones
Objectives
Monitoring of crop status

15. 15www.euxdat.eu
Monitoring of crop status
 The monitoring system will include a crop anomaly
detection component based on specific Sentinel 2 image
analysis algorithms to monitor crop condition and
presence of anomalies as deviation.
 It will then attribute stress based anomalies to disease or
pests by hyperspectral data and other data sources

16. 16www.euxdat.eu
 The service will provide stress maps of the targeted olive
plantations at the tree-polygon level
 Using these maps and the new treatments, farmers and
agronomists will be able to be more efficient and
effective in their detection and treatment of diseases
(Sentinel 2 multispectral indices)
 Identification will be achieved using UAV-enabled
hyperspectral imagery and its correlation to ground truth
Monitoring of crop status

17. 17www.euxdat.eu
Monitoring of crop status

18. 18www.euxdat.eu
 The user creates a polygon (or more) in a provided map (google
earth type). The input polygons will be in geojson format. The
assessment algorithms will be run on the selected polygons.
https://olive.test.euxdat.eu/
 The user is provided a list of dates to choose the imagery from
(imagery with cloud cover above a certain % are excluded from the
list)
 The user selects his dates the algorithms are run in the backend.
 When the results are ready (also in geojson), a table with the
statistics per pixel in each polygon is displayed to the user.
eg:
 Polygon 1:
• 20% Verticillium
• 10% Cycloconium
• 70% No stress
 Polygon 2:
• …
• …
• …
User interface
Monitoring of crop status

19. 19www.euxdat.eu
Example: CRI – Carotenoid Reflectance Index
Monitoring of crop status

20. 20www.euxdat.eu
Agroclimatic zones
 Freezing days
 Factors influencing temperature
• Elevation
• Distance from a water source
• Slope orientation
• Land cover
Influence

21. 21www.euxdat.eu
Agroclimatic zones
 Agro-climatic zones
 Elevation factor

22. 22www.euxdat.eu
Agroclimatic zones
 Agro-climatic zones
 e-Infrastructure components used
• Data connectors
• Meteo data (CDS- ERA5 / meteoblue API)
• EU-DEM
• Processing tools
• GRASS
• HPC aspects
• Parallelization
for different
time slices
• (Parallelization for hydrology effect
~ for each segment of water stream)

23. 23www.euxdat.eu
Agroclimatic zones
 Current status
 the algorithm for temperature adjustments according to
altitudes was changed into a procedure with required
inputs and outputs according to the diagram.
 the procedure for adjusting the temperature according to
the influence of water was fine-tuned.
 Work in progress
 automatic download of temperature data from Copernicus
according to user input (period from, to, step).
 Future plans
 focus on automatic download of DEM according to the
specified area and next part of the diagram.
 Integration to GUI

24. 24www.euxdat.eu
1.1 annual potential evapotranspiration, 50%
prob.
1.2 seasonal potential evapotranspiration,
50% prob.
1.3 seasonal water deficits, 50% probability,
1.4 seasonal water deficits, 10% probability,
100 mm
1.5 seasonal water deficits, 50% probability,
25 mm
1.6 seasonal water deficits, 10% probability
1.7 climatic moisture indices
2.1 average dates of last spring freeze, 0C
2.2 average dates of first fall freeze, 0C
2.3 average freeze-free period, 0C
2.4 dates of last spring freeze, 0C, 10% prob.
2.5 dates of first fall freeze, 0C, 10% probability
2.6 average dates of last spring freeze, -2C
2.7 average dates of first fall freeze, -2C
2.8 average freeze-free period, -2C
2.9 dates of last spring freeze, -2C, 10% prob.
2.10 dates of first fall freeze, -2C, 10% prob.
Agroclimatic zones

25. 25www.euxdat.eu
Looking for climatic patterns changes
 Statistical methods for trend recognition are being
evaluated

26. 26www.euxdat.eu
Information support for field use
recommendations
1. Should a tractor with fertilizer enter the field
2. should a tractor with a crop protection enter
the field
3. Should a harvester enter the field
4. Application map for fertilizer and spraying
All scenarios have identical requirements:
• Understanding of Field capacity of soil
moisture
• A good atmospheric correction of the Sentinel-
2 data
• A good biomass monitoring algorithm

27. 27www.euxdat.eu
Multiple notebooks generated to support
‘Information support’ , ‘Olu’ and other scenarios:
- Automatic reading of field borders
- Reading of field-based statistics for biomass as
well as for clouds

28. 28www.euxdat.eu
Multiple notebooks generated to support
‘Information support’ , ‘Olu’ and other scenarios:
- Investigation of atmospheric corrections
(comparison of 2 methods)

29. 29www.euxdat.eu
Figure 38. Soil moisture time-series for a station in Washington state,
USA demonstrating Field Capacities (FC) (red points). The dry/wet
condition classes will be established as a percentage of FCs for given
time-period.
Detection of field capacity required for:

30. 30www.euxdat.eu
Effective utilization of natural resources
 In process: integration of machinery data from various
sources and manufacturers

31. 31www.euxdat.eu
Determination of crop and growth stage
 A combined approach for crop detection and growth monitoring
would use land use maps for select cropping areas,
 use satellite images to build time series on those areas, classify the
time series into winter and summer crops (by using fallow periods
for separation and validating this with meteorological data – e.g.
precipitation sums to confirm growth or absence thereof),
 separating the crops using specific rules, such as typical periods
(sowing / emergence, booting, flowering, maturation, harvesting
times), colours (e.g. green for winer cereals vs. yellow for oilseed
rape), temperature min/max and sums , soil moisture.
 The resulting classification will be ground truthed with 3 methods:
manually , overlay with statistics, and check by users (customers)."

32. 32www.euxdat.eu
Calculate long time climate and geographic
position effects on the mass changes in Insects
 Moths (Tineidae) are an insect group containing many economic important
pests. Pupae formation and dormancy of this insects is triggered by day
length. Whereas day length is a function of the geographic position. The
development and the speed of development of this moths is depending on
temperature.
 By this proposition and site climate are selecting the part of population
which goes into dormancy and which forms the start population for the
upcoming year.
 Geo-location and the long time weather data in a 3 km grid from MeteoBlue
and Pessl Instruments can be used to calculate for any possible site the day
length dependence of the start population.
 This enables a big improve in the modelling of the mass changes in the
population during the season. The calculations are very intense and have to
be employed on high performance computers. It is needed to simulate the
population of every day in multiple objects describing the behaviour of an
individual insect form egg to adult to retrieve the information if this individual
insect will go the dormancy of will form a pupae. All insects which can not
reach the overwintering 5th larvae stage will not contribute to the population
in the next year."

33. 37www.euxdat.eu
Leveraging the EUXDAT
e-infrastructure
 Agro-climatic zones & climatic patterns example
GDAL/OGR, GRASS, Orfeo Toolbox

34. 38www.euxdat.eu
Leveraging the EUXDAT
e-infrastructure
 Agro-climatic zones & climatic patterns example
GDAL/OGR, GRASS, Orfeo Toolbox

35. 39www.euxdat.eu
Reports
 D2.4 EUXDAT e-Infrastructure Definition v2 - M15 (ATOS ES)
 D2.5 Updated Report on e-Infrastructure Requirements v2 –
M20 (WRLS)
 D2.6 EUXDAT e-Infrastructure Definition v3 - M24 (ATOS ES)

36. Thank you for your attention
jedlicka@wirelessinfo.cz
This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement No 777549
European e-Infrastructure
for Extreme Data Analytics
in Sustainable Development
www.EUXDAT.eu