Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Karel Jedlička - WP2 - Requirements and e-Infractructure Definition

335 views

Published on

EUXDAT project

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Karel Jedlička - WP2 - Requirements and e-Infractructure Definition

  1. 1. WP 2 – Requirements and e-Infrastructure Definition General Assembly (17-18th September, Thessaloniki) Karel Jedlička (WirelessInfo), This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 777549 www.EUXDAT.eu European e-Infrastructure for Extreme Data Analytics in Sustainable Development
  2. 2. 2www.euxdat.eu WP 2 – Requirements and e-Infrastructure Definition 1. Status of the WP2 2. Revision Plan M23-M24
  3. 3. 3www.euxdat.eu WP 2 – Objectives Objective 2.1: Describe the pilots proposed as the validators of the e- Infrastructure developments to be done in the rest of WPs. Objective 2.2: Identify and describe those end users communities that might benefit from the proposed e-Infrastructure, which may have new data management-related needs. Objective 2.3: Gather requirements from the pilots and from other stakeholders, in order to understand the current and future needs for computation and data management. Objective 2.4: Define the main features to be fulfilled by the e-Infrastructure, identifying the key components to be modified and the potential bottlenecks for scaling up to extremely large data analysis.
  4. 4. 4www.euxdat.eu WP 2 – Tasks Task 2.1: Pilots and Future Problems Description (WRLS) Task 2.2: Platform-Driven e-Infrastructure Requirements (WRLS) Task 2.3: High Level Definition of the e-Infrastructure (ATOSES)
  5. 5. 5www.euxdat.eu WP 2 – Deliverables • D2.1 Description of Proposed Pilots and Requirements M4 (WRLS) • D2.2 EUXDAT e-Infrastructure Definition v1 M6 (ATOSES) • D2.3 Updated Report on e-Infrastructure Requirements v1 M12 (WRLS) • D2.4 EUXDAT e-Infrastructure Definition v2 M15 (ATOSES) • D2.5 Updated Report on e-Infrastructure Requirements v2 M20 (WRLS) • D2.6 EUXDAT e-Infrastructure Definition v3 M24 (ATOSES)
  6. 6. 6www.euxdat.eu T2.1 Pilots description T2.2 e-Infrastructure requirements T2.3 e-Infrastructure definition WP 2 – Objectives – Tasks - Deliverables O2.1: describe pilots Agriculture experts group O2.3: pilots & users requirements O2.4: e-Infra- structure features D2.1, .3, .5: (Pilot) and e-Infrastructure requirements O2.2: describe users D2.2, .4, .6: e-Infra definition
  7. 7. 7www.euxdat.eu WP 2 – Progress Objective 2.1: Describe the pilots proposed as the validators of the e- Infrastructure developments to be done in the rest of WPs. Scenarios description • Morphometry characteristics for OpenLandUse • Monitoring of crop status • Agroclimatic zones • Looking for climatic patterns changes • Information support for field use recommendations • Effective utilization of natural resources • Determination of crop and growth stage • Calculate long time climate and geographic position effects on the mass changes in Insects
  8. 8. 8www.euxdat.eu Enrichment of OLU by geo- morphological characteristics Scenario 1: Open Land Use Map improvement • Example for a particular field min_elevation: 186.64203 max_elevation: 196.92177 mean_elevation: 190.70232 median_elevation: 190.44339 min_slope: 0.75737 max_slope: 1.56950 mean_slope: 1.23466 median_slope: 1.25559 min_azimuth: -179.98296 max_azimuth: 179.62314 mean_azimuth: -13.07384 median_azimuth: -30.85966
  9. 9. 9www.euxdat.eu Enrichment of OLU by geo- morphological characteristics Data generation  On the fly  The data on user request is generated onfly on ATOS cloud. The code used for generation on backend is in Python. The API is written in Flask microframework and deployed as container.  Pregenerated:  The data for the whole Europe are generated at once at HPC in Stuttgart and then dumped and sent to ATOS cloud. This approach generates all data at once and allows some analysis (selection by complex criteria) and visualization of the whole database
  10. 10. 10www.euxdat.eu Enrichment of OLU by geo- morphological characteristics Simple web-app example
  11. 11. 11www.euxdat.eu Enrichment of OLU by geo- morphological characteristics Orchestrator
  12. 12. 12www.euxdat.eu Enrichment of OLU by geo- morphological characteristics WMS from pregenerated data  https://mapserver.test.euxdat.eu/cgi- bin/mapserv?map=/maps/elevation/olu.map
  13. 13. 13www.euxdat.eu Current results Use Earth Observation (EO) data to define field boundaries...
  14. 14. 14www.euxdat.eu  Stress detection in olive trees  Enhance the ability to differentiate between biotic and abiotic crop stress  Pathogen identification between the surveyed ones Objectives Monitoring of crop status
  15. 15. 15www.euxdat.eu Monitoring of crop status  The monitoring system will include a crop anomaly detection component based on specific Sentinel 2 image analysis algorithms to monitor crop condition and presence of anomalies as deviation.  It will then attribute stress based anomalies to disease or pests by hyperspectral data and other data sources
  16. 16. 16www.euxdat.eu  The service will provide stress maps of the targeted olive plantations at the tree-polygon level  Using these maps and the new treatments, farmers and agronomists will be able to be more efficient and effective in their detection and treatment of diseases (Sentinel 2 multispectral indices)  Identification will be achieved using UAV-enabled hyperspectral imagery and its correlation to ground truth Monitoring of crop status
  17. 17. 17www.euxdat.eu Monitoring of crop status
  18. 18. 18www.euxdat.eu  The user creates a polygon (or more) in a provided map (google earth type). The input polygons will be in geojson format. The assessment algorithms will be run on the selected polygons. https://olive.test.euxdat.eu/  The user is provided a list of dates to choose the imagery from (imagery with cloud cover above a certain % are excluded from the list)  The user selects his dates the algorithms are run in the backend.  When the results are ready (also in geojson), a table with the statistics per pixel in each polygon is displayed to the user. eg:  Polygon 1: • 20% Verticillium • 10% Cycloconium • 70% No stress  Polygon 2: • … • … • … User interface Monitoring of crop status
  19. 19. 19www.euxdat.eu Example: CRI – Carotenoid Reflectance Index Monitoring of crop status
  20. 20. 20www.euxdat.eu Agroclimatic zones  Freezing days  Factors influencing temperature • Elevation • Distance from a water source • Slope orientation • Land cover Influence
  21. 21. 21www.euxdat.eu Agroclimatic zones  Agro-climatic zones  Elevation factor
  22. 22. 22www.euxdat.eu Agroclimatic zones  Agro-climatic zones  e-Infrastructure components used • Data connectors • Meteo data (CDS- ERA5 / meteoblue API) • EU-DEM • Processing tools • GRASS • HPC aspects • Parallelization for different time slices • (Parallelization for hydrology effect ~ for each segment of water stream)
  23. 23. 23www.euxdat.eu Agroclimatic zones  Current status  the algorithm for temperature adjustments according to altitudes was changed into a procedure with required inputs and outputs according to the diagram.  the procedure for adjusting the temperature according to the influence of water was fine-tuned.  Work in progress  automatic download of temperature data from Copernicus according to user input (period from, to, step).  Future plans  focus on automatic download of DEM according to the specified area and next part of the diagram.  Integration to GUI
  24. 24. 24www.euxdat.eu 1.1 annual potential evapotranspiration, 50% prob. 1.2 seasonal potential evapotranspiration, 50% prob. 1.3 seasonal water deficits, 50% probability, 1.4 seasonal water deficits, 10% probability, 100 mm 1.5 seasonal water deficits, 50% probability, 25 mm 1.6 seasonal water deficits, 10% probability 1.7 climatic moisture indices 2.1 average dates of last spring freeze, 0C 2.2 average dates of first fall freeze, 0C 2.3 average freeze-free period, 0C 2.4 dates of last spring freeze, 0C, 10% prob. 2.5 dates of first fall freeze, 0C, 10% probability 2.6 average dates of last spring freeze, -2C 2.7 average dates of first fall freeze, -2C 2.8 average freeze-free period, -2C 2.9 dates of last spring freeze, -2C, 10% prob. 2.10 dates of first fall freeze, -2C, 10% prob. Agroclimatic zones
  25. 25. 25www.euxdat.eu Looking for climatic patterns changes  Statistical methods for trend recognition are being evaluated
  26. 26. 26www.euxdat.eu Information support for field use recommendations 1. Should a tractor with fertilizer enter the field 2. should a tractor with a crop protection enter the field 3. Should a harvester enter the field 4. Application map for fertilizer and spraying All scenarios have identical requirements: • Understanding of Field capacity of soil moisture • A good atmospheric correction of the Sentinel- 2 data • A good biomass monitoring algorithm
  27. 27. 27www.euxdat.eu Multiple notebooks generated to support ‘Information support’ , ‘Olu’ and other scenarios: - Automatic reading of field borders - Reading of field-based statistics for biomass as well as for clouds
  28. 28. 28www.euxdat.eu Multiple notebooks generated to support ‘Information support’ , ‘Olu’ and other scenarios: - Investigation of atmospheric corrections (comparison of 2 methods)
  29. 29. 29www.euxdat.eu Figure 38. Soil moisture time-series for a station in Washington state, USA demonstrating Field Capacities (FC) (red points). The dry/wet condition classes will be established as a percentage of FCs for given time-period. Detection of field capacity required for:
  30. 30. 30www.euxdat.eu Effective utilization of natural resources  In process: integration of machinery data from various sources and manufacturers
  31. 31. 31www.euxdat.eu Determination of crop and growth stage  A combined approach for crop detection and growth monitoring would use land use maps for select cropping areas,  use satellite images to build time series on those areas, classify the time series into winter and summer crops (by using fallow periods for separation and validating this with meteorological data – e.g. precipitation sums to confirm growth or absence thereof),  separating the crops using specific rules, such as typical periods (sowing / emergence, booting, flowering, maturation, harvesting times), colours (e.g. green for winer cereals vs. yellow for oilseed rape), temperature min/max and sums , soil moisture.  The resulting classification will be ground truthed with 3 methods: manually , overlay with statistics, and check by users (customers)."
  32. 32. 32www.euxdat.eu Calculate long time climate and geographic position effects on the mass changes in Insects  Moths (Tineidae) are an insect group containing many economic important pests. Pupae formation and dormancy of this insects is triggered by day length. Whereas day length is a function of the geographic position. The development and the speed of development of this moths is depending on temperature.  By this proposition and site climate are selecting the part of population which goes into dormancy and which forms the start population for the upcoming year.  Geo-location and the long time weather data in a 3 km grid from MeteoBlue and Pessl Instruments can be used to calculate for any possible site the day length dependence of the start population.  This enables a big improve in the modelling of the mass changes in the population during the season. The calculations are very intense and have to be employed on high performance computers. It is needed to simulate the population of every day in multiple objects describing the behaviour of an individual insect form egg to adult to retrieve the information if this individual insect will go the dormancy of will form a pupae. All insects which can not reach the overwintering 5th larvae stage will not contribute to the population in the next year."
  33. 33. 37www.euxdat.eu Leveraging the EUXDAT e-infrastructure  Agro-climatic zones & climatic patterns example GDAL/OGR, GRASS, Orfeo Toolbox
  34. 34. 38www.euxdat.eu Leveraging the EUXDAT e-infrastructure  Agro-climatic zones & climatic patterns example GDAL/OGR, GRASS, Orfeo Toolbox
  35. 35. 39www.euxdat.eu Reports  D2.4 EUXDAT e-Infrastructure Definition v2 - M15 (ATOS ES)  D2.5 Updated Report on e-Infrastructure Requirements v2 – M20 (WRLS)  D2.6 EUXDAT e-Infrastructure Definition v3 - M24 (ATOS ES)
  36. 36. Thank you for your attention jedlicka@wirelessinfo.cz This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 777549 European e-Infrastructure for Extreme Data Analytics in Sustainable Development www.EUXDAT.eu

×