1. Processing conservation
indicators with open
source tools: lessons
learnt from the
Digital Observatory
for Protected Areas
Andrea Mandrici, Luca Battistella, Lucy Bastin
& Gregoire Dubois

2. European Commission (EC)
Executive Body of the European Union (EU)
based in Brussels
Joint Research Centre (JRC)
In-house science service of the EC; provides
independent scientific and technical support to
EU policy
JRC established in 1957
7 institutes in 5 countries with over 3,000 staff
(June 2016)
Restructured into 9 directorates
What is JRC?
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston

3. Earth Observation
Freshwater
Marine
Ecosystem Services
Food Security
Land Degradation
& Desertification
Soil
Forests
Invasive Alien Species
Carbon
Protected Areas
FOSS4G 2017 - Boston
JRC works on…
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston

4. Global reference information system developed by JRC within the Knowledge
Management Unit
Easy access for decision makers and other users to information
at Protected Area (PA), country and ecoregion level
Broad range of indicators on biodiversity values and threats/pressures to facilitate
comparison and prioritization
Supports EC, UN Convention on Biological Diversity (CBD), countries and other users in
planning, monitoring, reporting and/or funding allocation
DOPA is free and open access but respects data restrictions
FOSS4G 2017 - Boston
The Digital Observatory for Protected Areas (DOPA)
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston

5. DOPA Development Flow
GIS Analysis Database Dev REST API Dev Map Services Interface Design Application DevData
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston

6. DOPA REST APIs
REST API
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Services are distributed
through interface developed
in-house (PostgreSQL,
web.py)

7. DOPA Explorer
Application
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
● Performing search out of more than 21,000 protected areas.
● Get summary results in 20 ms for protected areas, countries,
ecoregions and soon regions.
● Dedicated pages for each geographical level

8. DOPA Explorer
Application
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston

9. DOPA Development Flow
GIS Analysis Database Dev REST API Dev Map Services Interface Design Application DevData
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston

10. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Robustness and transparency
peer-reviewed methods to calculate a range of indicators on PA state,
pressures and threats
Saura, S., L. Bastin, L. Battistella, A. Mandrici, G. Dubois (2017). Protected
areas in the world’s ecoregions: how well connected are they? Ecological
Indicators 76: 144-158 http://dx.doi.org/10.1016/j.ecolind.2016.12.047

11. Data sources
Data
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Reliability and openness of datasets
use of openly available datasets with clearly documented provenance
● in-house at JRC
● external

12. Data sources
Data
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Global Surface Water product was analyzed to record area and net of
permanent / seasonal water over 32 years within each PA.
Pekel et al. 2016.
High-resolution mapping of global surface water and its long-term changes. Nature 540, 418–422.
IN-HOUSE at JRC

13. Data sources
Data
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
IN-HOUSE at JRC
To evaluate human pressures on PAs, we use the Global Human Settlement map
Pesaresi et al. 2016.
Operating procedure for the production of the Global Human Settlement Layer from Landsat data of
the epochs 1975, 1990, 2000, and 2014. Publications Office of the European Union, EUR 27741 EN,
2016.

14. Data sources
Data
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
External: Countries
Global Administrative Unit Layer (GAUL)
http://www.fao.org
Exclusive Economic Zones (EEZ)
http://www.marineregions.org

15. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Countries
Merge GAUL+EEZ+ABNJ (POSTGIS, QGIS)

16. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Countries
Holes filling (GRASS, QGIS, ArcMap)

17. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Countries
Dissolve ISO3, JOIN ISO2, UN M49 (PostGIS)

18. Data sources
Data
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
External: Ecoregions
Terrestrial Ecoregion of the world (TEOW)
https://www.worldwildlife.org
Marine Ecoregions Of the World and
Pelagic Provinces Of the World
(MEOW and PPOW)
http://data.unep-wcmc.org

19. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Ecoregions
Merge TEOW,MEOW,PPOW (POSTGIS, QGIS)

20. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Countries+ecoregions ( Base Layer)
Merge GAUL, EEZ, TEOW,MEOW,PPOW (ArcGIS PRO, GRASS)

21. Data sources
Data
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
External: Protected Areas (Base Layer)
World Database on Protected Area (WDPA)
https://www.protectedplanet.net/

22. Data sources
Data
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
External: Protected Areas
World Database on Protected Area (WDPA)
https://www.protectedplanet.net/
● Technical challenges
● Legal restrictions

23. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
(pre)Processing: Protected Areas

24. Data sources
Data
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
External: Land Cover (Thematic Layer)
ESA Climate Change Initiative Land Cover (CCI LC)
https://www.esa-landcover-cci.org/

25. level 1 level 2 level 3 CCI-LC codes
Natural / semi-natural land Tree cover Tree cover, broadleaved, evergreen, closed to open (>15%) 50
Tree cover, broadleaved, deciduous, closed to open (>15%) 60, 61, 62
Tree cover, needleleaved, evergreen, closed to open (>15%) 70, 71, 72
Tree cover, needleleaved, deciduous, closed to open (>15%) 80, 81, 82
Tree cover, mixed leaf type (broadleaved and needleleaved) 90
Shrubland Shrubland 120, 121, 122
Grassland Grassland 130
Lichens and mosses Lichens and mosses 140
Mosaic tree, shrub and
herbaceous cover
Mosaic tree and shrub (>50%) / herbaceous cover (<50%) 100
Mosaic herbaceous cover (>50%) / tree and shrub (<50%) 110
Sparse vegetation Sparse vegetation (tree, shrub, herbaceous cover) (<15%) 150, 151, 152, 153
Bare areas Bare areas 200, 201, 202
Wetland, tree cover Tree cover, flooded, fresh water 160
Tree cover, flooded, saline water or brackish water 170
Wetlands, shrub or
herbaceous cover
Shrub or herbaceous cover, flooded, fresh/saline/brackish water 180
Mosaic natural / managed
land
Mosaic natural vegetation /
cropland
Mosaic natural vegetation (tree, shrub, herbaceous cover) (>50%) / cropland (<50%) 40
Mosaic cropland (>50%) / natural vegetation (tree, shrub, herbaceous cover) (<50%) 30
Cultivated / managed land Cropland Cropland, rainfed 10, 11, 12
Cropland, irrigated or post-flooding 20
Urban areas Urban areas 190
Water / snow and ice Water bodies Water bodies 210
Permanent snow and ice Permanent snow and ice 220
4 14 22 classes
GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Landcover
AGGREGATION LEVELS (GRASS)

26. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Landcover
Reclass, calculation of LCC (GRASS)

27. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: WDPA + Landcover

28. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Landcover
PostGIS splits each PA as VIEW
GRASS imports each VIEW as vector

29. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Landcover
Hadoop buffers 10 Kms
GRASS? PostGIS?

30. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Landcover
GDAL to create flat raster WDPA
PostGIS works for vectors

31. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Landcover
GRASS r.mapcalc to erase PA from buffers

32. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Landcover
STATISTICS: REGION+MASK (GRASS)

33. GIS Analysis
GIS Analysis
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Processing: Landcover
LOOP STATISTICS (GRASS)

34. Database Development
Database Dev
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
From Postgres to REST APIs

35. DOPA REST APIs
REST API Dev
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Services are distributed
through interface developed
in-house (PostgreSQL,
web.py)

36. DOPA Explorer
Application Dev
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston

37. Lesson Learnt
Andrea Mandrici & Luca Battistella, Lucy Bastin & Gregoire Dubois FOSS4G 2017 - Boston
Dedicated processing server Docker containers Avoid using different tools for
processing the same data
Adaptable scripts Code sharingSystematic workflow

38. Many thanks
http://dopa.jrc.ec.europa.eu
@EU_DOPA