A showcase of different access methods of Openstreetmap (OSM) data. Considering the business requirements in terms of area of interest and specific OSM objects, three access methods are highlighted: the built-in OSM Reader; binary OSM-files and OSM OverPass API. The presentation and comparison of the different OSM access methods is illustrated with three different use cases: - Data extraction of pharmacies based on OSM-tags - Data extraction to build a customized base map - An FME Server App acting as a self-serving data download tool to extract OSM data on a worldwide scale.

1. Integrate Openstreetmap data in your
organization

2. Presenter
The
Peak
of
Data
Integration
20
23
Dries Verdoodt
GIS analyst

3. The
Peak
of
Data
Integration
20
23
Agenda
Introduction
OpenStreetMap access methods
3 organisations
Objectives
User requirements
Technical implementation
Method1: Binary files
Method2: osm2pgsql
Method3: OSM Overpass API
Conclusion

4. The
Peak
of
Data
Integration
20
23
Introduction

5. The
Peak
of
Data
Integration
20
23
Define the appropriate method
to access OSM data

6. The
Peak
of
Data
Integration
20
23
OpenStreetMap access methods
Files
Manual download OSM or OSM Binary files
www.openstreetmap.org; www.geofabrik.de
API
Official API; Overpass API
Applications
Example QGIS plugin: QuickOSM
Command line tools
osmium; osm2pgsql

7. The
Peak
of
Data
Integration
20
23
3 organisations and their objectives
Swiss Humanitarian Aid (SHA)
Datasets to plan humanitarian interventions
Infrabel: Belgian railway infrastructure operator
Build a customised background map
Police of the Canton Vaud (CH)
Dataset of all the pharmacies

8. The
Peak
of
Data
Integration
20
23
Organisation Objective
User requirements
OSM access
method
Area of
interest
OSM
elements
Update
frequency
Acceptable
processing
time
Swiss
Humanitarian
Aid
Datasets to
plan
humanitarian
interventions
Worldwide Specific
elements:
schools,
healthcare
On-demand Hour Binary files
Belgian
Railway
Infrastructure
Operator
Build a
customised
background
map
National All
elements
Trimester Working day osm2pgsql
Police Canton
Vaud
Datasets of all
the
pharmacies
Regional /
Local
Specific
key-value
tag
On-demand Instant Overpass
API

9. The
Peak
of
Data
Integration
20
23
Technical implementation

10. The
Peak
of
Data
Integration
20
23
Introduction to OSM
OpenStreetMap
Community-driven map of the world
Based on local knowledge
Free to use under an open Licence
Elements’ geometry
nodes, ways, areas or relationships
Elements information
key-value tags
Example: conference hall in Bonn

11. The
Peak
of
Data
Integration
20
23
Method 1: OSM Binary files
Organisation Swiss Humanitarian Aid
Objective Download data for humanitarian
interventions
User requirements
Area of interest large areas, worldwide scale
OSM elements specific elements
Update frequency on-demand
Processing time hourly
OSM access method Download OSM binary files and
read specific OSM Map Features
http://download.geofabrik.de/europe/germany/
nordrhein-westfalen/

12. The
Peak
of
Data
Integration
20
23
FME Flow Hosted: Workspace App
Construct the extent of all the
OSM binary files worldwide
Download OSM binary file from the Geofabrik server
Select corresponding binary file, based on overlap with the AOI
Filter duplicate OSM elements and clip with the AOI
FTPServer
FME Form
Method 1: OSM Binary files
Save binary file in FME Flow resource folder
Write OSM Map Features dynamically
Read binary file

13. The
Peak
of
Data
Integration
20
23
Method 2: osm2pgsql
Organisation Infrabel
Objective Download the complete OSM dataset of
Belgium to build a customised background
map
User requirements
Area of interest national
OSM elements all elements
Update frequency trimester
Processing time working day
OSM access method osm2pgsql

14. The
Peak
of
Data
Integration
20
23
Method 2: osm2pgsql
osm2pgsql -d brussels -O flex -S streets.lua brussels.osm.pbf
Download OSM binary file of Belgium
Import data in PostGIS DB via the cmd-line tool osm2pgsql
only import, no update
preconfigured database tables
hstore datatype to store key-value tags
Postprocessing of the data
CRS reprojection
Drop and rebuild indexes
Copy data between tables
Refresh custom materialized views
Create DB tables
and custom
materialized views
PostGIS FME Form

15. The
Peak
of
Data
Integration
20
23
Method 3: OSM Overpass API
Organisation Police of the Canton Vaud
Objective Download a dataset of
Pharmacies in Canton Vaud (CH)
User requirements
Area of interest regional/local
OSM elements specific key-value tag
Update frequency on-demand
Processing time instantly
OSM access method OSM Overpass API
Example: https://overpass-turbo.eu/

16. The
Peak
of
Data
Integration
20
23
Method 3: OSM Overpass
FME Form
Split the AOI in tiles
Build a filter in the Query Language of the Overpass API request
Launch the HTTP Get request to download OSM files
Read OSM files
Write the results dynamically

17. The
Peak
of
Data
Integration
20
23
Organisation Objective
User requirements
OSM access
method
Area of
interest
OSM
elements
Update
frequency
Acceptable
processing
time
Swiss
Humanitarian
Aid
Datasets to
plan
humanitarian
interventions
Worldwide Specific
elements:
schools,
healthcare
On-demand Hour Binary files
Belgian
Railway
Infrastructure
Operator
Build a
customised
background
map
National All
elements
Trimester Working day osm2pgsql
Police Canton
Vaud
Datasets of all
the
pharmacies
Regional /
Local
Specific
key-value
tag
On-demand Instant Overpass
API

18. The
Peak
of
Data
Integration
20
23
Conclusion

19. The
Peak
of
Data
Integration
20
23
Choose the appropriate method
OSM (binary) file
Available on national / regional scale
Manual or automated download
Command line tool
Process OSM files: clipping, conversion, importation
API
Small Area of Interest
Up-to-date data
Easy implementation

20. The
Peak
of
Data
Integration
20
23
OSM data pitfalls
Incomplete information: missing tags
Elements represented by different geometries
Spatial filter in Readers is not supported
Warning - reader format `OSMPBF' does not support
querying by spatial predicate or envelope queries.
Automate the manual steps

21. The
Peak
of
Data
Integration
20
23
Enrich your applications with
this worldwide inexhaustible
dataset

22. Enrich your applications
Obtain new datasets
Replace or complete existing datasets
Involve the community

23. The
Peak
of
Data
Integration
20
23
Resources
Organisations:
Swiss Humanitarian Aid: SHA
Belgian Railway Operator: Infrabel
Openstreetmap:
OpenStreetMap: https://www.openstreetmap.org/about
osm2pgsql: https://osm2pgsql.org/doc/manual.html
Overpass Turbo: https://overpass-turbo.eu/
Overpass API: https://lz4.overpass-api.de/api/interpreter
GeoFabrik: http://download.geofabrik.de/

24. Try it!
drv@inser.ch
info@inser.ch