AnalysingAnalysing
OpenStreetMap DataOpenStreetMap Data
with QGISwith QGIS
JerryJerry CloughClough
SK53 on OpenStreetMap
@...
My BackgroundMy Background
● Biologist, Computer Scientist, Management Consultant
Naturalist
● GIS--, DB++
– OLAP platform...
OSM Need to KnowOSM Need to Know
● Open Vector Data
● 3 Geo-primitives
– Node (= point)
– Way (= linestring)
● Closed ways...
Some 'Interesting' Stats for GBSome 'Interesting' Stats for GB
(with apologies to Ordnance Survey)
● Pylons: 58,487 (OSGB:...
How I use QGISHow I use QGIS
● OSM data => PostGIS DB
● Initial analysis in QGIS
● PostGIS routines for more complex data
...
Case Study 1 : PubsCase Study 1 : Pubs
Pub Density in Great BritainPub Density in Great Britain
Cartograms based on PubsCartograms based on Pubs
Cartograms based on PubsCartograms based on Pubs
Case Study 2:Case Study 2:
Simulating Urban AtlasSimulating Urban Atlas
● 300+ EU cities population >100k
– 119 in April 2...
Examples of mapping OSM TagsExamples of mapping OSM Tags
to Urban Atlas Categoriesto Urban Atlas Categories
UA
Code
UA Des...
Painter’s Algorithm in QGISPainter’s Algorithm in QGIS
Case Study 3:Case Study 3:
Retail in OSMRetail in OSM
Retail Geo-dataRetail Geo-data

DriversDrivers
–Personal interest
• Used to consult to large retail chains & FMCG firm
–A...
FHRS 1
(local) Government Open Data
• Addresses
• Partial geolocation
– postcode
• Business Type
– Pub/Bar/Nightclub
– Sup...
Tracking my ownTracking my own
OSM MappingOSM Mapping
●
Plot premises by postcode centroid
●
OpenLayers plugin for backgro...
Conclusions Nottingham Retail 2
Conclusions Nottingham Retail 3
Classifying Retail
Areas
Case Study 4 : Street LightsCase Study 4 : Street Lights
Street Lights and OSM QualityStreet Lights and OSM Quality
Street Lights and OSM QualityStreet Lights and OSM Quality
Maps for DogsMaps for Dogs
Approaches to using OSM DataApproaches to using OSM Data
● Direct from OSM (API/ XML
files)
– Earlier Plugin (deprecated)
...
Postgre-SQL/GIS and osm2pgsqlPostgre-SQL/GIS and osm2pgsql
● osm2pgsql converts osm
data to postgres/postgis
– Slightly lo...
ProblemsProblems
● Polygon Handling
● Generalisation
● Missing data
● Free-form Tagging
The Problem with PolygonsThe Problem with Polygons
• No Area primitive in OSM
• Overlapping polygons
• OSM
– Broken polygo...
GeneralisationGeneralisation
• Multiple Ways
– Most objects will be formed
from many OSM ways (e.g,
Thames, M4)
• No simpl...
Tagging IssuesTagging Issues
• Synonymy
– natural=wood
– landuse=forest
• Variable Semantics
– highway=path
– place=hamlet...
Incomplete DataIncomplete Data
Other things I do in QGISOther things I do in QGIS
● Vice County maps using OSGB Open Data
– Plan to investigate Atlas mod...
ConclusionsConclusions
● QGIS fantastic tool for a wide range of manipulations of
OpenStreetMap data
– Particularly well s...
Supplementary SlidesSupplementary Slides
● Managing polygons for detailed analysis (Urban
Atlas)
PostGIS Processing
OSM
Polygons
OSM
Lines
Painter's
Algorithm
Rules
Clipped
Polygons
Clipped
Lines
Cleaned &
Clipped
Polyg...
Comparison 1
No OSM Data
Residential
Disagreement
Agreement
Nottingham Area
Comparison 2
No OSM Data
Residential
Disagreement
Agreement
Agreement
Supplementary SlidesSupplementary Slides
● Examples of OSM Mapping from Port-au-Prince
January 2010
Analysing OpenStreetMap Data with QGIS
Analysing OpenStreetMap Data with QGIS
Analysing OpenStreetMap Data with QGIS
Analysing OpenStreetMap Data with QGIS
Analysing OpenStreetMap Data with QGIS
Analysing OpenStreetMap Data with QGIS
Analysing OpenStreetMap Data with QGIS
Analysing OpenStreetMap Data with QGIS
Analysing OpenStreetMap Data with QGIS
Upcoming SlideShare
Loading in …5
×

Analysing OpenStreetMap Data with QGIS

3,773 views

Published on

Presentation to UK QGIS South East User Group, 2nd April 2014 at Imperial College London

Published in: Data & Analytics, Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,773
On SlideShare
0
From Embeds
0
Number of Embeds
99
Actions
Shares
0
Downloads
110
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Analysing OpenStreetMap Data with QGIS

  1. 1. AnalysingAnalysing OpenStreetMap DataOpenStreetMap Data with QGISwith QGIS JerryJerry CloughClough SK53 on OpenStreetMap @SK53onOSM SK53.osm@gmail.com
  2. 2. My BackgroundMy Background ● Biologist, Computer Scientist, Management Consultant Naturalist ● GIS--, DB++ – OLAP platforms since late 1980s ● OSM since Dec 2008 ● QGIS since Jan 2011 (1.1 => 2.0) ● Mainly analytical uses ● Interests: landuse, landcover, biotopes, local government open data, (pubs)
  3. 3. OSM Need to KnowOSM Need to Know ● Open Vector Data ● 3 Geo-primitives – Node (= point) – Way (= linestring) ● Closed ways may represent areas – Relations ● More complex geothings – Multipolygons – Geo-relations ● NO layers ● Volunteer Sourced – “Wiki map of the world” ● Free Tagging – aka Folksonomy ● Variable Coverage –
  4. 4. Some 'Interesting' Stats for GBSome 'Interesting' Stats for GB (with apologies to Ordnance Survey) ● Pylons: 58,487 (OSGB: 80,517) ● Post Boxes: 42,742 (93.728) ● Camp sites: 3,192 (8,908) ● Buildings: 1,890,835 (35,397,754) ● Bus Stops: 215,720 (354,099) ● Petrol Stations: (7,702) ● Addresses: 27,341,262 (OSGB); 532,886 ● Electricity Poles: 94,199 (183, 987) ● Road length: 522,627 km (407,532 km) ● 5 post boxes with Edward VIII cypher ● Only 110 War Memorials ● 847 Fire Hydrants ● 1,378 Real Ale pubs – 82 with Real Fires ● 4771 Cycle Parking ● 300 Wildlife Hides ● 5,552 Stiles ● 1,774 Canal Locks ● 2 Knitting Shops Ordnance Survey figures: /www.ordnancesurvey.co.uk/blog/2013/04/10-fascinating-facts-from- ordnance-survey/ OSM figures (April '13): /taginfo.openstreetmap.org.uk/
  5. 5. How I use QGISHow I use QGIS ● OSM data => PostGIS DB ● Initial analysis in QGIS ● PostGIS routines for more complex data manipulation ● R and other tools for stats/segmentation ● Visualisation in QGIS
  6. 6. Case Study 1 : PubsCase Study 1 : Pubs
  7. 7. Pub Density in Great BritainPub Density in Great Britain
  8. 8. Cartograms based on PubsCartograms based on Pubs
  9. 9. Cartograms based on PubsCartograms based on Pubs
  10. 10. Case Study 2:Case Study 2: Simulating Urban AtlasSimulating Urban Atlas ● 300+ EU cities population >100k – 119 in April 2010 – 228 in Sept. 2010 ● Baseline date 2006-7 ● Used 2.5 m imagery ● 5-6 year refresh cycle ● Minimum Map Unit (MMU) 0.25 ha urban / 1 ha rural http://sia.eionet.europa.eu/Land Monitoring Core Service/Urban Atlas
  11. 11. Examples of mapping OSM TagsExamples of mapping OSM Tags to Urban Atlas Categoriesto Urban Atlas Categories UA Code UA Description OSM Tags Comments 14100 Parks, Urban Green Space amenity=graveyard landuse=cemetery leisure=park leisure=village_green 14200 Sports Areas landuse=allotments landuse=recreation_ground leisure=golf_course leisure=pitch leisure=stadium 20000 Agricultural Land landuse=farm landuse=farmland landuse=pasture landuse=orchard landuse=vineyard leisure=nature_reserve natural=scrub,natural=heath natural=wetland natural=rock,natural=scree Additional OSM tags are also valid for this code (e.g., natural=glacier) 30000 Woods & Forest natural=wood landuse=forest 50000 Water landuse=reservoir waterway=riverbank natural=water
  12. 12. Painter’s Algorithm in QGISPainter’s Algorithm in QGIS
  13. 13. Case Study 3:Case Study 3: Retail in OSMRetail in OSM
  14. 14. Retail Geo-dataRetail Geo-data  DriversDrivers –Personal interest • Used to consult to large retail chains & FMCG firm –Article in Directions about Geolytix • Featured Nottingham, my main mapping location – Availability of Food Hygiene Open Data  QuestionsQuestions – How difficult was it to systematically get retail landuse and retail sites into OSM? – Was OSM data good enough for segmentation of landuse? Source: Geolytix in Directions Magazine
  15. 15. FHRS 1 (local) Government Open Data • Addresses • Partial geolocation – postcode • Business Type – Pub/Bar/Nightclub – Supermarket – Café/Restaurant – Other Retail • Covers at least 50-60% of retail outlets • Usually current – Typical inspection interval 6-12 months
  16. 16. Tracking my ownTracking my own OSM MappingOSM Mapping ● Plot premises by postcode centroid ● OpenLayers plugin for background ● Track areas visited and added to OSM in Excel Spreadsheet ● S/s linked in as layer ● Update to show places to map ● Push un-surveyed postcodes out as a GPX ● Load GPX on Garmin
  17. 17. Conclusions Nottingham Retail 2
  18. 18. Conclusions Nottingham Retail 3
  19. 19. Classifying Retail Areas
  20. 20. Case Study 4 : Street LightsCase Study 4 : Street Lights
  21. 21. Street Lights and OSM QualityStreet Lights and OSM Quality
  22. 22. Street Lights and OSM QualityStreet Lights and OSM Quality
  23. 23. Maps for DogsMaps for Dogs
  24. 24. Approaches to using OSM DataApproaches to using OSM Data ● Direct from OSM (API/ XML files) – Earlier Plugin (deprecated) – 2.0 method – ogr2ogr ● via Postgres DB – osm2pgsql – osmosis – imposm – osm2postgresql – osm2pgrouting ● via Shapefiles – Geofabrik ● Limited number of layers ● Limited sets of attributes – Roll your own http://wiki.openstreetmap.org/wiki/Osmosis http://wiki.openstreetmap.org/wiki/Osm2postgresql http://sourceforge.net/projects/osm2postgresql/ http://download.geofabrik.de/
  25. 25. Postgre-SQL/GIS and osm2pgsqlPostgre-SQL/GIS and osm2pgsql ● osm2pgsql converts osm data to postgres/postgis – Slightly lossy ● Relationship between members of multipolygons ● Road and other network topologies – Can choose projection ● default 3087 – Can tweak import rules ● Style files ● LUA – Fiddly under Windows ● osmconvert & osmfilter – Very useful tools to preprocess data for particular purposes ● Filter on OSM tag values ● Convert polygons to centroids ● ALWAYS USE -k option – Stores less widely used tags as an hstore column – Maximises flexibility – Throws away coastline by default (sometimes useful to keep it) http://wiki.openstreetmap.org/wiki/Osm2pgsql http://wiki.openstreetmap.org/wiki/Osmconvert http://wiki.openstreetmap.org/wiki/Osmfilter
  26. 26. ProblemsProblems ● Polygon Handling ● Generalisation ● Missing data ● Free-form Tagging
  27. 27. The Problem with PolygonsThe Problem with Polygons • No Area primitive in OSM • Overlapping polygons • OSM – Broken polygons – Intersecting polygons – osm2pgsql • In QGIS – Render OK – Geometry Operations fail • Essential tool: cleangeometry PostGIS function (SOGIS) http://www.sogis1.so.ch/sogis/dl/postgis/cleanGeometry.sql
  28. 28. GeneralisationGeneralisation • Multiple Ways – Most objects will be formed from many OSM ways (e.g, Thames, M4) • No simplified data – Dual carriageways – Roundabouts and flares – Built-up areas – Over noded for many uses • Fine-grain tagging • May require elaborate pre- processing
  29. 29. Tagging IssuesTagging Issues • Synonymy – natural=wood – landuse=forest • Variable Semantics – highway=path – place=hamlet – highway=trunk (gets changed every now & then) • Tagging for the Render – natural=sand for Golf bunker – landuse=grass Everywhere • Semantic Degradation – Tag with accepted semantics being used for something else – landuse=recreation_ground for Ski areas in US • Odd names – shop=mall Shopping Centre
  30. 30. Incomplete DataIncomplete Data
  31. 31. Other things I do in QGISOther things I do in QGIS ● Vice County maps using OSGB Open Data – Plan to investigate Atlas module now ● Distribution Maps of Trees in N. Hemisphere ● Attempts to analyse suburban structure based on building dates – Used Portland Oregon data – Huge Delauney triangulation
  32. 32. ConclusionsConclusions ● QGIS fantastic tool for a wide range of manipulations of OpenStreetMap data – Particularly well suited for ● Prototyping & visualisation ● Combining with other Open Data sources ● Recommend use with PostGIS – Maximises flexibility – Reduces complexity of potential learning curve for the OSM toolchain – Ability to manipulate data in PostGIS may be important ● Be aware of limitations and gotchas of OSM data
  33. 33. Supplementary SlidesSupplementary Slides ● Managing polygons for detailed analysis (Urban Atlas)
  34. 34. PostGIS Processing OSM Polygons OSM Lines Painter's Algorithm Rules Clipped Polygons Clipped Lines Cleaned & Clipped Polygons UA Shape Polygons Clean Geometry Gridded UA Classes Filter on Tags & Grid Gridded & Buffered UA Classes Tag Filter, Grid & Buffer Clip to Area Clip to Area Piecewise Union Union Step 1 Union Union Step 2 Merge Class Gridded Polygons Merge Grid Gridded UA Polygons Union Clipping areas by UA Class ClippingRegion Final Polygons Compare UA/OSM Union/Intersect/ Difference
  35. 35. Comparison 1 No OSM Data Residential Disagreement Agreement Nottingham Area
  36. 36. Comparison 2 No OSM Data Residential Disagreement Agreement
  37. 37. Agreement
  38. 38. Supplementary SlidesSupplementary Slides ● Examples of OSM Mapping from Port-au-Prince January 2010

×