Presented at the 46th Society of Cartographers Summer School in Manchester on September 10 2010. The abstract for the talk was as follows: "OpenStreetMap is coming of age, but as it starts to be used more in the mainstream, the age-old questions of quality and completeness are coming to the fore. A range of data sources have been used to build up the map in the UK, from GPS traces to aerial imagery, historic mapping, NaPTAN and the OS Open Data release, each with their own benefits and limitations. This talk looks at a number of studies and tools developed to quantify, compare and address accuracy and coverage of the project in the UK, in an attempt to answer the key questions - is it complete yet and just how good is it?"

1. OpenStreetMap The Quality Issue Oliver O’Brien

2. Contents Quality & Completeness Sources of Data Scans of Imagery for Tracing Imports Studies, Tools and Methods Comparison Other Considerations OSM in the Real World

3. Quality & Completeness OpenStreetMap – a crowdsourced spatial database of the world. How good is the data compared with real life? Is everything important on the ground in the database?

4. GPS Traces In the beginning (~2004) Tracks from GPS receivers Consumer-grade handheld devices Prone to random and systematic error Particularly in urban areas (multipath) Particularly if walking Walk both ways down street, take the “average”? In theory quality improves with multiple traces Metadata doesn’t include measured quality eCourier van GPS traces for London

5. GPS Traces

6. GPS Traces

7. GPS Traces

8. Out-of-Copyright Map Tracing Crown Copyright expires after 50 years Not so useful for urban areas Great for rural areas – less likely to change Old, low resolution map better than nothing Started with Richard Fairhurst’s collection in early 2006 Quality dependent on scan resolution and quality of rectification Paper warps, especially over 50 years

9. Out-of-Copyright Map Tracing

10. Yahoo Aerial Imagery Tracing Agreement in late 2006 that allowed tracing In the UK, some large urban areas covered Accuracy better than GPS but not as good as newer imagery sets (e.g. Google) Phantom roads Hidden barriers

11. Yahoo Aerial Imagery Tracing Coverage of many countries Iraq: “The whole road and rail network of the Baghdad area has been mapped using the Yahoo! imagery. As have some parks, forested areas, etc.” Australia (Sydney): “Completion of the greater metropolitan railway network… several complex road interchanges not practical to map on the bike”

12. Yahoo Aerial Imagery Tracing

13. NaPTAN Public transport nodes Once-off copy of DtT database in 2009 Most useful for “less glamorous” bus stops as railway station coverage was already complete in OpenStreetMap Rich metadata Envisaged as two-way, OSM would fix poor locations and might be fed back Imported on a county-by-county basis

14. NaPTAN NOVAM-Viewer Bus stops For reconciling NaPTAN/OSM Graphical tool Manage manual verification of the locations and metadata

15. Walking Papers Designed for large scale POI addition The source is annotations to existing OpenStreetMap data so ultimately dependent on the other sources Specially rendered to print out and add Points of Interest to Automatic georeferencing for tracing If you want it

16. Walking Papers

17. Imports A common theme so far – tracing Inherently poor quality due to user error Tiger (U.S.) AND (Netherlands) OS Open Data

18. Tiger (U.S.) From US Government Imported in late 2007 (data from 2005) Data quality low but good coverage More difficult to create a community to fix rather than map Specialist tools: CloudMade has realised potential of a cleaned-up set and has created web tools to help [animation]

19. AND (Netherlands) July 2007 Relatively straightforward as there was little existing OSM mapping for the Netherlands From a digital mapping data company with navigation and LBS products First “complete” country in OSM “15-20m” offsets in places between Yahoo aerial imagery and the AND data

20. OS OpenData (U.K.) No bulk imports (c.f. Andy Robinson) Community still debating UK coverage already too comprehensive Possible licensing concerns Building outlines possibly the biggest win Some test imports – automated feature detection from Street View rasters Isle of Sheppey

21. OS OpenData (U.K.)

22. Summary of Data Sources (UK)

23. Studies on Quality MukiHaklay (UCL CEGE) “Haklay, M., 2010, How good is volunteered geographical information? A comparative study of OpenStreetMap and Ordnance Survey datasets” Environment and Planning B: Planning and Design 37(4) 682 – 703 “on average within about 6 m of the position recorded by the OS, and with approximately 80% overlap of motorway objects between the two datasets” Analysis in 2008 (Meridian 2)

24. Studies on Quality MukiHaklay and AamerAther Beyond good enough? Spatial Data Quality and OpenStreetMap data OSM vsMasterMap I.T.N., looking at road lengths “The analysis shows that when A-roads, B-roads and a motorway from ITN are compared to OSM data, the overlap can reach values that are over 95%. … OSM is of better quality than Meridian 2.” “Positional accuracy is satisfactory for many applications. Attribute accuracy is also satisfactory.” Spring 2009

25. Studies on Quality “Just how good is it?” As good as many consumer-grade online mapping websites Depends on the care and keenness of the “local champion”

26. Studies on Completeness MukiHaklay Completeness in volunteered geographical information – the evolution of OpenStreetMap coverage (2008-2009) “OpenStreetMap already covers 65% of the area of England, although when details such as street names are taken into consideration, the coverage is closer to 25%. Significantly, this 25% of England’s area covers 45% of its population.” Affluence bias Covers to October 2009

27. Studies on Completeness MukiHaklay: OSM vs Meridian 2 (red) Left: Features only Right: With attributes

28. Studies on Completeness Peter Reed DfT highway length figures per county “Is it complete yet?” Depends on what you need Depends on where you need it Improving!

29. Tools for Completeness ITO World OS Locator

30. Tools for Completeness “Lonely Buses” tool – 20m buffer

31. Getting Completeness Mapping Parties May 2006: “Mapchester” Chris Perkins, Andy Robinson January 2007: Central London Steve Chilton Richard Fairhurst May 2009: Milton Keynes Me, Andy, Steve October 2009: Atlanta

32. Getting Completeness Milton Keynes 2 days 25 people [animation]

33. Getting Completeness

34. Getting Completeness London Mapping Marathon series One hour of mapping followed by pub 2008: Streets 2009: POIs 2010: Building outlines Build and keep the community Crises January 2010: Haiti earthquake High quality aerial imagery

35. Comparison Quality vs Google Maps POIs (Over)simplification of roads Completeness vs Google Maps POIs again Databases vscrowdsourcing Coverage… A few surprises – Pyongyang in OSM

36. Forbidden City in Beijing

37. Other Considerations

38. It Started Life as a Street Map It’s not Ordnance Survey MasterMap Roads are line features, not areal Not yet, anyway OpenStreetMap is just a database Anything can go in it…

39. Individual Trees in Bern!

40. Loose Taxonomy Need to consider the metadata too – is it on the map, is it in the right place, is it tagged appropriately? Community proposes and documents on wiki but doesn’t enforce Worldwide considerations Metadata sometimes present for source, always present for input/update date & user

41. Loose Taxonomy Partial order ensured by Steve Chilton’s OSM Mapnikstylesheet If you want to see your data appear on the map that “most” people see, follow the guidelines

42. Curves ...or lack of... Ability to tag ways as curved Still straight lines in DB

43. OSM in the Real World

44. CycleStreets “We'll be launching a page for Local Authorities v. soon about embedded versions of the CycleStreets planner on their sites” - CycleStreets

45. Skobbler Commercial Sat-Nav application based on OSM data with custom additions Motorway lanes for junction approaches Completeness crucial Display Google Maps on website for address finding (Google Local) but use OSM for routing and app 3D view Accuracy not so important

46. Skobbler

47. Skobbler Started in Germany The map is more complete Just launched in U.K. The map is probably good enough, though missing a lot of turn restrictions Users report bugs Skobbler fix remotely or highlight to OSM Both communities benefit

48. Flickr Beijing Olympics in 2008 Flickr is owned by Yahoo

49. Waze in Chile Recent accidental unattributed inclusion of OSM data from “unreliable source” Fine with attribution

50. Background for Visualisation http://oobrien.com/vis/bikes/ (London) http://osm.mapki.com/bikes/ (Minneapolis)

51. Attribution Various – Imagery of OpenStreetMap data which are CC-By-SA OpenStreetMap and contributors. Various – Imagery of Ordnance Survey OpenData Street View rasterswhich are Crown Copyright and Database Right 2010. p4 – Photograph of GPS unit by Steve Coast. p9 – Imagery of out-of-copyright map by Richard Fairhurst p16 – Photograph of Walking Papers in pub by Harry Wood. p27 – Maps produced by MukiHaklay. p28 – Map produced by Peter Reed. p42 –Screenshot of motorway junctions in OSM by Barry Crabtree. p43 – Photograph of ambulance by DBH NHS media services. p50 –Photograph of Boris Bike by Pocket Lint website. Other imagery may be copyright of the producers referred to on the same page.

52. More http://www.oliverobrien.co.uk/ My research blog @oobr on Twitter o.obrien@ucl.ac.uk