Osm Quality Assessment 2008


Published on

OSM Quality Assessment - presented in S4 event, London, 8th Jan 2008

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Osm Quality Assessment 2008

    1. 1. Understanding the quality of user generated mapping – comparing OpenStreetMap to Ordnance Survey geodata Dr Muki Haklay Department of Civil, Environmental and Geomatic Engineering, UCL [email_address]
    2. 2. Outline <ul><li>User Generated Geographical Information </li></ul><ul><li>Open Street Map – background </li></ul><ul><li>Evaluation and comparison </li></ul><ul><li>The future of User Generated Content Geographical Information ? </li></ul>
    3. 3. User Generated Geographical information <ul><li>Easy to use mapping websites, and wide availability of base mapping </li></ul><ul><li>Capture devices – from GPS receivers to mobile phones with integrated camera and A-GPS </li></ul><ul><li>User Generated Content – Flickr, YouTube </li></ul>
    4. 4. Flickr
    5. 5. Geograph.org.uk <ul><li>7,700 users </li></ul><ul><li>1.05m images </li></ul><ul><li>71.5% coverage </li></ul>
    6. 6. Google Map Maker
    7. 7. OpenStreetMap <ul><li>User generated (Crowdsourced) </li></ul><ul><li>Wiki style </li></ul><ul><li>Open access to data and software </li></ul><ul><li>‘ OpenStreetMap is a project aimed squarely at creating and providing free geographic data such as street maps to anyone who wants them’ </li></ul>(Image source: OpenStreetMap)
    8. 8. OpenStreetMap <ul><li>Started at UCL by Steve Coast, in the summer of 2004, with the aim to create a crowdsourced street map of the world </li></ul><ul><li>Many people joined in to help with the technical infrastructure and collect data. About 40-50 people form the core of the organisation </li></ul>
    9. 9. Creating Maps for OSM (Image source: OpenStreetMap)
    10. 10. OSM technological stack (cc) OpenStreetMap Haklay, M. And Weber, P., 2008, OpenStreetMap – User Generated Street Map, IEEE Pervasive Computing.
    11. 11. Simplified glue – OSM API vs. OGC WMS <ul><li>OpenStreetMap API: http:// api . openstreetmap.org / api /0.5/map?bbox=-71.00,42.00,-72.00,43.00 </li></ul><ul><li>OGC WFS API: </li></ul><ul><li>http://example.com/wfs?service=WFSSIMPLE&version=0.5&REQUEST=GetFeature&BBOX=-71.00,42.00,-72.00,43.00&TIME=2006-09-12/2006-09-22&OUTPUTFORMAT=text/xml </li></ul>Haklay, M. And Weber, P., 2008, OpenStreetMap – User Generated Street Map, IEEE Pervasive Computing.
    12. 12. (Image source: OpenStreetMap)
    13. 13. Mapping parties (cc) Urbanwide - flickr (cc) Nick black (c) Andrea Antonello
    14. 14. Achievements <ul><li>Tens of thousand of participants </li></ul><ul><li>Companies formed to commercialise outputs (CloudMade, Geofabrik) </li></ul><ul><li>Coverage of many places around the world where there is no commercial coverage by Teleatlas or Navteq </li></ul>
    15. 15. The quality issue <ul><li>How good it the data? </li></ul><ul><ul><li>Positional accuracy </li></ul></ul><ul><ul><li>Completeness </li></ul></ul><ul><ul><li>Attribute accuracy and completeness </li></ul></ul><ul><ul><li>Consistency </li></ul></ul><ul><ul><li>Semantic accuracy </li></ul></ul><ul><ul><li>Temporal quality (up-to-date-ness) </li></ul></ul>
    16. 16. The problem <ul><li>We know little about the people that collect it, their skills, knowledge or patterns of data collection </li></ul><ul><li>Loose coordination and no top-down quality assurance processes </li></ul>
    17. 17. Who collects?
    18. 18. Who collects? (c) Dair Grant (cc) Chris Fleming (cc) Shaun McDonald
    19. 19. Working together
    20. 20. Number of Users Area covered (Sq Km) 1 40021 2 20720 3 9136 4 4184 5 1986 6 936 7 448 8 269 9 139 10 and above 246
    21. 21. Users <ul><li>Participation inequality – small group of users collect most of the information, lots of users collect very little </li></ul><ul><li>Little ‘on the ground’ collaboration. Important as this is the main source of quality assurance in open source project - ‘Given enough eyeballs, all bugs are shallow’ (Raymond, 2001) </li></ul>
    22. 22. Accuracy and Completeness <ul><li>Comparing OSM to OS Meridian 2 roads layer </li></ul><ul><li>Maridian 2 -Motorways, major and minor roads are... Complex junctions are collapsed to single nodes and multi-carriageways to single links... some minor roads and cul-de-sacs less than 200m are not represented... Private roads and tracks are not included... </li></ul><ul><li>Nodes are derived from 1:1,250-1:2,500 mapping, with 20m filter around centre line generalisation </li></ul>
    23. 23. Positional Accuracy <ul><li>Meridian 2 and OSM – Motorway comparison </li></ul>A B
    24. 24. Goodchild and Hunter (1997), Hunter (1999) method <ul><li>Assuming that one dataset is of higher quality </li></ul><ul><li>Create buffer around the dataset with known width </li></ul><ul><li>Calculate the percentage of the evaluated dataset that falls within the buffer </li></ul>
    25. 25. Motorway comparison <ul><li>Buffer of 20m </li></ul><ul><li>Average of 80% - ranging from 59.81% to 88.80% </li></ul>Motorway Percentage Overlap M1 87.36% M2 59.81% M3 71.40% M4 84.09% M4 Spur 88.77% M10 64.05% M11 84.38% M20 87.18% M23 88.78% M25 88.80% M26 83.37% M40 72.78% A1(M) 85.70% A308(M) 78.27% A329(M) 72.11% A404 76.65%
    26. 26. Estimating positional accuracy
    27. 27. Positional accuracy <ul><li>On each tile, 100 points sample with evaluation of distance between OSM and Meridian 2 </li></ul><ul><li>Can see significant differences: from about 3m to over 8m </li></ul>Area Average difference (m) Barnet 6.77 Highgate 8.33 New Cross 6.04 South Norwood 3.17 Sutton 4.83 Total 5.83
    28. 28. Completeness – bulk method <ul><li>Assumption: as Meridian 2 is generalised, for each completed sq km: </li></ul><ul><ul><li>Total length(OSM roads)>Total length(Meridian 2 roads) </li></ul></ul><ul><li>Dividing England to 1km grid squares, and running a comparison for each cell </li></ul>
    29. 30. London
    30. 31. Birmingham
    31. 32. Manchester and Liverpool
    32. 33. Length comparison <ul><li>For 29.3% of the area of England, OSM is getting nearer completion and as good as Meridian 2 </li></ul><ul><li>When adding to this attributes, the percentage drops to 24.5% </li></ul><ul><li>Centres of major cities are well mapped </li></ul>
    33. 34. Completeness - visual comparison
    34. 35. Completeness – visual comparison
    35. 36. Completeness
    36. 37. Spatial justice and OSM
    37. 38. Spatial justice and OSM
    38. 39. So should I use OSM? <ul><li>OSM is fit for many purposes to which Meridian is suitable </li></ul><ul><li>Positional accuracy is satisfactory </li></ul><ul><li>Completeness in major urban area is satisfactory – and if the work is at a specific location, it is easy to improve and complete the dataset </li></ul>
    39. 40. Conclusions <ul><li>Impressive coverage by 150 participants with some help of 1000+ others </li></ul><ul><li>Possible to see completion within 3 years </li></ul><ul><li>Open questions: motivation, longevity of engagement, quality and multiple users, comparison with detailed datasets (MasterMap), giving information to participants about completion </li></ul>
    40. 41. Further reading <ul><li>Haklay, M., 2008, How good is OpenStreetMap information? A comparative study of OpenStreetMap and Ordnance Survey datasets for London and the rest of England , submitted to Environment and Planning B. </li></ul><ul><li>Haklay, M. And Weber, P., 2008, OpenStreetMap – User Generated Street Map , IEEE Pervasive Computing. </li></ul><ul><li>Haklay, M., Singleton, A., and Parker, C., 2008, Web mapping 2.0: the Neogeography of the Geoweb , Geography Compass </li></ul><ul><li>Haklay, M., 2008, Open Knowledge – learning from environmental information , presented at the Open Knowledge Conference (OKCon) 2008, London, 15 March. </li></ul><ul><li>Haklay, M., 2007, OSM and the public - what barriers need to be crossed? presented at State of the Map conference, Manchester, UK, 14-15 July. </li></ul><ul><li>To get a copy, write to [email_address] , or get them on povesham.wordpress.com </li></ul>