Beyond good enough? Spatial Data Quality and OpenStreetMap data

Beyond good enough? Spatial Data Quality and OpenStreetMap data
Dr Muki Haklay
Department of Civil, Environmental and Geomatic Engineering, UCL
m.haklay@ucl.ac.uk
With contributions from AamerAther (M.Eng 2009) and NaureenZulfiqar (M.Eng 2008)
Ordnance Survey data was kindly provided by the Ordnance Survey research unit.
OSM data was provided by GeoFabrik & CloudMade

Outline
Understanding quality of geographical information
Evaluation of OSM with Meridian data set
Evaluation of OSM with MasterMap
What does it all means?

The quality issue
How good it the data?
First question: good for what? Subjective quality – fitness for purpose/use
Second question: how to measure? Objective quality – but need to evaluate it in light of the first question

The quality issue
How good it the data?
Positional accuracy – the position of features or geographic objects in either two or three dimensions
Temporal accuracy – how up to date is the data? Does it presents the existing situation and when will it be updated?
Thematic/attribute accuracy – for quantitative attributes (width) and qualitative attributes (geographic names)
Completeness – The presence and absence of objects in a dataset at a particular point in time
Logical consistency –adherence to the logical rules of the data structure, attribution and relationships

The ‘problem’
We know little about the people that collect it, their skills, knowledge or patterns of data collection
Loose coordination and no top-down quality assurance processes – can’t produce good data
It is not complete and comprehensive – there are white areas

Who collects?

Who collects?
(c) Dair Grant
(cc) Shaun McDonald
(cc) Chris Fleming

Working together

Users
Participation inequality – small group of users collect most of the information, lots of users collect very little
Little ‘on the ground’ collaboration. Important as this is can be the main source of quality assurance - ‘Given enough eyeballs, all bugs are shallow’ (Raymond, 2001)
But does Linus’ law apply to OSM?!?

Accuracy and Completeness- Study I
Comparing OSM to OS Meridian 2 roads layer
Maridian 2 -Motorways, major and minor roads are... Complex junctions are collapsed to single nodes and multi-carriageways to single links... some minor roads and cul-de-sacs less than 200m are not represented... Private roads and tracks are not included...
Nodes are derived from 1:1,250-1:2,500 mapping, with 20m filter around centre line generalisation

Positional Accuracy
A
B
Meridian 2 and OSM – Motorway comparison

Goodchild and Hunter (1997), Hunter (1999) method
Assuming that one dataset is of higher quality
Create buffer around the dataset with known width
Calculate the percentage of the evaluated dataset that falls within the buffer

Motorway comparison
Buffer of 20m
Average of 80% - ranging from 59.81% to 88.80%

Estimating positional accuracy

Positional accuracy
On each tile, 100 points sample with evaluation of distance between OSM and Meridian 2
Can see significant variability: from about 3m to over 8m

Completeness – bulk method
Assumption: as Meridian 2 is generalised, for each completed sq km:
Total length(OSM roads)>Total length(Meridian 2 roads)
Dividing England to 1km grid squares, and running a comparison for each cell

London

Birmingham

Manchester and Liverpool

Length comparison
For 29.3% of the area of England, OSM is getting nearer completion and as good as Meridian 2 (March 2008). Estimated at %45-50 today.
When adding to this attributes, the percentage drops to 24.5% (March 2008). Estimated %35 today.
Centres of major cities are well mapped.

Completeness - visual comparison

Completeness – visual comparison

Completeness – difference by user?

Comparison II – Ordnance Survey Master Map
Data used for comparison: OS MasterMap Integrated Transport Network (ITN) layer
ITN consists of road network information
The most accurate and up-to-date geographic reference for Great Britain’s road structure
Any major real world changes are updated within 6 months
Used for numerous applications
e.g. Transport management systems, road routing, emergency planning...

Four test locations chosen:
TQ28se
TQ38se
TQ17ne
TQ37sw

Buffer analysis – again based on Goodchild and Hunter (1997) buffer comparison technique:
Buffer width (X):
X
ITN
OSM
Comparison methodology

Buffer overlap results:
109 roads examined covering over 328 km
Results of Master Map comparison

TQ38se (East London)
TQ28se (North/Central London)

TQ37sw (South London)
TQ17ne (West London)

Quality not linked to length

What does it mean?
OSM is better than Meridian 2 in terms of positional accuracy, and less accurate than MasterMap
The differences that were found in comparison I are a mix of the positional inaccuracies of both Meridian 2 and OSM. The higher overlap with MasterMap tells us that OSM was the more accurate of the two...

However ...

What are they paying for?
Meridian is officially not complete, clearly not accurate in terms of position, and without clear ‘6 month of major changes update’ rule
Hypothesis:
When people buy geodata, they pay for the errors, or the notion that the errors are well known and quantified.
Are they?

Putting a price tag on OSM ?
1 seat of Meridian 2 for England - £1272
OSM is 35% complete (positional and attribute) ...
... But higher positional accuracy than Meridian 2
So maybe £500 per seat?
If so, each Sq Km of OSM is worth about 40p or 0.5€ .

Linus’ law and OSM – inconclusive

So should I use OSM?
OSM is fit for many purposes to which Meridian 2 is suitable
Positional accuracy is satisfactory for many applications. Attribute accuracy is also satisfactory.
Completeness in major urban area is satisfactory – and if the work is at a specific location, it is easy to improve and complete the dataset

Conclusions
OSM quality is beyond good enough, it is a product that can be used for a wide range of activities
Better quality proxies, can be developed (for example, by user)
Quality procedures should also developed with passive sensing from mobile devices
More work is required on Linus’ Law

Further reading
Haklay, M., 2008, How good is OpenStreetMap information? A comparative study of OpenStreetMap and Ordnance Survey datasets for London and the rest of England, submitted to Environment and Planning B.
Haklay, M. And Weber, P., 2008, OpenStreetMap – User Generated Street Map, IEEE Pervasive Computing.
Haklay, M., Singleton, A., and Parker, C., 2008, Web mapping 2.0: the Neogeography of the Geoweb, Geography Compass
Haklay, M., 2008, Open Knowledge – learning from environmental information, presented at the Open Knowledge Conference (OKCon) 2008, London, 15 March.
Haklay, M., 2007, OSM and the public - what barriers need to be crossed?presented at State of the Map conference, Manchester, UK, 14-15 July.
To get a copy, write to m.haklay@ucl.ac.uk , or get them on povesham.wordpress.com

http://povesham.wordpress.com	617
http://adeshogues.wordpress.com	70
http://www.slideshare.net	31
http://geothink.textcube.com	1

Beyond good enough? Spatial Data Quality and OpenStreetMap data

by mukih on Jul 11, 2009

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 719

Statistics

Beyond good enough? Spatial Data Quality and OpenStreetMap data Presentation Transcript