Talk: "Using Open Data and Crowdsourcing to develop CycleStreets"


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Talk: "Using Open Data and Crowdsourcing to develop CycleStreets"

  1. 1. Using Open Data andCrowdsourcing to developCycleStreets Martin Lucas-Smith @CycleStreets
  2. 2. What does CycleStreets do?Cycle journey planner Photomap / CyclescapeOnline service, 2m journeys so far Campaigning tool
  3. 3. CycleStreets: who?Simon Nuttall Martin Lucas-SmithRoutemaster Webmaster … and various people helping out in various ways!
  4. 4. CycleStreets: history Cambridge-only cycle journey planner Originally written for Cambridge Cycling Campaign Launched June 2006 Google Map –based  5,000 lines drawn over satellite imagery  Google doesn’t give you data: just cartography 47,000 journeys planned 15,000 photos added
  5. 5. CycleStreets: history Lots of requests for same thing in other places around the UK Result is CycleStreets We are using OpenStreetMap for our data  We don’t have money for an OS license  Community aspect really important anyway OpenCycleMap cartography Went to public beta in March 2009  2m journeys so far  Mainly word-of-mouth so far
  6. 6. CycleStreets: UK-wide
  7. 7. CycleStreets Journey planner [Quick demo]
  8. 8. OpenStreetMap
  9. 9. Google (North Cambridge)
  10. 10. OpenStreetMap (North Cambridge)
  11. 11. “OpenStreetMap (OSM) is a collaborative project tocreate a free editable map of the world.” - WikipediaCollaborative:  Jul 2007: 9,000 people; April 2012: Almost 600,000Project:  Not just a map - mass of ideas, processes, data, outputsFree:  Free financially and Free as in openEditable:  Constantly changingOf the world:  Global, not just UK where it started
  12. 12. OpenStreetMap“OpenStreetMap creates and provides freegeographic data such as street maps to anyonewho wants them.“The project was started because most maps youthink of as free actually have legal or technicalrestrictions on their use, holding back peoplefrom using them in creative, productive, orunexpected ways.”
  13. 13. OpenStreetMapUK – Ordnance Survey:Very high quality, but ...Cost can be prohibitive  (particularly voluntary sector)Derivative data restrictions  Ordnance Survey has claimed derived data rights when you place something over one of their maps  Incompatible with direction of the Internet, where data is being ‘mashed’ together to make useful information and visualisationsCentral control – change slower
  14. 14. Crowdsourcing principle“Crowdsourcing is the act of taking a jobtraditionally performed by a designated agent(usually an employee) and outsourcing it to anundefined, generally large group of people in theform of an open call.” knows a little bit about something intheir area. Put that together and you get:
  15. 15. OpenStreetMap OpenStreetMap website default style
  16. 16. OpenStreetMap Cloudmade ‘Fresh’ style (#997)
  17. 17. OpenStreetMap Cloudmade ‘Googley’ style (#5138)
  18. 18. OpenStreetMap OpenCycleMap
  19. 19. OpenStreetMap OpenCycleMap
  20. 20. OpenStreetMap CycleStreets data view
  21. 21. OpenStreetMap CycleStreets data interrogation
  22. 22. OpenStreetMap
  23. 23. OpenStreetMap Glosm 3D (Russia)
  24. 24. OpenStreetMap
  25. 25. OpenStreetMap
  26. 26. OpenStreetMap
  27. 27. OpenStreetMap
  28. 28. OpenStreetMap Urban accessibility of Castelfiorentino
  29. 29. OpenStreetMap Bike Hub app, uses CycleStreets routing
  30. 30. OpenStreetMap First tactile map based on OSM data published on May 12, 2009
  31. 31. OpenStreetMap OpenPisteMap
  32. 32. CycleStreets Journey planner
  33. 33. OpenStreetMap Marikina Mapping Party cake (4th Mapping Party in the Philippines)
  34. 34. Data collection  Structured ground surveys – main source  Ground surveys, performed by a mapper  On foot, bicycle or in a car or boat.  Usually collected using a GPS unit  Government data sources  Landsat 7, US TIGER data, OS OpenData  Commercial data sources  AND from Netherlands  Traced from satellite imagery  e.g. Yahoo!, Microsoft Bing have donated
  35. 35. Objective data  OSM is a store of objective data  Everything must be verifiable  Subjective data is not welcome  Subjective assessment is the realm of the consumer of the data  E.g. Cycle journey planner decides on the likely niceness of a street based on objective attributes like speed limit, width, surface quality  My cycle to work would be different to my mum’s: we have different preferences for a ‘good’ route
  36. 36. OpenStreetMap ITO World animation OSM 2008 - A Year of Edits
  37. 37. Data collection Mapping takes place individually or in groups
  38. 38. Ground surveys Individuals or groups survey using GPS and taking notes Made easier by GPS technology  2000: Bill Clinton switches on wider GPS availability  Mid-2001: GPS units available for $100  2004: GPX standard (GPS data transfer) widespread Photo attribution unknown – please do contact us if you know
  39. 39. Mapping parties A group of openstreetmappers and novices Go to area & map it exhaustively, usually over a weekend Dividing up an area between participants and mapping it Mapping by car, cycle or walking Social aspect important: people can meet up and talk (usually at a pub) between mapping sessions Photo: David Earl Photo attribution unknown – please do contact us if you know
  40. 40. Mapping parties Photo attribution unknown – please do contact us if you know  e.g. Walking Papers: Print current state, annotate, load back in 
  41. 41. Social context Social context important  Community decides on data collection and structure norms appropriate to their situationThe mapkibera project istraining locals people ofKibera, Nairobi to create amap with OpenStreetMapTechnologies used dependon circumstances Map Kibera
  42. 42. Social context Importing other people’s data?  Massive debate within the OpenStreetMap community  (Assumes donated data is compatibly licensed)  One view: importing data gives the impression that an area doesn’t need to be mapped in person and reduces volunteer input  TIGER data import in US very problematical  Another view: importing data gives a massive head-start and means we can get into much more detailed mapping  Data creators vs Data consumers have different perspectives  CycleStreets needs a reasonably complete map!
  43. 43. Social context Is objectivity always possible?  WikiProject Gaza  Practical issues  How do you represent a location where only some people can enter/exit? Photo attribution unknown – please do contact us if you know
  44. 44. Social context How do you represent a location where only some people can enter/exit? Photo attribution unknown – please do contact us if you know
  45. 45. Social context Crisis Mapping: WikiProject Haiti Before January 12, 2010 Then NOAA, GeoEye, DigitalGlobe flew planes over the area, and donated their imagery for tracing purposes  People around the world at their computers contributed to effort Roads, buildings and refugee camps of Port-au-Prince mapped in just two days “The most complete digital map of Haitis roads”
  46. 46. Haiti The resulting data & maps have been used by several organisations providing relief aid, such as the World Bank, the European Commission Joint Research Centre, the Office for the Coordination of Humanitarian Affairs, UNOSAT, others
  47. 47. Data collection
  48. 48. Informal data structure No formal specification of how to represent things  No database schema – just key-value pairs Reflects the social context of the users  Users make it up as they go along  Communities of interest  norms  Conventions established, then stability  User/collector cycle embeds the convention
  49. 49. Informal data structure Nodes & Ways, Tags describes the (many) conventions formed so far Examples  Motorway represented as: “highway=motorway”  Local street: “highway=residential”  Guided bus! “highway=bus_guideway”  Fence: “barrier=fence”  Cycleway: “highway=cycleway”. But what type?  “cycleway=lane”  “cycleway=track”  “cycleway=opposite_lane”  POIs: “amenity=postbox”, “shop=charity”  Not to forget... “amenity=pub”
  50. 50. Adding data Potlatch 2 – (
  51. 51. Adding data Potlatch 2 – (
  52. 52. Potlatch 2 editor [Quick demo]
  53. 53. Adding data JOSM – Java OpenStreetMap Editor – advanced users
  54. 54. Adding dataThe ArcGIS Editor provides:• Simple tools to upload and download OSM data• An OSM-compatible geodatabase schema to locally store OSM data• An OSM symbology template for faster editing• Conflict-resolution tools for reconciling data back to the OSM database ArcGIS plugin for OpenStreetMap (free)
  55. 55. OSM / Google MapsGoogle doesn’t provide any data – just a pictureAlso doesn’t always have information needed by cyclists/walkers –park paths, cut-throughs, pubs! (Though is improving) OSM Google maps
  56. 56. OSM vs Ordnance Survey Depends what scale Question is intended use “Good enough” notion  OSM will never be good enough for utility companies needing exact location of pipes  But for many other uses, OSM appropriate and good enough Sutton Coldfield B72:
  57. 57. OSM vs Ordnance Survey Costs money – not free Big difference is the license – not free (libre) Plot points on a map and the OS claim some rights to that  Derivative data issues  Major problem in the age of the internet, where data is being shared, mixed, repurposed By contrast, OSM uses a Creative Commons license
  58. 58. Challenge to traditional mapping agencies Ordnance Survey seeing more competition OSM and internet sharing more generally forcing a change in business models? Lowering data use costs Lowering data collection costs Forcing derivative data restrictions to be removed Challenge in the small-scale map data area
  59. 59. Opens new opportunities Businesses like Microsoft, Google and others presumably spend a small fortune on mapping data Bing Maps (Microsoft) and MapQuest (AOL) now actively putting money and resources into OSM project  Perhaps speculatively  OSM will provide them with a cheaper way of providing data with far fewer restrictions in future?
  60. 60. Quality assurance issues Can we trust open data?  Depends whether it’s ‘good enough’ for your use Can we trust formalised data?  Tales of lorry satnavs for instance Balance between accuracy and speed/volume  Arbury Park in OSM as it was built – others slower Quality around the country variable  How can we ascertain this? Vandalism  But there’s the ability to watch an area for changes  More people = more vigilance or more vandalism?
  61. 61. Difficulties we face with OSM Coverage not uniform Lack of quality control: makes harder to engage Local Authorities Vandalism a concern for some though not in practice Subjective data? Maybe in future: lack of static IDs – unique numbers for features change Ability to engage local mappers when an area is deficient Many of these problems will go away as OSM matures
  62. 62. Challenge to traditional cartography Cartography is a major area of interest within the OpenStreetMap community Cartography is becoming more automated as Web 2.0 steams ahead  3D vector rendering – send data to device, not bitmaps
  63. 63. Cloudmade map renderer demo [Quick demo] Click ‘Edit map style’ Click on a design to start from Click ‘Clone Style’ in the bottom-right Use the ‘Object Visibility’ box on the right to remove/add features
  64. 64. OpenStreetMap ecosystem At the heart of the OpenStreetMap project is a database holding all the map data that people work with. Left: editors people use to enter data into the database Right: all sorts of interesting uses for the data, e.g. ...
  65. 65. OpenStreetMap uses Non-commercial Commercial / profit-making use absolutely fine  As long as people adhere to the license, i.e. give attribution and allow downstream users to share/re-use the data Maps of very many kinds Web routing SatNav devices Data analysis (e.g. accessibility analysis) Placefinding GPS background Humanitarian ...
  66. 66. OpenStreetMap: Summary Applies the Wikipedia approach of crowd-sourcing Extremely flexible Free (cost) and Free (libre) Challenging traditional map agencies / business models and government funding models Communities of interest and norms  Much scope for research Varied uses: maps, electronic devices, humanitarian, .. CycleStreets using it As more data goes in, more uses, so more people add data, so more people use it, so ...
  67. 67. CycleStreets Cycle journey planner
  68. 68. Journey planner: features Plan route from A-B, anywhere in UK Simple user interface (we hope!)  Click-click-plan, and simple Namefinder Gives set of route choices  (fastest, quietest, balanced) Takes accounts of hills Turn-by-turn directions Photos-en-route
  69. 69. Journey planner: features Distance, time, CO2 avoided, Calories Google Street View at any point Localised versions for easy linking  E.g. Link methods  E.g. ‘Fly in Google Earth’ Export to GPS Feedback system
  70. 70. Photomap
  71. 71. Photomap: features Icons on map (per type of feature)  Click to view image and info Add photo  Crowdsourcing: lots of people, but each donating a small effort Categorisation  E.g. “Show me all the cycle parking problems in Cambridge”
  72. 72. Mobile Key features on small screen iPhone app Android apps Mobile HTML5 app All open source – help welcomed! Jakob Nielsen: “Best Application Designs” - April 2012 (Lightweight Applications category
  73. 73. Mobile Other apps now incorporating our routing API - data interface Bike Hub – great world-first iPhone bike real-SatNav In the leading Boris Bike app, ‘London Cycle’
  74. 74. Why? Fundamentally, we want to see “More people cycling, more safely, more often” New cycle users face many challenges in UK:  Poor infrastructure, traffic hostility  Confidence cycling (address with training)  Cultural/identity issues: not yet mainstream  Lack of utility bikes in shops  Routes – different to car routes! We try to tackle the last problem ... and the first (through the Photomap)
  75. 75. How a routing engine works • Find route with lowest score, i.e. least ‘friction’ • ‘Shortest path algorithm’ - Standard problem in computer science, we use A* method
  76. 76. How it works (briefly) 1. Data comes from people collecting data on-street for OpenStreetMap  Remember: Is factual data only – e.g. presence of road, surface, type  NOT “I think this is a nice cycle route” 2. We take OSM data ‘off the shelf’  Though we’re part of the community in practice  Import several times a week: fresh data  Conversion process is complex – interpreting the data
  77. 77. How it works (briefly)3. Score each type of path:4. Take account of hills (add/remove penalty)5. Account for turn delays (work ongoing)6. Take account of detailed cyclist behaviour (ditto)
  78. 78. How it works (briefly) 7. Compress the network, to make the system much faster (system called ‘Cello’): A A 9 8 4 9: AC 10 7: AD,BD D 3 B B 6 6: BC C C 9 Park: 4 nodes & 7 ways After: 3 nodes & 3 ways
  79. 79. How it works (briefly) So each path / road / shortcut / etc. now has a score  Higher score = worse for cycling (more ‘friction’)User comes to the site 8. Find the lowest total score from A to B 9. Route is found 10. Repeat for quietest, fastest modes – each have different scores 11. Routes shown to user
  80. 80. Draw over the cartography We are using OpenCycleMap by Andy Allan ‘Tiles’ which form a static background once a route has been planned – i.e. we just put this behind a line we have calculated
  81. 81. Route feedback goes to OSM contacts
  82. 82. Getting involved: open sourcing All 3 mobile apps now open-sourced Main journey planner being open-sourced  Latest update at  Codebase currently harder to install than it should be  Currently modularising more heavily  Converting to Git Cyclescape open-source Very keen for greater involvement
  83. 83. Back in January 2011...Transport Direct CJP£2.4 million (from tax) £28k92,000 journeys planned 458,000 journeys planned (dated Jan 2011, total now = ??) (dated Jan 2011, reached 2m as of now)£26.09 per journey 6p per journey£1m – budget for 2011 £130k needed32 areas (professionally surveyed) UK-wide (but depends on OSM completeness)
  84. 84. UKGov... BUT things have now moved on a bitWe’re working with the DfT through their data contractor toget data into OSM – funded projectDfT have been very receptive to the open data potentialWe think cycle journey planning is most effective when doneby local people using Open DataMerging tool createdWe continue to work toensure that CycleStreetsis solution of choice
  85. 85. Big Society –compliantWe tick all the boxes:Collaborative: involves local peopleLow cost: datasets have no license fee,agile deliveryTrusted: for the people, by the peopleOpen DataCitizen involvement: combines skillsand input of large numbers of people(collecting data)Quality delivery: problems can befixed easilyTransparency: more people overseethe data and spot problems or potentialimprovements Cabinet Office
  86. 86. Was main feature on for months!
  87. 87. Local Authorities
  88. 88. UK Collision Map
  89. 89. Cyclescape Campaigning toolkit For campaign groups around UK Match, using geography, who is interested in what Ruby on Rails (new for us!) Really would welcome coders 
  90. 90. David Earl Martin Lucas-Smith, Twitter: @cyclestreets