MapQuest Routing using OSM Data David Nesbitt MapQuest, Inc.
Part of AOL Local Along with Patch and City’s Best First popular dynamic web application #2 in Online Mapping in the U.S. 50 million unique visitors per month Lack of investment in recent years  But that is changing! Recently launched major redesign at http://new.mapquest.com/
Mapquest and OSM MapQuest UK launched at SOTM in Girona  UK site first:  http://open.mapquest.co.uk/ Will launch Germany and France shortly MapQuest France is next New technology stack (for MapQuest) Mapnik, TileCache, Nominatim, OSM data Routing still uses proprietary engine Established $1MM internal fund for OpenStreetMap investment Focus: improve data within the United States Use resources from MapQuest and Patch to engage audiences to contribute Paying developers to contribute to open source projects Contact us at  [email_address]  with questions or comments
 
MapQuest Routing using OSM Data 1) We have ingested OSM data into our routing engine Deploying sites in Europe with OSM routing Localization work as we move to different countries/languages   Route narrative phrases, web page  2) Offer free routing services to developers http:// open.mapquestapi.com /directions / 3) Identify routing issues and develop tools to allow them to be fixed Working with members of the OSM community 4) Work towards using OSM data within US Plan to offer an open.mapquest.com site in the future US data needs improvements to geocoding and for navigability   MapQuest known for accurate directions   We have a large, engaged user base across multiple local properties in the US
MapQuest Routing Engine High performance and scalability (20K+ routes / minute across 12 hosts) A* heuristics, road filtering based on network level, efficient link marking and adjacency list structure Data layout optimized to improve memory / disk cache performance Route Services Additional routing services (developer services) supported Mobile turn-by-turn navigation support http://itunes.apple.com/app/mapquest-4-mobile/id316126557?mt=8 Route Quality Directional link-based algorithm for calculating routes -  simplifies issues with turn restrictions Turn/Intersection costing based on turn geometry and road classes at intersection “ Maneuver” penalties to help generate routes with less turns Link density - slightly favor rural areas vs. urban areas Driving Directions Data Format Format is designed to provide high performance for driving routes Data layout is optimized to improve cache performance  Route Narrative Improvements Tailored route narratives create concise, simple directions New enhanced narrative features like intersection counts, prior maneuvers, and gone-too-far guidance
MapQuest Route Services http://open.mapquestapi.com/directions/  - based on OSM data Multiple input/output formats   JSON, XML, key-value  pairs http://www.mapquestapi.com/directions/ Current developer services based on commercial data Route Matrix Optimized Routes (TSP) Path From Route Computes time/distance of a set of locations from a specific route Trim a set of locations to those that are on or close to a route   Time dependent routing Accounts for time dependent restrictions and real-time/historic speed data Draggable Routes Interactively drag a portion of the route Alternate Routes Provide alternate route paths and narrative
Route Matrix http://www.mapquestapi.com/directions/#matrix Calculates  drive time matrix  (time and distance) One-to-many Many-to-many Many-to-one (shortly) Supports wide range of applications As input to more robust route optimization (Traveling Salesman) method Enhance search results with drive time In-Car navigation: location finding aid Example: find the nearest gas station / convenience store while driving along an Interstate.  Analysis applications Meet in the middle Commuter optimization Drive time of bank employees to various “branches
Optimized Routes http://open.mapquestapi.com/directions/#optimized Traveling Salesman Problem (TSP) Combinatorial optimization problem Brute force approach is not feasible Well-designed heuristic algorithms can find “near-optimal” routes Simulated annealing approaches, Aarts, et al. (1988) MapQuest Optimized Routing Calculates drive time matrix (or pedestrian walking time) From each location to all others Find “near-optimal” route using simulated annealing method Reorder “intermediate” locations to match optimal path Run a “standard” MapQuest route Multi-leg Develops itinerary (narrative and route shape)
Route Data Ingestion osm4routing OpenStreetMap data parser that creates nodes-edges adapted for routing applications   Route DB   PostGRES DB created by osm4route   osm2ddf Reads RouteDB to create MapQuest DDF data (including route network) DDFPostProcess   Validates route network, applies turn costs, optimizes network data structures Driving Directions Format (DDF) Proprietary data format used by MapQuest routing engine osm4routing OSM XML Route DB osm2ddf DDFPostProcess DDF routing
osm4routing Modified version of code from  http://github.com/Tristramg/osm4routing Creates PostGRES Route DB with attributes required by MapQuest routing engine Primarily working with the edges table DB schema - removed some attributes not required and added some new ones Perform some data standardization and normalization Creates a relations/restrictions table Will look to contribute changes back in the future In the process of adding the ability to ingest OSM diff files osm4routing OSM XML Route DB osm2ddf DDFPostProcess DDF routing
osm2ddf Forms route network and optimizes data structure Computes link density for urban avoidance Converts OSM tags into internal attributes Maps highway tags to default speeds and “network level” Identifies and skip duplicate edges Creates name file and other files used for narrative guidance Did some work to “standardize” ref names: e.g., E 45    E45 Uses some country specific logic Computes compass directions of links (and begin/end of links) Allows rapid analysis of turns/intersections during guidance /narrative generation osm4routing OSM XML Route DB osm2ddf DDFPostProcess DDF routing
Next Steps Incremental / automated data update Start at weekly route data updates – move to daily Plan to build some route data validation tools / quality assurance checks Be able to reject and then fix major route issues  e.g., a broken connection along a major Interstate Willing to tolerate errors in local regions but not incorrect edits on major highways Develop tools to improve OSM data quality for routing Data validation and correction tools Working with the OSM community and contracting developers Tune routing engine Optimize performance and quality when using OSM data Pedestrian Routing Area landmark enhancements Traffic Message Channel Investigating ways to associate real time traffic information to OSM data
Pedestrian Routing Enhancements Area features to provide details since walkways and paths are often un-named http://www.mapquestapi.com/directions/v1/route?key=Dmjtd|lu612007nq%2C20%3Do5-50zah&callback=renderAdvancedNarrative&avoidTimedConditions=false&outFormat=json&routeType=pedestrian&narrativeType=microformat&enhancedNarrative=true&locale=en_BG&unit=m&from=1717%20Champa%20St,Denver,CO&to=100%20W%2014th%20Avenue%20Pkwy,Denver,CO
OSM data improvements for routing TIGER data import quality Driveways old/abandoned roads Duplicate edges / ways  Incorrect oneways Exit information (e.g. motorway exits) Lack of consistently applied data standards Model after what is recommended for Germany   Exit number, road name to which the exit connects, the city/town that the exit services   exit_to? Relations / Restrictions Often see intermediate roads being improperly ordered / connected Time dependent relations – may be more important in US than Europe?
OSM data improvements for routing (cont.) Bad connectivity, disjoint network areas Connection to Ferries Many are highway = service   http://www.openstreetmap.org/browse/way/4838402    Meant for alleyways, driveways, deliveries?   Disrupts road filtering / performance “ Isolated" ways that are drivable but not connected to any drivable links e.g., a drivable service road amongst pedestrian ways   City center search may find this as the closest road, driving routes fail Surface tags We care about paved vs. unpaved for vehicle access Too many free-form tags: e.g., surface = very horrible
Bad Connectivity Interstate connection without a ramp
Incorrect oneway http://www.openstreetmap.org/edit?lat=40.066952&lon=-76.287875&zoom=18 Thanks to Robert Chell @ 1Spatial Reversed !
Nice map! Route? http://www.openstreetmap.org/?minlon=-123.1313941&minlat=49.0071713&maxlon=-123.1313941&maxlat=49.0071713 Ferry near Calgary 16 edges!
Questions? David Nesbitt [email_address] 717-581-4874

SoTM US Routing

  • 1.
    MapQuest Routing usingOSM Data David Nesbitt MapQuest, Inc.
  • 2.
    Part of AOLLocal Along with Patch and City’s Best First popular dynamic web application #2 in Online Mapping in the U.S. 50 million unique visitors per month Lack of investment in recent years But that is changing! Recently launched major redesign at http://new.mapquest.com/
  • 3.
    Mapquest and OSMMapQuest UK launched at SOTM in Girona UK site first: http://open.mapquest.co.uk/ Will launch Germany and France shortly MapQuest France is next New technology stack (for MapQuest) Mapnik, TileCache, Nominatim, OSM data Routing still uses proprietary engine Established $1MM internal fund for OpenStreetMap investment Focus: improve data within the United States Use resources from MapQuest and Patch to engage audiences to contribute Paying developers to contribute to open source projects Contact us at [email_address] with questions or comments
  • 4.
  • 5.
    MapQuest Routing usingOSM Data 1) We have ingested OSM data into our routing engine Deploying sites in Europe with OSM routing Localization work as we move to different countries/languages Route narrative phrases, web page 2) Offer free routing services to developers http:// open.mapquestapi.com /directions / 3) Identify routing issues and develop tools to allow them to be fixed Working with members of the OSM community 4) Work towards using OSM data within US Plan to offer an open.mapquest.com site in the future US data needs improvements to geocoding and for navigability MapQuest known for accurate directions We have a large, engaged user base across multiple local properties in the US
  • 6.
    MapQuest Routing EngineHigh performance and scalability (20K+ routes / minute across 12 hosts) A* heuristics, road filtering based on network level, efficient link marking and adjacency list structure Data layout optimized to improve memory / disk cache performance Route Services Additional routing services (developer services) supported Mobile turn-by-turn navigation support http://itunes.apple.com/app/mapquest-4-mobile/id316126557?mt=8 Route Quality Directional link-based algorithm for calculating routes - simplifies issues with turn restrictions Turn/Intersection costing based on turn geometry and road classes at intersection “ Maneuver” penalties to help generate routes with less turns Link density - slightly favor rural areas vs. urban areas Driving Directions Data Format Format is designed to provide high performance for driving routes Data layout is optimized to improve cache performance Route Narrative Improvements Tailored route narratives create concise, simple directions New enhanced narrative features like intersection counts, prior maneuvers, and gone-too-far guidance
  • 7.
    MapQuest Route Serviceshttp://open.mapquestapi.com/directions/ - based on OSM data Multiple input/output formats JSON, XML, key-value pairs http://www.mapquestapi.com/directions/ Current developer services based on commercial data Route Matrix Optimized Routes (TSP) Path From Route Computes time/distance of a set of locations from a specific route Trim a set of locations to those that are on or close to a route Time dependent routing Accounts for time dependent restrictions and real-time/historic speed data Draggable Routes Interactively drag a portion of the route Alternate Routes Provide alternate route paths and narrative
  • 8.
    Route Matrix http://www.mapquestapi.com/directions/#matrixCalculates drive time matrix (time and distance) One-to-many Many-to-many Many-to-one (shortly) Supports wide range of applications As input to more robust route optimization (Traveling Salesman) method Enhance search results with drive time In-Car navigation: location finding aid Example: find the nearest gas station / convenience store while driving along an Interstate. Analysis applications Meet in the middle Commuter optimization Drive time of bank employees to various “branches
  • 9.
    Optimized Routes http://open.mapquestapi.com/directions/#optimizedTraveling Salesman Problem (TSP) Combinatorial optimization problem Brute force approach is not feasible Well-designed heuristic algorithms can find “near-optimal” routes Simulated annealing approaches, Aarts, et al. (1988) MapQuest Optimized Routing Calculates drive time matrix (or pedestrian walking time) From each location to all others Find “near-optimal” route using simulated annealing method Reorder “intermediate” locations to match optimal path Run a “standard” MapQuest route Multi-leg Develops itinerary (narrative and route shape)
  • 10.
    Route Data Ingestionosm4routing OpenStreetMap data parser that creates nodes-edges adapted for routing applications Route DB PostGRES DB created by osm4route osm2ddf Reads RouteDB to create MapQuest DDF data (including route network) DDFPostProcess Validates route network, applies turn costs, optimizes network data structures Driving Directions Format (DDF) Proprietary data format used by MapQuest routing engine osm4routing OSM XML Route DB osm2ddf DDFPostProcess DDF routing
  • 11.
    osm4routing Modified versionof code from http://github.com/Tristramg/osm4routing Creates PostGRES Route DB with attributes required by MapQuest routing engine Primarily working with the edges table DB schema - removed some attributes not required and added some new ones Perform some data standardization and normalization Creates a relations/restrictions table Will look to contribute changes back in the future In the process of adding the ability to ingest OSM diff files osm4routing OSM XML Route DB osm2ddf DDFPostProcess DDF routing
  • 12.
    osm2ddf Forms routenetwork and optimizes data structure Computes link density for urban avoidance Converts OSM tags into internal attributes Maps highway tags to default speeds and “network level” Identifies and skip duplicate edges Creates name file and other files used for narrative guidance Did some work to “standardize” ref names: e.g., E 45  E45 Uses some country specific logic Computes compass directions of links (and begin/end of links) Allows rapid analysis of turns/intersections during guidance /narrative generation osm4routing OSM XML Route DB osm2ddf DDFPostProcess DDF routing
  • 13.
    Next Steps Incremental/ automated data update Start at weekly route data updates – move to daily Plan to build some route data validation tools / quality assurance checks Be able to reject and then fix major route issues e.g., a broken connection along a major Interstate Willing to tolerate errors in local regions but not incorrect edits on major highways Develop tools to improve OSM data quality for routing Data validation and correction tools Working with the OSM community and contracting developers Tune routing engine Optimize performance and quality when using OSM data Pedestrian Routing Area landmark enhancements Traffic Message Channel Investigating ways to associate real time traffic information to OSM data
  • 14.
    Pedestrian Routing EnhancementsArea features to provide details since walkways and paths are often un-named http://www.mapquestapi.com/directions/v1/route?key=Dmjtd|lu612007nq%2C20%3Do5-50zah&callback=renderAdvancedNarrative&avoidTimedConditions=false&outFormat=json&routeType=pedestrian&narrativeType=microformat&enhancedNarrative=true&locale=en_BG&unit=m&from=1717%20Champa%20St,Denver,CO&to=100%20W%2014th%20Avenue%20Pkwy,Denver,CO
  • 15.
    OSM data improvementsfor routing TIGER data import quality Driveways old/abandoned roads Duplicate edges / ways Incorrect oneways Exit information (e.g. motorway exits) Lack of consistently applied data standards Model after what is recommended for Germany Exit number, road name to which the exit connects, the city/town that the exit services exit_to? Relations / Restrictions Often see intermediate roads being improperly ordered / connected Time dependent relations – may be more important in US than Europe?
  • 16.
    OSM data improvementsfor routing (cont.) Bad connectivity, disjoint network areas Connection to Ferries Many are highway = service http://www.openstreetmap.org/browse/way/4838402 Meant for alleyways, driveways, deliveries? Disrupts road filtering / performance “ Isolated" ways that are drivable but not connected to any drivable links e.g., a drivable service road amongst pedestrian ways City center search may find this as the closest road, driving routes fail Surface tags We care about paved vs. unpaved for vehicle access Too many free-form tags: e.g., surface = very horrible
  • 17.
    Bad Connectivity Interstateconnection without a ramp
  • 18.
  • 19.
    Nice map! Route?http://www.openstreetmap.org/?minlon=-123.1313941&minlat=49.0071713&maxlon=-123.1313941&maxlat=49.0071713 Ferry near Calgary 16 edges!
  • 20.
    Questions? David Nesbitt[email_address] 717-581-4874

Editor's Notes

  • #4 Big day today… Developers of this project here at SotM – great talent, know mapping at scale, please talk to them
  • #5 A look at the recent redesign Single-page interface, one-box search
  • #6 50 million users
  • #7 50 million users