Your SlideShare is downloading. ×
Maps2Graphs: A Socially Scalable Method for Generating High-Quality GIS Datasets Based on Google Maps API
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Maps2Graphs: A Socially Scalable Method for Generating High-Quality GIS Datasets Based on Google Maps API

97
views

Published on

GIS -- where Esri is probably the most prominent data provider -- is related to ITS by way of providing ITS research with road maps. Traditionally, GIS data come as .shp files in 2d- or 3dmesh …

GIS -- where Esri is probably the most prominent data provider -- is related to ITS by way of providing ITS research with road maps. Traditionally, GIS data come as .shp files in 2d- or 3dmesh format. The sad news is that (1) even 3dmesh is low resolution, and (2) native GIS roads maps have no context. This study presents a web application which extracts pairwise routes via Google Maps API and stores them in cloud storage (Google Drive, Dropbox, etc.). When done, pairwise routes are merged into the graph ready to use for ITS simulations. The webapp is socially scalable and can use any number of clients, which is important for large datasets.

Published in: Technology, Business

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
97
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. . . Problem Statement . Big Problem is... . ... that it is hard to come by simulations realistic road maps for large-scale ITS . . . Small Problem is... . ... that even with Google Maps, quotas make it difficult to create large datasets alone -- need social scaling (Google Maps quota: 2.5k reqs/day) . • EV battery replacement infrastructure simulations 03 • • supply chain research -- again, battery replacement infrastructures autodrive simulation? -- in real life by around 2020? • ... anyway who needs realistic road maps 03 myself+1 "...EV Use in Environments with V2H and Battery Replacement..." ITS Tokyo (2013) M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 2 /22 2/22
  • 2. . . Traditional GIS M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 3 /22 3/22
  • 3. . . GIS: The Shapefile • default in GIS datasets today • a non-GPS = special format -- see ArcGIS manual 11 • precision problems -- .SHP is all about relative shapes, not coordinates 11 Esri "ArcGIS Japan" http://www.esrij.com/products/arcgis (2013) M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 4 /22 4/22
  • 4. . . ArcGIS versus Google Maps • ArcGIS is ◦ paid ◦ needs datasets to work ◦ address2location and placename2location are restricted by quotas (credit) ◦ • no APIs -- just plugins for PowerPoint and smartphone apps GoogleMaps is ◦ free with quotas, pay 9$/m for nearly infinite quotas ◦ all data is in one place ◦ many APIs -- Location, Direction, Renderer, ... ◦ address2location and placename2location are seamless -- no need to care which you use M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 5 /22 5/22
  • 5. . . Google Maps : Route Datatype M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 6 /22 6/22
  • 6. . . Objectives M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 7 /22 7/22
  • 7. . . Objectives • an webapp a graph -- all-to-all roadmap -- from a which would create list of locations on top of Google Maps cloud based -- like Google Drive, Dropbox, etc... to ◦ terminology: serviceless app is a webapp that runs without a web • application has to be distribute work socially server ◦ otherwise need support from Google itself -- which is unlikely • application has to • scale socially -- full range from 1 to 10k collaborators rapid generation of datasets ◦ example: need graph of 1k locations by tomorrow! M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 8 /22 8/22
  • 8. . . Definitions socially scalable: same design for any number of collaborators • serverless webapp: • 1. a webapp that runs from a cloud drive 2. uses cloud drive to store data 3. can be shared by many collaborators • dataset: a graph (nodes and links) of routes among many locations 1. easily merged from many pairwise routes M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 9 /22 9/22
  • 9. . . maps2graphs: design M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 10 /22 10/22
  • 10. . . Basic Idea User M.Zhanikeev -- maratishe@gmail.com -- Data Webapp URL-based API Full RESTful API Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 11 /22 11/22
  • 11. . . Serverless Webapp User-Generated Content Slave copy (sync) Master copy More Detail Cloud Drive API Owner Content API M.Zhanikeev -- maratishe@gmail.com -- Collaborator Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 12 /22 12/22
  • 12. . . maps2graphs: Datatypes task.status.txt hA2A matrix of 5 v11 v11 v11 v11 v11 v21 v21 v21 v21 v31 v31 v31 v41 0100?001001?01 v41 becomes... task.list.json [ “address 1”, “address 2”, “address 3”, …. ] [collaborator].[time].[random tail] Cloud Drive M.Zhanikeev -- maratishe@gmail.com -- base64( JSON as string) …. … Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 13 /22 13/22
  • 13. . . maps2graphs: Batch • specific for Dropbox -- Dropbox JS API in 01 Collaborator Cloud Drive API Content API task.list.json New batch Randomize • minimize overlaps with other collaborators task.status.txt • new file (chunk) finalize file (set name) read/fill/update task.status.txt atomic: status string is updated task.status.txt is updated only when the batch file is successfully closed vxy write chunk (one line) randomization is important to • continuous: batch by batch 01 myself "Maps2Graphs Project Page" https://github.com/maratishe/maps2graphs (2013) M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 14 /22 14/22
  • 14. . . maps2graphs: social tricks M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 15 /22 15/22
  • 15. . . Social Features • maps2graphs is a crowdsourcing effort • much literature on crowdsourcing 04 05 06 07 • important feature is distribution: most people contribute very little, few contribute a lot most people will use the app for a few minutes, few will run is for days -- just leave it running in the browser • translation for maps2graphs: 04 G.Chatzimilioudis+3 "Crowdsourcing with Smartphones" IEEE J. Internet Computing (2012) 05 B.Yoo+2 "Travel Time Estimation Using Mobile Data" J. Transportation Studies (2005) 06 C.Wu+3 "Crowdsourcing Multimedia QoE Evaluation..." IEEE Trans. on Multimedia (2013) 07 D.Sui+2 "Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI)..." Springer Book (2013) M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 16 /22 16/22
  • 16. . . Actual Stats (later example) Client 1 Batch size 210 140 70 Client 2 Batch size 0 210 140 70 0 Client 3 Batch size 210 140 70 high intensity until out of quota, then hibernate until next 24H cycle . • high intensity: 1 req every 6 seconds • quota: 2500 req/day per IP • out of quota roughly in 3-5 hours 210 Client 4 Batch size 0 . Aggressive Approach . ... 140 70 • stay slightly below quota for personal use 0 0 20 40 60 80 100 Hour (relative to project start) M.Zhanikeev -- maratishe@gmail.com -- 120 140 Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 17 /22 17/22
  • 17. . . maps2graphs: implementation M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 18 /22 18/22
  • 18. . . Implementation • maps2graphs: source code in 01 • working code at 02 --- you can run it right now! http://tinyurl.com/maps2graphs ◦ or just go to ◦ 300 Family Marts in Northern Kyushu 01 myself "Maps2Graphs Project Page" https://github.com/maratishe/maps2graphs (2013) 02 myself "Maps2Graphs: Example Crowdsourcing Project" http://tinyurl.com/maps2graphs (2013) M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 19 /22 19/22
  • 19. . . Screenshot • simple interface • shows current volume (of current batch file) • shows details about current and past routes, quota status, etc. • if you need to abort, press commit and go M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 20 /22 20/22
  • 20. . . End Result M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 21 /22 21/22
  • 21. . . That’s all, thank you ... M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 22 /22 22/22
  • 22. . . [01] myself (2013) Maps2Graphs Project Page https://github.com/maratishe/maps2graphs [02] myself (2013) Maps2Graphs: Example Crowdsourcing Project http://tinyurl.com/maps2graphs [03] myself+1 (2013) ...EV Use in Environments with V2H and Battery Replacement... ITS Tokyo [04] G.Chatzimilioudis+3 (2012) Crowdsourcing with Smartphones IEEE J. Internet Computing [05] B.Yoo+2 (2005) Travel Time Estimation Using Mobile Data J. Transportation Studies M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 22 /22 22/22
  • 23. . . [06] C.Wu+3 (2013) Crowdsourcing Multimedia QoE Evaluation... IEEE Trans. on Multimedia [07] D.Sui+2 (2013) Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI)... Springer Book [08] Google (2013) Dropbox Core API https://www.dropbox.com/developers/core [09] Google (2013) Google Maps API https://developers.google.com/maps/ [10] (2013) JSON format www.json.org M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 22 /22 22/22
  • 24. . . [11] Esri (2013) ArcGIS Japan http://www.esrij.com/products/arcgis [12] W3C (2013) HTML5 W3C Candidate Recommendation at http://www.w3.org/TR/html5/ M.Zhanikeev -- maratishe@gmail.com -- Maps2Graphs: Social GIS Datasets Based on Google Maps API -- http://tinyurl.com/marat131204a --- 22 /22 22/22