Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Google Maps to Graphs
a howto on large-scale collaboration
using Google Maps and Dropbox
Marat Zhanikeev
2013/08/28
The Objective
1. Take a list of locations
ex: Family Mart convenience stores in some area
2. Use Google Maps API to get th...
The Reason for Crowdsourcing
• 100 locations need ~5000 Map API requests
• Daily quota of Maps API is 2500/day/IP
• You fi...
Step 1: Build Dropbox API for JS
• Dropbox does not have a JS API … yet
• https://www.dropbox.com/developers/core
• Dropbo...
Step 2: Learn Google Maps API
• Piece of cake!
• DirectionsService for routes (car, walking,…)
• Easy – you can feed it ad...
Step 3: Create a Serverless App
• Serverless Webapp: a web application which
does not require a web server
• Both GoogleMa...
Step 4: Make It Work for Crowds
• Concurrent access to data by many users
• Write of data are OK ( base64( JSON) = text)
•...
Step 5: Send it Out into the Wild
• Test run
• 300 Family Marts in Fukuoka, Japan
• ~35K requests
https://t.co/F5zPCeqhQw
...
It looks like this…
• Fully
automatic,
you can just
let it run
• Otherwise, if
you have to
run right now,
just press
COMMI...
Upcoming SlideShare
Loading in …5
×

Googlemaps4crowds

284 views

Published on

#maps2graphs project. The goals is simple: merge pairwise route information you get from Google Maps API into an all-to-all graph structure. Useful for various graph-based research like supply chains, topology optimization, etc. I use it for electric cars.

Published in: Technology, News & Politics
  • Be the first to comment

Googlemaps4crowds

  1. 1. Google Maps to Graphs a howto on large-scale collaboration using Google Maps and Dropbox Marat Zhanikeev 2013/08/28
  2. 2. The Objective 1. Take a list of locations ex: Family Mart convenience stores in some area 2. Use Google Maps API to get the route for each pair of locations if A→B = B→A, then roughly 0.5 * ( N^2) in code: for ( a = 0; a < n – 1; a++) { for ( b = a + 1; b < n; b++) { … }} 3. Split data in blocks and write each block to Dropbox • all clients write to the same Dropbox folder (yours) • use chunk_upload to split data in batches (otherwise too many files) • make it work for a large number of concurrent clients 4. Ask people for help (limited by single-client daily quota)
  3. 3. The Reason for Crowdsourcing • 100 locations need ~5000 Map API requests • Daily quota of Maps API is 2500/day/IP • You figure out the math • Reality: even a small crowd can create a dataset in no time • After all, it is all about creating datasets on demand
  4. 4. Step 1: Build Dropbox API for JS • Dropbox does not have a JS API … yet • https://www.dropbox.com/developers/core • Dropbox vs Google Drive – Only Dropbox is suitable – 24h valid upload_id for chunk_upload – Makes it possible to aggregates data into batches • Status: DONE (URL on the last page)
  5. 5. Step 2: Learn Google Maps API • Piece of cake! • DirectionsService for routes (car, walking,…) • Easy – you can feed it address strings, names of places, anything… • Status: DONE (URL on the last page)
  6. 6. Step 3: Create a Serverless App • Serverless Webapp: a web application which does not require a web server • Both GoogleMapsAPI and DropboxAPI use JS • You can pack it all into one HTML file… • … put it into your Dropbox folder …. • … and share it! • Status: DONE (URL on the last page)
  7. 7. Step 4: Make It Work for Crowds • Concurrent access to data by many users • Write of data are OK ( base64( JSON) = text) • Need status file to be shared by all – Need it for clients to know which routes to request – Overlaps are inevitable, but random selection of tasks will have to avoid too much overlap • Use a smart logic ( 1 req / 5s, safeguards, smarts, …) • Status: DONE (URL on the last page)
  8. 8. Step 5: Send it Out into the Wild • Test run • 300 Family Marts in Fukuoka, Japan • ~35K requests https://t.co/F5zPCeqhQw • Deploy Status: DONE (that t.co URL) • Crowds Status: WAITING • failed to get my tweets show up in #googlemapsapi #googlemaps #gis #datasets • miserably failed to have it noticed anywhere else • Don’t have that many friends • Deep Thoughts: is on-demand online collaboration actually possible?
  9. 9. It looks like this… • Fully automatic, you can just let it run • Otherwise, if you have to run right now, just press COMMIT and go.

×