Your SlideShare is downloading. ×
Lost In The Clouds
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Lost In The Clouds

367

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
367
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Lost in the Clouds An example of a schemaless database application in the cloud SlipstreamUSA :: March 25, 2009
  • 2. The Project
      • OpenStreetMap
      • 3. A free map of the planet
        • Free geospatial data
        • 4. Licensed under CC-BY-SA
      • Created from
        • Crowd-sourced contributions
        • 5. 90,000 registered users
        • 6. 1,000+ contributors per day
      • Used for:
    © Constantin Litvak
  • 12. Coverage CC-BY-SA 2.0 © OpenStreetMap
  • 13. The OpenStreetMap Database Server CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 20. Interfaces
      • API
        • For updates (1,000 users per day)
        • 21. Queries of up to 0.25 ² degrees
      • Weekly Planet Dump
        • 100Gbyte XML file
      • Changes
        • Feed of changes
        • 22. Daily / Hourly / Minutely
        • 23. One file per minute (~30Kb compressed)
    CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 24. Problem Statement
      • Problem
      • 25. Want to be able to query the data in bigger chunks and to do it more selectively
      • 26. Solution
      • 27. Cloud based schemaless database with simple to use query interface and lots of indexes
    CC-BY-SA 2.0 Image © Maning Sambale Data © OpenStreetMap
  • 28. XAPI Service
      • GT.M or Cache
      • 29. 250Gbyte database
      • Apache + serverLink
      • 35. Licensed as AGPL
    CC-BY-SA 2.0 Image © Maning Sambale Data © OpenStreetMap
  • 36. XAPI Service
      • In the cloud
      • 37. Queries of up to 100 ² degrees
      • 38. Query by tags and tag values
      • 39. Scaleable
      • 40. REST style interface
    http://xapi.openstreetmap.org/api/0.6/* [tourism=hotel] [bbox=-75.7,40.0,-75.4,40.2]
      • Import planet dump
      • 41. Import minute diff
    CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 42. XAPI Service
      • Geospatial Index – quadstrings
    adaabcdcabaadab
  • 43. XAPI Service ^way(27016525)=&quot;adaabcdcabaadab&quot; ^way(27016525,1)=296138118 ^way(27016525,2)=296138119 ^way(27016525,3)=296138120 ^way(27016525,4)=296138121 ^way(27016525,5)=296138118 ^waytag(27016525,&quot;addr:housenumber&quot;)=2 ^waytag(27016525,&quot;building&quot;)=&quot;yes&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27016525)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028298)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028299)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028326)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028327)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035972)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035973)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035974)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035975)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035984)=&quot;“ <way id='27016525'> <nd ref='296138118'/> <nd ref='296138119'/> <nd ref='296138120'/> <nd ref='296138121'/> <nd ref='296138118'/> <tag k='addr:housenumber' v='2'/> <tag k='building' v='yes'/> </way>
  • 44. Deployment
      • Currently 5 instances:
      • More servers needed:
        • Internet connectivity
        • 49. 300Gb disk storage
    CC-BY-SA 2.0 Image © Nikolaj Data © OpenStreetMap
  • 50. Wikipedia: Query to Map {{osm-query |way=name=Arizona Canal}} CC-BY-SA 2.0 © OpenStreetMap
  • 51. Internals
      • Debian
      • 52. Apache web-server
      • 53. REST
        • URI represents a resource
        • 54. Xpath flavoring
      • serverLink
        • WebLink emulator
      • GT.M or Cache
        • POM
      • AGPL
  • 55. XAPI Future
      • Load Sharing / balancing
      • 56. More Xpath-like queries
      • 57. JSON output
      • 58. Generic indexing service
      • 59. More servers
  • 60. Thank you Thank you Questions? CC-BY-SA 2.0 Image © ITO! Data © OpenStreetMap

×