Lost In The Clouds

  • 337 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
337
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Lost in the Clouds An example of a schemaless database application in the cloud SlipstreamUSA :: March 25, 2009
  • 2. The Project
      • OpenStreetMap
      • 3. A free map of the planet
        • Free geospatial data
        • 4. Licensed under CC-BY-SA
      • Created from
        • Crowd-sourced contributions
        • 5. 90,000 registered users
        • 6. 1,000+ contributors per day
      • Used for:
    © Constantin Litvak
  • 12. Coverage CC-BY-SA 2.0 © OpenStreetMap
  • 13. The OpenStreetMap Database Server CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 20. Interfaces
      • API
        • For updates (1,000 users per day)
        • 21. Queries of up to 0.25 ² degrees
      • Weekly Planet Dump
        • 100Gbyte XML file
      • Changes
        • Feed of changes
        • 22. Daily / Hourly / Minutely
        • 23. One file per minute (~30Kb compressed)
    CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 24. Problem Statement
      • Problem
      • 25. Want to be able to query the data in bigger chunks and to do it more selectively
      • 26. Solution
      • 27. Cloud based schemaless database with simple to use query interface and lots of indexes
    CC-BY-SA 2.0 Image © Maning Sambale Data © OpenStreetMap
  • 28. XAPI Service
      • GT.M or Cache
      • 29. 250Gbyte database
      • Apache + serverLink
      • 35. Licensed as AGPL
    CC-BY-SA 2.0 Image © Maning Sambale Data © OpenStreetMap
  • 36. XAPI Service
      • In the cloud
      • 37. Queries of up to 100 ² degrees
      • 38. Query by tags and tag values
      • 39. Scaleable
      • 40. REST style interface
    http://xapi.openstreetmap.org/api/0.6/* [tourism=hotel] [bbox=-75.7,40.0,-75.4,40.2]
      • Import planet dump
      • 41. Import minute diff
    CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 42. XAPI Service
      • Geospatial Index – quadstrings
    adaabcdcabaadab
  • 43. XAPI Service ^way(27016525)=&quot;adaabcdcabaadab&quot; ^way(27016525,1)=296138118 ^way(27016525,2)=296138119 ^way(27016525,3)=296138120 ^way(27016525,4)=296138121 ^way(27016525,5)=296138118 ^waytag(27016525,&quot;addr:housenumber&quot;)=2 ^waytag(27016525,&quot;building&quot;)=&quot;yes&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27016525)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028298)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028299)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028326)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028327)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035972)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035973)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035974)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035975)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035984)=&quot;“ <way id='27016525'> <nd ref='296138118'/> <nd ref='296138119'/> <nd ref='296138120'/> <nd ref='296138121'/> <nd ref='296138118'/> <tag k='addr:housenumber' v='2'/> <tag k='building' v='yes'/> </way>
  • 44. Deployment
      • Currently 5 instances:
      • More servers needed:
        • Internet connectivity
        • 49. 300Gb disk storage
    CC-BY-SA 2.0 Image © Nikolaj Data © OpenStreetMap
  • 50. Wikipedia: Query to Map {{osm-query |way=name=Arizona Canal}} CC-BY-SA 2.0 © OpenStreetMap
  • 51. Internals
      • Debian
      • 52. Apache web-server
      • 53. REST
        • URI represents a resource
        • 54. Xpath flavoring
      • serverLink
        • WebLink emulator
      • GT.M or Cache
        • POM
      • AGPL
  • 55. XAPI Future
      • Load Sharing / balancing
      • 56. More Xpath-like queries
      • 57. JSON output
      • 58. Generic indexing service
      • 59. More servers
  • 60. Thank you Thank you Questions? CC-BY-SA 2.0 Image © ITO! Data © OpenStreetMap