Your SlideShare is downloading. ×
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Lost In The Clouds
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Lost In The Clouds

370

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
370
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Lost in the Clouds An example of a schemaless database application in the cloud SlipstreamUSA :: March 25, 2009
  • 2. The Project
      • OpenStreetMap
      • 3. A free map of the planet
        • Free geospatial data
        • 4. Licensed under CC-BY-SA
      • Created from
        • Crowd-sourced contributions
        • 5. 90,000 registered users
        • 6. 1,000+ contributors per day
      • Used for:
    © Constantin Litvak
  • 12. Coverage CC-BY-SA 2.0 © OpenStreetMap
  • 13. The OpenStreetMap Database Server CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 20. Interfaces
      • API
        • For updates (1,000 users per day)
        • 21. Queries of up to 0.25 ² degrees
      • Weekly Planet Dump
        • 100Gbyte XML file
      • Changes
        • Feed of changes
        • 22. Daily / Hourly / Minutely
        • 23. One file per minute (~30Kb compressed)
    CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 24. Problem Statement
      • Problem
      • 25. Want to be able to query the data in bigger chunks and to do it more selectively
      • 26. Solution
      • 27. Cloud based schemaless database with simple to use query interface and lots of indexes
    CC-BY-SA 2.0 Image © Maning Sambale Data © OpenStreetMap
  • 28. XAPI Service
      • GT.M or Cache
      • 29. 250Gbyte database
      • Apache + serverLink
      • 35. Licensed as AGPL
    CC-BY-SA 2.0 Image © Maning Sambale Data © OpenStreetMap
  • 36. XAPI Service
      • In the cloud
      • 37. Queries of up to 100 ² degrees
      • 38. Query by tags and tag values
      • 39. Scaleable
      • 40. REST style interface
    http://xapi.openstreetmap.org/api/0.6/* [tourism=hotel] [bbox=-75.7,40.0,-75.4,40.2]
      • Import planet dump
      • 41. Import minute diff
    CC-BY-SA 2.0 Image © Ojw Data © OpenStreetMap
  • 42. XAPI Service
      • Geospatial Index – quadstrings
    adaabcdcabaadab
  • 43. XAPI Service ^way(27016525)=&quot;adaabcdcabaadab&quot; ^way(27016525,1)=296138118 ^way(27016525,2)=296138119 ^way(27016525,3)=296138120 ^way(27016525,4)=296138121 ^way(27016525,5)=296138118 ^waytag(27016525,&quot;addr:housenumber&quot;)=2 ^waytag(27016525,&quot;building&quot;)=&quot;yes&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27016525)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028298)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028299)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028326)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27028327)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035972)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035973)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035974)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035975)=&quot;&quot; ^wayx(&quot;building&quot;,&quot;*&quot;,&quot;adaabcdcabaadab&quot;,27035984)=&quot;“ <way id='27016525'> <nd ref='296138118'/> <nd ref='296138119'/> <nd ref='296138120'/> <nd ref='296138121'/> <nd ref='296138118'/> <tag k='addr:housenumber' v='2'/> <tag k='building' v='yes'/> </way>
  • 44. Deployment
      • Currently 5 instances:
      • More servers needed:
        • Internet connectivity
        • 49. 300Gb disk storage
    CC-BY-SA 2.0 Image © Nikolaj Data © OpenStreetMap
  • 50. Wikipedia: Query to Map {{osm-query |way=name=Arizona Canal}} CC-BY-SA 2.0 © OpenStreetMap
  • 51. Internals
      • Debian
      • 52. Apache web-server
      • 53. REST
        • URI represents a resource
        • 54. Xpath flavoring
      • serverLink
        • WebLink emulator
      • GT.M or Cache
        • POM
      • AGPL
  • 55. XAPI Future
      • Load Sharing / balancing
      • 56. More Xpath-like queries
      • 57. JSON output
      • 58. Generic indexing service
      • 59. More servers
  • 60. Thank you Thank you Questions? CC-BY-SA 2.0 Image © ITO! Data © OpenStreetMap

×