Your SlideShare is downloading. ×
0
Building MapAttack: A                          real-time geolocation                                   game               ...
geoloqi.com                           What is Geoloqi?geoloqi.com Sunday, February 12, 12
geoloqi.com                                Real-world solutions                          for thousands around the globeSun...
geoloqi.com                           Real-Time Location Tracking                          Real-World Interactive Experien...
geoloqi.com                          Personal LocationSunday, February 12, 12
geoloqi.com                          Location-Based NotesSunday, February 12, 12
geoloqi.com                          YOUR GPS GETS WEIRD.                               WE HELP FIX THAT.Sunday, February ...
geoloqi.com                          Real-Time ApplicationsSunday, February 12, 12
geoloqi.comSunday, February 12, 12
geoloqi.com   First Hackathon Attempt                 Layer in Geoloqi       Updates to MapAttack server  via HTTP REST AP...
Sunday, February 12, 12
Sunday, February 12, 12
Sunday, February 12, 12
Sunday, February 12, 12
geoloqi.com   There were some kinks      Used an REST HTTP service for   APNS (worked, but not “real-time”)        Server ...
geoloqi.com               MapAttack Server Issues            Blocking IO (requests to Geoloqi API held       everything up...
geoloqi.com                 HOW WE                DEALT WITH                    ITSunday, February 12, 12
geoloqi.comSunday, February 12, 12
geoloqi.com               A key-value store, and so much more               A great implementation of PUBLISH/SUBSCRIBE   ...
geoloqi.comSunday, February 12, 12
geoloqi.comSunday, February 12, 12
geoloqi.comSunday, February 12, 12
geoloqi.comSunday, February 12, 12
geoloqi.com                 HOW WE USE NODE JS    • Phones:    Custom binary protocol over UDP    • Web Browsers:    Web S...
geoloqi.com                          Socket.IO is awesome                           Adapter Pattern for realtime          ...
geoloqi.comSunday, February 12, 12
geoloqi.com                          Geoloqi API                ASYNC FTW!                Delayed updates when possible   ...
geoloqi.comSunday, February 12, 12
geoloqi.com                            The Reactor Pattern                      “The reactor design pattern is a concurren...
geoloqi.com                          Concurrent IO is a common problem                          All programming languages ...
geoloqi.com                                 MapAttack Server                            (or: How to make Ruby ROFLscale)  ...
geoloqi.com                    Sinatra::Synchrony                             kyledrake.net/sinatra-synchrony             ...
geoloqi.com                    Sinatra::SynchronySunday, February 12, 12
geoloqi.comSunday, February 12, 12
geoloqi.com                      Let’s Talk Performance.Sunday, February 12, 12
geoloqi.com                          Database = your real                          persistence problemSunday, February 12,...
geoloqi.com                      Make the relevant data                      stay outside of the slow                     ...
geoloqi.com    If you’re trying to solve a problem by      ROFLscaling your persistent store,             you’re doing it ...
geoloqi.com                          Examples of doing it wrong               “Let’s rewrite everything to use TrendDB”   ...
geoloqi.com     If you must speed up your persistent        store, here’s my practical advice.Sunday, February 12, 12
geoloqi.com                          Real Hardware FTW?                   Single master database? Get a real server! Reall...
geoloqi.com                          Real Hardware FTW?       The paradigm is diversifying, and the reason is SSD.       S...
geoloqi.com                          Real Hardware FTW?                            $500 PCIe SSD drives:                  ...
geoloqi.com                                     Long Term                          Multi-master experiment (Brewer’s CAP) ...
geoloqi.com                          In SummarySunday, February 12, 12
geoloqi.com            Geo-location games                 are FUN!Sunday, February 12, 12
geoloqi.com            Use the Geoloqi API to make           geolocation games! We did the                 hard work for y...
geoloqi.com                     MapAttack is open source!                          github.com/geoloqi/MapAttackSunday, Feb...
geoloqi.com                          Other uses of GeoloqiSunday, February 12, 12
geoloqi.com                          Thanks!Sunday, February 12, 12
Upcoming SlideShare
Loading in...5
×

NodePDX Slides

2,640

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,640
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "NodePDX Slides"

  1. 1. Building MapAttack: A real-time geolocation game Kyle DrakeSunday, February 12, 12
  2. 2. geoloqi.com What is Geoloqi?geoloqi.com Sunday, February 12, 12
  3. 3. geoloqi.com Real-world solutions for thousands around the globeSunday, February 12, 12
  4. 4. geoloqi.com Real-Time Location Tracking Real-World Interactive ExperiencesSunday, February 12, 12
  5. 5. geoloqi.com Personal LocationSunday, February 12, 12
  6. 6. geoloqi.com Location-Based NotesSunday, February 12, 12
  7. 7. geoloqi.com YOUR GPS GETS WEIRD. WE HELP FIX THAT.Sunday, February 12, 12
  8. 8. geoloqi.com Real-Time ApplicationsSunday, February 12, 12
  9. 9. geoloqi.comSunday, February 12, 12
  10. 10. geoloqi.com First Hackathon Attempt Layer in Geoloqi Updates to MapAttack server via HTTP REST API Processing through Geoloqi’s persistent store Finished in a weekend A lot of fun!Sunday, February 12, 12
  11. 11. Sunday, February 12, 12
  12. 12. Sunday, February 12, 12
  13. 13. Sunday, February 12, 12
  14. 14. Sunday, February 12, 12
  15. 15. geoloqi.com There were some kinks Used an REST HTTP service for APNS (worked, but not “real-time”) Server was getting backed up (updates were not async)Sunday, February 12, 12
  16. 16. geoloqi.com MapAttack Server Issues Blocking IO (requests to Geoloqi API held everything up) Lots of concurrent traffic Events had to be broadcast to all phones/ clients in the game synchronouslySunday, February 12, 12
  17. 17. geoloqi.com HOW WE DEALT WITH ITSunday, February 12, 12
  18. 18. geoloqi.comSunday, February 12, 12
  19. 19. geoloqi.com A key-value store, and so much more A great implementation of PUBLISH/SUBSCRIBE So simple, you can actually understand it Very fast and reliable Robust library supportSunday, February 12, 12
  20. 20. geoloqi.comSunday, February 12, 12
  21. 21. geoloqi.comSunday, February 12, 12
  22. 22. geoloqi.comSunday, February 12, 12
  23. 23. geoloqi.comSunday, February 12, 12
  24. 24. geoloqi.com HOW WE USE NODE JS • Phones: Custom binary protocol over UDP • Web Browsers: Web Sockets via Socket.IOSunday, February 12, 12
  25. 25. geoloqi.com Socket.IO is awesome Adapter Pattern for realtime Web Sockets, Flash, Long Polling! One interface for everything Easy to implement: ~100 LOCSunday, February 12, 12
  26. 26. geoloqi.comSunday, February 12, 12
  27. 27. geoloqi.com Geoloqi API ASYNC FTW! Delayed updates when possible Messages queues: Beanstalk Workers pick tasks off the stack Process data outside of the persistent store Simpler than AMQPSunday, February 12, 12
  28. 28. geoloqi.comSunday, February 12, 12
  29. 29. geoloqi.com The Reactor Pattern “The reactor design pattern is a concurrent programming pattern for handling service requests delivered concurrently to a service handler by one or more inputs” - Wikipedia My simplest description: Takes your blocking IO operation, shoves it into its own kernel thread behind the scenes, uses Unix kernel magic to make it rejoin the reactor queue when it’s ready. Linux: epoll(4) BSD: kqueue/keventSunday, February 12, 12
  30. 30. geoloqi.com Concurrent IO is a common problem All programming languages have trouble with it The Reactor pattern is a tool to resolve it Most languages have the Reactor pattern! JavaScript Node.js Python Twisted Ruby EventMachine (libem, C) Java JBoss_Netty PHP None yetSunday, February 12, 12
  31. 31. geoloqi.com MapAttack Server (or: How to make Ruby ROFLscale) MRI Ruby has a global interpreter lock But MRI Ruby does not block on IO JRuby and Rubinius 2: Native Threads And it has a reactor pattern, just like Node JSSunday, February 12, 12
  32. 32. geoloqi.com Sinatra::Synchrony kyledrake.net/sinatra-synchrony Implements EventMachine with Fibers The result: NO CALLBACKS < 100 LOC Only coding change is to use EM libs and fibers DON’T USE IT.Sunday, February 12, 12
  33. 33. geoloqi.com Sinatra::SynchronySunday, February 12, 12
  34. 34. geoloqi.comSunday, February 12, 12
  35. 35. geoloqi.com Let’s Talk Performance.Sunday, February 12, 12
  36. 36. geoloqi.com Database = your real persistence problemSunday, February 12, 12
  37. 37. geoloqi.com Make the relevant data stay outside of the slow persistence store.Sunday, February 12, 12
  38. 38. geoloqi.com If you’re trying to solve a problem by ROFLscaling your persistent store, you’re doing it wrong.Sunday, February 12, 12
  39. 39. geoloqi.com Examples of doing it wrong “Let’s rewrite everything to use TrendDB” Using EC2 for a single-master DB “because it scales” Buying something from Oracle “Ruby/PHP/JS is too slow for this, let’s rewrite it in TrendLang” Many NoSQL solutions have problems you don’t know about. Global write locks, single write master, et ceteraSunday, February 12, 12
  40. 40. geoloqi.com If you must speed up your persistent store, here’s my practical advice.Sunday, February 12, 12
  41. 41. geoloqi.com Real Hardware FTW? Single master database? Get a real server! Really! Cloud is only as fast as the fastest available HDD EBS performance is not great, even in RAID0 stripe See orion.heroku.com/past/2009/7/29/io_performance_on_ebs and perfcap.blogspot.com/2011/03/understanding-and-using-amazon-ebs.html and blog.dt.org/index.php/2010/06/amazon-ec2-io-performance-local-emphemeral-disks-vs-raid0- striped-ebs-volumes/ and endevver.com/2010/03/cost-analysis-of-an-amazon-ec2-deployment.html and mysqlperformanceblog.com/2011/02/21/death-match-ebs-versus-ssd-price-performance-and- qos and google.com/search?q=heroku+ebs+performance High end Xeon, 32GB ECC, fast 220GB SSD: ~$2KSunday, February 12, 12
  42. 42. geoloqi.com Real Hardware FTW? The paradigm is diversifying, and the reason is SSD. Say hello to the Fusion-io ioDrive Octal. PCIe x16 slot 5.12TB 6.0 GB/s read 4.4 GB/s write 1.19 MILLION IOPS Cost: $100,000 (plus $10 S&H). It’s a steal at this price.. but it will get cheaper.Sunday, February 12, 12
  43. 43. geoloqi.com Real Hardware FTW? $500 PCIe SSD drives: 220GB >100,000 IOPS ~700MB/s R/W 15,000 RPM SAS drives: 2TB+ ~175-210 IOPS <150MB/s R/WSunday, February 12, 12
  44. 44. geoloqi.com Long Term Multi-master experiment (Brewer’s CAP) Riak is my current favorite Cloud companies providing real hardware performance Map/Reduce != DatabaseSunday, February 12, 12
  45. 45. geoloqi.com In SummarySunday, February 12, 12
  46. 46. geoloqi.com Geo-location games are FUN!Sunday, February 12, 12
  47. 47. geoloqi.com Use the Geoloqi API to make geolocation games! We did the hard work for you.Sunday, February 12, 12
  48. 48. geoloqi.com MapAttack is open source! github.com/geoloqi/MapAttackSunday, February 12, 12
  49. 49. geoloqi.com Other uses of GeoloqiSunday, February 12, 12
  50. 50. geoloqi.com Thanks!Sunday, February 12, 12
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×