Download as KEY, PPTX
Uploaded byKyle Drake
Building MapAttack
This document discusses building a real-time geolocation game called MapAttack using the Geoloqi API and platform. It describes some initial issues encountered with the game server handling concurrent traffic and real-time updates. It then explains how using Node.js, EventMachine, and Geoloqi's publish/subscribe system helped address these issues by making the server asynchronous and event-driven. The document advocates for keeping relevant data outside persistent stores for better performance and discusses using high-performance hardware as an alternative to scaling database infrastructure. It promotes Geoloqi and building other real-time geolocation applications and games.
Building MapAttack
- 1. Building MapAttack: A real-timegeolocation game Kyle Drake
- 2. geoloqi.com What is Geoloqi? geoloqi.com
- 3. geoloqi.com Real-world solutions for thousands around the globe
- 4. geoloqi.com Real-Time LocationTracking Real-World Interactive Experiences
- 5.
- 6.
- 7. geoloqi.com YOUR GPS GETSWEIRD. WE HELP FIX THAT.
- 8.
- 9.
- 10. geoloqi.com First Hackathon Attempt Layer in Geoloqi Updates to MapAttack server via HTTP REST API Processing through Geoloqi’s persistent store Finished in a weekend A lot of fun!
- 15. geoloqi.com There were somekinks Used an REST HTTP service for APNS (worked, but not “real-time”) Server was getting backed up (updates were not async)
- 16. geoloqi.com MapAttackServer Issues Blocking IO (requests to Geoloqi API held everything up) Lots of concurrent traffic Events had to be broadcast to all phones/ clients in the game synchronously
- 17. geoloqi.com HOW WE DEALTWITH IT
- 18.
- 19. geoloqi.com A key-value store,and so much more A great implementation of PUBLISH/SUBSCRIBE So simple, you can actually understand it Very fast and reliable Robust library support
- 20.
- 21.
- 22.
- 23. geoloqi.com HOW WE USE NODE JS • Phones: Custom binary protocol over UDP • Web Browsers: Web Sockets via Socket.IO
- 24. geoloqi.com Socket.IO is awesome Adapter Pattern for realtime Web Sockets, Flash, Long Polling! One interface for everything Easy to implement: ~100 LOC
- 25.
- 26. geoloqi.com Geoloqi API ASYNC FTW! Delayed updates when possible Messages queues: Beanstalk Workers pick tasks off the stack Process data outside of the persistent store Simpler than AMQP
- 27.
- 28. geoloqi.com MapAttack Server (or: How to make Ruby ROFLscale) MRI Ruby has a global interpreter lock But MRI Ruby does not block on IO JRuby and Rubinius 2: Native Threads And it has a reactor pattern, just like Node JS!
- 29. geoloqi.com The Reactor Pattern “The reactor design pattern is a concurrent programming pattern for handling service requests delivered concurrently to a service handler by one or more inputs” - Wikipedia My simplest description: Takes your blocking IO operation, shoves it into its own kernel thread behind the scenes, uses Unix kernel magic to make it rejoin the reactor queue when it’s ready. Linux: epoll(4) BSD: kqueue/kevent
- 30. geoloqi.com Blocking IO aUNIVERSAL problem All programming languages have trouble with it The Reactor pattern is a tool to resolve it Most languages have the Reactor pattern! JavaScript Node.js Python Twisted Ruby EventMachine (libem, C) Java JBoss_Netty (wait, what? I thought Java had good threading..) PHP None yet (perhaps ever)
- 31. geoloqi.com Sinatra::Synchrony kyledrake.net/sinatra-synchrony Implements EventMachine with Fibers The result: NO CALLBACKS! < 100 LOC Only coding change is to use EM libs and fibers WOW, THAT WAS EASY.
- 32.
- 33.
- 34.
- 35. geoloqi.com Make the relevantdata stay outside of the slow persistence store.
- 36. geoloqi.com If you’re tryingto solve a problem by ROFLscaling your persistent store, you’re doing it wrong.
- 37. geoloqi.com Examples of doing it wrong “Let’s rewrite everything to use TrendDB” Using EC2 for a single-master DB “because it scales” Buying something from Oracle “Ruby/PHP/JS is too slow for this, let’s rewrite it in TrendLang” Most NoSQL solutions have dark secrets you don’t know about. Global write locks, single write master, et cetera
- 38. geoloqi.com If you mustspeed up your persistent store, here’s my practical advice.
- 39. geoloqi.com Real Hardware FTW? Single master database? Get a real server! Really! Cloud is only as fast as the fastest available HDD EBS performance is not great, even in RAID0 stripe See orion.heroku.com/past/2009/7/29/io_performance_on_ebs and perfcap.blogspot.com/2011/03/understanding-and-using-amazon-ebs.html and blog.dt.org/index.php/2010/06/amazon-ec2-io-performance-local-emphemeral-disks-vs-raid0- striped-ebs-volumes/ and endevver.com/2010/03/cost-analysis-of-an-amazon-ec2-deployment.html and mysqlperformanceblog.com/2011/02/21/death-match-ebs-versus-ssd-price-performance-and- qos and google.com/search?q=heroku+ebs+performance High end Xeon, 32GB ECC, fast 220GB SSD: ~$2K
- 40. geoloqi.com Real Hardware FTW? The paradigm is diversifying, and the reason is SSD. Say hello to the Fusion-io ioDrive Octal. PCIe x16 slot 5.12TB 6.0 GB/s read 4.4 GB/s write 1.19 MILLION IOPS Cost: $100,000 (plus $10 S&H). It’s a steal at this price.. but it will get cheaper.
- 41. geoloqi.com Real Hardware FTW? $500 PCIe SSD drives: 220GB >100,000 IOPS ~700MB/s R/W 15,000 RPM SAS drives: 2TB+ ~175-210 IOPS <150MB/s R/W
- 42. geoloqi.com Long Term Multi-master experiment (Brewer’s CAP) Riak is my current favorite Cloud companies providing real hardware performance
- 43.
- 44. geoloqi.com Geo-location games are FUN!
- 45. geoloqi.com Use theGeoloqi API to make geolocation games! We did the hard work for you.
- 46. geoloqi.com MapAttack is opensource! github.com/geoloqi/MapAttack
- 47.
- 48. geoloqi.com Other uses ofGeoloqi
- 49.