10. geoloqi.com
First Hackathon Attempt
Layer in Geoloqi
Updates to MapAttack server
via HTTP REST API
Processing through Geoloqi’s
persistent store
Finished in a weekend
A lot of fun!
Sunday, February 12, 12
15. geoloqi.com
There were some kinks
Used an REST HTTP service for
APNS (worked, but not “real-time”)
Server was getting backed up
(updates were not async)
Sunday, February 12, 12
16. geoloqi.com
MapAttack Server Issues
Blocking IO (requests to Geoloqi API held
everything up)
Lots of concurrent traffic
Events had to be broadcast to all phones/
clients in the game synchronously
Sunday, February 12, 12
17. geoloqi.com
HOW WE
DEALT WITH
IT
Sunday, February 12, 12
19. geoloqi.com
A key-value store, and so much more
A great implementation of PUBLISH/SUBSCRIBE
So simple, you can actually understand it
Very fast and reliable
Robust library support
Sunday, February 12, 12
24. geoloqi.com
HOW WE USE NODE JS
• Phones:
Custom binary protocol over UDP
• Web Browsers:
Web Sockets via Socket.IO
Sunday, February 12, 12
25. geoloqi.com
Socket.IO is awesome
Adapter Pattern for realtime
Web Sockets, Flash, Long Polling!
One interface for everything
Easy to implement: ~100 LOC
Sunday, February 12, 12
27. geoloqi.com
Geoloqi API
ASYNC FTW!
Delayed updates when possible
Messages queues: Beanstalk
Workers pick tasks off the stack
Process data outside of the persistent store
Simpler than AMQP
Sunday, February 12, 12
29. geoloqi.com
The Reactor Pattern
“The reactor design pattern is a concurrent
programming pattern for handling service
requests delivered concurrently to a service
handler by one or more inputs” - Wikipedia
My simplest description: Takes your blocking IO
operation, shoves it into its own kernel thread
behind the scenes, uses Unix kernel magic to make
it rejoin the reactor queue when it’s ready.
Linux: epoll(4)
BSD: kqueue/kevent
Sunday, February 12, 12
30. geoloqi.com
Concurrent IO is a common problem
All programming languages have trouble with it
The Reactor pattern is a tool to resolve it
Most languages have the Reactor pattern!
JavaScript Node.js
Python Twisted
Ruby EventMachine (libem, C)
Java JBoss_Netty
PHP None yet
Sunday, February 12, 12
31. geoloqi.com
MapAttack Server
(or: How to make Ruby ROFLscale)
MRI Ruby has a global interpreter lock
But MRI Ruby does not block on IO
JRuby and Rubinius 2: Native Threads
And it has a reactor pattern, just like Node JS
Sunday, February 12, 12
32. geoloqi.com
Sinatra::Synchrony
kyledrake.net/sinatra-synchrony
Implements EventMachine with Fibers
The result: NO CALLBACKS
< 100 LOC
Only coding change is to use EM libs and fibers
DON’T USE IT.
Sunday, February 12, 12
33. geoloqi.com
Sinatra::Synchrony
Sunday, February 12, 12
35. geoloqi.com
Let’s Talk Performance.
Sunday, February 12, 12
36. geoloqi.com
Database = your real
persistence problem
Sunday, February 12, 12
37. geoloqi.com
Make the relevant data
stay outside of the slow
persistence store.
Sunday, February 12, 12
38. geoloqi.com
If you’re trying to solve a problem by
ROFLscaling your persistent store,
you’re doing it wrong.
Sunday, February 12, 12
39. geoloqi.com
Examples of doing it wrong
“Let’s rewrite everything to use TrendDB”
Using EC2 for a single-master DB “because it scales”
Buying something from Oracle
“Ruby/PHP/JS is too slow for this, let’s rewrite it in
TrendLang”
Many NoSQL solutions have problems you don’t
know about. Global write locks, single write master, et
cetera
Sunday, February 12, 12
40. geoloqi.com
If you must speed up your persistent
store, here’s my practical advice.
Sunday, February 12, 12
41. geoloqi.com
Real Hardware FTW?
Single master database? Get a real server! Really!
Cloud is only as fast as the fastest available HDD
EBS performance is not great, even in RAID0 stripe
See orion.heroku.com/past/2009/7/29/io_performance_on_ebs
and perfcap.blogspot.com/2011/03/understanding-and-using-amazon-ebs.html
and blog.dt.org/index.php/2010/06/amazon-ec2-io-performance-local-emphemeral-disks-vs-raid0-
striped-ebs-volumes/
and endevver.com/2010/03/cost-analysis-of-an-amazon-ec2-deployment.html
and mysqlperformanceblog.com/2011/02/21/death-match-ebs-versus-ssd-price-performance-and-
qos
and google.com/search?q=heroku+ebs+performance
High end Xeon, 32GB ECC, fast 220GB SSD: ~$2K
Sunday, February 12, 12
42. geoloqi.com
Real Hardware FTW?
The paradigm is diversifying, and the reason is SSD.
Say hello to the Fusion-io ioDrive Octal.
PCIe x16 slot
5.12TB
6.0 GB/s read
4.4 GB/s write
1.19 MILLION IOPS
Cost: $100,000 (plus $10 S&H).
It’s a steal at this price.. but it will get cheaper.
Sunday, February 12, 12
43. geoloqi.com
Real Hardware FTW?
$500 PCIe SSD drives:
220GB
>100,000 IOPS
~700MB/s R/W
15,000 RPM SAS drives:
2TB+
~175-210 IOPS
<150MB/s R/W
Sunday, February 12, 12
44. geoloqi.com
Long Term
Multi-master experiment (Brewer’s CAP)
Riak is my current favorite
Cloud companies providing real hardware
performance
Map/Reduce != Database
Sunday, February 12, 12
45. geoloqi.com
In Summary
Sunday, February 12, 12
46. geoloqi.com
Geo-location games
are FUN!
Sunday, February 12, 12
47. geoloqi.com
Use the Geoloqi API to make
geolocation games! We did the
hard work for you.
Sunday, February 12, 12
48. geoloqi.com
MapAttack is open source!
github.com/geoloqi/MapAttack
Sunday, February 12, 12
49. geoloqi.com
Other uses of Geoloqi
Sunday, February 12, 12
