This document summarizes scaling Ruby applications for enterprise use. It discusses profiling tools like New Relic, databases like MongoDB and MySQL, queues like 0MQ and RabbitMQ, tools like Chef, testing with RSpec and Cucumber, dealing with legacy code, and adopting a component architecture with state on the wire. Specific techniques discussed include vertical scaling on EC2 by moving subsystems to different instance types, horizontal scaling by adding more machines and using Chef to provision them, and how these approaches improved metrics like throughput, database I/O, and size. The document emphasizes measuring performance, not prematurely optimizing, and allowing failure during development.

1. Scaling Ruby for
Enterprise Applications
MinneBar 2013

2. by Joel
Jensen
and John
Malone

3. How we got there...
Profiling
Databases
Queues
Tools
Testing
Legacy Code
Mentality
State on the Wire

4. Best Buy Epic App

5. Topology

6. Profiling
New Relic is our weapon of choice.

7. Stuff we used
Thread profiling
Monitoring
Daemons
Custom
instrumentation
Developer Mode
Deployment events
Slow query log

8. Developer Mode

9. Developer Mode
Query profiling

10. Databases
MongoDB
+Drivers of our language of choice
+Speed
+Flexibility
+Partition Tolerance
+Availability
-Consistency

11. Databases
Mysql
+Drivers for our language
-Flexibility
+Consistency

12. CAP theorem

13. Queues

14. 0MQ
+Speed (fastest you
can get)
+Very simple
design, great for
complex systems
+Drivers in all major
languages
-Hard to use
-Poor drivers
-No UI

15. RabbitMQ
+Speed
+Great web based
UI
Drivers in all major
languages

16. Tools
Chef
Automated,
scripted server
provisioning
Crucial for
modern systems

17. Testing
Integration test is the MOST important for us
Automate huge batches
Automate your metrics
Test with real data
Rspec and Cucumber

18. Legacy
Code

19. Mentality

20. Hack the Planet
Be a hacker (ala
hackers and
painters)
You are allowed to
fail
DRY
Abstraction
everywhere
Tests
No burnouts

21. Component
Architecture
Before State on the Wire

22. State on the Wire
Beat the Clock
Component => Input data, work on it, save it,
enqueue
Atomic Units of Work
Design of latency, serialize the message
Minimize database writes
Daemons gem

23. Component
Architecture
After State on the Wire

24. State on the Wire
Improvements
60% increase in throughput
90% decrease in database I/O
85% decrease in database size

25. Vertical Scaling
ec2instances.info

26. Throwing hardware at
the problem
Needed solid profiling
Realized we were running our CPU hot
Moved the Epic subsystem onto a High CPU
instance
Moved Bouncer subsystem onto a High
Memory instance

27. Vertical Scaling
Continued
8 cores were now put into play
200% increase in throughput
Daemons began autoscaling
Leave a core for your OS

28. Horizontal Scaling
Componentized architecture allows us to throw
more machines at the problem
Future state - autoscaling based upon queue
length
Chef stands machines up and down based on
CloudWatch queue length monitoring

29. Lessons Learned
You can’t improve
what you can’t
measure
Install good
monitoring and
alerting
Don’t ask for
permission
Say no to premature
optimization
1 million operations a
day means 12 ops
per second

30. Thank You
Aaron Gaalswyk
Anjali Raman
Arun Batchu
Arvind Naik
Carl Mueller
Chad Sears
Geoff Lory
Jayaraj Somasekharan
John Murphy
Josh Friberg
Kevin Matheny
Laurie Heavier