Capacity and Performance of Rails Applications

Performance and Capacity of Rails Applications Jim Van Fleet @bigﬂeet

$1B DATA CENTER PROJECT MAIDEN, NC DATE CLIENT Q4 2010 APPLE COMPUTER

$600M DATA CENTER PROJECT LENOIR, NC DATE CLIENT MAY, 2008 GOOGLE, INC.

$450M DATA CENTER PROJECT RUTHERFORD, NC DATE CLIENT Q2, 2012 FACEBOOK

What’s Running In Them? Apple: WebObjects (Java) Google: AppEngine (Java, Python) Facebook: PHP, C++

What’s Running In Them? Apple: WebObjects (Java) Google: AppEngine (Java, Python) Facebook: PHP, C++ TWITTER: “CUSTOM-BUILT” IN UTAH

So what?

So what? Big data.

ONE GIANT COMPUTER PROJECT LENOIR, NC DATE CLIENT MAY, 2008 GOOGLE, INC.

TWO GIANT COMPUTERS PROJECT LENOIR, NC DATE CLIENT MAY, 2008 GOOGLE, INC.

How do you split your data?

Big Data Approaches BigTable and Dynamo Google and Amazon Cassandra and Riak Sharding Basically everything else (MySQL, MongoDB)

Think about it!

Google, Amazon, Microsoft built their own tools. CHRISTOPHER BROWN VP, ENGINEERING AT OPSCODE

Google, Amazon, Microsoft built their own tools. Almost everyone else is here... CHRISTOPHER BROWN VP, ENGINEERING AT OPSCODE

Google, Amazon, Microsoft built their own tools. Almost everyone else is here... ... inexperienced or poorly equipped for the world in which we now operate. CHRISTOPHER BROWN VP, ENGINEERING AT OPSCODE

Plan for failure.

Being able to recover quickly from failure is more important than having failures less often. JOHN ALLSPAW VP, OPERATIONS AT ETSY (EX-FLICKR)

Being able to recover quickly from failure is more important than having failures less often. •NOT SAYING FAILURE SHOULD BE AN ACCEPTABLE CONDITION JOHN ALLSPAW VP, OPERATIONS AT ETSY (EX-FLICKR)

Being able to recover quickly from failure is more important than having failures less often. •NOT SAYING FAILURE SHOULD BE AN ACCEPTABLE CONDITION •SOME TYPES OF FAILURES SHOULD NEVER HAPPEN, LIKE ACCIDENTAL DATA LOSS JOHN ALLSPAW VP, OPERATIONS AT ETSY (EX-FLICKR)

“DUE TO THE CONFIGURATION ERROR, GITHUB’S PRODUCTION DATABASE WAS DESTROYED THEN RECREATED.” CHRIS WANSTRATH GITHUB, COFOUNDER

What would you do if your production DB was dropped?

What would you do if your database VM failed?

What would you do if one of your other VM’s failed?

Leaning Into It: Rails Fails Edition Catch an UnrecoverableError in application_controller. Wrap all connection errors. (MySQL, memcached, redis, etc.) Redundancy Plan for degraded service See if you can 200 without any supporting services running.

Leaning Into It: Rails Fails Edition Test Harness to stress your client code Load testing to stress your server code

Failure tolerance is an element of capacity.

Capacity != Performance

In Rails, your capacity will be deﬁned by your stack and VMs.

In Rails, your capacity will be deﬁned by your stack and VMs. UNLESS YOU SCREW IT UP

Platform As A Service

What is “the Cloud?”

If we're maxed out of one of the resources we need to add No software can use capacity, attempting to spin system resources that up a new node is only going to aren't there make the situation worse. JAMES GOLICK PARTNER, PROTOSE INC.

Make sure you’re ready for the cloud, whatever that means to you.

In Rails, your performance will be deﬁned by your bottlenecks.

Don’t “Design By Laptop”

Bottlenecks CPU RAM Network IO Disk IO

Bottlenecks: CPU You win!

Bottlenecks: CPU You win at winning!

Bottlenecks: RAM Swapping is a special case of a Disk IO ActiveRecord XML DOM Parsing Watch out for your concurrency level Be aware of passenger+REE Write high-concurrency endpoints in Sinatra

Bottlenecks: Net IO Timeouts 3rd party APIs (yes, including S3) Waiting on the database

Bottlenecks: Net IO MySQL edition Learn what explain does. Indexing avoids full-table scans Indexes make reads faster, writes are slower Indexes can be on multiple columns MySQL conﬁguration is very important

Bottlenecks: Net IO About Caching Use caching to avoid expensive data gathering. Page caching uses your web tier to avoid Rails Action caching does hit Rails Fragment caching can avoid speciﬁc DB calls by rendering an HTML fragment. The Peepcode is deﬁnitive on this subject. Memcached or File System

Bottlenecks: Net IO Web Tier Ensure your static assets aren’t served by your Rails app server. 404 in Rails is expensive Defer expensive work with dj or resque

Bottlenecks: Disk IO Virtual memory Shared ﬁlesystems s3fs/FUSE is unsupported everywhere NFS/GFS are always dangerous SSD + RAM are helpful

Bottlenecks: Diagnosis + Treatment NewRelic Aman Gupta’s “Debugging Ruby”

Questions?

Thanks! BEER TIME!

Capacity and Performance of Rails Applications

by Jim Van Fleet on Nov 17, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 48

Statistics

Capacity and Performance of Rails Applications — Presentation Transcript