Windy cityrails performance_tuning

Rails Performance John McCaffrey Railsperformance.blogspot.com

John McCaffrey: Its all about me
Doing Rails since 2007
3rd time presenting at WindyCityRails
2008: Advanced Firebug and JS unit testing
2009: PDF Generation in Rails
2010: Rails Performance Tuning
Addicted to Performance Tuning
railsperformance.blogspot.com
Feedback: spkr8.com/t/4413 Or bit.ly/PerfR8
@J_McCaffrey [email_address] (the slides will be available, with tons of extra references)

What are we gonna cover?
Performance touches a lot of layers
We have a very diverse audience

Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Ruby issues Q&A Starting off on the right foot

It all started... def organization_branches get_organizations .flatten.uniq.select{|org| org.children.size > 0} end def organization_leaves get_organizations .flatten.uniq.select{|org| org.children.size == 0} end def load_organizations branches = current_user .organization_branches leaves = current_user .organization_leaves @branches = branches.to_json(: only => [:id, :name]) @leaves = leaves.to_json(: only => [:id, :name]) @organizations = @criteria .branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end

def organization_branches get_organizations .flatten.uniq.select{|org| org.children.size > 0} end def organization_leaves get_organizations .flatten.uniq.select{|org| org.children.size == 0} end def load_organizations branches = current_user .organization_branches leaves = current_user .organization_leaves @branches = branches.to_json(: only => [:id, :name]) @leaves = leaves.to_json(: only => [:id, :name]) @organizations = @criteria .branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end It all started... 2811 queries! Down to 17!!

And then... Invoicing issue: If a customer resubmits an order with 2 years, they should be charged $9 but were only being charged $4.50 Do people ever calculate the ROI on software projects? 53626 * $4.50 = $241,317 (just in the last year)

Our theme of the day: Leverage Multiplying the power of your efforts, to achieve the greatest value

Performance and Business metrics: Bing.com Slower site = revenue drop 1 sec = 2.8% 2 sec = 4.3%

Performance and Business metrics: Shopzilla.com http://www.phpied.com/the-performance-business-pitch/ Went from 6 sec down to 1.2sec

Fred Wilson: Speed is the most important feature
10 Golden Principles of Successful Web Apps
Fred Wilson, Venture Capitalist www.avc.com
Speed
Instant Utility
Software is media
Less is More
Make it programmable
Make it personal
Make it RESTful
Discoverability
Clean
Playful

OK, so where do we begin?

Give yourself a fighting chance, make a plan
Alois Reitbauer, dynatrace
Anti-patterns
Thinking Scalability something you just sprinkle on later
Guessing, not testing
Ad-hoc, unstructured
Waiting until it hurts, not proactively planning
Thinking that any early planning is Premature optimization
Video on wiki.parleys.com

Premature optimization "Premature optimization is the root of all evil" -Donald Knuth

Premature optimization Some people really like Donald Knuth

Premature optimization, with DJ DK like, really, really like him

Premature optimization "Premature optimization is the root of all evil' -Donald Knuth "Quoting Knuth, as a way to avoid planning for performance issues, is a cop-out" @J_McCaffrey "Don't waste time on making optimization changes until you KNOW they are necessary"

If you can not measure it, you can not improve it. — Lord Kelvin

Let's do this the right way
Measure:
Understand the tools to use, at each layer, to measure and monitor performance
Create repeatable tests:
Be able to invoke the system and easily test
Isolate your changes:
One thing at a time
That's it: Measure, Test, Isolated changes

Terminology Response time (avg) == Latency Requests per second == Throughput Load : How much work is there? Utilization : How much of the resources are in use? Scalability : How well can we handle the load? Throughput : How many tasks can be done per unit of time? Concurrency : How many tasks can we do at once? Capacity : How big can our throughput go before things fall apart?

Terminology
Performance:
The resources and Time for a single request
Measured in Response Time
Throughput:
The number requests that can be handled per/time
Measured in Requests per second
Scalability:
Ability to grow and handle more requests, without significantly degrading the experience
Performance != Scalability

Terminology Let's say: 1 worker that can compute a job in .5 sec Throughput = 2 jobs per sec Latency = .5 sec per job Performance != Scalability Adding more workers will not improve the latency But not having enough workers can increase the response time, due to Queuing and delay

Resources: Things that we want to measure System Business Time Cpu Memory Network Disk Money Development time QA and testing time User time

There are performance opportunities at each layer

Waterfall: Twitter.com Waterfall Total time: 3815 ms Dynamic generation: 250 ms (~7%) Static: 3369 ms (~89%) * Will not add to 100%

Waterfall When the backend is actually doing work

Walmart: Waterfall Waterfall Total time: 5520 ms Dynamic generation: 388 ms (~7%) Static: 4580 ms (~82%)

Firebug Netpanel Simple and repeatable way to measure page load Usually, less than 20% of the time is spent in fetching the HTML So how do we optimize the other 80%?

YSlow
Created by Steve Souders
Observes the netpanel data, applies rules and gives you a score.
Deep understanding of how the browser works
Its the best way to have a repeatable, measurement of your site's load performance
http://developer.yahoo.com/yslow/

YSlow
Minimize HTTP Requests
Use a Content Delivery Network
Add an Expires or a Cache-Control Header
Gzip Components
Put StyleSheets at the Top
Put Scripts at the Bottom
Avoid CSS Expressions
Make JavaScript and CSS External
Reduce DNS Lookups
Minify JavaScript and CSS
Avoid Redirects
Remove Duplicate Scripts
Configure ETags
Make AJAX Cacheable
Use GET for AJAX Requests
Reduce the Number of DOM Elements
No 404s
Reduce Cookie Size
Use Cookie-Free Domains for Components
Avoid Filters
Do Not Scale Images in HTML
Make favicon.ico Small and Cacheable

Yslow: Created by Steve Souders http://stevesouders.com/cuzillion/

Google Page speed
Similar to YSlow
includes paint events
separates out ads, trackers from your content
better recommendations and optimizations
1 click to see optimized version of your files
http://code.google.com/speed/page-speed

Google Page speed

Make it repeatable
http://www.showslow.com/
track your site, or your competitors
will check every 24hrs for you
webpagetest.org
gtmetrix.org
zoompf.com
loadimpact.com
gomez.com

Summary
Page load is the place to start
Read everything in the Yslow and PageSpeed rules
Encourage other devs to play with YSlow, PageSpeed
Show Webpagtest.org, and zoompf.com
Now that we have a way to measure page performance, we can think about making changes

Isn't everyone already doing this?
Analysis of the Alexa top 1000 sites found:
42% don't gzip
44% have more than 2 css files
56% serve css from a cookied domain
62% don't minify
31% have more than 100k size css

Compress with Gzip
Can save you 50% to 80% of your bandwidth
Simplest performance improvement you can make
Beware differences in browsers
File types: html, js, css, xml, rss, txt, font, json
Probably the simplest and smartest thing you can do for your app!

Combine and minify javascript and css
Jsmin or YUICompressor, or Sprockets
Asset_packager
javascript_include_tag :cache => "cache/all"
Load common js libraries from google
http://github.com/rpheath/google_ajax_libraries_api
Sprites and image optimzation
spriteme.org
JqueryUI already gives you sprited images
Image optimization
Smush.it

Expires and Browser Caching Setting a far future expires tells the browser not to ask for the file Using the query string lets us break the caching by having the browser ask for a new file There is a problem with using a URL query string this way http://blog.eliotsykes.com/2010/05/06/why-rails-asset-caching-is-broken/

Rails Caching
Take your dynamic content, and make it static
Page
Action
Fragment
Full length tutorials
http://railslab.newrelic.com/
http://railscasts.com/episodes?search=caching
Greg Pollack Aloha on Rails http://vimeo.com/10860860

The Stack You are here

Request-log-analyzer • http://github.com/wvanbergen/request-log-analyzer • Request distribution per hour • Most requested • HTTP methods • HTTP statuses returned • Rails action cache hits • Request duration • View rendering time • Database time • Process blockers • Failed requests

Rack::Bug

Commercial tools
New Relic
Developer plugin is a good start
Free version doesn’t get you much
Bronze version with heroku
Scoutapp.com
Similar to new Relic
Large selection of community plugins

Testing tools Apache bench ab -n 10 -c 2 http://www.somewhere.com/ Httperf httperf --server localhost --port 3000 --uri / --num-conns 10000 Jmeter yes, its ugly, but its powerful

Database issues
Common database issues
Bad queries
Not utilizing explain
Inadequate indexes
N+1 queries
Selecting more data than is needed
Inconsistent queries for the same data

Query_reviewer
Common database issues
Bad queries
Not utilizing explain
http:// github.com/dsboulder/query_reviewer
Runs explain on all of your queries, outputs to div in page

Rails_indexes http://github.com/eladmeidar/rails_indexes Foreign key indexs • Columns that need to be sorted • Lookup fields • Columns that are used in a GROUP BY • Rake tasks to find missing indexes. New one I haven’t tried yet http:// github.com/samdanavia/ambitious_query_indexer

Bullet plugin Bullet plugin • http://github.com/flyerhzm/bullet Help you reduce the number of queries with alerts (and growl).

Slim_scrooge
Be more specific in your select
http://github.com/sdsykes/slim_scrooge
Instruments your code
Observes the usage pattern
Suggests/changes your select statement
When invoked within the exact same context

Ruby Issues
Most are due to memory problems
Slow GC
Not release references
Capturing scope
Profiling will reveal what’s going on
http:// guides.rubyonrails.org/performance_testing.html
If you want to learn everything there is to know about profiling
Aman Gupta
Joe Damato
http:// timetobleed.com /
http:// memprof.com /
They’ve already identified and fixed several memory leaks

Ruby issues: Faster Ruby Libraries C Extension++ • XML parser http://nokogiri.org/ • JSON parser http://github.com/brianmario/yajl-ruby/ • CSV parser http://www.toastyapps.com/excelsior/ • HTTP client http://github.com/pauldix/typhoeus Date http:// github.com/jeremyevans/home_run

Take home points
Performance
Performance affects the bottom line
The biggest performance win is usually in improving the load time
Continue to monitor and test your apps performance over time
Gzip, combine and minify to get a big boost
Lack of indexes is likely to be one of your biggest backend issues
Try to stay up to date with Gems, Plugins, and latest Rails
Upgrading to ruby 1.9 will give you a huge performance boost

Links: great info on performance
Performance
Scaling Rails series http:// railslab.newrelic.com /
RailsCasts.com
Velocity Conference http://en.oreilly.com/velocity2010
google io conference http://code.google.com/events/io/2010/
dynatrace site http:// ajax.dynatrace.com /pages/
http:// www.igvita.com /
http:// www.mysqlperformanceblog.com /

Links Questions? @j_mccaffrey [email_address]

Beyond ySlow: improving page performance outside the initial load MySql and Postgres Database tuning Heroku: Building Scalable apps from the start Tuning performance for Mobile devices Performance Improvements coming in Rails 3, and 3.1 Speeding up your tests

Wait, I made it through that?
Architectural musings
Background long running tasks
Email, image processing, reports, feeds
Leverage Rack, Sinatra or Padrino
Parallel processing, Event Machine
Learn from heroku, scalable from the start
Revaluate your cloudy-ness
Great RailsConf presentation by James Golick

Wait, I made it through that?
Designing for performance
Watch how they really use it
Multiple tabs?
Back and forth between list and detail view
Great presentation at WindyCityRails2009 by David Eisinger
Optimizing perceived performance
Make it feel fast

Questions for you
Architectural musings
Moving to or already on Ruby 1.9?
Serving your content from a separate domain?
Cookieless?

http://railsperformance.blogspot.com	680
http://www.railsperformance.com	173
http://speakerrate.com	34
http://feeds.feedburner.com	7
http://coderwall.com	1

Windy cityrails performance_tuning

by John McCaffrey

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 895

Statistics

Windy cityrails performance_tuning Presentation Transcript