Cvcc performance tuning

Web Performance
John McCaffrey Railsperformance.blogspot.com

John McCaffrey: Its all about me
Doing Java since 2000, Rails since 2007
Presented at WindyCityRails
2008: Advanced Firebug and JS unit testing
2009: PDF Generation in Rails
2010: Rails Performance Tuning
Addicted to Performance Tuning
railsperformance.blogspot.com
Feedback: http://spkr8.com/t/4961
@J_McCaffrey
[email_address]
(the slides will be available, with additional references)

What are we gonna cover?
Performance touches a lot of layers
We have a very diverse audience

Measuring Page Load
Fixing page load problems
Analyzing Server Performance
Database issues
Ruby issues
Q&A
Starting off on the right foot

It all started...
def organization_branches
get_organizations .flatten.uniq.select{|org| org.children.size > 0}
end
def organization_leaves
get_organizations .flatten.uniq.select{|org| org.children.size == 0}
end
def load_organizations
branches = current_user .organization_branches leaves = current_user .organization_leaves @branches = branches.to_json(: only => [:id, :name]) @leaves = leaves.to_json(: only => [:id, :name]) @organizations = @criteria .branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end

def organization_branches
get_organizations .flatten.uniq.select{|org| org.children.size > 0}
end
def organization_leaves
get_organizations .flatten.uniq.select{|org| org.children.size == 0}
end
def load_organizations
branches = current_user .organization_branches leaves = current_user .organization_leaves @branches = branches.to_json(: only => [:id, :name]) @leaves = leaves.to_json(: only => [:id, :name]) @organizations = @criteria .branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end
It all started...
2811 queries!
Down to 17!!

And then...
Invoicing issue: If a customer resubmits an order with 2 years, they should be charged $9 but were only being charged $4.50
How common are ROI calculations on software projects?
53626 * $4.50 = $241,317 (just in the last year)

Our theme of the day: Leverage
to achieve the greatest value

Performance and Business metrics: Bing.com
Slow site = revenue drop
1 sec = 2.8% 2 sec = 4.3%

Performance and Business metrics: Speed matters
Amazon: 100 ms delay caused a 1% drop in revenue . Google: 400 ms delay caused a 0.59% decrease in search requests per user. Yahoo!: 400 ms delay caused a 5-9% decrease in traffic . Bing: 2 sec delay caused a 4.3% drop in revenue per user. Mozilla made their download page 2.2 seconds faster and saw an increase of 15.4% in downloads . Google Maps reduced the file volume by 30% and observed a 30% increase in map requests . Netflix enabled gzip on the server; pages became 13-25% faster and saved 50% of traffic volume !

Performance and Business metrics: Shopzilla.com
http://www.phpied.com/the-performance-business-pitch/
Went from 6 sec down to 1.2sec

Fred Wilson: Speed is the most important feature
10 Golden Principles of Successful Web Apps
Fred Wilson, Venture Capitalist www.avc.com
Speed
Instant Utility
Software is media
Less is More
Make it programmable
Make it personal
Make it RESTful
Discoverability
Clean
Playful

OK, so where do we begin?

Give yourself a fighting chance, make a plan
Alois Reitbauer, dynatrace
Anti-patterns
Thinking Scalability something you just sprinkle on later
Guessing, not testing
Ad-hoc, unstructured
Waiting until it hurts, not proactively planning
Thinking that any early planning is Premature optimization
Video on Parleys.com

Premature optimization
"Premature optimization is the root of all evil" -Donald Knuth

Some people really like Donald Knuth

Premature optimization, with DJ DK
like, really, really like him

"Premature optimization is the root of all evil' -Donald Knuth
"Quoting Knuth, as a way to avoid planning for performance issues, is a cop-out" @J_McCaffrey
Don't waste time on making optimization changes until you KNOW they are necessary

If you can not measure it, you can not improve it. — Lord Kelvin

Let's do this the right way
Measure :
Understand the tools to use, at each layer, to measure and monitor performance
Create repeatable tests :
Be able to invoke the system and easily test
A/B, before/after tests, long-term monitoring
Isolate your changes :
One thing at a time
That's it: Measure, Test, and Isolate your changes

Terminology
Load : How much work is there?
Utilization : How much of the resources are in use?
Scalability : How well can we handle the load?
Throughput : How many tasks can be done per unit of time?
Concurrency : How many tasks can we do at once?
Capacity : How big can our throughput go before things fall apart?
Response time (avg) == Latency Requests per second == Throughput

Terminology
Performance:
The resources and Time for a single request
Measured in Response Time
Throughput:
The number requests that can be handled per/time
Measured in Requests per second
Scalability:
Ability to grow and handle more requests, without significantly degrading the experience
Performance != Scalability

Terminology
Let's say: 1 worker that can compute a job in .5 sec Throughput = 2 jobs per sec Latency = .5 sec per job
Performance != Scalability
Adding more workers will not improve the latency
But not having enough workers can increase the response time, due to Queuing and delay

There are performance opportunities at each layer

Waterfall: Twitter.com
Waterfall
When the html is downloaded When the rest of it is loaded
Total time: 3815 ms Dynamic generation: 250 ms (~7%) Static: 3369 ms (~89%) * Will not add to 100%

Waterfall
When the backend is actually doing work

Walmart: Waterfall
Waterfall
Total time: 5520 ms Dynamic: 388 ms (7%) Static: 4580 ms (~82%)

Firebug Netpanel
Simple and repeatable way to measure page load
Usually, less than 20% of the time is spent in fetching the HTML
So how do we optimize the other 80%?

YSlow
Created by Steve Souders
Observes the netpanel data, applies rules and gives you a score.
Deep understanding of how the browser works
Its the best way to have a repeatable, measurement of your site's load performance
http://developer.yahoo.com/yslow

YSlow
Minimize HTTP Requests
Use a Content Delivery Network
Add an Expires or a Cache-Control Header
Gzip Components
Put StyleSheets at the Top
Put Scripts at the Bottom
Avoid CSS Expressions
Make JavaScript and CSS External
Reduce DNS Lookups
Minify JavaScript and CSS
Avoid Redirects
Remove Duplicate Scripts
Configure ETags
Make AJAX Cacheable
Use GET for AJAX Requests
Reduce the Number of DOM Elements
No 404s
Reduce Cookie Size
Use Cookie-Free Domains for Components
Avoid Filters
Do Not Scale Images in HTML
Make favicon.ico Small and Cacheable

Yslow: Created by Steve Souders
http://stevesouders.com/cuzillion/

Google Page speed
Similar to YSlow
includes paint events
separates out ads, trackers from your content
better recommendations and optimizations
1 click to see optimized version of your files
http://code.google.com/speed/page-speed

Google Page speed

Make it repeatable
http://www.showslow.com/
track your site, or your competitors will check every 24hrs for you
webpagetest.org
gtmetrix.org
zoompf.com
loadimpact.com
gomez.com

Summary
Page load is the place to start
Read everything in the Yslow and PageSpeed rules
Encourage other devs to play with YSlow, PageSpeed
Show Webpagetest.org, and zoompf.com to the boss
Now that we have a way to measure page performance, we can think about making changes

Measuring Page Load
Database issues
Ruby issues
Q&A

Isn't everyone already doing this?
Analysis of the Alexa top 1000 sites found:
42% don't gzip
44% have more than 2 css files
56% serve css from a cookied domain
62% don't minify
31% have more than 100k size css
Top 1000 retail sites
50% aren't doing both keep-alive and compression (the 2 easiest things!!)

Compress with Gzip
Can save you 50% to 80% of your bandwidth
Simplest performance improvement you can make
Beware differences in browsers
File types: html, js, css, xml, rss, txt, font, json
Probably the simplest and smartest thing you can do for your app!

Combine and minify javascript and css
Rails
javascript_include_tag :all, :cache => "all"
Asset_packager, Jammit
Load common js libraries from google
http://github.com/rpheath/google_ajax_libraries_api Jsmin or YUICompressor Closure compiler Let's check it out!

Combine and minify javascript and css
Sprites and image optimization
spriteme.org
JqueryUI already gives you sprited images
Image optimization
Smush.it

Expires and Browser Caching
Setting a far future expires tells the browser not to ask for the file
Expires:Mon, 02 Sep 2030 21:23:08 GMT
/images/logo.png?1234567890
Using the query string lets us break the caching by having the browser ask for a new file
There is a problem with using a URL query string this way
http://blog.eliotsykes.com/2010/05/06/why-rails-asset-caching-is-broken/
/images/logo-fp-839b180ff39a24f8d6e0ee70e4c40fed.png

Rails Caching
Take your dynamic content, and make it static
Page
Action
Fragment
Full length tutorials
http://railslab.newrelic.com/
http://railscasts.com/episodes?search=caching
Greg Pollack Aloha on Rails http://vimeo.com/10860860

Measuring Page Load
Database issues
Ruby issues
Q&A

The Stack
You are here

Request-log-analyzer
• http://github.com/wvanbergen/request-log-analyzer • Request distribution per hour • Most requested • HTTP methods • HTTP statuses returned • Rails action cache hits • Request duration • View rendering time • Database time • Process blockers • Failed requests

Rack::Bug

Commercial tools
New Relic
Developer plugin is a good start
Free version doesn’t get you much (Bronze version with heroku)
Works for .NET
Scoutapp.com
Similar to new Relic
Large selection of community plugins

Testing tools
Apache bench ab -n 10 -c 2 http://www.somewhere.com/ Httperf httperf --server localhost --port 3000 --uri / --num-conns 10000 Jmeter yes, its ugly, but its powerful

Measuring Page Load
Database issues
Q&A

Database issues
Common database issues
Bad queries
Not utilizing explain
Inadequate indexes
N+1 queries
Selecting more data than is needed
Inconsistent queries for the same data

Query_reviewer
Common database issues
Bad queries
Not utilizing explain and slow query log
http:// github.com/dsboulder/query_reviewer
Runs explain on all of your queries, outputs to div in page

Rails_indexes
http://github.com/eladmeidar/rails_indexes Foreign key indexes • Columns that need to be sorted • Lookup fields • Columns that are used in a GROUP BY • Rake tasks to find missing indexes. New one I haven’t tried yet http:// github.com/samdanavia/ambitious_query_indexer

Bullet plugin
Bullet plugin • http://github.com/flyerhzm/bullet Help you reduce the number of queries with alerts (and growl).

Slim_scrooge
Be more specific in your select
http://github.com/sdsykes/slim_scrooge
Instruments your code
Observes the usage pattern
Suggests/changes your select statement
When invoked within the exact same context

Ruby Issues
Most are due to memory problems
Slow GC
Not release references
Capturing scope
Profiling will reveal what’s going on
http:// guides.rubyonrails.org/performance_testing.html
If you want to learn everything there is to know about profiling
Aman Gupta & Joe Damato
http:// timetobleed.com /
http:// memprof.com /
They’ve already identified and fixed several memory leaks

Ruby issues: Faster Ruby Libraries
C Extension++ • XML parser http://nokogiri.org/ • JSON parser http://github.com/brianmario/yajl-ruby/ • CSV parser http://www.toastyapps.com/excelsior/ • HTTP client http://github.com/pauldix/typhoeus Date
http:// github.com/jeremyevans/home_run

Take home points
Performance
Performance affects the bottom line
The biggest performance win is usually in improving the load time
Continue to monitor and test your apps performance over time
Gzip, combine and minify to get a big boost
Lack of indexes is likely to be one of your biggest backend issues
Try to stay up to date with Libraries and patches
A well tested codebase is easier to tune!
Upgrading to ruby 1.9 will give you a huge performance boost

Links: great info on performance
Performance
Scaling Rails series http:// railslab.newrelic.com
RailsCasts.com
Velocity Conf http://en.oreilly.com/velocity2010
Google io conf http://code.google.com/events/io/2010/
dynatrace site http:// ajax.dynatrace.com /pages/
http:// www.igvita.com
http:// www.mysqlperformanceblog.com

Wait, I made it through that?
Designing for performance
Watch how they really use it
Multiple tabs?
Back and forth between list and detail view
Great presentation at WindyCityRails2009 by David Eisinger
Optimizing perceived performance
Make it feel fast

Wait, I made it through that?
Architectural musings
Background long running tasks
Email, image processing, reports, feeds
Leverage Rack, Sinatra or Padrino
Parallel processing, Event Machine
Learn from heroku, scalable from the start
Revaluate your cloudy-ness
Great RailsConf presentation by James Golick

Links
Questions? @j_mccaffrey [email_address]

Beyond ySlow: improving page performance outside the initial load
MySql and Postgres Database tuning
Heroku: Building Scalable apps from the start
Tuning performance for Mobile devices
Performance Improvements coming in Rails 3, and 3.1
Speeding up your tests

Questions for you
Architectural musings
Moving to or already on Ruby 1.9?
Serving your content from a separate domain
Cookieless?

Cvcc performance tuning

by John McCaffrey on Nov 13, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 1

Statistics

Cvcc performance tuning — Presentation Transcript