Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Web Performance
John McCaffrey
Railsperformance.blogspot.com
John McCaffrey: Its all about me
• Doing Java since 2000, Rails since 2007
• Presented at WindyCityRails
– 2008: Advanced ...
What are we gonna cover?
• Performance touches a lot of layers
• We have a very diverse audience
Measuring
Page Load
Fixing page
load problems
Analyzing
Server
Performance
Database issues
Ruby issues Q&A
Starting off on...
It all started...
def organization_branches
get_organizations.flatten.uniq.select{|org| org.children.size > 0}
end
def org...
def organization_branches
get_organizations.flatten.uniq.select{|org| org.children.size > 0}
end
def organization_leaves
g...
And then...
Invoicing issue:
If a customer resubmits an order with 2 years,
they should be charged $9 but were only being
...
Our theme of the day: Leverage
to achieve the greatest value
Performance and Business metrics: Bing.com
Slow site = revenue drop
1 sec = 2.8%
2 sec = 4.3%
Performance and Business metrics: Speed matters
Amazon: 100 ms delay caused a 1% drop in revenue.
Google: 400 ms delay cau...
Performance and Business metrics: Shopzilla.com
Went from 6 sec
down to 1.2sec
http://www.phpied.com/the-performance-busin...
Fred Wilson: Speed is the most important feature
10 Golden Principles of Successful Web Apps
Fred Wilson, Venture Capitali...
OK, so where do we begin?
Give yourself a fighting chance, make a plan
Alois Reitbauer, dynatrace
Anti-patterns
• Thinking Scalability something you...
Premature optimization
"Premature optimization is
the root of all evil"
-Donald Knuth
Premature optimization
Some people really like Donald Knuth
Premature optimization, with DJ DK
like, really, really like him
Premature optimization
"Premature optimization is
the root of all evil'
-Donald Knuth
"Quoting Knuth, as a way to
avoid pl...
If you can not measure it,
you can not improve it.
— Lord Kelvin
Let's do this the right way
Measure:
• Understand the tools to use, at each layer, to measure
and monitor performance
Crea...
Terminology
Load:
How much work is
there?
Utilization:
How much of the
resources are in use?
Scalability:
How well can we
...
Terminology
Performance:
• The resources and Time for a single request
• Measured in Response Time
Throughput:
• The numbe...
Terminology
Let's say:
1 worker that can compute a job in .5 sec
Throughput = 2 jobs per sec
Latency = .5 sec per job
Perf...
There are performance opportunities at each layer
Waterfall: Twitter.com
Waterfall
Total time: 3815 ms
Dynamic generation: 250 ms
(~7%)
Static: 3369 ms (~89%)
* Will not ad...
Waterfall
When the backend is
actually doing work
Walmart: Waterfall
Waterfall
Total time: 5520 ms
Dynamic: 388 ms (7%)
Static: 4580 ms (~82%)
Firebug Netpanel
Simple and repeatable way to measure page load
Usually, less than 20% of the time is spent in fetching th...
YSlow
• Created by Steve Souders
• Observes the netpanel data, applies rules and gives you a
score.
• Deep understanding o...
YSlow
• Minimize HTTP Requests
• Use a Content Delivery Network
• Add an Expires or a Cache-Control Header
• Gzip Componen...
Yslow: Created by Steve Souders
http://stevesouders.com/cuzillion/
Google Page speed
• Similar to YSlow
• includes paint events
• separates out ads, trackers from your content
• better reco...
Google Page speed
Google Page speed
Make it repeatable
http://www.showslow.com/
track your site, or your competitors
will check every 24hrs for you
• webpaget...
Summary
• Page load is the place to start
• Read everything in the Yslow and PageSpeed rules
• Encourage other devs to pla...
Measuring
Page Load
Fixing page
load problems
Analyzing
Server
Performance
Database issues
Ruby issues Q&A
Starting off on...
Isn't everyone already doing this?
Analysis of the Alexa top 1000 sites found:
• 42% don't gzip
• 44% have more than 2 css...
Compress with Gzip
• Can save you 50% to 80% of your bandwidth
• Simplest performance improvement you can make
• Beware di...
Combine and minify javascript and css
Rails
• javascript_include_tag :all, :cache => "all"
• Asset_packager, Jammit
Load c...
Combine and minify javascript and css
Sprites and image optimization
• spriteme.org
• JqueryUI already gives you sprited i...
Expires and Browser Caching
Setting a far future expires tells the browser not to ask for the file
Expires:Mon, 02 Sep 203...
Rails Caching
Take your dynamic content, and make it static
• Page
• Action
• Fragment
Full length tutorials
• http://rail...
Measuring
Page Load
Fixing page
load problems
Analyzing
Server
Performance
Database issues
Ruby issues Q&A
Starting off on...
The Stack
You are here
Request-log-analyzer
• http://github.com/wvanbergen/request-log-analyzer
• Request distribution per hour
• Most requested
...
Rack::Bug
Rack::Bug
Commercial tools
New Relic
• Developer plugin is a good start
• Free version doesn’t get you much (Bronze version with her...
Testing tools
Apache bench
ab -n 10 -c 2 http://www.somewhere.com/
Httperf
httperf --server localhost --port 3000 --uri / ...
Measuring
Page Load
Fixing page
load problems
Analyzing
Server
Performance
Database issues
Q&A
Starting off on
the right f...
Database issues
Common database issues
• Bad queries
• Not utilizing explain
• Inadequate indexes
• N+1 queries
• Selectin...
Query_reviewer
Common database issues
• Bad queries
• Not utilizing explain and slow query log
• http://github.com/dsbould...
Rails_indexes
http://github.com/eladmeidar/rails_indexes
Foreign key indexes
• Columns that need to be sorted
• Lookup fie...
Bullet plugin
Bullet plugin
• http://github.com/flyerhzm/bullet
Help you reduce the number of queries
with alerts (and gro...
Slim_scrooge
Be more specific in your select
• http://github.com/sdsykes/slim_scrooge
• Instruments your code
• Observes t...
Ruby Issues
Most are due to memory problems
• Slow GC
• Not release references
• Capturing scope
• Profiling will reveal w...
Ruby issues: Faster Ruby Libraries
C Extension++
• XML parser http://nokogiri.org/
• JSON parser http://github.com/brianma...
Take home points
Performance
• Performance affects the bottom line
• The biggest performance win is usually in improving t...
Links: great info on performance
Performance
• Scaling Rails series http://railslab.newrelic.com
• RailsCasts.com
• Veloci...
Wait, I made it through that?
Designing for performance
• Watch how they really use it
• Multiple tabs?
• Back and forth b...
Wait, I made it through that?
Architectural musings
•Background long running tasks
• Email, image processing, reports, fee...
Links
Questions?
@j_mccaffrey
Railsperformance@gmail.com
Beyond ySlow:
improving page
performance
outside the
initial load
MySql and
Postgres
Database tuning
Heroku: Building
Scal...
Questions for you
Architectural musings
• Moving to or already on Ruby 1.9?
• Serving your content from a separate domain
...
Upcoming SlideShare
Loading in …5
×

Cvcc performance tuning

1,918 views

Published on

Web Performance tuning presentation given at http://www.chippewavalleycodecamp.com/

Covers basic http flow, measuring performance, common changes to improve performance now, and several tools and techniques you can use now.

Published in: Technology, Design
  • Be the first to comment

Cvcc performance tuning

  1. 1. Web Performance John McCaffrey Railsperformance.blogspot.com
  2. 2. John McCaffrey: Its all about me • Doing Java since 2000, Rails since 2007 • Presented at WindyCityRails – 2008: Advanced Firebug and JS unit testing – 2009: PDF Generation in Rails – 2010: Rails Performance Tuning • Addicted to Performance Tuning • railsperformance.blogspot.com • Feedback: http://spkr8.com/t/4961 @J_McCaffrey railsperformance@gmail.com (the slides will be available, with additional references)
  3. 3. What are we gonna cover? • Performance touches a lot of layers • We have a very diverse audience
  4. 4. Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Ruby issues Q&A Starting off on the right foot
  5. 5. It all started... def organization_branches get_organizations.flatten.uniq.select{|org| org.children.size > 0} end def organization_leaves get_organizations.flatten.uniq.select{|org| org.children.size == 0} end def load_organizations branches = current_user.organization_branches leaves = current_user.organization_leaves @branches = branches.to_json(:only => [:id, :name]) @leaves = leaves.to_json(:only => [:id, :name]) @organizations = @criteria.branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end
  6. 6. def organization_branches get_organizations.flatten.uniq.select{|org| org.children.size > 0} end def organization_leaves get_organizations.flatten.uniq.select{|org| org.children.size == 0} end def load_organizations branches = current_user.organization_branches leaves = current_user.organization_leaves @branches = branches.to_json(:only => [:id, :name]) @leaves = leaves.to_json(:only => [:id, :name]) @organizations = @criteria.branch ? leaves.collect {|o| [ o.name.titleize, o.id ] }.sort : branches.collect {|o| [ o.name.titleize, o.id ] }.sort end It all started... 2811 queries! Down to 17!!
  7. 7. And then... Invoicing issue: If a customer resubmits an order with 2 years, they should be charged $9 but were only being charged $4.50 How common are ROI calculations on software projects? 53626 * $4.50 = $241,317 (just in the last year)
  8. 8. Our theme of the day: Leverage to achieve the greatest value
  9. 9. Performance and Business metrics: Bing.com Slow site = revenue drop 1 sec = 2.8% 2 sec = 4.3%
  10. 10. Performance and Business metrics: Speed matters Amazon: 100 ms delay caused a 1% drop in revenue. Google: 400 ms delay caused a 0.59% decrease in search requests per user. Yahoo!: 400 ms delay caused a 5-9% decrease in traffic. Bing: 2 sec delay caused a 4.3% drop in revenue per user. Mozilla made their download page 2.2 seconds faster and saw an increase of 15.4% in downloads. Google Maps reduced the file volume by 30% and observed a 30% increase in map requests. Netflix enabled gzip on the server; pages became 13-25% faster and saved 50% of traffic volume!
  11. 11. Performance and Business metrics: Shopzilla.com Went from 6 sec down to 1.2sec http://www.phpied.com/the-performance-business-pitch/
  12. 12. Fred Wilson: Speed is the most important feature 10 Golden Principles of Successful Web Apps Fred Wilson, Venture Capitalist www.avc.com •Speed• Instant Utility • Software is media • Less is More • Make it programmable • Make it personal • Make it RESTful • Discoverability • Clean • Playful
  13. 13. OK, so where do we begin?
  14. 14. Give yourself a fighting chance, make a plan Alois Reitbauer, dynatrace Anti-patterns • Thinking Scalability something you just sprinkle on later • Guessing, not testing • Ad-hoc, unstructured • Waiting until it hurts, not proactively planning • Thinking that any early planning is Premature optimization Video on Parleys.com
  15. 15. Premature optimization "Premature optimization is the root of all evil" -Donald Knuth
  16. 16. Premature optimization Some people really like Donald Knuth
  17. 17. Premature optimization, with DJ DK like, really, really like him
  18. 18. Premature optimization "Premature optimization is the root of all evil' -Donald Knuth "Quoting Knuth, as a way to avoid planning for performance issues, is a cop-out" @J_McCaffrey Don't waste time on making optimization changes until you KNOW they are necessary
  19. 19. If you can not measure it, you can not improve it. — Lord Kelvin
  20. 20. Let's do this the right way Measure: • Understand the tools to use, at each layer, to measure and monitor performance Create repeatable tests: • Be able to invoke the system and easily test • A/B, before/after tests, long-term monitoring Isolate your changes: • One thing at a time That's it: Measure, Test, and Isolate your changes
  21. 21. Terminology Load: How much work is there? Utilization: How much of the resources are in use? Scalability: How well can we handle the load? Throughput: How many tasks can be done per unit of time? Concurrency: How many tasks can we do at once? Capacity: How big can our throughput go before things fall apart? Response time (avg) == Latency Requests per second == Throughput
  22. 22. Terminology Performance: • The resources and Time for a single request • Measured in Response Time Throughput: • The number requests that can be handled per/time • Measured in Requests per second Scalability: • Ability to grow and handle more requests, without significantly degrading the experience Performance != Scalability
  23. 23. Terminology Let's say: 1 worker that can compute a job in .5 sec Throughput = 2 jobs per sec Latency = .5 sec per job Performance != Scalability Adding more workers will not improve the latency But not having enough workers can increase the response time, due to Queuing and delay
  24. 24. There are performance opportunities at each layer
  25. 25. Waterfall: Twitter.com Waterfall Total time: 3815 ms Dynamic generation: 250 ms (~7%) Static: 3369 ms (~89%) * Will not add to 100% When the html is downloaded When the rest of it is loaded
  26. 26. Waterfall When the backend is actually doing work
  27. 27. Walmart: Waterfall Waterfall Total time: 5520 ms Dynamic: 388 ms (7%) Static: 4580 ms (~82%)
  28. 28. Firebug Netpanel Simple and repeatable way to measure page load Usually, less than 20% of the time is spent in fetching the HTML So how do we optimize the other 80%?
  29. 29. YSlow • Created by Steve Souders • Observes the netpanel data, applies rules and gives you a score. • Deep understanding of how the browser works • Its the best way to have a repeatable, measurement of your site's load performance http://developer.yahoo.com/yslow
  30. 30. YSlow • Minimize HTTP Requests • Use a Content Delivery Network • Add an Expires or a Cache-Control Header • Gzip Components • Put StyleSheets at the Top • Put Scripts at the Bottom • Avoid CSS Expressions • Make JavaScript and CSS External • Reduce DNS Lookups • Minify JavaScript and CSS • Avoid Redirects • Remove Duplicate Scripts • Configure ETags • Make AJAX Cacheable • Use GET for AJAX Requests • Reduce the Number of DOM Elements • No 404s • Reduce Cookie Size • Use Cookie-Free Domains for Components • Avoid Filters • Do Not Scale Images in HTML • Make favicon.ico Small and Cacheable
  31. 31. Yslow: Created by Steve Souders http://stevesouders.com/cuzillion/
  32. 32. Google Page speed • Similar to YSlow • includes paint events • separates out ads, trackers from your content • better recommendations and optimizations • 1 click to see optimized version of your files http://code.google.com/speed/page-speed
  33. 33. Google Page speed
  34. 34. Google Page speed
  35. 35. Make it repeatable http://www.showslow.com/ track your site, or your competitors will check every 24hrs for you • webpagetest.org • gtmetrix.org • zoompf.com • loadimpact.com • gomez.com
  36. 36. Summary • Page load is the place to start • Read everything in the Yslow and PageSpeed rules • Encourage other devs to play with YSlow, PageSpeed • Show Webpagetest.org, and zoompf.com to the boss Now that we have a way to measure page performance, we can think about making changes
  37. 37. Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Ruby issues Q&A Starting off on the right foot
  38. 38. Isn't everyone already doing this? Analysis of the Alexa top 1000 sites found: • 42% don't gzip • 44% have more than 2 css files • 56% serve css from a cookied domain • 62% don't minify • 31% have more than 100k size css Top 1000 retail sites 50% aren't doing both keep-alive and compression (the 2 easiest things!!)
  39. 39. Compress with Gzip • Can save you 50% to 80% of your bandwidth • Simplest performance improvement you can make • Beware differences in browsers • File types: html, js, css, xml, rss, txt, font, json Probably the simplest and smartest thing you can do for your app!
  40. 40. Combine and minify javascript and css Rails • javascript_include_tag :all, :cache => "all" • Asset_packager, Jammit Load common js libraries from google http://github.com/rpheath/google_ajax_libraries_api Jsmin or YUICompressor Closure compiler Let's check it out!
  41. 41. Combine and minify javascript and css Sprites and image optimization • spriteme.org • JqueryUI already gives you sprited images Image optimization • Smush.it
  42. 42. Expires and Browser Caching Setting a far future expires tells the browser not to ask for the file Expires:Mon, 02 Sep 2030 21:23:08 GMT /images/logo.png?1234567890 Using the query string lets us break the caching by having the browser ask for a new file There is a problem with using a URL query string this way http://blog.eliotsykes.com/2010/05/06/why-rails-asset-caching-is-broken/ /images/logo-fp-839b180ff39a24f8d6e0ee70e4c40fed.png
  43. 43. Rails Caching Take your dynamic content, and make it static • Page • Action • Fragment Full length tutorials • http://railslab.newrelic.com/ • http://railscasts.com/episodes?search=caching • Greg Pollack Aloha on Rails http://vimeo.com/10860860
  44. 44. Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Ruby issues Q&A Starting off on the right foot
  45. 45. The Stack You are here
  46. 46. Request-log-analyzer • http://github.com/wvanbergen/request-log-analyzer • Request distribution per hour • Most requested • HTTP methods • HTTP statuses returned • Rails action cache hits • Request duration • View rendering time • Database time • Process blockers • Failed requests
  47. 47. Rack::Bug
  48. 48. Rack::Bug
  49. 49. Commercial tools New Relic • Developer plugin is a good start • Free version doesn’t get you much (Bronze version with heroku) • Works for .NET • Scoutapp.com • Similar to new Relic • Large selection of community plugins
  50. 50. Testing tools Apache bench ab -n 10 -c 2 http://www.somewhere.com/ Httperf httperf --server localhost --port 3000 --uri / --num-conns 10000 Jmeter yes, its ugly, but its powerful
  51. 51. Measuring Page Load Fixing page load problems Analyzing Server Performance Database issues Q&A Starting off on the right foot
  52. 52. Database issues Common database issues • Bad queries • Not utilizing explain • Inadequate indexes • N+1 queries • Selecting more data than is needed • Inconsistent queries for the same data
  53. 53. Query_reviewer Common database issues • Bad queries • Not utilizing explain and slow query log • http://github.com/dsboulder/query_reviewer • Runs explain on all of your queries, outputs to div in page
  54. 54. Rails_indexes http://github.com/eladmeidar/rails_indexes Foreign key indexes • Columns that need to be sorted • Lookup fields • Columns that are used in a GROUP BY • Rake tasks to find missing indexes. New one I haven’t tried yet http://github.com/samdanavia/ambitious_query_indexer
  55. 55. Bullet plugin Bullet plugin • http://github.com/flyerhzm/bullet Help you reduce the number of queries with alerts (and growl).
  56. 56. Slim_scrooge Be more specific in your select • http://github.com/sdsykes/slim_scrooge • Instruments your code • Observes the usage pattern • Suggests/changes your select statement • When invoked within the exact same context
  57. 57. Ruby Issues Most are due to memory problems • Slow GC • Not release references • Capturing scope • Profiling will reveal what’s going on • http://guides.rubyonrails.org/performance_testing.html If you want to learn everything there is to know about profiling • Aman Gupta & Joe Damato • http://timetobleed.com/ • http://memprof.com/ • They’ve already identified and fixed several memory leaks
  58. 58. Ruby issues: Faster Ruby Libraries C Extension++ • XML parser http://nokogiri.org/ • JSON parser http://github.com/brianmario/yajl-ruby/ • CSV parser http://www.toastyapps.com/excelsior/ • HTTP client http://github.com/pauldix/typhoeus Date http://github.com/jeremyevans/home_run
  59. 59. Take home points Performance • Performance affects the bottom line • The biggest performance win is usually in improving the load time • Continue to monitor and test your apps performance over time • Gzip, combine and minify to get a big boost • Lack of indexes is likely to be one of your biggest backend issues • Try to stay up to date with Libraries and patches • A well tested codebase is easier to tune! • Upgrading to ruby 1.9 will give you a huge performance boost
  60. 60. Links: great info on performance Performance • Scaling Rails series http://railslab.newrelic.com • RailsCasts.com • Velocity Conf http://en.oreilly.com/velocity2010 • Google io conf http://code.google.com/events/io/2010/ • dynatrace site http://ajax.dynatrace.com/pages/ • http://www.igvita.com • http://www.mysqlperformanceblog.com
  61. 61. Wait, I made it through that? Designing for performance • Watch how they really use it • Multiple tabs? • Back and forth between list and detail view • Great presentation at WindyCityRails2009 by David Eisinger • Optimizing perceived performance • Make it feel fast
  62. 62. Wait, I made it through that? Architectural musings •Background long running tasks • Email, image processing, reports, feeds •Leverage Rack, Sinatra or Padrino •Parallel processing, Event Machine •Learn from heroku, scalable from the start •Revaluate your cloudy-ness • Great RailsConf presentation by James Golick
  63. 63. Links Questions? @j_mccaffrey Railsperformance@gmail.com
  64. 64. Beyond ySlow: improving page performance outside the initial load MySql and Postgres Database tuning Heroku: Building Scalable apps from the start Tuning performance for Mobile devices Performance Improvements coming in Rails 3, and 3.1 Speeding up your tests
  65. 65. Questions for you Architectural musings • Moving to or already on Ruby 1.9? • Serving your content from a separate domain • Cookieless?

×