6 tips for improving ruby performance

7,980 views
6,877 views

Published on

This presentation was prepared for a Webcast where John Yerhot, Engine Yard US Support Lead, and Chris Kelly, Technical Evangelist at New Relic discussed how you can scale and improve the performance of your Ruby web apps. They shared detailed guidance on issues like:

Caching strategies
Slow database queries
Background processing
Profiling Ruby applications
Picking the right Ruby web server
Sharding data
Attendees will learn how to:

Gain visibility on site performance
Improve scalability and uptime
Find and fix key bottlenecks

See the on-demand replay:

http://pages.engineyard.com/6TipsforImprovingRubyApplicationPerformance.html

Published in: Technology

6 tips for improving ruby performance

  1. In Today’s Session You Will Learn howto:•  Gain visibility on site performance•  Improve scalability and uptime•  Find and fix key bottlenecks
  2. New Relic + Engine Yard•  Database•  Web Servers•  Caching•  Background Processing
  3. Web Request Overview
  4. Web Application Overview
  5. DATABASE
  6. Database PerformanceLazy loading associated data can quicklylead to an N+1 query problem.ORMs (ActiveRecord, DataMapper, etc.) make it easy toget our data but also make it easy to forget to optimize andrefactor.N+1 problems are hard to spot in development since youare working with limited data sets.
  7. N+1 Query Creep# app/models/customer.rbclass Customer < ActiveRecord::Base has_many :addressesend# app/models/address.rbclass Address < ActiveRecord::Base belongs_to :customerend# app/controllers/customers_controller.rbclass CustomersController < ApplicationController def index @customers = Customer.all endend# app/views/customers/index.html.erb<% @customers.each do |customer| %> <%= content_tag :h1, customer.name %><% end %>
  8. N+1 Query Creep# app/views/customers/index.html.erb<% @customers.each do |customer| %> <%= content_tag :h1, customer.name %> <%= content_tag :h2, customer.addresses.first.city %><% end %>If @customers has 100 records, youll have 101 queries:SELECT "customers".* FROM "customers"SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 1 AND "addresses"."primary" = t LIMIT 1SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 2 AND "addresses"."primary" = t LIMIT 1SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 3 AND "addresses"."primary" = t LIMIT 1......SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" = 100 AND "addresses"."primary" = t LIMIT 1
  9. Eager Loading with .includes# app/controllers/customers_controller.rbclass CustomersController < ApplicationController def index @customers = Customer.includes(:addresses).all endendIf @customers has 100 records, now we only have 2 queries:SELECT "customers".* FROM "customers"SELECT "addresses".* FROM "addresses" WHERE "addresses"."customer_id" IN(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21,22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40,41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59,60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78,79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97,98, 99, 100)
  10. Finding N+1 in New RelicNew Relic > App Server > Web Transactions > Performance Breakdown
  11. Missing Indexes == Slow Queries
  12. Adding an Index is Simple# db/migrate/20120201040247_add_index_for_shop_id_on_orders.rbclass AddIndexForShopIdOnOrders < ActiveRecord::Migration def change add_index :orders, :shop_id endend Index Protips: •  Searching an index on a table with 1,000 rows is 100x faster than searching a table without an index. •  Put an index on any columns you will likely query against, its better to have too many than too few indexes. •  Adding an index to a table will lock the table!
  13. RUBYWEBSERVERS
  14. Passenger 3•  Simple to operate•  Simple configuration•  Handles worker management•  Great for multi-application environments•  Great for low resource environments•  Attached to Nginx/Apache HTTPD
  15. Passenger Request Queuesolo i-c3f2d8a2 ~ # passenger-status----------- General information -----------max = 3count = 3active = 0inactive = 3Waiting on global queue: 0----------- Application groups -----------/data/john_yerhot_org/current: App root: /data/john_yerhot_org/current * PID: 19802 Sessions: 0 Processed: 3 Uptime: 3h 10m 13s/data/scalingrails/current: App root: /data/scalingrails/current * PID: 28726 Sessions: 0 Processed: 3 Uptime: 59m 22s/data/sites/clmeisinger/current: App root: /data/sites/clmeisinger/current * PID: 22147 Sessions: 0 Processed: 70 Uptime: 10h 45m 57s
  16. Unicorn•  Independent of front end web server•  More configuration options•  Master process will reap children on timeout•  Great for single application environments•  Allows for zero downtime deploys
  17. Unicorn Request Queue?Raindropssolo i-5b74313d ~ # gem install raindropsFetching: raindrops-0.10.0.gem (100%)Building native extensions. This could take a while...Successfully installed raindrops-0.10.01 gem installedsolo i-5b74313d ~ # ruby -rubygems -e "require raindrops; putsRaindrops::Linux.unix_listener_stats([/var/run/engineyard/unicorn_appname.sock]).inspect"{"/var/run/engineyard/unicorn_appname.sock"=>#<struct Raindrops::ListenStatsactive=0, queued=0>}
  18. Request Queuing in New Relic
  19. Request Queuing in New Relic NOT COOL
  20. Request Queuing in New Relic •  Time between first ActionContoller hit - X-Queue-Start = Time spent in queuing.Internet => LB inserts X-Queue-Start => Nginx => Ruby Webserver => Rack =>ApplicationTrack Rack Middleware as welldef call(env) env["HTTP_X_MIDDLEWARE_START"] = "t=#{(Time.now.to_f * 1000000).to_i}" @app.call(env)end
  21. CACHING
  22. Cache Everything Rails makes it stupid easy to cache everything. Do it.
  23. Static Files & Nginx The best cache is a static file served by Nginx.# create it on #index, #show, etc..caches_page :index# expire it on #creates, #updates, #destory, etc...expire_page :action => :index
  24. A Note About Static Files:Use the front end server.upstream upstream_enki { server unix:/var/run/engineyard/unicorn_enki.sock fail_timeout=0;}location ~ ^/(images|assets|javascripts|stylesheets)/ { try_files $uri $uri/index.html /last_assets/$uri /last_assets/$uri.html@app_enki; expires 10y;}location / { if (-f $document_root/system/maintenance.html) { return 503; } try_files $uri $uri/index.html $uri.html @app_enki;}
  25. Memcached: The Standard# config/initializers/memcached.rbconfig.cache_store =:mem_cache_store, "server-1:11211", "server-2:11211", "server-3:11211","server-4:11211"
  26. Next Best: ActionCachingWill still go through Rack/Rails, but the action getscached.before_filter :make_sure_youre_okcaches_action :all_the_thingsdef all_the_things @all_things = Thing.all_in_a_complex_wayenddef expire expire_action :action => :all_the_thingsend
  27. Fragment Caching<% cache(my_cache_key) do %> <%= render_large_tag_cloud %><% end %>...def update_large_tag_cloud TagCloud.update expire_fragment(my_cache_key)end
  28. BaremetalRails.cache.write("john", "yerhot")Rails.cache.read("john")# => "yerhot"# execute a block on miss and cache it.Rails.cache.fetch("miss") do "yerhot"endRails.fetch("miss")# => "yerhot"Rails.cache.exists("john") # => trueRails.cache.delete("john") # => trueRails.cache.exists("john") # => false
  29. BackgroundProcessing
  30. Why Background Processing?•  send email•  process images•  grab feeds and cache them•  complex computations/reports•  create/expire caches/pages (like Reddit)
  31. Best Practice: Use a utility server for background jobs and cron.
  32. Resque to the Rescue
  33. Resque in New Relic
  34. Delayed Job Too
  35. Rails 4Background Processing baked in.•  Allow an application to switch job systems with minimal code change due to common API•  Very basic queuing system built in•  Roll your own wrapper class that responds to push & pop # application.rb config.queue = QueueName Rails.queue.push(Job.new)
  36. Review•  You need to be monitoring your application.•  Performance has to be reviewed on a regular basis.•  Database indexes are cheap, make lots of them.•  Every application can take advantage of some level of caching: page, action or fragment.•  Background any work that you can.•  Dont neglect front-end performance.
  37. How to Install New RelicNew Relic Standard is Free at Engine Yard 1.  If you’re an Engine Yard Customer, select your plan in your Engine Yard Account Settings 2.  Add newrelic_rpm to your Gemfile 3.  Enable monitoring in the Engine Yard DashboardFull Installation Details: http://ey.io/install-newrelic
  38. Questions?
  39. Chris Kelly John YerhotThanks for @amateurhuman www.newrelic.com @yerhot www.engineyard.comWatching!

×