Download as PDF, PPTX
![cacheable
class PostsController < ApplicationController
def show
response_cache do
@post = @shop.posts.find(params[:id])
respond_with(@post)
end
end
def cache_key_data
{
:action => action_name,
:format => request.format,
:params => params.slice(:id),
:shop_version => @shop.version
}
end
end](https://image.slidesharecdn.com/howshopifyscalesrails2-130502225237-phpapp02/75/How-Shopify-Scales-Rails-38-2048.jpg)
![Identity Cache
class Product < ActiveRecord::Base
include IdentityCache
has_many :images
cache_index [:shop_id, :id]
cache_has_many :images, :embed => true
end
@product = Product.fetch_by_shop_id_and_id(shop_id, id)
@images = @product.fetch_images](https://image.slidesharecdn.com/howshopifyscalesrails2-130502225237-phpapp02/75/How-Shopify-Scales-Rails-42-2048.jpg)
![Resque
class AddressGeolocationJob
max_retries 3
def self.perform(params)
object = params[:model].constantize.find(params[:id])
object.latitude, object.longitude = Geocoder.geocode(object)
object.save!
end
end
Resque.enqueue(AddressGeolocationJob, :id => 1, :model => 'Address')](https://image.slidesharecdn.com/howshopifyscalesrails2-130502225237-phpapp02/75/How-Shopify-Scales-Rails-49-2048.jpg)
More Related Content
![[오픈소스컨설팅] 스카우터 사용자 가이드 2020](https://cdn.slidesharecdn.com/ss_thumbnails/2020scouteruserguide-200122014357-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to How Shopify Scales Rails
How Shopify Scales Rails
- 1. shopify How Shopify ScalesRails John Duff
- 2. • The Shopifystack • Knowing what to scale • How we cache • Scaling beyond caching • Splitting things up Overview
- 3.
- 7.
- 8. • Ruby 1.9.3-p385 •Rails 3.2 • Percona MySQL 5.5 • Unicorn 4.5 • Memcached 1.4.14 • Redis 2.6 The Stack
- 9. The Stack • 53App Servers • 1590 Unicorn Workers • 5 Job Servers • 370 Job Workers Nginx Unicorn Rails 3.2 Ruby 1.9.3-p385
- 10. The Stack Firewall Load Balancer AppServers Redis Job Servers Database Memcached Search
- 11. • 55,873 Linesof application code • 15,968 Lines of CoffeeScript application code • 81,892 Lines of test code • 211 Controllers • 468 Models The Stack
- 12.
- 13. 9.9 M Orders Anorder every 3.2 seconds
- 14. 2,008 Sales perMinute Cyber Monday
- 15. 50,000 RPM 45 msresponse time
- 16.
- 17. Looking Back, toLook Ahead
- 18. • First lineof code written in 2004 • Shopify released June, 2006 • Same codebase • Over 9 years of Rails upgrades, improvements and changes Looking Back, to Look Ahead
- 19. Looking Back, toLook Ahead • 6,702 Lines of application code (55,873) • 4,386 Lines of test code (81,892) • 38 Controllers (211) • 77 Models (468)
- 20. Looking Back, toLook Ahead • Ruby 1.8.2 • Rails 0.13.1 • MySQL 4.1 • Lighttpd • Memcached
- 21.
- 22. One Request, OneProcess
- 23. RPM = W* 1/R
- 24. RPM = 1590* 60 / 0.072
- 25. 1,325,000 = 1172* 60 / 0.072
- 26.
- 27. Know The System •Avoid network calls during requests • Speed up unavoidable network calls • The Storefront and Checkout • The Chive
- 28.
- 29. Measure ALL THETHINGS
- 30. Measure ALL THETHINGS • New Relic • Splunk • StatsD • Cacti • Conan
- 31.
- 32.
- 35.
- 36.
- 37. cacheable • https://github.com/Shopify/cacheable • servegzip’d content • ETag and 304 Not Modified • generational caching • no explicit expiry
- 38. cacheable class PostsController <ApplicationController def show response_cache do @post = @shop.posts.find(params[:id]) respond_with(@post) end end def cache_key_data { :action => action_name, :format => request.format, :params => params.slice(:id), :shop_version => @shop.version } end end
- 39.
- 40.
- 41. Identity Cache • https://github.com/Shopify/identity_cache •cache full model objects in memcached • can include associated objects in cache • must opt in to the cache • explicit, but automatic expiry
- 42. Identity Cache class Product< ActiveRecord::Base include IdentityCache has_many :images cache_index [:shop_id, :id] cache_has_many :images, :embed => true end @product = Product.fetch_by_shop_id_and_id(shop_id, id) @images = @product.fetch_images
- 43.
- 44. Get Out ofMy Process
- 45. Delayed Job • Jobsstored in the db • Workers run in their own process • Workers poll for jobs periodically • https://github.com/collectiveidea/delayed_job
- 46. Resque • Redis backed •O(1) operation to pop jobs • Faster (300 jobs/sec vs 120 jobs/sec) • Extensible • https://github.com/defunkt/resque
- 47. Resque • Sending Email •Processing Payments • Geolocation • Import / Export • Indexing for Search • 86 Other things...
- 48.
- 49. Resque class AddressGeolocationJob max_retries 3 defself.perform(params) object = params[:model].constantize.find(params[:id]) object.latitude, object.longitude = Geocoder.geocode(object) object.save! end end Resque.enqueue(AddressGeolocationJob, :id => 1, :model => 'Address')
- 50. Redis • Inventory reservationsystem • Sessions • Theme uploads • Throttling • Carts
- 51. All Roads LeadTo MySQL
- 52. MySQL Hardware • 4x 8 Core Processor • SSD • 256 GB Ram • Full working set in memory
- 53. MySQL Query Optimization •pt-query-digest • Avoid queries that generate temp tables • Adding the right indexes • Forcing / Ignoring Indexes
- 54. MySQL Tuning • disableinnodb_stats_on_metadata • increase table_open_cache • replace glibc memory allocator with tcmalloc • innodb_autoinc_lock_mode=‘interleaved’
- 55.
- 56. after_commit • After transactionhas been committed • Webhooks • Cache expiry • Update associated objects
- 57. after_commit class OrderObserver <ActiveRecord::Observer observe :order def after_save(order) if order.changes.keys.include?(:financial_status) order.flag_for_after_commit(:update_customer) end end def after_commit(order) if order.flagged_for_after_commit?(:update_customer) Resque.enqueue(UpdateCustomerJob, :id => order.id) end end end
- 58.
- 59. Services • Split outstandalone services as needed • Independently scaled • Segmented metrics • Overall system is more complex • Limit to what is necessary
- 60.
- 61. Adapt and Evolveas Needed Using data and knowledge of the system to drive decisions
- 62. Summary • Know yourapplication and infrastructure. • Keep slow IO or CPU tasks out of the main process. • Measure your optimizations. You can make it worse.
- 63.