Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Real Life Rails Scaling
Agenda• Scale your App	• Scale your Team	• Scale yourself
Nikon D5100http://d5100.nikon-asia.com
Homepage
Features
Competition
Numbersirb(main):001:0> Submission.count	=> 179430	irb(main):002:0> User.count	=> 206546	# du -sh public/system/	24GB 	 pu...
Tune your App
Infrastructure• Use GIT & get a tool your team can use	• GIT commit eMails	• Google Docs for shared documents	• Try to ban...
Monitoring• Consider monitoring software	• Find slow requests, slow queries etc	• We used new relic RPM on a trial:	    !•
Deployment• Staging system on the same server	• Auto deploy to Staging on commit	 • Useful when in 2+-part team	• One comm...
Server setup• Database on a second server	 • Direct link via private network	• Shared storage via NFS possible, better if ...
HTML best practices• Less code == better code	 • Compact HTML markup	• Compact CSS & JS	• Short class names	• Think: Every...
HTML tuning• http://validator.w3.org/	• http://developer.yahoo.com/yslow/	• Rails 3.1: Asset pipeline for compression
Rails request tuning• Think: Return to user as soon as possible	• Think: Every request can block another	• Don’t: Process ...
Background processing• Beanstalk is heavy for tons of requests, but  great for real time	• Consider cronjob for background...
(Image-) File uploads• Uploading images will take a while	• Store image immediately & return	• Create thumbnails later & i...
(Image-) File uploads• Conditional versions are kick ass!
(Image-) File uploads• One directory can only keep 32768 files...	• Sharding: distribute files into subdirectories          ...
Facebook integration• Before: fb_graph gem to read user photos	• Problem: 1000 requests per minute &  facebook blocks the ...
Big data sets• Identify key data	• Fragment cache each
  item	• Use JS for post-
  processing
Big data setsOnly select what is really needed	Submission.accepted_submissions.select(latest_cols).where(:is_photo_of_the_...
memcached• Use it.
Tune your team
Communication• Project plan	• Communicate clear about bottlenecks	 • Like eMail, server storage
GIT access to everyone• Deploy to staging on each commit
Tune yourself
Your tools+• Many good Rails blogs are blocked by GFW	 • Get a non-free VPN	• During an intense project, stuff comes up	 •...
Delta File
Podcasts• Watch & apply:	 • railscasts.com	 • codeschool.com
Upcoming SlideShare
Loading in …5
×

Rails scaling

2,121 views

Published on

A short presentation I gave in a GZ Ruby meetup. See www.gzruby.org for details.

Published in: Technology
  • Be the first to comment

Rails scaling

  1. 1. Real Life Rails Scaling
  2. 2. Agenda• Scale your App • Scale your Team • Scale yourself
  3. 3. Nikon D5100http://d5100.nikon-asia.com
  4. 4. Homepage
  5. 5. Features
  6. 6. Competition
  7. 7. Numbersirb(main):001:0> Submission.count => 179430 irb(main):002:0> User.count => 206546 # du -sh public/system/ 24GB public/system/
  8. 8. Tune your App
  9. 9. Infrastructure• Use GIT & get a tool your team can use • GIT commit eMails • Google Docs for shared documents • Try to ban eMail (impossible...)
  10. 10. Monitoring• Consider monitoring software • Find slow requests, slow queries etc • We used new relic RPM on a trial: !•
  11. 11. Deployment• Staging system on the same server • Auto deploy to Staging on commit • Useful when in 2+-part team • One command to deploy (cap deploy) to production • Full Root access == full fun
  12. 12. Server setup• Database on a second server • Direct link via private network • Shared storage via NFS possible, better if hosted via secondary nodes (asset host) • More RAM == more happy
  13. 13. HTML best practices• Less code == better code • Compact HTML markup • Compact CSS & JS • Short class names • Think: Every 1KB matters • Include JS per page only, not global
  14. 14. HTML tuning• http://validator.w3.org/ • http://developer.yahoo.com/yslow/ • Rails 3.1: Asset pipeline for compression
  15. 15. Rails request tuning• Think: Return to user as soon as possible • Think: Every request can block another • Don’t: Process images immediately • Don’t: Recalculate lists / caches in a request • Do: Use background jobs • Do: Set flags for further actions
  16. 16. Background processing• Beanstalk is heavy for tons of requests, but great for real time • Consider cronjob for background tasks • whenever gem
  17. 17. (Image-) File uploads• Uploading images will take a while • Store image immediately & return • Create thumbnails later & in background • We used carrierwave & conditional versions
  18. 18. (Image-) File uploads• Conditional versions are kick ass!
  19. 19. (Image-) File uploads• One directory can only keep 32768 files... • Sharding: distribute files into subdirectories Carrierwave rocks! public/system/uploads/ 1/pic.jpg 2/pic.jpg 3/pic.jpg 4/pic.jpg 5/pic.jpg public/system/uploads/ 2011-11-23 2011-11-24 1/pic.jpg 2/pic.jpg 3/pic.jpg 4/pic.jpg 5/pic.jpg
  20. 20. Facebook integration• Before: fb_graph gem to read user photos • Problem: 1000 requests per minute & facebook blocks the access -> site down • Solved: fb connect JavaScript
  21. 21. Big data sets• Identify key data • Fragment cache each
 item • Use JS for post-
 processing
  22. 22. Big data setsOnly select what is really needed Submission.accepted_submissions.select(latest_cols).where(:is_photo_of_the_week => false).order(sort) latest_cols =[:id, :user_id, :fb_count, :fb_count_en, :shard, :picture, :picture_is_landscape, :created_at, "TO_CHAR(submissions.updated_at, YYYYMMDDHH24MISS.US)as updated_at"]
  23. 23. memcached• Use it.
  24. 24. Tune your team
  25. 25. Communication• Project plan • Communicate clear about bottlenecks • Like eMail, server storage
  26. 26. GIT access to everyone• Deploy to staging on each commit
  27. 27. Tune yourself
  28. 28. Your tools+• Many good Rails blogs are blocked by GFW • Get a non-free VPN • During an intense project, stuff comes up • Create a delta file • Do testing and automate using a build server
  29. 29. Delta File
  30. 30. Podcasts• Watch & apply: • railscasts.com • codeschool.com

×