Real Life Rails Scaling
Agenda• Scale your App	• Scale your Team	• Scale yourself
Nikon D5100http://d5100.nikon-asia.com
Homepage
Features
Competition
Numbersirb(main):001:0> Submission.count	=> 179430	irb(main):002:0> User.count	=> 206546	# du -sh public/system/	24GB 	 pu...
Tune your App
Infrastructure• Use GIT & get a tool your team can use	• GIT commit eMails	• Google Docs for shared documents	• Try to ban...
Monitoring• Consider monitoring software	• Find slow requests, slow queries etc	• We used new relic RPM on a trial:	    !•
Deployment• Staging system on the same server	• Auto deploy to Staging on commit	 • Useful when in 2+-part team	• One comm...
Server setup• Database on a second server	 • Direct link via private network	• Shared storage via NFS possible, better if ...
HTML best practices• Less code == better code	 • Compact HTML markup	• Compact CSS & JS	• Short class names	• Think: Every...
HTML tuning• http://validator.w3.org/	• http://developer.yahoo.com/yslow/	• Rails 3.1: Asset pipeline for compression
Rails request tuning• Think: Return to user as soon as possible	• Think: Every request can block another	• Don’t: Process ...
Background processing• Beanstalk is heavy for tons of requests, but  great for real time	• Consider cronjob for background...
(Image-) File uploads• Uploading images will take a while	• Store image immediately & return	• Create thumbnails later & i...
(Image-) File uploads• Conditional versions are kick ass!
(Image-) File uploads• One directory can only keep 32768 files...	• Sharding: distribute files into subdirectories          ...
Facebook integration• Before: fb_graph gem to read user photos	• Problem: 1000 requests per minute &  facebook blocks the ...
Big data sets• Identify key data	• Fragment cache each
  item	• Use JS for post-
  processing
Big data setsOnly select what is really needed	Submission.accepted_submissions.select(latest_cols).where(:is_photo_of_the_...
memcached• Use it.
Tune your team
Communication• Project plan	• Communicate clear about bottlenecks	 • Like eMail, server storage
GIT access to everyone• Deploy to staging on each commit
Tune yourself
Your tools+• Many good Rails blogs are blocked by GFW	 • Get a non-free VPN	• During an intense project, stuff comes up	 •...
Delta File
Podcasts• Watch & apply:	 • railscasts.com	 • codeschool.com
Upcoming SlideShare
Loading in …5
×

Rails scaling

2,004 views

Published on

A short presentation I gave in a GZ Ruby meetup. See www.gzruby.org for details.

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,004
On SlideShare
0
From Embeds
0
Number of Embeds
151
Actions
Shares
0
Downloads
0
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Rails scaling

  1. 1. Real Life Rails Scaling
  2. 2. Agenda• Scale your App • Scale your Team • Scale yourself
  3. 3. Nikon D5100http://d5100.nikon-asia.com
  4. 4. Homepage
  5. 5. Features
  6. 6. Competition
  7. 7. Numbersirb(main):001:0> Submission.count => 179430 irb(main):002:0> User.count => 206546 # du -sh public/system/ 24GB public/system/
  8. 8. Tune your App
  9. 9. Infrastructure• Use GIT & get a tool your team can use • GIT commit eMails • Google Docs for shared documents • Try to ban eMail (impossible...)
  10. 10. Monitoring• Consider monitoring software • Find slow requests, slow queries etc • We used new relic RPM on a trial: !•
  11. 11. Deployment• Staging system on the same server • Auto deploy to Staging on commit • Useful when in 2+-part team • One command to deploy (cap deploy) to production • Full Root access == full fun
  12. 12. Server setup• Database on a second server • Direct link via private network • Shared storage via NFS possible, better if hosted via secondary nodes (asset host) • More RAM == more happy
  13. 13. HTML best practices• Less code == better code • Compact HTML markup • Compact CSS & JS • Short class names • Think: Every 1KB matters • Include JS per page only, not global
  14. 14. HTML tuning• http://validator.w3.org/ • http://developer.yahoo.com/yslow/ • Rails 3.1: Asset pipeline for compression
  15. 15. Rails request tuning• Think: Return to user as soon as possible • Think: Every request can block another • Don’t: Process images immediately • Don’t: Recalculate lists / caches in a request • Do: Use background jobs • Do: Set flags for further actions
  16. 16. Background processing• Beanstalk is heavy for tons of requests, but great for real time • Consider cronjob for background tasks • whenever gem
  17. 17. (Image-) File uploads• Uploading images will take a while • Store image immediately & return • Create thumbnails later & in background • We used carrierwave & conditional versions
  18. 18. (Image-) File uploads• Conditional versions are kick ass!
  19. 19. (Image-) File uploads• One directory can only keep 32768 files... • Sharding: distribute files into subdirectories Carrierwave rocks! public/system/uploads/ 1/pic.jpg 2/pic.jpg 3/pic.jpg 4/pic.jpg 5/pic.jpg public/system/uploads/ 2011-11-23 2011-11-24 1/pic.jpg 2/pic.jpg 3/pic.jpg 4/pic.jpg 5/pic.jpg
  20. 20. Facebook integration• Before: fb_graph gem to read user photos • Problem: 1000 requests per minute & facebook blocks the access -> site down • Solved: fb connect JavaScript
  21. 21. Big data sets• Identify key data • Fragment cache each
 item • Use JS for post-
 processing
  22. 22. Big data setsOnly select what is really needed Submission.accepted_submissions.select(latest_cols).where(:is_photo_of_the_week => false).order(sort) latest_cols =[:id, :user_id, :fb_count, :fb_count_en, :shard, :picture, :picture_is_landscape, :created_at, "TO_CHAR(submissions.updated_at, YYYYMMDDHH24MISS.US)as updated_at"]
  23. 23. memcached• Use it.
  24. 24. Tune your team
  25. 25. Communication• Project plan • Communicate clear about bottlenecks • Like eMail, server storage
  26. 26. GIT access to everyone• Deploy to staging on each commit
  27. 27. Tune yourself
  28. 28. Your tools+• Many good Rails blogs are blocked by GFW • Get a non-free VPN • During an intense project, stuff comes up • Create a delta file • Do testing and automate using a build server
  29. 29. Delta File
  30. 30. Podcasts• Watch & apply: • railscasts.com • codeschool.com

×