Rails scaling

Real Life Rails Scaling

Agenda• Scale your App • Scale your Team • Scale yourself

Nikon D5100http://d5100.nikon-asia.com

Homepage

Features

Competition

Numbersirb(main):001:0> Submission.count => 179430 irb(main):002:0> User.count => 206546 # du -sh public/system/ 24GB public/system/

Tune your App

Infrastructure• Use GIT & get a tool your team can use • GIT commit eMails • Google Docs for shared documents • Try to ban eMail (impossible...)

Monitoring• Consider monitoring software • Find slow requests, slow queries etc • We used new relic RPM on a trial: !•

Deployment• Staging system on the same server • Auto deploy to Staging on commit • Useful when in 2+-part team • One command to deploy (cap deploy) to production • Full Root access == full fun

Server setup• Database on a second server • Direct link via private network • Shared storage via NFS possible, better if hosted via secondary nodes (asset host) • More RAM == more happy

HTML best practices• Less code == better code • Compact HTML markup • Compact CSS & JS • Short class names • Think: Every 1KB matters • Include JS per page only, not global

HTML tuning• http://validator.w3.org/ • http://developer.yahoo.com/yslow/ • Rails 3.1: Asset pipeline for compression

Rails request tuning• Think: Return to user as soon as possible • Think: Every request can block another • Don’t: Process images immediately • Don’t: Recalculate lists / caches in a request • Do: Use background jobs • Do: Set ﬂags for further actions

Background processing• Beanstalk is heavy for tons of requests, but great for real time • Consider cronjob for background tasks • whenever gem

(Image-) File uploads• Uploading images will take a while • Store image immediately & return • Create thumbnails later & in background • We used carrierwave & conditional versions

(Image-) File uploads• Conditional versions are kick ass!

(Image-) File uploads• One directory can only keep 32768 ﬁles... • Sharding: distribute ﬁles into subdirectories Carrierwave rocks! public/system/uploads/ 1/pic.jpg 2/pic.jpg 3/pic.jpg 4/pic.jpg 5/pic.jpg public/system/uploads/ 2011-11-23 2011-11-24 1/pic.jpg 2/pic.jpg 3/pic.jpg 4/pic.jpg 5/pic.jpg

Facebook integration• Before: fb_graph gem to read user photos • Problem: 1000 requests per minute & facebook blocks the access -> site down • Solved: fb connect JavaScript

Big data sets• Identify key data • Fragment cache each  item • Use JS for post-  processing

Big data setsOnly select what is really needed Submission.accepted_submissions.select(latest_cols).where(:is_photo_of_the_week => false).order(sort) latest_cols =[:id, :user_id, :fb_count, :fb_count_en, :shard, :picture, :picture_is_landscape, :created_at, "TO_CHAR(submissions.updated_at, YYYYMMDDHH24MISS.US)as updated_at"]

memcached• Use it.

Tune your team

Communication• Project plan • Communicate clear about bottlenecks • Like eMail, server storage

GIT access to everyone• Deploy to staging on each commit

Tune yourself

Your tools+• Many good Rails blogs are blocked by GFW • Get a non-free VPN • During an intense project, stuff comes up • Create a delta ﬁle • Do testing and automate using a build server

Delta File

Podcasts• Watch & apply: • railscasts.com • codeschool.com

http://rubylive.fr	109
http://paper.li	6
http://news.humancoders.com	6
http://www.hanrss.com	3
http://www.twylah.com	1

Rails scaling

by Sebastian Roth on Nov 28, 2011

Accessibility

Categories

Upload Details

Usage Rights

5 Embeds 125

Statistics

Rails scaling Presentation Transcript