Can railsscale
Upcoming SlideShare
Loading in...5
×
 

Can railsscale

on

  • 293 views

 

Statistics

Views

Total Views
293
Views on SlideShare
293
Embed Views
0

Actions

Likes
1
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Can railsscale Can railsscale Presentation Transcript

  • Can Rails (finally) scale? Experience at Ask.fm Tuesday, September 24, 13
  • CanRailsScale.com Tuesday, September 24, 13
  • Ask.fm • 80M registered users • 200M monthly unique users • 400M page-views a day • ~1M users online Tuesday, September 24, 13
  • Ask.fm • 600 questions a second • 20k requests / second • 50k likes / minute Tuesday, September 24, 13
  • Ask.fm • ~300 ruby on rails nodes • 16 Cores • 16Gb RAM • 24 passengers Tuesday, September 24, 13
  • Team 10 developers 2 Ruby developers Tuesday, September 24, 13
  • Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • Persistence Layer Tuesday, September 24, 13
  • Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • Connection Pool Tuesday, September 24, 13
  • Connection Pool Tuesday, September 24, 13
  • Connection Pool Tuesday, September 24, 13
  • Connection Pool Tuesday, September 24, 13
  • Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • Datasource switching Database ActiveRecord db_charmer* Data Access Logic Business Logic * https://github.com/kovyrin/db-charmer Tuesday, September 24, 13
  • Example production: bar: adapter: mysql username: blah host: blah.local database: blah foo: adapter: mysql username: foo host: foo.local database: foo class Foo < ActiveRecord::Base db_magic :connection => :foo end class Bar < ActiveRecord::Base db_magic :connection => :bar, :slave => :bar_slave end Tuesday, September 24, 13
  • Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • High-Availability Database ActiveRecord db_charmer* Data Access Logic Business Logic * https://github.com/kovyrin/db-charmer Tuesday, September 24, 13
  • • Multiple datasources • Master - Slave • Sharding • High-Availability • Read & Write • Read • None High-Availability Tuesday, September 24, 13
  • Sharding types • Modulo function • Data-dictionary Tuesday, September 24, 13
  • Example (1) question_shard_244_247: <<: *question_shard_host_base_srvb71 database: shard244_247 shards: 244..247 question_shard_248_251: <<: *question_shard_host_base_srvb71 database: shard248_251 shards: 248..251 Tuesday, September 24, 13
  • Example (2) class User < ActiveRecord::Base def questions(access_mode=:read) on_question_shard(self, access_mode) end end user.questions().answered.latest user.questions(:read_write).build({...}) Tuesday, September 24, 13
  • Example (3) def on_question_shard(access_mode=:read_write, &block) accessor = shard_accessor(access_mode) question_data_source.shard_for(accessor, nil, &block) end def shard_for_key(accessor) raise ShardAccessError, 'accessor missing' unless accessor raise ShardAccessError, 'access denied' unless @shard_access.shard_accessible?(accessor.key, accessor.access_mode) shard_connection_config( data_source_name(accessor.key) ) end Tuesday, September 24, 13
  • Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • Redis Sharding • ShardMap • HashRing • DataDictionary Tuesday, September 24, 13
  • Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • Ruby threads • Multithreading in Ruby sucks :) Tuesday, September 24, 13
  • Ruby threads • Parallel DB calls • Third-party services • Any service :) Tuesday, September 24, 13
  • Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • Configuration • database.yml • email.yml • redis.yml • ... • system.yml • ... Tuesday, September 24, 13
  • Centralized configuration Tuesday, September 24, 13
  • Centralized configuration if defined?(PhusionPassenger) PhusionPassenger.on_event(:starting_worker_process) do |forked| if forked Rlt::Configuration::UpdateWatcher::start(askfm_config.store) else end end else Rlt::Configuration::UpdateWatcher::start(askfm_config.store) end Tuesday, September 24, 13
  • Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • Queue • Resque Tuesday, September 24, 13
  • Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • Upload • Goliath* application (non-blocking ruby web server) • 4 servers • 2.5M Uploads a day * https://github.com/postrank-labs/goliath/ Tuesday, September 24, 13
  • Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • Graylog Tuesday, September 24, 13
  • Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • Statistics • Business metrics • Application metrics • Performance metrics • Errors Tuesday, September 24, 13
  • Statistics Tuesday, September 24, 13
  • Statistics Tuesday, September 24, 13
  • Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • Infrastructure • Memory leaks • Timeout Tuesday, September 24, 13
  • Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • Code scalability • MVC • ActiveRecord • Tests • DB Migrations Tuesday, September 24, 13
  • Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • Rails tools scalability • NewRelic • Exceptional.io • Google Analytics • Capistrano Tuesday, September 24, 13
  • Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • GC Tunning Tuesday, September 24, 13
  • Ruby 2.0 Tuesday, September 24, 13
  • jRuby Tuesday, September 24, 13
  • Event Machine Tuesday, September 24, 13