Can Rails (finally) scale?
Experience at Ask.fm
Tuesday, September 24, 13
CanRailsScale.com
Tuesday, September 24, 13
Ask.fm
• 80M registered users
• 200M monthly unique users
• 400M page-views a day
• ~1M users online
Tuesday, September 24...
Ask.fm
• 600 questions a second
• 20k requests / second
• 50k likes / minute
Tuesday, September 24, 13
Ask.fm
• ~300 ruby on rails nodes
• 16 Cores
• 16Gb RAM
• 24 passengers
Tuesday, September 24, 13
Team
10 developers
2 Ruby developers
Tuesday, September 24, 13
Proposed topics
• Persistence Layer
• Application Layer
• Infrastructure
• Code scalability
• Rails Tools Scalability
• Mi...
Proposed topics
• Persistence Layer
• Application Layer
• Infrastructure
• Code scalability
• Rails Tools Scalability
• Mi...
Persistence Layer
Tuesday, September 24, 13
Persistence Layer Issues
• Connection pooling
• Datasource switching
• High-Availability
• Redis
Tuesday, September 24, 13
Persistence Layer Issues
• Connection pooling
• Datasource switching
• High-Availability
• Redis
Tuesday, September 24, 13
Connection Pool
Tuesday, September 24, 13
Connection Pool
Tuesday, September 24, 13
Connection Pool
Tuesday, September 24, 13
Connection Pool
Tuesday, September 24, 13
Persistence Layer Issues
• Connection pooling
• Datasource switching
• High-Availability
• Redis
Tuesday, September 24, 13
Datasource switching
Database
ActiveRecord
db_charmer*
Data Access Logic
Business Logic
* https://github.com/kovyrin/db-ch...
Example
production:
bar:
adapter: mysql
username: blah
host: blah.local
database: blah
foo:
adapter: mysql
username: foo
h...
Persistence Layer Issues
• Connection pooling
• Datasource switching
• High-Availability
• Redis
Tuesday, September 24, 13
High-Availability
Database
ActiveRecord
db_charmer*
Data Access Logic
Business Logic
* https://github.com/kovyrin/db-charm...
• Multiple datasources
• Master - Slave
• Sharding
• High-Availability
• Read & Write
• Read
• None
High-Availability
Tues...
Sharding types
• Modulo function
• Data-dictionary
Tuesday, September 24, 13
Example (1)
question_shard_244_247:
<<: *question_shard_host_base_srvb71
database: shard244_247
shards: 244..247
question_...
Example (2)
class User < ActiveRecord::Base
def questions(access_mode=:read)
on_question_shard(self, access_mode)
end
end
...
Example (3)
def on_question_shard(access_mode=:read_write, &block)
accessor = shard_accessor(access_mode)
question_data_so...
Persistence Layer Issues
• Connection pooling
• Datasource switching
• High-Availability
• Redis
Tuesday, September 24, 13
Redis Sharding
• ShardMap
• HashRing
• DataDictionary
Tuesday, September 24, 13
Proposed topics
• Persistence Layer
• Application Layer
• Infrastructure
• Code scalability
• Rails Tools Scalability
• Mi...
Application Layer
• Ruby threads
• Configuration
• Queue
• Uploads
• Production Log
• Statistics
Tuesday, September 24, 13
Application Layer
• Ruby threads
• Configuration
• Queue
• Uploads
• Production Log
• Statistics
Tuesday, September 24, 13
Ruby threads
• Multithreading in Ruby sucks :)
Tuesday, September 24, 13
Ruby threads
• Parallel DB calls
• Third-party services
• Any service :)
Tuesday, September 24, 13
Application Layer
• Ruby threads
• Configuration
• Queue
• Uploads
• Production Log
• Statistics
Tuesday, September 24, 13
Configuration
• database.yml
• email.yml
• redis.yml
• ...
• system.yml
• ...
Tuesday, September 24, 13
Centralized configuration
Tuesday, September 24, 13
Centralized configuration
if defined?(PhusionPassenger)
PhusionPassenger.on_event(:starting_worker_process) do |forked|
if ...
Application Layer
• Ruby threads
• Configuration
• Queue
• Uploads
• Production Log
• Statistics
Tuesday, September 24, 13
Queue
• Resque
Tuesday, September 24, 13
Application Layer
• Ruby threads
• Configuration
• Queue
• Uploads
• Production Log
• Statistics
Tuesday, September 24, 13
Upload
• Goliath* application (non-blocking ruby
web server)
• 4 servers
• 2.5M Uploads a day
* https://github.com/postran...
Application Layer
• Ruby threads
• Configuration
• Queue
• Uploads
• Production Log
• Statistics
Tuesday, September 24, 13
Graylog
Tuesday, September 24, 13
Application Layer
• Ruby threads
• Configuration
• Queue
• Uploads
• Production Log
• Statistics
Tuesday, September 24, 13
Statistics
• Business metrics
• Application metrics
• Performance metrics
• Errors
Tuesday, September 24, 13
Statistics
Tuesday, September 24, 13
Statistics
Tuesday, September 24, 13
Proposed topics
• Persistence Layer
• Application Layer
• Infrastructure
• Code scalability
• Rails Tools Scalability
• Mi...
Infrastructure
• Memory leaks
• Timeout
Tuesday, September 24, 13
Proposed topics
• Persistence Layer
• Application Layer
• Infrastructure
• Code scalability
• Rails Tools Scalability
• Mi...
Code scalability
• MVC
• ActiveRecord
• Tests
• DB Migrations
Tuesday, September 24, 13
Proposed topics
• Persistence Layer
• Application Layer
• Infrastructure
• Code scalability
• Rails Tools Scalability
• Mi...
Rails tools scalability
• NewRelic
• Exceptional.io
• Google Analytics
• Capistrano
Tuesday, September 24, 13
Proposed topics
• Persistence Layer
• Application Layer
• Infrastructure
• Code scalability
• Rails Tools Scalability
• Mi...
GC Tunning
Tuesday, September 24, 13
Ruby 2.0
Tuesday, September 24, 13
jRuby
Tuesday, September 24, 13
Event Machine
Tuesday, September 24, 13
Upcoming SlideShare
Loading in...5
×

Can railsscale

233

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
233
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Can railsscale

  1. 1. Can Rails (finally) scale? Experience at Ask.fm Tuesday, September 24, 13
  2. 2. CanRailsScale.com Tuesday, September 24, 13
  3. 3. Ask.fm • 80M registered users • 200M monthly unique users • 400M page-views a day • ~1M users online Tuesday, September 24, 13
  4. 4. Ask.fm • 600 questions a second • 20k requests / second • 50k likes / minute Tuesday, September 24, 13
  5. 5. Ask.fm • ~300 ruby on rails nodes • 16 Cores • 16Gb RAM • 24 passengers Tuesday, September 24, 13
  6. 6. Team 10 developers 2 Ruby developers Tuesday, September 24, 13
  7. 7. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  8. 8. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  9. 9. Persistence Layer Tuesday, September 24, 13
  10. 10. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  11. 11. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  12. 12. Connection Pool Tuesday, September 24, 13
  13. 13. Connection Pool Tuesday, September 24, 13
  14. 14. Connection Pool Tuesday, September 24, 13
  15. 15. Connection Pool Tuesday, September 24, 13
  16. 16. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  17. 17. Datasource switching Database ActiveRecord db_charmer* Data Access Logic Business Logic * https://github.com/kovyrin/db-charmer Tuesday, September 24, 13
  18. 18. Example production: bar: adapter: mysql username: blah host: blah.local database: blah foo: adapter: mysql username: foo host: foo.local database: foo class Foo < ActiveRecord::Base db_magic :connection => :foo end class Bar < ActiveRecord::Base db_magic :connection => :bar, :slave => :bar_slave end Tuesday, September 24, 13
  19. 19. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  20. 20. High-Availability Database ActiveRecord db_charmer* Data Access Logic Business Logic * https://github.com/kovyrin/db-charmer Tuesday, September 24, 13
  21. 21. • Multiple datasources • Master - Slave • Sharding • High-Availability • Read & Write • Read • None High-Availability Tuesday, September 24, 13
  22. 22. Sharding types • Modulo function • Data-dictionary Tuesday, September 24, 13
  23. 23. Example (1) question_shard_244_247: <<: *question_shard_host_base_srvb71 database: shard244_247 shards: 244..247 question_shard_248_251: <<: *question_shard_host_base_srvb71 database: shard248_251 shards: 248..251 Tuesday, September 24, 13
  24. 24. Example (2) class User < ActiveRecord::Base def questions(access_mode=:read) on_question_shard(self, access_mode) end end user.questions().answered.latest user.questions(:read_write).build({...}) Tuesday, September 24, 13
  25. 25. Example (3) def on_question_shard(access_mode=:read_write, &block) accessor = shard_accessor(access_mode) question_data_source.shard_for(accessor, nil, &block) end def shard_for_key(accessor) raise ShardAccessError, 'accessor missing' unless accessor raise ShardAccessError, 'access denied' unless @shard_access.shard_accessible?(accessor.key, accessor.access_mode) shard_connection_config( data_source_name(accessor.key) ) end Tuesday, September 24, 13
  26. 26. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  27. 27. Redis Sharding • ShardMap • HashRing • DataDictionary Tuesday, September 24, 13
  28. 28. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  29. 29. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  30. 30. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  31. 31. Ruby threads • Multithreading in Ruby sucks :) Tuesday, September 24, 13
  32. 32. Ruby threads • Parallel DB calls • Third-party services • Any service :) Tuesday, September 24, 13
  33. 33. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  34. 34. Configuration • database.yml • email.yml • redis.yml • ... • system.yml • ... Tuesday, September 24, 13
  35. 35. Centralized configuration Tuesday, September 24, 13
  36. 36. Centralized configuration if defined?(PhusionPassenger) PhusionPassenger.on_event(:starting_worker_process) do |forked| if forked Rlt::Configuration::UpdateWatcher::start(askfm_config.store) else end end else Rlt::Configuration::UpdateWatcher::start(askfm_config.store) end Tuesday, September 24, 13
  37. 37. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  38. 38. Queue • Resque Tuesday, September 24, 13
  39. 39. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  40. 40. Upload • Goliath* application (non-blocking ruby web server) • 4 servers • 2.5M Uploads a day * https://github.com/postrank-labs/goliath/ Tuesday, September 24, 13
  41. 41. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  42. 42. Graylog Tuesday, September 24, 13
  43. 43. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  44. 44. Statistics • Business metrics • Application metrics • Performance metrics • Errors Tuesday, September 24, 13
  45. 45. Statistics Tuesday, September 24, 13
  46. 46. Statistics Tuesday, September 24, 13
  47. 47. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  48. 48. Infrastructure • Memory leaks • Timeout Tuesday, September 24, 13
  49. 49. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  50. 50. Code scalability • MVC • ActiveRecord • Tests • DB Migrations Tuesday, September 24, 13
  51. 51. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  52. 52. Rails tools scalability • NewRelic • Exceptional.io • Google Analytics • Capistrano Tuesday, September 24, 13
  53. 53. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  54. 54. GC Tunning Tuesday, September 24, 13
  55. 55. Ruby 2.0 Tuesday, September 24, 13
  56. 56. jRuby Tuesday, September 24, 13
  57. 57. Event Machine Tuesday, September 24, 13
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×