Can railsscale
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
322
On Slideshare
322
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
3
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Can Rails (finally) scale? Experience at Ask.fm Tuesday, September 24, 13
  • 2. CanRailsScale.com Tuesday, September 24, 13
  • 3. Ask.fm • 80M registered users • 200M monthly unique users • 400M page-views a day • ~1M users online Tuesday, September 24, 13
  • 4. Ask.fm • 600 questions a second • 20k requests / second • 50k likes / minute Tuesday, September 24, 13
  • 5. Ask.fm • ~300 ruby on rails nodes • 16 Cores • 16Gb RAM • 24 passengers Tuesday, September 24, 13
  • 6. Team 10 developers 2 Ruby developers Tuesday, September 24, 13
  • 7. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • 8. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • 9. Persistence Layer Tuesday, September 24, 13
  • 10. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • 11. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • 12. Connection Pool Tuesday, September 24, 13
  • 13. Connection Pool Tuesday, September 24, 13
  • 14. Connection Pool Tuesday, September 24, 13
  • 15. Connection Pool Tuesday, September 24, 13
  • 16. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • 17. Datasource switching Database ActiveRecord db_charmer* Data Access Logic Business Logic * https://github.com/kovyrin/db-charmer Tuesday, September 24, 13
  • 18. Example production: bar: adapter: mysql username: blah host: blah.local database: blah foo: adapter: mysql username: foo host: foo.local database: foo class Foo < ActiveRecord::Base db_magic :connection => :foo end class Bar < ActiveRecord::Base db_magic :connection => :bar, :slave => :bar_slave end Tuesday, September 24, 13
  • 19. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • 20. High-Availability Database ActiveRecord db_charmer* Data Access Logic Business Logic * https://github.com/kovyrin/db-charmer Tuesday, September 24, 13
  • 21. • Multiple datasources • Master - Slave • Sharding • High-Availability • Read & Write • Read • None High-Availability Tuesday, September 24, 13
  • 22. Sharding types • Modulo function • Data-dictionary Tuesday, September 24, 13
  • 23. Example (1) question_shard_244_247: <<: *question_shard_host_base_srvb71 database: shard244_247 shards: 244..247 question_shard_248_251: <<: *question_shard_host_base_srvb71 database: shard248_251 shards: 248..251 Tuesday, September 24, 13
  • 24. Example (2) class User < ActiveRecord::Base def questions(access_mode=:read) on_question_shard(self, access_mode) end end user.questions().answered.latest user.questions(:read_write).build({...}) Tuesday, September 24, 13
  • 25. Example (3) def on_question_shard(access_mode=:read_write, &block) accessor = shard_accessor(access_mode) question_data_source.shard_for(accessor, nil, &block) end def shard_for_key(accessor) raise ShardAccessError, 'accessor missing' unless accessor raise ShardAccessError, 'access denied' unless @shard_access.shard_accessible?(accessor.key, accessor.access_mode) shard_connection_config( data_source_name(accessor.key) ) end Tuesday, September 24, 13
  • 26. Persistence Layer Issues • Connection pooling • Datasource switching • High-Availability • Redis Tuesday, September 24, 13
  • 27. Redis Sharding • ShardMap • HashRing • DataDictionary Tuesday, September 24, 13
  • 28. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • 29. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • 30. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • 31. Ruby threads • Multithreading in Ruby sucks :) Tuesday, September 24, 13
  • 32. Ruby threads • Parallel DB calls • Third-party services • Any service :) Tuesday, September 24, 13
  • 33. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • 34. Configuration • database.yml • email.yml • redis.yml • ... • system.yml • ... Tuesday, September 24, 13
  • 35. Centralized configuration Tuesday, September 24, 13
  • 36. Centralized configuration if defined?(PhusionPassenger) PhusionPassenger.on_event(:starting_worker_process) do |forked| if forked Rlt::Configuration::UpdateWatcher::start(askfm_config.store) else end end else Rlt::Configuration::UpdateWatcher::start(askfm_config.store) end Tuesday, September 24, 13
  • 37. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • 38. Queue • Resque Tuesday, September 24, 13
  • 39. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • 40. Upload • Goliath* application (non-blocking ruby web server) • 4 servers • 2.5M Uploads a day * https://github.com/postrank-labs/goliath/ Tuesday, September 24, 13
  • 41. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • 42. Graylog Tuesday, September 24, 13
  • 43. Application Layer • Ruby threads • Configuration • Queue • Uploads • Production Log • Statistics Tuesday, September 24, 13
  • 44. Statistics • Business metrics • Application metrics • Performance metrics • Errors Tuesday, September 24, 13
  • 45. Statistics Tuesday, September 24, 13
  • 46. Statistics Tuesday, September 24, 13
  • 47. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • 48. Infrastructure • Memory leaks • Timeout Tuesday, September 24, 13
  • 49. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • 50. Code scalability • MVC • ActiveRecord • Tests • DB Migrations Tuesday, September 24, 13
  • 51. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • 52. Rails tools scalability • NewRelic • Exceptional.io • Google Analytics • Capistrano Tuesday, September 24, 13
  • 53. Proposed topics • Persistence Layer • Application Layer • Infrastructure • Code scalability • Rails Tools Scalability • Misc Tuesday, September 24, 13
  • 54. GC Tunning Tuesday, September 24, 13
  • 55. Ruby 2.0 Tuesday, September 24, 13
  • 56. jRuby Tuesday, September 24, 13
  • 57. Event Machine Tuesday, September 24, 13