Scaling Rails Applications in the Cloud




                            100 new users
                             in < 2 ...
A Typical Case Study
                                                                                                     ...
I’m Mike Subelsky
@subelsky / @otherinbox
What is Cloud
Computing?
Solving problems
across a flexible grid
      of nodes
...which you consume
      as a utility
The nodes are
   elastic
The nodes are
        loosely coupled
                 Tasking System




Computing   Computing    Computing   Computing
 ...
The nodes are
 distributed
The nodes are
 ephemeral
The nodes are
 disposable
What are the
 benefits?
Lower cost
Greater efficiency
Greater flexibility
You want flexibility
My Cloud Grows With Me!
                                                                                                  ...
What are the costs?
Some loss of control
Potential for lock-in
Greater complexity
You chose a language
  that sublimates
     complexity
aws-S3
  RightAWS

                  Ruby/EventMachine


     Ruby Shines Here
                  Hpricot
memcache-client
 ...
Capistrano
     Chef
   Rubber
  PoolParty
     Scalr
Roll Your Own
What do you need
 to get started?
Tasking System




Computing   Computing    Computing   Computing
 Instance    Instance     Instance    Instance

        ...
Our Sky
        SMTP                 Sync




                                    Staging
                                ...
Web Cloud
                                        Mongrels &
                                        Memcached
           ...
Staging &
Experiments
Sync Cloud
Autoscaling
SMTP Cloud
                           o
                    et                          Ta
                               ...
Loose Coupling Is How I Sleep
         At Night
  US-EAST-1C
                               to                     Ta
    ...
Loose Coupling Is How I Sleep
         At Night
   US-EAST-1C
                                to                     Ta
  ...
Loose Coupling Is How I Sleep
         At Night


                                                                        ...
Loose Coupling Is How I Sleep
         At Night


                                                       X
   US-EAST-1C
 ...
Loose Coupling Is How I Sleep
         At Night


                                      X
   US-EAST-1C
                  ...
Loose Coupling Is How I Sleep
         At Night
What It Looks Like
Processing Cloud
                                                                                     MessageParser
      ...
Workers
•   MessageExporter

•   DailyUpdateSender

•   RegistrationWorker

•   BackupWorker
The Database Is a
 Single Point of
     Failure
How We Cope
                            Hourly
              EBS Volume
                           Snapshots


           ...
A Dark But
Illustrative Tale
with QuickTime
Manual Stuff

    mkdir /vol
 mount /vol /dev/sdj

sudo god start mysqld
Manual Stuff

change database.yml by hand (yuck)

     queue mail to redeliver
(Wasted)
Troubleshooting:
     15 mins
Provisioning new
   database:
    15 mins
If there was a next
       time...
(standard programmer
   optimism applies)
     < 10 minutes
Replication?
 Sharding?
Key-Value?
Columns?
Logging gem        Splunk




              Logging


                   Good ol’ fashion tail
Zenoss
          Hoptoad

                 Slow query log
   Dash


            Monitoring
          RightScale

Alertra  ...
Questions?
 mike@oib.com

   @subelsky
  @otherinbox
Join our team
Upgrade your job            Open Positions
•   Smart, fun people
                            • Ruby on Rails...
rate.sxsw.com
Thank
 you!
Scaling Rails Applications In The Cloud
Scaling Rails Applications In The Cloud
Scaling Rails Applications In The Cloud
Scaling Rails Applications In The Cloud
Scaling Rails Applications In The Cloud
Upcoming SlideShare
Loading in …5
×

Scaling Rails Applications In The Cloud

3,996 views

Published on

Mike Subelsky from OtherInbox.com describes how OtherInbox uses cloud computing to scale their Ruby on Rails application.

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,996
On SlideShare
0
From Embeds
0
Number of Embeds
870
Actions
Shares
0
Downloads
72
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Scaling Rails Applications In The Cloud

  1. 1. Scaling Rails Applications in the Cloud 100 new users in < 2 hours The hashtag for this talk is: #SX09-573
  2. 2. A Typical Case Study 32 The Inevitable Mass Adoption of Our Product Public Beta 8 Techcrunch50 Me, Josh, Friends & Family and a 4 Private Beta few other people The Inexorable March of Time
  3. 3. I’m Mike Subelsky @subelsky / @otherinbox
  4. 4. What is Cloud Computing?
  5. 5. Solving problems across a flexible grid of nodes
  6. 6. ...which you consume as a utility
  7. 7. The nodes are elastic
  8. 8. The nodes are loosely coupled Tasking System Computing Computing Computing Computing Instance Instance Instance Instance Permanent Storage
  9. 9. The nodes are distributed
  10. 10. The nodes are ephemeral
  11. 11. The nodes are disposable
  12. 12. What are the benefits?
  13. 13. Lower cost Greater efficiency Greater flexibility
  14. 14. You want flexibility
  15. 15. My Cloud Grows With Me! 32 Heady, Viral, Asymptomtotic Growth of Userbase Public Beta 8 Techcrunch50 Me, Josh, Friends & Family and a 4 Private Beta few other people The Blink of an Eye
  16. 16. What are the costs?
  17. 17. Some loss of control Potential for lock-in Greater complexity
  18. 18. You chose a language that sublimates complexity
  19. 19. aws-S3 RightAWS Ruby/EventMachine Ruby Shines Here Hpricot memcache-client ActiveSupport
  20. 20. Capistrano Chef Rubber PoolParty Scalr Roll Your Own
  21. 21. What do you need to get started?
  22. 22. Tasking System Computing Computing Computing Computing Instance Instance Instance Instance Permanent Storage Bonus: Non-RDBMS
  23. 23. Our Sky SMTP Sync Staging Staging Staging Staging Staging Staging Processing Master DB Staging Staging Staging Staging Staging Experiment Workers Web
  24. 24. Web Cloud Mongrels & Memcached JSON Mongrels & HTML Apache HAProxy Master DB Memcached Atom Mongrels & Memcached Message Bodies Mailbox Export Requests, Domain Registration SQS S3
  25. 25. Staging & Experiments
  26. 26. Sync Cloud
  27. 27. Autoscaling
  28. 28. SMTP Cloud o et Ta SQS m s na e kin e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  29. 29. Loose Coupling Is How I Sleep At Night US-EAST-1C to Ta e SQS am skin en e Fil ars g S3 P Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 US-EAST-1B to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  30. 30. Loose Coupling Is How I Sleep At Night US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X XX US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  31. 31. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  32. 32. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  33. 33. Loose Coupling Is How I Sleep At Night X US-EAST-1C to Ta me SQS s na e kin ile rs g 3 F Pa S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3 X US-EAST-1B o t Ta me SQS s a kin en e Fil ars g 3P S Postfix MailSender MailReceiver MailParser Master DB le Ra Fi w Mb x bo ox M Fil w e Ra S3
  34. 34. Loose Coupling Is How I Sleep At Night
  35. 35. What It Looks Like
  36. 36. Processing Cloud MessageParser h s pu cks Tasks MessageParsing Queue a allb e MessageClassfier Queue MessageClassifier c rd nam o AnyFeatureWeWant Queue ec file R ive S3 ct A AnyFeatureWorker Message Created
  37. 37. Workers • MessageExporter • DailyUpdateSender • RegistrationWorker • BackupWorker
  38. 38. The Database Is a Single Point of Failure
  39. 39. How We Cope Hourly EBS Volume Snapshots Weekly Master DB Dumps Expensive Queries Memcached
  40. 40. A Dark But Illustrative Tale with QuickTime
  41. 41. Manual Stuff mkdir /vol mount /vol /dev/sdj sudo god start mysqld
  42. 42. Manual Stuff change database.yml by hand (yuck) queue mail to redeliver
  43. 43. (Wasted) Troubleshooting: 15 mins
  44. 44. Provisioning new database: 15 mins
  45. 45. If there was a next time...
  46. 46. (standard programmer optimism applies) < 10 minutes
  47. 47. Replication? Sharding?
  48. 48. Key-Value? Columns?
  49. 49. Logging gem Splunk Logging Good ol’ fashion tail
  50. 50. Zenoss Hoptoad Slow query log Dash Monitoring RightScale Alertra New Relic
  51. 51. Questions? mike@oib.com @subelsky @otherinbox
  52. 52. Join our team Upgrade your job Open Positions • Smart, fun people • Ruby on Rails Developer • Startup environment • Ruby on Rails Intern • Cutting edge tech • Product Manager • Competitive pay • Stock options We pay $1,000 for successful referrals!
  53. 53. rate.sxsw.com
  54. 54. Thank you!

×