Successfully reported this slideshow.
Scaling Rails
a system-wide approach to performance optimization



         Exploratory Case Study: http://escolinhas.pt
...
< Problem >



2   Scaling Rails: a system-wide approach to performance optimization
Web 2.0




3   Scaling Rails: a system-wide approach to performance optimization
Better web
                                        Better tools




4   Scaling Rails: a system-wide approach to performan...
Ruby on Rails




5   Scaling Rails: a system-wide approach to performance optimization
Popularity brings                                                   USERS

    USERS need scalability
6   Scaling Rails: a...
Rails doesn’t scale




7   Scaling Rails: a system-wide approach to performance optimization
Twitter




8   Scaling Rails: a system-wide approach to performance optimization
9   Scaling Rails: a system-wide approach to performance optimization
Twitter abandoning Rails?




10   Scaling Rails: a system-wide approach to performance optimization
“FWIW: Twitter currently has no plans to
     abandon RoR. Lots of our code is not in
     RoR, already, though. Maybe tha...
YellowPages




12   Scaling Rails: a system-wide approach to performance optimization
More users?

          “Just throw hardware at it.”



13   Scaling Rails: a system-wide approach to performance optimizat...
Rails doesn’t scale




14   Scaling Rails: a system-wide approach to performance optimization
Hulu




15   Scaling Rails: a system-wide approach to performance optimization
Scribd




16   Scaling Rails: a system-wide approach to performance optimization
They scaled
                  They gave keynotes on

                                         HOW
17   Scaling Rails: a sy...
What’s missing here?




18   Scaling Rails: a system-wide approach to performance optimization
No general approaches to scaling




19   Scaling Rails: a system-wide approach to performance optimization
Problem Overview




20   Scaling Rails: a system-wide approach to performance optimization
< Problem Overview >

 Rails doesn’t scale?
  Twitter
  YellowPages

 Rails doesn’t scale?
  Hulu
  Scribd

 General appro...
< Motivation >



22   Scaling Rails: a system-wide approach to performance optimization
WEB

                                                      WEB
                                                WEB
       ...
USERS

                                                  USERS
                                         USERS
            ...
IMPORTANCE



                                                    IMPORTANCE


                                     IMPORT...
Better web




26   Scaling Rails: a system-wide approach to performance optimization
Better tools




27   Scaling Rails: a system-wide approach to performance optimization
UX involves reasonable response
     times and service uptime




28   Scaling Rails: a system-wide approach to performanc...
Most recent tools are great




29   Scaling Rails: a system-wide approach to performance optimization
“Ruby on Rails is a breakthrough in lowering the
     barriers of entry to programming. Powerful web
       applications t...
Everything can be improved

                   There is no perfect tool

                There are always pitfalls


31   ...
What’s really motivating?




32   Scaling Rails: a system-wide approach to performance optimization
Fixing pitfalls on great tools

           Improving UX for everyone



33   Scaling Rails: a system-wide approach to perf...
Motivation Overview




34   Scaling Rails: a system-wide approach to performance optimization
< Motivation Overview >
 Web keeps growing
  More users, importance
  Web 2.0

 User Experience
  Also dependant on:
    R...
< Goals >



36   Scaling Rails: a system-wide approach to performance optimization
What about scaling?




37   Scaling Rails: a system-wide approach to performance optimization
Everything is easy

         Scaling should also be easy



38   Scaling Rails: a system-wide approach to performance opti...
Creation of




39   Scaling Rails: a system-wide approach to performance optimization
Guidelines and conventions
            for high performance
              Rails applications



40   Scaling Rails: a syst...
Optimal Rails-oriented
     configurations for every involved
                component



41    Scaling Rails: a system-w...
Patches to improve the
     components performance from a
           Rails-perspective



42    Scaling Rails: a system-wi...
Objectives Overview




43   Scaling Rails: a system-wide approach to performance optimization
< Objectives Overview >

 Scaling should be easy

 Creation of:
   Generic guidelines and conventions
   Optimal configura...
< State of the Art >



45   Scaling Rails: a system-wide approach to performance optimization
< Operating Systems >




46   Scaling Rails: a system-wide approach to performance optimization
Most used OS in servers: Linux




47   Scaling Rails: a system-wide approach to performance optimization
BSD faster than Linux




48   Scaling Rails: a system-wide approach to performance optimization
Linux faster than BSD




49   Scaling Rails: a system-wide approach to performance optimization
Linux Ruby 1.8 2x faster
                 than Windows Ruby 1.8




50   Scaling Rails: a system-wide approach to performa...
Linux Ruby 1.9 70% faster
              than Windows Ruby 1.9




51   Scaling Rails: a system-wide approach to performanc...
Rails widely deployed in Linux




52   Scaling Rails: a system-wide approach to performance optimization
< Ruby >




53   Scaling Rails: a system-wide approach to performance optimization
Ruby 1.8 is the most widely used




54    Scaling Rails: a system-wide approach to performance optimization
JRuby is the fastest implementation




55    Scaling Rails: a system-wide approach to performance optimization
Ruby 1.8 will soon be outdated




56   Scaling Rails: a system-wide approach to performance optimization
Ruby 1.9 will soon be outdated




57   Scaling Rails: a system-wide approach to performance optimization
Ruby 1.9 has only one interpreter




58    Scaling Rails: a system-wide approach to performance optimization
Ruby MRI 1.9




59   Scaling Rails: a system-wide approach to performance optimization
Ruby MRI 1.9

                               Rails bottleneck: GC




60   Scaling Rails: a system-wide approach to perfor...
< Web servers >




61   Scaling Rails: a system-wide approach to performance optimization
Webrick




62   Scaling Rails: a system-wide approach to performance optimization
Webrick

                                          Pioneer, slow




63   Scaling Rails: a system-wide approach to perform...
Mongrel




64   Scaling Rails: a system-wide approach to performance optimization
Mongrel

        Much faster, excellent HTML parser




65   Scaling Rails: a system-wide approach to performance optimiza...
Thin




66   Scaling Rails: a system-wide approach to performance optimization
Thin

                           Mongrel’s HTML parser




67   Scaling Rails: a system-wide approach to performance optim...
Thin

            New I/O back-end: EventMachine




68   Scaling Rails: a system-wide approach to performance optimization
Thin

                         Faster than those before




69   Scaling Rails: a system-wide approach to performance opti...
Passenger




70   Scaling Rails: a system-wide approach to performance optimization
Passenger

                                   Not self-contained




71   Scaling Rails: a system-wide approach to perform...
Passenger

                        Module for Apache/Nginx




72   Scaling Rails: a system-wide approach to performance o...
Passenger

        Good performance, great at scaling




73   Scaling Rails: a system-wide approach to performance optimi...
Thin is the fastest

                 Passenger scales better



74   Scaling Rails: a system-wide approach to performance...
< Database >




75   Scaling Rails: a system-wide approach to performance optimization
MySQL




76   Scaling Rails: a system-wide approach to performance optimization
PostgreSQL




77   Scaling Rails: a system-wide approach to performance optimization
MySQL is faster and more popular




78   Scaling Rails: a system-wide approach to performance optimization
Schema-less databases




79   Scaling Rails: a system-wide approach to performance optimization
Schema-less databases

 Said to have better performance than SQL




80   Scaling Rails: a system-wide approach to perform...
MongoDB




81   Scaling Rails: a system-wide approach to performance optimization
CouchDB




82   Scaling Rails: a system-wide approach to performance optimization
Benchmarks are scarce




83   Scaling Rails: a system-wide approach to performance optimization
Major improvement in all DB:

                     Caching, caching, caching!




84   Scaling Rails: a system-wide approa...
< Rails >




85   Scaling Rails: a system-wide approach to performance optimization
Rails 2.3




86   Scaling Rails: a system-wide approach to performance optimization
Many performance pitfalls
               improved in Rails 3




87   Scaling Rails: a system-wide approach to performance...
Rails 3

                         Beta release in February




88   Scaling Rails: a system-wide approach to performance o...
Immediate need: form helpers, i18n




89   Scaling Rails: a system-wide approach to performance optimization
< Application >




90   Scaling Rails: a system-wide approach to performance optimization
Porting to Ruby 1.9 (from 1.8)




91   Scaling Rails: a system-wide approach to performance optimization
Porting to Rails 3 (from 2.3)




92   Scaling Rails: a system-wide approach to performance optimization
Check code for common pitfalls

                           Eager loading
                    Fetching records in batches
 ...
< Methodology >



94   Scaling Rails: a system-wide approach to performance optimization
State of the Art narrows options down




95   Scaling Rails: a system-wide approach to performance optimization
Benchmarking

         Controlled tests to choose the best
           software for each component




96   Scaling Rails: ...
Tweaking

        Try out different configurations and
                    architectures




97   Scaling Rails: a system-...
Testing

     Measuring of results, finding bottlenecks




98    Scaling Rails: a system-wide approach to performance opt...
Developing

                    Improving over bottlenecks




99   Scaling Rails: a system-wide approach to performance o...
Benchmarking
                                       Tweaking
                                        Testing
             ...
Recipe: repeat for all components




101   Scaling Rails: a system-wide approach to performance optimization
Rails perspective!




102   Scaling Rails: a system-wide approach to performance optimization
Escolinhas




103   Scaling Rails: a system-wide approach to performance optimization
104   Scaling Rails: a system-wide approach to performance optimization
Butterfly Effect




105   Scaling Rails: a system-wide approach to performance optimization
106   Scaling Rails: a system-wide approach to performance optimization
Consider sensitive dependencies




107    Scaling Rails: a system-wide approach to performance optimization
< Work schedule >



108   Scaling Rails: a system-wide approach to performance optimization
109   Scaling Rails: a system-wide approach to performance optimization
Activity                                                                 Start Date   Duration (weeks)
  OS testing and ch...
< Thank you >
http://snaprails.tumblr.com



                      Gonçalo S. Silva
                      Advisor: Ademar ...
Upcoming SlideShare
Loading in …5
×

Scaling Rails: a system-wide approach to performance optimization

8,132 views

Published on

Second keynote on my master's thesis. It's a state of the art review.

Published in: Technology
  • Be the first to comment

Scaling Rails: a system-wide approach to performance optimization

  1. Scaling Rails a system-wide approach to performance optimization Exploratory Case Study: http://escolinhas.pt February 2010 FEUP
  2. < Problem > 2 Scaling Rails: a system-wide approach to performance optimization
  3. Web 2.0 3 Scaling Rails: a system-wide approach to performance optimization
  4. Better web Better tools 4 Scaling Rails: a system-wide approach to performance optimization
  5. Ruby on Rails 5 Scaling Rails: a system-wide approach to performance optimization
  6. Popularity brings USERS USERS need scalability 6 Scaling Rails: a system-wide approach to performance optimization
  7. Rails doesn’t scale 7 Scaling Rails: a system-wide approach to performance optimization
  8. Twitter 8 Scaling Rails: a system-wide approach to performance optimization
  9. 9 Scaling Rails: a system-wide approach to performance optimization
  10. Twitter abandoning Rails? 10 Scaling Rails: a system-wide approach to performance optimization
  11. “FWIW: Twitter currently has no plans to abandon RoR. Lots of our code is not in RoR, already, though. Maybe that's why people are confused.” - Evan Williams, Twitter CEO 11 Scaling Rails: a system-wide approach to performance optimization
  12. YellowPages 12 Scaling Rails: a system-wide approach to performance optimization
  13. More users? “Just throw hardware at it.” 13 Scaling Rails: a system-wide approach to performance optimization
  14. Rails doesn’t scale 14 Scaling Rails: a system-wide approach to performance optimization
  15. Hulu 15 Scaling Rails: a system-wide approach to performance optimization
  16. Scribd 16 Scaling Rails: a system-wide approach to performance optimization
  17. They scaled They gave keynotes on HOW 17 Scaling Rails: a system-wide approach to performance optimization
  18. What’s missing here? 18 Scaling Rails: a system-wide approach to performance optimization
  19. No general approaches to scaling 19 Scaling Rails: a system-wide approach to performance optimization
  20. Problem Overview 20 Scaling Rails: a system-wide approach to performance optimization
  21. < Problem Overview > Rails doesn’t scale? Twitter YellowPages Rails doesn’t scale? Hulu Scribd General approaches to scaling? 21 Scaling Rails: a system-wide approach to performance optimization
  22. < Motivation > 22 Scaling Rails: a system-wide approach to performance optimization
  23. WEB WEB WEB WEB WEB 23 Scaling Rails: a system-wide approach to performance optimization
  24. USERS USERS USERS USERS USERS 24 Scaling Rails: a system-wide approach to performance optimization
  25. IMPORTANCE IMPORTANCE IMPORTANCE IMPORTANCE IMPORTANCE 25 Scaling Rails: a system-wide approach to performance optimization
  26. Better web 26 Scaling Rails: a system-wide approach to performance optimization
  27. Better tools 27 Scaling Rails: a system-wide approach to performance optimization
  28. UX involves reasonable response times and service uptime 28 Scaling Rails: a system-wide approach to performance optimization
  29. Most recent tools are great 29 Scaling Rails: a system-wide approach to performance optimization
  30. “Ruby on Rails is a breakthrough in lowering the barriers of entry to programming. Powerful web applications that formerly might have taken weeks or months to develop can be produced in a matter of days.” - Tim O'Reilly, Founder of O'Reilly Media 30 Scaling Rails: a system-wide approach to performance optimization
  31. Everything can be improved There is no perfect tool There are always pitfalls 31 Scaling Rails: a system-wide approach to performance optimization
  32. What’s really motivating? 32 Scaling Rails: a system-wide approach to performance optimization
  33. Fixing pitfalls on great tools Improving UX for everyone 33 Scaling Rails: a system-wide approach to performance optimization
  34. Motivation Overview 34 Scaling Rails: a system-wide approach to performance optimization
  35. < Motivation Overview > Web keeps growing More users, importance Web 2.0 User Experience Also dependant on: Response times and availability Recent tools are great There are always pitfalls Fixing pitfalls on great tools Improving UX for everyone 35 Scaling Rails: a system-wide approach to performance optimization
  36. < Goals > 36 Scaling Rails: a system-wide approach to performance optimization
  37. What about scaling? 37 Scaling Rails: a system-wide approach to performance optimization
  38. Everything is easy Scaling should also be easy 38 Scaling Rails: a system-wide approach to performance optimization
  39. Creation of 39 Scaling Rails: a system-wide approach to performance optimization
  40. Guidelines and conventions for high performance Rails applications 40 Scaling Rails: a system-wide approach to performance optimization
  41. Optimal Rails-oriented configurations for every involved component 41 Scaling Rails: a system-wide approach to performance optimization
  42. Patches to improve the components performance from a Rails-perspective 42 Scaling Rails: a system-wide approach to performance optimization
  43. Objectives Overview 43 Scaling Rails: a system-wide approach to performance optimization
  44. < Objectives Overview > Scaling should be easy Creation of: Generic guidelines and conventions Optimal configurations Patches for all components All from a performance-oriented philosophy centered in Ruby on Rails! 44 Scaling Rails: a system-wide approach to performance optimization
  45. < State of the Art > 45 Scaling Rails: a system-wide approach to performance optimization
  46. < Operating Systems > 46 Scaling Rails: a system-wide approach to performance optimization
  47. Most used OS in servers: Linux 47 Scaling Rails: a system-wide approach to performance optimization
  48. BSD faster than Linux 48 Scaling Rails: a system-wide approach to performance optimization
  49. Linux faster than BSD 49 Scaling Rails: a system-wide approach to performance optimization
  50. Linux Ruby 1.8 2x faster than Windows Ruby 1.8 50 Scaling Rails: a system-wide approach to performance optimization
  51. Linux Ruby 1.9 70% faster than Windows Ruby 1.9 51 Scaling Rails: a system-wide approach to performance optimization
  52. Rails widely deployed in Linux 52 Scaling Rails: a system-wide approach to performance optimization
  53. < Ruby > 53 Scaling Rails: a system-wide approach to performance optimization
  54. Ruby 1.8 is the most widely used 54 Scaling Rails: a system-wide approach to performance optimization
  55. JRuby is the fastest implementation 55 Scaling Rails: a system-wide approach to performance optimization
  56. Ruby 1.8 will soon be outdated 56 Scaling Rails: a system-wide approach to performance optimization
  57. Ruby 1.9 will soon be outdated 57 Scaling Rails: a system-wide approach to performance optimization
  58. Ruby 1.9 has only one interpreter 58 Scaling Rails: a system-wide approach to performance optimization
  59. Ruby MRI 1.9 59 Scaling Rails: a system-wide approach to performance optimization
  60. Ruby MRI 1.9 Rails bottleneck: GC 60 Scaling Rails: a system-wide approach to performance optimization
  61. < Web servers > 61 Scaling Rails: a system-wide approach to performance optimization
  62. Webrick 62 Scaling Rails: a system-wide approach to performance optimization
  63. Webrick Pioneer, slow 63 Scaling Rails: a system-wide approach to performance optimization
  64. Mongrel 64 Scaling Rails: a system-wide approach to performance optimization
  65. Mongrel Much faster, excellent HTML parser 65 Scaling Rails: a system-wide approach to performance optimization
  66. Thin 66 Scaling Rails: a system-wide approach to performance optimization
  67. Thin Mongrel’s HTML parser 67 Scaling Rails: a system-wide approach to performance optimization
  68. Thin New I/O back-end: EventMachine 68 Scaling Rails: a system-wide approach to performance optimization
  69. Thin Faster than those before 69 Scaling Rails: a system-wide approach to performance optimization
  70. Passenger 70 Scaling Rails: a system-wide approach to performance optimization
  71. Passenger Not self-contained 71 Scaling Rails: a system-wide approach to performance optimization
  72. Passenger Module for Apache/Nginx 72 Scaling Rails: a system-wide approach to performance optimization
  73. Passenger Good performance, great at scaling 73 Scaling Rails: a system-wide approach to performance optimization
  74. Thin is the fastest Passenger scales better 74 Scaling Rails: a system-wide approach to performance optimization
  75. < Database > 75 Scaling Rails: a system-wide approach to performance optimization
  76. MySQL 76 Scaling Rails: a system-wide approach to performance optimization
  77. PostgreSQL 77 Scaling Rails: a system-wide approach to performance optimization
  78. MySQL is faster and more popular 78 Scaling Rails: a system-wide approach to performance optimization
  79. Schema-less databases 79 Scaling Rails: a system-wide approach to performance optimization
  80. Schema-less databases Said to have better performance than SQL 80 Scaling Rails: a system-wide approach to performance optimization
  81. MongoDB 81 Scaling Rails: a system-wide approach to performance optimization
  82. CouchDB 82 Scaling Rails: a system-wide approach to performance optimization
  83. Benchmarks are scarce 83 Scaling Rails: a system-wide approach to performance optimization
  84. Major improvement in all DB: Caching, caching, caching! 84 Scaling Rails: a system-wide approach to performance optimization
  85. < Rails > 85 Scaling Rails: a system-wide approach to performance optimization
  86. Rails 2.3 86 Scaling Rails: a system-wide approach to performance optimization
  87. Many performance pitfalls improved in Rails 3 87 Scaling Rails: a system-wide approach to performance optimization
  88. Rails 3 Beta release in February 88 Scaling Rails: a system-wide approach to performance optimization
  89. Immediate need: form helpers, i18n 89 Scaling Rails: a system-wide approach to performance optimization
  90. < Application > 90 Scaling Rails: a system-wide approach to performance optimization
  91. Porting to Ruby 1.9 (from 1.8) 91 Scaling Rails: a system-wide approach to performance optimization
  92. Porting to Rails 3 (from 2.3) 92 Scaling Rails: a system-wide approach to performance optimization
  93. Check code for common pitfalls Eager loading Fetching records in batches Database Indexes Caching 93 Scaling Rails: a system-wide approach to performance optimization
  94. < Methodology > 94 Scaling Rails: a system-wide approach to performance optimization
  95. State of the Art narrows options down 95 Scaling Rails: a system-wide approach to performance optimization
  96. Benchmarking Controlled tests to choose the best software for each component 96 Scaling Rails: a system-wide approach to performance optimization
  97. Tweaking Try out different configurations and architectures 97 Scaling Rails: a system-wide approach to performance optimization
  98. Testing Measuring of results, finding bottlenecks 98 Scaling Rails: a system-wide approach to performance optimization
  99. Developing Improving over bottlenecks 99 Scaling Rails: a system-wide approach to performance optimization
  100. Benchmarking Tweaking Testing Developing and Improving 100 Scaling Rails: a system-wide approach to performance optimization
  101. Recipe: repeat for all components 101 Scaling Rails: a system-wide approach to performance optimization
  102. Rails perspective! 102 Scaling Rails: a system-wide approach to performance optimization
  103. Escolinhas 103 Scaling Rails: a system-wide approach to performance optimization
  104. 104 Scaling Rails: a system-wide approach to performance optimization
  105. Butterfly Effect 105 Scaling Rails: a system-wide approach to performance optimization
  106. 106 Scaling Rails: a system-wide approach to performance optimization
  107. Consider sensitive dependencies 107 Scaling Rails: a system-wide approach to performance optimization
  108. < Work schedule > 108 Scaling Rails: a system-wide approach to performance optimization
  109. 109 Scaling Rails: a system-wide approach to performance optimization
  110. Activity Start Date Duration (weeks) OS testing and choosing 15/02/2010 1 OS kernel testing and tuning 22/02/2010 1 Web server testing and bottleneck research 01/03/2010 1 Web server bottleneck improvements 08/03/2010 1 Ruby interpreter testing and bottleneck research 15/03/2010 1 Ruby interpreter bottleneck improvements 22/03/2010 1.5 DB testing and bottleneck research 31/03/2010 1 DB bottleneck improvements 07/04/2010 1 Rails bottleneck research 14/04/2010 1 Rails bottleneck improvements 21/04/2010 2.5 Escolinhas analysis and bottleneck research 10/05/2010 1 Escolinhas improvements 07/05/2010 3 Writing of dissertation report 07/06/2010 4 110 Scaling Rails: a system-wide approach to performance optimization
  111. < Thank you > http://snaprails.tumblr.com Gonçalo S. Silva Advisor: Ademar Aguiar http://goncalossilva.com http://twitter.com/goncalossilva

×