Deploying and Monitoring
Ruby on Rails
A practical guide




Mathias Meyer and Jonathan Weiss, 25.05.2009
Peritor GmbH
Who are we?

                           Jonathan Weiss

                           •  Consultant for Peritor GmbH in Berli...
Who are we?

                      Mathias Meyer

                      •  Consultant for Peritor GmbH in Berlin
         ...
Agenda




 Infrastructure   Deployment   Practical Session   Monitoring   Q&A




                                       ...
Infrastructure
  Infrastructure




                   5
Simple Rails Setup




       One Rails instance handles all requests
       Rails is single-threaded: There is only one c...
Rails Setup




              7
Rails Setup




              8
Typical Rails Setup

 •  A load-balancer distributes the incoming requests
 •  Some load-balancers will deliver static req...
Rails Setup Options




                      10
Deployment Questions
                                           Apache?          Pound?
             mod_proxy_balancer?

...
What we are going to cover today

Rails Application Server             Proxy/Web Server
 •  FastCGI                       ...
FastCGI




          13
FastCGI

 •  Protocol to communicate with long-running CGI applications
 •  Usage of either mod_fcgi with Apache 1.3 or mo...
FastCGI




          15
FastCGI




          16
17
Mongrel




          18
Mongrel

•  Developed by Zed Shaw as an alternative to FastCGI
•  Complete HTTP-Server that can load arbitrary Ruby-servle...
Mongrel




          20
Mongrel




          Apache 2.2 – mod_proxy_balancer
          Lighttpd
          Nginx
          HA-Proxy
          Poun...
Mongrel Cluster



 Utility to manage several Mongrel instances




                                               22
Mongrel Cluster



 Control Mongrels




                    23
Mongrel and Apache 2.2




                         Define Mongrel Cluster
                         in Apache




         ...
Simple Mongrel and Apache 2.2



 Redirect all traffic to the Mongel cluster




                                          ...
A more complex example

•  Redirect dynamic requests
•  Serve static content
•  Support cached pages
•  Support maintenanc...
Mongrel

•  Very robust
•  Strict HTTP parser
•  Easy to debug (HTTP!)
•  Used to be defacto deployment setup with Apache ...
mod_rails




            28
mod_rails a.k.a Phusion Passenger

 •  Module for Apache 2.2 (and Nginx)
 •  Allows Apache to control Rails instances
 •  ...
Install Phusion Passenger

 Install Apache module




 Load and activate in Apache




                               30
Install Phusion Passenger

 Install Passenger with nginx (installs custom nginx)




 Configure nginx




                 ...
Customized Phusion Passenger


 Control Rails instance number




                                 32
Control Phusion Passenger



 Restart after deployment:




                             33
Phusion Passenger




      One machine

                    34
Phusion Passenger




      One machine   Multiple machines

                                        35
Phusion Passenger

 •  Ready for production
 •  Makes setup easier – on the single machine level
 •  Multiple servers stil...
JRuby




        37
•  Ruby Runtime on the Java Virtual Machine
•  Implemented in Java and Ruby
•  Compiles Ruby into Java-bytecode
•  Integra...
JRuby on Rails




                 39
JRuby on Glassfish




      One machine

                    40
JRuby on Glassfish




      One machine   Multiple machines

                                        41
Setup JRuby on Glassfish

 1.  Download JRuby and Glassfish
 2.  From http://blog.headius.com/2008/08/zero-to-production-in-...
Warble Configuration



                      Define min/max
                      Rails runtimes




                      ...
Glassfish for development


 Shrunk-down version of Glassfish v3 - packaged as a gem
 Can run any Rack-based application



...
Rails Setup




              45
Proxy Options




                46
Proxy Requirements

 •  Hide cluster backend from the user
 •  Load-balancer backend instances
 •  Recognize down hosts
 •...
Apache 2.2

 •  Apache 2.2 introduced mod_proxy_balancer
 •  mod_proxy_balancer can speak to multiple backends and balance...
Apache 2.2

 •  Apache 2.2 introduced mod_proxy_balancer
 •  mod_proxy_balancer can speak to multiple backends and balance...
Apache 2.2

 •  Apache 2.2 introduced mod_proxy_balancer
 •  mod_proxy_balancer can speak to multiple backends and balance...
Apache 2.2

Pro
 •  Stable, robust, and mature
 •  Many people know how to work with Apache
 •  Integrates well with other...
Nginx – From Russia with love




 •  Nginx - popular Russian webserver with good proxy support
 •  Can load-balance multi...
Nginx Configuration

Simple proxy example




Get complete version here:

http://brainspl.at/nginx.conf.txt




           ...
Nginx

Pro
 •  Stable, robust, and fast
 •  Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files
...
Lighttpd

 •  Lightweight and fast webserver
 •  Balancing proxy support
 •  Good FastCGI support
 •  Used to be popular –...
Lighttpd Configuration

 Simple proxy example




                        56
Lighttpd

Pro
 •  Fast and lightweight
 •  Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files
 ...
HA-Proxy


•  HAProxy – reliable, high performance TCP/HTTP load balancer
•  Proxying and content inspection
•  No content...
HAProxy


Simple proxy example




                       59
HAProxy

Pro
 •  Mature, stable, robust, and fast
 •  TCP and HTTP balancing




Con
 •  Few Rails examples
 •  Usually no...
Recommended Setups




                     61
Small Site

             Recommendation
              Apache 2.2 with mod_rails /
              Phusion Passenger




    ...
Medium Site

              Recommendation
               •  Apache/Nginx as the frontend proxy
               •  Passenger...
Large Rails Setup


                    Recommendation
                     •  Redundant load-balancer
                   ...
Heavy Static Files



Recommendation
 •  Deliver static assets
    through separate web
    server farm
 •  Passenger/mod_...
Java Shop

            Recommendation
             •  Deliver a Rails-WAR file and you are
                done
           ...
67
Application Server Handler




                             68
Application Server Handler




                             69
Application Server Handler




                             70
Rack




       71
Rack Handler




               72
Rack




       73
Rack Middleware




                  74
Rack Middleware

•  Authentication (HTTP, OpenID)
•  Sessions
•  Routing
•  Caching
•  Logging
•  Debugging




          ...
Support

Frameworks    Web Server
 •  Rails      •  CGI
 •  Merb       •  FastCGI
 •  Camping    •  Mongrel
 •  Sinatra   ...
Remarks




          77
Ruby Enterprise Edition

 •  Copy-On-Write patches to Ruby 1.8
 •  Saves memory when spawning several Rails instances
 •  ...
Thin, Ebb, Evented Mongrel & Co.

 •  Alternatives to Mongrel
 •  Claim to be faster, lighter, and what have you
 •  Rende...
Infrastructure
  Deployment




                 80
81
Deployment Options




                     82
Deployment Options




                     83
Deployment Options




                     84
Deployment Options




                     85
What does Capistrano do?




                           86
Capistrano Deployment Cycle




                              87
Requirements




               88
What doesn’t Capistrano do?

 •  Plan your initial server setup
 •  Configure basic services




                          ...
Basic Ingredients

 •  The cap command
 •  Variables
 •  Roles
 •  Tasks
 •  Namespaces




                      90
Basic Ingredients - cap

 Your one-stop deployment shop




                                 91
Basic Ingredients - Variables

 •  Configure basic project information
 •  Override Capistrano’s default assumptions
 •  On...
Basic Ingredients - Roles

 •  Define types of servers
 •  Default roles
     •  :www
     •  :app
     •  :db
 •  All can ...
Basic Ingredients - Roles

 Define custom roles as you please




 Can be reused when defining tasks




                   ...
Basic Ingredients - Tasks

 •  Define an atomic set of actions for Capistrano
 •  Can be called from the command line
 •  O...
Basic Ingredients - Tasks
 To find all the tasks available in your project, use




                                       ...
Basic Ingredients - Namespaces


 Group tasks together logically




 Namespaces and tasks are separated with “:”




    ...
Get Your Capistrano On




                         98
Get Your Capistrano On

 Capfile, the place to include more recipes




                                             99
Get Your Capistrano On

 config/deploy.rb, application specific configuration




                                           ...
Capistrano’s Defaults

  •  Your SCM is Subversion
  •  Deployment directory is /u/apps/#{application_name}
  •  User for ...
Get Your Capistrano On




                         102
Get Your Capistrano On

 •  Capistrano expects a directory structure
 •  Can be created with cap deploy:setup




        ...
The Deployment Lifecycle




                           104
The Deployment Lifecycle

 Check the prerequisites:




                            105
The Deployment Lifecycle

 Set up your application for the first time




                                             106
The Deployment Lifecycle

 The initial deployment




     1.  Checks the revision from the local machine
     2.  Checks ...
The Deployment Lifecycle




                           108
The Deployment Lifecycle

 Subsequent deployments




    1.  Checks the revision from the local machine
    2.  Checks ou...
The Deployment Lifecycle




                           110
Common Capistrano Tasks

 Deploy and run migrations




 Run only the migrations




 Restart application servers




 Rol...
Have Your Shell, and Eat It Too




                                  112
Invoking any Command




                       113
Deployment Strategies




                        114
Deployment Strategies

 Direct checkout (from scratch) on the servers




                                                ...
Deployment Strategies

 Keep a cached copy of the current SCM head




                                              116
Deployment Strategies

 Check out locally and transfer




                                  117
Give it a little Spin

  Capistrano expects a script called spin in script/process




  For Passenger




               ...
Customizing Capistrano




                         119
Write your own Tasks




                       120
Write your own Tasks




                       121
Namespace your Tasks




                       122
Callbacks

 Execute a task before another runs




 Execute a task after another has finished




 Callbacks are run in the...
Useful Variables




                   124
Transactions




               125
Transactions




               126
Transactions




               127
Transactions




               128
The Rest

 •  Gem dependencies
 •  Support for deploying through gateway servers
 •  Parallel execution on multiple server...
Vlad the Deployer




                    130
Vlad the Deployer

 •  Simple, Rake-based deployment solution
 •  Comes with defaults for most popular SCM, app and web se...
Vlad the Deployer

 Load




 Configure




                    132
Vlad the Deployer




 Deployment




                    133
Vlad the Deployer

 •  Simple, sometimes too simple
 •  No callback mechanism, instead the Rake syntax must be used




 •...
Chef

•  System Integration Framework
•  Written in Ruby – inspired by Puppet
•  Manage multiple nodes
   •  Configuration
...
Architecture




               136
Recipes

Install and upgrade packages




                               137
Recipes

Create and update remote files




                                138
Recipes

Create and update remote files




                                139
chef-deploy

Enable




Use in recipe




                140
Chef and chef-deploy

 •  In heavy development: sometimes unstable and hard to debug
 •  Be prepared to get your hands dir...
One Click Deploy




                   142
Webistrano




             143
Webistrano

•  Web-UI to Capistrano
•  Manages projects and their stages
•  Alerting and Accounting
•  Scriptable and exte...
Macistrano

 •  Mac-GUI to Webistrano
 •  Fire and monitor deployments from your desktop




                             ...
Infrastructure
Practical Session




                    146
Practical Capistrano




                       147
EC2 Instances

 •  Users
     •  root / railswaycon
     •  deploy / railswaycon
 •  MySQL
     •  root / ‘’
 •  Apache Co...
Git Repository

 •  git://github.com/mattmatt/railscasts.git
 •  Setup the servers
 •  Deploy!




                       ...
Tasks

 •  Make memcached restart with every deploy
 •  (Usually not a good idea, but why not?)




                      ...
Tasks

 •  Make Apache recognize maintenance.html and display it if exists




                                           ...
Maintenance Page

•  Create symbolic link to /etc/apache2/mods-available/rewrite.load in /etc/apache2/
   mods-enabled
•  ...
Tasks

 •  Deploy a Rack-based application
 •  Install Sinatra locally and on the server
 •  git://github.com/mattmatt/rai...
Infrastructure
   Monitoring




                 154
The two questions of monitoring




                                  155
1. Is everything still running?




                                  156
2. What are the trends?




                          157
Monit

•  Process-level monitoring
•  Checks PID-files, ports, and permissions
•  Reacts by executing a script and/or alert...
Monit

MySQL




        159
Monit

Apache




         160
Monit

Mongrel




          161
God

•  Ruby-based configuration
•  Easily extendible
•  Can work with event- and poll-based conditions
•  Comes with less ...
God

Mongrel




          163
Munin

•  Host-level monitoring
•  Master periodically asks nodes for local data
•  Checks system resources and records hi...
Munin




        165
Munin




        166
Munin




        167
Exception Monitoring

 •  Monitor errors raised by your application
 •  Usually notifying you by email
 •  Good old except...
Hoptoad




          169
Exceptional




              170
New Relic

 •  Rails Performance monitoring
 •  Live view into your application




                                      ...
Other Tools

 •  Nagios
 •  Big Brother
 •  New Relic RPM
 •  FiveRuns
 •  JMX




                    172
Infrastructure
Advanced Deployment




                      173
Search




         174
Search

Full text search




Can become very slow on big data sets




                                        175
Full Text Search Engine

Separate Service
 •  Creates full text index
 •  Application queries search daemon
 •  Index upda...
Search Slave

Database replication slave
 •  Has complete dataset
 •  Migrates slow search queries from master
 •  Can use...
Database Index

PostgreSQL Tsearch2
 •  Core since 8.3
 •  Allows to create full text index on multiple columns
    or arb...
What to use?

Different characteristics
 •  Real-time updates and stale data
 •  Lost updates
 •  Performance
 •  Document...
Background Processing




                        180
Problem

Long running tasks
 •  Resizing uploaded images
 •  Mailing
 •  Computing an expensive operation
 •  Accessing sl...
Solution

Asynchronous processing in the background




      Message/Queue                         Scheduler



         ...
Background Processing




                        183
Options



 Options for message bus:   Options for background process:
 •  Database                •  (Ruby) Daemon
 •  Am...
Database/Ruby daemon example




                               185
Cloud Infrastructure




                       186
Cloud Infrastructure

Servers come and go
 •  You do not know your servers before deploying
 •  Restarting is the same as ...
Solution #1

Query and manually adjust
 •  Servers do not change that often
 •  New nodes probably need manual interventio...
Solution #2

Use a central directory service
 •  A central place to manage your running instances
 •  Instances query the ...
Solution #2

Use a central directory service
 •  A central place to manage your running instances
 •  Instances query the ...
Central Directory

Different Implementations
 •  File on S3
 •  SimpleDB
 •  A complete service,
    capable of monitoring...
Q&A




      192
Peritor GmbH

Teutonenstraße 16
14129 Berlin
Telefon: +49 (0)30 69 20 09 84 0
Telefax: +49 (0)30 69 20 09 84 9

Internet: ...
Upcoming SlideShare
Loading in...5
×

Deploying And Monitoring Rails

11,745

Published on

While it’s easy to get started developing applications using Ruby on Rails, the hard part is (as with most technologies) how to put it into production? There are a lot of options like Mongrel, nginx, Apache and JRuby, all of them difficult to evaulate without deeper knowledge of Rails and your application’s scaling needs. We will look at the available options to get a clearer picture how each setup is in different situations, and how you can utilize Capistrano for an easy deployment process in those situations. Simple as well as more complex setups will be discussed. We’ll try to include your problems as well as your experiences to discuss specific issues with deployment and scalability.

Published in: Technology
1 Comment
43 Likes
Statistics
Notes
No Downloads
Views
Total Views
11,745
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
460
Comments
1
Likes
43
Embeds 0
No embeds

No notes for slide

Deploying And Monitoring Rails

  1. 1. Deploying and Monitoring Ruby on Rails A practical guide Mathias Meyer and Jonathan Weiss, 25.05.2009 Peritor GmbH
  2. 2. Who are we? Jonathan Weiss •  Consultant for Peritor GmbH in Berlin •  Specialized in Rails, Scaling, Deployment, and Code Review •  Webistrano - Rails deployment tool •  FreeBSD Rubygems and Ruby on Rails maintainer http://www.peritor.com http://blog.innerewut.de 2
  3. 3. Who are we? Mathias Meyer •  Consultant for Peritor GmbH in Berlin •  Specialized in Rails, Performance/Database Tuning, Deployment, and Refactoring •  Maintainer of Macistrano – Webistrano-Client for Mac OS X (and soon the iPhone), run_later and acts_as_solr http://www.paperplanes.de 3
  4. 4. Agenda Infrastructure Deployment Practical Session Monitoring Q&A 4
  5. 5. Infrastructure Infrastructure 5
  6. 6. Simple Rails Setup One Rails instance handles all requests Rails is single-threaded: There is only one concurrent request 6
  7. 7. Rails Setup 7
  8. 8. Rails Setup 8
  9. 9. Typical Rails Setup •  A load-balancer distributes the incoming requests •  Some load-balancers will deliver static requests themselves •  Several Rails instances handle all requests •  Number of concurrent requests equals number of Rails instances 9
  10. 10. Rails Setup Options 10
  11. 11. Deployment Questions Apache? Pound? mod_proxy_balancer? FastCGI? Mongrel? Proxy? Load-balancer? Nginx? mod_rails? Ebb? HA-Proxy? Reverse Proxy? Swiftiply? Phusion Passenger? Thin? Pen? Rails Application Server? Lighttpd? 11
  12. 12. What we are going to cover today Rails Application Server Proxy/Web Server •  FastCGI •  Apache2 •  Mongrel •  Nginx •  mod_rails / Phussion Passenger •  Lighttpd •  JRuby + Glassfish & Co. •  HA-Proxy 12
  13. 13. FastCGI 13
  14. 14. FastCGI •  Protocol to communicate with long-running CGI applications •  Usage of either mod_fcgi with Apache 1.3 or mod_fcgi with Lighttpd •  Proxy local and remote FastCGI instances •  Oldest way of deploying Rails •  Deprecated and unstable •  Hard to debug (FastCGI protocol) 14
  15. 15. FastCGI 15
  16. 16. FastCGI 16
  17. 17. 17
  18. 18. Mongrel 18
  19. 19. Mongrel •  Developed by Zed Shaw as an alternative to FastCGI •  Complete HTTP-Server that can load arbitrary Ruby-servlets •  Built-in Rails support 19
  20. 20. Mongrel 20
  21. 21. Mongrel Apache 2.2 – mod_proxy_balancer Lighttpd Nginx HA-Proxy Pound 21
  22. 22. Mongrel Cluster Utility to manage several Mongrel instances 22
  23. 23. Mongrel Cluster Control Mongrels 23
  24. 24. Mongrel and Apache 2.2 Define Mongrel Cluster in Apache 24
  25. 25. Simple Mongrel and Apache 2.2 Redirect all traffic to the Mongel cluster 25
  26. 26. A more complex example •  Redirect dynamic requests •  Serve static content •  Support cached pages •  Support maintenance page •  Enable client-side caching of images, stylesheets, and JavaScript •  Compress output if supported 26
  27. 27. Mongrel •  Very robust •  Strict HTTP parser •  Easy to debug (HTTP!) •  Used to be defacto deployment setup with Apache 2.2 and mod_proxy_balancer •  Can be a bit difficult to setup (mongrel_cluster, ports, Apache) •  Not so easy on mass/virtual hosting 27
  28. 28. mod_rails 28
  29. 29. mod_rails a.k.a Phusion Passenger •  Module for Apache 2.2 (and Nginx) •  Allows Apache to control Rails instances •  Apache starts and stops application instances depending on the application load •  Able to run any Rack-compatible Ruby application (Merb, Sinatra & Co.) •  Only manages Rails on one host - no remote instances •  Combine with HTTP-Proxy / balancing solution 29
  30. 30. Install Phusion Passenger Install Apache module Load and activate in Apache 30
  31. 31. Install Phusion Passenger Install Passenger with nginx (installs custom nginx) Configure nginx 31
  32. 32. Customized Phusion Passenger Control Rails instance number 32
  33. 33. Control Phusion Passenger Restart after deployment: 33
  34. 34. Phusion Passenger One machine 34
  35. 35. Phusion Passenger One machine Multiple machines 35
  36. 36. Phusion Passenger •  Ready for production •  Makes setup easier – on the single machine level •  Multiple servers still require load balancer •  Suitable for mass-hosting •  upcoming standard way of deploying Rails 36
  37. 37. JRuby 37
  38. 38. •  Ruby Runtime on the Java Virtual Machine •  Implemented in Java and Ruby •  Compiles Ruby into Java-bytecode •  Integrates with Java code and libraries •  Java’s promises of native threads and JIT •  Allows for Ruby/Rails applications to be packaged as WAR files •  WAR files deployable on any J2EE-container: Glassfish, JBoss, Tomcat, Jetty, … 38
  39. 39. JRuby on Rails 39
  40. 40. JRuby on Glassfish One machine 40
  41. 41. JRuby on Glassfish One machine Multiple machines 41
  42. 42. Setup JRuby on Glassfish 1.  Download JRuby and Glassfish 2.  From http://blog.headius.com/2008/08/zero-to-production-in-15-minutes.html 42
  43. 43. Warble Configuration Define min/max Rails runtimes 43
  44. 44. Glassfish for development Shrunk-down version of Glassfish v3 - packaged as a gem Can run any Rack-based application Installation: •  $ jruby -S gem install glassfish •  $ glassfish •  There is no Step 3 44
  45. 45. Rails Setup 45
  46. 46. Proxy Options 46
  47. 47. Proxy Requirements •  Hide cluster backend from the user •  Load-balancer backend instances •  Recognize down hosts •  Fair scheduler •  (Deliver static content) 47
  48. 48. Apache 2.2 •  Apache 2.2 introduced mod_proxy_balancer •  mod_proxy_balancer can speak to multiple backends and balance requests •  Apache can acts as a pure proxy or can also serve static files 48
  49. 49. Apache 2.2 •  Apache 2.2 introduced mod_proxy_balancer •  mod_proxy_balancer can speak to multiple backends and balance requests •  Apache can acts as a pure proxy or can also serve static files 49
  50. 50. Apache 2.2 •  Apache 2.2 introduced mod_proxy_balancer •  mod_proxy_balancer can speak to multiple backends and balance requests •  Apache can acts as a pure proxy or can also serve static files 50
  51. 51. Apache 2.2 Pro •  Stable, robust, and mature •  Many people know how to work with Apache •  Integrates well with other modules (SVN, DAV, Auth, …) Con •  Apache can be complicated to configure •  The stock Apache is quite resource-hungry compared to pure proxy solutions 51
  52. 52. Nginx – From Russia with love •  Nginx - popular Russian webserver with good proxy support •  Can load-balance multiple backends and deliver static content •  Quite popular with Mongrel as the Rails backend •  Recent Passenger support 52
  53. 53. Nginx Configuration Simple proxy example Get complete version here: http://brainspl.at/nginx.conf.txt 53
  54. 54. Nginx Pro •  Stable, robust, and fast •  Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files •  Simpler configuration file •  Can directly talk to memcached - SSI Con •  More documentation would be nice •  No equivalent for many Apache modules 54
  55. 55. Lighttpd •  Lightweight and fast webserver •  Balancing proxy support •  Good FastCGI support •  Used to be popular – until Mongrel came around 55
  56. 56. Lighttpd Configuration Simple proxy example 56
  57. 57. Lighttpd Pro •  Fast and lightweight •  Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files •  Simpler configuration file Con •  Unstable for some people •  Slow development cycle •  More documentation would be nice •  Configuration file can be too simple (virtual host aliasing) •  No equivalent for many Apache modules •  (No Passenger support) 57
  58. 58. HA-Proxy •  HAProxy – reliable, high performance TCP/HTTP load balancer •  Proxying and content inspection •  No content serving, just a proxy •  Mature proxy module (fair scheduler) •  ACL support See also similar Pound and Pen 58
  59. 59. HAProxy Simple proxy example 59
  60. 60. HAProxy Pro •  Mature, stable, robust, and fast •  TCP and HTTP balancing Con •  Few Rails examples •  Usually not needed in a Rails setup 60
  61. 61. Recommended Setups 61
  62. 62. Small Site Recommendation Apache 2.2 with mod_rails / Phusion Passenger 62
  63. 63. Medium Site Recommendation •  Apache/Nginx as the frontend proxy •  Passenger/mod_rails as the backend •  Deliver static files with Apache 63
  64. 64. Large Rails Setup Recommendation •  Redundant load-balancer •  Redundant proxy / web •  Passenger/mod_rails 64
  65. 65. Heavy Static Files Recommendation •  Deliver static assets through separate web server farm •  Passenger/mod_rails 65
  66. 66. Java Shop Recommendation •  Deliver a Rails-WAR file and you are done •  Integrate with existing Java landscape and infrastructure 66
  67. 67. 67
  68. 68. Application Server Handler 68
  69. 69. Application Server Handler 69
  70. 70. Application Server Handler 70
  71. 71. Rack 71
  72. 72. Rack Handler 72
  73. 73. Rack 73
  74. 74. Rack Middleware 74
  75. 75. Rack Middleware •  Authentication (HTTP, OpenID) •  Sessions •  Routing •  Caching •  Logging •  Debugging 75
  76. 76. Support Frameworks Web Server •  Rails •  CGI •  Merb •  FastCGI •  Camping •  Mongrel •  Sinatra •  Phusion Passenger •  Mack •  JRuby Servlets •  Maveric •  WEBrick •  Halcyon •  Ebb •  Ramaze •  Thin •  Rum •  LiteSpeed •  Vintage •  Swiftcore •  Waves •  Unicorn 76
  77. 77. Remarks 77
  78. 78. Ruby Enterprise Edition •  Copy-On-Write patches to Ruby 1.8 •  Saves memory when spawning several Rails instances •  Used by Phusion Passenger if available 78
  79. 79. Thin, Ebb, Evented Mongrel & Co. •  Alternatives to Mongrel •  Claim to be faster, lighter, and what have you •  Rendering “Hello World” is usually not your bottleneck Stick with stable and robust Mongrel, or better yet, with Passenger 79
  80. 80. Infrastructure Deployment 80
  81. 81. 81
  82. 82. Deployment Options 82
  83. 83. Deployment Options 83
  84. 84. Deployment Options 84
  85. 85. Deployment Options 85
  86. 86. What does Capistrano do? 86
  87. 87. Capistrano Deployment Cycle 87
  88. 88. Requirements 88
  89. 89. What doesn’t Capistrano do? •  Plan your initial server setup •  Configure basic services 89
  90. 90. Basic Ingredients •  The cap command •  Variables •  Roles •  Tasks •  Namespaces 90
  91. 91. Basic Ingredients - cap Your one-stop deployment shop 91
  92. 92. Basic Ingredients - Variables •  Configure basic project information •  Override Capistrano’s default assumptions •  Once set, variables are available globally •  Defined using the set method 92
  93. 93. Basic Ingredients - Roles •  Define types of servers •  Default roles •  :www •  :app •  :db •  All can point to the same server •  But all three must be defined •  At least one database server needs to be primary 93
  94. 94. Basic Ingredients - Roles Define custom roles as you please Can be reused when defining tasks 94
  95. 95. Basic Ingredients - Tasks •  Define an atomic set of actions for Capistrano •  Can be called from the command line •  Or other tasks 95
  96. 96. Basic Ingredients - Tasks To find all the tasks available in your project, use 96
  97. 97. Basic Ingredients - Namespaces Group tasks together logically Namespaces and tasks are separated with “:” 97
  98. 98. Get Your Capistrano On 98
  99. 99. Get Your Capistrano On Capfile, the place to include more recipes 99
  100. 100. Get Your Capistrano On config/deploy.rb, application specific configuration 100
  101. 101. Capistrano’s Defaults •  Your SCM is Subversion •  Deployment directory is /u/apps/#{application_name} •  User for SCM and SSH is the currently logged-in user •  Commands are run with sudo 101
  102. 102. Get Your Capistrano On 102
  103. 103. Get Your Capistrano On •  Capistrano expects a directory structure •  Can be created with cap deploy:setup 103
  104. 104. The Deployment Lifecycle 104
  105. 105. The Deployment Lifecycle Check the prerequisites: 105
  106. 106. The Deployment Lifecycle Set up your application for the first time 106
  107. 107. The Deployment Lifecycle The initial deployment 1.  Checks the revision from the local machine 2.  Checks out the code on the remote machines 3.  Sets a link called current pointing to the lates release 4.  Runs the migrations 5.  Fires up application servers 107
  108. 108. The Deployment Lifecycle 108
  109. 109. The Deployment Lifecycle Subsequent deployments 1.  Checks the revision from the local machine 2.  Checks out the code on the remote machines 3.  Updates current link 4.  Restarts application servers 109
  110. 110. The Deployment Lifecycle 110
  111. 111. Common Capistrano Tasks Deploy and run migrations Run only the migrations Restart application servers Rollback to the previous release 111
  112. 112. Have Your Shell, and Eat It Too 112
  113. 113. Invoking any Command 113
  114. 114. Deployment Strategies 114
  115. 115. Deployment Strategies Direct checkout (from scratch) on the servers 115
  116. 116. Deployment Strategies Keep a cached copy of the current SCM head 116
  117. 117. Deployment Strategies Check out locally and transfer 117
  118. 118. Give it a little Spin Capistrano expects a script called spin in script/process For Passenger 118
  119. 119. Customizing Capistrano 119
  120. 120. Write your own Tasks 120
  121. 121. Write your own Tasks 121
  122. 122. Namespace your Tasks 122
  123. 123. Callbacks Execute a task before another runs Execute a task after another has finished Callbacks are run in the order they’re defined 123
  124. 124. Useful Variables 124
  125. 125. Transactions 125
  126. 126. Transactions 126
  127. 127. Transactions 127
  128. 128. Transactions 128
  129. 129. The Rest •  Gem dependencies •  Support for deploying through gateway servers •  Parallel execution on multiple servers •  Server setup with deprec gem 129
  130. 130. Vlad the Deployer 130
  131. 131. Vlad the Deployer •  Simple, Rake-based deployment solution •  Comes with defaults for most popular SCM, app and web servers 131
  132. 132. Vlad the Deployer Load Configure 132
  133. 133. Vlad the Deployer Deployment 133
  134. 134. Vlad the Deployer •  Simple, sometimes too simple •  No callback mechanism, instead the Rake syntax must be used •  Very non-verbose output, using --trace recommended 134
  135. 135. Chef •  System Integration Framework •  Written in Ruby – inspired by Puppet •  Manage multiple nodes •  Configuration •  Packages •  Custom scripts and commands 135
  136. 136. Architecture 136
  137. 137. Recipes Install and upgrade packages 137
  138. 138. Recipes Create and update remote files 138
  139. 139. Recipes Create and update remote files 139
  140. 140. chef-deploy Enable Use in recipe 140
  141. 141. Chef and chef-deploy •  In heavy development: sometimes unstable and hard to debug •  Be prepared to get your hands dirty •  Better suited for configuration than deployment •  Community recipes at http://github.com/opscode/cookbooks 141
  142. 142. One Click Deploy 142
  143. 143. Webistrano 143
  144. 144. Webistrano •  Web-UI to Capistrano •  Manages projects and their stages •  Alerting and Accounting •  Scriptable and extendable •  BSD License http://labs.peritor.com/webistrano 144
  145. 145. Macistrano •  Mac-GUI to Webistrano •  Fire and monitor deployments from your desktop http://github.com/mattmatt/macistrano 145
  146. 146. Infrastructure Practical Session 146
  147. 147. Practical Capistrano 147
  148. 148. EC2 Instances •  Users •  root / railswaycon •  deploy / railswaycon •  MySQL •  root / ‘’ •  Apache Configuration •  /etc/apache2/sites-enabled/default •  Rails •  /srv/www 148
  149. 149. Git Repository •  git://github.com/mattmatt/railscasts.git •  Setup the servers •  Deploy! 149
  150. 150. Tasks •  Make memcached restart with every deploy •  (Usually not a good idea, but why not?) 150
  151. 151. Tasks •  Make Apache recognize maintenance.html and display it if exists 151
  152. 152. Maintenance Page •  Create symbolic link to /etc/apache2/mods-available/rewrite.load in /etc/apache2/ mods-enabled •  Update virtual host configuration with rewrite rules •  Run cap deploy:web:disable •  Hook it into the deployment process 152
  153. 153. Tasks •  Deploy a Rack-based application •  Install Sinatra locally and on the server •  git://github.com/mattmatt/railswaycon.git •  Create deployment configuration •  Create Apache configuration 153
  154. 154. Infrastructure Monitoring 154
  155. 155. The two questions of monitoring 155
  156. 156. 1. Is everything still running? 156
  157. 157. 2. What are the trends? 157
  158. 158. Monit •  Process-level monitoring •  Checks PID-files, ports, and permissions •  Reacts by executing a script and/or alerting •  Has a web GUI for administering nodes 158
  159. 159. Monit MySQL 159
  160. 160. Monit Apache 160
  161. 161. Monit Mongrel 161
  162. 162. God •  Ruby-based configuration •  Easily extendible •  Can work with event- and poll-based conditions •  Comes with less recipes than Monit 162
  163. 163. God Mongrel 163
  164. 164. Munin •  Host-level monitoring •  Master periodically asks nodes for local data •  Checks system resources and records historical data •  Allows to recognize trends and make predictions •  Alerting support 164
  165. 165. Munin 165
  166. 166. Munin 166
  167. 167. Munin 167
  168. 168. Exception Monitoring •  Monitor errors raised by your application •  Usually notifying you by email •  Good old exception_notification Plugin •  Exceptions as a Service 168
  169. 169. Hoptoad 169
  170. 170. Exceptional 170
  171. 171. New Relic •  Rails Performance monitoring •  Live view into your application 171
  172. 172. Other Tools •  Nagios •  Big Brother •  New Relic RPM •  FiveRuns •  JMX 172
  173. 173. Infrastructure Advanced Deployment 173
  174. 174. Search 174
  175. 175. Search Full text search Can become very slow on big data sets 175
  176. 176. Full Text Search Engine Separate Service •  Creates full text index •  Application queries search daemon •  Index update through application or database Possible Engines •  Ferret •  Sphinx •  Solr •  Lucene •  … 176
  177. 177. Search Slave Database replication slave •  Has complete dataset •  Migrates slow search queries from master •  Can use different database table engine 177
  178. 178. Database Index PostgreSQL Tsearch2 •  Core since 8.3 •  Allows to create full text index on multiple columns or arbitrary SQL expressions MySQL MyISAM FULLTEXT index •  Only works with MySQL <= 5.0 and MyISAM tables •  Full text index on multiple columns 178
  179. 179. What to use? Different characteristics •  Real-time updates and stale data •  Lost updates •  Performance •  Document content and format •  Complexity 179
  180. 180. Background Processing 180
  181. 181. Problem Long running tasks •  Resizing uploaded images •  Mailing •  Computing an expensive operation •  Accessing slow back-ends When running inside request-response-cycle •  Blocks user •  Blocks Rails instance •  Hard to monitor and debug 181
  182. 182. Solution Asynchronous processing in the background Message/Queue Scheduler 182
  183. 183. Background Processing 183
  184. 184. Options Options for message bus: Options for background process: •  Database •  (Ruby) Daemon •  Amazon SQS •  Cron job with script/runner •  Drb •  Forked process •  Memcache •  Delayed Job / BJ / (Backgroundrb) •  ActiveMQ •  run_later •  … •  …. 184
  185. 185. Database/Ruby daemon example 185
  186. 186. Cloud Infrastructure 186
  187. 187. Cloud Infrastructure Servers come and go •  You do not know your servers before deploying •  Restarting is the same as introducing a new machine You can’t hardcode IPs database.yml 187
  188. 188. Solution #1 Query and manually adjust •  Servers do not change that often •  New nodes probably need manual intervention •  Use AWS ElasticIPs to ease the pain Set servers dynamically AWS Elastic IP 188
  189. 189. Solution #2 Use a central directory service •  A central place to manage your running instances •  Instances query the directory and react 189
  190. 190. Solution #2 Use a central directory service •  A central place to manage your running instances •  Instances query the directory and react 190
  191. 191. Central Directory Different Implementations •  File on S3 •  SimpleDB •  A complete service, capable of monitoring and controlling your instances 191
  192. 192. Q&A 192
  193. 193. Peritor GmbH Teutonenstraße 16 14129 Berlin Telefon: +49 (0)30 69 20 09 84 0 Telefax: +49 (0)30 69 20 09 84 9 Internet: www.peritor.com E-Mail: kontakt@peritor.com 193 Peritor GmbH - Alle Rechte vorbehalten 193
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×