Deploying And Monitoring Rails
Upcoming SlideShare
Loading in...5
×
 

Deploying And Monitoring Rails

on

  • 16,070 views

While it’s easy to get started developing applications using Ruby on Rails, the hard part is (as with most technologies) how to put it into production? There are a lot of options like Mongrel, ...

While it’s easy to get started developing applications using Ruby on Rails, the hard part is (as with most technologies) how to put it into production? There are a lot of options like Mongrel, nginx, Apache and JRuby, all of them difficult to evaulate without deeper knowledge of Rails and your application’s scaling needs. We will look at the available options to get a clearer picture how each setup is in different situations, and how you can utilize Capistrano for an easy deployment process in those situations. Simple as well as more complex setups will be discussed. We’ll try to include your problems as well as your experiences to discuss specific issues with deployment and scalability.

Statistics

Views

Total Views
16,070
Views on SlideShare
16,006
Embed Views
64

Actions

Likes
42
Downloads
451
Comments
1

8 Embeds 64

http://www.slideshare.net 53
http://localhost:3000 5
http://localhost:3002 1
http://127.0.0.1:3000 1
http://ror.pmar.inav2.com 1
http://www.inav2.com 1
http://localhost 1
http://seekr-artemis.heroku.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Deploying And Monitoring Rails Deploying And Monitoring Rails Presentation Transcript

  • Deploying and Monitoring Ruby on Rails A practical guide Mathias Meyer and Jonathan Weiss, 25.05.2009 Peritor GmbH
  • Who are we? Jonathan Weiss •  Consultant for Peritor GmbH in Berlin •  Specialized in Rails, Scaling, Deployment, and Code Review •  Webistrano - Rails deployment tool •  FreeBSD Rubygems and Ruby on Rails maintainer http://www.peritor.com http://blog.innerewut.de 2
  • Who are we? Mathias Meyer •  Consultant for Peritor GmbH in Berlin •  Specialized in Rails, Performance/Database Tuning, Deployment, and Refactoring •  Maintainer of Macistrano – Webistrano-Client for Mac OS X (and soon the iPhone), run_later and acts_as_solr http://www.paperplanes.de 3
  • Agenda Infrastructure Deployment Practical Session Monitoring Q&A 4
  • Infrastructure Infrastructure 5
  • Simple Rails Setup One Rails instance handles all requests Rails is single-threaded: There is only one concurrent request 6
  • Rails Setup 7
  • Rails Setup 8
  • Typical Rails Setup •  A load-balancer distributes the incoming requests •  Some load-balancers will deliver static requests themselves •  Several Rails instances handle all requests •  Number of concurrent requests equals number of Rails instances 9
  • Rails Setup Options 10
  • Deployment Questions Apache? Pound? mod_proxy_balancer? FastCGI? Mongrel? Proxy? Load-balancer? Nginx? mod_rails? Ebb? HA-Proxy? Reverse Proxy? Swiftiply? Phusion Passenger? Thin? Pen? Rails Application Server? Lighttpd? 11
  • What we are going to cover today Rails Application Server Proxy/Web Server •  FastCGI •  Apache2 •  Mongrel •  Nginx •  mod_rails / Phussion Passenger •  Lighttpd •  JRuby + Glassfish & Co. •  HA-Proxy 12
  • FastCGI 13
  • FastCGI •  Protocol to communicate with long-running CGI applications •  Usage of either mod_fcgi with Apache 1.3 or mod_fcgi with Lighttpd •  Proxy local and remote FastCGI instances •  Oldest way of deploying Rails •  Deprecated and unstable •  Hard to debug (FastCGI protocol) 14
  • FastCGI 15
  • FastCGI 16
  • 17
  • Mongrel 18
  • Mongrel •  Developed by Zed Shaw as an alternative to FastCGI •  Complete HTTP-Server that can load arbitrary Ruby-servlets •  Built-in Rails support 19
  • Mongrel 20
  • Mongrel Apache 2.2 – mod_proxy_balancer Lighttpd Nginx HA-Proxy Pound 21
  • Mongrel Cluster Utility to manage several Mongrel instances 22
  • Mongrel Cluster Control Mongrels 23
  • Mongrel and Apache 2.2 Define Mongrel Cluster in Apache 24
  • Simple Mongrel and Apache 2.2 Redirect all traffic to the Mongel cluster 25
  • A more complex example •  Redirect dynamic requests •  Serve static content •  Support cached pages •  Support maintenance page •  Enable client-side caching of images, stylesheets, and JavaScript •  Compress output if supported 26
  • Mongrel •  Very robust •  Strict HTTP parser •  Easy to debug (HTTP!) •  Used to be defacto deployment setup with Apache 2.2 and mod_proxy_balancer •  Can be a bit difficult to setup (mongrel_cluster, ports, Apache) •  Not so easy on mass/virtual hosting 27
  • mod_rails 28
  • mod_rails a.k.a Phusion Passenger •  Module for Apache 2.2 (and Nginx) •  Allows Apache to control Rails instances •  Apache starts and stops application instances depending on the application load •  Able to run any Rack-compatible Ruby application (Merb, Sinatra & Co.) •  Only manages Rails on one host - no remote instances •  Combine with HTTP-Proxy / balancing solution 29
  • Install Phusion Passenger Install Apache module Load and activate in Apache 30
  • Install Phusion Passenger Install Passenger with nginx (installs custom nginx) Configure nginx 31
  • Customized Phusion Passenger Control Rails instance number 32
  • Control Phusion Passenger Restart after deployment: 33
  • Phusion Passenger One machine 34
  • Phusion Passenger One machine Multiple machines 35
  • Phusion Passenger •  Ready for production •  Makes setup easier – on the single machine level •  Multiple servers still require load balancer •  Suitable for mass-hosting •  upcoming standard way of deploying Rails 36
  • JRuby 37
  • •  Ruby Runtime on the Java Virtual Machine •  Implemented in Java and Ruby •  Compiles Ruby into Java-bytecode •  Integrates with Java code and libraries •  Java’s promises of native threads and JIT •  Allows for Ruby/Rails applications to be packaged as WAR files •  WAR files deployable on any J2EE-container: Glassfish, JBoss, Tomcat, Jetty, … 38
  • JRuby on Rails 39
  • JRuby on Glassfish One machine 40
  • JRuby on Glassfish One machine Multiple machines 41
  • Setup JRuby on Glassfish 1.  Download JRuby and Glassfish 2.  From http://blog.headius.com/2008/08/zero-to-production-in-15-minutes.html 42
  • Warble Configuration Define min/max Rails runtimes 43
  • Glassfish for development Shrunk-down version of Glassfish v3 - packaged as a gem Can run any Rack-based application Installation: •  $ jruby -S gem install glassfish •  $ glassfish •  There is no Step 3 44
  • Rails Setup 45
  • Proxy Options 46
  • Proxy Requirements •  Hide cluster backend from the user •  Load-balancer backend instances •  Recognize down hosts •  Fair scheduler •  (Deliver static content) 47
  • Apache 2.2 •  Apache 2.2 introduced mod_proxy_balancer •  mod_proxy_balancer can speak to multiple backends and balance requests •  Apache can acts as a pure proxy or can also serve static files 48
  • Apache 2.2 •  Apache 2.2 introduced mod_proxy_balancer •  mod_proxy_balancer can speak to multiple backends and balance requests •  Apache can acts as a pure proxy or can also serve static files 49
  • Apache 2.2 •  Apache 2.2 introduced mod_proxy_balancer •  mod_proxy_balancer can speak to multiple backends and balance requests •  Apache can acts as a pure proxy or can also serve static files 50
  • Apache 2.2 Pro •  Stable, robust, and mature •  Many people know how to work with Apache •  Integrates well with other modules (SVN, DAV, Auth, …) Con •  Apache can be complicated to configure •  The stock Apache is quite resource-hungry compared to pure proxy solutions 51
  • Nginx – From Russia with love •  Nginx - popular Russian webserver with good proxy support •  Can load-balance multiple backends and deliver static content •  Quite popular with Mongrel as the Rails backend •  Recent Passenger support 52
  • Nginx Configuration Simple proxy example Get complete version here: http://brainspl.at/nginx.conf.txt 53
  • Nginx Pro •  Stable, robust, and fast •  Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files •  Simpler configuration file •  Can directly talk to memcached - SSI Con •  More documentation would be nice •  No equivalent for many Apache modules 54
  • Lighttpd •  Lightweight and fast webserver •  Balancing proxy support •  Good FastCGI support •  Used to be popular – until Mongrel came around 55
  • Lighttpd Configuration Simple proxy example 56
  • Lighttpd Pro •  Fast and lightweight •  Uses fewer resources (CPU and RAM) than Apache for proxy-mode and static files •  Simpler configuration file Con •  Unstable for some people •  Slow development cycle •  More documentation would be nice •  Configuration file can be too simple (virtual host aliasing) •  No equivalent for many Apache modules •  (No Passenger support) 57
  • HA-Proxy •  HAProxy – reliable, high performance TCP/HTTP load balancer •  Proxying and content inspection •  No content serving, just a proxy •  Mature proxy module (fair scheduler) •  ACL support See also similar Pound and Pen 58
  • HAProxy Simple proxy example 59
  • HAProxy Pro •  Mature, stable, robust, and fast •  TCP and HTTP balancing Con •  Few Rails examples •  Usually not needed in a Rails setup 60
  • Recommended Setups 61
  • Small Site Recommendation Apache 2.2 with mod_rails / Phusion Passenger 62
  • Medium Site Recommendation •  Apache/Nginx as the frontend proxy •  Passenger/mod_rails as the backend •  Deliver static files with Apache 63
  • Large Rails Setup Recommendation •  Redundant load-balancer •  Redundant proxy / web •  Passenger/mod_rails 64
  • Heavy Static Files Recommendation •  Deliver static assets through separate web server farm •  Passenger/mod_rails 65
  • Java Shop Recommendation •  Deliver a Rails-WAR file and you are done •  Integrate with existing Java landscape and infrastructure 66
  • 67
  • Application Server Handler 68
  • Application Server Handler 69
  • Application Server Handler 70
  • Rack 71
  • Rack Handler 72
  • Rack 73
  • Rack Middleware 74
  • Rack Middleware •  Authentication (HTTP, OpenID) •  Sessions •  Routing •  Caching •  Logging •  Debugging 75
  • Support Frameworks Web Server •  Rails •  CGI •  Merb •  FastCGI •  Camping •  Mongrel •  Sinatra •  Phusion Passenger •  Mack •  JRuby Servlets •  Maveric •  WEBrick •  Halcyon •  Ebb •  Ramaze •  Thin •  Rum •  LiteSpeed •  Vintage •  Swiftcore •  Waves •  Unicorn 76
  • Remarks 77
  • Ruby Enterprise Edition •  Copy-On-Write patches to Ruby 1.8 •  Saves memory when spawning several Rails instances •  Used by Phusion Passenger if available 78
  • Thin, Ebb, Evented Mongrel & Co. •  Alternatives to Mongrel •  Claim to be faster, lighter, and what have you •  Rendering “Hello World” is usually not your bottleneck Stick with stable and robust Mongrel, or better yet, with Passenger 79
  • Infrastructure Deployment 80
  • 81
  • Deployment Options 82
  • Deployment Options 83
  • Deployment Options 84
  • Deployment Options 85
  • What does Capistrano do? 86
  • Capistrano Deployment Cycle 87
  • Requirements 88
  • What doesn’t Capistrano do? •  Plan your initial server setup •  Configure basic services 89
  • Basic Ingredients •  The cap command •  Variables •  Roles •  Tasks •  Namespaces 90
  • Basic Ingredients - cap Your one-stop deployment shop 91
  • Basic Ingredients - Variables •  Configure basic project information •  Override Capistrano’s default assumptions •  Once set, variables are available globally •  Defined using the set method 92
  • Basic Ingredients - Roles •  Define types of servers •  Default roles •  :www •  :app •  :db •  All can point to the same server •  But all three must be defined •  At least one database server needs to be primary 93
  • Basic Ingredients - Roles Define custom roles as you please Can be reused when defining tasks 94
  • Basic Ingredients - Tasks •  Define an atomic set of actions for Capistrano •  Can be called from the command line •  Or other tasks 95
  • Basic Ingredients - Tasks To find all the tasks available in your project, use 96
  • Basic Ingredients - Namespaces Group tasks together logically Namespaces and tasks are separated with “:” 97
  • Get Your Capistrano On 98
  • Get Your Capistrano On Capfile, the place to include more recipes 99
  • Get Your Capistrano On config/deploy.rb, application specific configuration 100
  • Capistrano’s Defaults •  Your SCM is Subversion •  Deployment directory is /u/apps/#{application_name} •  User for SCM and SSH is the currently logged-in user •  Commands are run with sudo 101
  • Get Your Capistrano On 102
  • Get Your Capistrano On •  Capistrano expects a directory structure •  Can be created with cap deploy:setup 103
  • The Deployment Lifecycle 104
  • The Deployment Lifecycle Check the prerequisites: 105
  • The Deployment Lifecycle Set up your application for the first time 106
  • The Deployment Lifecycle The initial deployment 1.  Checks the revision from the local machine 2.  Checks out the code on the remote machines 3.  Sets a link called current pointing to the lates release 4.  Runs the migrations 5.  Fires up application servers 107
  • The Deployment Lifecycle 108
  • The Deployment Lifecycle Subsequent deployments 1.  Checks the revision from the local machine 2.  Checks out the code on the remote machines 3.  Updates current link 4.  Restarts application servers 109
  • The Deployment Lifecycle 110
  • Common Capistrano Tasks Deploy and run migrations Run only the migrations Restart application servers Rollback to the previous release 111
  • Have Your Shell, and Eat It Too 112
  • Invoking any Command 113
  • Deployment Strategies 114
  • Deployment Strategies Direct checkout (from scratch) on the servers 115
  • Deployment Strategies Keep a cached copy of the current SCM head 116
  • Deployment Strategies Check out locally and transfer 117
  • Give it a little Spin Capistrano expects a script called spin in script/process For Passenger 118
  • Customizing Capistrano 119
  • Write your own Tasks 120
  • Write your own Tasks 121
  • Namespace your Tasks 122
  • Callbacks Execute a task before another runs Execute a task after another has finished Callbacks are run in the order they’re defined 123
  • Useful Variables 124
  • Transactions 125
  • Transactions 126
  • Transactions 127
  • Transactions 128
  • The Rest •  Gem dependencies •  Support for deploying through gateway servers •  Parallel execution on multiple servers •  Server setup with deprec gem 129
  • Vlad the Deployer 130
  • Vlad the Deployer •  Simple, Rake-based deployment solution •  Comes with defaults for most popular SCM, app and web servers 131
  • Vlad the Deployer Load Configure 132
  • Vlad the Deployer Deployment 133
  • Vlad the Deployer •  Simple, sometimes too simple •  No callback mechanism, instead the Rake syntax must be used •  Very non-verbose output, using --trace recommended 134
  • Chef •  System Integration Framework •  Written in Ruby – inspired by Puppet •  Manage multiple nodes •  Configuration •  Packages •  Custom scripts and commands 135
  • Architecture 136
  • Recipes Install and upgrade packages 137
  • Recipes Create and update remote files 138
  • Recipes Create and update remote files 139
  • chef-deploy Enable Use in recipe 140
  • Chef and chef-deploy •  In heavy development: sometimes unstable and hard to debug •  Be prepared to get your hands dirty •  Better suited for configuration than deployment •  Community recipes at http://github.com/opscode/cookbooks 141
  • One Click Deploy 142
  • Webistrano 143
  • Webistrano •  Web-UI to Capistrano •  Manages projects and their stages •  Alerting and Accounting •  Scriptable and extendable •  BSD License http://labs.peritor.com/webistrano 144
  • Macistrano •  Mac-GUI to Webistrano •  Fire and monitor deployments from your desktop http://github.com/mattmatt/macistrano 145
  • Infrastructure Practical Session 146
  • Practical Capistrano 147
  • EC2 Instances •  Users •  root / railswaycon •  deploy / railswaycon •  MySQL •  root / ‘’ •  Apache Configuration •  /etc/apache2/sites-enabled/default •  Rails •  /srv/www 148
  • Git Repository •  git://github.com/mattmatt/railscasts.git •  Setup the servers •  Deploy! 149
  • Tasks •  Make memcached restart with every deploy •  (Usually not a good idea, but why not?) 150
  • Tasks •  Make Apache recognize maintenance.html and display it if exists 151
  • Maintenance Page •  Create symbolic link to /etc/apache2/mods-available/rewrite.load in /etc/apache2/ mods-enabled •  Update virtual host configuration with rewrite rules •  Run cap deploy:web:disable •  Hook it into the deployment process 152
  • Tasks •  Deploy a Rack-based application •  Install Sinatra locally and on the server •  git://github.com/mattmatt/railswaycon.git •  Create deployment configuration •  Create Apache configuration 153
  • Infrastructure Monitoring 154
  • The two questions of monitoring 155
  • 1. Is everything still running? 156
  • 2. What are the trends? 157
  • Monit •  Process-level monitoring •  Checks PID-files, ports, and permissions •  Reacts by executing a script and/or alerting •  Has a web GUI for administering nodes 158
  • Monit MySQL 159
  • Monit Apache 160
  • Monit Mongrel 161
  • God •  Ruby-based configuration •  Easily extendible •  Can work with event- and poll-based conditions •  Comes with less recipes than Monit 162
  • God Mongrel 163
  • Munin •  Host-level monitoring •  Master periodically asks nodes for local data •  Checks system resources and records historical data •  Allows to recognize trends and make predictions •  Alerting support 164
  • Munin 165
  • Munin 166
  • Munin 167
  • Exception Monitoring •  Monitor errors raised by your application •  Usually notifying you by email •  Good old exception_notification Plugin •  Exceptions as a Service 168
  • Hoptoad 169
  • Exceptional 170
  • New Relic •  Rails Performance monitoring •  Live view into your application 171
  • Other Tools •  Nagios •  Big Brother •  New Relic RPM •  FiveRuns •  JMX 172
  • Infrastructure Advanced Deployment 173
  • Search 174
  • Search Full text search Can become very slow on big data sets 175
  • Full Text Search Engine Separate Service •  Creates full text index •  Application queries search daemon •  Index update through application or database Possible Engines •  Ferret •  Sphinx •  Solr •  Lucene •  … 176
  • Search Slave Database replication slave •  Has complete dataset •  Migrates slow search queries from master •  Can use different database table engine 177
  • Database Index PostgreSQL Tsearch2 •  Core since 8.3 •  Allows to create full text index on multiple columns or arbitrary SQL expressions MySQL MyISAM FULLTEXT index •  Only works with MySQL <= 5.0 and MyISAM tables •  Full text index on multiple columns 178
  • What to use? Different characteristics •  Real-time updates and stale data •  Lost updates •  Performance •  Document content and format •  Complexity 179
  • Background Processing 180
  • Problem Long running tasks •  Resizing uploaded images •  Mailing •  Computing an expensive operation •  Accessing slow back-ends When running inside request-response-cycle •  Blocks user •  Blocks Rails instance •  Hard to monitor and debug 181
  • Solution Asynchronous processing in the background Message/Queue Scheduler 182
  • Background Processing 183
  • Options Options for message bus: Options for background process: •  Database •  (Ruby) Daemon •  Amazon SQS •  Cron job with script/runner •  Drb •  Forked process •  Memcache •  Delayed Job / BJ / (Backgroundrb) •  ActiveMQ •  run_later •  … •  …. 184
  • Database/Ruby daemon example 185
  • Cloud Infrastructure 186
  • Cloud Infrastructure Servers come and go •  You do not know your servers before deploying •  Restarting is the same as introducing a new machine You can’t hardcode IPs database.yml 187
  • Solution #1 Query and manually adjust •  Servers do not change that often •  New nodes probably need manual intervention •  Use AWS ElasticIPs to ease the pain Set servers dynamically AWS Elastic IP 188
  • Solution #2 Use a central directory service •  A central place to manage your running instances •  Instances query the directory and react 189
  • Solution #2 Use a central directory service •  A central place to manage your running instances •  Instances query the directory and react 190
  • Central Directory Different Implementations •  File on S3 •  SimpleDB •  A complete service, capable of monitoring and controlling your instances 191
  • Q&A 192
  • Peritor GmbH Teutonenstraße 16 14129 Berlin Telefon: +49 (0)30 69 20 09 84 0 Telefax: +49 (0)30 69 20 09 84 9 Internet: www.peritor.com E-Mail: kontakt@peritor.com 193 Peritor GmbH - Alle Rechte vorbehalten 193