packaging ruby and rails apps
  to production

          Fabio
Kung
          Rodrigo
Vaz
@fabiokung
@rodrigovaz
no offense, but...
cap deploy
w"?
     
OMG...

  gem vs system pkgs
       again?

                       disclaimer
it probably doesn’t
    apply to PaaS


                      disclaimer
sysadmin love


     h+p://www.flickr.com/photos/theamarand/2879573829/
http://www.flickr.com/photos/sequelguy/4390001903/
sysadmins
and
sysops

know
OS/distro
tools



                 http://www.flickr.com/photos/sequelguy/4390001903/
dpkg -l
$ dpkg -l
ii adduser               3.112ubuntu1
ii apt                0.7.25.3ubuntu9.3
ii apt-utils            0.7.25.3ubuntu9.3
ii aptitude             0.4.11.11-1ubuntu10
ii autoconf              2.65-3ubuntu1
ii automake               1:1.11.1-1
ii autotools-dev            20090611.1
ii base-files            5.0.0ubuntu20.10.04.2
ii base-passwd              3.5.22
ii bash                4.1-2ubuntu3
ii bash-completion           1:1.1-3ubuntu2
ii binutils            2.20.1-3ubuntu7
ii build-essential         11.4build1
ii bzip2               1.0.5-4ubuntu0.1
ii cdbs                0.4.62+nmu1ubuntu9
ii cmake                2.8.1-4~lucid1
ii cmake-data              2.8.1-4~lucid1
ii coreutils           7.4-2ubuntu2
ii cpp                4:4.4.3-1ubuntu1
ii cpp-4.4               4.4.3-4ubuntu5
ii cpu-checker             0.1-0ubuntu2
ii cron               3.0pl1-106ubuntu5
ii dash                0.5.5.1-3ubuntu2
ii debconf               1.5.28ubuntu4
ii debconf-i18n             1.5.28ubuntu4
ii debhelper              7.4.15ubuntu1
ii debianutils           3.2.2
ii debootstrap             1.0.28ubuntu1~lucid1
ii devscripts            2.10.61ubuntu5
...
dpkg -L myapp
dpkg --contents myapp.deb
       $ dpkg -L vim
       /.
       /usr
       /usr/bin
       /usr/bin/vim.basic
       /usr/share
       /usr/share/lintian
       /usr/share/lintian/overrides
       /usr/share/lintian/overrides/vim
       /usr/share/bug
       /usr/share/bug/vim
       /usr/share/bug/vim/presubj
       /usr/share/doc
       /usr/share/doc/vim
dpkg -S /etc/myapp.conf

    $ dpkg -S /usr/bin/vim.basic
    vim: /usr/bin/vim.basic
apticron, cron-apt
apticron report [Mon, 18 Apr 2011 18:25:02 -0300]
=====================================================

apticron has detected that some packages need upgrading on:


   foo.locaweb.com.br

   [ 10.0.0.1 ]

The following packages are currently pending an upgrade:


   myapp 5.1.0-3

   myapp-assets 5.1.0-3

======================================================

Package Details:

Reading changelogs...
--- Changes for myapp (myapp myapp-assets) ---
myapp (5.1.0-3) stable; urgency=low

 * new awesome feature
 * lots of bug fixes

-- Fabio Kung <fabio.kung@locaweb.com.br>  Tue, 12 Apr 2011 22:31:11 -0300
http://www.flickr.com/photos/atibens/4578265134/




post install
health, sanity check
db/seeds.rb
rake db:migrate
rake assets:export
rake i18n:js:export
           ...
package repositories
       distribute it
signed packages
dev toolchain
 in production servers?
build pipeline
tes;ng
x
stable
multiple archs
dpkg -i myapp-oldversion
btw...



 rake db:migrate VERSION=old
dependencies
(re)deploying services
       must be ultra easy
package everything
    OH
WAIT...
configuration managers
recipes
                               cf-agent


                  promisses


cfengine, chef,        rules   cf-agent
     puppet




                               cf-agent
package "sudo" do
  action :upgrade
end

user "sshservice" do
  system true
  action :create
end

template "/etc/sudoers" do
  source "sudoers.erb"
  mode 0440
  owner "root"
  group "root"
  variables(:user => "sshservice")
  action :create
end
bundle common control
{
    bundlesequence => { "ssh" };
    inputs => { "cfengine_stdlib.cf" }
}

bundle agent ssh
{
    packages:
        "sudo"
            package_policy => "add"
            package_method => "deb"
    files:
        "/etc/sudoers"
            edit_defaults => empty,
            edit_line => expand_template("sudoers.in"),
            perms => mog("0440", root, root),
            create => "true";
    methods:
        "sshservice" usebundle => create_user("sshservice")
}
rule of thumb
keys
ssh, access, crypt, passwords, ...
init scripts
/etc/init.d/myapp start|stop|restart
configuration files
   <any>.yml.example
database.yml.example
                       CFEngine3
       production:
        adapter: mysql2
        encoding: utf8
        database: ${db_name}
        username: ${db_user}
        password: ${db_password}
        host: ${db_address}

edit_line => expand_template("database.yml.example")
database.yml.example
           Chef,
Puppet
 production:
  adapter: mysql2
  encoding: utf8
  database: <%= db_name %>
  username: <%= db_user %>
  password: <%= db_password %>
  host: <%= db_address %>
packages are addictive
creating packages
      http://www.flickr.com/photos/miadcommunicationdesign/3169997945/
debian/control
Source: myapp
Section: web
Priority: extra
Maintainer: Fabio Kung <fabio.kung@locaweb.com.br>
Build-Depends: debhelper (>= 7)
Standards-Version: 3.8.3
Homepage: https://github.com/locaweb/myapp

Package: myapp
Architecture: any
Depends: rubygems (>= 1.3.6), libapache2-mod-passenger (>= 2.2)
Description: my great new app
 Visit https://mycompany.com/myapp for more information
debian/changelog
myapp (0.0.2) unstable; urgency=low

 * Applying SOLID after attending railsconf 2011
 * using Devise

-- Fabio Kung <fabiokung@locaweb.com.br>   Wed, 18 May 2011 08:00:30 -0300



myapp (0.0.1) unstable; urgency=low

 * Initial release

-- Rodrigo Vaz <rodvaz@locaweb.com.br>   Wed, 17 May 2011 18:05:20 -0300
debian/rules
                  


              
      



      
dh_make --createorig
• dpkg-buildpackage
• pbuilder
• cowbuilder
why is it unusual?
debian-ruby-
        maintainers
Giving up on Ruby packaging - Lucas Nussbaum




http://www.lucas-nussbaum.net/blog/?p=617
FPM
                    awesome
        https://github.com/jordansissel/fpm



I
want
a
simple
way
to
create
packages

without
all
the
bullshit.
In
my
own

infrastructure,
I
have
no
interest
in
Debian

policy
and
RedHat
packaging
guidelines
‐
I

have
interest
in
my
group's
own
style
culture

and
have
a
very
strong
interest
in
ge6ng
work

done.
                               ‐‐
Jordan
Sissel
http://www.flickr.com/photos/brightbox/2298191057/
http://morethanseven.net/2011/01/16/Why-developers-should-
              care-about-system-packages.html




 Why Developers Should Care
   About System Packages
                                   -- Gareth Rushgrove
what about other
   platforms?
blaming game
what are we doing to end it?
github.com/locaweb/debundler
a long way to go...
RPM
bundle debianize (?)
  bundler deeper integration, plugin
                        
                             
                        
    just some ideas... rvm is great :-)
vendor everything vs
 package everything
      gem dependencies
packaging gems
• fpm
• gem2deb
• gem2rpm
• checkinstall
• debian/
  changelog from
  git log
build servers
bricklayer
github.com/locaweb/bricklayer
REST API
POST /project:
  name="myapp"
  version="1.0.1"
  git_url="git://host/project/repository.git"
  branch="master"
  build_cmd="rake prepare"
  install_cmd="rake install PREFIX=debian/tmp"
  respository_url="ftp.location.com"
  repository_user="a_user"
  repository_passwd="a_g00dP455w0rd"
launchpad
 dev.launchpad.net
pain points

• old rubygems official system packages
• bundler (!)
• package yourself
• PPA?
Thanks!
              fabio.kung at gmail, @fabiokung
             rodrigo.vaz at gmail, @rodrigovaz




                            additional info:
                           https://wiki.ubuntu.com/PackagingGuide
http://morethanseven.net/2011/01/16/Why-developers-should-care-about-system-packages.html
                              https://github.com/jordansissel/fpm
                         http://www.lucas-nussbaum.net/blog/?p=617

Ruby and Rails Packaging to Production

Editor's Notes

  • #2 \n
  • #3 infrastructure coding\n
  • #4 \n
  • #5 \n
  • #6 \n
  • #7 \n
  • #8 NO! and yes. :-)\n**my** packages on **my** servers\nprivate repositories\n
  • #9 \n
  • #10 \n
  • #11 multiple systems\nmultiple languages\nmultiple architectures\nsystem packages provide a lot of useful features\n
  • #12 what is installed?\nwhat are the versions?\nwhat is the _current_ version of my app?\n
  • #13 what are the config files?\nwhat is included?\n
  • #14 what is the package of this file?\n
  • #15 \n
  • #16 \n
  • #17 learning from operating systems, self-test boot\ndiff .example .conf\nexternal systems, api versions, required assets, filesystem, required users, fixtures, ...\nit&amp;#x2019;s all about dependencies! (other than package dependencies)\n
  • #18 \n
  • #19 bytecode precompilation\n
  • #20 package =&gt; many projects\nsame package =&gt; many machines\nmultiple versions\n
  • #21 \n
  • #22 avoid compilation/build on production servers \ngcc, make, g++, libc-dev, etc\nrvm, git, ... the more crap you have in prod servers =&gt; the more to exploit\n\n
  • #23 **same package** being deployed at different staging environments\n
  • #24 \n
  • #25 \n
  • #26 \n
  • #27 \n
  • #28 pre/postrm\n
  • #29 \n
  • #30 rebuildable servers\nservers/machines as code\nrecreating machines =&gt; easy\n
  • #31 what goes inside the package? what doesn&amp;#x2019;t?\n
  • #32 \n
  • #33 Configuration Managers\n
  • #34 chef receipt\n
  • #35 cf promisse\n
  • #36 it belongs to your app =&gt; put it inside the package\nunless you expect it to be changed\nare people going to edit it after installing the package?\n
  • #37 \n
  • #38 deb, rpm offer easy ways to create them\n
  • #39 as we are all doing\n
  • #40 \n
  • #41 \n
  • #42 assets server\n
  • #43 dh_make\ndebian/{control,rules,changelog}\ndpkg-buildpackage -&gt; rules, sign\ndebian maintainers policy, ubuntu packaging guide\n
  • #44 \n
  • #45 \n
  • #46 \n
  • #47 \n
  • #48 \n
  • #49 why things seem to be slower in the ruby vs system packages camp?\nwhy just a few people seems to be caring?\n
  • #50 \n
  • #51 \n
  • #52 \n
  • #53 rubygems, ruby and passenger debs\n
  • #54 \n
  • #55 tools\nnot usual to &amp;#x201C;build&amp;#x201D; stuff on production servers\nplatforms offer easy ways to make binary/system packages\nruby is probably missing that. Nobody is caring\n
  • #56 \n
  • #57 \n
  • #58 \n
  • #59 \n
  • #60 other ideas:\n - rvm gemset package\n - rvm rubies package\n
  • #61 \n
  • #62 \n
  • #63 \n
  • #64 cloud IaaS and VMs help a lot\nbricklayer\nlaunchpad\nCI servers (hudson, bamboo, teamcity, ...)\n\n
  • #65 - Watch git repositories\n- Build snapshots for each commit\n- Watch for tags testing_1.2.3 and stable_1.2.3\n- Promote packages to a ftp server (which runs reprepro or any other debian repository tool)\n- Has a rest API\n- Has cocoa-like web interface wrote with cappuccino framework\n- Batteries are included except redis server\n\n\n
  • #66 \n
  • #67 - Lauchpad generate binaries out of your source as long as your application is already ready to be packaged\n- Build packages for multiple archs\n- Host the package in a repository called PPA\n- Developed and open source by canonical\n
  • #68 \n
  • #69 \n