From Dev to DevOpsCarlos Sanchez@csanchezhttp://carlossanchez.eu
Apache  @csanchez   Maven                       Apache   ASF                 Archiva  Member                 ApacheEclipse...
Dev... What?
Agile           planning   iterative development  continuous integrationrelease soon, release often
DevQaOps ?
DevOps addresses       Fear of change     Risky deployments It works on my machine!         SiloisationDev Change vs. Ops ...
Individuals and interactions over processes and tools Working software over comprehensive documentation  Customer collabor...
Our highest priority is to satisfy the customer through early and            continuous delivery of valuable software. Wel...
Dev
What developers do today tospecify target environments is         NOT enough
Ops
requirementsOperating Systemconfig filespackages installedmulti stage configurationsdevQApre-productionproduction
DeploymentHow do I deploy this?documentationmanual stepsprone to errors
CloudHow do I deploy this?to hundreds of servers
Infrastructure as CodeFollow development best practices              tagging            branching             releasing   ...
DevOps
Should I worry about my OPS            job?
yes
yes my job is to make otherpeople’s jobs unnecessary
yes
yes
yes no
yes noyou should see the NOOPS           guys
DevOps is NOT about the tools
Nice, BUThow can I implement IT
Nice, BUThow can I implement IT
Tools can enablechange in behavior  and eventually  change culture     Patrick Debois
Man is the only animal thatchanges environment instead of           adapting
Man is the only animal thatchanges environment instead of           adapting          (using tools)
DevOps toolseveryone thinks isintelligent enoughevery tool thinks it’s cloudenabledevery tool thinks it’sDevOps
DevOps toolseveryone thinks isintelligent enoughevery tool thinks it’s cloudenabledevery tool thinks it’sDevOps
DevOps tools: infrastructure automation                 infrastructure as code      it’s all invented, now it’s standardized
DEVOPS“MY MACHINES ARE BEING PROVISIONED”
Puppet    infrastructure IS code       declarative modelstate vs process - no scripting           manifests             ru...
Puppet file structure      /etc/puppet    manifests/site.pp   manifests/nodes.pp
Puppet exampleuser { dave:  ensure     => present,  uid        => 507,  gid        => admin,  shell      => /bin/zsh,  hom...
Vagrant
Vagrant     Oracle VirtualBox cmdline automation       Easy Puppet and Chef provisioning  Keep VM configuration for differe...
Vagrant base boxes  www.vagrantbox.esanywhere! just (big) files
using Vagrant$ gem install vagrant$ vagrant box add centos-6.0-x86_64       http://dl.dropbox.com/u/1627760/centos-6.0-x86...
VagrantVagrant::Config.run do |config|  # Every Vagrant virtual environment requires a box to build off of.  config.vm.box...
manifests/base.pppackage { jdk:  ensure => installed,  name    => $operatingsystem ? {     centOS => "java-1.6.0-openjdk-d...
VeeWee
VeeWee   easily build Vagrant base boxeshttps://github.com/jedi4ever/veewee
using VeeWee$ gem install veewee$ vagrant basebox templates$ vagrant basebox define my-ubuntu-server    ubuntu-11.04-serve...
Geppetto
http://cloudsmith.github.com/geppetto
Puppet DSL
Puppet resourcesuser { dave:  ensure     =>   present,  uid        =>   507,  gid        =>   admin,  shell      =>   /bin...
Puppet resources $ puppet resource user rootuser { root:    home => /var/root,    shell => /bin/sh,    uid => 0,    ensure...
Puppet resources$ puppet resource user daveensure=present shell="/bin/zsh" home="/home/dave" managehome=truenotice: /User[...
Puppet resourcesfile {testfile:  path    => /tmp/testfile,  ensure => present,  mode    => 0640,  content => "Im a test fi...
Puppet standalone$ puppet apply my_test_manifest.pp
orderingfile {/tmp/test1:  ensure => present,  content => "Hi.",}notify {/tmp/test1 has alreadybeen synced.:  require => F...
orderingfile {/tmp/test1:  ensure => present,  content => "Hi.",} ->notify {/tmp/test1 has alreadybeen synced.:}
package / file /service / subscribepackage { openssh-server:  ensure => present,  before => File[/etc/ssh/sshd_config],}fil...
variables$longthing = "Imagine I have something reallylong in here. Like an SSH key, lets say."file {authorized_keys:  pat...
factshost {self:  ensure         =>   present,  name           =>   $::fqdn,  host_aliases   =>   [puppet, $::hostname],  ...
conditionalsif $is_virtual {  service {ntpd:    ensure => stopped,    enable => false,  }}else {  service { ntpd:    name ...
conditionalscase $operatingsystem {  centos, redhat: { $apache = "httpd" }  debian, ubuntu: { $apache = "apache2" }  defau...
class definitionclass ntp {    package { ntp:      ensure => installed,    }    service { ntp:      name      => ntpd,     ...
class declaration (1)class {ntp: }
class declaration (2)include ntp
parameterized classesclass paramclassexample ($value1, $value2 = "Defaultvalue") {  notify {"Value 1 is ${value1}.":}  not...
modules{module}/    files/    lib/    manifests/         init.pp         {class}.pp         {defined type}.pp         {nam...
templatingfile {/etc/foo.conf:  ensure => file,  require => Package[foo],  content => template(foo/foo.conf.erb),}
templates/foo.conf.erbOS is <%= $::operatingsystem %>
node configuration# nodes.ppnode someserver.domain.com inherits basenode {    $web_fqdn = www.domain.com    include generic...
Maven and Puppet
What am I doing to automate deployment             Ant tasks plugin             ssh commands             Assembly plugin  ...
What can I do to automate deployment Handle full deployment including infrastructure             not just webapp deploymen...
Maven-Puppet module         A Maven Puppet modulehttps://github.com/maestrodev/puppet-maven   fetches Maven artifacts from...
Requirements   Java JDK    Maven
Installing Maven$repo1 = {  id => "myrepo",  username => "myuser",  password => "mypassword",  url => "http://repo.acme.co...
New Maven typemaven { "/tmp/maven-core-2.2.1.jar":  id => "org.apache.maven:maven-core:jar:2.2.1",  repos => ["http://repo...
New Maven typemaven { "/tmp/maven-core-2.2.1.jar":  groupId => "org.apache.maven",  artifactId => "maven-core",  version =...
Example code                    Available at          http://github.carlossanchez.eu           http://blog.carlossanchez.e...
Thanks!http://maestrodev.comhttp://carlossanchez.eucsanchez@maestrodev.comcarlos@apache.org@csanchez
Photo Credits               Son of Man Lego - Alex Eylarhttp://www.flickr.com/photos/hoyvinmayvin/4702772452/              ...
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
From Dev to DevOps - ApacheCON NA 2011
Upcoming SlideShare
Loading in...5
×

From Dev to DevOps - ApacheCON NA 2011

9,588

Published on

More info at http://blog.carlossanchez.eu/2011/11/15/from-dev-to-devops-slides-from-apachecon-na-vancouver-2011/

The DevOps movement aims to improve communication between developers and operations teams to solve critical issues such as fear of change and risky deployments. But the same way that Agile development would likely fail without continuous integration tools, the DevOps principles need tools to make them real, and provide the automation required to actually be implemented. Most of the so called DevOps tools focus on the operations side, and there should be more than that, the automation must cover the full process, Dev to QA to Ops and be as automated and agile as possible. Tools in each part of the workflow have evolved in their own silos, and with the support of their own target teams. But a true DevOps mentality requires a seamless process from the start of development to the end in production deployments and maintenance, and for a process to be successful there must be tools that take the burden out of humans.
Apache Maven has arguably been the most successful tool for development, project standardization and automation introduced in the last years. On the operations side we have open source tools like Puppet or Chef that are becoming increasingly popular to automate infrastructure maintenance and server provisioning.
In this presentation we will introduce an end-to-end development-to-production process that will take advantage of Maven and Puppet, each of them at their strong points, and open source tools to automate the handover between them, automating continuous build and deployment, continuous delivery, from source code to any number of application servers managed with Puppet, running either in physical hardware or the cloud, handling new continuous integration builds and releases automatically through several stages and environments such as development, QA, and production.

Published in: Technology
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
9,588
On Slideshare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
0
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide
  • \n
  • question? #developers vs #ops vs #devops\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • question? what do you do to specify ops requirements?\n
  • \n
  • question? what do you do to specify ops requirements?\n
  • \n
  • \n
  • \n
  • \n
  • question? who&amp;#x2019;s able to roll back to a previous version of software? and system?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • From Dev to DevOps - ApacheCON NA 2011

    1. 1. From Dev to DevOpsCarlos Sanchez@csanchezhttp://carlossanchez.eu
    2. 2. Apache @csanchez Maven Apache ASF Archiva Member ApacheEclipse IAM Continuum
    3. 3. Dev... What?
    4. 4. Agile planning iterative development continuous integrationrelease soon, release often
    5. 5. DevQaOps ?
    6. 6. DevOps addresses Fear of change Risky deployments It works on my machine! SiloisationDev Change vs. Ops stability
    7. 7. Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
    8. 8. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late in development. Agile processes harness change for the customers competitive advantage. Deliver working software frequently, from a couple of weeksto a couple of months, with a preference to the shorter timescale. Business people and developers must work together daily throughout the project. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation. Agile processes promote sustainable development. Thesponsors, developers, and users should be able to maintain a constant pace indefinitely.
    9. 9. Dev
    10. 10. What developers do today tospecify target environments is NOT enough
    11. 11. Ops
    12. 12. requirementsOperating Systemconfig filespackages installedmulti stage configurationsdevQApre-productionproduction
    13. 13. DeploymentHow do I deploy this?documentationmanual stepsprone to errors
    14. 14. CloudHow do I deploy this?to hundreds of servers
    15. 15. Infrastructure as CodeFollow development best practices tagging branching releasing dev, QA, production
    16. 16. DevOps
    17. 17. Should I worry about my OPS job?
    18. 18. yes
    19. 19. yes my job is to make otherpeople’s jobs unnecessary
    20. 20. yes
    21. 21. yes
    22. 22. yes no
    23. 23. yes noyou should see the NOOPS guys
    24. 24. DevOps is NOT about the tools
    25. 25. Nice, BUThow can I implement IT
    26. 26. Nice, BUThow can I implement IT
    27. 27. Tools can enablechange in behavior and eventually change culture Patrick Debois
    28. 28. Man is the only animal thatchanges environment instead of adapting
    29. 29. Man is the only animal thatchanges environment instead of adapting (using tools)
    30. 30. DevOps toolseveryone thinks isintelligent enoughevery tool thinks it’s cloudenabledevery tool thinks it’sDevOps
    31. 31. DevOps toolseveryone thinks isintelligent enoughevery tool thinks it’s cloudenabledevery tool thinks it’sDevOps
    32. 32. DevOps tools: infrastructure automation infrastructure as code it’s all invented, now it’s standardized
    33. 33. DEVOPS“MY MACHINES ARE BEING PROVISIONED”
    34. 34. Puppet infrastructure IS code declarative modelstate vs process - no scripting manifests ruby ERB templates master - agent architecture
    35. 35. Puppet file structure /etc/puppet manifests/site.pp manifests/nodes.pp
    36. 36. Puppet exampleuser { dave: ensure => present, uid => 507, gid => admin, shell => /bin/zsh, home => /home/dave, managehome => true,}file {/tmp/test1: ensure => present, content => "Hi.",}
    37. 37. Vagrant
    38. 38. Vagrant Oracle VirtualBox cmdline automation Easy Puppet and Chef provisioning Keep VM configuration for different projectsShare boxes and configuration files across teams base box + configuration files
    39. 39. Vagrant base boxes www.vagrantbox.esanywhere! just (big) files
    40. 40. using Vagrant$ gem install vagrant$ vagrant box add centos-6.0-x86_64 http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box$ vagrant init myproject$ vagrant up$ vagrant ssh$ vagrant suspend$ vagrant resume$ vagrant destroy
    41. 41. VagrantVagrant::Config.run do |config| # Every Vagrant virtual environment requires a box to build off of. config.vm.box = "centos-6.0-x86_64" # The url from where the config.vm.box box will be fetched config.vm.box_url = "http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box" # Boot with a GUI so you can see the screen. (Default is headless) #config.vm.boot_mode = :gui # Assign this VM to a host only network IP, allowing you to access it via the IP. # config.vm.network "33.33.33.10" # Forward a port from the guest to the host, which allows for outside # computers to access the VM, whereas host only networking does not. config.vm.forward_port "sonar", 9000, 19000 # Enable provisioning with Puppet stand alone. config.vm.share_folder("templates", "/tmp/vagrant-puppet/templates", "templates") config.vm.provision :puppet do |puppet| puppet.manifest_file = "base.pp" puppet.module_path = "mymodules" puppet.options = ["--templatedir","/tmp/vagrant-puppet/templates"] puppet.options = "-v -d" endend
    42. 42. manifests/base.pppackage { jdk: ensure => installed, name => $operatingsystem ? { centOS => "java-1.6.0-openjdk-devel", Ubuntu => "openjdk-6-jdk", default => "jdk", },}
    43. 43. VeeWee
    44. 44. VeeWee easily build Vagrant base boxeshttps://github.com/jedi4ever/veewee
    45. 45. using VeeWee$ gem install veewee$ vagrant basebox templates$ vagrant basebox define my-ubuntu-server ubuntu-11.04-server-amd64# customize definitions/my-ubuntu-server$ vagrant basebox build my-ubuntu-server$ vagrant basebox validate my-ubuntu-server$ vagrant basebox export my-ubuntu-server$ vagrant box add my-ubuntu-server my-ubuntu-server.box$ vagrant init my-ubuntu-server
    46. 46. Geppetto
    47. 47. http://cloudsmith.github.com/geppetto
    48. 48. Puppet DSL
    49. 49. Puppet resourcesuser { dave: ensure => present, uid => 507, gid => admin, shell => /bin/zsh, home => /home/dave, managehome => true,}
    50. 50. Puppet resources $ puppet resource user rootuser { root: home => /var/root, shell => /bin/sh, uid => 0, ensure => present, password => *, gid => 0, comment => System Administrator}
    51. 51. Puppet resources$ puppet resource user daveensure=present shell="/bin/zsh" home="/home/dave" managehome=truenotice: /User[dave]/ensure: createduser { dave: ensure => present, home => /home/dave, shell => /bin/zsh}
    52. 52. Puppet resourcesfile {testfile: path => /tmp/testfile, ensure => present, mode => 0640, content => "Im a test file.",}notify {"Im notifying you.":}
    53. 53. Puppet standalone$ puppet apply my_test_manifest.pp
    54. 54. orderingfile {/tmp/test1: ensure => present, content => "Hi.",}notify {/tmp/test1 has alreadybeen synced.: require => File[/tmp/test1],}
    55. 55. orderingfile {/tmp/test1: ensure => present, content => "Hi.",} ->notify {/tmp/test1 has alreadybeen synced.:}
    56. 56. package / file /service / subscribepackage { openssh-server: ensure => present, before => File[/etc/ssh/sshd_config],}file { /etc/ssh/sshd_config: ensure => file, mode => 600, source => /root/learning-manifests/sshd_config,}service { sshd: ensure => running, enable => true, hasrestart => true, hasstatus => true, subscribe => File[/etc/ssh/sshd_config],}
    57. 57. variables$longthing = "Imagine I have something reallylong in here. Like an SSH key, lets say."file {authorized_keys: path => /root/.ssh/authorized_keys, content => $longthing,}
    58. 58. factshost {self: ensure => present, name => $::fqdn, host_aliases => [puppet, $::hostname], ip => $::ipaddress,}file {motd: ensure => file, path => /etc/motd, mode => 0644, content => "Welcome to ${::hostname},na ${::operatingsystem} island in the sea of ${::domain}.n",}
    59. 59. conditionalsif $is_virtual { service {ntpd: ensure => stopped, enable => false, }}else { service { ntpd: name => ntpd, ensure => running, enable => true, hasrestart => true, require => Package[ntp], }}
    60. 60. conditionalscase $operatingsystem { centos, redhat: { $apache = "httpd" } debian, ubuntu: { $apache = "apache2" } default: { fail("Unrecognized operating system forwebserver") }}$apache = $operatingsystem ? { centos => httpd, redhat => httpd, /(?i)(ubuntu|debian)/ => "apache2-$1", # (Dont actually use that package name.) default => undef, }
    61. 61. class definitionclass ntp { package { ntp: ensure => installed, } service { ntp: name => ntpd, ensure => running, enable => true, subscribe => File[ntp.conf], }}
    62. 62. class declaration (1)class {ntp: }
    63. 63. class declaration (2)include ntp
    64. 64. parameterized classesclass paramclassexample ($value1, $value2 = "Defaultvalue") { notify {"Value 1 is ${value1}.":} notify {"Value 2 is ${value2}.":}}class {paramclassexample: value1 => Something, value2 => Something else,}class {paramclassexample: value1 => Something,}
    65. 65. modules{module}/ files/ lib/ manifests/ init.pp {class}.pp {defined type}.pp {namespace}/ {class}.pp {class}.pp templates/ tests/
    66. 66. templatingfile {/etc/foo.conf: ensure => file, require => Package[foo], content => template(foo/foo.conf.erb),}
    67. 67. templates/foo.conf.erbOS is <%= $::operatingsystem %>
    68. 68. node configuration# nodes.ppnode someserver.domain.com inherits basenode { $web_fqdn = www.domain.com include genericwebserver include some_other_service}node ldapmaster.domain.com inherits basenode { include s_ldap::master}node humanresources.domain.com inherits basenode { include c_humanresources}
    69. 69. Maven and Puppet
    70. 70. What am I doing to automate deployment Ant tasks plugin ssh commands Assembly plugin Cargo
    71. 71. What can I do to automate deployment Handle full deployment including infrastructure not just webapp deployment Help Ops with clear, automated manifests Ability to reproduce production environments in local box using Vagrant / VirtualBox / VMWare Use the right tool for the right job
    72. 72. Maven-Puppet module A Maven Puppet modulehttps://github.com/maestrodev/puppet-maven fetches Maven artifacts from the repo manages them with Puppet no more extra packaging
    73. 73. Requirements Java JDK Maven
    74. 74. Installing Maven$repo1 = { id => "myrepo", username => "myuser", password => "mypassword", url => "http://repo.acme.com",}# Install Mavenclass { "maven::maven": version => "2.2.1",} -># Create a settings.xml with the repo credentialsclass { "maven::settings" : servers => [$repo1],}
    75. 75. New Maven typemaven { "/tmp/maven-core-2.2.1.jar": id => "org.apache.maven:maven-core:jar:2.2.1", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"],}
    76. 76. New Maven typemaven { "/tmp/maven-core-2.2.1.jar": groupId => "org.apache.maven", artifactId => "maven-core", version => "2.2.1", packaging => "jar", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"],}
    77. 77. Example code Available at http://github.carlossanchez.eu http://blog.carlossanchez.euhttps://github.com/maestrodev/puppet-modules
    78. 78. Thanks!http://maestrodev.comhttp://carlossanchez.eucsanchez@maestrodev.comcarlos@apache.org@csanchez
    79. 79. Photo Credits Son of Man Lego - Alex Eylarhttp://www.flickr.com/photos/hoyvinmayvin/4702772452/ Brick wall - Luis Argerich http://www.flickr.com/photos/lrargerich/4353397797/ Agile vs. Iterative flow - Christopher Littlehttp://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg DevOps - Rajiv.Pant http://en.wikipedia.org/wiki/File:Devops.png Pimientos de Padron - Howard Walfish http://www.flickr.com/photos/h-bomb/4868400647/ Compiling - XKCD http://xkcd.com/303/ Printer in 1568 - Meggs, Philip B http://en.wikipedia.org/wiki/File:Printer_in_1568-ce.png Relativity - M. C. Escherhttp://en.wikipedia.org/wiki/File:Escher%27s_Relativity.jpg

    ×