From Dev to DevOps - Apache Barcamp Spain 2011

7,023 views

Published on

UPDATE: updated slides at http://www.slideshare.net/carlossg/from-dev-to-devops-conferencia-agile-spain-2011

The DevOps movement aims to improve communication between developers and operations teams to solve critical issues such as fear of change and risky deployments. But the same way that Agile development would likely fail without continuous integration tools, the DevOps principles need tools to make them real, and provide the automation required to actually be implemented. Most of the so called DevOps tools focus on the operations side, and there should be more than that, the automation must cover the full process, Dev to QA to Ops and be as automated and agile as possible. Tools in each part of the workflow have evolved in their own silos, and with the support of their own target teams. But a true DevOps mentality requires a seamless process from the start of development to the end in production deployments and maintenance, and for a process to be successful there must be tools that take the burden out of humans.
Apache Maven has arguably been the most successful tool for development, project standardization and automation introduced in the last years. On the operations side we have open source tools like Puppet or Chef that are becoming increasingly popular to automate infrastructure maintenance and server provisioning.
In this presentation we will introduce an end-to-end development-to-production process that will take advantage of Maven and Puppet, each of them at their strong points, and open source tools to automate the handover between them, automating continuous build and deployment, continuous delivery, from source code to any number of application servers managed with Puppet, running either in physical hardware or the cloud, handling new continuous integration builds and releases automatically through several stages and environments such as development, QA, and production.

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,023
On SlideShare
0
From Embeds
0
Number of Embeds
4,541
Actions
Shares
0
Downloads
0
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • From Dev to DevOps - Apache Barcamp Spain 2011

    1. 1. From Dev to DevOpsCarlos SanchezMaestroDev
    2. 2. Hola!Solutions Architect atMaestroDevMember of ApacheSoftware FoundationMaven PMC, Continuum,ArchivaEclipse IAM co-leadetc...
    3. 3. Dev... What?
    4. 4. Agile
    5. 5. Agile planning iterative development continuous integrationrelease soon, release often
    6. 6. DevQaOps ?
    7. 7. DevOps addresses
    8. 8. DevOps addresses Fear of change Risky deployments It works on my machine! SiloisationDev Change vs. Ops stability
    9. 9. Agile Manifesto
    10. 10. Agile ManifestoIndividuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
    11. 11. Agile Manifesto
    12. 12. Agile ManifestoOur highest priority is to satisfy the customer through early and continuous delivery of valuable software.
    13. 13. Agile Manifesto
    14. 14. Agile Manifesto Welcome changing requirements, even late indevelopment. Agile processes harness change for the customers competitive advantage.
    15. 15. Agile Manifesto
    16. 16. Agile ManifestoDeliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
    17. 17. Agile Manifesto
    18. 18. Agile ManifestoBusiness people and developers must work together daily throughout the project.
    19. 19. Agile Manifesto
    20. 20. Agile ManifestoBuild projects around motivated individuals. Give them theenvironment and support they need, and trust them to get the job done.
    21. 21. Agile Manifesto
    22. 22. Agile Manifesto The most efficient and effective method of conveyinginformation to and within a development team is face- to-face conversation.
    23. 23. Agile Manifesto
    24. 24. Agile ManifestoWorking software is the primary measure of progress.
    25. 25. Agile Manifesto
    26. 26. Agile ManifestoAgile processes promote sustainable development. Thesponsors, developers, and users should be able to maintain a constant pace indefinitely.
    27. 27. Agile Manifesto
    28. 28. Agile ManifestoContinuous attention to technical excellence and good design enhances agility.
    29. 29. Agile Manifesto
    30. 30. Agile ManifestoSimplicity--the art of maximizing the amount of work not done--is essential.
    31. 31. Agile Manifesto
    32. 32. Agile ManifestoThe best architectures, requirements, and designs emerge from self-organizing teams.
    33. 33. Agile Manifesto
    34. 34. Agile ManifestoAt regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
    35. 35. DevOps toolseveryone thinks isintelligent enoughevery tool thinks it’s cloudenabledevery tool thinks it’sDevOps
    36. 36. DevOps toolseveryone thinks isintelligent enoughevery tool thinks it’s cloudenabledevery tool thinks it’sDevOps
    37. 37. DevOps tools: infrastructure automation
    38. 38. DevOps tools: infrastructure automation infrastructure as code it’s all invented, now it’s standardized
    39. 39. Dev
    40. 40. Ops
    41. 41. requirements
    42. 42. requirements Operating System config files packages installedmulti stage configurations dev QA pre-production production
    43. 43. DeploymentHow do I deploy this?documentationmanual stepsprone to errors
    44. 44. CloudHow do I deploy this?to hundreds of servers
    45. 45. Infrastructure as Code
    46. 46. Infrastructure as CodeFollow development best practices tagging branching releasing dev, QA, production
    47. 47. Nice, buthow can I do IT
    48. 48. Nice, buthow can I do IT
    49. 49. DevOps
    50. 50. Puppet
    51. 51. Puppet infrastructure IS code declarative modelstate vs process - no scripting manifests ruby ERB templates master - agent architecture
    52. 52. Puppet file structure
    53. 53. Puppet file structure /etc/puppet manifests/site.pp manifests/nodes.pp
    54. 54. Puppet resources
    55. 55. Puppet resourcesuser { dave: ensure => present, uid => 507, gid => admin, shell => /bin/zsh, home => /home/dave, managehome => true,}
    56. 56. Puppet resources
    57. 57. Puppet resources $ puppet resource user rootuser { root: home => /var/root, shell => /bin/sh, uid => 0, ensure => present, password => *, gid => 0, comment => System Administrator}
    58. 58. Puppet resources
    59. 59. Puppet resources$ puppet resource user daveensure=present shell="/bin/zsh" home="/home/dave" managehome=truenotice: /User[dave]/ensure: createduser { dave: ensure => present, home => /home/dave, shell => /bin/zsh}
    60. 60. Puppet resources
    61. 61. Puppet resourcesfile {testfile: path => /tmp/testfile, ensure => present, mode => 0640, content => "Im a test file.",}notify {"Im notifying you.":}
    62. 62. Puppet standalone
    63. 63. Puppet standalone$ puppet apply my_test_manifest.pp
    64. 64. ordering
    65. 65. orderingfile {/tmp/test1: ensure => present, content => "Hi.",}notify {/tmp/test1 has alreadybeen synced.: require => File[/tmp/test1],}
    66. 66. ordering
    67. 67. orderingfile {/tmp/test1: ensure => present, content => "Hi.",} ->notify {/tmp/test1 has alreadybeen synced.:}
    68. 68. package / file /service / subscribe
    69. 69. package / file /service / subscribepackage { openssh-server: ensure => present, before => File[/etc/ssh/sshd_config],}file { /etc/ssh/sshd_config: ensure => file, mode => 600, source => /root/learning-manifests/sshd_config,}service { sshd: ensure => running, enable => true, hasrestart => true, hasstatus => true, subscribe => File[/etc/ssh/sshd_config],}
    70. 70. variables
    71. 71. variables$longthing = "Imagine I have something reallylong in here. Like an SSH key, lets say."file {authorized_keys: path => /root/.ssh/authorized_keys, content => $longthing,}
    72. 72. facts
    73. 73. factshost {self: ensure => present, name => $::fqdn, host_aliases => [puppet, $::hostname], ip => $::ipaddress,}file {motd: ensure => file, path => /etc/motd, mode => 0644, content => "Welcome to ${::hostname},na ${::operatingsystem} island in the sea of ${::domain}.n",}
    74. 74. conditionals
    75. 75. conditionalsif $is_virtual { service {ntpd: ensure => stopped, enable => false, }}else { service { ntpd: name => ntpd, ensure => running, enable => true, hasrestart => true, require => Package[ntp], }}
    76. 76. conditionals
    77. 77. conditionalscase $operatingsystem { centos, redhat: { $apache = "httpd" } debian, ubuntu: { $apache = "apache2" } default: { fail("Unrecognized operating systemfor webserver") }}$apache = $operatingsystem ? { centos => httpd, redhat => httpd, /(?i)(ubuntu|debian)/ => "apache2-$1", # (Dont actually use that package name.) default => undef, }
    78. 78. class definition
    79. 79. class definitionclass ntp { package { ntp: ensure => installed, } service { ntp: name => ntpd, ensure => running, enable => true, subscribe => File[ntp.conf], }}
    80. 80. class declaration (1)
    81. 81. class declaration (1)class {ntp: }
    82. 82. class declaration (2)
    83. 83. class declaration (2)include ntp
    84. 84. parameterized classes
    85. 85. parameterized classesclass paramclassexample ($value1, $value2 ="Default value") { notify {"Value 1 is ${value1}.":} notify {"Value 2 is ${value2}.":}}class {paramclassexample: value1 => Something, value2 => Something else,}class {paramclassexample: value1 => Something,}
    86. 86. modules
    87. 87. modules{module}/ files/ lib/ manifests/ init.pp {class}.pp {defined type}.pp {namespace}/ {class}.pp {class}.pp templates/ tests/
    88. 88. templating
    89. 89. templatingfile {/etc/foo.conf: ensure => file, require => Package[foo], content => template(foo/foo.conf.erb),}
    90. 90. templates/foo.conf.erb
    91. 91. templates/foo.conf.erbOS is <%= $::operatingsystem %>
    92. 92. node configuration
    93. 93. node configuration# nodes.ppnode someserver.domain.com inherits basenode { $web_fqdn = www.domain.com include genericwebserver include some_other_service}node ldapmaster.domain.com inherits basenode { include s_ldap::master}node humanresources.domain.com inherits basenode { include c_humanresources}
    94. 94. Vagrant
    95. 95. Vagrant
    96. 96. Vagrant Oracle VirtualBox cmdline automation Easy Puppet and Chef provisioning Keep VM configuration for different projectsShare boxes and configuration files across teams base box + configuration files
    97. 97. Vagrant base boxes
    98. 98. Vagrant base boxes www.vagrantbox.es anywhere! just (big) filesVeeWee tool to automate installations https://github.com/jedi4ever/veewee
    99. 99. using Vagrant
    100. 100. using Vagrant$ gem install vagrant$ vagrant box add centos-6.0-x86_64 http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box$ vagrant init myproject$ vagrant up$ vagrant ssh$ vagrant suspend$ vagrant resume$ vagrant destroy
    101. 101. Vagrant
    102. 102. VagrantVagrant::Config.run do |config| # Every Vagrant virtual environment requires a box to build off of. config.vm.box = "centos-6.0-x86_64" # The url from where the config.vm.box box will be fetched config.vm.box_url = "http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box" # Boot with a GUI so you can see the screen. (Default is headless) #config.vm.boot_mode = :gui # Assign this VM to a host only network IP, allowing you to access it via the IP. # config.vm.network "33.33.33.10" # Forward a port from the guest to the host, which allows for outside # computers to access the VM, whereas host only networking does not. config.vm.forward_port "sonar", 9000, 19000 # Enable provisioning with Puppet stand alone. config.vm.share_folder("templates", "/tmp/vagrant-puppet/templates", "templates") config.vm.provision :puppet do |puppet| puppet.manifest_file = "base.pp" puppet.module_path = "mymodules" puppet.options = ["--templatedir","/tmp/vagrant-puppet/templates"] puppet.options = "-v -d" endend
    103. 103. manifests/base.pp
    104. 104. manifests/base.pppackage { jdk: ensure => installed, name => $operatingsystem ? { centOS => "java-1.6.0-openjdk-devel", Ubuntu => "openjdk-6-jdk", default => "jdk", },}
    105. 105. Maven and Puppet
    106. 106. What can I do to automate deployment
    107. 107. What can I do to automate deployment Handle full deployment including infrastructure not just webapp deployment Help Ops with clear, automated manifests Ability to reproduce production environments in local box using Vagrant / VirtualBox / VMWare
    108. 108. Maven-Puppet module
    109. 109. Maven-Puppet module A Maven Puppet modulehttps://github.com/maestrodev/puppet-maven fetches Maven artifacts from the repo manages them with Puppet no more extra packaging
    110. 110. Requirements
    111. 111. Requirements Java JDK Apache Buildrpreinstalled in the boxchicken and egg problem
    112. 112. New Maven type
    113. 113. New Maven type maven { "/tmp/maven-core-2.2.1.jar": id => "org.apache.maven:maven-core:jar:2.2.1", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"],}
    114. 114. New Maven type
    115. 115. New Maven type maven { "/tmp/maven-core-2.2.1.jar": groupId => "org.apache.maven", artifactId => "maven-core", version => "2.2.1", packaging => "jar", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"],}
    116. 116. Example code
    117. 117. Example code Available at http://github.carlossanchez.eu http://blog.carlossanchez.euhttps://github.com/maestrodev/puppet-modules
    118. 118. Questions?
    119. 119. Thanks!http://maestrodev.comhttp://carlossanchez.eucsanchez@maestrodev.comcarlos@apache.orgcsanchez
    120. 120. Photo Credits
    121. 121. Photo Credits Son of Man Lego - Alex Eylarhttp://www.flickr.com/photos/hoyvinmayvin/4702772452/ Brick wall - Luis Argerich http://www.flickr.com/photos/lrargerich/4353397797/ Agile vs. Iterative flow - Christopher Littlehttp://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg DevOps - Rajiv.Pant http://en.wikipedia.org/wiki/File:Devops.png

    ×