Puppet for Java developers - JavaZone NO 2012

36,637 views
33,056 views

Published on

Example code at https://github.com/carlossg/puppet-for-java-devs
More info at http://blog.carlossanchez.eu/tag/devops
Video at http://vimeo.com/49483627

Puppet is an infrastructure-as-code tool that allows easy and automated provisioning of servers, defining the packages, configuration, services,... in code. Enabling DevOps culture, tools like Puppet help drive Agile development all the way to operations and systems administration, and along with continuous integration tools like Jenkins, it is a key piece to accomplish repeatability and continuous delivery, automating the operations side during development, QA or production, and enabling testing of systems configuration.
Traditionally a field for system administrators, Puppet can empower developers, allowing both to collaborate coding the infrastructure needed for their developments, whether it runs in hardware, virtual machines or cloud. Developers and sysadmins can define what JDK version must be installed, application server, version, configuration files, war and jar files,... and easily make changes that propagate across all nodes.
Using Vagrant, a command line automation layer for VirtualBox, they can also spin off virtual machines in their local box, easily from scratch with the same configuration as production servers, do development or testing and tear them down afterwards.
We’ll show how to install and manage Puppet nodes with JDK, multiple application server instances with installed web applications, database, configuration files and all the supporting services. Including getting up and running with Vagrant and VirtualBox for quickstart and Puppet experiments, as well as setting up automated testing of the Puppet code.

Published in: Technology
0 Comments
41 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
36,637
On SlideShare
0
From Embeds
0
Number of Embeds
16,966
Actions
Shares
0
Downloads
0
Comments
0
Likes
41
Embeds 0
No embeds

No notes for slide
  • \n
  • question? #developers vs #ops vs #devops\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • question? what do you do to specify ops requirements?\n
  • \n
  • question? what do you do to specify ops requirements?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • question? who’s able to roll back to a previous version of software? and system?\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Puppet for Java developers - JavaZone NO 2012

    1. Puppet for JavadevelopersCarlos Sanchez@csanchezhttp://carlossanchez.euhttp://maestrodev.com
    2. Apache @csanchez Maven Apache ASF Archiva Member ApacheEclipse IAM Continuum blog.carlossanchez.eu
    3. DevOps
    4. DevQaOps ?
    5. Agile planning iterative development continuous integrationrelease soon, release often
    6. DevOps addresses Fear of change Risky deployments It works on my machine! SiloisationDev Change vs. Ops stability
    7. Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan
    8. DevOps is NOT about the tools
    9. Nice, BUThow can I implement IT
    10. Nice, BUThow can I implement IT
    11. Tools can enablechange in behavior and eventually change culture Patrick Debois
    12. DevOps toolseveryone is intelligentenoughevery tool is cloudenabledevery tool is DevOps(tm)
    13. DevOps toolseveryone is intelligentenoughevery tool is cloudenabledevery tool is DevOps(tm)
    14. DevOps tools: infrastructure automation infrastructure as code it’s all invented, now it’s standardized
    15. Infrastructure as CodeNew solutions bring new challengesFollow development best practices tagging branching releasing dev, QA, production
    16. New solutions bring new problems DEVOPS “MY MACHINES ARE BEING PROVISIONED”
    17. Puppet exec { "maven-untar":manifests command => "tar xf /tmp/x.tgz", cwd => "/opt", creates => "/opt/apache-maven-${version}",ruby-like path => ["/bin"], } -> ERB templates file { "/usr/bin/mvn": ensure => link, target => "/opt/apache-maven-${version}/bin/mvn", } file { "/usr/local/bin/mvn": ensure => absent, require => Exec["maven-untar"], } file { "$home/.mavenrc": mode => "0600", owner => $user, content => template("maven/mavenrc.erb"), require => User[$user], }
    18. Puppetinfrastructure IS code package { openssh-server: ensure => present, }
    19. Puppetdeclarative modelstate vs process no scripting service { ntp: name => ntpd, ensure => running, }
    20. master - agentarchitecture
    21. Vagrant
    22. Vagrant Oracle VirtualBox cmdline automation Easy Puppet and Chef provisioning Keep VM configuration for different projectsShare boxes and configuration files across teams base box + configuration files http://vagrantup.com/
    23. Vagrant base boxes www.vagrantbox.esanywhere! just (big) files
    24. using Vagrant$ vagrant box add centos-6.0-x86_64 http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box$ vagrant init myproject$ vagrant up$ vagrant ssh$ vagrant suspend$ vagrant resume$ vagrant destroy
    25. VagrantVagrant::Config.run do |config| # Every Vagrant virtual environment requires a box to build off of. config.vm.box = "centos-6.0-x86_64" # The url from where the config.vm.box box will be fetched config.vm.box_url = "http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box" # Boot with a GUI so you can see the screen. (Default is headless) #config.vm.boot_mode = :gui # Assign this VM to a host only network IP, allowing you to access it viathe IP. # config.vm.network "33.33.33.10" # Forward a port from the guest to the host, which allows for outside # computers to access the VM, whereas host only networking does not. config.vm.forward_port "sonar", 9000, 19000 config.vm.provision :puppet do |puppet| puppet.manifest_file = "site.pp" puppet.module_path = "mymodules" endend
    26. manifests/base.pppackage { jdk: ensure => installed, name => $operatingsystem ? { centOS => "java-1.6.0-openjdk-devel", Ubuntu => "openjdk-6-jdk", default => "jdk", },}
    27. VeeWee
    28. VeeWee easily build Vagrant base boxeshttps://github.com/jedi4ever/veewee
    29. using VeeWee$ gem install veewee$ vagrant basebox templates$ vagrant basebox define my-ubuntu-server ubuntu-11.04-server-amd64# customize definitions/my-ubuntu-server$ vagrant basebox build my-ubuntu-server$ vagrant basebox validate my-ubuntu-server$ vagrant basebox export my-ubuntu-server$ vagrant box add my-ubuntu-server my-ubuntu-server.box$ vagrant init my-ubuntu-server
    30. Editors
    31. Geppetto
    32. http://cloudsmith.github.com/geppetto
    33. TextMate
    34. https://github.com/masterzen/puppet-textmate-bundle
    35. Puppet DSL
    36. Puppet resourcesuser { dave: ensure => present, uid => 507, gid => admin, shell => /bin/zsh, home => /home/dave, managehome => true,}
    37. Puppet resourcesfile {testfile: path => /tmp/testfile, ensure => present, mode => 0640, content => "Im a test file.",}notify {"Im notifying you.":}
    38. Puppet standalone$ puppet apply my_test_manifest.pp
    39. orderingfile {/tmp/test1: ensure => present, content => "Hi.",}notify {/tmp/test1 has alreadybeen synced.: require => File[/tmp/test1],}
    40. orderingfile {/tmp/test1: ensure => present, content => "Hi.",} ->notify {/tmp/test1 has alreadybeen synced.:}
    41. package / file /service / subscribepackage { openssh-server: ensure => present, before => File[/etc/ssh/sshd_config],}file { /etc/ssh/sshd_config: ensure => file, mode => 600, source => /root/learning-manifests/sshd_config,}service { sshd: ensure => running, enable => true, hasrestart => true, hasstatus => true, subscribe => File[/etc/ssh/sshd_config],}
    42. variables$longthing = "Imagine I have something reallylong in here. Like an SSH key, lets say."file {authorized_keys: path => /root/.ssh/authorized_keys, content => $longthing,}
    43. factshost {self: ensure => present, name => $::fqdn, host_aliases => [puppet, $::hostname], ip => $::ipaddress,}file {motd: ensure => file, path => /etc/motd, mode => 0644, content => "Welcome to ${::hostname},na ${::operatingsystem} island in the sea of ${::domain}.n",}
    44. conditionalsif $is_virtual { service {ntpd: ensure => stopped, enable => false, }}else { service { ntpd: name => ntpd, ensure => running, enable => true, hasrestart => true, require => Package[ntp], }}
    45. conditionalscase $operatingsystem { centos, redhat: { $apache = "httpd" } debian, ubuntu: { $apache = "apache2" } default: { fail("Unrecognized operating system forwebserver") }}$apache = $operatingsystem ? { centos => httpd, redhat => httpd, /(?i)(ubuntu|debian)/ => "apache2-$1", # (Dont actually use that package name.) default => undef, }
    46. class definitionclass ntp { package { ntp: ensure => installed, } service { ntp: name => ntpd, ensure => running, enable => true, subscribe => File[ntp.conf], }}
    47. class declarationclass {myclass: param1 => a, param2 => b,}
    48. class declaration (2)include ntp
    49. parameterized classesclass paramclassexample ($value1, $value2 = "Defaultvalue") { notify {"Value 1 is ${value1}.":} notify {"Value 2 is ${value2}.":}}class {paramclassexample: value1 => Something, value2 => Something else,}class {paramclassexample: value1 => Something,}
    50. definitionsdefine myfile ($path) { file {$path: content => Something}}myfile {a: path => /tmp/a,}myfile {b: path => /tmp/b,}
    51. templatingfile {/etc/foo.conf: ensure => file, require => Package[foo], content => template(foo/foo.conf.erb),}
    52. templates/foo.conf.erbOS is <%= $::operatingsystem %>
    53. node configuration# nodes.ppnode someserver.domain.com inherits basenode { $web_fqdn = www.domain.com include genericwebserver include some_other_service}node ldapmaster.domain.com inherits basenode { include s_ldap::master}node humanresources.domain.com inherits basenode { include c_humanresources}
    54. Puppet modules
    55. module tool$ puppet module generate csanchez-mymodule
    56. modules{module}/ files/ lib/ manifests/ init.pp {class}.pp {defined type}.pp {namespace}/ {class}.pp {class}.pp templates/ spec/ tests/
    57. Modulefilename csanchez-mymoduleversion 0.0.1source UNKNOWNauthor csanchezlicense Apache License,Version 2.0summary UNKNOWNdescription UNKNOWNproject_page UNKNOWN## Add dependencies, if any:# dependency username/name, >= 1.2.0
    58. building a module$ puppet module build
    59. Puppetlabs Forgehttp://forge.puppetlabs.com A library of modules
    60. librarian-puppethttp://librarian-puppet.com/$ librarian-puppet init# edit Puppetfile$ librarian-puppet install
    61. Puppetfileforge "http://forge.puppetlabs.com"# mod puppetlabs/stdlib# mod ntp,# :git => git://github.com/puppetlabs/puppetlabs-ntp.git# mod apt,# :git => https://github.com/puppetlabs/puppetlabs-apt.git,# :ref => feature/master/dans_refactor
    62. Maven and Puppet
    63. What am I doing to automate deployment Ant tasks plugin ssh commands Assembly plugin Cargo Capistrano
    64. What can I do to automate deployment Handle full deployment including infrastructure not just webapp deployment Help Ops with clear, automated manifests Ability to reproduce production environments in local box using Vagrant / VirtualBox / VMWare Use the right tool for the right job
    65. Maven-Puppet module A Maven Puppet modulehttps://github.com/maestrodev/puppet-maven fetches Maven artifacts from the repo manages them with Puppet no more extra packaging
    66. Installing Maven$repo1 = { id => "myrepo", username => "myuser", password => "mypassword", url => "http://repo.acme.com",}# Install Mavenclass { "maven::maven": version => "2.2.1",} -># Create a settings.xml with the repo credentialsclass { "maven::settings" : servers => [$repo1],}
    67. New Maven typemaven { "/tmp/maven-core-2.2.1.jar": id => "org.apache.maven:maven-core:jar:2.2.1", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"],}
    68. New Maven typemaven { "/tmp/maven-core-2.2.1.jar": groupId => "org.apache.maven", artifactId => "maven-core", version => "2.2.1", packaging => "jar", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"],}
    69. Examples
    70. Tomcat cluster + postgres postgres database db.acme.com tomcat servers tomcat1.acme.com tomcat2.acme.com ... web server www.acme.com
    71. Modules required puppetlabs/stdlib puppetlabs/java puppetlabs/apache inkling/postgresqlcamptocamp/tomcat maestrodev/maven maestrodev/wget stahnma/epel
    72. Puppetfileforge http://forge.puppetlabs.commod puppetlabs/stdlib, 3.0.1mod puppetlabs/java, 0.1.6mod puppetlabs/apache, 0.2.2mod inkling/postgresqlmod camptocamp/tomcat, :git => git://github.com/camptocamp/puppet-tomcat.git, :ref => 16e498mod maestrodev/maven, ‘0.0.1’mod maestrodev/wget, 0.0.1mod stahnma/epel, 0.0.2
    73. inkling - postgresclass { postgresql::server: config_hash => { ip_mask_allow_all_users => 192.168.0.0/0, listen_addresses => *, manage_redhat_firewall => true, postgres_password => postgres, },}postgresql::db { appfuse: user => appfuse, password => appfuse, grant => all,}
    74. camptocamp - tomcattomcat::instance { appfuse: ensure => present, http_port => 8080,}$webapp = /srv/tomcat/appfuse/webapps/ROOT
    75. maestrodev - mavenmaven { "${webapp}.war": id => org.appfuse:appfuse-spring:2.1.0:war, require => File[/srv/tomcat/appfuse/webapps], notify => Service[tomcat-centrepoint],}
    76. Puppet Rspec require spec_helperdescribe db.acme.com do let(:facts) { { :osfamily => RedHat, :operatingsystem => CentOS, :operatingsystemrelease => 6.3} } it { should_not contain_class(java) } it { should contain_class(postgresql::server)}end
    77. Example code and slides Available at http://slideshare.carlossanchez.eu http://github.carlossanchez.euhttps://github.com/carlossg/puppet-for-java-devs http://blog.carlossanchez.euhttps://github.com/maestrodev/puppet-modules
    78. Takk!http://blog.carlossanchez.euhttp://maestrodev.comcsanchez@maestrodev.comcarlos@apache.org@csanchez
    79. Photo Credits Son of Man Lego - Alex Eylarhttp://www.flickr.com/photos/hoyvinmayvin/4702772452/ Brick wall - Luis Argerich http://www.flickr.com/photos/lrargerich/4353397797/ Agile vs. Iterative flow - Christopher Littlehttp://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg DevOps - Rajiv.Pant http://en.wikipedia.org/wiki/File:Devops.png Pimientos de Padron - Howard Walfish http://www.flickr.com/photos/h-bomb/4868400647/ Compiling - XKCD http://xkcd.com/303/ Printer in 1568 - Meggs, Philip B http://en.wikipedia.org/wiki/File:Printer_in_1568-ce.png Relativity - M. C. Escherhttp://en.wikipedia.org/wiki/File:Escher%27s_Relativity.jpg Teacher and class - Herald Post http://www.flickr.com/photos/heraldpost/5169295832/

    ×