Your SlideShare is downloading. ×
Infrastructure testing with Jenkins, Puppet and Vagrant - Agile Testing Days 2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Infrastructure testing with Jenkins, Puppet and Vagrant - Agile Testing Days 2013

21,862
views

Published on

Extend Continuous Integration to automatically test your infrastructure. …

Extend Continuous Integration to automatically test your infrastructure.

Continuous Integration can be extended to test deployments and production environments, in a Continuous Delivery cycle, using infrastructure-as-code tools like Puppet, allowing to manage multiple servers and their configurations, and test the infrastructure the same way continuous integration tools do with developers’ code.

Puppet is an infrastructure-as-code tool that allows easy and automated provisioning of servers, defining the packages, configuration, services, … in code. Enabling DevOps culture, tools like Puppet help drive Agile development all the way to operations and systems administration, and along with continuous integration tools like Jenkins, it is a key piece to accomplish repeatability and continuous delivery, automating the operations side during development, QA or production, and enabling testing of systems configuration.

Using Vagrant, a command line automation layer for VirtualBox, we can easily spin off virtual machines with the same configuration as production servers, run our test suite, and tear them down afterwards.

We will show how to set up automated testing of an application and associated infrastructure and configurations, creating on demand virtual machines for testing, as part of your continuous integration process.

Published in: Technology

0 Comments
40 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
21,862
On Slideshare
0
From Embeds
0
Number of Embeds
21
Actions
Shares
0
Downloads
218
Comments
0
Likes
40
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Infrastructure testing with Jenkins, Puppet and Vagrant Carlos Sanchez @csanchez http://csanchez.org http://maestrodev.com
  • 2. @csanchez ASF Member Apache Maven Eclipse Foundation csanchez.org maestrodev.com
  • 3. How we got here
  • 4. Agile planning iterative development continuous integration release soon, release often
  • 5. Fear of change Risky deployments It works on my machine! Siloisation Dev Change vs. Ops stability
  • 6. OPs requirements Operating System config files packages installed multi stage configurations dev QA pre-production production
  • 7. Deployment How do I deploy this? documentation manual steps prone to errors
  • 8. Cloud How do I deploy this? to hundreds of servers
  • 9. DevOps
  • 10. DevQaOps ?
  • 11. DEV QA OPS
  • 12. QA DEV OPS
  • 13. Specs Packages Versions DEV PROD
  • 14. DEV QA Testability
  • 15. Metrics Logs Security updates DEV PROD
  • 16. is not about the tools but how can I implement IT
  • 17. Tools can enable change in behavior and eventually change culture Patrick Debois
  • 18. everyone is intelligent enough every tool is cloud enabled every tool is DevOps(tm)
  • 19. 3 key concepts
  • 20. Continuous Delivery
  • 21. Continuous delivery
  • 22. Infrastructure as Code it’s all been invented, now it’s standardized
  • 23. manifests ruby-like ERB templates exec { "maven-untar": command => "tar xf /tmp/x.tgz", cwd => "/opt", creates => "/opt/apache-maven-${version}", path => ["/bin"], } -> file { "/usr/bin/mvn": ensure => link, target => "/opt/apache-maven-${version}/bin/mvn", } file { "/usr/local/bin/mvn": ensure => absent, require => Exec["maven-untar"], } file { "$home/.mavenrc": mode => "0600", owner => $user, content => template("maven/mavenrc.erb"), require => User[$user], }
  • 24. package { 'openssh-server': ensure => present, } infrastructure IS code
  • 25. service { 'ntp': name => 'ntpd', ensure => running, } declarative model state vs process no scripting
  • 26. Follow development best practices tagging branching releasing dev, QA, production new solutions new challenges
  • 27. Self servicing
  • 28. Infrastructure always available virtualization & cloud empower developers reduce time-to-market
  • 29. devs buy-in With great power comes great responsibility
  • 30. Vagrant empower developers dev-ops collaboration automation
  • 31. Vagrant
  • 32. Vagrant Virtual and cloud automation VirtualBox VMWare Fusion AWS Rackspace Easy Puppet and Chef provisioning Keep VM configuration for different projects Share boxes and configuration files across teams base box + configuration files
  • 33. Vagrant base boxes www.vagrantbox.es puppet-vagrant-boxes.puppetlabs.com anywhere! just (big) files
  • 34. using Vagrant $ gem install vagrant $ vagrant box add centos-6.0-x86_64 http://dl.dropbox.com/u/1627760/centos-6.0-x86_64.box $ $ $ $ $ $ vagrant vagrant vagrant vagrant vagrant vagrant init myproject up ssh suspend resume destroy
  • 35. Vagrant Vagrant.configure("2") do |config| # Every Vagrant virtual environment requires a box to build off of. config.vm.box = "CentOS-6.4-x86_64-minimal" config.vm.box_url = "https://.../CentOS-6.4-x86_64-minimal.box" # web server config.vm.define :www do |config| config.vm.hostname = "www.acme.local" config.vm.network "forwarded_port", guest: 80, host: 10080 config.vm.network "private_network", ip: "192.168.33.12" end config.vm.provision :puppet do |puppet| puppet.module_path = "modules" puppet.manifest_file = "site.pp" end end
  • 36. Geppetto
  • 37. http://cloudsmith.github.com/geppetto
  • 38. Maven and Puppet
  • 39. What am I doing to automate deployment Ant tasks plugin ssh commands Assembly plugin Cargo Capistrano
  • 40. What can I do to automate deployment Handle full deployment including infrastructure not just webapp deployment Help Ops with clear, automated manifests Ability to reproduce production environments in local box using Vagrant / VirtualBox / VMWare Use the right tool for the right job
  • 41. Maven-Puppet module A Maven Puppet module https://github.com/maestrodev/puppet-maven fetches Maven artifacts from the repo manages them with Puppet no more extra packaging
  • 42. Installing Maven $repo1 = { id => "myrepo", username => "myuser", password => "mypassword", url => "http://repo.acme.com", } # Install Maven class { "maven::maven": version => "2.2.1", } -> # Create a settings.xml with the repo credentials class { "maven::settings" : servers => [$repo1], }
  • 43. New Maven type maven { "/tmp/maven-core-2.2.1.jar": id => "org.apache.maven:maven-core:jar:2.2.1", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"], }
  • 44. New Maven type maven { "/tmp/maven-core-2.2.1.jar": groupId => "org.apache.maven", artifactId => "maven-core", version => "2.2.1", packaging => "jar", repos => ["http://repo1.maven.apache.org/maven2", "http://mirrors.ibiblio.org/pub/mirrors/maven2"], }
  • 45. Examples
  • 46. Infrastructure www httpd Jenkins Archiva QA httpd, tomcat, postgres tomcat1 tomcat db postgres
  • 47. Tomcat cluster + postgres httpd www.acme.com tomcat tomcat1.acme.com tomcat tomcat2.acme.com postgres db.acme.com ...
  • 48. Continuous Delivery commit git repo post commit hook jenkins developer build db production tomcat* www QA vm binary repository integration testing
  • 49. Puppet Modules required $ bundle install && librarian-puppet install forge 'http://forge.puppetlabs.com' mod 'puppetlabs/java', '>=1.0.0' mod 'puppetlabs/apache', '>=0.9.0' mod 'puppetlabs/postgresql', '>=3.0.0' mod 'puppetlabs/firewall' mod 'camptocamp/tomcat', :git => 'https://github.com/carlossg/puppet-tomcat.git', :ref => 'setclasspath' mod 'maestrodev/maven', '>=1.0.0' mod 'stahnma/epel', '>=0.0.2' mod 'maestrodev/avahi', '>=1.0.0' mod 'acme', :path => 'mymodules/acme'
  • 50. mymodules/acme/manifests/db_node.pp class 'acme::db_node' { class { 'postgresql::server': ip_mask_allow_all_users => '192.168.0.0/0', listen_addresses => '*', postgres_password => 'postgres', } -> postgresql::server::db { 'appfuse': user => 'appfuse', password => 'appfuse', grant => 'all', } firewall proto port action } } { '100 allow postgres': => 'tcp', => '5432', => 'accept',
  • 51. mymodules/acme/manifests/tomcat_node.pp I class acme::tomcat_node( $db_host = 'db.local', $repo = 'http://carlos-mbook-pro.local:8000/repository/all/', $appfuse_version = '2.2.2-SNAPSHOT', $service = 'tomcat-appfuse', $app_name = 'appfuse', $tomcat_root = '/srv/tomcat') { # install java class { 'java': } # install tomcat class { 'tomcat': } -> # create a tomcat server instance for appfuse # It allows having multiple independent tomcat servers in different ports tomcat::instance { 'appfuse': ensure => present, http_port => 8080, } # where the war needs to be installed $webapp = "${tomcat_root}/${app_name}/webapps/ROOT"
  • 52. mymodules/acme/manifests/tomcat_node.pp II # install maven and download appfuse war file from our archiva instance class { 'wget': } -> class { 'maven::maven' : version => '3.0.4', } -> # clean up to deploy the next snapshot exec { "rm -rf ${webapp}*": } -> maven { "${webapp}.war": id => "org.appfuse:appfuse-spring:${appfuse_version}:war", repos => [$repo], require => File["${tomcat_root}/${app_name}/webapps"], notify => Service[$service], } -> firewall proto port action } } { '100 allow tomcat': => 'tcp', => '8080', => 'accept',
  • 53. mymodules/acme/manifests/www_node.pp class acme::www_node($tomcat_host = 'tomcat1.local') { class { 'apache': } class { 'apache::mod::proxy_http': } # create a virtualhost that will proxy the tomcat server apache::vhost { "${::hostname}.local": port => 80, docroot => '/var/www', proxy_dest => "http://${tomcat_host}:8080", } firewall { '100 allow apache': proto => 'tcp', port => '80', action => 'accept', } }
  • 54. manifests/site.pp import 'nodes/*.pp' node ‘parent’ { class {'epel': } -> class {'avahi': firewall => true, } }
  • 55. manifests/nodes/tomcat.pp # tomcat1.acme.com, tomcat2.acme.com,... node /tomcatd..*/ inherits ‘parent’ { file {'/etc/motd': content => ”tomcat server: ${::hostname}n”, } class {'acme::tomcat_node'} }
  • 56. manifests/nodes/qa.pp node /qa..*/ inherits ‘parent’ { class {'acme::db_node': } class {'acme::tomcat_node': db_host => 'localhost', } class {'acme::www_node': tomcat_host => 'localhost', } }
  • 57. Infrastructure unit testing
  • 58. rspec-puppet Unit testing your puppet manifests Ensuring packages, config files, services,...
  • 59. spec/hosts/db_spec.pp require 'rspec-puppet' describe 'db.acme.com' do let(:facts) { { :osfamily => 'RedHat', :operatingsystem => 'CentOS', :operatingsystemrelease => ‘6.3’} } it { should contain_class('postgresql::server') } end
  • 60. spec/hosts/www_spec.pp require 'rspec-puppet' describe 'www.acme.com' do let(:facts) { { :osfamily => 'RedHat', :operatingsystem => 'CentOS', :operatingsystemrelease => ‘6.3’} } it { should contain_package('httpd') } end
  • 61. Example code and slides Available at http://slideshare.csanchez.org http://github.csanchez.org http://blog.csanchez.org
  • 62. Danke! http://csanchez.org http://maestrodev.com csanchez@maestrodev.com carlos@apache.org @csanchez
  • 63. Photo Credits Brick wall - Luis Argerich http://www.flickr.com/photos/lrargerich/4353397797/ Agile vs. Iterative flow - Christopher Little http://en.wikipedia.org/wiki/File:Agile-vs-iterative-flow.jpg DevOps - Rajiv.Pant http://en.wikipedia.org/wiki/File:Devops.png Pimientos de Padron - Howard Walfish http://www.flickr.com/photos/h-bomb/4868400647/ Compiling - XKCD http://xkcd.com/303/ Printer in 1568 - Meggs, Philip B http://en.wikipedia.org/wiki/File:Printer_in_1568-ce.png Relativity - M. C. Escher http://en.wikipedia.org/wiki/File:Escher%27s_Relativity.jpg Teacher and class - Herald Post http://www.flickr.com/photos/heraldpost/5169295832/