Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Dev ops for developers

853 views

Published on

  • Be the first to comment

Dev ops for developers

  1. 1. Johann-Peter Hartmann / Mayflower GmbHPractical DevOps forDevelopers
  2. 2. Hi!
  3. 3. Nice to meet You!
  4. 4. I am Johann.
  5. 5. Who are You?
  6. 6. Developers? Ops?Anything else?
  7. 7. Developers: PHP? Python? Ruby? Java?Anything else?
  8. 8. What do you expect to hear today? (if you got a good explanationi‘ll leave out the motivation slides)
  9. 9. Motivation orWhy it‘s more fun to do DevOps as a Dev right now.
  10. 10. Hype!Googles CIO Ben Fried 11 days ago: „Cult Of DevOps“
  11. 11. Actually:Because we need it.
  12. 12. Development in the dark ages ...
  13. 13. Golden ImageVMware /KVM
  14. 14. It wasn‘t reliable anymore.
  15. 15. And our applications changed
  16. 16. BrowserApache with PHP MySQL
  17. 17. Browser HipApache with PHP MySQL NoSQL Server
  18. 18. BrowserApache with PHP MySQL Hip Memcache NoSQL Performance FTW! Server
  19. 19. Browser HipApache with PHP MySQL NoSQL Server Memcache, oops, Gearman Redis now
  20. 20. Browser HipApache with PHP MySQL NoSQL ServerMemcache, oops, Gearman, ahum, eJabberD Redis now ActiveMQ
  21. 21. Browser HipApache with Apache with MySQL NoSQL PHP PHP ServerMemcache, Memcache, Gearman, oops, oops, ahum, eJabberDRedis now Redis now ActiveMQ
  22. 22. Browser HipApache with Apache with MySQL NoSQL PHP PHP ServerMemcache, Memcache, Gearman, oops, oops, ahum, eJabberDRedis now Redis now ActiveMQ
  23. 23. Apa Apa MyS Hip Apa Apa MyS HipMe Me Gea eJa Me Me Gea eJa Development Continuous Integration Apa Apa Apa Apa MyS MyS Hip Hip Me Me Me Me Gea Gea eJa eJa Production
  24. 24. Version X+2 Version X+1 Development Continuous Integration Version X Production
  25. 25. New! With Redis! Not so new with MemcachedVersion X+2 Version X+1 Development Continuous Integration Still without proper Caching :-( Version X Production
  26. 26. 10th Floor Test http://www.flickr.com/photos/75905404@N00/
  27. 27. Collection of Fails Failsafety Simplicity Fast Setup TimeRepeatability Self-ServiceConsistency Version Management http://www.flickr.com/photos/turtlemom_nancy/
  28. 28. ?
  29. 29. DevOps for Devs in3 (actually not so easy) steps
  30. 30. 1. Manage your Dev-Setup with Vagrant and VeeWee (Install VirtualBox and Ruby first)
  31. 31. Vagrant Virtualbox based automatic creation and management of VMs based on Puppet / Chef
  32. 32. Better use rvm, see http://www.jedi.be/blog/2011/03/28/using-vagrant-as-a-team/~# gem install vagrant~# gem install veewee~# vagrant basebox templatesvagrant basebox define <boxname> archlinux-i386... (40 baseboxes)vagrant basebox define <boxname> windows-2008R2-amd64~# vagrant basebox define natty ubuntu-11.04-server-amd64~# vagrant basebox build natty~# vagrant basebox export natty~# vagrant box add natty natty.box
  33. 33. http://vagrantbox.es
  34. 34. ~# vagrant init natty~# vagrant up~# vagrant sshWelcome to Ubuntu 11.04 (GNU/Linux 2.6.38-8-server x86_64) * Documentation: http://www.ubuntu.com/server/docvagrant@natty:~$
  35. 35. ~# vagrantTasks: vagrant basebox # Commands to manage baseboxes vagrant box # Commands to manage system boxes vagrant destroy # Destroy the environment, deleting the created virtualmachines vagrant halt # Halt the running VMs in the environment vagrant help [TASK] # Describe available tasks or one specific task vagrant init [box_name] [box_url] # Initializes the current folder forVagrant usage vagrant package # Package a Vagrant environment for distribution vagrant provision # Rerun the provisioning scripts on a running VM vagrant reload # Reload the environment, halting it then restartingit. vagrant resume # Resume a suspended Vagrant environment. vagrant ssh # SSH into the currently running Vagrant environment. vagrant ssh_config # outputs .ssh/config valid syntax for connecting tothis environment via ssh vagrant status # Shows the status of the current Vagrant environment. vagrant suspend # Suspend a running Vagrant environment. vagrant up # Creates the Vagrant environment vagrant version # Prints the Vagrant version information
  36. 36. 2. Manage your configuration
  37. 37. Similarities
  38. 38. • Configuration as (Ruby-)Code• Client-only or Client-server Setup• there is a series-b funded company in the background• both are officially supported by amazon• there are a lot of BIG customers using the tool• good documentation• good, vibrant communities
  39. 39. • both know the current configuration (ohai and facter)• You define your nodes (Servers)• using a lot of pre-existing resources• and a lot of default community-built cookbooks / modules• it‘s easy to extend using ruby• use configuration file templates• use providers as platform abstractions (for packaging, ...)
  40. 40. • Chef is actually Ruby, Puppet provides a DSL• puppet has the bigger community• puppet has more documentation• but chef is growing fast in both regards• puppet = europe, chef = usa• chef is more flexible because of native ruby• chef is more flexible because of clever data structures
  41. 41. There is no „better“ tool.
  42. 42. There is no „better“ tool.But we prefer Puppet. Less Ruby :-)
  43. 43. user { johann: ensure => present, uid => 507, gid => admin, shell => /bin/bash, home => /home/johann, managehome => true,}
  44. 44. user "johann" do username "johann" password "$1$P$WXmqrQEVj88fVTHevErxq." shell "/bin/bash" system true supports :manage_home => trueend
  45. 45. Back to your setup ...
  46. 46. Vagrant::Config.run do |config| config.vm.box = "natty"end
  47. 47. ~# cat VagrantfileVagrant::Config.run do |config| config.vm.provision :puppet, :module_path => "modules" do|puppet| puppet.manifests_path = "manifests" puppet.manifest_file = "development.pp" end config.vm.define :web do |web_config| web_config.vm.box = "natty" web_config.vm.host_name = "webserver01" # web_config.vm.boot_mode = :gui web_config.vm.network "33.33.33.10" web_config.vm.forward_port "http", 80, 8080 web_config.vm.forward_port "ssh", 22, 20022 web_config.vm.share_folder "v-data", "/srv/www", "../silex-demo" endend
  48. 48. ~# cat manifests/development.ppimport "classes/*"node "webserver01" { include web}node "dbserver01" { include db}node "ciserver01" { include ci}
  49. 49. ~# cat manifests/classes/web.ppclass web inherits basenode { include apache include apache::php apache::vhost { silex-demo.local: port => 80, docroot => /srv/www/docroot, } package { ["mysql-client", "php5-cli", "phpunit", "php5-curl", "php5-dev", "php5-gd", "php5-imagick", "php5-mcrypt","php5-mysql", "php5-xdebug","php5-suhosin", "php-pear","php-codesniffer" ]: ensure => present, }}
  50. 50. ~# cat manifests/classes/ci.ppclass ci inherits basenode { include apache include apache::php exec { "pear_autodiscover": command => "/usr/bin/pear config-set auto_discover 1", } package { ["pear.phpunit.de/PHP_CodeBrowser","pear.phpunit.de/PHPUnit_MockObject", "pear.phpunit.de/PHPUnit_Selenium", "pear.phpunit.de/PHP_CodeCoverage","pear.phpunit.de/PHP_Timer", "pear.phpunit.de/phpcpd","pear.phpunit.de/phploc"]: ensure => latest, provider => "pear", require => Exec["pear_autodiscover"] }}
  51. 51. 3. Make the configuration part of your sourcecode
  52. 52. • application• data• docs• library• public• scripts • jobs • build • configuration • VagrantFile • manifests • modules• temp
  53. 53. Ok, that was a lot of work. Why did i do that again?
  54. 54. Collection of Wins Failsafety Simplicity Fast Setup TimeRepeatability Self-ServiceConsistency Version Management http://www.flickr.com/photos/turtlemom_nancy/
  55. 55. There is no golden Image anymore
  56. 56. There is just one directory in your source ...
  57. 57. „vagrant up“„vagrant provision“
  58. 58. Even more vagrant fun ...
  59. 59. Fail Safety with Vagrant-Snapvagrant snap take -d “snap1“vagrant snap listvagrant snap go “snap1“
  60. 60. What if i need tosimulate 20
  61. 61. McCloudWrapper likeVagrant around FogTransparent local & cloud usageSupports EC2, OpenStack, KVM, etc
  62. 62. What if i need tosimulate 2000machines?
  63. 63. mCollectivedssh/ssh-for-loop on steroidsfast management for loads of serversuses puppet/facter or chef/ohai, MQ- based
  64. 64. $ mc-package -W "architecture=x86" status apache * [ ============================================================> ] 10 / 10host01.example.com version = apache-2.2.9-7host02.example.com version = apache-2.2.9-7host03.example.com version = apache-2.2.9-7host04.example.com version = apache-2.2.9-7host05.example.com version = apache-2.2.9-7host06.example.com version = apache-2.2.9-7host07.example.com version = apache-2.2.9-7host08.example.com version = apache-2.2.9-7host09.example.com version = apache-2.2.9-7host10.example.com version = apache-2.2.9-7---- package agent summary ---- Nodes: 10 / 10 Versions: 10 * 0.25.5-1.el5 Elapsed Time: 1.03 s
  65. 65. The Foreman - Machine Life CycleManagementActs as a web front end for PuppetShows You the system inventoryCreates new machines and takes care of provisioning
  66. 66. Knowing everything about yourserver configuration is great!
  67. 67. Create new servers
  68. 68. Your (possible) Todolist
  69. 69. 1. Install Vagrant, Veewee etc
  70. 70. 2. Add anconfigurationfolder to your
  71. 71. 3. Configure Your Vagrant images using chef or puppet
  72. 72. 4. Setup a chef-or puppet-server using this configuration
  73. 73. 5. Move your CI, Staging andProduction-Setup to chef/puppet, too
  74. 74. 6. Create a self-service-plattform for thedevelopment team
  75. 75. More cool stuff you can do:
  76. 76. Test-Driven infrastructure using chef/puppet-cucumber
  77. 77. Automatedmonitoring usingnagios or munin
  78. 78. Automatedreporting using graphite
  79. 79. Thanks!Johann-Peter Hartmannjohann__ @ freenode I hope you enjoyed it!Mail / Jabber:johann-peter.hartmann@mayflower.deFurther reading:http://www.planetdevops.nethttp://dzone.comhttp://twitter.com/#!/DEVOPS_BORAThttp://github.com/johannhartmann/
  80. 80. Vegetable Test DrivenInfrastructure Kontinuierliche Entiwcklung - und dann? I Mayflower GmbH I 28. Oktober 2010 I 87
  81. 81. actually a behavior driven design toolused for test driven infrastructurecucumber-puppetchef-cucumber
  82. 82. Feature: Manualsearch In order to find an article As an developer I want to use the search function Scenario: Search for bdd and check resulting page Given I go to "http://it-republik.de/php/" When I fill in "search_itr" with "bdd" And I click "search2" Then I should see "Suche"
  83. 83. Given /^I go to "([^"]*)"$/ do |url| visit urlendWhen /^I fill in "([^"]*)" with "([^"]*)"$/ do |field, value| fill_in field, :with => valueendWhen /^I click "([^"]*)"$/ do |button| click_button(button)endThen /^I should see "([^"]*)"$/ do |text| response_body.should include(text)end
  84. 84. johann$ cucumberFeature: Manualsearch In order to find an article As an developer I want to use the search function Scenario: Search for bdd and check resulting page # features/search.feature:5 Given I go to "http://it-republik.de/php/" # features/step_definitions/search_steps.rb:1 When I fill in "search_itr" with "bdd" # features/step_definitions/search_steps.rb:5 And I click "search2" # features/step_definitions/search_steps.rb:9 Then I should see "Suche" # features/step_definitions/search_steps.rb:131 scenario (1 passed)4 steps (4 passed)0m1.615s
  85. 85. Feature: Install inetd In order to serve the web the httpd service must be installed Scenario: Setup httpd Given a node of class “web“ When I compile the catalog Then package “httpd“ should be “installed“ Then there should be a resource “Service[httpd]“ And the service should have “enable“ set to “true“ And State should be “running“ And the service should require “Package[httpd]“ Kontinuierliche Entiwcklung - und dann? I Mayflower GmbH I 28. Oktober 2010 I 92
  86. 86. Hey, i understandOperations - and i‘m working in marketing
  87. 87. Unit TestsAcceptance TestsMetricsCoding Style... you get theidea.
  88. 88. ... and evenmore ...PackagingInfrastructureUpdatesReleasesDeployment
  89. 89. WetwareRefactoring
  90. 90. Work together
  91. 91. Own together
  92. 92. Eat together http://www.flickr.com/photos/vilavelosa/ 3815032524/
  93. 93. Plantogether Kontinuierliche Entiwcklung - und dann? I Mayflower GmbH I 28. Oktober 2010 I 103
  94. 94. MayflowerDevOpsification
  95. 95. Wetware - Mayflower 1-2 Ops per team • Admin & Development tasks • full time working within the team
  96. 96. Wetware - MayflowerClose cooperation with central operationsFull root access for any development-infrastructre
  97. 97. Wetware - Mayflower1+n Puppet-Master • central company master • Team puppet master per team / project • company master is starting point for the project puppet configuration
  98. 98. Wetware - MayflowerExample Setup: • local Developer VM • CI-Deployment-Server in the DMZ • Staging in the private Cloud • Beta happens in the Amazon-Cloud • Production in AWS, too
  99. 99. Wetware - Mayflowerlocal GIT- / Gitorious-ServerEucalyptus-Cloud in the DMZ - as Self-Service!
  100. 100. Wetware - Mayflower Vagrant for Development Scrum => KanBan Puppet Nagios

×