DevOps for Developers

Johann-Peter Hartmann / Mayflower GmbHPractical DevOps forDevelopers

Nice to meet You!

I am Johann.

Who are You?

Developers? Ops?Anything else?

Developers: PHP? Python? Ruby? Java?Anything else?

What do you expect to hear today? (if you got a good explanationi‘ll leave out the motivation slides)

Motivation orWhy it‘s more fun to do DevOps as a Dev right now.

Hype!Googles CIO Ben Fried 11 days ago: „Cult Of DevOps“

Actually:Because we need it.

Development in the dark ages ...

Golden ImageVMware /KVM

It wasn‘t reliable anymore.

And our applications changed

BrowserApache with PHP MySQL

Browser HipApache with PHP MySQL NoSQL Server

BrowserApache with PHP MySQL Hip Memcache NoSQL Performance FTW! Server

Browser HipApache with PHP MySQL NoSQL Server Memcache, oops, Gearman Redis now

Browser HipApache with PHP MySQL NoSQL ServerMemcache, oops, Gearman, ahum, eJabberD Redis now ActiveMQ

Browser HipApache with Apache with MySQL NoSQL PHP PHP ServerMemcache, Memcache, Gearman, oops, oops, ahum, eJabberDRedis now Redis now ActiveMQ

Apa Apa MyS Hip Apa Apa MyS HipMe Me Gea eJa Me Me Gea eJa Development Continuous Integration Apa Apa Apa Apa MyS MyS Hip Hip Me Me Me Me Gea Gea eJa eJa Production

Version X+2 Version X+1 Development Continuous Integration Version X Production

New! With Redis! Not so new with MemcachedVersion X+2 Version X+1 Development Continuous Integration Still without proper Caching :-( Version X Production

10th Floor Test http://www.flickr.com/photos/75905404@N00/

Collection of Fails Failsafety Simplicity Fast Setup TimeRepeatability Self-ServiceConsistency Version Management http://www.flickr.com/photos/turtlemom_nancy/

DevOps for Devs in3 (actually not so easy) steps

1. Manage your Dev-Setup with Vagrant and VeeWee (Install VirtualBox and Ruby first)

Vagrant Virtualbox based automatic creation and management of VMs based on Puppet / Chef

Better use rvm, see http://www.jedi.be/blog/2011/03/28/using-vagrant-as-a-team/~# gem install vagrant~# gem install veewee~# vagrant basebox templatesvagrant basebox define <boxname> archlinux-i386... (40 baseboxes)vagrant basebox define <boxname> windows-2008R2-amd64~# vagrant basebox define natty ubuntu-11.04-server-amd64~# vagrant basebox build natty~# vagrant basebox export natty~# vagrant box add natty natty.box

http://vagrantbox.es

~# vagrant init natty~# vagrant up~# vagrant sshWelcome to Ubuntu 11.04 (GNU/Linux 2.6.38-8-server x86_64)* Documentation: http://www.ubuntu.com/server/docvagrant@natty:~$

~# vagrantTasks: vagrant basebox # Commands to manage baseboxes vagrant box # Commands to manage system boxes vagrant destroy # Destroy the environment, deleting the created virtualmachines vagrant halt # Halt the running VMs in the environment vagrant help [TASK] # Describe available tasks or one specific task vagrant init [box_name] [box_url] # Initializes the current folder forVagrant usage vagrant package # Package a Vagrant environment for distribution vagrant provision # Rerun the provisioning scripts on a running VM vagrant reload # Reload the environment, halting it then restartingit. vagrant resume # Resume a suspended Vagrant environment. vagrant ssh # SSH into the currently running Vagrant environment. vagrant ssh_config # outputs .ssh/config valid syntax for connecting tothis environment via ssh vagrant status # Shows the status of the current Vagrant environment. vagrant suspend # Suspend a running Vagrant environment. vagrant up # Creates the Vagrant environment vagrant version # Prints the Vagrant version information

2. Manage your configuration

Similarities

• Configuration as (Ruby-)Code• Client-only or Client-server Setup• there is a series-b funded company in the background• both are officially supported by amazon• there are a lot of BIG customers using the tool• good documentation• good, vibrant communities

• both know the current configuration (ohai and facter)• You define your nodes (Servers)• using a lot of pre-existing resources• and a lot of default community-built cookbooks / modules• it‘s easy to extend using ruby• use configuration file templates• use providers as platform abstractions (for packaging, ...)

• Chef is actually Ruby, Puppet provides a DSL• puppet has the bigger community• puppet has more documentation• but chef is growing fast in both regards• puppet = europe, chef = usa• chef is more flexible because of native ruby• chef is more flexible because of clever data structures

There is no „better“ tool.

There is no „better“ tool.But we prefer Puppet. Less Ruby :-)

user { johann: ensure => present, uid => 507, gid => admin, shell => /bin/bash, home => /home/johann, managehome => true,}

user "johann" do username "johann" password "$1$P$WXmqrQEVj88fVTHevErxq." shell "/bin/bash" system true supports :manage_home => trueend

Back to your setup ...

Vagrant::Config.run do |config| config.vm.box = "natty"end

~# cat VagrantfileVagrant::Config.run do |config| config.vm.provision :puppet, :module_path => "modules" do|puppet| puppet.manifests_path = "manifests" puppet.manifest_file = "development.pp" end config.vm.define :web do |web_config| web_config.vm.box = "natty" web_config.vm.host_name = "webserver01" # web_config.vm.boot_mode = :gui web_config.vm.network "33.33.33.10" web_config.vm.forward_port "http", 80, 8080 web_config.vm.forward_port "ssh", 22, 20022 web_config.vm.share_folder "v-data", "/srv/www", "../silex-demo" endend

~# cat manifests/development.ppimport "classes/*"node "webserver01" { include web}node "dbserver01" { include db}node "ciserver01" { include ci}

~# cat manifests/classes/web.ppclass web inherits basenode { include apache include apache::php apache::vhost { silex-demo.local: port => 80, docroot => /srv/www/docroot, } package { ["mysql-client", "php5-cli", "phpunit", "php5-curl", "php5-dev", "php5-gd", "php5-imagick", "php5-mcrypt","php5-mysql", "php5-xdebug","php5-suhosin", "php-pear","php-codesniffer" ]: ensure => present, }}

~# cat manifests/classes/ci.ppclass ci inherits basenode { include apache include apache::php exec { "pear_autodiscover": command => "/usr/bin/pear config-set auto_discover 1", } package { ["pear.phpunit.de/PHP_CodeBrowser","pear.phpunit.de/PHPUnit_MockObject", "pear.phpunit.de/PHPUnit_Selenium", "pear.phpunit.de/PHP_CodeCoverage","pear.phpunit.de/PHP_Timer", "pear.phpunit.de/phpcpd","pear.phpunit.de/phploc"]: ensure => latest, provider => "pear", require => Exec["pear_autodiscover"] }}

3. Make the configuration part of your sourcecode

• application• data• docs• library• public• scripts • jobs • build • configuration • VagrantFile • manifests • modules• temp

Ok, that was a lot of work. Why did i do that again?

Collection of Wins Failsafety Simplicity Fast Setup TimeRepeatability Self-ServiceConsistency Version Management http://www.flickr.com/photos/turtlemom_nancy/

There is no golden Image anymore

There is just one directory in your source ...

„vagrant up“„vagrant provision“

Even more vagrant fun ...

Fail Safety with Vagrant-Snapvagrant snap take -d “snap1“vagrant snap listvagrant snap go “snap1“

What if i need tosimulate 20

McCloudWrapper likeVagrant around FogTransparent local & cloud usageSupports EC2, OpenStack, KVM, etc

What if i need tosimulate 2000machines?

mCollectivedssh/ssh-for-loop on steroidsfast management for loads of serversuses puppet/facter or chef/ohai, MQ- based

$ mc-package -W "architecture=x86" status apache * [ ============================================================> ] 10 / 10host01.example.com version = apache-2.2.9-7host02.example.com version = apache-2.2.9-7host03.example.com version = apache-2.2.9-7host04.example.com version = apache-2.2.9-7host05.example.com version = apache-2.2.9-7host06.example.com version = apache-2.2.9-7host07.example.com version = apache-2.2.9-7host08.example.com version = apache-2.2.9-7host09.example.com version = apache-2.2.9-7host10.example.com version = apache-2.2.9-7---- package agent summary ---- Nodes: 10 / 10 Versions: 10 * 0.25.5-1.el5 Elapsed Time: 1.03 s

The Foreman - Machine Life CycleManagementActs as a web front end for PuppetShows You the system inventoryCreates new machines and takes care of provisioning

Knowing everything about yourserver configuration is great!

Create new servers

Your (possible) Todolist

1. Install Vagrant, Veewee etc

2. Add anconﬁgurationfolder to your

3. Conﬁgure Your Vagrant images using chef or puppet

4. Setup a chef-or puppet-server using this conﬁguration

5. Move your CI, Staging andProduction-Setup to chef/puppet, too

6. Create a self-service-plattform for thedevelopment team

More cool stuff you can do:

Test-Driven infrastructure using chef/puppet-cucumber

Automatedmonitoring usingnagios or munin

Automatedreporting using graphite

Thanks!Johann-Peter Hartmannjohann__ @ freenode I hope you enjoyed it!Mail / Jabber:johann-peter.hartmann@mayflower.deFurther reading:http://www.planetdevops.nethttp://dzone.comhttp://twitter.com/#!/DEVOPS_BORAThttp://github.com/johannhartmann/

Vegetable Test DrivenInfrastructure Kontinuierliche Entiwcklung - und dann? I Mayﬂower GmbH I 28. Oktober 2010 I 87

actually a behavior driven design toolused for test driven infrastructurecucumber-puppetchef-cucumber

Feature: Manualsearch In order to find an article As an developer I want to use the search function Scenario: Search for bdd and check resulting page Given I go to "http://it-republik.de/php/" When I fill in "search_itr" with "bdd" And I click "search2" Then I should see "Suche"

johann$ cucumberFeature: Manualsearch In order to find an article As an developer I want to use the search function Scenario: Search for bdd and check resulting page # features/search.feature:5 Given I go to "http://it-republik.de/php/" # features/step_definitions/search_steps.rb:1 When I fill in "search_itr" with "bdd" # features/step_definitions/search_steps.rb:5 And I click "search2" # features/step_definitions/search_steps.rb:9 Then I should see "Suche" # features/step_definitions/search_steps.rb:131 scenario (1 passed)4 steps (4 passed)0m1.615s

Feature: Install inetd In order to serve the web the httpd service must be installed Scenario: Setup httpd Given a node of class “web“ When I compile the catalog Then package “httpd“ should be “installed“ Then there should be a resource “Service[httpd]“ And the service should have “enable“ set to “true“ And State should be “running“ And the service should require “Package[httpd]“ Kontinuierliche Entiwcklung - und dann? I Mayﬂower GmbH I 28. Oktober 2010 I 92

Hey, i understandOperations - and i‘m working in marketing

Unit TestsAcceptance TestsMetricsCoding Style... you get theidea.

... and evenmore ...PackagingInfrastructureUpdatesReleasesDeployment

WetwareRefactoring

Work together

Own together

Eat together http://www.flickr.com/photos/vilavelosa/ 3815032524/

Plantogether Kontinuierliche Entiwcklung - und dann? I Mayﬂower GmbH I 28. Oktober 2010 I 103

MayflowerDevOpsification

Wetware - Mayflower 1-2 Ops per team • Admin & Development tasks • full time working within the team

Wetware - MayflowerClose cooperation with central operationsFull root access for any development-infrastructre

Wetware - Mayflower1+n Puppet-Master • central company master • Team puppet master per team / project • company master is starting point for the project puppet configuration

Wetware - MayflowerExample Setup: • local Developer VM • CI-Deployment-Server in the DMZ • Staging in the private Cloud • Beta happens in the Amazon-Cloud • Production in AWS, too

Wetware - Mayflowerlocal GIT- / Gitorious-ServerEucalyptus-Cloud in the DMZ - as Self-Service!

Wetware - Mayflower Vagrant for Development Scrum => KanBan Puppet Nagios

http://agile.dzone.com	728
http://paper.li	21
http://a0.twimg.com	9
http://php.dzone.com	3
http://www.dzone.com	2
http://ruby.dzone.com	1

DevOps for Developers

by Mayflower GmbH on Oct 13, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

6 Embeds 764

Statistics

DevOps for Developers — Presentation Transcript