Devops: The War is over - If you want it

DEV OPSPatrick Debois http://jedi.be/blog

When I was a young developer http://www.ﬂickr.com/photos/ninahiironniemi/1118868590/sizes/l/

I loved to develop simple applicationshttp://1000awesomethings.ﬁles.wordpress.com/2008/11/campﬁre.jpg

I kept my own little server going http://www.rookgordijn.net/wp-content/uploads/2009/08/Texel_21.JPG

Most of the time , the results were ﬁne http://www.foodgeekery.com/fullimg/ribfest09-cowboy2.jpg

Sometimes the applications would crash and I didn’t really know why http://www.ﬂickr.com/photos/abbyladybug/2644163076/sizes/l/

But overall my friends seemed to like my work http://www.motoer75.nl/images/BBQ2009030.jpg

I learned from the pro’shttp://www.bfeedme.com/wp-content/uploads/2007/07/legends-of-texas-bbq-cookbook-of-the-day-7-30-07.jpg

Read about softwarearchitecturehttp://www.freepatentsonline.com/6557545-0-large.jpg

I even got my Scrum certiﬁcation http://patiopalace.com/assets/images/gas-grills/webber/certiﬁedexperts3.jpg

Still I was experiencing the occasional problems http://www.ﬂickr.com/photos/tradica/3071394619/sizes/l/

I blamed theoperations guys

I decided to leave the cowboys http://media.photobucket.com/image/bbq%20cowboy/NotleyQue/The%20BBQ%20Family/CowboyChili.jpg

And make plans of my own http://www.ﬂickr.com/photos/iimaj/3959147070/sizes/o/

I really wanted to focus on the customer http://www.ﬂickr.com/photos/brachiator/107510145/sizes/l/

I had the developers usethe coolest programming languages http://www.dongeovanny.com/images/meatpage2kobe.jpg

Use the best frameworks availablehttp://www.bbqsaucereviews.com/wp-content/uploads/2009/06/bbqsaucereview-001.jpg

We had theDBA’s takecare of the database schemas http://jeffreysmeatmarketblog.ﬁles.wordpress.com/2009/06/picture-4-3-0412.jpg

Explained them that User Interface is importanthttp://www.ﬂickr.com/photos/bhamsandwich/3601018752/sizes/l/ http://www.ﬂickr.com/photos/jayneandd/4078134280/sizes/l/

Up until now we had done only some manual checks. http://lovefeasttable.com/blog/wp-content/uploads/2009/05/may09c067.jpg

So we moved to unit tests http://www.ﬂickr.com/photos/cookthink/210948100/sizes/l/

Even Behavior Driven Development http://blogs.suntimes.com/food/4-29_white_BBQ_1.jpg

We were stillexperiencing problems with our application servers.

We brought in bigger storage boxeshttp://www.ﬂickr.com/photos/80417459@N00/2056898218/

gave thesysadmins thebiggest servers http://euroross.blogspot.com/bbq.jpg

created virtual machines tocreate different environments http://propanesmoker.org/wp-content/uploads/2009/11/Propane_smoker.jpg

Used the cloud to solve some scalability problems http://www.bbqpits.com/people_of_bbq/greg_nj3.jpg

Still the customer wasn’t happy http://www.ﬂickr.com/photos/theogeo/3317291208/sizes/l/in/photostream/

And the IT department was firefighting all the time http://www.funnypictures.net.au/images/firefighter-bbq1.jpg

Then there was a complete power failure http://www.ﬂickr.com/photos/atesca/2457127172/sizes/l/

Nobody had checked the power installation http://www.ﬂickr.com/photos/zanyﬂash/2278286656/sizes/o/

Leaving the application in an unknown state http://www.ﬂickr.com/photos/cirox/363284194/sizes/l/

Resulting again in a major outage http://badgerdiary.ﬁles.wordpress.com/2008/12/closed-sign-smer.jpg

Our small ﬁxes would not help anymore http://www.nonplusultra.nl/data/producten/12727-groot.jpg

All backups were corrupt because we could only backup properly when the application was down http://www.proexotics.com/faq_images/soaking_water_monitor.jpg

http://www.ﬂickr.com/photos/jmv/16366498/sizes/o/ Luckily we had a cold Standby Backup

kr.com/photos/tom-poes/60373623/ Time to take a step back

We worked hard onminimizing technical debt in our code http://allwecaneat.ﬁles.wordpress.com/2008/08/_igp7826_sm.jpg

We started to improve our deployment tools http://www.eggcookers.net/images/pictures/progressive-stainless-steel-bbq-tongs-bbq505.jpg http://ecx.images-amazon.com/images/I/41wSaS%2BMy8L.jpg

Put monitoring inour development and test environment http://www.ﬂickr.com/photos/wiphey/66525468/sizes/o/

Reused it in the production environment http://3.bp.blogspot.com/_ySMs_t20v08/Sw8m5JjVg1I/AAAAAAAAAFQ/jr2nXJYSfbI/s1600/DSC_4835.jpg

Implemented scaling from the ground up http://weblog.greenpeace.org/makingwaves/welldone430.jpg

We made deployment reproducible http://www.universalbarbeque.com/images/untitled-2_copy.jpg

Improved the way to collect metricshttp://www.ﬂickr.com/photos/cjscott69/400160562/sizes/l/

To ofﬂoad some of the appservers we installed a caching system http://www.ﬂickr.com/photos/namealus/3114627540/

And loadbalancers solved the webserver scaling problem http://www.ﬂickr.com/photos/benchilada/4231838748/sizes/l/

Content Delivery Networks http://www.izzysburgerspa.com/images/uploaded/image/Delivery%20Truck%201.jpg

Made our application security proof http://strongdogz.com/wp-content/uploads/2008/09/beauceron.jpg

Comply toregulations http://www.ﬂickr.com/photos/loop_oh/3086747855/sizes/o/

Integrated backups on the application level http://www.mzmfarm.net/linked/img_3507.jpg

Solved the multi-user problem problem http://carrotsncake.com/wp-content/uploads/2009/03/img_6705.jpg

Self healing on our databaseFor some know We would automatically states reset the system http://assets.whirlpool.com/ﬁles/usr/970/FilterReset5.jpg

So you want to be a great agile chef?

#1 care for the whole lifecycle #2 look at it from a business perspective #3 work together as a team PresentationSoftware InfrastructureProject Operations Testing http://www.damniwish.com/wp-content/uploads/2009/12/IMG_1265.jpg

It’s not about the It’s not about the software hardware

I like my servers, software steak well donehttp://www.ﬂickr.com/photos/spenceke/2250116189/sizes/l/

IT Tooling is changing

Virtualization • kvm • xen • solaris zones • vmware • virtualbox • ...

Cloud showedus how to scale

Abstraction APIhttp://libvirt.org/ http://www.jclouds.org/ Adrian Cole @jclouds

Platform as a Service

Hudson as Service http://www.cloudbees.com/

But who will manage your IAAS platform?

First steps of Automation http://fai-project.org/https://fedorahosted.org/cobbler/ http://spacewalk.redhat.com/

Cloning

Virtual Sprawl

Conﬁguration Management Chefhttp://www.cfengine.org/ http://www.opscode.com/chef https://github.com/ hugoduncan/pallet Pallethttp://www.puppetlabs.com

Infrastructure as code# tomcat.ppclass tomcat {! package {! ! "java-1.6.0-sun-compat":! ! ! ensure => "installed";! ! "tomcat6":! ! ! require => [Package["java-1.6.0-sun-compat"], Package["ntc-tomcat-log4j"]],! ! ! ensure => "installed";! ! "tomcat6-admin-webapps":! ! ! ensure => "installed";! ! "log4j":! ! ! ensure => "installed";! ! "tanukiwrapper":! ! ! ensure => "installed";! ! "mysql-connector-java":! ! ! ensure => "installed";! }! file {! ! "/etc/init.d/tomcat6": mode => "0755", owner => "root", group => "root", source => "puppet:///tomcat/tomcat6.init"; }! user {! ! "tomcat":! ! ! ensure => "present",! ! ! require => Package["tomcat6"];! }}!

Model Driven Infrastructure

Deﬁnition of your infrastructure John E.Vincent @lusishttps://github.com/lusis/vogeler

Version Control

Reusable componentshttps://github.com/camptocamp

Code/Server Smells Private!Playground The sysadmin uses the system as his toy playground, doesnt clean up. • /tmp & /var/tmp full of old install files • / full of files Gready!Server One server that does every function • combined mail and web and dns and fileshares • all users on the same system Root is the cause of all!evil • last show login all root • no sudo is activated • no sshd keys for logins • nfs share/root? • Chmod 777 • most processes run as root

Sharing Recipes/Code http://cookbooks.opscode.com/http://forge.puppetlabs.com/

Automationwithout tests is

cucumber-nagiosFeature: google.com It should be up And I should be able to search for things Scenario: Searching for things When I visit "http://www.google.com" And I fill in "q" with "wikipedia" And I press "Google Search" Then I should see "www.wikipedia.org"$ cucumber-nagios google.featureCritical: 0, Warning: 0, 4 okay | passed=4, failed=0 Lindsay Holmwood @auxesis

cucumber-apacheapache.featureFeature: In order to deploy my application As a systems administrator I want to know that the config files are legal Scenario: Valid Apache config Given a config file template apache2.conf in etc/apache2 When I generate it Then there should be a file called apache2.conf in etc/ap And it should be valid Julian Simpsonhttp://pastie.org/693713 @builddoctor

cucumber-puppetFeature: cucumber-puppet In order to run my puppet manifests test suite As an admin I want the cucumber-puppet gem installed Scenario: Install cucumber-puppet Given a node of class "cucumber-puppet" When I compile the catalog Then gem "cucumber-puppet" should be "installed" Nikolay Sturmhttps://github.com/nistude/cucumber-puppet @nistude

Orchestrationhttp://controltier.com/ http://marionett-collective.org @damonedwards @ripienaar

Vagrant Allows an easy way to use the production recipes in your own dev environmenthttp://vagrantup.com/ Mitchell Hashimoto @mitchellh

IAAS and PAAS blurringPull in both your recipes and your code

Agile Infrastructure & Agile Sysadminhttp://blog.crisp.se/henrikkniberg/2008/07/14/1216002720000.html

Comparing Agile and ITILAgile ITIL http://www.winstonind.com/images/brands/traditional_cooking_process.gif

Certiﬁcations

ProcessFundamentalism

Manifesto’s for the world Meat Cloud Manifesto == Rules == On Infrastructure —————– There is one system, not a collection of systems.Agile Manifesto The desired state of the system should be a known quantity. The “known quantity” must be machine parseable. The actual state of the system must self-correct to the desired state. The only authoritative source for the actual state of the system is the system The entire system must be deployable using source media and text ﬁles. On Buying Software ——————- ..... James White Manifesto

Truck Factor 10th ﬂoor The number of people on your team test The ability to take a random machine in your infrastructure who have to be hit with a truck before , drop it from the 10th ﬂoor, the project is in serious trouble and be able to restore your infrastructure to working within 5-10 minu

Backlog of items• User Stories • Tickets• Features • Incidents • Problems

Requirements Non-Functional Functional • Security• Login/Logout • Capacity• Print order • Stability• CRUD order • Scalability

Business Value Ordered by Ordered by the valuethe value gained protectedIncrease Change Avoid Change

Understand what the business wants Proxy ServiceCustomers Manager

Support the team Service DeliveryScrum Manager Manager

Ideally one product ownerProduct Product Product Project1 Project 2 Project 3Owner1 Owner2 Owner3 Scrum Master Operations Manager Development Team Operations Team

OutsourcingOutsourced Call CentersDevelopment

Specialist & Generalists Rockstar Mentality Analysts SecurityTesters UI Designers Network Frontend Sysadmins Storage Backend

Global Ownership It’s the It’s thejavascript network It’s the It’s the backend server It’s the It’s the database storage

Sprint Review Change Advisory Meeting Board Approve Changes

Iterations & Flow

Kanban

1day sprintsOperations

Sprint ServiceVelocity Levels Story Estimate Uptime 99% Points Improves over time Team Approved

Continuous Improvement Lean Thinking Lean Software Lean ITILDevelopment

Value Stream Mapping

Technical waste

I know all thatWhat’s new?

The map is notthe territory Alfred Korzybski

Optimize the whole notjust the individual parts Systems Thinking

People make it happen

Culture & Mindset

Collaborationnot only when things go wrong Sysadmins Testers Analysts SecurityDevelopers Network UI Architects Designers

No rock star mentality Mine are bigger then yours

Craftsmanship

It’s about customer valuehttp://www.ﬂickr.com/photos/spenceke/2250116189/sizes/l/

Interesting Books

Want to know more? #devops on twitter & irc http://devopsdays.org http://devopscafe.org http://planetdevops.net devops googlegroupdevops-toolchain googlegroup

The End

http://www.jedi.be	455
http://agile.dzone.com	303
http://clujengineeringblog	243
http://www.planetdevops.net	108
http://java.dzone.com	82
http://jedi.be	53
http://www.linkedin.com	33
http://lasterday.posterous.com	21
http://www.aninki.net	19
http://paper.li	13
http://laz.jedi.be	10
http://www.scalebig.com	7
http://server.rdesign.nl	5
http://www.slideshare.net	5
http://server.dzone.com	3
http://asieholden.wordpress.com	3
http://edicolaeuropea.blogspot.com	2
http://a0.twimg.com	2
http://dotnet.dzone.com	2
file://	2
url_unknown	1
https://pramati.qontext.com	1
http://www.techgig.com	1
http://fasoulas.posterous.com	1
http://static.slidesharecdn.com	1
http://vbsuite.dev.thesite.org.uk	1
http://beta.scalebig.com	1
http://bset.royans.net	1
http://www.jobnet.nl	1
http://www.hackzq8search.appspot.com	1
http://translate.googleusercontent.com	1
https://g-proxy.appspot.com	1
http://10.111.3.12	1

Devops: The War is over - If you want it

by Patrick Debois on Nov 25, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

33 Embeds 1,384

Statistics

Devops: The War is over - If you want it — Presentation Transcript