Downloaded 26 times
Uploaded byJulian Dunn
Configuration Management Isn't Everything
This document discusses configuration management (CM) projects and what leads to their success and failure. It notes that while CM is important, revolutionizing IT requires more than just CM. Successful CM projects have realistic expectations, dedicate resources to own the CM process, conduct candid assessments of current processes, make incremental changes, and plan in advance. They focus on people, process, and products rather than just tools. Unsuccessful projects have unrealistic expectations, rely too much on consultants, take a big bang approach, and lack advance planning. The key is to focus on CM as a long-term investment that improves team collaboration and experimentation.
More Related Content
Similar to Configuration Management Isn't Everything
Configuration Management Isn't Everything
- 1. Configuration Management Isn’t Everything JulianDunn Senior Consultant, Chef Software, Inc. jdunn@getchef.com
- 2. What Cred DoI Have? • 15 years experience in IT • Consulting Engineer at Chef • “Consultants are called when things are really screwed up”
- 3.
- 4. Revolutionizing IT ina company takes a lot more than just using configuration management. configuration management.
- 5.
- 6. Executives: Speed isNew Currency
- 7. Executives / Managers •“It takes forever to do anything around here” • “Our site/apps are down too often” • “Why can’t we be like Amazon.com?” • “I have an iPad with all these apps”
- 8. System Administrators /Engineers • Configuration drift leading to failures/outages • Handcrafted systems with unknown state • Janky & error-prone one-off scripts • Developers spend too much time “setting up environment” • Constant firefighting and reactivity
- 9. Commonalities • Frustration withspeed of IT • Frustration with bureaucracy • Poor working conditions for staff • Along comes automation...
- 11. The Real EndGoal • IT velocity • IT as a core competency • Successful companies will be IT companies
- 12. When Do CMProjects Fail? When Do They Succeed? When Do They Succeed?
- 13. Failure: Unrealistic Expectations •Timelines • Effort • Scope of change • “Magic”
- 14. “They see thedemo, don't understand what was done, and think it shits miracles.”
- 15. Success: Correct Expectations •Investment • People • Code • Time • Open to changing almost anything
- 16. Fail: Not dedicatingresources • “DevOps Team” • Reliance on consultants • Reliance on contractors • Not their “real job”
- 17. Success: Own theCM • Engineers with domain expertise writing code • Part of their “real job” • Own the changes
- 18. “Why do wedo things this way?”
- 19. Success: Candid ProcessAssessment • Value-stream mapping • Origins of ITIL & safety gates • Loosen controls in proportion to consistency guarantees
- 20. Fail: Big BangApproach • Hardest problem • Biggest problem • All at once
- 21. Success: Incremental Change •Pick a small project • Make change incrementally • Choose a route that will minimize likelihood of failure
- 22. Fail: Lack ofAdvance Planning • You can’t automate what you don’t understand • What do you even have?
- 23. Success: Advance Planning •Spend time up front building the model • Writing CM code on day 1 is ineffective • Workflow is as important as the work
- 24. Fail: Doing ItBy Force • Arbitrary deadlines with no business reason • Panic • “Failure is not an option”
- 25. Failure is alwaysan option.
- 26. “When bad thingshappened, we just calmly laid out all the options, and failure was not one of them. We never panicked, and we never gave up on finding a solution.” - Gene Krantz
- 27.
- 28. All that said... •Certain tools are complementary with CM • Primary: Tools that improve team communication, collaboration and experimentation • Secondary: Tools that complement CM’s consistency guarantees
- 29. Source Control • Whyis Git so popular? • Easy experimentation • Full control offline • Collaboration & communication • Use whatever source control system lets you have these features
- 31. Artifact Consistency • Artifactserver • Consistency • Reproducibility • Immutability • Complementary to CM system
- 32. Control Flow ofVendor Patches • “Artifact server” for patches coming from upstream vendor • RedHat Satellite • Spacewalk/Katello • WSUS • Ubuntu Landscape • others?
- 33.
- 34. The Three P’s •People • Process • Product
- 35. People • Own theskills • Develop internal resources • Jez Humble: “Stop Hiring ‘Devops Experts’ And Start Growing Them”
- 36. Process • Using aCM tool to capture bad process doesn’t get you very far • Understand current value-stream map
- 37. Product • CM systemis the “star” • “Supporting cast”
- 39. Thank You! • JezHumble: http://www.slideshare.net/jezhumble/stop-hiring-devops-e • Mark Imbriaco, ChatOps: https://www.youtube.com/watch?v=pCVvYCjvoZI
Editor's Notes
- #4 This talk started as a conversation between a fellow consultant and I. Often we go into shops and they think DevOps (or Chef, whatever) is something you can just order from Amazon.com or NewEgg and suddenly your business will be accelerated. Maybe this is Stockholm Syndrome from years of enterprise software sales reps telling them “just buy our software, we will solve XYZ for you”
- #5 - Revolutionizing IT in an org is gonna take a lot more than just buying/implementing a CM product. - It’s amazing how many leaders don’t get this. But they have the purchasing authority.
- #9 Janky scripts - “There’s nothing more permanent in IT than a temporary solution”
- #10 Speed Bureaucracy Poor working conditions - Usually this is at the SA level, demoralized because they feel like they’re just line cooks, and business is always breathing down their neck even though they’re engaging in heroics to keep the systems alive
- #11 “Automate all the things” is not a useful goal for either executives or sysadmins. You could replace “automate all the things” with “cloudify all the things” or “devops all the things” or any other buzzword, it still doesn’t make for a sensible objective. Automating for the sake of automating is pointless. Automation is the medicine for your headache; have you actually solved why you have a headache in the first place?
- #12 How many of you have read the Phoenix Project? How many of you could relate to it? I’m going to give away the ending a little bit here, but you should still read it, if for nothing else to know that you aren’t alone in your experiences in IT. Able to keep up with speed of biz, responsive etc. Owning your IT and being in control of it will let you leapfrog over competitors that don’t have the same competencies. Doesn’t matter whether you are Tesco or Deutsche Bank. Those of us in IT often say “we’d like to improve communications between the biz & IT” because we just get unrealistic orders dumped on our lap. Well, here’s how you do it -- when IT is part of the biz, it’s a lot harder for the business to treat you as a cost centre or short order cook. If you’re a pharmaceutical company, for example, you’d never ask your scientists to run a clinical trial & ship a drug in a month. You have some idea how long it takes and have clearer expectations because that’s your business.
- #14 Timelines and Effort: * “We want to move everything to the cloud in the next two quarters” -- well think about how long it took to build that stuff in the first place. And how do you know you can move everything? What about your EOL AIX servers? Your AS/400s? * Unprepared for the amount of change required to fully automate. Scope of change: I say “one foot on the dock and one foot on the boat” -- not 100% committed to change. And if you ride a bike too slowly it will fall over. For example, the client agrees that automation is great, but they later freak out when they realize they have to give up direct control over things. I’ve seen customers flip out about not making manual changes on the server or revoking ssh or sudo access to admins -- all the things are done through Chef. Well, are you buying into the consistency & reproducibility guarantees of Chef or not? Existing processes that they want to shoehorn CM into. “How can we implement Chef within the confines of our existing ITIL process?” Security team freaks out and doesn't want anything running as root. Client doesn't even own their own hardware, therefore making changes is very painful when they have to depend on a third party. Have you had those conversations with the MSP? These are financial and contractual implications that customers don’t think about but stand in the way of a successful implementation. And just to touch on “magic”, sometimes that’s our fault as salespeople -- or in your organization, if you’re “selling” CM up to management. Next slide:
- #15 I said I wasn’t going to swear in this presentation... sorry. Paraphrasing a wise person who may or may not be in this room...
- #17 Jez Humble: http://continuousdelivery.com/2012/10/theres-no-such-thing-as-a-devops-team/ Some customer engagements go badly because staff are pulled off to go do “real work” -- guess what, the CM system *is* the real work from now on.
- #18 Own the changes - what happens if I come in, “eat shoot and leave?” Next time you need to make a change you’ll be lost. “Hire goats and give them rope” -- Michael D.
- #19 How many of you are cat people? This story will be familiar to those of you who own cats. Once upon a time in a certain village in India there lived a guru. Every evening the guru would sit on his seat and deliver a lecture to the public. It so happened that the guru had a cat, and just at the time of giving the lecture the cat would create a big disturbance. Being greatly annoyed by the cat, the guru decided to tie the cat to a tree before starting his lecture. So doing, the guru then delivered the lecture without disturbance. It worked so well that the guru regularly tied the cat to the tree before beginning his discourse. After some years the guru died. His disciples carried on the guru’s program. They also continued tying the cat to the tree. When the cat died, they bought another cat and thus the ritual of tying a cat to a tree continued generation after generation. In the fifth generation that followed the guru, one of the renowned followers wrote an elaborate treatise on the spiritual significance of tying a cat to a tree before beginning one’s studies of the scriptures. It's tradition!
- #20 If you analyze why ITIL, CABs, etc. and other layers of bureaucracy exist, it’s often to “guarantee” safety in IT systems because we had no other way of doing so. So managers often want
- #21 Big bang with process change is sure to get people’s backs up.
- #22 Don’t pick the biggest, most gnarliest project up frontMake change incrementally -- no “big bang” implementation. It is a lifestyle changeChoose a route that will minimize likelihood of failure -- analogize to high-speed rail projects in America Tell the story of high speed rail in North America and why it’s so bad compared to Europe & Japan
- #23 MongoDB or document-oriented databases -- “schemaless” doesn’t mean “don’t plan the schema ahead of time”
- #24 Model is not only a diagram/inventory of what you have and roughly how it was built, but what data structures, cookbooks, etc. you are going to use and have for your CM tool. Do forensics on your running infra, if necessary -- that’s why we often suggest to customers that we do an “infrastructure assessment” before embarking on a CM project. Developing a model on the fly is harder than having a baseline to start, even if the baseline changes down the road. Workflow’s importance: what I learned from working in the media, where reporters had the most screwed-up workflows and it inhibited them from producing great content as a result.
- #26 You can’t do automation (or any IT projects) by force -- dead-drop dates don’t work. You may just end up launching garbage and end up fixing them incrementally in the worst possible way: publicly. Also, Gene Kranz *never* said this IRL. That was made up by the Apollo 13 writers… what he did say was “When bad things happened, we just calmly laid out all the options, and failure was not one of them. We never panicked, and we never gave up on finding a solution.”
- #27 Again, why you don’t choose the riskiest, most gnarly project to start with. Have buffer room for failure & experimentation. Not failure of the entire project, but room to experiment & not build up too much tech debt.
- #32 Easy experimentation -- branching and merging ops can be done completely offline, and you can rewrite history & rebase before publishing Collaboration & collaboration were popularized by GitHub and you can either do that, or BitBucket, or Gitlab, or a myriad of others If Mercurial, Stash (Git server variant), Perforce, etc. let you do these things, then use them if they make you comfortable
- #33 Many companies implement peer-to-peer chat systemsLync, Skype, GChat, etc.What’s lacking is group chatHipChat, Campfire, IRCWhy group chat? (list examples here, link to Mark Imbriaco’s GitHub ChatOps talk) - Need to make the case for why
- #34 Publish your builds to an artifact server* Enforce policies on the artifact server (immutable objects, who can publish builds, etc.)* Mirror external dependencies there (RubyGems, Java libraries, etc.)Objective: every time you install a system or even “mvn compile”, you get the same thing* Examples: Artifactory, Sonatype Nexus
- #35 WSUS - Windows Server Update Services
- #37 Maybe here is where we say “just add water” doesn’t work Tell them, not gonna sell you on DevOps culture or anything, that’s not what this conference is about “Every presentation must have a pie chart, so I have one” -- Sean Carolan
- #40 “Reinforce culture with technology” - Adam
- #41 CM is not a magic bullet for all your IT problems. But many companies think that it is.