This document discusses different approaches to implementing continuous infrastructure and collaboration between development and operations teams. It argues that starting with operations automating their own workflows first allows them to understand the tools and processes before enforcing them on developers, prevents technical debt, and enables delivery and provisioning from the start. Having dedicated operations resources focus on automation leads to improved collaboration and quality over having "devops" teams dictate tools to operations.

1. Continuous Infrastructure
First
Kris Buytaert
@krisbuytaert

2. Kris BuytaertKris Buytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● CTO and Open Source Consultant @CTO and Open Source Consultant @inuits.euinuits.eu
● Everything is a freaking DNS ProblemEverything is a freaking DNS Problem
● Evangelizing devopsEvangelizing devops
● Organiser of #devopsdays, #cfgmgmtcamp,Organiser of #devopsdays, #cfgmgmtcamp,
#loadays, ….#loadays, ….

3. Case 1: Chaotic OpsCase 1: Chaotic Ops
● Complete ChaosComplete Chaos
● 10% reproducability10% reproducability
● CI infra hides under a dev’s deskCI infra hides under a dev’s desk
● Ops in Debug ModeOps in Debug Mode
● No standardisationNo standardisation
● Apollo MomentApollo Moment

4. Case 1: T0+3monthsCase 1: T0+3months
● Build a Reproducable Jenkins + SlavesBuild a Reproducable Jenkins + Slaves
● CI for Puppet by OPSCI for Puppet by OPS
● Test your codeTest your code
● Promotion Stage for Infrastructure CodePromotion Stage for Infrastructure Code
● Split confg out of codeSplit confg out of code
● Keep delivering updatesKeep delivering updates

5. Case 1: T0+6 monthsCase 1: T0+6 months
● Stack AlignmentStack Alignment
•
1 jdk, 1 jboss , ...1 jdk, 1 jboss , ...
● Project Dolly :Project Dolly :
•
Puppet for everythingPuppet for everything
● 90% reproducability90% reproducability
● Standardised BuildsStandardised Builds
● Increased Test CoverageIncreased Test Coverage
● Java Developers contribute to Infra TestsJava Developers contribute to Infra Tests

6. Case 1: ConclusionCase 1: Conclusion
● Started with preparing ops folks to automateStarted with preparing ops folks to automate
● Learned the same tools developers useLearned the same tools developers use
● Developers help the ops folks to improveDevelopers help the ops folks to improve
● Collaboration + Progress ++Collaboration + Progress ++

7. Case 2: CI by DevsCase 2: CI by Devs
● Some devs have testsSome devs have tests
● Some dev teams have “CI”Some dev teams have “CI”
● Deployments are ChaosDeployments are Chaos
● Ops nags about Artifact QualityOps nags about Artifact Quality

8. Case 2: T0+18 monthsCase 2: T0+18 months
● Found the frst ops skills in the orgFound the frst ops skills in the org
● Mostly overworked Brent’sMostly overworked Brent’s
● Move them out of their ofcesMove them out of their ofces
● Focus teamFocus team
● Teach AgileTeach Agile
● Adopt IAC (puppet)Adopt IAC (puppet)
● First SuccessesFirst Successes
● Move people back to teamsMove people back to teams

9. Case 2: T0+24 monthsCase 2: T0+24 months
● Grey Beard Ops person has converted to AgileGrey Beard Ops person has converted to Agile
EvangelistEvangelist
● Preaches Kanban (for ops) and ScrumPreaches Kanban (for ops) and Scrum
● Writes Test for his CodeWrites Test for his Code
● Coaches developers to achieve CI/CDCoaches developers to achieve CI/CD

10. Case 2: ConclusionsCase 2: Conclusions
● Starting with dev delayed the collaboration forStarting with dev delayed the collaboration for
1+ year1+ year
● Ops were fre fghting and not involvedOps were fre fghting and not involved
● Once ops resources were dedicatedOnce ops resources were dedicated
collaboration and quality improvement startedcollaboration and quality improvement started
to happento happen

11. Case 4:Ops NOT involvedCase 4:Ops NOT involved
● Large TransformationLarge Transformation
● ““devops” team dictates tools they have neverdevops” team dictates tools they have never
used them selvesused them selves
● Tools they as a team don’t need themselvesTools they as a team don’t need themselves
● Developers complain about unusable toolsDevelopers complain about unusable tools
● Developers complain about broken toolsDevelopers complain about broken tools
● Tools enforce a manual processTools enforce a manual process

12. Case 4: 2 years laterCase 4: 2 years later
● Average “devops” role stays for 2 months ,Average “devops” role stays for 2 months ,
then leavesthen leaves
● Senior IT management has left (2x)Senior IT management has left (2x)
● Only In house analysts remainOnly In house analysts remain
● Mostly contract based developersMostly contract based developers
● Failing Cloud StrategyFailing Cloud Strategy
● Legacy Container EcosystemLegacy Container Ecosystem

13. A CI EcosystemA CI Ecosystem
● Version ControlVersion Control
● DeploymentDeployment
● Build ToolingBuild Tooling
● Artifcact RepositoryArtifcact Repository
● Code Coverage ToolingCode Coverage Tooling
● Testing ToTesting To
How many of those tools is your average opsHow many of those tools is your average ops
person used to use ?person used to use ?

14. Understanding CIUnderstanding CI
● Understanding Dependency Management,Understanding Dependency Management,
Testing, Packaging , Deploying =>Testing, Packaging , Deploying =>
•
Same LanguageSame Language
•
Same ToolsSame Tools
•
Same PainSame Pain

15. Typical Dev FirstTypical Dev First
● Dev lacks environments or uses wrong stackDev lacks environments or uses wrong stack
● Dev can’t deploy yetDev can’t deploy yet
● No PipelinesNo Pipelines
•
Technical debt is createdTechnical debt is created
● Ops is catching up with envsOps is catching up with envs
● NFR get ignoredNFR get ignored
● Brent stays a heroBrent stays a hero

16. Why ops frst ?Why ops frst ?
● You can’t support / understand what you don’tYou can’t support / understand what you don’t
do yourselve.do yourselve.
● Code = CodeCode = Code
● Unblock deliveryUnblock delivery
● Unblock provisioningUnblock provisioning
● Metrics & Monitoring Build inMetrics & Monitoring Build in

17. Dev OoopsDev Ooops #container edition#container edition
● ““Put this Code Live, here's a DockerPut this Code Live, here's a Docker
Image ”Image ”
● No machines available ?No machines available ?
● What database ? Where to store theWhat database ? Where to store the
data ?data ?
● Security ? What distro is this even ?Security ? What distro is this even ?
Bad Cows ?Bad Cows ?
● How do we monitor his ?How do we monitor his ?
● Backups ?Backups ?
● How did you build this ?How did you build this ?
● Has DockerHub been hacked again ?Has DockerHub been hacked again ?

18. Culture vs ToolsCulture vs Tools

19. Culture Hack:Culture Hack:
Set up CI/CD for your CI/CD infrastructureSet up CI/CD for your CI/CD infrastructure
frst, If the people running your infra don'tfrst, If the people running your infra don't
know how CI/CD works , how do youknow how CI/CD works , how do you
expect them to support / teach yourexpect them to support / teach your
application teams ?application teams ?

20. Devopsdays Gent 2019Devopsdays Gent 2019
29-30 october 201929-30 october 2019
Vooruit GhentVooruit Ghent
https://devopsdays.org/https://devopsdays.org/
events/2019-ghent/events/2019-ghent/
Early Bird = OPENEarly Bird = OPEN
CFP = OPENCFP = OPEN