My Ad Hoc Ignite presentation for #devopsdays Kiev 2019

1. Continuous Infrastructure
First
Kris Buytaert
@krisbuytaert

2. Kris BuytaertKris Buytaert
● I used to be a Dev,I used to be a Dev,
● Then Became an OpThen Became an Op
● CTO and Open Source Consultant @CTO and Open Source Consultant @inuits.euinuits.eu
● Everything is a freaking DNS ProblemEverything is a freaking DNS Problem
● Evangelizing devopsEvangelizing devops
● Organiser of #devopsdays, #cfgmgmtcamp,Organiser of #devopsdays, #cfgmgmtcamp,
#loadays, ….#loadays, ….

3. Case 1: Chaotic OpsCase 1: Chaotic Ops
● Complete ChaosComplete Chaos
● 10% reproducability10% reproducability
● CI infra hides under a dev’s deskCI infra hides under a dev’s desk
● Ops in Debug ModeOps in Debug Mode
● No standardisationNo standardisation
● Apollo MomentApollo Moment

4. Case 1: T0+3monthsCase 1: T0+3months
● Build a Reproducable Jenkins + SlavesBuild a Reproducable Jenkins + Slaves
● CI for Puppet by OPSCI for Puppet by OPS
● Test your codeTest your code
● Promotion Stage for Infrastructure CodePromotion Stage for Infrastructure Code
● Split confg out of codeSplit confg out of code
● Keep delivering updatesKeep delivering updates

5. Case 1: T0+6 monthsCase 1: T0+6 months
● Stack AlignmentStack Alignment
•
1 jdk, 1 jboss , ...1 jdk, 1 jboss , ...
● Project Dolly :Project Dolly :
•
Puppet for everythingPuppet for everything
● 90% reproducability90% reproducability
● Standardised BuildsStandardised Builds
● Increased Test CoverageIncreased Test Coverage
● Java Developers contribute to Infra TestsJava Developers contribute to Infra Tests

6. Case 1: ConclusionCase 1: Conclusion
● Started with preparing ops folks to automateStarted with preparing ops folks to automate
● Learned the same tools developers useLearned the same tools developers use
● Developers help the ops folks to improveDevelopers help the ops folks to improve
● Collaboration + Progress ++Collaboration + Progress ++

7. Case 2: CI by DevsCase 2: CI by Devs
● Some devs have testsSome devs have tests
● Some dev teams have “CI”Some dev teams have “CI”
● Deployments are ChaosDeployments are Chaos
● Ops nags about Artifact QualityOps nags about Artifact Quality

8. Case 2: T0+18 monthsCase 2: T0+18 months
● Found the frst ops skills in the orgFound the frst ops skills in the org
● Mostly overworked Brent’sMostly overworked Brent’s
● Move them out of their ofcesMove them out of their ofces
● Focus teamFocus team
● Teach AgileTeach Agile
● Adopt IAC (puppet)Adopt IAC (puppet)
● First SuccessesFirst Successes
● Move people back to teamsMove people back to teams

9. Case 2: T0+24 monthsCase 2: T0+24 months
● Grey Beard Ops person has converted to AgileGrey Beard Ops person has converted to Agile
EvangelistEvangelist
● Preaches Kanban (for ops) and ScrumPreaches Kanban (for ops) and Scrum
● Writes Test for his CodeWrites Test for his Code
● Coaches developers to achieve CI/CDCoaches developers to achieve CI/CD

10. Case 2: ConclusionsCase 2: Conclusions
● Starting with dev delayed the collaboration forStarting with dev delayed the collaboration for
1+ year1+ year
● Ops were fre fghting and not involvedOps were fre fghting and not involved
● Once ops resources were dedicatedOnce ops resources were dedicated
collaboration and quality improvement startedcollaboration and quality improvement started
to happento happen

11. Case 4:Ops NOT involvedCase 4:Ops NOT involved
● Large TransformationLarge Transformation
● ““devops” team dictates tools they have neverdevops” team dictates tools they have never
used them selvesused them selves
● Tools they as a team don’t need themselvesTools they as a team don’t need themselves
● Developers complain about unusable toolsDevelopers complain about unusable tools
● Developers complain about broken toolsDevelopers complain about broken tools
● Tools enforce a manual processTools enforce a manual process

12. Case 4: 2 years laterCase 4: 2 years later
● Average “devops” role stays for 2 months ,Average “devops” role stays for 2 months ,
then leavesthen leaves
● Senior IT management has left (2x)Senior IT management has left (2x)
● Only In house analysts remainOnly In house analysts remain
● Mostly contract based developersMostly contract based developers
● Failing Cloud StrategyFailing Cloud Strategy
● Legacy Container EcosystemLegacy Container Ecosystem

13. A CI EcosystemA CI Ecosystem
● Version ControlVersion Control
● DeploymentDeployment
● Build ToolingBuild Tooling
● Artifcact RepositoryArtifcact Repository
● Code Coverage ToolingCode Coverage Tooling
● Testing ToTesting To
How many of those tools is your average opsHow many of those tools is your average ops
person used to use ?person used to use ?

14. Understanding CIUnderstanding CI
● Understanding Dependency Management,Understanding Dependency Management,
Testing, Packaging , Deploying =>Testing, Packaging , Deploying =>
•
Same LanguageSame Language
•
Same ToolsSame Tools
•
Same PainSame Pain

15. Typical Dev FirstTypical Dev First
● Dev lacks environments or uses wrong stackDev lacks environments or uses wrong stack
● Dev can’t deploy yetDev can’t deploy yet
● No PipelinesNo Pipelines
•
Technical debt is createdTechnical debt is created
● Ops is catching up with envsOps is catching up with envs
● NFR get ignoredNFR get ignored
● Brent stays a heroBrent stays a hero

16. Why ops frst ?Why ops frst ?
● You can’t support / understand what you don’tYou can’t support / understand what you don’t
do yourselve.do yourselve.
● Code = CodeCode = Code
● Unblock deliveryUnblock delivery
● Unblock provisioningUnblock provisioning
● Metrics & Monitoring Build inMetrics & Monitoring Build in

17. Dev OoopsDev Ooops #container edition#container edition
● ““Put this Code Live, here's a DockerPut this Code Live, here's a Docker
Image ”Image ”
● No machines available ?No machines available ?
● What database ? Where to store theWhat database ? Where to store the
data ?data ?
● Security ? What distro is this even ?Security ? What distro is this even ?
Bad Cows ?Bad Cows ?
● How do we monitor his ?How do we monitor his ?
● Backups ?Backups ?
● How did you build this ?How did you build this ?
● Has DockerHub been hacked again ?Has DockerHub been hacked again ?

18. Culture vs ToolsCulture vs Tools

19. Culture Hack:Culture Hack:
Set up CI/CD for your CI/CD infrastructureSet up CI/CD for your CI/CD infrastructure
frst, If the people running your infra don'tfrst, If the people running your infra don't
know how CI/CD works , how do youknow how CI/CD works , how do you
expect them to support / teach yourexpect them to support / teach your
application teams ?application teams ?

20. Devopsdays Gent 2019Devopsdays Gent 2019
29-30 october 201929-30 october 2019
Vooruit GhentVooruit Ghent
https://devopsdays.org/https://devopsdays.org/
events/2019-ghent/events/2019-ghent/
Early Bird = OPENEarly Bird = OPEN
CFP = OPENCFP = OPEN