2. Eric’s Bio
I’m a Lead Consultant at Urbancode
where I helps customers get the
most out of their build, deploy and
release processes. I have 9 years of
automation experience throughout
Eric Minick the application life-cycle in roles as
eric@urbancode.com
@EricMinick a developer, test automation
engineer, and support engineer. I’ve
been at the forefront of CI & CD for
7+ years
2
4. Why do we accumulate technical debt?
• We leverage technical
debt to deliver more
faster.
Scope Time
• De-leveraging is rarely
accounted for in project
planning. Resources
• Green-Shifting*
4 * http://www.drdobbs.com/191600661
5. Why should we care?
Baggage that slows the team Quality issues
• Lack of automated tests • Lack of tests results in
lengthen QA cycles buggier code
• Fear of merging work
• Releases are error prone
and lead to unnecessary
• Unrefactored code slow to
work with outages
• Slow build / deploy processes
delay learning and release
pace
5
6. The limits of what we know
• Known Knowns: Bugs confirmed and tracked
• Known Unknowns: Undiscovered bugs
• Unknown unknowns: One of our project
teams is using a GPL’d library making their
product impossible to ship
6
7. Where Automation Helps
• “Testing” for debt: Automated tests, code
scans and reports can help identify (and
quantify) problems.
• Automation as a learning experience: The act
of automating brings surprises.
7
9. Testing for Debt: Continuous Integration
• On code commit, build and test the software
• Roll up changes to build-time or runtime
dependencies and test those too to identify
API incompatibilities
9
10. Testing for Debt: Code Inspection
Code Reviews – Rapidly detect issues
Static Code Analysis – Tool based checks for
bugs, code duplication, code theft, & non-
compliance with dev standards.
10
11. Testing for Debt: Code Inspection
Code Reviews – Rapidly detect issues
Static Code Analysis – Tool based checks for
bugs, code duplication, code theft, & non-
compliance with dev standards.
11
14. An example safety net
• Continuous build & unit test
• Nightly slow tests / code scans
• Emails identifying new issues – ideally tied
against source code changes
• Regular inspection of trends
• Bugs / Stories entered around issues
14
15. Automation as a learning experience
Implementing the safety net helps us discover
unacknowledged debt
15
17. Automation Examples: The Build
• Build time dependencies not understood
• Build scripts missing or incomplete
• “Magic build server” anti-pattern
17 http://www.urbancode.com/html/resources/webinars/Role_of_Binary_repo
sitories_in_Software_Configuration_Management.html
18. Automation Examples: Deployment
• Deployment scripts scarce
• “Special Instructions” with most deployments
indicate non-repeatable process
• Environmental differences handled poorly
• Separation of duties less than real
18
19. Automation Example: Test Automation
• Requirements less well understood
• Existing tests are few, stale, un-optimized
• Application not architected for testability
19
20. Expect the unexpected when automating
• At scale, Green-Shifting, has hidden issues
• Include these discoveries in ROI estimations
for automation (positively and negatively)
• This is a happy side effect
20
21. Direct and Indirect Automation Benefits
• Direct: We tested for problems and found
them.
• Indirect: In attempting to be more
efficient, we automated, and accidently
discovered problems.
21
22. Automating for the team
• Provide a “safety net” to detect and recognize
issues.
• Diagnose and repair lack of repeatability in
build-deploy-test
• Quantify accumulating debt in support of
fighting scope creep
• Team level tooling is fine
22
23. Automating the Enterprise
• Benefits extend beyond aggregate team level
benefits
• Central Automation and Reporting gets us:
– Identify who can use shared configuration
– Who has tests, who doesn’t
– Who is using what tools
– Build / deploy failure rates
23
24. Favorite Examples: Deployment Failures
• Problem: High rate of deployment failures a
problem
• Goal: Reduce failure rate from 40% to 5%
• Approach: Avoid manually fixing a
deployment. Fix the automation and redeploy.
• Enforcement: Monthly / weekly spreadsheet
of success to CIO with a six month plan.
24
25. Favorite Examples: “End of Day” Commits
• Problem: Developers commit breaking
changes at the end of the day
• Goal: Avoid other devs staying late to clean up
• Approach: Report on failures, and react to
negative patterns as a team.
25
27. Favorite Examples: Low numbers of tests
• Problem: Unit testing discipline breaking
down over time
• Goal: Maintain high or improving coverage
• Approach: Standard coverage tools and an
emphasis on upward trends.
• Enforcement: Trending report
on monitor over CIO’s door
27
28. Summary
• We accumulate technical debt as we race to
deliver more, faster.
• This causes us to eventually release
less, slower, with worse quality
• Automation directly and indirectly helps us find
issues.
• There are benefits at both team and enterprise
levels.
28
29. More References
http://urbancode.com/resources
• Enterprise CD Maturity Model
• Lean Build & Deployment Automation
• Managing Release Risks with Metrics
Blogs.urbancode.com
Twitter.com/UrbanCodeSoft
Facbebook.com/UrbanCodeSoft
Slideshare.net/Urbancode
29
30. Yes, we sell products for this
• AnthillPro
– Build automation and build promotion
• uDeploy
– Deployment and release management
30
Collected technical work left undone.When a feature changes, there’s immediate debt to documentation. When we branch, as we work longer in parallel, we accumulate a debt towards integrating. CI minimizes the accumulation of that debt.