Continuous integration (CI) and continuous deployment (CD) have been key to increasing the pace at which development teams can deliver value to the business while keeping quality high. However, many teams are finding that their build pipelines are hard to manage, with builds taking so long that they often halt the development process. Abraham Marin-Perez explores the negative effects of long builds and discusses the most common solutions that organizations apply—many of which involve unfortunate compromises that reduce the benefits of CI/CD. In order to fix this problem without compromises, Abraham shares a technique to rebuild only the parts of the system affected by a change and metrics that you can use to assess the overall health of the build pipeline. Learn how to equip your team with the necessary tools to identify when builds are too long—and what to do about it.

1. DW4
DevOps Implementation
11/16/2016 1:30:00 PM
Make Your Continuous Deployment
Pipeline as Fast as Possible
Presented by:
Abraham Marin-Perez
Equal Experts
Brought to you by:
350 Corporate Way, Suite 400, Orange Park, FL 32073
888--‐268--‐8770 ·∙ 904--‐278--‐0524 - info@techwell.com - http://www.stareast.techwell.com/

2. Abraham Marin-Perez
Equal Experts
Abraham Marin-Perez is a Java programmer, author, speaker, and agile
consultant. Abraham has been working in software development for a decade,
paying special attention to what maximizes a team's capacity to deliver in the
long term. This has led him to study everyday programming habits and
techniques that ensure the sustainability of the development process, sharing his
views in Real-World Maintainable Software and more informally in his blog and
on Twitter (@AbrahamMarin). He helps run the London Java Community—an
organization which is an elected member of the Java Community Process
Executive Committee, and contributes as a Java news editor at InfoQ.

3. 11/1/2016
1
Keeping Your CI / CD Pipeline
Abraham Marin-Perez
@AbrahamMarin
Keeping Your CI / CD Pipeline
as Fast as It Needs to Be
@AbrahamMarin
fromfragiletoagile.com
#BetterSoftwareCon#BetterSoftwareCon @AbrahamMarin @EqualExperts@AbrahamMarin @EqualExperts
About Me

4. 11/1/2016
2
About Me
About Me

5. 11/1/2016
3
About Me
About Me

6. 11/1/2016
4
About Me
About Me

7. 11/1/2016
5
About This Talk

8. 11/1/2016
6
About This Talk
 Continuous Integration: check everything is still
working after every commit
 Continuous Deployment: every successful
commit turns into a releasecommit turns into a release
About This Talk

9. 11/1/2016
7
About This Talk
About This Talk

10. 11/1/2016
8
About This Talk

11. 11/1/2016
9
SUPER
APP
# Files: 75
# Tests: 800
Build Time: 4 min
Output: superapp.war
SUPER
APP
# Files: 113
# Tests: 1200
Build Time: 6 min
Output: superapp.war

12. 11/1/2016
10
SUPER
APP
# Files: 169
# Tests: 1800
Build Time: 9 min
Output: superapp.war
Slow feedback
The Problems Of Size
Broken builds mask issues
Development paralysis
Impact on ability to meet our SLAsp y
Pay per use
Missed business opportunities

13. 11/1/2016
11
Live with it
How Organisations Manage Size
Partial CD: only quick tests
Phased CD: split into components
Test Deprecation Policy
Microservices
Microservices

14. 11/1/2016
12
Microservices
Microservices

15. 11/1/2016
13

16. 11/1/2016
14
SUPER
APP
# Files: 169
# Tests: 1800
Build Time: 9 min
Output: superapp.war
APP
# Files: 72
# Tests: 800
Build Time: 4 min
APP
BACKEN
D
SUPER APP
# Files: 115
# Tests: 1200
Build Time: 6 min
Output: superapp.war
Build Time: 4 min
Output: appbackend.jar
Microservices?

17. 11/1/2016
15
Microservices
Microservices

18. 11/1/2016
16
Microservices
Microservices

19. 11/1/2016
17
Microservices
Microservices

20. 11/1/2016
18
Build Pipeline Becomes aBuild Pipeline Becomes a
Network
Scalable Continuous Deployment
With Maven
https://goo.gl/LvkkRq

21. 11/1/2016
19
A real case scenario

22. 11/1/2016
20
Service
Service
Service
Parent
POM
Logging
Service
28%
Service
Service
Parent
POM
Logging

23. 11/1/2016
21
Service
28%
28%
Service
28%
Service
Parent
POM
Logging
Service28%
28%
20%
Service
28%
Service
Parent
POM
Logging

24. 11/1/2016
22
Service
48%
28%
20%
Service
28%
Service
Parent
POM
Logging
Service
Service
Service
Parent
POM
Logging

25. 11/1/2016
23
Service
Service
Service
Useful Metrics
 Build Time (BT): time an individual build takes
to run
 Change Rate (CR): percentage of commits upon
an individual build with respect to the whole
system

26. 11/1/2016
24
Service
Service
Servi
ce
Service28%
Service
Service
Parent
POM
Logging

27. 11/1/2016
25
 Impact Time (IT): total time to run a build and all the
builds that will be triggered as a result
Useful Metrics
builds that will be triggered as a result
No dependants 
Useful Metrics
IT(A) = BT(A)
A

28. 11/1/2016
26
Serial execution 
Useful Metrics
IT(A) = BT(A) + IT(B) + IT(C)
B
A
C
Parallel execution 
IT(A) BT(A) + ma (IT(B) IT(C))
Useful Metrics
IT(A) = BT(A) + max(IT(B), IT(C))
B
A
C

29. 11/1/2016
27
Service
Service
Service
Useful Metrics
Weighted Impact Time (WIT): impact time of a build
weighted according to its change rage
WIT(A) = IT(A) * CR(A)

30. 11/1/2016
28
Average Impact Time (AIT): total time needed on
Useful Metrics
Average Impact Time (AIT): total time needed, on
average, to execute all necessary builds after any
given commit anywhere in the system
AIT = WIT(A) + WIT(B) + ... + WIT(Z)AIT WIT(A) WIT(B) ... WIT(Z)
Sample Thresholds

31. 11/1/2016
29
Sample Thresholds
Average Impact Time
Average Impact Time is what indicates how well you
have scaled your systemy y
M i I t Ti
Sample Thresholds
Maximum Impact Time
In a worst-case scenario, a build won’t take longer
than this.

32. 11/1/2016
30
Maximum Impact Time for Critical Components
Sample Thresholds
Maximum Impact Time for Critical Components
The same, but only for your most sensitive modules
(log-in, payment gateway, etc.)
Beware of dependencies!
Service
Service
Service

33. 11/1/2016
31
M l iManual processing
takes time...
Automating Build Analysis
 Most CI systems provide an API
 Calculations aren’t complex
 Multiple graphical tools availablep g p

34. 11/1/2016
32
github.com/quiram/build-hotspots
Build Hotspots
https://https://commons.wikimedia.orgcommons.wikimedia.org/wiki/File:2012_Italian_GP_/wiki/File:2012_Italian_GP_--_Lotus_wheel.jpg_Lotus_wheel.jpg

35. 11/1/2016
33
Thank YouThank You
fromfragiletoagile.com
@AbrahamMarin
#FastCI #BetterSoftwareCon
@EqualExper
ts
equal-
experts
equalexperts.com