Slides for the CI Tutorial delivered at STPcon 2009.

1. Core Principles of CI
STPcon
1

2. Intro
2

3. Who Am I?
• Principal of OpenSource
Connections
• Contributer to CruiseControl
and Continuum CI projects
• Member of Apache Software
Foundation
• Presenter at conferences
(OSCON, ApacheCON, jTDS,
ExpoQA, STPcon)
• Fascinated by the art of
software development
3

4. Author
4

5. Father
5

6. Agilista
6

7. Schedule
Core Principles of
9 to 10:15
CI
Break 15 minutes
Hudson to the
10:30 to 11:15
Rescue
Agile and CI 11:15 to 12
7

8. Rules of the Road
• Cell phones on Silent.
• Ask Questions!
• Feel free to get up and
move around the room.
8

9. What is CI?
9

10. Conceptual
10

11. Definition
a fully automated and reproducible build, including testing, that
runs many times a day - Martin Fowler
http://martinfowler.com/articles/continuousIntegration.html
11

12. Martin Fowler
• Coined the term
Continuous Integration
• Chief Scientist for
ThoughtWorks
• Speaks and Blogs on
software:
• http://martinfowler.com/
12

13. Mapping
diagram credit to Brian Di Croce @ http://blog.briandicroce.com/2008/03/17/setting-up-a-continuous-
integration-environment-preface/
13

14. Activity
Turn to the person next to you, and see how many
non-obvious things you have in common.
14

15. Arm Bands!
• Elisabeth Hendrickson is
TestObsessed.com
• Focuses on Testing in the Agile
community
• Really great blogger: Specialized
Test Management Systems are an
Agile Impediment and Adventures
with Auto-Generated Tests and
RSpec are typical posts
15

16. CI in a nutshell
16

17. Demo
17

18. People
18

19. Version Control
Repository
19

20. CI Server
20

21. Build Script
• Automated Script is Key!
• Needs to be robust and simple
21

22. Feedback Mechanism
From:
cruisecontrol@opensourceconnections.com
ubject:
[CruiseControl] agenda_direcciones build 370 fixed
S
Date:
October 7, 2008 2:22:50 PM EDT
To:
epugh@opensourceconnections.com
The build has been fixed.
CHANGES
-------
New revision 370 detected
Revision 370 committed by epugh on 2008-10-07 18:22:03
mysql esta mejor que sqlite
M /branches/ERIC_EXPO_08_DEMO/agenda_direcciones/config/database.yml
See http://cruise.opensourceconnections.com:3333/builds/agenda_direcciones/370 for details.
22

23. photo credit to Jay Grieves @ www.bigswingingdeveloper.com
23

24. Aspects of CI
24

25. Source Code
Compilation
• Eclipse “Export to War”
is NOT the solution
25

26. Database Integration
• Database is integral to • Don’t require a multi gig
your code database file to run
tests!
• DDL, stored
procedures are • Look at techniques like
treated as source fixtures or mock objects
to reduce database
• Involve your DBA in coupling
your project
• Developers need
• DBUnit for Java database sandbox
projects!
26

27. Testing
with out testing, CI is just a glorified compiler- Eric Pugh
• Testing proves the intent
of the programmer was
met.
• Typically provides the
“checkability” that James
Bach talks about
• Multiple Levels of testing
27

28. Inspection
• Code Coverage
• Code Complexity
• Copy’n’pasting
• Architectural Standards
• Lint Checking
• Static Analysis
28

29. Deployment
• Turns what is normally a difficult process
into a non event...
• ant deploy
• Simplifies standup new environments
29

30. Documentation
30

31. Feedback
• Email
• IRC
• Devices
• Blog Posts
• Twitter Notifications
• Tray Icons
31

32. So What is the Value?
32

33. Life with out CI
• Unstable code, integration is difficult
• Many build errors reported
• Only one person on the team can build
the entire project
• Getting “test ready” product is hard
• Feedback cycle is long
• Same bug keeps coming back
Every day is a struggle to be productive
33

34. Life with CI
• The build process is simple and
repeatable
• Elimination of Human Error
• Bugs stay fixed!
• Feedback cycle is very rapid
Every day you know you can produce better software!
34

35. Risk Reduction
• What is in the build?
• What has changed
between builds?
• How do I verify
functionality?
35

36. Reduce Human Error
36

37. My Ah-hah! moment
37

38. Reduce Repetitive
Processes
• DRY: Don’t Repeat
Yourself
• Script Everything!
• “ant deploy”
38

39. Make Painful Things
Painless!
photo credit to Paul Duvall @ http://www.integratebutton.com/
39

40. Produce Deployable
Software
• It’s what we are here to do!
• Always in a “Demo Ready” state
• No last minute administrivia tasks to get
the product ready
• Installer is built
• Documentation is written
• Changelog is written
40

41. Enable Project Visibility
41

42. 42

43. Increase Project
Confidence
43

44. 44

45. photo credit to ThoughtWorks
45

46. Facilitate
Communication
46

47. photo credit to Eric Pugh @ www.opensourceconnections.com
47

48. 48

49. 49

50. Activity
Exquisite corpse (also known as "exquisite
cadaver" or "rotating corpse") is a method
by which a collection of words or images is
collectively assembled, the result being
known as the exquisite corpse or cadavre
exquis in French.
50

51. What Prevents Teams
from Using CI?
51

52. Cultural Challenges
52

53. Fear of Commitment
• Developers and Testers
need to commit changes
frequently!
• Incremental small changes
make finding integration
issues simpler
drawing credit to Tom Armitage @ http://infovore.org/archives/2007/03/14/the-continuous-integration-
barometer/
53

54. This isn’t 1984
54

55. Fear of Blame
55

56. • CI needs a champion who acts as the
ambassador to the leaders of the
organization.
• Thought leaders in the organization who
can help encourage team to accept the
process changes that CI brings
• A very successful test case
• A greenfield project if possible
56

57. 57

58. Structural Challenges
58

59. Testability
• Modularity? • Perform code
complexity analysis to
• Coupling to underlying see if some parts are
more amenable to
platform?
testing.
• Requires external
services to be up and
available for testing?
59

60. Tight Coupling
photo credit to Jay Grieves @ www.bigswingingdeveloper.com
60

61. CI Tool Stability
Challenges
61

62. Invest in CI Server
• The CI system must be as well
X
maintained as the Source Control
system.
• The CI system performs builds very
✓
quickly.
• Who has responsibility for CI? It’s very
important to have a “throat to choke”.
• No false alarms. If there are false
alarms then the team loses confidence
in the CI system.
62

63. Monkey CI
photo credit to Eric Pugh @ www.opensourceconnections.com
63

64. When Should You
Adopt CI?
64

65. From
good CI
to
great CI!
65

66. 10 minute rule
photo credit to Jay Grieves @ www.bigswingingdeveloper.com
66

67. Multi Stage CI
• Reduce the amount of
integration required
• Teams integrate as
modules are marked as
ready.
• Maven2 was built with
this in mind!
Damon Poole @ http://damonpoole.blogspot.com/2008/01/advanced-multi-stage-continous.html
67

68. Break up Tests
• “preflight” or “smoke” tests
• longer integration tests
• periodic full site generation and report
generation
68

69. Predictive Testing
• Some commercial tools support predicting
which tests will fail based on previous
behavior, and running those first!
69

70. Cup of Coffee Metric
What happens in the time it takes to get a cup of coffee can
tell you a lot about the quality of your continuous integration
system. - Paul Julius
• Builds shouldn’t take any longer!
• Getting a cup of coffee or drink of
water breaks up your day
• Gives you a chance to review your
work
• Requires a moduler fast build
http://pauljulius.com/blog/2009/09/14/cup-of-coffee-metric-for-continuous-integration/
70

71. Paul Julius
• co-founder of CITCON
• Original developer of
CruiseControl
• Speaks and Blogs on CI
extensively
• http://pauljulius.com/blog/
71

72. Virtuous Circle
• Metrics are hard to get
• Teams (Developers?)
rebel against on high
management
• Need good processes
that support and
encourage other good
processes
72

73. HackyStat
73

74. CI isn’t one specific
method
74

75. !F!2!89.+;D+!89!:G:7.!+67
789:;0<69;:= 8;>76! 789!:2!07?9! ?0>?86!0 78.?8!
!"#$%#&'
!"#$%&'"()*+ ?-#4+!@)1-#)+ 0)/)3*)31A+ 95$1B'3C+ ,-'&*+D54E+
,-'&*+.#)/" ,-'&*+.#)/" :)/4"'#45A !"#$%&!'(#%)* >2+.3$/"(4#"
0)*'1$#)*+ ,-'&*+6&-"#)5 .)1-5)+
,-'&*+2$1('3) !"#$%&!'(#%)*
643#'3-4-"+
.4&'*+ ,-'&*"
.4-51)+643#54&
Enterprise CI
!"#$%&'()
789:;0<69;:= 8;>76! 789!:2!07?9! ?0>?86!0 78.?8!
H)&/)5+.15'/#" D-&&A+.15'/#)* .)&IJ.)5K'1)+ G-"(J,-##43+ ?-#4E$#'1+
9)"#+0)/&4A" G54*+0)/&4AE)3# G54*-1#'43+
0)/&4A"
?-#4+0)/&4A+#4+ N$#)*O+?-#4E$#'1+
;3)+!3KL G54E4#'43"
Maturity Model
24"#&A+ D-&&A+.#$3*$5*'M)*+
.#$3*$5*'M)* 0)/&4A"
789:;0<69;:= 8;>76! 789!:2!07?9! ?0>?86!0 78.?8!
Came out of discussions at 2009
!"#!$%&
.4E)++ :)C5)""'43+ .4E)+.#$#'1+ !@#)3"'K)+ QRRS+
9)"#+?-#4E$#'43 9)"#"+:-3+P'#(+ ?3$&A"'"+<")* <3'#+9)"#" 9)"#+64K)5$C)
,-'&*
CITcon. Developed by Eric ?-#4E$#)*+
D-31#'43$&+9)"#"
!@#)3"'K)+
?-#4E$#)*+
D-31#'43+9)"#"
Minick of UrbanCode :'"BJ,$")*+
2$3-$&+9)"#"
.)1-5'#A+.1$3"
!"#$!%&'(
789:;0<69;:= 8;>76! 789!:2!07?9! ?0>?86!0 78.?8!
944&J,$")*+ .'&4)*+:)/45#'3C H'"#45'1$&+ 95)3*'3C G5)*'1#'K)
:)/45#" :)/45#" 24*)&'3C
G-%&'"(+ 654""J.'&4
>'"'%'&'#AT+:)/45#+ F$#)"#+:)/45#" 654""+D-31#'43+ ?3$&A"'"
:-33)5 :)/45#+?11)""
,$"'1+
:)/45#+?3$&A"'"
'!()*+,"$-!).$/ !"#$%&"!'
!"#$%&'()*&)+(&,%-$(.(/0$&12&3"-+(/-4&3)5$6&78/-$.".$(
!""#$%%&&&'()"!*++#,-'.-/%!"/+%,01-2,.01%&!*"03#(#0,1
75

76. 789:;0<69;:= 8;>76! 789!:2!07?9! ?0>?86!0 78.?8!
!"#$%#&'
!"#$%&'"()*+ ?-#4+!@)1-#)+ 0)/)3*)31A+ 95$1B'3C+ ,-'&*+D54E+
,-'&*+.#)/" ,-'&*+.#)/" :)/4"'#45A !"#$%&!'(#%)* >2+.3$/"(4#"
0)*'1$#)*+ ,-'&*+6&-"#)5 .)1-5)+
,-'&*+2$1('3) !"#$%&!'(#%)*
643#'3-4-"+
.4&'*+ ,-'&*"
.4-51)+643#54&
76

77. !"#$%&'()
789:;0<69;:= 8;>76! 789!:2!07?9! ?0>?86!0 78.?8!
H)&/)5+.15'/#" D-&&A+.15'/#)* .)&IJ.)5K'1)+ G-"(J,-##43+ ?-#4E$#'1+
9)"#+0)/&4A" G54*+0)/&4AE)3# G54*-1#'43+
0)/&4A"
?-#4+0)/&4A+#4+ N$#)*O+?-#4E$#'1+
;3)+!3KL G54E4#'43"
24"#&A+ D-&&A+.#$3*$5*'M)*+
.#$3*$5*'M)* 0)/&4A"
77

78. 789:;0<69;:= 8;>76! 789!:2!07?9! ?0>?86!0 78.?8!
!"#!$%&
.4E)++ :)C5)""'43+ .4E)+.#$#'1+ !@#)3"'K)+ QRRS+
9)"#+?-#4E$#'43 9)"#"+:-3+P'#(+ ?3$&A"'"+<")* <3'#+9)"#" 9)"#+64K)5$C)
,-'&*
?-#4E$#)*+ !@#)3"'K)+
D-31#'43$&+9)"#" ?-#4E$#)*+
D-31#'43+9)"#"
:'"BJ,$")*+
2$3-$&+9)"#"
.)1-5'#A+.1$3"
78

79. !"#$!%&'(
789:;0<69;:= 8;>76! 789!:2!07?9! ?0>?86!0 78.?8!
944&J,$")*+ .'&4)*+:)/45#'3C H'"#45'1$&+ 95)3*'3C G5)*'1#'K)
:)/45#" :)/45#" 24*)&'3C
G-%&'"(+ 654""J.'&4
>'"'%'&'#AT+:)/45#+ F$#)"#+:)/45#" 654""+D-31#'43+ ?3$&A"'"
:-33)5 :)/45#+?11)""
,$"'1+
:)/45#+?3$&A"'"
79

80. What about People?
80

81. CI on a Dollar a Day
"From now on, our code in revision control will always build
successfully and pass its tests.". - James Shore
• Build Computer
• Ridiculous Toy
• Annoying Bell
• Automated Build
• Group Agreement
http://jamesshore.com/Blog/Continuous-Integration-on-a-Dollar-a-Day.html
81

82. credit to Paul Julius @ http://www.anthillpro.com/blogs/anthillpro-blog/2009/09/11/
enterprise_ci_cultural_maturity.html
82

83. Lots of
advanced
techniques!
credit to Christine Mitterbauer @ Microdoc
83

84. Perils of CI
84

85. Getting too focused on
Green Bar
Glare of the Green Bar in CI can
obscure issues with your product-
James Bach
85

86. Making CI the core of too
many systems
86

87. Making decisions based
on your available
tooling
87

88. Selecting a CI System
88

89. Approaches
• Big Hand Rolled System
• Cloud Based Solution
• Completely Hosted
89

90. Hand Rolled System
Pros Cons
Complete control of Rolling your own can take
configuration more work
Data and code inside Dealing with disparate
firewall platform elements is harder
Works with any type of Multiple kinds of projects can
system be hard to manage
Must provide Sys Admin
Cheapest
work
90

91. CI Matrix
Matrix of differences between CI tools: http://tinyurl.com/39hol7
91

92. Cloud Based Solution
Pros Cons
Data and Code outside
Very Scalable
firewall
Use CPU resources only
Cloud computing still new
when needed
Deal well with multiple
GUI apps hard to test
kinds of projects
Must provide Sys Admin
?
work
92

93. CI in a Box
• Amazon EC2 Hosted
instance
• http://www.ciinabox.com/
93

94. Hosted Solution
Pros Cons
Typically limited in language
Very Scalable
support
data and code outside
No Sys Admin tasks
firewall
Pricing is fairly
GUI apps hard to test
straightforward
? vendors are still new
94

95. RunCodeRun.com
95

96. MikeCI.com
96

97. Where to Next?
97

98. CITCON
• Open Spaces Style
• Twice a year, US and International
• Raleigh-Durham, NC, April 2010
• http://www.citconf.com/
98

99. 99

100. CI Yahoo Group
http://tech.groups.yahoo.com/group/citcon/
100

101. The Book
Continuous Integration:
Improving Software Quality and
Reducing Risk
101

102. The CI guys
http://ci-guys.com/
Free online consulting
102