James Lewis: Microservices - Systems That Are #neverdone at I T.A.K.E. Unconference 2015

1. 1
never done
building systems that are
jalewis@thoughtworks.com
@boicy

2. 2
agile
XP
TDD
BDD
YAGNI
DRY
SOLID
Continuous Delivery
Refactoring
GoF
GRASP
emergent design
World of Warcraft
KISS

3. 3
the future is scary

4. 4
"ever accelerating progress of technology and changes in the mode
of human life, which gives the appearance of approaching some
essential singularity in the history of the race beyond which human
affairs, as we know them, could not continue”
John von Neumann, as recorded by Ulam, 1958

5. 5
“Around the world, laboring women produce forty-five thousand babies a day,
representing 1023 MIPS of processing power. Also around the world, fab lines
casually churn out thirty million microprocessors a day, representing 1023 MIPS.
In another ten months, most of the MIPS being added to the solar system will be
machine-hosted for the first time”

6. 6

7. 7
“Self-Driving Trucks Are Going to Hit Us Like a Human-Driven Truck”
https://medium.com/basic-income/self-driving-trucks-are-going-to-hit-us-like-a-human-driven-truck-b8507d9c5961
source: Morgan Stanley
This is where we’re at and this is
what we face as we look
towards a quickly approaching
horizon of over 3 million
unemployed truckers and
millions more unemployed
service industry workers in
small towns all over the country
dependent on truckers as
consumers of their services.

8. “If you aren’t worried about strong AI within the next 5-10
years then you aren’t working closely enough with AI
teams”
(paraphrasing Elon Musk)
8

9. 9

10. 9

11. 10
closer to home?

12. 11

13. 12
Singularity

14. 12
Singularity
JavaScript

15. 12
Singularity
JavaScript
Container

16. 12
Singularity
JavaScript
Log aggregation
Container

17. 13
even closer to home

18. HOW WE DESIGN SOFTWARE IS CHANGING
14

19. 15

20. 16
“the first post-devops architectural style”
Neal Ford

21. 17
replaceable component architectures
Dan North

22. 18
never done

23. 19
never done
Incomplete
adjective
not having all the necessary or appropriate parts

24. 19
never done
Incomplete
adjective
not having all the necessary or appropriate parts

25. 20
never done

26. 20
“This, milord, is my family's axe.
We have owned it for almost nine
hundred years, see. Of course,
sometimes it needed a new blade.
And sometimes it has required a
new handle, new designs on the
metalwork, a little refreshing of the
ornamentation . . . but is this not
the nine hundred-year-old axe of
my family? And because it has
changed gently over time, it is still a
pretty good axe, y'know. Pretty
good.”
never done

27. 21
microservices should be:
cheap to replace
and should allow us to go as
“fast as possible”?
quick to scale
able to withstand failure

28. 22
Hardest things to do:

29. 22
Hardest things to do: End-to-end testing

30. 22
Hardest things to do: End-to-end testing
Independent deployment

31. 22
Hardest things to do: End-to-end testing
Independent deployment
Service versioning / evolution

32. TESTING MICROSERVICES IS HARD
23
Service A
Service
Large
Medium
Small

33. TESTING MICROSERVICES IS HARD
23
Service A
Service
Large
Medium
Small

34. Service
Stub
TESTING MICROSERVICES IS HARD
23
Service A
Large
Medium
Small

35. INTEGRATING MICROSERVICES IS HARD
24

36. INTEGRATING MICROSERVICES IS HARD
24
Integration Test

37. INTEGRATING MICROSERVICES IS HARD
24
Integration Test Prod…

38. INTEGRATING MICROSERVICES IS HARD
24
Integration Test Prod…

39. INTEGRATING MICROSERVICES IS HARD
24
Integration Test Prod…

40. INTEGRATING MICROSERVICES IS HARD
24
Integration Test Prod…

41. 25

42. 26
WHAT DOES THIS MEAN
FOR THE CRAFT OF
SOFTWARE DEVELOPMENT?

43. 27
<thinks>

44. 28

45. 29
well crafted software
steadily adding value

46. 30

47. 31
Nat Pryce
Steve Freeman
Dan North
Sydney ‘Hoppalong’ Redelinghuys
Jim Webber
Ian Robinson
Ivan Moore
Liz Keogh
Simon Stewart
Jez Humble
Dave Farley
Jay Fields
Dan Worthington-Bodart
Joe Walnes

48. 31
Nat Pryce
Steve Freeman
Dan North
Sydney ‘Hoppalong’ Redelinghuys
Jim Webber
Ian Robinson
Ivan Moore
Liz Keogh
Simon Stewart
Jez Humble
Dave Farley
Jay Fields
Dan Worthington-Bodart
Joe Walnes
APPEAL TO AUTHORITY
WARNING!

49. 32
TW is an XP shop

50. 33
agile
XP
TDD
BDD
YAGNI
DRY
SOLID
Continuous Delivery
Refactoring
GoF
GRASP
emergent design
World of Warcraft
KISS

51. 34
does this still make sense?

52. 35

53. 35
Gemini Project, Rogallo wing
Source: wikipedia.org

54. 36
<thinks>

55. 37
Gemini Project, Rogallo wing
Source: wikipedia.org
it’s turtles all the way down

56. 38
agile
XP
TDD
BDD
YAGNI
DRY
SOLID
Continuous Delivery
Refactoring
GoF
GRASP
emergent design
World of Warcraft
KISS

57. 39
agile
XP
TDD
BDD
YAGNI
DRY
SOLID
Continuous Delivery
Refactoring
GoF
GRASP
emergent design
World of Warcraft
KISS

58. 40
http://martinfowler.com/bliki/Yagni.html

59. 4141

60. 4242

61. 4343

62. 4444
Build out
services as
you need
them

63. 45
agile
XP
TDD
BDD
YAGNI
DRY
SOLID
Continuous Delivery
Refactoring
GoF
GRASPemergent design
World of Warcraft
KISS

64. 46

65. 47
Fulfilment
Retail

66. 48
Fulfilment Retail

67. 49
Fulfilment Retail

68. 50
Fulfilment Retail

69. 51
Fulfilment Retail
High cohesion
Low coupling

70. 52
(incidentally, if you were playing
the Conway’s law lottery, that’s
when you number came up)

71. 53
agile
XP
TDD
BDD
YAGNI
DRY
SOLID
Continuous Delivery
Refactoring
GoF
GRASP
emergent design
World of Warcraft
KISS

72. 5454

73. 55
git clone https://github.com/boicy/service-template
(note this doesn’t exist)

74. 56

75. 57
DRY within services
duplication between
services

76. 58
agile
XP
TDD
BDD
YAGNI
DRY
SOLID
Continuous Delivery
Refactoring
GoF
GRASP
emergent design
World of Warcraft
KISS

77. 59
“The London school
of Test Driven
Development”
Mike Feathers

78. 60
should we write unit tests?

79. 61
Nat Pryce
Steve Freeman
Dan North
Sydney ‘Hoppalong’ Redelinghuys
Jim Webber
Ian Robinson
Ivan Moore
Liz Keogh
Simon Stewart
Jez Humble
Dave Farley
Jay Fields
Dan Worthington-Bodart
Joe Walnes

80. 62
http://moleseyhill.com/blog/2009/08/27/dreyfus-model/

81. should we write unit tests?
63
personally I think it’s more
important than *ever*

82. 64
agile
XP
TDD
BDD
YAGNI
DRY
Continuous Delivery
Refactoring
GoF
GRASP
emergent design
World of Warcraft
KISS
SOLID

83. 65
a class should be no bigger than my head

84. 6666
a:Class

85. 6767
a:Class
a:Class
a:Class
a:Class

86. 6868

87. 6969

88. 70
SRP
a service should
be no bigger than
my head

89. 71
agile
XP
TDD
BDD
YAGNI
DRY
SOLID
Continuous Delivery
Refactoring
GoF
GRASP
emergent design
World of Warcraft
KISS

90. 72

91. 73
WWJD?

92. 73
WWJD?
(what would Joe do?)

93. 74

94. 75
cron, python, boto, pydot, graphviz

95. 75
cron, python, boto, pydot, graphviz

96. 76
cron, python, boto, pydot, graphviz
Do the simplest
thing possible

97. 77
but what about testing and deployment

98. 78

99. 78

100. 78
SEMANTIC MONITORING

101. 79
Customer
Service
Web Shop

102. 79
Customer
Service
Web Shop
Small
Medium
Large

103. 79
Customer
Service
Web Shop
Small
Medium
Large

104. 79
Customer
Service
Web Shop
Small
Medium
Large
Consumer Driven Contracts

105. 80
Customer
Service
Web Shop

106. 80
Customer
Service
Web Shop
Expectations

107. 80
Customer
Service
Web Shop
Expectations

108. 80
Customer
Service
Web Shop
Expectations
Prod

109. 80
Customer
Service
Web Shop
Expectations
Prod

110. 81

111. 81
https://github.com/realestate-com-au/pact

112. 82
Prod
Prod
Prod
Prod
QA
Good Monitoring
Fast Remediation
TESTING IN PRODUCTION

113. 83
CHANGING SERVICES INDEPENDENTLY

114. 84
Thomas J. Allen, 1977

115. 85
0 10 20 30 40 50 60 70 80 90 100
0
0.05
0.10
0.15
0.20
0.25
0.30
m
Probability of
weekly interaction
x
x x
x x
xxxx
xx
xx
x
x x
x
x x
x x
The effect of distance on communication

116. 86
Low change rate
inter-company integration
High stability
Semantic Versioning
Contract Testing
Tolerant Reader
“Conversational change”

117. 87
higher change rate
inter-team integration
lower stability
Semantic Versioning
Contract Testing
Tolerant Reader
“Conversational change”

118. 88
highest change rate
intra-team
lowest stability
Semantic Versioning
Contract Testing
Tolerant Reader
“Conversational change”

119. 89
integration environment
the death of the

120. 90
production != live

121. 91
the future is scary

122. we are learning how to:
Craft my families axe
Deploy microservices independently
Test microservices in isolation
Test microservices in production

123. 93
the future is bright

124. 94
we have old techniques that apply:
SRP
GRASP
YAGNI
KISS
TDD
DRY

125. and new techniques to apply:
Consumer Driven Contracts
Semantic Monitoring
Semantic Versioning
Testing in Production

126. 96

127. 96
software craftsmanship is around the code too

128. 96
software craftsmanship is around the code too
chunk up and think about the “gaps”

129. 96
software craftsmanship is around the code too
chunk up and think about the “gaps”
and

130. 96
software craftsmanship is around the code too
chunk up and think about the “gaps”
do the simplest thing possible
and

131. 97
Now…

132. 98
Go forth and craft software
jalewis@thoughtworks.com
@boicy

133. 98
Go forth and craft software
jalewis@thoughtworks.com
@boicy
Thanks!
Go forth and craft software
drink beer!

134. 99
I, for one, welcome
our new overlords