1. @koenighotze
10 (or so) tips for failing
at microservices - badly

2. @koenighotze
David Schmitz / @koenighotze
Senacor Technologies

3. @koenighotze
Disclaimers

4. @koenighotze
This is
NOT
about:
you,
your project,
your customer,
my customers,
or any future customer
or project

5. @koenighotze
Anecdote
IS NOT
Evidence

6. @koenighotze
So, what’s in it for
you?

7. @koenighotze
Your boss wants to modernise
You don’t want to modernise
Things to do to sabotage your project

8. @koenighotze
So…your Bossread
about microservices?

9. @koenighotze
Extreme agile
cloud native digital
microservices!
Disrupt or be
disrupted!
Sounds awesome.
And cloud and agile
equal good?!?
Industry 4.0!
FTW!

10. @koenighotze
“IT People! Make it happen!
Build me some microservices”

11. @koenighotze
What the hell is a microservice and why should I care?

12. @koenighotze
Microservices
Single bounded context
UI and services
Independent
Self-contained
Local storage
Share nothing
User Interface
Services
Persistence
Operations

13. @koenighotze
BLAH BLAH…
User Interface
Services
Persistence
Operations
Too lazy to actually read all
this stuff…looks boring and I
want to look at funny cat
pics.

14. @koenighotze
Just ask yourself…

15. @koenighotze
Do you prefer change…
https://flic.kr/p/bVPzUE

16. @koenighotzehttps://flic.kr/p/bajCVX
…or rest?

17. @koenighotze
Sabotage countdown

18. @koenighotze
10

19. @koenighotze
Go full scale polyglot

20. @koenighotze
Where is Waldo?

21. @koenighotze

22. @koenighotze

23. @koenighotze
Still successful?

24. @koenighotze
Semantic Versioning
Major.Minor.BugfixMajor.Minor.BugfixMajor.Minor.BugfixMajor.Minor.Bugfix
Semantic Versioning

25. @koenighotze
$ npm ls | grep @0

26. @koenighotze
ui-broker@0.1.0
│ ├─┬ fbjs@0.8.8
│ │ │ ├─┬ encoding@0.1.12
│ │ │ │ └── iconv-lite@0.4.15
│ │ └── ua-parser-js@0.7.12
├─┬ react-scripts@0.8.5
│ │ ├── normalize-range@0.1.2
│ │ │ │ └── json5@0.5.1
│ │ │ └── …

27. @koenighotze
William-Wallace-Tribute-team!
Groundhog code
Maintenance field trips
Synergies: polyglot UI with beta status
Find new friends on Stackoverflow
Heisenops

28. @koenighotze
Polyglot?

29. @koenighotze
Polyglot?
thx @Codepitbull

30. @koenighotze
9

31. @koenighotze
The data monolith
(c) Disney Corporation

32. @koenighotze

33. @koenighotze
Credit Account Payment
Expensive
enterprise SQL
Datastore

34. @koenighotze
Credit Account Payment

35. @koenighotze
Account name must be
VARCHAR(200) instead of
VARCHAR(50)

36. @koenighotze
Credit Account Payment

37. @koenighotze
Sharing is a Good ThingTM
Avoid table or schema ownership
Ubiquitous domain modelling paralysis
Keep people awake at night with
insidious dependencies

38. @koenighotze
8

39. @koenighotze
The event monolith

40. @koenighotze
Eventsourcing marketing pitch

41. @koenighotze
Customer
Homeaddress
Updated
Credit
acquired
New
Customer
Created
…
Direction of time
Customer Credit
Event Store

42. @koenighotze
Audit trail - immutable history for free
Distributed data management
Decoupled microservices with CQRS
…and much more!

43. @koenighotze
‘Das schöne
Österreich’ will no longer be
accepted as a country code

44. @koenighotze
Country:
“AU”
Country:
“Das schöne
Österreich”
…
Direction of time
Customer
Event Store
Credit

45. @koenighotze
How to deal with shared code?

46. @koenighotze
- C P

47. @koenighotze
Country:
“AU”
Country:
“Das schöne
Österreich”
…
Direction of time
Customer Credit
Event Store

48. @koenighotze
2 months later…

49. @koenighotze
Customer Credit Reporting Accounting Profile
Loan Trading Exchange Broker Purchase

50. @koenighotze
Don’t come up with a data cleansing strategy
WET is the new DRY
Never question the design
All systems should be eventsourced
Eventual consistency = “Eventuell konsistent”

51. @koenighotze
7

52. @koenighotze
The home grown monolith

53. @koenighotze
– Carl Sagan (slightly paraphrased)
“If you wish to make
MICROSERVICES from
scratch, you must first
create a FRAMEWORK”

54. @koenighotze
{
"type": "NewCustomerCreated",
"id": 1234,
"payload": {
}
}
CustomerAccount Reporting
ES V1 ES V1 ES V1

55. @koenighotze
The customer system needs
special, non-numeric ids

56. @koenighotze
Major.Minor.Bugfix

57. @koenighotze
Customer V2
{
"type": "NewCustomerCreated",
"id": "1234-5678",
"payload": {
...
}
}
ES V2

58. @koenighotze
And at runtime???

59. @koenighotze
Customer V2Account Reporting
ES V1 ES V1ES V2
{
"type": "NewCustomerCreated",
"id": "1234-5678",
"payload": {
...
}
}

60. @koenighotze
Change the world event
Customer V2Account V2 Reporting V2
ES V2 ES V2ES V2

61. @koenighotze
Customer
Account
Reporting
Danger: independent releases

62. @koenighotze
Customer
Account
Reporting
Success: release monolith
Big bang

63. @koenighotze
Day rates!

64. @koenighotze
Dependencies as social tools
Job insurance via invasive frameworks
Building frameworks is fun!
Ideas: collections, string-utils, logging,
ORM, JavaScript frontend framework!

65. @koenighotze
No time to create a framework?

66. @koenighotze
10/Oct/1999:21:15:05 +0500
Sep 19 16:27:49
2007-08-31 19:22:21.469 ADT
060516 22:38:54
Fri Sep 09 10:42:29.902022 2011

67. @koenighotze
6

68. @koenighotze
The meat cloud

69. @koenighotze
Bonus hint:
optimise your revenue!

70. @koenighotze
W K
P T

71. @koenighotze
Who KNOWS how
to do it?K
P T
Who WANTS it
to be done?
Who has the
required
PRIVILEGES?
Who has TIME
to actually
do it?

72. @koenighotze
Day rates!

73. @koenighotze
Clever insight:
CD is very useful

74. @koenighotze
Delivery
pipeline
Production

75. @koenighotze
And the rest of the
world?

76. @koenighotze
Delivery pipeline
Production
…
Handcrafted
with love
Soulless
automation

77. @koenighotze
Cattle not pets

78. @koenighotze
Ask yourself:
Whom do YOU love more?

79. @koenighotze
Mr. Wiskers Walking steak

80. @koenighotze
W
Quickly…release a new
Customer microservice
K
P T

81. @koenighotze
Enter the DevOps TeamTM

82. @koenighotze
Ops toolchain?
P T
W K
P T

83. @koenighotze
P T
KW

84. @koenighotze
P T
W K
P T

85. @koenighotze
P T
W K
P T

86. @koenighotze

87. @koenighotze
Infrastructure is expensive…be proud to
take care of it
Microsoft Word is some kind of automation
Play the “we are not Google” card
Combine with polyglot ops for maximum
chaos

88. @koenighotze
5

89. @koenighotze
The prescriptive cloud

90. @koenighotze
Agility
Flexibility
Scale as you grow
Use what you need

91. @koenighotze
One Size Fits All
Cloud
The

92. @koenighotze
One Size Fits All
Cloud
The

93. @koenighotze
Rails
TODO app

94. @koenighotze
200 users now…
…but we might become the next Google

95. @koenighotze
Docker
Rails
TODO app

96. @koenighotze
Node
Pod
Docker
Rails
TODO app
EtcdService
FEK
Volume
Pod
Docker
Rails
TODO app

97. @koenighotze
VPC Security groupCloudwatch
ELB IAMEBS
EC2 EC2 EC2 EC2
EC2 EC2 EC2 EC2

98. @koenighotze
[610420.800771] kube-state-metr invoked oom-killer: gfp_mask=0x24000c0, order=0, oom_score_adj=-998
[610420.805428] kube-state-metr cpuset=32c394ad4709430cddb71194ea406d598b82159c2715d42b64927ac5e67f0228 mems_allowed=0
[610420.810504] CPU: 3 PID: 9403 Comm: kube-state-metr Tainted: G E 4.4.65-k8s #1
[610420.813712] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016
[610420.813712] 0000000000000286 00000000c5185d6f ffffffff812f67b5 ffff8803e7163e20
[610420.813712] ffff8803e71c8400 ffffffff811d8855 ffffffff81826173 ffff8803e71c8400
[610420.813712] ffffffff81a6b740 0000000000000206 0000000000000002 ffff880308bbf438
[610420.813712] Call Trace:
[610420.813712] [<ffffffff812f67b5>] ? dump_stack+0x5c/0x77
[610420.813712] [<ffffffff811d8855>] ? dump_header+0x62/0x1d7
[610420.813712] [<ffffffff8116ded1>] ? oom_kill_process+0x211/0x3d0
[610420.813712] [<ffffffff811d0f4f>] ? mem_cgroup_iter+0x1cf/0x360
[610420.813712] [<ffffffff811d2de3>] ? mem_cgroup_out_of_memory+0x283/0x2c0
[610420.813712] [<ffffffff811d3abd>] ? mem_cgroup_oom_synchronize+0x32d/0x340
[610420.813712] [<ffffffff811cf170>] ? mem_cgroup_begin_page_stat+0x90/0x90
[610420.813712] [<ffffffff8116e5b4>] ? pagefault_out_of_memory+0x44/0xc0
[610420.813712] [<ffffffff815a65b8>] ? page_fault+0x28/0x30
[610426.700118] Task in /kubepods/pod401c0ec6-97d9-11e7-adb9-021e5eba15b7/32c394ad4709430cddb71194ea406d598b82159
[610426.711591] memory: usage 16384kB, limit 16384kB, failcnt 629216
[610426.715097] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0
[610426.719063] kmem: usage 0kB, limit 9007199254740988kB, failcnt 0
[610426.722823] Memory cgroup stats for /kubepods/pod401c0ec6-97d9-11e7-adb9-021e5eba15b7: cache:0KB rss:0KB rss_huge:0KB
[610426.749544] Memory cgroup stats for /kubepods/pod401c0ec6-97d9-11e7-adb9-021e5eba15b7/32c394ad4709430cddb71194ea406d598b82159c2715d42b64927ac5e67f0228:
[610426.765166] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
[610426.769304] [ 7288] 0 7288 257 1 4 2 0 -998 pause
[610426.773553] [20340] 0 20340 11395 7113 27 5 0 -998 kube-state-metr
[610426.778753] Memory cgroup out of memory: Kill process 20340 (kube-state-metr) score 769 or sacrifice child
…
…
Approx. 3 billion lines of garbage logs
to analyse for fun

99. @koenighotze
Not enough?

100. @koenighotze
Multi-Cloud

101. @koenighotze
Rails
TODO app
Rails
TODO app
Rails
TODO app

102. @koenighotze
PolyWTF works for ops

103. @koenighotze

104. @koenighotze
Still successful?
Process to the rescue

105. @koenighotze
Checklists (Excel or PPT!)
Arbitrary naming conventions
External (but unknown)
stakeholders
Last minute documents
(Templates, Fonts!)

106. @koenighotze
Be proud of complexity, we are
engineers for a reason
Ops tooling puts JavaScript to
shame
If Twitter runs on it, so should you
Combine with Meat cloud

107. @koenighotze
4

108. @koenighotze
The SPA monolith

109. @koenighotze
Customer REST Credit RESTAccount REST
Sophisticated portal
frontend

110. @koenighotze
Add a mandatory
social-number field to the
customer

111. @koenighotze
But not that mandatory. E.g. if
the customer is not committed, yet

112. @koenighotze
Customer Domain Credit DomainAccount Domain
Comprehensive UI Domain
Model

113. @koenighotze
Domain is spread around
Customer REST Credit RESTAccount REST

114. @koenighotze
Where should the logic go?

115. @koenighotze

116. @koenighotze
Advantage?

117. @koenighotze
UI-driven changes
Customer Domain Credit DomainAccount Domain
Sophisticated portal
frontend

118. @koenighotze
Microservices are REST only (hint: service!)
Meet people while discussion ownership
and conflicting features
Keep the testers busy with side effects
Single point of failure - blame the script
kids working on the UI

119. @koenighotze
3

120. @koenighotze
The decision monolith

121. @koenighotze
– Some Developer who does not understand ENTERPRISE IT!
“But we need Java 7…it is 2017 after all!”

122. @koenighotze
Agile
Rx
Microservice
Actors Augmented realityCloud
AWS
WebRTC IoT
Blockchain
Developer madness

123. @koenighotze
Architecture & Design
Sieve of Sanity

124. @koenighotze
JEE
WebSphere
Agile
Rx
Microservice
Actors Augmented realityCloud
AWS
WebRTC IoT
Blockchain

125. @koenighotze
Frustrate your coworkers
Irritate motivated colleagues
Do not play around with technology
If you are older than 30…play the “This is
_real_ enterprise, kid” card
Ivory towers are beautiful

126. @koenighotze
2

127. @koenighotze
The business
monolith

128. @koenighotze
Business people cannot understand
complex any architecture

129. @koenighotze
Credit Account Payment
… … …
Product Toolset Something

130. @koenighotze
Let’s force the user to
always enter a valid address

131. @koenighotze
No, the customer
should be able to enter partial
addresses

132. @koenighotze
Requirements Ping Pong

133. @koenighotze
Credit Account Payment
Product Toolset Something
?????

134. @koenighotze
Change management

135. @koenighotze
Requirements management

136. @koenighotze
Day rates!

137. @koenighotze
Bonus:
works with ops, too

138. @koenighotze
Credit Account Payment
… … …
Product Toolset Something
What the…?

139. @koenighotze
Development
Ops
Business
Waterscrumfall

140. @koenighotze
Avoid X-functional teams
Slow means more time for Twitter!
The servant of many kings is a free man
Change / Requirement mgmt. are your
allies
Conway’s law works both ways

141. @koenighotze
1

142. @koenighotze
The staffing monolith

143. @koenighotze
It is easier to search for React
developers than for general
craftswomen

144. @koenighotze
Frontend
Development
Service
Development
DevOps Team
Test
Business

145. @koenighotze
But…why?

146. @koenighotze
Frequent Flyer Program
Rail-Bonus Program
Hotel Credit Points

147. @koenighotze
Dysfunctional setup in
3 easy steps

148. @koenighotze
Step 1
Establish distributed
communication

149. @koenighotze
Magic
Specification
Success
Template

150. @koenighotze
Step 2
Setup firewalls

151. @koenighotze
Frontend Microservice
Service Microservice
Persistence Stuff
Team A
Team B
Team C

152. @koenighotze
Step 3
Use bureaucracy

153. @koenighotze
The frontend team
The service team
The database guys
“Product
Owner”
“Product
Owner”
“Product
Owner”

154. @koenighotze
Day rates!

155. @koenighotze
But…what if this
works?

156. @koenighotze
Enter the fakeholder!

157. @koenighotze

158. @koenighotze
“Product
Owner”
“UX
Product
Owner”
“Back
Product
Owner”
“Overall
Product
Owner”
“Product
Owner”
“Product
Owner”
Doesn’t
care
Cannot
decide

159. @koenighotze
Even sounds “agile”!

160. @koenighotze
Technical benefits?

161. @koenighotze
Frontend Microservice
Service Microservice
Persistence Stuff
Shared
stability
Shared
release
Convoluted
domains

162. @koenighotze
Programmers do not like other people
Nobody should have a big picture, keep
them in the dark
“War rooms” sound really important
Avoid real ownership

163. @koenighotze
And: In case of bugs…

164. @koenighotze
…it’s always
the other team’s fault!

165. @koenighotze
Wrapping up….

166. @koenighotze
The key to failure is
the hidden monolith

167. @koenighotze
Microservices mean work
Brownfield - The perfect excuse
Think of the organisational impact
Never involve Ops or Business
Microservices are a fad
Embrace the monolith

168. @koenighotze
Thank you!
Questions? Disagree? Let’s talk!
@Koenighotze