Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The 36th Chamber
of Shaolin
Improve Your Microservices Kung Fu
in 36 Easy Steps
freiheit.com technologies - Hamburg, Septe...
This talk derives from more
than 17+ years of experience
building large-scale Internet
systems.
© freiheit.com
2
Internally we are calling these
principles BLACK & WHITE.
© freiheit.com
3
Black & White principles are
principles that are true most
of the time.
© freiheit.com
4
This talk is about
Microservices.
But as you will see, most of
these principles are true for all
kinds of systems.
© freih...
Don’t build your own cloud
or cluster management.
People with far more experience have already done this.
Use what is give...
Don’t build your own data
center and availability zone
failover.
Again: People with far more experience have already
done ...
Don’t roll your own service
discovery.
People with far more experience … - you know the spiel.
© freiheit.com
8
One container. One service.
This is how you should use Docker.
© freiheit.com
9
I am a big fan of Kubernetes
running on GCE or AWS.
© freiheit.com
10
A microservice provides a set
of remote procedure calls that
are closely related to each
other (high cohesion).
● It shoul...
Service API
Receives requests
Returns responses
No state,
no session,
“non-sticky”
A MICROSERVICE
© freiheit.com
12
A microservice starts up fast.
Start-up time < 10 sec.
© freiheit.com
13
A microservice has a low
memory footprint.
It doesn’t cache large amounts of data.
© freiheit.com
14
A microservice has well
defined resource constraints.
Memory/heap limits, CPU limits, swap disabled
(or constrained), etc....
A microservice has a fast
response time.
Like 15ms/50th, 50ms/95th and 100ms/99th percentile.
© freiheit.com
16
A microservice has a low
error rates.
There will be some 404, 500, 503, etc. But not many. And
you should know and underst...
A microservice differentiates
between technical errors and
usage errors.
Technical is like out of memory, usage is like re...
A microservice must be
monitored at all times.
Monitor response time, request count and errors.
Define thresholds and aler...
© freiheit.com
20
You must be able to trace a
request from the client down
through all services calls.
Each request needs to have a unique I...
A microservice never shares
its persistent state (database)
with any other microservice.
Exception: Importer service write...
It is totally okay to use a
relational database.
Don’t shoot yourself in the foot with Cassandra and NoSQL.
Rather use rea...
A microservice should never
include both batch and online
processing at the same time.
Separate batch from online processi...
Microservices will and can
call other microservices.
Call as few other microservices as possible.
Prevent circular calls.
...
A microservice answers
requests asynchronously.
© freiheit.com
26
One thread
handling
requests fast,
handing off to
worker threads
Receives requests
Returns responses
Many worker
threads r...
A microservice sends requests
to other microservices
asynchronously.
Take a look at reactive programming Rx* or other stuf...
ASYNC CALLING OTHER SERVICES
Async
Async Async
© freiheit.com
29
If a downstream service fails
your service should still work.
Use graceful degradation to return useful responses
as long ...
If you use REST,
use it asynchronously.
This doesn’t mean “message queues”.
© freiheit.com
31
It is better to use specialized
RPC protocols instead of
REST that can handle
interface changes better.
Like gRPC, Thrift,...
Each request needs a
reasonable time-out and
must be cancelable.
Like with gRPC/Context.
© freiheit.com
33
All asynchronous downstream
work resulting from a request
must be cancelable, too.
Like with gRPC/Context.
© freiheit.com
...
A request must be repeatable
providing the same result.
Requests must be idempotent.
© freiheit.com
35
Log the right amount of data
and create actionable,
meaningful log entries.
Or you will drown in your logs with unusable i...
If you need two or more
requests to succeed, you need
to implement retry/rollback on
error.
This is a distributed system. ...
No branches. All new code is
in the HEAD branch.
Update your microservices
with very small commits.
Few large commits are ...
You must be able to identify
which version of your code is
currently running in production.
You can use the commit rev as ...
Design from the beginning
how to run integration tests.
How to run a test over dozens or hundreds of services,
all having ...
Share only code where a
change doesn’t force
everybody to update all their
services.
Exception: Crypto, security-related c...
No commit without tests.
The “good feelings” start at code coverage > 85 %.
70 % unit tests, 20 % integration tests, 10 % ...
Actively manage your
microservices’ lifecycle.
Handle SIGTERM, readiness & lifeness probes, etc.
© freiheit.com
43
Deploy new versions to a
canary context first.
In Kubernetes you can use namespaces to separate
contexts. Measure and comp...
Backend-for-frontend helps
you to reduce the number of
requests and bandwidth
consumption of your clients.
Fewer requests....
Simulate real-world traffic
for load testing.
One option is to record live traffic for replay.
See https://github.com/buge...
Run a chaos monkey in
production.
It randomly kills services and you will learn if everything is
really under control.
© f...
DevOps doesn’t work if it
gets big.
You need dedicated Site Reliability Engineers (SREs) to
team up with your software eng...
I lied. There were
more than 36 steps ...
Thank you.
freiheit.com technologies gmbh 2016
Want to work with us and create t...
Upcoming SlideShare
Loading in …5
×

The 36th Chamber of Shaolin - Improve Your Microservices Kung Fu in 36 Easy Steps

1,199 views

Published on

We all know, that software development can not be learned by following checklists and that writing distributed systems is not easy. But wouldn't it be nice, if you had a checklist of the most important things to keep in mind when you start building your own microservices project?
This is what my talk is about. I presented it at codetalks 2016 in Hamburg, Germany. (YouTube video of the talk will follow soon).
No technical details about how to use Kafka or which cloud manager is the best. Just some general principles that will help you to focus on the right things from the beginning. You don't have to agree on all principles, but I am pretty sure that if you just focus on 80% you will be better off. Feedback and comments are very welcome so this list can grow (and maybe even shrink) based on new project experiences.

Published in: Software
  • Be the first to comment

The 36th Chamber of Shaolin - Improve Your Microservices Kung Fu in 36 Easy Steps

  1. 1. The 36th Chamber of Shaolin Improve Your Microservices Kung Fu in 36 Easy Steps freiheit.com technologies - Hamburg, September 29, 2016 code.talks 2016 - Keynote Track
  2. 2. This talk derives from more than 17+ years of experience building large-scale Internet systems. © freiheit.com 2
  3. 3. Internally we are calling these principles BLACK & WHITE. © freiheit.com 3
  4. 4. Black & White principles are principles that are true most of the time. © freiheit.com 4
  5. 5. This talk is about Microservices. But as you will see, most of these principles are true for all kinds of systems. © freiheit.com 5
  6. 6. Don’t build your own cloud or cluster management. People with far more experience have already done this. Use what is given. © freiheit.com 6
  7. 7. Don’t build your own data center and availability zone failover. Again: People with far more experience have already done this. It is difficult. Use what is given. © freiheit.com 7
  8. 8. Don’t roll your own service discovery. People with far more experience … - you know the spiel. © freiheit.com 8
  9. 9. One container. One service. This is how you should use Docker. © freiheit.com 9
  10. 10. I am a big fan of Kubernetes running on GCE or AWS. © freiheit.com 10
  11. 11. A microservice provides a set of remote procedure calls that are closely related to each other (high cohesion). ● It should be as small as needed. ● It should do only one thing and do it well. ● It should do 'one thing' entirely. ● It's better to split later than to build it too small at first. ● A split should be simple. © freiheit.com 11
  12. 12. Service API Receives requests Returns responses No state, no session, “non-sticky” A MICROSERVICE © freiheit.com 12
  13. 13. A microservice starts up fast. Start-up time < 10 sec. © freiheit.com 13
  14. 14. A microservice has a low memory footprint. It doesn’t cache large amounts of data. © freiheit.com 14
  15. 15. A microservice has well defined resource constraints. Memory/heap limits, CPU limits, swap disabled (or constrained), etc. © freiheit.com 15
  16. 16. A microservice has a fast response time. Like 15ms/50th, 50ms/95th and 100ms/99th percentile. © freiheit.com 16
  17. 17. A microservice has a low error rates. There will be some 404, 500, 503, etc. But not many. And you should know and understand why they are happening. © freiheit.com 17
  18. 18. A microservice differentiates between technical errors and usage errors. Technical is like out of memory, usage is like receiving wrong requests from clients the service is unable to reply and therefore denies to answer. © freiheit.com 18
  19. 19. A microservice must be monitored at all times. Monitor response time, request count and errors. Define thresholds and alerts when thresholds are reached. © freiheit.com 19
  20. 20. © freiheit.com 20
  21. 21. You must be able to trace a request from the client down through all services calls. Each request needs to have a unique ID. © freiheit.com 21
  22. 22. A microservice never shares its persistent state (database) with any other microservice. Exception: Importer service writes to database. Online service reads from / writes to database. © freiheit.com 22
  23. 23. It is totally okay to use a relational database. Don’t shoot yourself in the foot with Cassandra and NoSQL. Rather use really flat data models, no joins, no complicated queries in a relational database. © freiheit.com 23
  24. 24. A microservice should never include both batch and online processing at the same time. Separate batch from online processing to make sure that your online services aren’t getting slow when the batch processing starts. © freiheit.com 24
  25. 25. Microservices will and can call other microservices. Call as few other microservices as possible. Prevent circular calls. © freiheit.com 25
  26. 26. A microservice answers requests asynchronously. © freiheit.com 26
  27. 27. One thread handling requests fast, handing off to worker threads Receives requests Returns responses Many worker threads running in the background ASYNC REQUEST HANDLING © freiheit.com 27
  28. 28. A microservice sends requests to other microservices asynchronously. Take a look at reactive programming Rx* or other stuff ... © freiheit.com 28
  29. 29. ASYNC CALLING OTHER SERVICES Async Async Async © freiheit.com 29
  30. 30. If a downstream service fails your service should still work. Use graceful degradation to return useful responses as long as possible. © freiheit.com 30
  31. 31. If you use REST, use it asynchronously. This doesn’t mean “message queues”. © freiheit.com 31
  32. 32. It is better to use specialized RPC protocols instead of REST that can handle interface changes better. Like gRPC, Thrift, Finagle. © freiheit.com 32
  33. 33. Each request needs a reasonable time-out and must be cancelable. Like with gRPC/Context. © freiheit.com 33
  34. 34. All asynchronous downstream work resulting from a request must be cancelable, too. Like with gRPC/Context. © freiheit.com 34
  35. 35. A request must be repeatable providing the same result. Requests must be idempotent. © freiheit.com 35
  36. 36. Log the right amount of data and create actionable, meaningful log entries. Or you will drown in your logs with unusable information. © freiheit.com 36
  37. 37. If you need two or more requests to succeed, you need to implement retry/rollback on error. This is a distributed system. It can fail anytime. © freiheit.com 37
  38. 38. No branches. All new code is in the HEAD branch. Update your microservices with very small commits. Few large commits are more unpredictable than lots of small commits that are pushed continuously to your cluster. Your monitoring and alerting will protect you. © freiheit.com 38
  39. 39. You must be able to identify which version of your code is currently running in production. You can use the commit rev as a Docker release tag. © freiheit.com 39
  40. 40. Design from the beginning how to run integration tests. How to run a test over dozens or hundreds of services, all having separate databases? © freiheit.com 40
  41. 41. Share only code where a change doesn’t force everybody to update all their services. Exception: Crypto, security-related code, etc. © freiheit.com 41
  42. 42. No commit without tests. The “good feelings” start at code coverage > 85 %. 70 % unit tests, 20 % integration tests, 10 % end-to-end tests. © freiheit.com 42
  43. 43. Actively manage your microservices’ lifecycle. Handle SIGTERM, readiness & lifeness probes, etc. © freiheit.com 43
  44. 44. Deploy new versions to a canary context first. In Kubernetes you can use namespaces to separate contexts. Measure and compare key KPIs with your production context. © freiheit.com 44
  45. 45. Backend-for-frontend helps you to reduce the number of requests and bandwidth consumption of your clients. Fewer requests. The least amount of data transferred. © freiheit.com 45
  46. 46. Simulate real-world traffic for load testing. One option is to record live traffic for replay. See https://github.com/buger/gor. © freiheit.com 46
  47. 47. Run a chaos monkey in production. It randomly kills services and you will learn if everything is really under control. © freiheit.com 47
  48. 48. DevOps doesn’t work if it gets big. You need dedicated Site Reliability Engineers (SREs) to team up with your software engineers. © freiheit.com 48
  49. 49. I lied. There were more than 36 steps ... Thank you. freiheit.com technologies gmbh 2016 Want to work with us and create the future? Talk to us! jobs@freiheit.com

×