Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A Microservices Reference Architecture


Published on

A highly opinionated, strawman reference architecture for success with Microservices.

Presented at the fifth Sydney Microservices Meetup on Feb 18th 2015.

Published in: Technology

A Microservices Reference Architecture

  1. 1. icreseruices A Reference Architecture @yuumehn Co-founder, Sixtree
  2. 2. Microservices I _¢, 0i"“" A, ‘ Reference Architecture @yaamehn Co-founder, Sixtree
  3. 3. The Prerequisites
  4. 4. The Prerequisites Trading code complexity for operational complexity http: //blog. ingineering. it/ post/110741562854/microservices-have-you-met-devops
  5. 5. The Prerequisites Trading code complexity for operational complexity Supporting Infrastructure Needed as Proportion Of Thing http: //blog. ingineering. it/ post/110741562854/microservices-have-you-met-devops
  6. 6. The Prerequisites Trading code complexity for operational complexity S I N P O T http: //blog. ingineering. it/ post/110741562854/microservices-have-you-met-devops
  7. 7. _ I’ve got 99 problems, MQNQLI-[Hi but not distributed systems ones
  8. 8. We shall now talk about these weird grey lines
  9. 9. A Friendly Guide to Vamerfs 3 LOQ
  10. 10. A Friendly Guide to Vamerfs 3 LOQ (Levels of Conviction)
  11. 11. A Friendly Guide to Vamerfs 3 LOG (Levels of Conviction)
  12. 12. A Friendly Guide to Vamen’s 3 LOC (Levels of Conviction) Kramer
  13. 13. A Friendly Guide to Vamerfs 3 LOC (Levels of Conviction)
  14. 14. l 9&9. what’s that? 51:?
  15. 15. 1. Use Docher Strong Configured via Ecosystem Environment _/ __{—— VM + Runtime + Binary Distribution Emerging p°| "¢Gb'° —J"'{i Standard Package Language Minimal Dependencies Agnostic
  16. 16. Only two types of containers 4:? /_: ’j Stateless Stateful J .1.. .’ / T_. E3
  17. 17. Even databases are microservices ‘€37 T
  18. 18. 2. Configure Using Environment Z__. _ {M Template Configuration + Environment + Startup Script = Configured System Prefer consistent dependency on environment variables over: ° Properties files ° Command line arguments ° Other configuration options On startup, the container should: - Configure service based on environment ° Then start the service
  19. 19. Example Template Systems Ruby-based with ERB Python-based with linja2 Go-based with Go templates Shell-based with sed Doesn’t matter, choose one, teach the team, stick with it
  20. 20. 2a. Configure Dynamically Using Consul 4:" Use consul-template to: I fl V‘ t“‘” ° Watch consul for configuration J changes ° Reload files and services on changes For scenarios where dynamic reloads are appropriate
  21. 21. 3. Centralise Logging ljjjfl
  22. 22. 3a. Use Logstash & Elasticsearch
  23. 23. 4. Enable Distributed Tracing *1/5: ii? 57%? ?? my , pg’; / // ’_{: jm—>
  24. 24. But don’t be naive OOO FOIL comaoert EE ‘'5 Look to Dapper (Google) and Ziphin (Twitter) for inspiration Unfortunately no good language neutral approach exists. We end up writing our own client libraries to facilitate this. More on client libraries later!
  25. 25. 5. Standardise Monitoring , _/_____. )lJ
  26. 26. 5a. Expose Stats for Pulling T/ ‘:—’_ (T [if T ‘P CIRCQNUS L2) collectd _, _/_ Expose / stats endpoint Periodically polls all Docher containers Insert your favourite external Return standardised JSON structure Retrieves stats from / stats monitoring service here Provide health and statistics Retrieves other details from daemon Pushes container and host data to Circonus
  27. 27. Your Friendly Docher Daemon The Docher daemon running on each host exposes (at least) the following for all running and stopped containers through its API: ° Environment variable configuration - CPU, memory and other vital stats - Exposed ports Program built to work against Docher daemon API can use this information regardless of what’s inside the daemon TIP: Use environment variables such as STATS_ENABLED, STATS_PORT and STATS_UR| to signal the monitoring capabilities of a container to the outside world
  28. 28. e. Enable Events
  29. 29. ea. Use Kafka as the Unified Log / >1 7 L7 | Alternatives: - R bb'tMQ / I L. ..) - A: riv; Mo / ‘ l - Any Event Store (ideally with Push capabilities) Distributed durable _. , partitioned, falult-tolerdnt, lightning fast
  30. 30. ob. Use Standard Event Structures I 7; Define organisation wide event structure and metadata Define standard for event ‘scope’, for example: - Does the event contain a snapshot of the ‘new’ entity at that time? - Or just the fact that ‘something happened’ with a reference URI for callback? - Or a full trace of the ‘before’ and ‘after’ and all relevant changes? Model domain events as part of service design Publish available events to a centralised registry (ergh. .. no good standard for this yet)
  31. 31. ,/ U m , L
  32. 32. Ta. lust Use HTTP Seriously
  33. 33. 7b. Never go full REST vouius1_n_nn tr - IIEII , I11 0 I110 gcnoratorznel
  34. 34. 7c. Have a Data Format Standard ISON API is good XML schemas still work Pick one or two
  35. 35. id. Have a Spec and a Portal We like RAML
  36. 36. 7e. Use Consul, Registrator & DNS E 1. Discover 2_ Register . /_‘_”_"_“ é__ / _n: :* ’T‘ 4. Call (J ' nul __, 4. L,
  37. 37. 7f. Provide Client Libraries ° Many things are very hard to get right on the client side: ° Retries, backoff and circuit breakers ° Versioning and Poste| ’s Law ° Serialisation and deserialization - Reading documentation © ° Either: ° Use a framework like Finagle to make this much simpler (but lock you into NM), or ° Standardise a way of creating client libraries for each service that implements the above concerns
  38. 38. 7g. Consumer Driven Contracts Use pact or pacto to ensure your consumers don’t break on upgrades Or, like, provide client libraries The main point is that you want to know the new service isn’t going to break anything Testing, synthetic transactions, easy rollbacks, resilient clients etc all work . , I '
  39. 39. 8. Use Continuous Delivery f3’E; _'fl* oc>C>C>©O©O/
  40. 40. Docker makes lfe simpler BUILD RELEASE PROMOTE mvn deploy mvn vcrsionstsul $vERSION echo "Ready for Production” mvn ClOCl<El“IlJ3Cl<fl°€ § docker‘ tag $VERSION docker tag lates: docker push $VERSION 1» docker push latest mvn scmztag Auto deploys to DEV Ready for deployment Ready for deployment with to SIT and UAT with to PRD with docker pull latest docker pull $VERSION docker pull $VERSION d0CkeI" Pun latest docker run $VERSION docker run $VERSION
  41. 41. 8. Use Docker Orchestration ’’ ‘a Q E e E) 1}] EEIE llilfllfl . :j—. _——-
  42. 42. All Together Now
  43. 43. The Production Line