J1 2015 "Building a Microservice Ecosystem: Some Assembly Still Required"
•Download as PPTX, PDF•
9 likes•6,659 views
Microservice platforms are finally becoming a reality: Mesos, Kubernetes, and a whole bunch of PaaS-style offerings are available, but the reality is that these platforms still don’t provide everything you need in order to build a fully functional microservice ecosystem. Come to this session to learn about the essential deployment, orchestration, and glue components that often have to be self-assembled. The presentation begins by looking at deployment techniques and tools and examines where to test (QA, staging, or production), how to test (integration and contracts), and how to separate deployment and release. It then discusses orchestration, configuration, and service discovery. Finally it looks at essential glue such as logging, monitoring, and alerting.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to J1 2015 "Building a Microservice Ecosystem: Some Assembly Still Required"
Similar to J1 2015 "Building a Microservice Ecosystem: Some Assembly Still Required" (20)
More from Daniel Bryant
More from Daniel Bryant (20)
Recently uploaded
Recently uploaded (20)
J1 2015 "Building a Microservice Ecosystem: Some Assembly Still Required"
- 1. Building a Microservice Ecosystem: Some Assembly (Still) Required Daniel Bryant @danielbryantuk
- 2. Agenda • What is a microservice platform/ecosystem • Local development of services can be hard • Think about your build pipeline(s) • Testing requires a paradigm shift • Consider the infrastructure • Plan for the inevitable… 28/10/2015 @danielbryantuk
- 3. Who Am I? • London Java Community Associate • Adopt OpenJDK and JSR • InfoQ Editor, DZone MVB, Voxxed 28/10/2015 @danielbryantuk • Principal Consultant at OpenCredo • Technical/digital transformation • Java, Golang, CI/CD, DevOps • Microservices, cloud and containers • Maintainer of muservicesweekly.com
- 4. So, What is a Microservice? “Loosely coupled service oriented architecture with bounded contexts” Adrian Cockcroft “Applications that fit in your head” James Lewis 28/10/2015 @danielbryantuk
- 5. What do I mean by ‘ecosystem’? • Build – Local development, pipelines and integration • Test – From local integration to E2E • Deploy • Operate • Observe – Monitoring/logging/alerting 28/10/2015 @danielbryantuk
- 6. Adrian Cockcroft’s Thoughts 28/10/2015 @danielbryantuk www.slideshare.net/adriancockcroft/microxchg-microservices
- 7. Alexis Richardson’s Thoughts 28/10/2015 @danielbryantuk gotocon.com/goto-london-2015/#!#schedulePopupExtras-7011
- 8. Joe Beda’s Thoughts www.eightypercent.net/post/layers-in-the-stack.html 28/10/2015 @danielbryantuk
- 11. Core (up front?) Decisions • Platform – Cloud Foundry – Amazon ECS, Deis, Flynn – Docker Swarm (Tutum) – Kubernetes – Mesos – HashiCorp Nomad – DIY (maybe leverage Netflix’s stuff?) 28/10/2015 @danielbryantuk
- 12. Core (up front?) Decisions • Packaging – Services/tasks/jobs – ’buildpacked’ – VMs – Containers • Is Docker a de facto standard?… 28/10/2015 @danielbryantuk
- 14. Developing Locally: The Basics • GitHub’s Boxen (Puppet) • Pivotal’s Sprout (Chef) • Mac-dev-playbook (Ansible) • Vagrant (Otto?) • Docker Compose (Docker machine) 28/10/2015 @danielbryantuk
- 15. Developing Locally • The naïve approach – Replication of env per service – …and dependencies and data stores and... – Soon gets crazy • Local profiles + mocking/stubbing – Spring profiles + Mockito etc 28/10/2015 @danielbryantuk
- 16. Developing Locally • Service virtualisation – Mountebank, Wiremock (Saboteur), Mirage • ‘Production-in-a-box’ (IFTTT) – Docker Compose, Vagrant, cf_nise_installer • Environment leasing – Create your own env (e.g. Hailo) 28/10/2015 @danielbryantuk
- 18. Create a Pipeline 28/10/2015 @danielbryantuk
- 19. Multi-service / Multi-pipeline? 28/10/2015 @danielbryantuk ?
- 22. Testing Basics • Toby Clemson’s article – martinfowler.com/articles/microservice-testing 28/10/2015 @danielbryantuk
- 23. Multi-service / Multi-pipeline? 28/10/2015 @danielbryantuk ?
- 24. Pipeline • Multiple pipelines? – Beware of rubber stamping (distributed monolith) – ‘semver’ if you must (semver.org) e.g. 1.2.1 • Shared staging – Critical path testing (why not prod?) • Ensured consistency – Consumer-based contracts – Backwards compatibility 28/10/2015 @danielbryantuk
- 25. Integration Testing • Contracts – Pact-JVM github.com/DiUS/pact-jvm – PACT broker github.com/bethesque/pact_broker – Examples: github.com/mstine/microservices-pact 28/10/2015 @danielbryantuk
- 26. My Opinions • BDD services API – Serenity BDD • Contract tests (failure is a conversation) • Component test and unit test (as normal) – Maven surefire/failsafe • BDD critical paths throughout application – Including API journey 28/10/2015 @danielbryantuk
- 27. Final Words on Testing • Don’t forget the “ilities” • Security / reliability – ZAP (from the OWASP team) – github.com/continuumsecurity/bdd-security • Performance / scalability – Jmeter (Jenkins Performance plugin) – flood.io 28/10/2015 @danielbryantuk
- 29. Separate Deploy and Release • Feature flags – Difficult at scale (and distribution) – Enable at ingress • Incremental (phased) rollout • Canary vs blue/green • Avoid datastore migrations (if possible) 28/10/2015 @danielbryantuk
- 31. Centralise Configuration • Consul & consul-template • Etcd/ZK & confd • Netflix Archaius • Spring Cloud • Watch out for application rollback! 28/10/2015 @danielbryantuk
- 33. Building Blocks • Standardise on an OS – Amazon Linux vs mainstream distros • HashiCorp Terraform – VMWare? cloud-init and vcloud-tools – “Boot my secure government cloud” • “CAPS” – Chef, Ansible, Puppet, SaltStack – Automated sysadmin 28/10/2015 @danielbryantuk
- 34. DevOps and Programmable Infra • It involves programming… • Introduce SOLID principles • Good CI/CD principles – Gitflow etc 28/10/2015 @danielbryantuk
- 35. Service Discovery • External – HAProxy / nginx / ELB etc • Client-side – Netflix Ribbon (with Prana) – ‘Baker Street’ (extending SmartStack) – srv-router • Kubernetes and CF are good to go 28/10/2015 @danielbryantuk
- 39. Service Discovery • External – HAProxy / nginx / ELB etc • Client-side – Netflix Ribbon (with Prana) – ‘Baker Street’ (extending SmartStack) – srv-router • Kubernetes and CF are good to go 28/10/2015 @danielbryantuk
- 41. Monitoring and People 28/10/2015 @danielbryantuk www.infoq.com/news/2015/06/too-big-to-fail
- 42. Start with the Basics • Health checks – Coda Hale (DropWizard) Metrics – Spring Boot actuator • KPIs for apps (and business) – Assertions / invariants – Throughput – Queue length 28/10/2015 @danielbryantuk
- 43. Logging • “What every engineer should know” • “10 Tips for Proper Application Logging” • ElasticSearch-Logstash-Kibana (ELK) – Buffer/proxy log sending or… – Mount directory into container 28/10/2015 @danielbryantuk
- 45. Distributed Tracing • github.com/daniel-bryant-uk/correlation-id-async • MDC logging (logback.qos.ch/manual/mdc.html ) • Zipkin/Brave (github.com/openzipkin/docker-zipkin ) 28/10/2015 @danielbryantuk
- 46. Monitoring • Push – Spring Boot actuator e.g. InfluxDbExporter • Pull – E.g. Telegraf (shout to Tareq Abedrabbo) • InfluxDB vs prometheus vs graphite vs opentsdb • Information radiators – Aggregate vs individual 28/10/2015 @danielbryantuk
- 47. Aggregation: Sick Cattle, Not Sick Pets
- 49. Problems? • Rob Ewaschuk’s “Philosophy on Alerting” • Brendan Gregg’s USE method – “check utilization, saturation, and errors.” • “DevOps Troubleshooting” – Kyle Rankin 28/10/2015 @danielbryantuk
- 50. Summary • Looked at “microservice platforms” • Local development of 3+ services is hard • Think about your build pipeline(s) • Testing requires a paradigm shift • Consider the infrastructure • Plan for the inevitable issues… • Try not to create your own platform... 28/10/2015 @danielbryantuk
- 52. Thanks Feedback is always welcomed! daniel.bryant@opencredo.com @danielbryantuk www.muservicesweekly.com 28/10/2015 @danielbryantuk
Editor's Notes
- https://github.com/codecentric/spring-boot-starter-batch-web/blob/master/src/main/java/de/codecentric/batch/metrics/InfluxdbMetricsExporter.java