Downloaded 137 times
Uploaded byDaniel Bryant
J1 2015 "Building a Microservice Ecosystem: Some Assembly Still Required"
The document discusses the challenges and considerations in building a microservice ecosystem, including aspects such as local development, testing paradigms, and infrastructure planning. Key elements like build pipelines, deployment strategies, service discovery, and observability are emphasized. The author encourages careful planning to avoid common pitfalls and to leverage existing platforms and tools instead of creating a custom solution.
More Related Content
Similar to J1 2015 "Building a Microservice Ecosystem: Some Assembly Still Required"
J1 2015 "Building a Microservice Ecosystem: Some Assembly Still Required"
- 1. Building a MicroserviceEcosystem: Some Assembly (Still) Required Daniel Bryant @danielbryantuk
- 2. Agenda • What isa microservice platform/ecosystem • Local development of services can be hard • Think about your build pipeline(s) • Testing requires a paradigm shift • Consider the infrastructure • Plan for the inevitable… 28/10/2015 @danielbryantuk
- 3. Who Am I? •London Java Community Associate • Adopt OpenJDK and JSR • InfoQ Editor, DZone MVB, Voxxed 28/10/2015 @danielbryantuk • Principal Consultant at OpenCredo • Technical/digital transformation • Java, Golang, CI/CD, DevOps • Microservices, cloud and containers • Maintainer of muservicesweekly.com
- 4. So, What isa Microservice? “Loosely coupled service oriented architecture with bounded contexts” Adrian Cockcroft “Applications that fit in your head” James Lewis 28/10/2015 @danielbryantuk
- 5. What do Imean by ‘ecosystem’? • Build – Local development, pipelines and integration • Test – From local integration to E2E • Deploy • Operate • Observe – Monitoring/logging/alerting 28/10/2015 @danielbryantuk
- 6. Adrian Cockcroft’s Thoughts 28/10/2015@danielbryantuk www.slideshare.net/adriancockcroft/microxchg-microservices
- 7. Alexis Richardson’s Thoughts 28/10/2015@danielbryantuk gotocon.com/goto-london-2015/#!#schedulePopupExtras-7011
- 8.
- 9.
- 10.
- 11. Core (up front?)Decisions • Platform – Cloud Foundry – Amazon ECS, Deis, Flynn – Docker Swarm (Tutum) – Kubernetes – Mesos – HashiCorp Nomad – DIY (maybe leverage Netflix’s stuff?) 28/10/2015 @danielbryantuk
- 12. Core (up front?)Decisions • Packaging – Services/tasks/jobs – ’buildpacked’ – VMs – Containers • Is Docker a de facto standard?… 28/10/2015 @danielbryantuk
- 13.
- 14. Developing Locally: TheBasics • GitHub’s Boxen (Puppet) • Pivotal’s Sprout (Chef) • Mac-dev-playbook (Ansible) • Vagrant (Otto?) • Docker Compose (Docker machine) 28/10/2015 @danielbryantuk
- 15. Developing Locally • Thenaïve approach – Replication of env per service – …and dependencies and data stores and... – Soon gets crazy • Local profiles + mocking/stubbing – Spring profiles + Mockito etc 28/10/2015 @danielbryantuk
- 16. Developing Locally • Servicevirtualisation – Mountebank, Wiremock (Saboteur), Mirage • ‘Production-in-a-box’ (IFTTT) – Docker Compose, Vagrant, cf_nise_installer • Environment leasing – Create your own env (e.g. Hailo) 28/10/2015 @danielbryantuk
- 17.
- 18. Create a Pipeline 28/10/2015@danielbryantuk
- 19. Multi-service / Multi-pipeline? 28/10/2015@danielbryantuk ?
- 20.
- 21.
- 22. Testing Basics • TobyClemson’s article – martinfowler.com/articles/microservice-testing 28/10/2015 @danielbryantuk
- 23. Multi-service / Multi-pipeline? 28/10/2015@danielbryantuk ?
- 24. Pipeline • Multiple pipelines? –Beware of rubber stamping (distributed monolith) – ‘semver’ if you must (semver.org) e.g. 1.2.1 • Shared staging – Critical path testing (why not prod?) • Ensured consistency – Consumer-based contracts – Backwards compatibility 28/10/2015 @danielbryantuk
- 25. Integration Testing • Contracts –Pact-JVM github.com/DiUS/pact-jvm – PACT broker github.com/bethesque/pact_broker – Examples: github.com/mstine/microservices-pact 28/10/2015 @danielbryantuk
- 26. My Opinions • BDDservices API – Serenity BDD • Contract tests (failure is a conversation) • Component test and unit test (as normal) – Maven surefire/failsafe • BDD critical paths throughout application – Including API journey 28/10/2015 @danielbryantuk
- 27. Final Words onTesting • Don’t forget the “ilities” • Security / reliability – ZAP (from the OWASP team) – github.com/continuumsecurity/bdd-security • Performance / scalability – Jmeter (Jenkins Performance plugin) – flood.io 28/10/2015 @danielbryantuk
- 28.
- 29. Separate Deploy andRelease • Feature flags – Difficult at scale (and distribution) – Enable at ingress • Incremental (phased) rollout • Canary vs blue/green • Avoid datastore migrations (if possible) 28/10/2015 @danielbryantuk
- 30.
- 31. Centralise Configuration • Consul& consul-template • Etcd/ZK & confd • Netflix Archaius • Spring Cloud • Watch out for application rollback! 28/10/2015 @danielbryantuk
- 32.
- 33. Building Blocks • Standardiseon an OS – Amazon Linux vs mainstream distros • HashiCorp Terraform – VMWare? cloud-init and vcloud-tools – “Boot my secure government cloud” • “CAPS” – Chef, Ansible, Puppet, SaltStack – Automated sysadmin 28/10/2015 @danielbryantuk
- 34. DevOps and ProgrammableInfra • It involves programming… • Introduce SOLID principles • Good CI/CD principles – Gitflow etc 28/10/2015 @danielbryantuk
- 35. Service Discovery • External –HAProxy / nginx / ELB etc • Client-side – Netflix Ribbon (with Prana) – ‘Baker Street’ (extending SmartStack) – srv-router • Kubernetes and CF are good to go 28/10/2015 @danielbryantuk
- 36.
- 37.
- 38.
- 39. Service Discovery • External –HAProxy / nginx / ELB etc • Client-side – Netflix Ribbon (with Prana) – ‘Baker Street’ (extending SmartStack) – srv-router • Kubernetes and CF are good to go 28/10/2015 @danielbryantuk
- 40.
- 41. Monitoring and People 28/10/2015@danielbryantuk www.infoq.com/news/2015/06/too-big-to-fail
- 42. Start with theBasics • Health checks – Coda Hale (DropWizard) Metrics – Spring Boot actuator • KPIs for apps (and business) – Assertions / invariants – Throughput – Queue length 28/10/2015 @danielbryantuk
- 43. Logging • “What everyengineer should know” • “10 Tips for Proper Application Logging” • ElasticSearch-Logstash-Kibana (ELK) – Buffer/proxy log sending or… – Mount directory into container 28/10/2015 @danielbryantuk
- 44.
- 45. Distributed Tracing • github.com/daniel-bryant-uk/correlation-id-async •MDC logging (logback.qos.ch/manual/mdc.html ) • Zipkin/Brave (github.com/openzipkin/docker-zipkin ) 28/10/2015 @danielbryantuk
- 46. Monitoring • Push – SpringBoot actuator e.g. InfluxDbExporter • Pull – E.g. Telegraf (shout to Tareq Abedrabbo) • InfluxDB vs prometheus vs graphite vs opentsdb • Information radiators – Aggregate vs individual 28/10/2015 @danielbryantuk
- 47. Aggregation: Sick Cattle,Not Sick Pets
- 48.
- 49. Problems? • Rob Ewaschuk’s“Philosophy on Alerting” • Brendan Gregg’s USE method – “check utilization, saturation, and errors.” • “DevOps Troubleshooting” – Kyle Rankin 28/10/2015 @danielbryantuk
- 50. Summary • Looked at“microservice platforms” • Local development of 3+ services is hard • Think about your build pipeline(s) • Testing requires a paradigm shift • Consider the infrastructure • Plan for the inevitable issues… • Try not to create your own platform... 28/10/2015 @danielbryantuk
- 51.
- 52. Thanks Feedback is alwayswelcomed! daniel.bryant@opencredo.com @danielbryantuk www.muservicesweekly.com 28/10/2015 @danielbryantuk
Editor's Notes
- #47 https://github.com/codecentric/spring-boot-starter-batch-web/blob/master/src/main/java/de/codecentric/batch/metrics/InfluxdbMetricsExporter.java