The way in which communication is handled within a cloud native application has changed over the past few years. Kubernetes has become the de facto platform infrastructure, and inter-service communication is now handled via a service mesh. This session will explore how to integrate the open source Ambassador Kubernetes API gateway and the Consul Connect service mesh into your Java apps. Learn about Kubernetes ingress and inter-service communication Understand the tradeoffs of using different technologies to implement cloud native communication Explore how these technologies integrate well – or not – with new and existing Java applications Hear about lessons learned in production, at 3 a.m., with lots of coffee

1. Cloud Native Communication:
Using API Gateway and Service Mesh with Java Apps
Daniel Bryant
Product Architect, Datawire

2. tl;dr
▪ Moving to cloud and containers (cloud native) brings benefits and challenges
– Ingress and service-to-service communications change
▪ API gateway handles ingress traffic: you don’t control the client
▪ Service mesh handles service-to-service comms: you influence the client
▪ You can implement new comms via two patterns
– Outside-in, using an API gateway
– Balkanization, using a service mesh on a segment of services

3. Product Architect at Datawire, Freelance Tech Consultant and Writer
Java Champion, avid reader, conference tourist
@danielbryantuk

4. Motivations: Acceleration
▪ Lead time
▪ Deployment frequency
▪ Mean time to restore (MTTR)
▪ Change fail percentage
CIOs: “We want to go faster, and not fall over
(and if it breaks we want to detect and fix it fast)”

5. App Modernisation
▪ Refactoring, repurposing, or consolidation of heritage software to align it
more closely with current business needs
▪ Decoupling applications from infrastructure
– Moving workloads to take advantage of cloud-based (AI) services
– Retiring old systems (saving infra/hosting costs)
– Reducing operational burden (e.g. toil and security patching)

6. App Modernisation
▪ Microservices!
▪ Cloud!
▪ Containers!
▪ Kubernetes!

7. App modernisation: Not an overnight thing

8. End-to-end communication

9. End-to-end communication

10. API Gateway

11. API Gateway: Edge proxy, ingress, ADC...
▪ Exposes internal services to end-users (often via multiple domains)
▪ Encapsulates backends: k8s, VMs, bare metal etc
▪ Focused on managing ingress (“north-south”) traffic
▪ You don’t control the client

12. API Gateway: Self-Serve Routing & Security
▪ Self-serve routing
– Traffic routing, splitting, and shaping (to dynamic backends)
– Release functionality (A/B, canary, dark launch etc)
▪ Security
– End-user authentication/authorization
– TLS termination, rate limiting, WAF, DDoS protection, etc

13. API Gateway Options

14. API Gateway: Decisions Points
Java-based Traditional Cloud Native
Configuration language Java Scripts Kubernetes CRDs,
YAML
Typical owners Developers Ops or API team Developers & Platform
Learning curve Low-Medium Medium-High Medium-High
Performance
(anecdotal)
Low (e.g. DIY)
High (e.g. Netty)
High High (especially in
volatile envs)
Adding functionality DIY Plugins, modules Plugins, modules
Biggest pain point Maintenance
Coupling/cohesion
Configuration New workflow (DevOps?)

15. Ambassador config

16. Service Mesh

17. “Service mesh”, you say?
https://twitter.com/cesarTronLozai/status/1175327326218915840
https://twitter.com/wm/status/1173350339946274816

18. Service Mesh: Proxy mesh, Fabric model...
▪ Exposes internal services to internal consumers
▪ Encapsulates service infra: across k8s, VMs, bare metal etc
▪ Dynamic routing for service-to-service (“east-west”) traffic
▪ You generally control the client (or at least can influence this...)

19. Exploring end-to-end communication

21. Service Mesh: Three Pillars
▪ Observability
– “Golden signals”: latency, errors, traffic, saturation (USE, RED)
– Both global and service-to-service
▪ Reliability
– Abstracting health checks, retries, circuit breakers etc.
– Providing sane default to protect system
▪ Security
– Authn/z propagation, mTLS, ACLs, network segmentation

22. Service Mesh: Three Pillars
▪ Observability
– “Golden signals”: latency, errors, traffic, saturation (USE, RED)
– Both global and service-to-service
▪ Reliability
– Abstracting health checks, retries, circuit breakers etc.
– Providing sane default to protect system
▪ Security
– Authn/z propagation, mTLS, ACLs, network segmentation
https://www.infoq.com/podcasts/

23. Service Mesh Options

24. Service Mesh: Decisions Points
None:
Simple Service
Discovery
Java “Service Mesh”
(in-process)
Sidecar Service Mesh
(out-of-process)
Configuration Scripts Java Kubernetes CRDs, YAML
Typical owners Dev / ops Developers Platform
Learning curve Low Medium-High Medium-High
Performance
(anecdotal)
Highest High High
Comms-based
NFR support
DIY Circuit-breaking, retries,
bulkheads, timeouts
Circuit-breaking, reties,
bulkheads, timeouts
Cross-language Yes No (or not really) Yes
Biggest
pain point
Running support services,
Configuration
Running support services,
maintaining config (coupling)
Installation and
maintainence

25. Consul config

26. Lessons learned...

27. Typical Problems
▪ No clear use case
▪ Not working with the ops team…
▪ Turtles all the way down
▪ NFR-handling implemented
multiple places in stack

28. Migration tactics
▪ Outside in
– Start with a gateway
– Identify a endpoint/service
▪ Balkanization
– Start with a service mesh
– Identify a service segment
▪ Easy install
▪ Conceptually easy to understand
▪ Less intrusive for all platforms
▪ (Potentially) higher blast radius
▪ Less new functionality
▪ Potentially high value functionality
▪ “Easy” to deploy in Kubernetes
▪ Can support multi-cluster (beta)
▪ Operationally complex
▪ (Potentially) challenging to unwind
▪ Expectation management… :-)

29. bit.ly/2mr58C1

30. Explore in browser
https://instruqt.com/hashicorp/tracks/sock-shop-tutorial
Hat tip to:
Todd Radel, Nic Jackson & Eric Veld!

31. Conclusion
▪ Moving to cloud and containers (cloud native) brings benefits and challenges
– Ingress and service-to-service communications change
▪ API gateway handles ingress traffic: you don’t control the client
▪ Service mesh handles service-to-service comms: you influence the client
▪ You can implement new comms via two patterns
– Outside-in, using an API gateway
– Balkanization, using a service mesh on a segment of services

32. References
▪ Context:
– https://www.infoq.com/articles/api-gateway-service-mesh-app-modernisation/
▪ Reference:
– https://www.getambassador.io/user-guide/consul-connect-ambassador/
– https://www.getambassador.io/user-guide/consul/
– https://www.consul.io/docs/platform/k8s/ambassador.html
– https://www.hashicorp.com/blog/hashicorp-consul-supports-microsoft-s-new-service-mesh-framework
Experiment in an Instruqt sandbox: https://instruqt.com/hashicorp/tracks/sock-shop-tutorial
Code examples: https://github.com/emojify-app

33. Copyright © 2019 HashiCorp
Thanks!
@danielbryantuk

34. Copyright © 2019 HashiCorp
Bonus

35. App Modernisation: It’s a balancing act
Bare Metal / DIY VMs Hybrid Cloud Only
Private Cloud Only Public Cloud OnlyHybrid

36. https://blog.christianposta.com/microservices/api-gateways-are-going-through-an-identity-crisis/

37. https://blog.envoyproxy.io/service-mesh-data-plane-vs-control-plane-2774e720f7fc
Control planes and data planes
Data plane
Control plane

38. Control planes: Differing use cases
▪ North-south
– Unknown / untrusted clients
– Limited exposure of services (Mapping)
– Centralised ops ingress defaults + decentralised product team cfg
▪ East-west
– Dynamic service information update required (multiple sources)
– Identity required for all services (mTLS + ACLs)
– “Sane” internal defaults + decentralised dev cfg