API Gateways provide functionality like rate limiting, authentication, request routing, reporting, and more. If you’ve been following the rise in service-mesh technologies, you’ll notice there is a lot of overlap with API Gateways when solving some of the challenges of microservices. If service mesh can solve these same problems, you may wonder whether you really need a dedicated API Gateway solution? The reality is there is some nuance in the problems solved at the edge (API Gateway) compared to service-to-service communication (service mesh) within a cluster. But with the evolution of cluster-deployment patterns, these nuances are becoming less important. What’s more important is that the API Gateway is evolving to live at a layer above service mesh and not directly overlapping with it. In other words, API Gateways are evolving to solve application-level concerns like aggregation, transformation, and deeper context and content-based routing as well as fitting into a more self-service, GitOps style workflow. In this talk we put aside the “API Gateway” infrastructure as we know it today and go back to first principles with the “API Gateway pattern” and revisit the real problems we’re trying to solve. Then we’ll discuss pros and cons of alternative ways to implement the API Gateway pattern and finally look at open source projects like Envoy, Kubernetes, and GraphQL to see how the “API Gateway pattern” actually becomes the API for our applications while coexisting nicely with a service mesh (if you adopt a service mesh).

1. 1 | Copyright © 2019
API Gateways are going through an
identity crisis
@christianposta

2. 2 | Copyright © 2019 @christianposta
CHRISTIAN POSTA
• Field CTO @ Solo.io
• Author of a few books
• Contributor to many open-source projects
• Architect, blogger, speaker, mentor, leader
@christianposta
christian@solo.io
https://blog.christianposta.com
https://slideshare.net/ceposta

3. 3 | Copyright © 2019 @christianposta
WROTE THE FIRST BOOK ON ISTIO…

4. 4 | Copyright © 2019 @christianposta
WRITING ISTIO BOOK FOR MANNING
https://bit.ly/istio-in-action

5. 5 | Copyright © 20195 | Copyright © 2019
Identity Crisis

6. 6 | Copyright © 2019 @christianposta
INSPIRED BY A BLOG…
http://bit.ly/gateway-identity-crisis

7. 8 | Copyright © 2019 @christianposta
• API Management solutions
• Kubernetes/cluster ingress
• Service Mesh proxies, gateways
• Application-built gateways
Identity crisis?

8. 9 | Copyright © 2019 @christianposta
“I am just going to
use service mesh,
doesn’t that make
gateways obsolete?”
IDENTITY CRISIS?

9. 10 | Copyright © 201910 | Copyright © 2019
Alternative talk title:
Role of edge gateways in relation to service
mesh adoption

10. 11 | Copyright © 2019 @christianposta
• Problem statement, challenges, tradeoffs
• Useful patterns to set context
• Envoy proxy as the foundation
• Architecture patterns to consider
• Demo
• Wrap up
What to expect from rest of this talk:

11. 12 | Copyright © 2019 @christianposta
• Augment, complement, replace existing API infrastructure
• Support a microservices, cloud environment
• Need better traffic control and observability
• As little disruption as possible, target multiple compute
• Improve security posture
Goals we see

12. 13 | Copyright © 2019
THE PROBLEM
HOW DO YOU
OBSERVE?
HOW DO YOU
MANAGE APIS?
HOW CAN ENFORCE
SECURITY?
MONOLITH MICROSERVICES

13. 14 | Copyright © 2019 @christianposta
• Traffic control
• Traffic routing
• Secure communications
• Application-level network observability
• Policy enforcement
Solving challenges between services within the organization

14. 15 | Copyright © 2019
LARGE, CENTRALIZED, LOW-TRUST, SHARED ENVIRONMENTS

15. 16 | Copyright © 2019
CENTRALIZED VS DECENTRALIZED

16. 17 | Copyright © 2019
CENTRALIZED VS DECENTRALIZED WITHOUT GAPS

17. 18 | Copyright © 201918 | Copyright © 2019
How to get there?

18. 19 | Copyright © 2019 @christianposta

19. 20 | Copyright © 2019 @christianposta
• Start with one proxy, grow to more
• Pick a subset of traffic applications
• Get demonstrable value from it
• Data plane matters
• Leverage high-value patterns for application traffic
Start with a gateway approach

20. 21 | Copyright © 201921 | Copyright © 2019
Three important gateway patterns

21. 22 | Copyright © 2019 @christianposta
“Edge” concerns, North-South vs East-West

22. 23 | Copyright © 2019 @christianposta
“Edge” concerns, North-South vs East-West
Capability Service Mesh Edge
Traffic Control ✔ ✔
Traffic Routing ✔ ✔
TLS/mTLS ✔ ✔
Network Observability ✔ ✔
Policy Enforcement ✔ ✔

23. 24 | Copyright © 2019 @christianposta
“Edge” concerns, North-South vs East-West
Capability Service Mesh Edge
OAuth/OIDC ✘ ✔
Web Application Firewall ✘ ✔
Message transformation ✘ ✔
Request/response caching ✘ ✔
Domain-specific rate limit ✘ ✔
HMAC, request path security ✘ ✔
Understand API surface,
intended decoupling ✘ ✔

24. 25 | Copyright © 2019 @christianposta
API Gateway Pattern
“A service that’s the entry point into the microservices-
based application … is responsible for request routing,
API composition, protocol translation …
and edge functions like authentication,
authorization, rate limiting, caching, et. al.”
See also:
https://microservices.io/patterns/apigateway.html
https://www.manning.com/books/microservices-patterns

25. 26 | Copyright © 2019 @christianposta
API Gateway Pattern: what it does
• Tailored API for different kinds of clients
• Abstracts backend services, protocols, message shapes, etc
• Powerful matching/routing, discovery, transformation,
aggregation, protocol translation
• Backends for front-ends
• A developer-first construct
• Intended to be decentralized and change at the pace of your
applications
• Network-level functions including rate-limiting,
authentication/authorization, caching, metrics collection

26. 27 | Copyright © 2019 @christianposta
Mediator pattern
http://en.wikipedia.org/wiki/Design_Patterns
Credit: https://springframework.guru/gang-of-four-design-patterns/mediator-pattern/

27. 28 | Copyright © 2019 @christianposta

28. 29 | Copyright © 2019 @christianposta

29. 30 | Copyright © 2019 @christianposta

30. 31 | Copyright © 2019 @christianposta

31. 32 | Copyright © 201932 | Copyright © 2019
Envoy proxy as a gateway

32. 33 | Copyright © 2019 @christianposta
Meet Envoy Proxy
http://envoyproxy.io

33. 34 | Copyright © 2019 @christianposta
Envoy Proxy implements:
• zone aware, least request load balancing
• circuit breaking
• outlier detection
• retries, retry policies
• timeout (including budgets)
• traffic shadowing
• rate limiting
• access logging, statistics collection
• Many other features!

34. 35 | Copyright © 2019 @christianposta

35. 36 | Copyright © 2019 @christianposta

36. 37 | Copyright © 2019 @christianposta
Edge Gateway built on Envoy
https://github.com/solo-io/gloo

37. 38 | Copyright © 2019 @christianposta
What is Gloo?
● Enterprise Envoy Proxy
● API-level routing, decoupling
● Complements any service mesh
● Traffic control, canary releases
● OAuth flows
● TLS termination, passthrough, mTLS
● Rate limiting, Caching
● Request/Response transformation
● Kubernetes CRDs (when deployed to Kubernetes)
https://gloo.solo.io

38. 39 | Copyright © 2019 @christianposta
Edge Gateway built on Envoy

39. 40 | Copyright © 2019 @christianposta
Gloo companion project: Sqoop
Query
Monolith Microservice
s
Cloud Functions
Result
https://sqoop.solo.io

40. 41 | Copyright © 2019 @christianposta
Demo!

41. 42 | Copyright © 201942 | Copyright © 2019
Gateway adoption patterns
(waypoint architecture) on the journey
to service mesh

42. 43 | Copyright © 2019 @christianposta
Start with single proxy

43. 44 | Copyright © 2019 @christianposta
Bring in decoupling points (multi-tier gateway)

44. 45 | Copyright © 2019 @christianposta
Gateway per product/domain/bounded context

45. 46 | Copyright © 2019 @christianposta
Push gateways down as you grow,
avoid death star architecture!

46. 47 | Copyright © 2019 @christianposta
Push gateways down as you grow,
avoid death star architecture!

47. 48 | Copyright © 2019 @christianposta
• Crawl, walk, run approach
• Leverage shared gateways, path for decentralization
• Envoy/Gloo proven open-source projects, successful adoption
• Reduce risk, target multi-platform compute, move at your own
pace
Final thoughts

48. 49 | Copyright © 2019 @christianposta
Check out Solo.io!

49. 50 | Copyright © 2019 @christianposta
Sneak peak, https://servicemeshhub.io

50. 51 | Copyright © 2019 @christianposta
CHRISTIAN POSTA
@christianposta
christian@solo.io
https://blog.christianposta.com
https://slideshare.net/ceposta

51. 52 | Copyright © 201952 | Copyright © 2019
@soloio_inc