Both API Gateways and Service Meshes offer similar features and capabilities but are each implemented on a different axis in your application’s data plane. Draw those capabilities in a Venn diagram, you’ll see a lot of overlap between the API Gateways and Service Meshes. This is NOT a talk about the overlap. This is a talk about the things outside the overlap and why they matter (#HereBeDragons). The talk focuses on three questions around North/South & East/West Traffic. This talk is for a senior developer/architect on where to best inject cross-cutting concerns around security, observability and resiliency enabled by API Gateways and Service Meshes.

1. © 2022 Thoughtworks | Confidential
API Gateways & Service Mesh
What’s Best for Us?

2. © 2022 Thoughtworks | Confidential

3. API Gateways and Service Meshes offer similar
capabilities (though they handle traffic at a different
axis in the request — often distinguished by terms like
North/South and East/West).
While there is extensive overlap, there are cases where
each excel.

4. © 2022 Thoughtworks | Confidential
Wesley Reisz (he/him)
Technical Principal
Creator/Co-host of #TheInfoQPodcast
Chairperson QConSF/Plus
(
Fall)
@wesreisz
wesley.reisz@thoughtworks.com
4

5. API Gateway & Service Mesh
Some Questions to Think About
Deployment Patterns
Agenda

6. © 2022 Thoughtworks | Confidential
API Gateway & Service Mesh
Level Set on North/South & East/West traffic
6

7. © 2022 Thoughtworks | Confidential
API Gateway responsible for
client-to-service communication
(or north-south traffic) guards the
boundary of the application and
controls how the application
makes itself available to other
externally.
7
© 2022 Thoughtworks
Consumer
Provider

8. © 2022 Thoughtworks | Confidential
API Gateway responsible for
client-to-service communication
(or north-south traffic) guards the
boundary of the application and
controls how the application
makes itself available to other
externally.
8
© 2022 Thoughtworks
Consumer
Provider
Gateway

9. The API gateway pattern describes an additional hop in the
network that every request will have to go through in order to
consume the underlying APIs. In this context, some people call the
API gateway a centralized deployment.

10. API Gateway
API Gateway Control Plane
Service Service Service
Data Data Data

11. API
Composition
Request/Response
Transformation
WAF
Data Loss
Prevention
API Gateway
(North/South)
API as a
Product
Rate
Throttling
Enable
Deployment
Patterns
Policy
Layer 7
Security

12. © 2022 Thoughtworks | Confidential 12
© 2022 Thoughtworks
© 2022 Thoughtworks
Consumer Provider
A service mesh responsible
for service-to-service
communication (east/west traffic)
is a dedicated infrastructure layer
that you can add to your
applications.

13. © 2022 Thoughtworks | Confidential 13
© 2022 Thoughtworks
Consumer Provider
A service mesh responsible
for service-to-service
communication (east/west traffic)
is a dedicated infrastructure layer
that you can add to your
applications.
Often referred
to as a sidecar

14. A service mesh provides a generic mechanism for intercepting
microservice communications. It enables us to transparently
introduce aspects such as security, routing, monitoring, and
testing with no changes to the service themselves — in fact,
reduce complexity within the service. Because of the distributed
approach, the service mesh is considered more decentralized
than an API Gateway approach.

15. Control Plane
Data Plane
Ingress Traffic Sidecar Proxy
Service A
Sidecar Proxy Egress Traffic
Service B
Control
Discovery
Certs
Config

16. Observability
Resiliency
Service Mesh
(East/West)
Rate
Throttling
AuthN/AuthZ
Enable
Deployment
Patterns
Policy
Layer 7
Layer 4
Traffic Routing/
Shaping
Security

17. API Gateway is responsible for the flow of requests between the
client and the services, aggregating multiple services and
creating and sending the final response to the client (often at a
perimeter). Service Mesh is responsible for the flow of requests
between services (often inside a perimeter).

18. Seems straight forward, what’s the confusion?

19. API Gateway and Service Mesh have overlapping functionalities, such
as rate-limiting, security, service discovery, tracing, etc. but they work
on different levels and solve different problems.

20. Observability
Resiliency
Service Mesh
(East/West)
Rate
Throttling
AuthN/AuthZ
Enable
Deployment
Patterns
Policy
Layer 7
API Gateway
(North/South)
Layer 4
Traffic Routing/
Shaping
Security
API
Composition
Request/Response
Transformation
WAF
Data Loss
Prevention
API as a
Product
Rate
Throttling
Enable
Deployment
Patterns
Policy
Layer 7
Security

21. Observability
Resiliency
Service Mesh
(East/West)
API
Composition
Rate
Throttling
Request/Response
Transformation
AuthN/AuthZ
Enable
Deployment
Patterns
WAF
Data Loss
Prevention
Policy
Layer 7
API Gateway
(North/South)
API as a
Product
Layer 4
Traffic Routing/
Shaping
Security

22. While API Gateway and Service Mesh have overlapping
capabilities. Things get more complicated in the realm of
Kubernetes.
Ingress, Service Mesh Ingress, and API Gateways can also do very
similar things for North/South traffic.

23. Pod Pod
Cluster
Service
Kubernetes

24. Ingress
(
Gateway API
)
Pod Pod
North/South
Traffic
Cluster
Service
Kubernetes with Ingress

25. Ingress
(
Gateway API
)
Pod Pod
North/South
Traffic
East/West
Traffic
Cluster
Sidecar
Sidecar
Kubernetes with Service Mesh

26. API Gateway
Ingress
(
Gateway API
)
Pod Pod
North/South
Traffic
East/West
Traffic
Cluster
Sidecar
Sidecar
Kubernetes with Service Mesh & API Gateway
Envoy (a highly
performant
proxy) is often
used in all.

28. Should I use an API Gateway or Service Mesh?

29. © 2022 Thoughtworks | Confidential
Similar Capabilities, Different Focus
Some questions to think about
29

30. Observability
Resiliency
Service Mesh
(East/West)
API
Composition
Rate
Throttling
Request/Response
Transformation
AuthN/AuthZ
Enable
Deployment
Patterns
WAF
Data Loss
Prevention
Policy
Layer 7
API Gateway
(North/South)
API as a
Product
Layer 4
Traffic Routing/
Shaping
Security

31. I think as an industry, frankly, we've done a poor job of using
consistent nomenclature here.
I think you're going to hear people say API Gateway, edge proxy,
Ingress controller, and in many ways, they're going to use them
interchangeably. I don't even know that I could honestly tell you what
the difference is because from my perspective, and I would consider
myself an expert here, I don't think there really is much of a difference.
I think it's more useful for me to come at it from the perspective of, in
modern internet architectures, I have an edge component that is
sitting between the internet and my backend systems. And I can call
this the Ingress or the API Gateway or the edge proxy, but it's better
to focus on the functionality.
-
Matt Klein, Creator Envoy
https://www.infoq.com/podcasts/matt-klein-envoy-gateway

32. Are your APIs a
product?
Questions to Consider
Is your Traffic
Internal or
External?
Where are you
okay with
Complexity?
© 2022 Thoughtworks

33. API as a product that other developers, partners or
teams will consume.
Are your APIs a
product?

34. Examples of Product’s as an API
● Plaid: The company builds a data transfer network that
powers fintech and digital finance products. Plaid's product, a
technology platform, enables applications to connect with users’ bank
accounts. It allows consumers and businesses to interact with their
bank accounts, check balances, and make payments through different
financial technology applications
● Stripe: Stripe, Inc. is an Irish-American financial services and software
as a service
(
SaaS
)
company dual-headquartered in San Francisco,
United States and Dublin, Ireland. The company primarily offers
payment processing software and application programming interfaces
(
APIs) for e-commerce websites and mobile applications.
● Twilio: Twilio is an American company based in San Francisco,
California, which provides programmable communication tools for
making and receiving phone calls, sending and receiving text
messages, and performing other communication functions using its
web service APIs.
Are your APIs a
product?

35. • Create your services
• Create a self-service portal for developer self registration.
• Create a billing/monitoring service
• Create an AuthN and AuthZ for access / control
• Generate API keys for users
• Create a dashboard (reports) for API usage
• Produce documentation
• Load balance, Rate Limit, Proxy, & apply Policies
To make a sellable API, what do you need to do?
Are your APIs a
product?

36. Service
API Gateway API Management
Dev Portal
Developer Consuming
Product API
AuthN/AuthZ
Dashboard
(reports)
Self-service portal
Docs
API Keys
Load Balancing
Rate Limit
Are your APIs a
product?
Service Service

37. https://docs.konghq.com/hub/

38. External calls tend to be more focused on the
perimeter and often have different sets of non-
functional requirements.
Internal calls tend to be more homogeneous and
often more focused on the added network
between services (often concerned with
reliability and resiliency).
Is your Traffic Internal vs External?
Is your Traffic
Internal or
External?

39. The fundamental difference between
edge routing (north/south) and internal
(east/west) routing is that with the
edge, you don’t control the client.
-
Richard Li, CEO Ambassador Labs
Is your Traffic
Internal or
External?

40. Is your Traffic
Internal or
External?
Client Server
1
Requests protected resource
2
Presents cert (server.cer)
CA
3
Verifies cert (server.cer)
4
Presents cert (client.cer)
5
Verifies cert
(client.cer)
6
Returns protected Resource

41. Is your Traffic
Internal or
External?
https://www.datacenterdynamics.com/en/news/spotify-sees-hour-long-global-
outage-forgot-renew-certificates/

42. Ingress Traffic Sidecar Proxy
Service A
Sidecar Proxy Egress Traffic
Service B
Data Plane
Control
Discovery
Certs
Config Control Plane

43. Ingress Traffic Sidecar Proxy
Service A
Sidecar Proxy Egress Traffic
Service B
Data Plane
Control
Discovery
Certs
Config Control Plane

44. Observability
At the sidecar, you have insight into the components of a
request.
At the gateway, you have insight into total time of the request
coming in and out of your network.
Is your Traffic
Internal or
External?

45. https://www.moesif.com/blog/technical/aws-api-gateway/How-to-Monitor-API-Usage-and-Performance-with-the-Moesif-Plugin-for-AWS-API-Gateway/

48. !
Warning: No silver bullets.
Out of the box observability from a service mesh
gives you a leg up, but it’s rarely enough to
effectively operate a large microservice
environment alone. Providers repeatedly point out
developers still need to instrument code for
effective observability strategies.
Is your Traffic
Internal or
External?

49. Similar Capabilities, Different Lens
● Deployment Patterns
(
Canaries or blue/green deployment)
● Resources used
(
OpEx vs CapEx)
● Cache strategies
● Simplify / Consolidate Architecture
● Defense
(
Bot blocking / WAF
)
Is your Traffic
Internal or
External?

50. Are you okay trading complexity
for flow on a development team?
API Gateways are a simple piece of infrastructure. Service
Mesh are not.
Where are you
okay with
Complexity?

51. Let’s Build an App
iOS, Android Web
Mobile BFF Web BFF
Order Detail
Order Detail
Consumer
Service
Order
Service
Delivery
Service
Data Data Data
Where are you
okay with
Complexity?

52. Let’s Build an App
Resiliency
Observability
Security
Service
Discovery
Deployment
Patterns
Business
Logic
Frameworks Data
Developer
Where are you
okay with
Complexity?

53. Let’s Build an App
Resiliency
Observability
Security
Service
Discovery
Deployment
Patterns
Business
Logic
Developer
Frameworks
Data
Value
Chain
Where are you
okay with
Complexity?
Custom/Unique Commodity

54. Let’s Build an App
Resiliency
Observability
Security
Service
Discovery
Deployment
Patterns
Business
Logic
Developer
Frameworks
Data
Value
Chain
Where are you
okay with
Complexity?
Custom/Unique Commodity

55. Let’s Build an App
Business
Logic
Developer
Frameworks
Data
Value
Chain
Observability
Where are you
okay with
Complexity?
Custom/Unique Commodity

56. Operating things a service mesh comes with
complexity. It is worth it for many
organizations to tackle that complexity and
reduce the cognitive load on development
teams if it improves overall flow. Where are you
okay with
Complexity?

57. Are your APIs a
product?
Questions to Consider
Is your Traffic
Internal or
External?
Where are you
okay with
Complexity?
© 2022 Thoughtworks

58. © 2022 Thoughtworks | Confidential
Deployment Patterns
Common Scenarios using API Gateway/Service Mesh
58

59. API Gateway
Service A Service B Service C
Cache
Logging
Rate limiting
SSL Offloading
Routing
Deploying API Gateway
API Gateway Control Plane
North/South
Traffic
Layer 7
Data Loss
Prevention
WAF

60. Deploying a Service Mesh
Istio Gateway
Sidecar
Service A
Sidecar
Service B
Sidecar
Service C
Sidecar
Service D
Sidecar
Service E
Sidecar
Service F
North/South
Traffic
East / West
Traffic
Tracing
Rate limiting
Circuit Breakers
Routing
Deployment
Patterns
Logging
Resiliency
Policy

61. Deploying API Gateway with a Service Mesh
Sidecar
Service A
Sidecar
Service B
Sidecar
Service C
Sidecar
Service D
Sidecar
Service E
Sidecar
Service F
North/South
Traffic
East / West
Traffic
Istio Gateway
API Gateway
Cache
Logging
Rate limiting SSL Offloading
Layer 7
Data Loss
Prevention
WAF
Tracing
Rate limiting
Circuit Breakers
Resiliency
Policy

62. Deploying API Gateway into a Service Mesh
API Gateway
Sidecar
Sidecar
Service A
Sidecar
Service B
Sidecar
Service C
Sidecar
Service D
Sidecar
Service E
Sidecar
Service F
North/South
Traffic
East / West
Traffic
Cache
Logging
Rate limiting
SSL Offloading
Layer 7
Data Loss
Prevention
WAF
Tracing
Rate limiting
Circuit Breakers
Resiliency
Policy

63. API Gateway
Sidecar
Service A
Sidecar
Service C
Sidecar
Service D
Sidecar
Service F
East / West
Traffic
API Gateway
Sidecar
Service A
Sidecar
Service C
Sidecar
Service D
Sidecar
Service F
East / West
Traffic
Internal API Gateway / Service Mesh
Tracing
Rate limiting
Circuit Breakers
Resiliency
Policy
Tracing
Rate limiting
Circuit Breakers
Resiliency
Policy
API as Product

64. API Gateway & Service Mesh
Some Questions to Think About
Deployment Patterns
Agenda

65. API Gateways and Service Meshes offer similar
capabilities (though they handle traffic at a different
axis in the request — often distinguished by terms like
North/South and East/West).
While there is extensive overlap, there are cases where
each excel.

66. © 2022 Thoughtworks | Confidential
Key Takeaways
• API Gateway & Service Mesh capabilities are converging. Focus
less on what the tech is called and more on the functionality
you need.
• Among other things, API Gateway excel at the perimeter;
Service Meshes excel at improving the network reliability in a
microservices environment
• If just starting out, considering starting with an API Gateway
over a service mesh
• Regardless of your choice, you have options when it comes to
deployments
@wesreisz

67. © 2022 Thoughtworks | Confidential

68. © 2022 Thoughtworks | Confidential
We look forward to
working with you
Wesley Reisz (he/him)
Technical Principal
Creator/Co-host of #TheInfoQPodcast
Chairperson QConSF/Plus
(
Fall)
@wesreisz
wesley.reisz@thoughtworks.com
68