This document discusses building a service mesh with NGINX. It notes that operating distributed applications is difficult due to issues like slow and unreliable calls between services, distributed fault finding, and continuous updates occurring in production. It reviews existing approaches like using an NGINX proxy per pod or a simple mesh. A full service mesh provides more capabilities but also more complexity. The document outlines NGINX's plans to build a service mesh focused on hybrid applications, with lightweight and performant data and control planes using open source projects like SPIRE and OpenTracing where possible.

2. Building a Service Mesh
with NGINX
Owen Garrett, Faisal Memon
Products & Engineering, NGINX

3. Operating a distributed application is hard
Static, Predictable Monolith: Dynamic, Distributed App:
Fast, reliable function calls
Local debugging
Local profiling
Calendared, big-bang upgrades
‘Integration hell’ contained in dev
Slow, unreliable API calls
Distributed fault finding
Distributed tracing
In-place dynamic updates
‘Continuous integration’ live in prod
More things can go wrong, it’s harder to find the faults, everything happens live

4. FrontEnd
RecommendEng
InventoryDB
ImageRender
AdminConnector
UserState
PricingEng
API-fe
N
• Security
• Instrumentation
• Tracing
• Traffic Control

5. Precursors to Service Mesh
1

6. NGINX per-Pod Proxy
Use NGINX Per-Pod Proxy:
• To intercept traffic to a single pod
• Implement access control, metrics and
tracing, web app firewall for that
service
Complexity: Simple
• Single point of configuration, simple
• Fully integrated into pod – easy build,
test and deployment
Each Pod in Service B has a
dedicated proxy.
We rely on K8s (kubeproxy) to
load-balance traffic to Service B
B
A kubeproxy

7. What about egress traffic?
• Why might this be a problem?
1. Cannot automatically perform mTLS. Need to rely on application to make
TLS requests, or rely on overlay network and K8s Network Policies for
encryption and authorization
2. Metrics and traces are generated on server-side, not client-side, so do not
measure latency effect of K8s network
No control of egress traffic, exiting pod Full control of ingress traffic, entering pod

8. Simple Mesh
Use NGINX Simple Mesh:
• When the application only needs to talk
to a small, well-known set of external
services
Complexity: Not very simple!
• Need to know all egress targets in
advance
• Fully integrated into pod – easy build,
test and deployment
Ingress Traffic – exactly as
the per-Pod proxy
configuration.
Egress Traffic – application
talks to local NGINX IP
address, achieved by e.g. DNS
manipulation or IP tables.
NGINX needs a virtual server
for each egress service.

9. Service Mesh Use a Service Mesh (Istio, Linkerd):
• When you don’t know the topology of the
application
• When you want an off-the-shelf solution
• When the specific service mesh capabilities
match your requirements
Complexity: Medium-High
Service Mesh technology is not fully mature, so
operating, troubleshooting and debugging it
requires considerable technical expertise.
Control Plane
e.g. Istio’s Pilot/Mixer/Citadel
Sidecar Proxy
Service A
Sidecar Proxy
Service B
Sidecar Proxy
Service C
Sidecar Proxy
Service D

10. NGINX Service Mesh
2

11. Why is NGINX building a Service
Mesh?
What are the attributes of modern
applications?
• Hybrid (microservice and legacy)
• Web and API
• Multiple technology stacks
What is needed to deliver these
applications?
• Load Balancer / App Delivery
• Specialized API management
• Internal Service Mesh

12. NGINX Features and Principles
Initial Features
• Mutual TLS
• Instrumentation
• Tracing
• Traffic Control
Core Principles
• Hybrid Applications first
• Lightweight and Performant
• Architectural Simplicity
• Use CNCF etc. projects
whenever possible

13. What is NGINX building?
Data Plane
East/West traffic
Control
Plane
Management
Plane
Infrastructure
Kubernetes VMware AWS Bare Metal
SVC SVC
SVC
SVC
SVC SVC
Conf Db
Kubernetes Service Registry Inventory VMware, AWS… Inventory

14. What is NGINX building?
Data Plane
East/West traffic
Control
Plane
Management
Plane
Infrastructure
Kubernetes VMware AWS Bare Metal
SVC SVC
SVC
SVC
SVC SVC
Conf Db
Kubernetes Service Registry Inventory VMware, AWS… Inventory

15. What is NGINX building?
Data Plane
East/West traffic
Control
Plane
Management
Plane
Infrastructure
Kubernetes VMware AWS Bare Metal
NGINX Service Mesh control plane
SVC SVC
SVC
SVC
SVC SVC
Topology Policies
Conf Db
CLI / API
$>_
Kubernetes Service Registry Inventory VMware, AWS… Inventory

16. What is NGINX building?
Data Plane
East/West traffic
Control
Plane
Management
Plane
Infrastructure
Kubernetes VMware AWS Bare Metal
NGINX Service Mesh control plane
SVC SVC
SVC
SVC
SVC SVC
Topology Policies
Conf Db
CLI / API
$>_
Kubernetes Service Registry Inventory VMware, AWS… Inventory
SPIRE

17. What is NGINX building?
Data Plane
East/West traffic
Control
Plane
Management
Plane
Infrastructure
Kubernetes VMware AWS Bare Metal
NGINX Service Mesh control plane
SVC SVC
SVC
SVC
SVC SVC
Topology Policies
Conf Db
CLI / API
$>_
Kubernetes Service Registry Inventory VMware, AWS… Inventory
SPIRE Grafana OpenTracing

18. What’s in the dataplane?
• NGINX
◦ OpenTracing module (3rd party, open source)
◦ Prometheus module (open sourced)
◦ SPIFFE support (to-be-open sourced)
◦ NGINX Plus
◦ Sophisticated configuration for fully-hitless reloads
SVC

19. Walkthrough
3
19

20. Concluding Points
4
20

21. Find the balance
Cost
to
operate
Complexity, Interdependencies, Speed of Change
Single simple app Many complex, interdependent apps
Using native Kubernetes
and other services
Using
service mesh
As service meshes mature,
their cost will go down

22. Solve today’s problem, today
• Identify the problem
◦ Security? Visualization? Tracing? Advanced traffic control?
• Identify the solution with the minimum technology to solve
the problem
◦ Vanilla Kubernetes with Ingress Controller
◦ Per-Service or per-Pod load balancer
◦ “Full-fat” Service Mesh

23. Owen Garrett, Faisal Memon
owen@nginx.com / faisal@nginx.com
Thank you