Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and HAProxy


Published on

Modern cloud applications today are built as distributed microservices. These microservices talk to each other over L7 protocols: HTTP, gRPC, Redis, Kafka, and more. In this world, L7 proxies have assumed a crucial role in managing and observing L7 protocols. In this talk, I’ll discuss the evolution of service architectures, the role L7 proxies play in this world, and how there is now a battle raging between Envoy Proxy, HAProxy, and NGINX. I’ll wrap by talking about why we chose Envoy Proxy as the anchor of our Ambassador API Gateway and show how that has enabled a number of new capabilities.

Published in: Software
  • Be the first to comment

  • Be the first to like this

The rise of Layer 7, microservices, and the proxy war with Envoy, NGINX, and HAProxy

  1. 1. 1 TheRiseofLayerSeven 1 2 We had to build centralized, local applications * (Relatively) simple to reason about and debug * Hard to accomplish certain things! Led to multiprocessing, multithreading, and distributed computing ALongTimeAgo… 2
  2. 2. 3 First distributed computing push was all about custom stuff * Applications had their own protocols (mostly directly on L4) * What we did to help one application didn’t help others Stacking approach: tried to push intelligence down into the network * Lots of crazy things in old IP switches Not just hard, but REALLY HARD. Now what? ALongTimeAgo… 3 4 Second decentralization push: individual services, independent lifecycles, independent teams This time we have effective Layer 7 communications * HTTP * gRPC * WebSockets * (exceptions are e.g., Redis, PostgreSQL, …) Under it all still a distributed system! MicroservicesArchitecture 4
  3. 3. 5 8FallaciesofDistributedComputing The network is reliable. Latency is zero. Bandwidth is infinite. The network is secure. Topology doesn’t change. There is one administrator. Transport cost is zero. The network is homogeneous. 5 6 Routability: make sure this request goes to an instance that is availableResilience: If no response with 5 seconds, retry again Observability: What’s the p99 latency of the server? ManagingtheFallacies 6
  4. 4. 7 Resilience Handling network/service failures. Retries, timeouts, circuit breakers, … Observability Being able to troubleshoot the system based on external outputs Distributed tracing, metrics monitoring, logs, … Routability Managing where requests go Load balancing, canary, shadowing, … ManagingtheFallacies 7 8 For example, look at HTTP: * URL and headers provide routing information * HTTP status provides metadata for health monitoring or circuit breaking * Clear semantics of “a request” for latency/throughput monitoring from outside Similar benefits with gRPC (slightly less for websockets) Often application-independent! ManagingtheFallacies 8
  5. 5. MANAGINGL7 9 10 Eureka Stubby Secret Google libraries FirstCut(~2010):SmartRPCLibraries 10
  6. 6. 11 FirstCut(~2010):SmartApplications 11 12 Single platform • Want the benefits? Gotta write for that platform! • Hard to take advantage of good stuff on another platform Distributed monolith * Tightly-coupled application-specific protocol makes it hard to develop independently * Versioning, software distribution, dependency nightmares Nothing’sPerfect… 12
  7. 7. 13 = + 2013:Proxy&Sidecar 13 14 Proxy transparently introduces resilience, observability, and routability. Service sends request to a proxy running locally (i.e., on same VM), which then proxies the request to the destination. 2013:Proxy&Sidecar 14
  8. 8. 15 HAProxy dates to 2006 and after awhile its development stagnated • Observability was there from the start • Never really designed for a large number of services • Zero-downtime reloads and runtime API didn’t arrive until late 2017 Zookeeper was a bear to run and deploy • Consul and etcd started to enter the picture Nothing’sPerfect… 15 16 Zero-downtime reloads Observability Runtime API (in 2017) 2015:EnterNGINX 16
  9. 9. 17 Observability and zero-downtime reloads from the beginning No Zookeeper to wrangle However, NGINX Plus locked many of the good features away. * API, advanced load balancing, etc. Nothing’sPerfect… 17 2016:EnterEnvoy 18 Zero-downtime reloads Observability Runtime API Shadowing gRPC Resilience 18
  10. 10. 19 Neutral Governance Growing Upstream Community 2018:EnvoyCommunity 19 20 Envoy offers a lot of what we want… * Observability * Zero-downtime reloads * API …but the cost is complexity. * Envoy is very powerful and very flexible * API is really designed for machine configuration This makes it challenging for humans to work with. Nothing’sPerfect… 20
  11. 11. CONTROLPLANES 21 22 Control Plane ControlPlane 22
  12. 12. 23 Istio. Control plane for east/west traffic. Ambassador. Kubernetes-based control plane for edge management. go-control-plane. DIY Envoy control plane. EnvoyControlPlanes 23 24 TheAmbassador ControlPlane Kubernetes manifests Ambassador is notified of configuration changes, and manages the Envoy configuration All traffic flows through Envoy Kubernetes services 24
  13. 13. 25 Ambassadorconfiguration --- apiVersion: v1 kind: Service metadata: name: httpbin annotations: | --- apiVersion: ambassador/v0 kind: Mapping name: httpbin_mapping prefix: /httpbin/ service: host_rewrite: spec: ports: - name: httpbin port: 80 { "timeout_ms": 3000, "prefix": "/httpbin/", "prefix_rewrite": "/", "host_rewrite": "", "weighted_clusters": { "clusters": [{ "name": "cluster_httpbin_org_80", "weight": 100.0 }] } } { "name": "cluster_httpbin_org_80", "connect_timeout_ms": 3000, "type": "strict_dns", "lb_type": "round_robin", "hosts": [{ "url": "tcp://" }] } Envoyequivalent 25 SUMMARY 26
  14. 14. 27 2012 2013 … 20162011 2017 • 1.8 released • First support for zero-downtime reloads, runtime API • NGINX Plus R13 (Aug 2017) • Runtime API • Shadowing 2018 • SmartStack switches to Envoy 27 28 Managing L7 is critical to modern cloud-native applications HAProxy, NGINX, and Envoy Proxy are evolving to meet these new requirements With neutral governance and the fastest growing community, Envoy Proxy looks to be the new standard for L7 proxies Most users don’t use Envoy directly; they use a control plane instead Summary 28