Uploaded byChiradeep Vittal
Load Balancing for Containers and Cloud Native Architecture
The document discusses the evolution and significance of load balancing in the context of containers and microservices, emphasizing the need for integration with container cluster managers. It outlines various architectures, technologies, and emerging patterns that enhance performance, reliability, and security in cloud-native environments. The content also highlights the importance of continuous monitoring, automation, and the use of advanced resilience patterns in modern load balancing solutions.
More Related Content
Similar to Load Balancing for Containers and Cloud Native Architecture
Load Balancing for Containers and Cloud Native Architecture
- 1. LOAD BALANCING FOR CONTAINERS August2016 Copyright: Citrix Systems, Inc
- 2. About me ¨ DistinguishedEngineer at Citrix ¨ Apache CloudStack PMC ¨ Work on Citrix Netscaler and containers Copyright: Citrix Systems, Inc
- 3. Load balancers areimportant (again) ¨ Containers and Microservices ¨ Teaching old load balancers new tricks ¨ Emerging patterns for Load Balancing ¨ Future directions Copyright: Citrix Systems, Inc
- 4.
- 5. What’s changed? ¨ DevOps+ Automation ¨ Microservices / Containers Image by BMW Werk Leipzig - http://bmw-werk-leipzig.de, CC BY-SA 2.0 deCopyright: Citrix Systems, Inc
- 6. Containers or Microservices? ¨Microservices == architecture ¨ Containers == implementation ¨ Containers win over VMs for Microservices Copyright: Citrix Systems, Inc
- 7. Monoliths vs. Microservices LB Web Data Copyright:Citrix Systems, Inc
- 8. Load Balancing: Traditionalvs. Cloud Native Static Applications, well defined topology Dynamic Microservices, changing topology W W W A A A A A W M M TrafficismostlyN-S M M M M M M M M M Traffic is mix of N-S and E-W Copyright: Citrix Systems, Inc
- 9. Cloud-Native Patterns of architecture and organization that deliver softwarewith speed and reliability ¨ Auto Scale ¨ Continuous Delivery ¨ Baked-in Resilience ¨ Deep Monitoring ¨ Collaboration Copyright: Citrix Systems, Inc
- 10. Cloud Native +Containers is Network Intensive ¨ Implications on ¤ Performance ¤ Reliability ¤ Security ¤ Routing ¤ Naming and discovery ¤ Monitoring Copyright: Citrix Systems, Inc
- 11. Cloud Native Landscape- Microservices Microservices APIs Performance Resilience Security Visibility Continuous Delivery AutoScal e Circuit Breaker Load Balancing Throttling Discovery Audit Segmentation E2E Encryption Routing Chaos Monkey Distributed Debug Back-off Lifecycle Management Auth Copyright: Citrix Systems, Inc
- 12. Containers Container Networking Container Security Integrity Isolation AuthN/Z Container Orchestration Infrastructure Orchestration Network Orchestration Cloud Orchestration Multi-cloud Orchestration Private Cloud OrchestrationHypervisor Orchestration Network Orchestration IPAM/ DNS Overlay / Underlay Microservices Lifecycle Management Container Performance Network Perf scheduler Container Storage Storage Orchestration Storage Orchestration Cloud Native Landscape - Infrastructure Copyright: Citrix Systems, Inc
- 13. Container Managers tothe rescue? Docker Swarm Kubernetes (from Google) Mesosphere DCOS (based on Apache Mesos) Copyright: Citrix Systems, Inc
- 14. Container Cluster Managers ¨Manage / orchestrate multi- host clusters of containers ¨ Enable DevOps automation with APIs ¨ Manage network and storage as well as compute α1 β1 δ1 β2 α2 δ3 α3 α4 δ2 α5 α6 HostH1 HostH2 HostH3 HostH4 δ4 ClusterManager Cluster API Copyright: Citrix Systems, Inc
- 15. Cluster Managers –Sweet Spots ¨ Infrastructure independence ¨ Scheduling ¨ Discovery ¨ Scaling (partially) ¨ Security (a little bit) ¨ Load balancing (limited) Copyright: Citrix Systems, Inc
- 16. Load Balancing forcontainer clusters – Ingress / Edge α1 β1 δ1 HostH1 β2 α2 δ3 HostH2 α3 α4 δ2 HostH3 Public Endpoint https://alpha:443 LB α1 β1 δ1 HostH1 β2 α2 δ3 HostH2 α3 α4 δ2 HostH3 α5 α6 HostH4 Public Endpoint https://alpha:443 LB Scale out Copyright: Citrix Systems, Inc
- 17. Load Balancing –intra-cluster α1 β1 HostH1 α2 δ3 HostH2 α4 δ2 HostH3 LB α1 β1 HostH1 α2 δ3 HostH2 α4 δ2 HostH3 LBLB LB LB per endpoint LB per container host Copyright: Citrix Systems, Inc
- 18. Load Balancing forContainer Clusters ¨ High performance ¨ Few instances ¨ Somewhat frequent reconfiguration ¨ Hardware or virtual ¨ Advanced: TLS, WAF, content switching ¨ Stateful ¨ Frequent reconfiguration ¨ Short lifespans ¨ Large number of instances ¨ Lower performance ¨ L4 common, L7 desirable ¨ Stateless Ingress (edge) Intra-cluster Copyright: Citrix Systems, Inc
- 19. Reconfiguration of LoadBalancer α1 β1 HostH1 β2 α2 HostH2 α3 α4 HostH3 Ingress LB α5 α6 HostH4 ClusterManager LB Controller Cluster API Container Events Reconfigur ation Container State Container State Query Copyright: Citrix Systems, Inc
- 20. Orchestration vs. Choreography ¨Cluster Manager / Orchestrator drives predefined (often hardcoded) process ¨ Points in the process can be “plugged in”. ¨ E.g., Load Balancer controller can be plugged in. ¨ Brittle, hard to change ¨ Typical of IaaS stacks (e.g., OpenStack, CloudStack) ¨ Cluster Manager emits events ¨ Controllers react to events asynchronously ¨ Easy to swap controllers ¨ Loosely coupled ¨ Harder to debug. ¨ No “god view” of state of the system. ¨ Typical of Cluster Managers (Kubernetes) Orchestration Choreography Copyright: Citrix Systems, Inc
- 21. Typical Form factors ¨Virtual, hardware (F5, Netscaler), or service (ELB) ¨ Usually Proprietary ¨ Containerized OSS (HAProxy, Nginx) ¨ Written from scratch (kube-proxy, Uber hyperbahn, linkerd, traefik) Ingress LB Intra-cluster Copyright: Citrix Systems, Inc
- 22. VPX Hypervisor Virtual Run Anywhere = NetScaler CPX:NetScaler in a Container MPX Physical Price-Performance CPX (new) Container SDX Platform Multi-Tenant = Copyright: Citrix Systems, Inc
- 23. Netscaler CPX Containerized Netscaler for Developers/ Cloud native deployments ¨ Proprietary ¨ Same hardened code, same features as bigger form factors ¨ Unified control plane with Ingress and other LB (Netscaler MAS) ¨ Unified monitoring, logging and analytics Copyright: Citrix Systems, Inc
- 24. Netscaler Family Automation Network and LoadBalancer Full Featured Un-compromised Network/ Load Balancer functionality All Appliance/ Workloads Containers, VM, Physical form factors Investment Protection Keep what you already have Single Management Platform One manager for all your appliances Copyright: Citrix Systems, Inc
- 25. Packaged as DockerContainer ¨ Investment protection ¤ Same code bits à container form factor ¤ Managed like any other NetScaler platform ¤ Seamless transition from Development to Production ¨ Functionality: ¤ Load balancing ¤ Content Switching ¤ All traffic types and protocols ¤ SSL offloading ¤ DNS ¤ Monitoring and logging ¤ Nitro API You Can Deploy In Seconds! Server Linux OS Docker Engine AppA bin/libs App B bin/libs App C bin/libs C P X bin/libs Copyright: Citrix Systems, Inc
- 26. Netscaler CPX Express ¨“Developer” Edition* ¨ No license required ¨ Available for download from Docker Hub in Q3 ¨ Drop-in replaceable with licensed version *Not for production use Copyright: Citrix Systems, Inc
- 27. MAS Turns NetScalersinto a Pool – Control One to Thousands NetScaler MAS CPX VPX MPX Container Management: Mesos & Marathon Docker Swarm Kubernetes Service Discovery Orchestrator: Self Service Portal NetScaler MAS Functions App-centric Life Cycle Configuration at Scale Visibility and Insights Copyright: Citrix Systems, Inc
- 28. What do WeMean by App-Centricity App configs vs. network configs Provide role-based, partitioned access to application owners Provide tools to map app config to NetScaler: Stylebooks Copyright: Citrix Systems, Inc
- 29. ¨ Template-driven configurationfor Citrix Netscaler. ¨ Intent-driven ¨ Sharing, collaboration and re-use ¨ Automate via an API App-centric: Netscaler Stylebooks Copyright: Citrix Systems, Inc
- 30. Configuration at Scale Copyright:Citrix Systems, Inc
- 31. ¨ The ADChas visibility to users on one side and apps/infrastructure on the other ¨ Outbound: Data provides insights on customer usage and behavior ¨ Inbound: Insights on app health and performance ¨ Security: Recognizing and mitigating increasingly sophisticated attacks Analytics at Scale: Insight from Network Data Users Apps Devices Copyright: Citrix Systems, Inc
- 32.
- 33. Cluster Managers: NativeLB support ¨ Built-in LB based on IPVS ¨ No ingress LB ¨ Native Service abstraction ¨ Kube-proxy for intra-cluster L4 load balancing (uses iptables) ¨ “Ingress” object for edge routing, but not built-in Docker Swarm Kubernetes ¨ Native ‘Task’ (service) abstraction ¨ Iptables-based L4 LB ¨ No built-in Ingress LB Mesos / Marathon Copyright: Citrix Systems, Inc
- 34. HostH1 Ingress Challenge (e.g.,Kubernetes) • Hardware / Virtual LB has to “participate” in overlay (usually VxLAN) • Interact with overlay manager / SDN controller • Kubernetes: Kube- Proxy introduces extra hop Intra Cluster Network (Overlay / Routing) External Network LB Ingress α1 β1 β2 α2 HostH2 α5 α6 HostHn Copyright: Citrix Systems, Inc
- 35. Emerging LB patterns ¨Client-side LB ¨ Sophisticated routing ¨ Resilience patterns ¨ Visibility / Insights Copyright: Citrix Systems, Inc
- 36. Client-side LB ¤ Embeddedinto calling application n e.g., Netflix Ribbon (with Eureka) ¤ Run as side-cars (alongside each application or one-per-host) n Netflix Prana n Twitter Finagle n Linkerd from Buoyant.io (based on Finagle) n AirBnB Smartstack/Synapse (uses HAProxy) n Uber Hyperbahn (like Finagle, switched from HAProxy) n Kube Proxy Eureka M M M M M M Copyright: Citrix Systems, Inc
- 37. Advanced Resilience ¨ LBimplements / assists resilience patterns such as: ¤ Circuit Breaker (e.g., Netflix Hystrix) ¤ Anti-DDOS ¤ Throttling ¤ Chaos Monkey Credit: http://martinfowler.com/bliki/CircuitBreaker.html Copyright: Citrix Systems, Inc
- 38. Routing ¨ Red-black deploys ¨Content routing M M M M Version N M M M M M Version N+1 10% 90% CD Pipeline Copyright: Citrix Systems, Inc
- 39. Visibility ¨ Compliance ¨ Debug ¨Topology ¨ Protocol Insights M M M M M M M M M M M Analytics Copyright: Citrix Systems, Inc
- 40. Wrap-up ¨ Load balancingis different for containers / microservices ¨ Integration with container cluster managers is needed ¨ Consider using the same LB technology for ingress and intra-cluster ¨ Emerging patterns solidify the importance of the load balancer ¨ LB in the wire brings ¤ Simplicity ¤ Resilience ¤ Future proofing Copyright: Citrix Systems, Inc