In this new world of remote work, the demand for real-time cloud-based communication services has skyrocketed. 8x8, a company that provides a global cloud communications platform for its customers, moved its SaaS application from the data center to public cloud using Kubernetes and a microservices-based architecture to meet increased demand. In this webinar, we’ll share the firsthand experience of the DevOps team at 8x8, covering key considerations and learnings from successfully migrating a VoIP offering from an on-premises environment to AWS at scale. Learnings will cover designing application delivery infrastructure, automation, analytics, security, observability, and rapid scaling.

1. © 2018 Citrix | Confidential
Learnings from 8x8’s Experience
JUNE 18, 2020
Business Continuity with
Microservices-Based Apps and DevOps
Pankaj Gupta, Citrix PankajOnCloud
Lance Johnson, 8x8
Michael Laws, 8x8

2. 2 © 2018 Citrix | Confidential
Micro-services Based Application for Business Continuity
Portability
Modular App
Architecture
Break application in small
services, which can be
developed, deployed and
auto-scaled independently.
Faster Releases
Very frequent application
updates (even many times a
day)
Continuous Delivery to
automate deployment
Self contained runtime
Application portability
across Clouds
Read “A Pandemic Plan for Application Architecture” at https://tinyurl.com/K8sApp

3. 3 © 2018 Citrix | Confidential3 © 2019 Citrix | Confidential
Diverse Stakeholders Have Unique Needs for Business
Continuity
Application & infrastructure security
Container security & API gateways
Automation
DevSecOps
SRE
Application availability
Observability
Incident response
Postmortems
DevOps
Faster release &
deployment cycles
CI/CD & automation
Canary & progressive
rollout
User experience
Troubleshooting
Microservice discovery &
routing
Developers
Network policy & compliance
Manage, control & monitor network
Resources & capacity planning
(visibility)
NetOps
Platform Team
Platform governance
Operational
efficiency
Developer agility

4. 4 © 2018 Citrix | Confidential4 © 2019 Citrix | Confidential
Citrix Micro Services App Delivery Principles
Integrated With Your
Platform & Tools
Get apps to production fast with vast K8s
platform & opensource tools support
Performance
& Scale
Support large clusters &
very dynamic microservices
Consistent App &
API Security
Break the silos
of monoliths & microservices
Architecture
Flexibility
Move at speed of your IT skill set
Balance benefits & complexity
Complete
Observability Stack
Gain visibility & troubleshoot
problems faster, Break the silos
Production-Grade Solution at the Speed of Business
Read “7 Key Considerations for Microservices-Based Application Delivery” at https://tinyurl.com/7Consideration

5. 5 © 2018 Citrix | Confidential
For Your Every Step for Transition to Hybrid Multi Cloud
Single Software Code Base
across Portfolio for
Features, APIs & Operational Consistency, and Innovation Velocity
Hardware
Appliances
DC & Private Cloud, MPX
Virtual Appliances
DC & Private Cloud, VPX
In Public Cloud
VPX for AWS, Azure, GCP
Containers
Public & Private Cloud, CPX
Bare Metal
Private & Public Cloud, BLX
Multi-Tenant
MSP & Private Cloud,
SDX
Citrix ADC Portfolio: Built for Multi-Cloud
Most Comprehensive, Feature Rich & Software-Centric

6. 6 © 2018 Citrix | Confidential6 © 2019 Citrix | Confidential
Complexity
Benefits
Unified Ingress
2-Tier Ingress
Service Mesh Lite
Service Mesh
Low High
High
Balance the tradeoff
between benefits and complexity
Simplest and quickest way
to production
Simple for network-savvy
platform teams
Best observability &
security but complex
Service mesh-like
benefits but simpler
Microservices App Delivery Architecture Choices
Read “The Best Way to Select a Proxy Architecture for Microservices Application Delivery” https://tinyurl.com/ProxyArch

7. 7 | © 8x8, Inc. All Rights Reserved.
DevOps “Three Body Problem”

8. 8 | © 8x8, Inc. All Rights Reserved.
Introductions
Lance Johnson, Director of Engineering, Cloud R&D
https://twitter.com/johnsonrl
https://github.com/rljohnsn
https://www.linkedin.com/in/lancejohnsn/
Michael Laws, Sr. Site Reliability Engineer @8x8
https://www.linkedin.com/in/mikelaws/

9. 9 | © 8x8, Inc. All Rights Reserved.
Metrics

10. 10 | © 8x8, Inc. All Rights Reserved.
Unified Communications

11. 11 | © 8x8, Inc. All Rights Reserved.
At a Glance

12. 12 | © 8x8, Inc. All Rights Reserved.
Unified Communications
Meetings
Chat
Voice
Call Center

13. 13 | © 8x8, Inc. All Rights Reserved.
The Three Body Problem
Unsolvable mathematical problem

14. 14 | © 8x8, Inc. All Rights Reserved.
DevOps Three Body Problem
Deployments
• Ssh
• Jenkins Pipelines
• HyperLoop
• Ansible
• Chef
• Terraform
• Cloudformation
Customers
• DNS
• LTM
• GTM
• Route53
• Geo
• Proxy
• CloudFlare
• Ambassador/Envoy
Dependencies
• Data Centers
• Clouds
• Files
• Databases
• Random VPC
• Public
• Private
How did 8x8 approach the problem?

15. 15 | © 8x8, Inc. All Rights Reserved.
“What is your Quest?”

16. 16 | © 8x8, Inc. All Rights Reserved.
DevOps Fundamentals
• Safety
• Provide a way for changes to be validated and tested.
• Automation
• Deliver the changes the same way for any environment
• Knowledge Sharing
• Learn from each other with open collaboration
• Autonomy
• Immediate visibility that changes deployed are safe
• Continuously verified for every change

17. 17 | © 8x8, Inc. All Rights Reserved.
Adoption of Microservices: 100s of Them!
● Which
○ Foundational Framework services (Cloud8)
○ Shared Services
○ Stateless Application Services
● Why
○ Divide and conquer
○ Technology refresh
○ Team Autonomy

18. 18 | © 8x8, Inc. All Rights Reserved.
Design Goals
● Uniform deployment regardless of environments
● Service Discovery
● High availability and automatic recovery
● Container Packaging
● Native Kubernetes
● Public Charts
● Scaffolding Charts

19. 19 | © 8x8, Inc. All Rights Reserved.
Deployment Environments - Cloud

20. 20 | © 8x8, Inc. All Rights Reserved.
Deployment Environments - Hybrid

21. 21 | © 8x8, Inc. All Rights Reserved.
Deployment Environment - On Premise
* Mainline K8s (1.17+)
* L3 Leaf-Spine
* 3 Zones/DC
* Zone Storage
(& CSI Topology)
* BGP (Calico + Bird)
- Citrix ADC
- Route Reflectors
- BGP to Host

22. 22 | © 8x8, Inc. All Rights Reserved.
Deployment Environment - Anatomy of a Zone
Requirements:
* Horizontal Scale
* Zone isolation
topology.kubernetes.io/region=us-west
topology.kubernetes.io/zone=sjc01a
* Specialized network
Use-cases (DSR, ECMP, Anycast)
* Service IPs (VIPs) via BGP
* K8s-native APIs/resources
kind: Service
type: LoadBalancer

23. 23 | © 8x8, Inc. All Rights Reserved.
Load Balancing Tier - Citrix ADC
Key Decision Factors:
Supports Kubernetes Services and Ingresses (and Ingress Classes)
Supports Complex, Zone-Aware Infrastructures
Full-Feature BGP Support (e.g. Route Reflectors, Auth, RHI, ECMP)
Horizontal/Linear Scale (using VPX in VMware)
Rich Feature Support via CRD/Annotation (Cert Mgmt, Rules, Policy, IPAM, etc.)
Simple, Hands-Off, Reliable Deployment and Configuration
Flexible Licensing Options & Editions (MPX, VPX, CPX, etc.)

24. 24 | © 8x8, Inc. All Rights Reserved.
Recommendations
Reserve System Resources05
● System level, guarantee the OS a percentage
● Namespace LimitResource defaults
● Right size your workers
Cloud First04
● Started in the cloud for a uniform experience
● 100% of the resources were automatable
Self Service03
● Both engineering and operations could react
independently
Alignment02
● Strong collaboration between operations and
development
● Shared goals between teams
Choose your battles01
● Only stateless microservices initially
● New framework from the ground up

25. 25 | © 8x8, Inc. All Rights Reserved.
Recommendations
Centralize Metrics and Logs10
● Prometheus has vast library of existing dashboards
● Collect and ship using Daemonsets
● Newrelic with Pagerduty
Common Logging Format09
● Simply it for the teams, send to standard out
● Build into your framework or endorse a specific format
Ingress vs Services08
● Use an ingress strategy
● Saves money
● Provides Telemetry
Security07
● Limit kubectl usage
● Helm safer
● Cluster Role for a power user
Embrace A Service Mesh06
● Telemetry
● Better Security
● Connect desperate clusters

26. 26 | © 8x8, Inc. All Rights Reserved.