Cloud-Native Application Debugging with Envoy and Service Mesh

1. Cloud-native Application Debugging with Envoy and Service Mesh Christian Posta Field CTO – Solo.io

3. 3 | Copyright © 2020 01 02 03 04 05 06 Challenges of microservices, debugging Introduction to our lab environment Distributed tracing with a service mesh Debugging microservices Debugging in production with record and replay Proactive debugging with chaos experimentation Approximate flow of workshop

8. 8 | Copyright © 2020 December 11, 2018 2018 TOP WOMEN ENTREPRENEURS IN CLOUD INNOVATION Seventh Annual Award Honors Women Founders for Outstanding Accomplishments in Cloud and Emerging Technologies, Sponsored by Facebook, Intel, and Google. Award Winning Innovation Key Industry Collaborations

12. 12 | Copyright © 2020 Cloud application networking challenges • Edge/DMZ routing • Surgical / fine / per-request routing • A/B rollout • Traffic shaping • Request racing • Internal releases / dark launches • Request shadowing • Fault injection

16. 16 | Copyright © 202016 | Copyright © 2020 Decentralized, language-independent observability in the network Foundational technology to help solve these challenges in a cloud-native application architecture

18. 18 | Copyright © 2020 Envoy implements: • zone aware, least request load balancing • circuit breaking • outlier detection • retries, retry policies • timeout (including budgets) • traffic shadowing • request racing • rate limiting • access logging, statistics collection • Many other features!

20. 20 | Copyright © 2020 Why Envoy? • C++ • Built ground-up for services environment • Large, diverse, vibrant community • Dynamic configuration model • Highly extensible (in C++  we’ll come back to this) • Many out of the box L7 filters (HTTP, HTTP2, grpc, redis, mysql, DynamoDB, thrift, zookeeper, kafka, et. al.) • Incredible trove of telemetry, tracing out of the box • Very versatile deployment options (as we’ll see)

26. 26 | Copyright © 2020 Service mesh technologies provide the following: • Service discovery / Load balancing • Secure service-to-service communication • Traffic control / shaping / shifting • Policy / Intention based access control • Traffic metric collection • Service resilience • API / programmable interface

41. 41 | Copyright © 2020 THE PROBLEM A MONOLITHIC APPLICATION CONSISTS OF A SINGLE PROCESS AN ATTACHED DEBUGGER ALLOWS VIEWING THE COMPLETE STATE OF THE APPLICATION DURING RUNTIME A MICROSERVICES APPLICATION CONSISTS OF POTENTIALLY HUNDREDS OF PROCESSES IS IT POSSIBLE TO GET A COMPLETE VIEW OF THE STATE OF A SUCH APPLICATION?!

45. 45 | Copyright © 2020 -> ls -l /proc/self/ns total 0 lrwxrwxrwx 1 idit idit 0 Dec 7 01:14 cgroup -> cgroup:[4026531835] lrwxrwxrwx 1 idit idit 0 Dec 7 01:14 ipc -> ipc:[4026531839] lrwxrwxrwx 1 idit idit 0 Dec 7 01:14 mnt -> mnt:[4026531840] lrwxrwxrwx 1 idit idit 0 Dec 7 01:14 net -> net:[4026532009] lrwxrwxrwx 1 idit idit 0 Dec 7 01:14 pid -> pid:[4026531836] lrwxrwxrwx 1 idit idit 0 Dec 7 01:14 pid_for_children -> pid:[4026531836] lrwxrwxrwx 1 idit idit 0 Dec 7 01:14 user -> user:[4026531837] lrwxrwxrwx 1 idit idit 0 Dec 7 01:14 uts -> uts:[4026531838] -> inod of mnt namespace (unique identifier to the container namespace) via CRI api call ExecSyncRequest Node Namespace: ns-a s-dlv CRI c1 We need to translate the pid of the process (application that run in the container) to the host pid namespace to allow debugger to attach. Namespace: Squash

60. 60 | Copyright © 2020 Gloo Data Plane and Control Plane EXTERNAL AUTH RATE LIMITING GLOO FILTERS ROUTER UPSTREAM EXTERNAL AUTH SERVER RATE LIMITING SERVER CACHING DATA LOSS PREVENTION LAMBDA NATS.IO TRANSFORMATION WEB APPLICATION FIREWALL (WAF)

61. 61 | Copyright © 2020 API Gateway built on Envoy ENVIRONMENT SECRET CONFIGURATION Data Plane Upstream gRPC-JSON transcoder Rate limiting External AUTH … Control Plane Configure and manage envoy’s plugins Router

62. 62 | Copyright © 2020 Gloo API Gateway • Unify backend APIs running in Kubernetes, VMs, Physical, FaaS, etc • Decentralized configuration: allow service teams to move fast • Declarative configuration • Provides a control plane for Envoy • Security (Oauth/ODIC, API Key, TLS, SNI, OPA, HMAC, custom) • Kubernetes native / run outside Kube as well • Highly pluggable/extensible • “If you know Kubernetes, you know Gloo”  user quote

67. 67 | Copyright © 2020 @christianposta CHAOS ENGINEERING THINK OF A VACCINE OR A FLU SHOT INJECT YOURSELF WITH SOMETHING HARMFUL IN ORDER TO PREVENT A FUTURE ISSUE. CAREFULLY INJECTING THIS HARM INTO YOUR SYSTEMS TO TEST THE SYSTEM’S ABILITY TO RESPOND TO IT. “BREAK THINGS ON PURPOSE" IN ORDER TO LEARN HOW TO BUILD MORE RESILIENT SYSTEMS.

70. 70 | Copyright © 2020 @christianposta CONTROL EXPERIMENT ⍄ DEFINE EXPERIMENTS (SET OF: MESSAGE DELAYS, NETWORK FAULTS) ⍄ RUN EVERY INTERVAL (E.G. EVERY FRIDAY AT 9PM) ⍄ GATHERED METRICS – COMPARE BASELINE ⍄ STOP EXPERIMENT IF CONDITION REACHED

71. 71 | Copyright © 2020 @christianposta GLOOSHOT GLOOSHOT ALLOWS YOU TO PERFORM CHAOS EXPERIMENTS AT THE SERVICE MESH LEVEL. DEFINE ERROR CONDITIONS IN TERMS OF SUCH FAILURE MODES: ⍄ MESSAGE DELAYS ⍄ NETWORK FAULTS. RUN EXPERIMENTS UNTIL A STOP CONDITION IS MET. GLOOSHOT INTERFACES WITH ALL MAJOR SERVICE MESHES THROUGH SERVICE MESH INTERFACE (SMI).

79. 79 | Copyright © 2020 • https://solo.io • https://slack.solo.io • https://gloo.solo.io • https://envoyproxy.io • https://istio.io • https://webassemblyhub.io • https://servicemeshhub.io • https://blog.christianposta.com