1.
Dumb Services In
Smart Nets
Ant Weiss, Otomato Software
Deploy Like a Ninja with a Service
Mesh

2.
whoami: Anton (Ant) Weiss
@antweiss
Otomato Software Delivery
http://otomato.link

3.
@antweiss
while True:
intellect = 0.05

4.
@antweiss
while True:
intellect = 0.05
dumbness = 0.95

5.
@antweiss
Microservices !!!
API
gateway
User
mgmt
Product
Service
Pricing
Service
Orders
Service
Billing
Service
Payment
Service
MainUI
DB DB DB
DB DB DB
UserUI
MobileUI

6.
@antweiss
Microservices !!!
API
gateway
User
mgmt
Product
Service
Pricing
Service
Orders
Service
Billing
Service
Payment
Service
MainUI
DB DB DB
DB DB DB
UserUI
MobileUI
MacroPAIN !!!

7.
@antweiss

8.
@antweiss
The Traps of Distributed SysteMS

9.
@antweiss
● Misconfigured Fallback Options
The Traps of Distributed SysteMS

10.
@antweiss
● Misconfigured Fallback Options
● Retry Storms
The Traps of Distributed SysteMS

11.
@antweiss
● Misconfigured Fallback Options
● Retry Storms
● Outages When a Downstream is Overloaded
The Traps of Distributed SysteMS

12.
@antweiss
● Misconfigured Fallback Options
● Retry Storms
● Outages When a Downstream is Overloaded
● Cascading Failures When a SPOF crashes
The Traps of Distributed SysteMS

13.
@antweiss
http://principlesofchaos.org
The Traps of Distributed SysteMS

14.
@antweiss
PATTERNS of RESILIENCE

15.
@antweiss
● connection pools
PATTERNS of RESILIENCE

16.
@antweiss
● connection pools
● failure detectors
PATTERNS of RESILIENCE

17.
@antweiss
● connection pools
● failure detectors
● failover strategies:
○ circuit breaking
○ exponential back-offs
PATTERNS of RESILIENCE

18.
@antweiss
● connection pools
● failure detectors
● failover strategies:
○ circuit breaking
○ exponential back-offs
● load-balancers
PATTERNS of RESILIENCE

19.
@antweiss
● connection pools
● failure detectors
● failover strategies:
○ circuit breaking
○ exponential back-offs
● load-balancers
● back-pressure techniques
○ rate limiting, etc
PATTERNS of RESILIENCE

20.
@antweiss
ADDITIONAL CONCERNS
Service Discovery

21.
@antweiss
ADDITIONAL CONCERNS
Service Discovery
Observability:
● Distributed Tracing
● Network Telemetry

22.
@antweiss
Service Discovery
Observability:
● Distributed Tracing
● Network Telemetry
Security:
● Point-to-point mTLS
ADDITIONAL CONCERNS

23.
@antweiss
ADDITIONAL CONCERNS
Service Discovery
Observability:
● Distributed Tracing
● Network Telemetry
Security:
● Point-to-point mTLS
Continuous Delivery

24.
@antweiss
ADDITIONAL CONCERNS
Service Discovery
Observability:
● Distributed Tracing
● Network Telemetry
Security:
● Point-to-point mTLS
Progressive Delivery

25.
@antweiss
Progressive Delivery Strategies
● Rolling updates

26.
@antweiss
● Rolling updates
● Dark Launch
Progressive Delivery Strategies

27.
@antweiss
● Rolling updates
● Dark Launch
● Blue-Green
Progressive Delivery Strategies

28.
@antweiss
● Rolling updates
● Dark Launch
● Blue-Green
● Canary
Progressive Delivery Strategies

29.
@antweiss
● Rolling updates
● Dark Launch
● Blue-Green
● Canary
● Traffic Mirroring
Kiaaaiiii !!
Progressive Delivery Strategies

30.
@antweiss
Go to a Library?
Finagle
Ribbon
Hystrix
Zipkin
Spring-sleuth
Polly (for .NET)

31.
@antweiss
The Cloud-Native Answer: Service Mesh
Istio

32.
@antweiss
Istio Perks
● Smart Routing and Load Balancing

33.
@antweiss
● Smart Routing and Load Balancing
● Detailed Network Telemetry
Istio Perks

34.
@antweiss
● Smart Routing and Load Balancing
● Detailed Network Telemetry
● Security (TLS) Built-In
Istio Perks

35.
@antweiss
● Smart Routing and Load Balancing
● Detailed Network Telemetry
● Security (TLS) Built-In
● Platform(?) and Language Independent
Istio Perks

36.
@antweiss
● Smart Routing and Load Balancing
● Detailed Network Telemetry
● Security (TLS) Built-In
● Platform(?) and Language Independent
No Changes to App Code!!! (Well, almost)
Istio Perks

37.
@antweiss
ISTIO ARCHITECTURE

38.
@antweiss
Demo App
front
bethaleph
envoy envoy
envoy
istio-ingress-gateway
envoy
https://github.com/otomato-gh/istio-games

39.
@antweiss
YAM
L
who?

40.
@antweiss
The BirdWatch Loop
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: birdwatches.otomato.link
WAIT
FOR
CANARY
Found Canary:
Service: aleph
Version: 0.4
Release
some
traffic to
the canary
Canary
Healthy?
Canary Flies to
Production
yes no
Canary Dies
https://github.com/otomato-gh/birdwatch-otomator

41.
@antweiss
Real NInjas Use:
K8S
Operators
Manage Complex (Stateful)
Apps
K8S
Otomators
Automate Cluster
Management
VS.
https://github.com/otomato-gh/birdwatch-otomator

42.
@antweiss
● Won’t DevOps get Stuck in the Mesh?
SOME HARD QuestIONS:

43.
@antweiss
SOME HARD QuestIONS:
● Won’t DevOps get Stuck in the Mesh?
● Who’s Gonna Build the Otomators?

44.
@antweiss
SOME HARD QuestIONS:
● Won’t DevOps get Stuck in the Mesh?
● Who’s Gonna Build the Otomators?
● Why Python and not Golang?

45.
@antweiss
SUMMARIZE:
● Service Mesh Protects Us From the Traps
● Solves the Shared Libraries Issues
● A Good Fit if you like Centralised Control
● Smarter Pipelines -> Dumber Services
● Without Otomators The Mesh Can Get Clogged
● Dumb Services in a Clogged Mesh = Catastrophe
● Therefore Real Ninjas use Otomators

46.
@antweiss
The Future:
● Alternatives:
○ Linkerd 2.0 (from Bouyant)
○ Consul Connect (Hashicorp)
● Commercial Options
○ Aspen Mesh (F5 incubator)
○ AWS, Azure, GCP
● SMI Spec - the Standard Service Mesh API

47.
@antweiss
Thank You
@antweiss
http://otomato.link
http://devopstrain.pro