4. INSERT DESIGNATOR, IF NEEDED4
DISTRIBUTED SERVICES
ARCHITECTURES
Fallacies of Distributed Computing
● The network is reliable.
● Latency is zero.
● Bandwidth is infinite.
● The network is secure.
● Topology doesn't change.
● There is one administrator.
● Transport cost is zero.
● The network is homogeneous.
wikipedia.org/wiki/Fallacies_of_distributed_computing
5. INSERT DESIGNATOR, IF NEEDED5
DISTRIBUTED SERVICES
ARCHITECTURES
Applications must deal with
● Unpredictable failure modes
● End-to-end application correctness
● System degradation
● Topology changes
● Elastic/ephemeral/transient resources
A
E
B C
F G
D
H
I
Client
6. INSERT DESIGNATOR, IF NEEDED6
POSSIBLE SOLUTIONS
Today, Developers do this:
● Circuit Breaking
● Bulkheading
● Timeouts/Retries
● Service Discovery
● Client-side Load Balancing
7. INSERT DESIGNATOR, IF NEEDED7
TOO MUCH INFRASTRUCTURE IN BUSINESS
LOGIC
Platform
Microservice
Netflix OSS
Netflix OSS
Business Logic
Container Platform
Microservice
Business Logic
9. Istio Control Plane
ISTIO - A ROBUST SERVICE MESH FOR
MICROSERVICES
Istio Pilot Istio Mixer Istio Auth
Pod
Container
Service A
Envoy Proxy
Pod
Container
Service A
Envoy Proxy
Pod
Container
Service A
Envoy ProxyIstio Data
Plane
● service discovery
● load balancing
● TLS termination
● HTTP/2 & gRPC proxying,
● circuit breakers,
● health checks,
● staged rollouts fault injection
● rich metrics.
● access control
● usage policies
● telemetry collection
● traffic mgmt
● discovery
● authentication
● policy enforcement
● Id & credentials
11. GOAL FOR LAB
In this lab you will learn:
● How to install Istio onto OpenShift Container Platform
● How to deploy apps with sidecar proxies
● How to generate and visualize deep metrics for apps
● How to alter routing dynamically
● How to inject faults for testing
● How to do rate limiting
● How Istio implements circuit breaking and distributed tracing
12. LAB: DETECTING AND PREVENTING ISSUES
IN DISTRIBUTED APPS WITH ISTIO
SCENARIO 7 PREVENT AND DETECT ISSUES IN A DISTRIBUTED SYSTEM
14. RESULT OF LAB
In this lab you learned:
● How to install Istio onto OpenShift Container Platform
● How to deploy apps with sidecar proxies
● How to generate and visualize deep metrics for apps
● How to alter routing dynamically
● How to inject faults for testing
● How to do rate limiting
● How Istio implements circuit breaking and distributed tracing
● Use cases for service mesh
Why do we want to architect apps from a distributed services standpoint:
The benefits, we hope to achieve are primary related to:
Performance
Reliability
Resiliency
Extensibility
Availability
Robustness
Often application architectures must write code to facilitate these elements rather then just concentrating on core business logic.
Why is it important to ensure robustness among these distributed services elements?
Because of this list of fallacies in Distributed Computing environments:
The network is reliable.
Latency is zero.
Bandwidth is infinite.
The network is secure.
Topology doesn't change.
There is one administrator.
Transport cost is zero.
The network is homogeneous.
We typically can not trust that these items will be available in our environments….
Therefore, applications must deal with:
Unpredictable failure modes
End-to-end application correctness
System degradation
Topology changes
Elastic/ephemeral/transient resources or resources that simply go away…
One can argue that a highly available cluster environment, such as OpenShift running at scale would help mitigate many of these issues….?
What do you all think…?
Depending on the specific context, I’m good with this though…
We’ve seen how health-checks work today…
We understand how OCP can schedule multiple pods within the cluster adding resiliency to our individual micro-services.
We’ve seen pipeline automation for CI/CD and testing scenarios complete with advanced deployment techniques -- again, working together to ensure robustness
So, how can we make this environment even more resilient….?
Entire suites of frameworks were built to help developers address these resilience concerns… (e.g. netflix oss)
Netflix OSS has been the de facto way of building these supporting services for running microservices which required a number of standalone services to maintain and operate as well as boilerplate code that is required to talk to these services
Netflix OSS while a great collection of frameworks and libraries was build a couple of years ago for the VM-based AWS infrastructure.
Fast forward to today, many of the services that didn’t exist at that time for the VMs and Netflix had to invent them, are provided as base services in container platforms
However, recognizing that there is too much infrastructure in our business logic… what other solutions can we turn too?
entire suites of frameworks so for every language/framework combination, you need...
service discovery
retries
timeouts
load balancing
bulk heading
circuit breaking
rate limiting
built to help developers address these resilience concerns (e.g. netflix oss)
adaptive routing
deadlines
back pressure
outlier detection
health checking
traffic shaping
request shadowing
edge/dmz routing
surgical / fine / per-request routing
A/B testing rollout
dark launches
fault injection
stats, metric collection
observability
Originally built at Lyft, Envoy is a high performance C++ distributed proxy designed for single services and applications, as well as a for large scale microservice “service mesh” architectures.
Built on the learnings of solutions such as NGINX, HAProxy, hardware load balancers, and cloud load balancers.
Envoy runs alongside every application (Pod -- the concept of a side-car) and abstracts the network by providing common features in a platform-agnostic manner.
When all service traffic in an infrastructure flows via an Envoy mesh, it becomes easy to visualize problem areas via consistent observability, and tune overall performance.
As a service mesh grows in size and complexity, it can become harder to understand and manage. Istio adds requirements including: discovery, load balancing, failure recovery, metrics, and monitoring, and often more complex operational requirements such as A/B testing, canary releases, rate limiting, access control, and end-to-end authentication.
Traffic Management. Control the flow of traffic and API calls between services, make calls more reliable, and make the network more robust in the face of adverse conditions.
Observability. Gain understanding of the dependencies between services and the nature and flow of traffic between them, providing the ability to quickly identify issues.
Policy Enforcement. Apply organizational policy to the interaction between services, ensure access policies are enforced and resources are fairly distributed among consumers. Policy changes are made by configuring the mesh, not by changing application code.
Service Identity and Security. Provide services in the mesh with a verifiable identity and provide the ability to protect service traffic as it flows over networks of varying degrees of trustability.
In addition to these behaviors, Istio is designed for extensibility to meet diverse deployment needs:
Platform Support. Istio is designed to run in a variety of environments including ones that span Cloud, on-premise, Kubernetes, Mesos etc. We’re initially focused on Kubernetes but are working to support other environments soon.
Integration and Customization. The policy enforcement component can be extended and customized to integrate with existing solutions for ACLs, logging, monitoring, quotas, auditing and more.
The goal: Is that these capabilities greatly decrease the coupling between application code, the underlying platform, and policy. This decreased coupling not only makes services easier to implement, but also makes it simpler for operators to move application deployments between environments or to new policy schemes. Applications become inherently more portable as a result.
***Istio currently only supports service deployment on Kubernetes, though other environments will be supported in future versions.
entire suites of frameworks so for every language/framework combination, you need...
service discovery
retries
timeouts
load balancing
bulk heading
circuit breaking
rate limiting
built to help developers address these resilience concerns (e.g. netflix oss)
adaptive routing
deadlines
back pressure
outlier detection
health checking
traffic shaping
request shadowing
edge/dmz routing
surgical / fine / per-request routing
A/B testing rollout
dark launches
fault injection
stats, metric collection
observability
Earlier this year Google, IBM and Lyft announced the Istio project, which aims to make it easier to develop and connect AND manage our complex and distributed microservices applications. First generation microservices were mainly A DIY effort - you had to do a lot of defensive programming in the app itself. Istio basically delivers on the “microservices 2.0” idea of minimizing requirements on developers for dealing with the distributed nature of their apps.
In this diagram from Christian Posta, chief architect for cloud app dev at red hat, you can roughly see how this translates to apps - it pushes the real magic behind distributed apps like routing, rate limiting or circuit breaking into lower networking layers, out of reach and out of control of app developers. It also means that the service mesh benefits all applications running on it, regardless of programming language and communication protocol - think databases or IoT binary streaming apps.
An Istio service mesh is logically split into a data plane and a control plane.
The data plane is composed of a set of intelligent proxies (Envoy) deployed as sidecars that mediate and control all network communication between microservices.
The control plane is responsible for managing and configuring proxies to route traffic, as well as enforcing policies at runtime.
The following diagram shows the different components that make up each plane:
##################
To give you an idea of what this looks like, here’s a rough diagram of the components at play. Istio creates a cross-cutting platform-level service mesh to address common microservices architecture concerns, and there are a lot of them: communication, load balancing, traffic routing, metrics, quotas, authentication, rate limiting, circuit breakers, timeouts, automatic retries, and on and on.. Things that developers and operations have to deal with today. It does this by injecting we are called side car proxies for each service, acting as a frontend to the service and managing traffic to it according to policy. As services come and go, their presence, absence, or general health are tracked by The control plane components and traffic is shaped accordingly.
Red Hat customers and the greater OpenShift and Kubernetes communities will benefit from this platform level support for microservice architectures, so stay tuned as we work to bring a developer preview of this technology by Red Hat Summit in May and then later on in the year fully integrate it into OpenShift and the RHOAR getting started experience.
To give you an idea of what this looks like, here’s a rough diagram of the components at play. Istio creates a cross-cutting platform-level service mesh to address common microservices architecture concerns, and there are a lot of them: communication, load balancing, traffic routing, metrics, quotas, authentication, rate limiting, circuit breakers, timeouts, automatic retries, and on and on.. Things that developers and operations have to deal with today. It does this by injecting we are called side car proxies for each service, acting as a frontend to the service and managing traffic to it according to policy. As services come and go, their presence, absence, or general health are tracked by The control plane components and traffic is shaped accordingly.
Red Hat customers and the greater OpenShift and Kubernetes communities will benefit from this platform level support for microservice architectures, so stay tuned as we work to bring a developer preview of this technology by Red Hat Summit in May and then later on in the year fully integrate it into OpenShift and the RHOAR getting started experience.
We’re going to use a sample app within the Istio framework to run through these activities….
################
To give you an idea of what this looks like, here’s a rough diagram of the components at play. Istio creates a cross-cutting platform-level service mesh to address common microservices architecture concerns, and there are a lot of them: communication, load balancing, traffic routing, metrics, quotas, authentication, rate limiting, circuit breakers, timeouts, automatic retries, and on and on.. Things that developers and operations have to deal with today. It does this by injecting we are called side car proxies for each service, acting as a frontend to the service and managing traffic to it according to policy. As services come and go, their presence, absence, or general health are tracked by The control plane components and traffic is shaped accordingly.
Red Hat customers and the greater OpenShift and Kubernetes communities will benefit from this platform level support for microservice architectures, so stay tuned as we work to bring a developer preview of this technology by Red Hat Summit in May and then later on in the year fully integrate it into OpenShift and the RHOAR getting started experience.
We’re going to use a sample app within the Istio framework to run through these activities….Here’s a diagram of the logical architecture for our last scenario.