Building Cloud-Native App Series - Part 5 of 11
Microservices Architecture Series
Microservices Architecture,
Monolith Migration Patterns
- Strangler Fig
- Change Data Capture
- Split Table
Infrastructure Design Patterns
- API Gateway
- Service Discovery
- Load Balancer
1. @arafkarsh arafkarsh
ARAF KARSH HAMID
Co-Founder / CTO
MetaMagic Global Inc., NJ, USA
@arafkarsh
arafkarsh
Microservices
Architecture Series
Building Cloud Native Apps
Microservices,
Monolithic Migration
Containers / Kubernetes
Infrastructure Design Patterns
Part 5 of 11
2. @arafkarsh arafkarsh 2
Slides are color coded based on the topic colors.
Microservices
Architecture
1
Monolithic to
Microservices Migration
2
Intro to
Containers &
Kubernetes
3
Infrastructure Design
Patterns 4
10. @arafkarsh arafkarsh 10
100s Microservices
1,000s Releases / Day
10,000s Virtual Machines
100K+ User actions / Second
81 M Customers Globally
1 B Time series Metrics
10 B Hours of video streaming
every quarter
Source: NetFlix: : https://www.youtube.com/watch?v=UTKIT6STSVM
10s OPs Engineers
0 NOC
0 Data Centers
So what do NetFlix think about DevOps?
No DevOps
Don’t do lot of Process / Procedures
Freedom for Developers & be Accountable
Trust people you Hire
No Controls / Silos / Walls / Fences
Ownership – You Build it, You Run it.
11. @arafkarsh arafkarsh 11
50M Paid Subscribers
100M Active Users
60 Countries
Cross Functional Team
Full, End to End ownership of features
Autonomous
1000+ Microservices
Source: https://microcph.dk/media/1024/conference-microcph-2017.pdf
1000+ Tech Employees
120+ Teams
12. @arafkarsh arafkarsh
Microservices definition
12
In short, the microservice architectural style is an approach to
developing a single application as a suite of small services, each
running in its own process and communicating with lightweight
mechanisms, often an HTTP resource API. These services are built
around business capabilities and independently deployable by
fully automated deployment machinery. There is a bare minimum
of centralized management of these services, which may be
written in different programming languages and use different
data storage technologies.
https://martinfowler.com/articles/microservices.html
By James Lewis and Martin Fowler
Bolo Definition Kya hai?
Tell me what’s the definition
13. @arafkarsh arafkarsh
Microservices Characteristics
13
We can scale our operation independently, maintain
unparalleled system availability, and introduce new
services quickly without the need for massive
reconfiguration. —
Werner Vogels, CTO, Amazon Web Services
Modularity ... is to a technological economy
what the division of labor is to a
manufacturing one.
W. Brian Arthur,
author of e Nature of Technology
The key in making great and growable systems is
much more to design how its modules communicate
rather than what their internal properties and
behaviors should be.
Alan Kay, 1998 email to the Squeak-dev list
Components
via
Services
Organized around
Business
Capabilities
Products
NOT
Projects
Smart
Endpoints
& Dumb Pipes
Decentralized
Governance &
Data Management
Infrastructure
Automation
Design for
Failure
Evolutionary
Design
14. @arafkarsh arafkarsh
When should I use them (Microservices)?
14
• Strong Module Boundaries:
Microservices reinforce modular
structure, which is particularly important
for larger teams.
• Independent Deployment: Simple
services are easier to deploy, and since
they are autonomous, are less likely to
cause system failures when they go
wrong.
• Technology Diversity: With microservices
you can mix multiple languages,
development frameworks and data-
storage technologies.
When you have What’s the Cost
Distribution: Distributed systems are harder to
program, since remote calls are slow and are
always at risk of failure.
Eventual Consistency: Maintaining strong
consistency is extremely difficult for a
distributed system, which means everyone has
to manage eventual consistency.
Operational Complexity: You need a mature
operations team to manage lots of services,
which are being redeployed regularly.
Source: https://www.martinfowler.com/microservices/
15. @arafkarsh arafkarsh
What is the right size for a Microservice?
15
• Rather than the size what matters is the Business Function /
Domain of the service.
• One Microservice may have half a dozen entities and other a
couple of dozen entities. What’s more important is the role
Microservices plays.
• Bounded Context from DDD helps you to decompose a large
multi domain Monolith into a Microservice for each Bounded
Context.
• Focusing on User stories will help you clearly define the
boundaries of the Business Domain.
17. @arafkarsh arafkarsh
17
Monolithic vs. Microservices Example
Traditional Monolithic App using
Single Technology Stack
Micro Services with Multiple Technology Stack
This 3-tier model is obsolete now.
Source: Gartner Market Guide for Application
Platforms. Nov 23, 2016
Event Stream / Queues / Pub-Sub / Storage
UI Layer
Web Services
Business Logic
Database Layer
Micro
Service
4
EE 7
Inventory
UI Layer
Web Services
Business Logic
Database Layer
Micro
Service
1
Customer
SE 8
UI Layer
Web Services
Business Logic
Database Layer
Micro
Service
3
Shopping
Cart
UI Layer
Web Services
Business Logic
Database Layer
Micro
Service
2
Order
UI Layer
WS
BL
DL
Database
Shopping
Cart
Order
Customer
Inventory
API Gateway (Zuul Edge Server)
Load Balancer
(Ribbon)
Circuit Breaker
(Hystrix)
Service Discovery (Eureka)
Load Balancer
(Ribbon)
Circuit Breaker
(Hystrix)
Load Balancer
(Ribbon)
Circuit Breaker
(Hystrix)
Load Balancer
(Ribbon)
Circuit Breaker
(Hystrix)
12
18. @arafkarsh arafkarsh 18
SOA vs. Microservices Example
Traditional Monolithic App with SOA
Micro Services with Multiple Technology Stack
Event Stream / Queues/ Pub-Sub / Storage
UI Layer
Web Services
Business Logic
Database Layer
Micro
Service
1
Customer
SE 8
UI Layer
Web Services
Business Logic
Database Layer
Micro
Service
3
Shopping
Cart
UI Layer
Web Services
Business Logic
Database Layer
Micro
Service
2
Order
API Gateway
Load Balancer
Circuit Breaker
Service Discovery
Load Balancer
Circuit Breaker
Load Balancer
Circuit Breaker
UI Layer
Database
Shopping
Cart
Order
Customer
Inventory
Enterprise Service
Bus
Messaging
REST / SOAP
HTTP
MOM
JMS
ODBC / JDBC
Translation
Web Services
XML
WSDL
Addressing
Security
Registry
Management
Producers
Shared
Database
Consumers
3rd Party Apps
Smart Pipes
Lot of Business logic
resides in the Pipe
19. @arafkarsh arafkarsh
Microservices Deployment Model
Microservices with Multiple Technology Stack – Software Stack for Networking
Event Stream / Queues / Pub-Sub / Storage
Users
Service
Discovery
(Eureka)
Config
Server
(Spring)
API (Zuul) Gateway
UI Layer
Web Services
Business Logic
Database Layer
Micro
Service
2
Shopping
Cart
SE 8
LB = Ribbon
CB = Hystrix
LB = Ribbon
CB = Hystrix
UI Layer
Web Services
Business Logic
Database Layer
Product
SE 8
Micro Service 1
With 4 node cluster
LB = Ribbon
CB = Hystrix
UI Layer
Web Services
Business Logic
Database Layer
Order
SE 8
Micro Service 3
With 2 node
Cluster
LB = Ribbon
CB = Hystrix
UI Layer
Web Services
Business Logic
Database Layer
Customer
Micro Service 4
With 2 node
cluster
HTTP Server
All UI Code is
bundled
Virtual
Private
Network
19
20. @arafkarsh arafkarsh
Shopping Portal – Docker / Kubernetes – Network Stack
20
/ui
/cart
/order
Load Balancer
Ingress
Deployment / Replica / Pod Nodes
Kubernetes Objects
Firewall
Cart Pod
Cart Pod
Cart Pod
Cart Service
N4
N3
MySQL
DB
EndPoints
Internal
Load Balancers
Users
Routing based on Layer 3,4 and 7
Order Pod
Order Pod
Order Pod
Order Service
N4
N3
N1
EndPoints
Mongo
DB
UI Pod
UI Pod
UI Pod
UI Service
N1
N2
N2
EndPoints
Redis
DB
21. @arafkarsh arafkarsh
Shopping Portal – Docker / Kubernetes – Network Stack
21
/customer
/product
/review
Load Balancer
Ingress
Deployment / Replica / Pod Nodes
Kubernetes Objects
Firewall
Customer Pod
Customer Pod
Customer Pod
Customer
Service
N1
N2
N2
EndPoints
Product Pod
Product Pod
Product Pod
Product
Service
N4
N3
MySQL
DB
EndPoints
Review Pod
Review Pod
Review Pod
Review
Service
N4
N3
N1
Service Call
Kube DNS
EndPoints
Internal
Load Balancers
Users
Routing based on Layer 3,4 and 7
Redis
DB
Mongo
DB
22. @arafkarsh arafkarsh
Software Network Stack Vs Network Stack
22
Pattern Software Stack Java Software Stack .NET Kubernetes
1 API Gateway Zuul Server SteelToe Istio Envoy
2 Service Discovery Eureka Server SteelToe Kube DNS
3 Load Balancer Ribbon Server SteelToe Istio Envoy
4 Circuit Breaker Hysterix SteelToe Istio
5 Config Server Spring Config SteelToe Secrets, Env - K8s Master
Web Site https://netflix.github.io/ https://steeltoe.io/ https://kubernetes.io/
The Developer needs to write code to integrate with the Software Stack
(Programming Language Specific. For Ex. Every microservice needs to subscribe to
Service Discovery when the Microservice boots up.
Service Discovery in Kubernetes is based on the Labels assigned to Pod and Services
and its Endpoints (IP Address) are dynamically mapped (DNS) based on the Label.
26. @arafkarsh arafkarsh
Scale Cube and Micro Services
26
1. Y Axis Scaling – Functional Decomposition : Business Function as a Service
2. Z Axis Scaling – Database Partitioning : Avoid locks by Database Sharding
3. X Axis Scaling – Cloning of Individual Services for Specific Service Scalability
27. @arafkarsh arafkarsh 27
12
Factor
App
Methodology
4 Backing Services Treat Backing services like DB, Cache as attached resources
5 Build, Release, Run Separate Build and Run Stages
6 Process Execute App as One or more Stateless Process
7 Port Binding Export Services with Specific Port Binding
8 Concurrency Scale out via the process Model
9 Disposability Maximize robustness with fast startup and graceful exit
10 Dev / Prod Parity
Keep Development, Staging and Production as similar as possible
Checkout the Shift – Left in DevOps (Slide 157)
11 Logs Treat logs as Event Streams
12 Admin Process
Run Admin Tasks as one of Process (Eg. DB Migration, Software
upgrades, etc..)
Factors Description
1 Codebase One Code base tracked in revision control
2 Dependencies Explicitly declare dependencies
3 Configuration Configuration driven Apps
Source:
https://12factor.net/
28. @arafkarsh arafkarsh
Catalogues of Microservices
28
System Z Model From Spotify
• Different types of Components Z Supports
• Libraries
• Data Pipelines
• Views in the client
• Data Store
• Service
29. @arafkarsh arafkarsh
Pros
1. Adds Complexity
2. Skillset shortage
3. Confusion on getting the
right size
4. Team need to manage
end-to-end of the
Service (From UI to
Backend to Running in
Production).
29
1. Robust
2. Scalable
3. Testable (Local)
4. Easy to Change and
Replace
5. Easy to Deploy
6. Technology Agnostic
Cons
Microservices Pros and Cons
30. @arafkarsh arafkarsh
Monolithic > Microservices
• Forrester Research
• Modernization journey
• Assess and classify your app portfolio
• Plan and prioritize
30
2
31. @arafkarsh arafkarsh 31
Any Organization that designs a system
(defined broadly) will produce a design
whose structure is a copy of the
organization’s communication structure.
32. @arafkarsh arafkarsh
* For IT Services : They can do one more project of same size with ZERO COST in Platform Licensing (Based on 20 developer pack USD $50K per month for 3 months License cost = $150K)
32
33. @arafkarsh arafkarsh
Agile
Scrum (4-6 Weeks)
Developer Journey
Monolithic
Domain Driven Design
Event Sourcing and CQRS
Waterfall
Optional
Design
Patterns
Continuous Integration (CI)
6/12 Months
Enterprise Service Bus
Relational Database [SQL]
Development QA / QC Ops
33
Microservices
Domain Driven Design
Event Sourcing and CQRS
Scrum / Kanban (1-5 Days)
Mandatory
Design
Patterns
Infrastructure Design Patterns
CI
DevOps
Event Streaming / Replicated Logs
SQL NoSQL
CD
Container Orchestrator Service Mesh
34. @arafkarsh arafkarsh
Modernization Journey
34
Start new features as Microservices
Incrementally establish the success early.
Expose Legacy On-Premise Apps API’s
If legacy Apps cant be shifted to Cloud
Refactor Monolithic features to Microservices
Breakdown and Deploy Feature by Feature
Containerize the Microservice
Reduce costs, simplifies the operations and consistent
environment between Dev, QA and Production
Monolith De-commission Plan
Incrementally sunset the Monolith
Velocity as you
transform
Increase your
Delivery Velocity
along the Journey
High
Future
Present
Low
Inspired by a paper from IBM
35. @arafkarsh arafkarsh
Assess and Classify your App Portfolio
35
Take inventory of your Apps
Classify the Apps based on
technology, complexity.
Align Apps to your Business
Priorities
Identify the Apps that are critical for
Modernization.
Identify Business
Modernization Requirements
Create a Roadmap with faster go to
market strategy
Understand the effort and
Cost
Evaluate all possible Modernization
options
Container Refactor Expose APIs
Lift & Shift
Business
Value
Cost
Complexity
Product Catalogue
Product Review
Inventory Shopping Cart
Customer
Profile
Order Management
Inspired by a paper from IBM
36. @arafkarsh arafkarsh
Lift and Shift
Rehost:
o Moving an On-Premise App without any redesign to the
Cloud Service Provider.
o Migration can be fast and relatively inexpensive.
o However, the OpEx can be on higher side as Application is
not using leveraging the Cloud efficiencies.
Quick Savings
o Down Jones lowered IT costs by more than 25%.
o GE Oil & Gas realized 52% cost savings
Source: https://www.netapp.com/knowledge-center/what-is-lift-and-shift/.
36
37. @arafkarsh arafkarsh
Design Pattern
1 Decomposition Identify the Modules and Boundaries
2 Strangler Fig Product Service (Query Only with Apache Solr)
3 Branch By Abstraction Shopping Cart as Service using Monolithic DB
4 Decorating Collaborator New Shipping Service with Database
5 Change Data Capture Enhance Loyalty Module as a new Service
6 Aggregate API Expose Customer API on Monolith
7 Split Table
Product Split into 2 Services as Product Management
and Warehouse
8 Change Data Ownership Order Service using Monolithic DB
9 Move Foreign Key to Code
Foreign Key in Order (item) with Product Table moved to
Order Service Code
10 Strangler Fig Move out Customer Module and decommission the Monolith
Shopping Portal Legacy Migration Plan
• Prioritize the Modules / Services For Implementation & Deployment
Setup
Transition Plan
UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product
Monolith
37
38. @arafkarsh arafkarsh
1. Decomposition Pattern – Tier 2 : Backend
Shopping Portal Monolithic DB
Product
Catalogue
Reviews
Product
Order Item
Shipping
Methods
Address
Payments
Inventory
Items
Category
Inventory Event
Cart Items
Wish List Price Event
Category
Order
Added From
Cart
uses
uses
Customer
Brand
Addresses
Cards
Shopping Portal Monolithic App
1. Focus on the User Journey
2. Identify the Epics and User Stories
3. Consider each Epic as a Cohesive
Unit
Epics
1. Customer Management
2. Product Catalogue
3. Shopping Cart
4. Order Management
5. Inventory System
Source: Monolith to Microservices By Sam Newman
38
39. @arafkarsh arafkarsh
1. Decomposition Pattern
An e-Comm App User’s Journey can
run across multiple Bounded
Context / Microservices.
User Journey X
Product
Catalogue
Shopping
Cart
Order
User Journey Y
Bounded
Context
Bounded
Context
Bounded
Context
Shopping Portal Monolithic DB
Shopping Portal Monolithic App
Product
Catalogue
Reviews
Product
Order Item
Shipping
Methods
Address
Payments
Inventory
Items
Category
Inventory Event
Cart Items
Wish List Price Event
Category
Order
Added From
Cart
uses
uses
Customer
Brand
Addresses
Cards
Customer
Order
Cart
Product
Inventory
Source: Monolith to Microservices By Sam Newman
39
40. @arafkarsh arafkarsh
1. Decomposition Pattern – Tier 3 : Database
An e-Comm App User’s Journey can
run across multiple Bounded
Context / Microservices.
User Journey X
Product
Catalogue
Shopping
Cart
Order
User Journey Y
Bounded
Context
Bounded
Context
Bounded
Context
Customer
Shopping Portal Monolithic App
Product
Catalogue
Reviews
Product
Order Item
Shipping
Methods
Address
Payments
Inventory
Items
Category
Inventory Event
Cart Items
Wish List Price Event
Category
Order
Added From
Cart
uses
uses
Customer
Brand
Addresses
Cards
Customer
Order
Cart
Product
Inventory
Cart
Order
Product
Inventory
Source: Monolith to Microservices By Sam Newman
40
41. @arafkarsh arafkarsh
Plan and Prioritize
41
Complexity Cost Value Score Rank
Weightage 35% 25% 40%
Customer
Med
3
Med
3
Low
1
2.20
7
6
Product
Reviews
Med
3
High
5
Med
3
3.50
11
3
Product
Catalogue
Med
3
Med
3
High
5
4.80
11
1
Shopping
Cart
High
5
Med
3
Med
3
3.70
11
4
Order
Very High
7
Med
3
High
5
5.20
15
2
Inventory
Very High
7
High
5
Med
3
4.90
15
5
Prioritize
Low Priority projects are
good test cases but does not
bring much business value.
Quick Wins
Identify a feature / project
which has good business
value and low in complexity.
Project Duration
Focus on shorter duration
projects with high Business
Value.
Shopping Portal Features Prioritization
Inspired by a paper from IBM
42. @arafkarsh arafkarsh
2. Strangler Fig Pattern
API Gateway / Proxy
Web Services
Business Logic
Database Layer
Product
Micro
Service
Product
SE 8
UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product
UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product
Web Services
Business Logic
Database Layer
Product
Micro
Service
Product
SE 8
API Gateway / Proxy
Step 1
Identity the Module
Step 2
Move the Service
Step 3
Redirect the Calls to New Service
Source: Monolith to Microservices By Sam Newman
1. New Service
(Code)
2. Better Search
Capabilities
Product DB will be used
by other modules in
Monolithic App
Monolith Monolith Monolith
Only used for Search
Capabilities and
Scalability.
42
43. @arafkarsh arafkarsh
3. Branch By Abstraction
UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product
Web Services
Business Logic
Database Layer
Cart Service
Uses Monolithic DB
Cart
SE 8
Step 1
Create an Abstraction for
Product Module
Step 2
Use Abstraction
Cart Mono
Implements
Cart Service
UI Layer
WS
BL
DL
Database
Customer
Cart
Step 3
Create new
Implementation
UI Layer
WS
BL
DL
Database
Customer
Cart
Web Services
Business Logic
Database Layer
Cart Service
Uses Monolithic DB
Cart
SE 8
Step 4
Switch Implementation
UI Layer
WS
BL
DL
Database
Customer
Cart
UI Layer
WS
BL
DL
Database
Customer
Step 5
Clean up
Web Services
Business Logic
Database Layer
Cart Service
Uses Monolithic DB
Cart
Monolith Monolith
Source: Monolith to Microservices By Sam Newman
43
44. @arafkarsh arafkarsh
4. Decorating Collaborator Pattern
WS
BL
DL
Database
Cart
Order
Customer
Product
Monolith
Request Response
Order Proxy
Web Services
Business Logic
Database Layer
Shipping
Micro
Service
Shipping
Place
Order
Success
/ Failed If Success
Proceed for
Shipping
1. In the Strangler Fig Pattern Proxy did
the routing.
2. Order Proxy Sends the data to Shipping
Service
3. Eventually Order Proxy will completely
migrate to Order Microservice.
Source: Monolith to Microservices By Sam Newman
Add Shipping Feature integrated Order
Module if the Order is finalized.
1. Least number of changes to Monolith
2. Add new feature as new Microservice
44
45. @arafkarsh arafkarsh
5. Change Data Capture Pattern
WS
BL
DL
Cart
Order
Customer
Product
Monolith
Transaction Log
Insert into Loyalty
Values (“ABC”, “123-456-789”, 100)
Customer ID Loyalty ID Value
ABC 123-456-789 100
On Commit
Web Services
Business Logic
Database Layer
Loyalty
Micro
Service
Loyalty
1. Monolithic has basic Loyalty Feature
2. Loyalty Program is further enhanced in a Microservice.
Connector
Loyalty Topic
3. Initial Data Captured in
Monolithic is transferred to
Loyalty Microservice.
4. Further feature is developed
in Loyalty Service
5. CDC is better than Decorating
Collaborator in this Use Case
Source: Monolith to Microservices By Sam Newman
45
46. @arafkarsh arafkarsh
6. Aggregate Exposing Monolith Pattern
WS
BL
DL
Cart
Order
Customer
Product
Monolith
Web Services
Business Logic
Database Layer
Loyalty
Micro
Service
Loyalty
1. Customer Aggregate is Exposed as
Service for all the CRUD handling.
2. As Customer reference (As a
Foreign Key) available in multiple
modules, Shopping Cart, Order
etc.
3. It’s a good idea to keep the
Customer As an Aggregate Service
in the Monolith.
4. Until all the services are moved
out.
ID Customer Name Title
ABC John Doe CEO
API
Source: Monolith to Microservices By Sam Newman
46
47. @arafkarsh arafkarsh
7. Split Table Patterns
WS
BL
DL
Cart
Order
Customer
Product
Monolith
SKU Item Name Qty
ABC iPhone 12 610
DEF iPhone 11 987
GHI iPhone 10 1597
Web Services
Business Logic
Database Layer
Order
Micro
Service
Product
Admin
Web Services
Business Logic
Database Layer
Order
Micro
Service
Warehouse
SKU Item Name
ABC iPhone 12
DEF iPhone 11
GHI iPhone 10
SKU Qty
ABC 610
DEF 987
GHI 1597
1. Split the Table in the existing Monolithic DB
2. The Module Product Split into Product Admin and Warehouse
3. Each Microservice will has its own Split Table.
4. Make sure that only Soft Delete is allowed in Product table.
Source: Monolith to Microservices By Sam Newman
47
48. @arafkarsh arafkarsh
8. Change Data Ownership Pattern
WS
BL
DL
Cart
Order
Customer
Product
Monolith
Web Services
Business Logic
Database Layer
Order
Micro
Service
order
1. Order Module Created as a Service as uses Monolith as the Database..
2. In the Second Iteration Order Service will have its own DB Carved out of Monolith Application
3. Change of Order DB from Monolith to Microservices “Order Service”
WS
BL
DL
Cart
Customer
Product
Monolith
Web Services
Business Logic
Database Layer
Order
Micro
Service
order
Order
ID Customer SKU Qty Value
100 John Doe ABC 5 $100
100 John Doe DEF 2 $200
ID Customer SKU Qty Value
100 John Doe ABC 5 $100
100 John Doe DEF 2 $200
Source: Monolith to Microservices By Sam Newman
48
49. @arafkarsh arafkarsh
9. Move Foreign Key Relationship to Code Pattern
WS
BL
DL
Cart
Order
Customer
Product
Monolith
1. SKU in the Order is a Foreign Key to Product Table in the Monolith
2. When Order is Moved as a Separate Service SKU Foreign Key relationship is lost.
3. Move the Foreign Key Relationship into Order Service Code or Embrace Duplication - Copy Item
name in Product Table to Order when the Order is created.
Web Services
Business Logic
Database Layer
Order
Service
order
ID Customer SKU Qty Value
100 John Doe ABC 5 $100
100 John Doe DEF 2 $200
ID Customer SKU Qty Value
100 John Doe ABC 5 $100
100 John Doe DEF 2 $200
SKU Item Name
ABC iPhone 12
DEF iPhone 11
Web Services
Business Logic
Database Layer
Product
Admin
Service
Product
Admin
SKU Item Name
ABC iPhone 12
DEF iPhone 11
Source: Monolith to Microservices By Sam Newman
1.
Query the Order
2.
Get SKU
From Product
Admin Service
3.
Fetch SKU
49
50. @arafkarsh arafkarsh
Strangler Fig Pattern
API Gateway / Proxy
Web Services
Business Logic
Database Layer
Customer
Micro
Service
Customer
SE 8
UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product
UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product
Web Services
Business Logic
Database Layer
Customer
Micro
Service
Customer
SE 8
API Gateway / Proxy
Step 1
Identity the Module
Step 2
Move the Service
Step 3
Redirect the Calls to New Service
Source: Monolith to Microservices By Sam Newman
1. New Service
(Code)
2. Better Search
Capabilities
Monolith Monolith Monolith
• Customer Module the last Module migrated to Microservices
• Legacy Shopping Portal App is de-commissioned.
50
51. @arafkarsh arafkarsh
10. Parallel Run Pattern
51
/order
API Gateway
Firewall
Order v2
v2
v2
v1
Order v1
Order
Service
Destination
Rule
v1
Users
Production Data is mirrored to new release for real-time
testing
Source: Monolith to Microservices By Sam Newman
100% = v1
Mirror = v2
Scenario 1
95% = v1
5% = v2
Scenario 2
100% = v2
Scenario 3
52. @arafkarsh arafkarsh
Design Pattern
1 Decomposition Identify the Modules and Boundaries
2 Strangler Fig Product Service (Query Only with Apache Solr)
3 Branch By Abstraction Shopping Cart as Service using Monolithic DB
4 Decorating Collaborator New Shipping Service with Database
5 Change Data Capture Enhance Loyalty Module as a new Service
6 Aggregate API Expose Customer API on Monolith
7 Split Table
Product Split into 2 Services as Product Management
and Warehouse
8 Change Data Ownership Order Service using Monolithic DB
9 Move Foreign Key to Code
Foreign Key in Order (item) with Product Table moved to
Order Service Code
10 Strangler Fig Move out Customer Module and decommission the Monolith
Shopping Portal Legacy Migration Plan
• Prioritize the Modules / Services For Implementation & Deployment
Setup
Transition Plan
UI Layer
WS
BL
DL
Database
Cart
Order
Customer
Product
Monolith
52
53. @arafkarsh arafkarsh
Monolithic to Microservices Summary
1. Classify your Apps into Following areas
1. Lift and Shit
2. Containerize
3. Refactor
4. Expose API
2. Prioritize High Business Value Low
Technical Complexity
3. Focus on Shorter Duration – From
Specs to Operation
Migration Design Patterns
1. Decomposition
2. Strangler Fig Pattern
3. Branch By Abstraction
4. Decorating Collaborator
5. Change Data Capture
6. Aggregate API
7. Split Table
8. Change Data Ownership
9. Move Foreign Key to Code
10. Parallel Run
53
54. @arafkarsh arafkarsh
Discovering Microservices Principles….
Components
via
Services
Organized around
Business
Capabilities
Products
NOT
Projects
Smart
Endpoints
& Dumb Pipes
Decentralized
Governance &
Data Management
Infrastructure
Automation
Design for
Failure
Evolutionary
Design
What did we Learn Today?
54
55. @arafkarsh arafkarsh
Discovering Microservices Principles….
Components
via
Services
Organized around
Business
Capabilities
Products
NOT
Projects
Smart
Endpoints
& Dumb Pipes
Decentralized
Governance &
Data Management
Infrastructure
Automation
Design for
Failure
Evolutionary
Design
What did we Learn Today?
55
57. @arafkarsh arafkarsh
RESTful Guidelines
57
1. Endpoints as nouns, NOT verbs
Ex. /catalogues
/orders
/catalogues/products
and NOT
/getProducts/
/updateProducts/
2. Use plurals
Ex. /catalogues/{catalogueId}
and NOT
/catalogue/{catalogueId}
3. Documenting
4. Paging
5. Use SSL
6. HTTP Methods
GET / POST / PUT / DELETE / OPTIONS / HEAD
7. HTTP Status Codes (Effective usage)
8. Versioning
Media Type Version
GET /account/5555 HTTP/1.1
Accept: application/vnd.catalogues.v1+json
URL path version
https://domain/v1/catalogues/products
58. @arafkarsh arafkarsh
RESTful Guidelines – Query Examples
58
Search All
Products
Search Products By
Catalogue ID
Search Products By
Catalogue ID & Product ID
60. @arafkarsh arafkarsh 60
# Name * Who Uses Pros Cons
1
Media Type Versioning
Accept:
Application/vnd.api.article+xml;
version=1.0
Med GitHub
• Version Directly @
resource level
• Preserve URI
• Close to RESTful Specs
• Harder to Test
• Distort HTTP Headers purpose
• Tools required for testing
2
Custom Headers Versioning
X-API-Version: 2.
Med Microsoft • Preservers URI
• Harder to Test
• Tools required for testing
3
URI Versioning
api.example.com/v1/resource
High
Google
Twitter
Amazon
• Most common method
• Versions can be explored
using Browser
• Easy to use
• Disrupts RESTful Compliance.
URI should represent resource
and not versions
4
Domain Versioning
apiv1.example.com/resource
Low Facebook
• Same as are URI
Versioning
• Same as URI Versioning
5
Request Parameter
Versioning
GET /something/?version=0.1
High
Pivotal
NetFlix
• Similar to URI versioning • It can get messy
6
Date Versioning
First request saves the date.
Low Clearbit
• New APIs can be shipped
without changing the
end points
• Complex to implement
• Traceability is difficult.
API Versioning
61. @arafkarsh arafkarsh
Restful API Summary
61
o Endpoints as Nouns not VERBS
o /catalogues, /orders, /products/category
o API Versioning
o Use the best suited to your environment
o Use all the HTTP Verbs
o GET, POST, PUT, DELETE
63. @arafkarsh arafkarsh
Servers / Virtual Machines / Containers
Hardware
Host OS
HYPERVISOR
App 1 App 1 App 1
Guest
OS
BINS
/ LIB
Guest
OS
BINS
/ LIB
Guest
OS
BINS
/ LIB
Type 1 Hypervisor
App 2
App 3
App 2
OS
Hardware
Desktop / Laptop
BINS
/ LIB
App
BINS
/ LIB
App
Container 1 Container 2
Type 2 Hypervisor
Hardware
HYPERVISOR
App 1 App 1 App 1
Guest
OS
BINS
/ LIB
Guest
OS
BINS
/ LIB
Guest
OS
BINS
/ LIB
App 2
App 3
App 2
Guest OS
Hardware
Type 2 Hypervisor
BINS
/ LIB
App
BINS
/ LIB
App
BINS
/ LIB
App
Container 1 Container 2 Container 3
HYPERVISOR
Virtualizes the OS
Create Secure Sandboxes in OS
Virtualizes the Hardware
Creates Virtual Machines
Hardware
OS
BINS / LIB
App
1
App
2
App
3
Server
Data Center
No Virtualization
Cloud Elastic Computing
63
64. @arafkarsh arafkarsh
What’s a Container?
64
Virtual
Machine
Looks like a
Walks like a
Runs like a
Containers are a Sandbox inside Linux Kernel sharing the kernel with
separate Network Stack, Process Stack, IPC Stack etc.
65. @arafkarsh arafkarsh
Docker containers are Linux Containers
CGROUPS
NAME
SPACES
Copy on
Write
DOCKER
CONTAINER
65
• Kernel Feature
• Groups Processes
• Control Resource
Allocation
• CPU, CPU Sets
• Memory
• Disk
• Block I/O
• Images
• Not a File System
• Not a VHD
• Basically a tar file
• Has a Hierarchy
• Arbitrary Depth
• Fits into Docker
Registry
• The real magic behind
containers
• It creates barriers
between processes
• Different Namespaces
• PID Namespace
• Net Namespace
• IPC Namespace
• MNT Namespace
• Linux Kernel Namespace
introduced between
kernel 2.6.15 – 2.6.26
docker run
lxc-start
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01
66. @arafkarsh arafkarsh
Docker Container – Linux and Windows
66
Control Groups
cgroups
Namespaces
Pid, net, ipc, mnt, uts
Layer Capabilities
Union File Systems:
AUFS, btrfs, vfs
Control Groups
Job Objects
Namespaces
Object Namespace, Process
Table. Networking
Layer Capabilities
Registry, UFS like
extensions
Namespaces: Building blocks of the Containers
67. @arafkarsh arafkarsh
Linux Kernel
67
HOST OS (Ubuntu)
Client
Docker Daemon
Cent OS
Alpine
Debian
Linux
Kernel
Host Kernel
Host Kernel
Host Kernel
All the containers
will have the same
Host OS Kernel
If you require a
specific Kernel
version then Host
Kernel needs to be
updated
HOST OS (Windows 10)
Client
Docker Daemon
Nano Server
Server Core
Nano Server
Windows
Kernel
Host Kernel
Host Kernel
Host Kernel
Windows Kernel
68. @arafkarsh arafkarsh
Docker Images in the Github Workshop
68
Ubuntu
JRE 8 JRE 11
Tomcat 8 Tomcat 9
My App 1
Tomcat 9
My App 3
Spring Boot
My App 4
From Ubuntu
Build My Ubuntu
From My Ubuntu
Build My JRE8
From My Ubuntu
Build My JRE11
From My JRE 11
Build My Boot
From My Boot
Build My App 4
From My JRE8
Build My TC8
From My TC8
Build My App 1
My App 2
Source: https://github.com/meta-magic/kubernetes_workshop
69. @arafkarsh arafkarsh
Docker Daemon
Docker Client
How Docker works….
69
$ docker search ….
$ docker build ….
$ docker container create ..
Docker Hub
Images
Containers
$ docker container run ..
$ docker container start ..
$ docker container stop ..
$ docker container ls ..
$ docker push ….
$ docker swarm ..
2
1
3
4
1. Search for the Container
2. Docker Daemon Sends the request to Hub
3. Downloads the image
4. Run the Container from the image
70. @arafkarsh arafkarsh
Kubernetes Key Concepts
o Declarative Model
Infrastructure as a code
o Desired State
Required state of your App /
Microservice
o Current State
Current State of the App due to
error or load factor
Pods
Replicas
Deploy
Service
70
71. @arafkarsh arafkarsh
Deployment – Updates and rollbacks, Canary Release
D
ReplicaSet – Self Healing, Scalability, Desired State
R
Worker Node 1
Master Node (Control Plane)
Kubernetes
Architecture
71
POD
POD itself is a Linux
Container, Docker
container will run inside
the POD. PODs with single
or multiple containers
(Sidecar Pattern) will share
Cgroup, Volumes,
Namespaces of the POD.
(Cgroup / Namespaces)
Scheduler
Controller
Manager
Using yaml or json
declare the desired
state of the app.
State is stored in
the Cluster store.
Self healing is done by Kubernetes using watch loops if the desired state is changed.
POD POD POD
BE
1.2
10.1.2.34
BE
1.2
10.1.2.35
BE
1.2
10.1.2.36
BE
15.1.2.100
DNS: a.b.com 1.2
Service Pod IP Address is dynamic, communication should
be based on Service which will have routable IP
and DNS Name. Labels (BE, 1.2) play a critical role
in ReplicaSet, Deployment, & Services etc.
Cluster
Store
etcd
Key Value
Store
Pod Pod Pod
Label Selector selects pods based on the Labels.
Label
Selector
Label Selector
Label Selector
Node
Controller
End Point
Controller
Deployment
Controller
Pod
Controller
….
Labels
Internet
Firewall K8s Virtual
Cluster
Cloud Controller
For the cloud providers to manage
nodes, services, routes, volumes etc.
Kubelet
Node
Manager
Container
Runtime
Interface
Port 10255
gRPC
ProtoBuf
Kube-Proxy
Network Proxy
TCP / UDP Forwarding
IPTABLES / IPVS
Allows multiple
implementation of
containers from v1.7
RESTful yaml / json
$ kubectl ….
Port 443
API Server
Pod IP ...34 ...35 ...36
EP
• Declarative Model
• Desired State
Key Aspects
N1
N2
N3
Namespace
1
N1
N2
N3
Namespace
2
• Pods
• ReplicaSet
• Deployment
• Service
• Endpoints
• StatefulSet
• Namespace
• Resource Quota
• Limit Range
• Persistent
Volume
Kind
Secrets
Kind
• apiVersion:
• kind:
• metadata:
• spec:
Declarative Model
• Pod
• ReplicaSet
• Service
• Deployment
• Virtual Service
• Gateway, SE, DR
• Policy, MeshPolicy
• RbaConfig
• Prometheus, Rule,
• ListChekcer …
@
@
Annotations
Names
Cluster IP
Node
Port
Load
Balancer
External
Name
@
Ingress
72. @arafkarsh arafkarsh
Docker Networking Vs. Kubernetes Networking
72
Container 1
172.17.3.2
Web Server 8080
Veth: eth0
Container 2
172.17.3.3
Microservice 9002
Veth: eth0
Container 3
172.17.3.4
Microservice 9003
Veth: eth0
Container 4
172.17.3.5
Microservice 9004
Veth: eth0
IP tables rules
eth0
10.130.1.101/24
Node 1
Docker0 Bridge 172.17.3.1/16
Veth0 Veth1 Veth2 Veth3
Container 1
172.17.3.2
Web Server 8080
Veth: eth0
Container 2
172.17.3.3
Microservice 9002
Veth: eth0
Container 3
172.17.3.4
Microservice 9003
Veth: eth0
Container 4
172.17.3.5
Microservice 9004
Veth: eth0
IP tables rules
eth0
10.130.1.102/24
Node 2
Docker0 Bridge 172.17.3.1/16
Veth0 Veth1 Veth2 Veth3
Pod 1
172.17.3.2
Web Server 8080
Veth: eth0
Pod 2
172.17.3.3
Microservice 9002
Veth: eth0
Pod 3
172.17.3.4
Microservice 9003
Veth: eth0
Pod 4
172.17.3.5
Microservice 9004
Veth: eth0
IP tables rules
eth0
10.130.1.101/24
Node 1
L2 Bridge 172.17.3.1/16
Veth0 Veth1 Veth2 Veth3
Same IP Range. NAT Required Uniq IP Range. netFilter, IP Tables / IPVS. No NAT required
Pod 1
172.17.3.6
Web Server 8080
Veth: eth0
Pod 2
172.17.3.7
Microservice 9002
Veth: eth0
Pod 3
172.17.3.8
Microservice 9003
Veth: eth0
Pod 4
172.17.3.9
Microservice 9004
Veth: eth0
IP tables rules
eth0
10.130.1.102/24
Node 2
L2 Bridge 172.17.3.1/16
Veth0 Veth1 Veth2 Veth3
73. @arafkarsh arafkarsh
Kubernetes Networking
3 Networks
73
Source: https://github.com/meta-magic/kubernetes_workshop
eth0 10.130.1.102/24
Node 1
veth0
eth0
Pod 1
Container 1
172.17.4.1
eth0
Pod 2
Container 1
172.17.4.2
veth1
eth0
10.130.1.103/24
Node 2
veth1
eth0
Pod 1
Container 1
172.17.5.1
eth0
10.130.1.104/24
Node 3
veth1
eth0
Pod 1
Container 1
172.17.6.1
Service
EP EP EP
VIP
192.168.1.2/16
1. Physical Network
2. Pod Network
3. Service Network
End Points
handles
dynamic IP
Addresses of
the Pods
selected by a
Service based
on Pod Labels
Virtual IP doesn’t have any
physical network card or
system attached.
Virtual Network - L2 / L3 /Overlay / Cloud
74. @arafkarsh arafkarsh
Kubernetes Workload Portability
Goals
1. Abstract away Infrastructure
Details
2. Decouple the App Deployment
from Infrastructure (On-Premise
or Cloud)
To help Developers
1. Write Once, Run Anywhere
(Workload Portability)
2. Avoid Vendor Lock-In
Cloud
On-Premise
74
75. @arafkarsh arafkarsh
Shopping Portal
75
/ui
/productms
/auth
/order
Gateway
Virtual Service
Deployment / Replica / Pod Nodes
Istio Sidecar - Envoy
Load Balancer
Kubernetes
Objects
Istio Objects
Firewall
P M C
Istio Control Plane
UI Pod N5
v2
Canary
v2
User X = Canary
Others = Stable
A / B Testing using
Canary Deployment
v1 UI Pod
UI Pod
UI Pod
UI
Service
N1
N2
N2
Destination
Rule
Stable / v1
EndPoints
Internal
Load Balancers
Source:
https://github.com/meta-magic/kubernetes_workshop
Users
Product Pod
Product Pod
Product Pod
Product
Service
MySQL
Pod
N4
N3
Destination
Rule
EndPoints
Review Pod
Review Pod
Review Pod
Review
Service
N1
N4
N3
Service Call
Kube DNS
EndPoints
76. @arafkarsh arafkarsh
Shopping Portal
76
/ui
/productms
/auth
/order
Gateway
Virtual Service
Deployment / Replica / Pod Nodes
Istio Sidecar - Envoy
Load Balancer
Kubernetes
Objects
Istio Objects
Firewall
P M C
Istio Control Plane
UI Pod N5
v2
Canary
v2
v1 UI Pod
UI Pod
UI Pod
UI
Service
N1
N2
N2
Destination
Rule
Stable / v1
EndPoints
Internal
Load Balancers
Source:
https://github.com/meta-magic/kubernetes_workshop
Users
Product Pod
Product Pod
Product Pod
Product
Service
MySQL
Pod
N4
N3
Destination
Rule
EndPoints
Review Pod
Review Pod
Review Pod
Review
Service
N1
N4
N3
Service Call
Kube DNS
EndPoints
Traffic Shifting
Canary Deployment
10% = Canary
90% = Stable
77. @arafkarsh arafkarsh
Shopping Portal
77
/ui
/productms
/auth
/order
Gateway
Virtual Service
Deployment / Replica / Pod Nodes
Load Balancer
Kubernetes
Objects
Firewall
P M C
Istio Control Plane
UI Pod N5
v2
Canary
v2
v1 UI Pod
UI Pod
UI Pod
UI
Service
N1
N2
N2
Destination
Rule
Stable / v1
EndPoints
Internal
Load Balancers
Source:
https://github.com/meta-magic/kubernetes_workshop
Users
Product Pod
Product Pod
Product Pod
Product
Service
MySQL
Pod
N4
N3
Destination
Rule
EndPoints
Review Pod
Review Pod
Review Pod
Review
Service
N1
N4
N3
Service Call
Kube DNS
EndPoints
Blue Green Deployment
100% = Stable
When you want to shift to v2
Change 100% to Canary
Istio Sidecar - Envoy
Istio Objects
78. @arafkarsh arafkarsh
Shopping Portal
78
/UI
/productms
/auth
/order
Gateway
Virtual Service
Deployment / Replica / Pod Nodes
External
Load Balancer
Kubernetes
Objects
Firewall
P M C
Istio Control Plane
UI Pod N5
v2
Canary
v2
v1 UI Pod
UI Pod
UI Pod
UI
Service
N1
N2
N2
Destination
Rule
Stable / v1
EndPoints
Internal
Load Balancers
Source:
https://github.com/meta-magic/kubernetes_workshop
Users
Product Pod
Product Pod
Product Pod
Product
Service
MySQL
Pod
N4
N3
Destination
Rule
EndPoints
Review Pod
Review Pod
Review Pod
Review
Service
N1
N4
N3
Service Call
Kube DNS
EndPoints
Mirror Deployment
100% = Stable
Mirror = Canary
Production Data is mirrored to
new release for real-time testing
Istio Sidecar - Envoy
Istio Objects
79. @arafkarsh arafkarsh 79
Shopping Portal
/ui
/productms
/auth
/order
Gateway
Virtual Service
Deployment / Replica / Pod Nodes
Load Balancer
Firewall
P M C
Istio Control Plane
Fault Injection
MySQL
Pod
N4
N3
Destination
Rule
Product Pod
Product Pod
Product Pod
Product
Service
Service Call
Kube DNS
EndPoints
Internal
Load Balancers
79
Source:
https://github.com/meta-magic/kubernetes_workshop
Fault Injection
Delay = 2 Sec
Abort = 10%
Kubernetes
Objects
Users
Review Pod
Review Pod
Review Pod
Review
Service
N1
N4
N3
EndPoints
UI Pod
UI Pod
UI Pod
UI
Service
N1
N2
N2
Destination
Rule
v1
EndPoints
Istio Sidecar - Envoy
Istio Objects
80. @arafkarsh arafkarsh
K8s Network Policies L3/L4
80
Kubernetes blocks the
Product UI to access
Database or Product
Review directly.
You can create
Network policies
across name spaces,
services etc., for both
incoming (Ingress) and
outgoing (Egress)
traffic.
Product UI Pod
Product UI Pod
Product UI Pod
Product Pod
Product Pod
Product Pod
Review Pod
Review Pod
Review Pod
MySQL
Pod
Mongo
Pod
Order UI Pod
Order UI Pod
Order UI Pod
Order Pod
Order Pod
Order Pod
Oracle
Pod
Blocks Access
Blocks Access
81. @arafkarsh arafkarsh
Network Security Policy for Microservices
81
Product Review
Microservice
Product
Microservice
172.27.1.2
L3 / L4
L7 – API
GET /live
GET /ready
GET /reviews/{id}
POST /reviews
PUT /reviews/{id}
DELETE /reviews/{id}
GET /reviews/192351
Product review can be accessed ONLY by
Product. IP Tables enforces this rule.
Exposed
Exposed
Exposed
Exposed
Exposed
All other method calls are also
exposed to Product Microservice.
iptables –s 172.27.1.2
-p tcp –dport 80
-j accept
82. @arafkarsh arafkarsh
Network Security Policy for Microservices
82
Product Review
Microservice
Product
Microservice
L3 / L4
L7 – API
GET /live
GET /ready
GET /reviews/{id}
POST /reviews
PUT /reviews/{id}
DELETE /reviews/{id}
GET /reviews/192351
Rules are implemented by BPF (Berkeley
Packet Filter) at Linux Kernel level.
From Product Microservice
only GET /reviews/{id}
allowed.
BPF / XDP performance is much
superior to IPVS.
Except GET /reviews All other
calls are blocked for Product
Microservice
83. @arafkarsh arafkarsh
BPF / XDP (eXpress Data Path)
83
Network Driver Software Stack
Network Card
BPF
Regular BPF (Berkeley Packet Filter) mode
Network Driver Software Stack
Network Card
BPF
XDP allows BPF program to run inside the network driver with access to DMA buffer.
Berkeley Packet Filters (BPF) provide a powerful tool for intrusion detection analysis.
Use BPF filtering to quickly reduce large packet captures to a reduced set of results
by filtering based on a specific type of traffic.
Source: https://www.ibm.com/support/knowledgecenter/en/SS42VS_7.3.2/com.ibm.qradar.doc/c_forensics_bpf.html
84. @arafkarsh arafkarsh
XDP (eXpress Data Path)
84
BPF Program can
drop millions packet
per second when
there is DDoS attack.
Network Driver Software Stack
Network Card
BPF
Drop
Stack
Network Driver Software Stack
Network Card
BPF
Drop
Stack
LB & Tx
BPF can perform
Load Balancing and
transmit out the
data to wire again.
Source: http://www.brendangregg.com/ebpf.html
85. @arafkarsh arafkarsh
Kubernetes Container Network Interface
85
Container Runtime
Container Network Interface
Weave Calico Romana Cilium Flannel
Layer 3
BGP
BGP Route Reflector
Network Policies
IP Tables
Stores data in Etcd
Project Calico
Layer 3
VXLAN (No Encryption)
IPSec
Overlay Network
Host-GW (L2)
Stores data in Etcd
https://coreos.com/
Layer 3
IPSec
Network Policies
Multi Cloud NW
Stores data in Etcd
https://www.weave.works/
Layer 3
L3 + BGP & L2 +VXLAN
IPSec
Network Policies
IP Tables
Stores data in Etcd
https://romana.io/
Layer 3 / 7
BPF / XDP
L7 Filtering using BPF
Network Policies
L2 VXLAN
API Aware (HTTP, gRPC,
Kafka, Cassandra… )
Multi Cluster Support
https://cilium.io/
BPF (Berkeley Packet Filter) – Runs inside the Linux Kernel
On-Premise Ingress Load Balancer
Mostly Mostly Yes Yes Yes
86. @arafkarsh arafkarsh
Container & Kubernetes Summary
86
1. Containers are NOT Virtual Machines
2. Containers are isolated area in the OS kernel
3. Kubernetes – is a Container Orchestration
Platform.
4. Kubernetes abstracts the cloud vendor (AWS,
Azure, GCP) scalability features.
5. Kubernetes Concepts – Declarative Model,
Desired State and Current State.
87. @arafkarsh arafkarsh
Discovering Microservices Principles….
Components
via
Services
Organized around
Business
Capabilities
Products
NOT
Projects
Smart
Endpoints
& Dumb Pipes
Decentralized
Governance &
Data Management
Infrastructure
Automation
Design for
Failure
Evolutionary
Design
What did we Learn Today?
87
88. @arafkarsh arafkarsh
Discovering Microservices Principles….
Components
via
Services
Organized around
Business
Capabilities
Products
NOT
Projects
Smart
Endpoints
& Dumb Pipes
Decentralized
Governance &
Data Management
Infrastructure
Automation
Design for
Failure
Evolutionary
Design
What did we Learn Today?
88
89. @arafkarsh arafkarsh
Infrastructure Design Patterns
• API Gateway
• Load balancer
• Service discovery
• Circuit breaker
• Service Aggregator
• Let-it crash pattern
89
4
90. @arafkarsh arafkarsh
API Gateway Design Pattern – Software Stack
UI
Layer
WS
BL
DL
Database
Shopping Cart
Order
Customer
Product
Firewall
Users
API Gateway
Load
Balancer
Circuit
Breaker
UI
Layer
Web
Services
Business
Logic
Database
Layer
Product
SE
MySQL
DB
Product
Microservice
With 4 node
cluster
Load
Balancer
Circuit
Breaker
UI
Layer
Web
Services
Business
Logic
Database
Layer
Customer
Redis
DB
Customer
Microservice
With 2 node
cluster
Users
Access the
Monolithic
App
Directly
API Gateway (Reverse Proxy Server) routes the traffic
to appropriate Microservices (Load Balancers)
90
91. @arafkarsh arafkarsh
API Gateway – Kubernetes Implementation
/customer
/product
/cart
/order
API Gateway
Ingress
Deployment / Replica / Pod Nodes
Kubernetes Objects
Firewall
Customer Pod
Customer Pod
Customer Pod
Customer
Service
N1
N2
N2
EndPoints
Product Pod
Product Pod
Product Pod
Product
Service
N4
N3
MySQL
DB
EndPoints
Review Pod
Review Pod
Review Pod
Review
Service
N4
N3
N1
Service Call
Kube DNS
EndPoints
Internal
Load Balancers
Users
Routing based on Layer 3,4 and 7
Redis
DB
Mongo
DB
Load Balancer
91
92. @arafkarsh arafkarsh 92
API Gateway – Kubernetes / Istio
/customer
/product
/auth
/order
API Gateway
Virtual Service
Deployment / Replica / Pod Nodes
Load Balancer
Firewall
P M C
Istio Control Plane
MySQL
Pod
N4
N3
Destination
Rule
Product Pod
Product Pod
Product Pod
Product
Service
Service Call
Kube DNS
EndPoints
Internal
Load Balancers
92
Kubernetes
Objects
Users
Review Pod
Review Pod
Review Pod
Review
Service
N1
N4
N3
EndPoints
Customer Pod
Customer Pod
Customer Pod
Customer
Service
N1
N2
N2
Destination
Rule
EndPoints
Redis
DB
Mongo
DB
Istio Sidecar - Envoy
Istio Objects
93. @arafkarsh arafkarsh
Load Balancer Design Pattern
Firewall
Users
API Gateway
Load
Balancer
Circuit
Breaker
UI
Layer
Web
Services
Business
Logic
Database
Layer
Product
SE
MySQL
DB
Product
Microservice
With 4 node
cluster
Load
Balancer
CB
=
Hystrix
UI
Layer
Web
Services
Business
Logic
Database
Layer
Customer
Redis
DB
Customer
Microservice
With 2 node
cluster
API Gateway (Reverse Proxy Server) routes
the traffic to appropriate Microservices
(Load Balancers)
Load Balancer Rules
1. Round Robin
2. Based on
Availability
3. Based on
Response Time
93
94. @arafkarsh arafkarsh
Ingress
Load Balancer – Kubernetes Model
Kubernetes
Objects
Firewall
Users
Product 1
Product 2
Product 3
Product
Service
N4
N3
N1
EndPoints
Internal
Load Balancers
DB
Load Balancer
API Gateway
N1
N2
N2
Customer 1
Customer 2
Customer 3
Customer
Service
EndPoints
DB
Internal
Load Balancers
Pods Nodes
• Load Balancer receives the (request) packet from the User and it picks up
a Virtual Machine in the Cluster to do the internal Load Balancing.
• Kube Proxy using IP Tables redirect the Packet using internal load
Balancing rules.
• Packet enters Kubernetes Cluster and reaches Node (of that specific Pod)
and Node handover the packet to the Pod.
/customer
/product
/cart
94
95. @arafkarsh arafkarsh
Service Discovery – NetFlix Network Stack Model
Firewall
Users
API Gateway
Load
Balancer
Circuit
Breaker
Product
MySQL
DB
Product
Microservice
With 4 node
cluster
Load
Balancer
Circuit
Breaker
UI
Layer
Web
Services
Business
Logic
Database
Layer
Customer
Redis
DB
Customer
Microservice
With 2 node
cluster
• In this model Developers write the
code in every Microservice to register
with NetFlix Eureka Service Discovery
Server.
• Load Balancers and API Gateway also
registers with Service Discovery.
• Service Discovery will inform the Load
Balancers about the instance details
(IP Addresses).
Service Discovery
95
96. @arafkarsh arafkarsh
Ingress
Service Discovery – Kubernetes Model
Kubernetes
Objects
Firewall
Users
Product 1
Product 2
Product 3
Product
Service
N4
N3
N1
EndPoints
Internal
Load Balancers
DB
API Gateway
N1
N2
N2
Customer 1
Customer 2
Customer 3
Customer
Service
EndPoints
DB
Internal
Load Balancers
Pods Nodes
• API Gateway (Reverse Proxy Server) doesn't know the instances (IP
Addresses) of News Pod. It knows the IP address of the Services
defined for each Microservice (Customer / Product etc.)
• Services handles the dynamic IP Addresses of the pods. Services
Endpoints will automatically discover the new Pods based on Labels.
Service Definition
from Kubernetes
Perspective
/customer
/product
/cart
Service Call
Kube DNS
96
97. @arafkarsh arafkarsh
Circuit Breaker Pattern
97
/ui
/productms
If Product Review is not
available Product service
will return the product
details with a message
review not available.
Reverse Proxy Server
Ingress
Deployment / Replica / Pod Nodes
Kubernetes Objects
Firewall
UI Pod
UI Pod
UI Pod
UI Service
N1
N2
N2
EndPoints
Product Pod
Product Pod
Product Pod
Product
Service
N4
N3
MySQL
Pod
EndPoints
Internal
Load Balancers
Users
Routing based on Layer 3,4 and 7
Review Pod
Review Pod
Review Pod
Review
Service
N4
N3
N1
Service Call
Kube DNS
EndPoints
98. @arafkarsh arafkarsh
Service Aggregator Pattern
/newservice
Reverse Proxy Server
Ingress
Deployment / Replica / Pod Nodes
Kubernetes
Objects
Firewall
Service Call
Kube DNS
Users
Internal
Load Balancers
EndPoints News Pod
News Pod
News Pod
News
Service
N4
N3
N2
News Service Portal
• News Category wise
Microservices
• Aggregator Microservice to
aggregate all category of news.
Auto Scaling
• Sports Events (IPL / NBA) spikes
the traffic for Sports Microservice.
• Auto scaling happens for both
News and Sports Microservices.
N1
N2
N2
National
National
National
National
Service
EndPoints
Internal
Load Balancers
DB
N1
N2
N2
Politics
Politics
Politics
Politics
Service
EndPoints
DB
Sports
Sports
Sports
Sports
Service
N4
N3
N1
EndPoints
Internal
Load Balancers
DB
98
100. @arafkarsh arafkarsh
Service Aggregator Pattern
100
/artist
Reverse Proxy Server
Ingress
Deployment / Replica / Pod Nodes
Kubernetes
Objects
Firewall
Service Call
Kube DNS
Users
Internal
Load Balancers
EndPoints Artist Pod
Artist Pod
Artist Pod
Artist
Service
N4
N3
N2
Spotify Microservices
• Artist Microservice combines all
the details from Discography,
Play count and Playlists.
Auto Scaling
• Scaling of Artist and downstream
Microservices will automatically
scale depends on the load factor.
N1
N2
N2
Discography
Discography
Discography
Discography
Service
EndPoints
Internal
Load Balancers
DB
N1
N2
N2
Play Count
Play Count
Play Count
Play Count
Service
EndPoints
DB
Playlist
Playlist
Playlist
Playlist
Service
N4
N3
N1
EndPoints
Internal
Load Balancers
DB
101. @arafkarsh arafkarsh
Config Store – Spring Config Server
Firewall
Users
API Gateway
Load
Balancer
Circuit
Breaker
Product
MySQL
DB
Product
Microservice
With 4 node
cluster
Load
Balancer
Circuit
Breaker
UI
Layer
Web
Services
Business
Logic
Database
Layer
Customer
Redis
DB
Customer
Microservice
With 2 node
cluster
• In this model Developers write the
code in every Microservice to
download the required configuration
from a Central server (Ex. Spring
Config Server for the Java World).
• This creates an explicit dependency
order in which service to come up will
be critical.
Config Server
101
102. @arafkarsh arafkarsh
Service Mesh – Sidecar Design Pattern
102
CB – Circuit Breaker
LB – Load Balancer
SD – Service Discovery
Microservice
Process
1
Process
2
Service Mesh Control Plane
Service
Discovery
Routing
Rules
Control Plane will have all the rules for Routing and
Service Discovery. Local Service Mesh will download the
rules from the Control pane will have a local copy.
Service Discovery Calls
Service
Mesh
Calls
Customer Microservice
Application Localhost calls
http://localhost/api/order/
Router
Network Stack
LB
CB SD
Service
Mesh
Sidecar
UI Layer
Web Services
Business Logic
Order Microservice
Application Localhost calls
http://localhost/api/payment/
Router
Network Stack
LB
CB SD
Service
Mesh
Sidecar
UI Layer
Web Services
Business Logic
Data Plane
103. @arafkarsh arafkarsh
Shopping Portal
103
/ui
/productms
/auth
/order
Gateway
Virtual Service
Deployment / Replica / Pod Nodes
Load Balancer
Kubernetes
Objects
Firewall
P M C
Istio Control Plane
UI Pod N5
v2
Canary
v2
User X = Canary
Others = Stable
A / B Testing using
Canary Deployment
v1 UI Pod
UI Pod
UI Pod
UI
Service
N1
N2
N2
Destination
Rule
Stable / v1
EndPoints
Internal
Load Balancers
Source:
https://github.com/meta-magic/kubernetes_workshop
Users
Product Pod
Product Pod
Product Pod
Product
Service
MySQL
Pod
N4
N3
Destination
Rule
EndPoints
Review Pod
Review Pod
Review Pod
Review
Service
N1
N4
N3
Service Call
Kube DNS
EndPoints
Istio Sidecar - Envoy
Istio Objects
104. @arafkarsh arafkarsh
Let-it-Crash Design Pattern – Erlang Philosophy
104
• The Erlang view of the world is that everything is a process and that processes can
interact only by exchanging messages.
• A typical Erlang program might have hundreds, thousands, or even millions of processes.
• Letting processes crash is central to Erlang. It’s the equivalent of unplugging your router
and plugging it back in – as long as you can get back to a known state, this turns out to be
a very good strategy.
• To make that happen, you build supervision trees.
• A supervisor will decide how to deal with a crashed process. It will restart the process, or
possibly kill some other processes, or crash and let someone else deal with it.
• Two models of concurrency: Shared State Concurrency, & Message Passing Concurrency.
The programming world went one way (toward shared state). The Erlang community
went the other way.
• All languages such as C, Java, C++, and so on, have the notion that there is this stuff called
state and that we can change it. The moment you share something you need to bring
Mutex a Locking Mechanism.
• Erlang has no mutable data structures (that’s not quite true, but it’s true enough). No
mutable data structures = No locks. No mutable data structures = Easy to parallelize.
105. @arafkarsh arafkarsh
Let-it-Crash Design Pattern
105
1. The idea of Messages as the first-class citizens of a system, has been
rediscovered by the Event Sourcing / CQRS community, along with a strong
focus on domain models.
2. Event Sourced Aggregates are a way to Model the Processes and NOT things.
3. Each component MUST tolerate a crash and restart at any point in time.
4. All interaction between the components must tolerate that peers can crash.
This means ubiquitous use of timeouts and Circuit Breaker.
5. Each component must be strongly encapsulated so that failures are fully
contained and cannot spread.
6. All requests sent to a component MUST be self describing as is practical so
that processing can resume with a little recovery cost as possible after a
restart.
106. @arafkarsh arafkarsh
Let-it-Crash : Comparison Erlang Vs. Microservices Vs. Monolithic Apps
106
Erlang Philosophy Micro Services Architecture Monolithic Apps (Java, C++, C#, Node JS ...)
1 Perspective
Everything is a
Process
Event Sourced Aggregates are a way to
model the Process and NOT things.
Things (defined as Objects) and
Behaviors
2
Crash
Recovery
Supervisor will
decide how to
handle the
crashed process
Kubernetes Manager monitors all the
Pods (Microservices) and its Readiness
and Health. K8s terminates the Pod if
the health is bad and spawns a new
Pod. Circuit Breaker Pattern is used
handle the fallback mechanism.
Not available. Most of the monolithic
Apps are Stateful and Crash Recovery
needs to be handled manually and all
languages other than Erlang focuses
on defensive programming.
3 Concurrency
Message Passing
Concurrency
Domain Events for state changes within
a Bounded Context & Integration Events
for external Systems.
Mostly Shared State Concurrency
4 State
Stateless :
Mostly Immutable
Structures
Immutability is handled thru Event
Sourcing along with Domain Events and
Integration Events.
Predominantly Stateful with Mutable
structures and Mutex as a Locking
Mechanism
5 Citizen Messages
Messages are 1st class citizen by Event
Sourcing / CQRS pattern with a strong
focus on Domain Models
Mutable Objects and Strong focus on
Domain Models and synchronous
communication.
107. @arafkarsh arafkarsh
Infrastructure Design Patterns Summary
107
1. API Gateway
2. Service Discovery
3. Load Balancer
4. Circuit Breaker
5. Side Car Pattern
6. Service Aggregator Pattern
7. Let It Crash Pattern
108. @arafkarsh arafkarsh
Discovering Microservices Principles….
Components
via
Services
Organized around
Business
Capabilities
Products
NOT
Projects
Smart
Endpoints
& Dumb Pipes
Decentralized
Governance &
Data Management
Infrastructure
Automation
Design for
Failure
Evolutionary
Design
What did we Learn Today?
108
109. @arafkarsh arafkarsh
Discovering Microservices Principles….
Components
via
Services
Organized around
Business
Capabilities
Products
NOT
Projects
Smart
Endpoints
& Dumb Pipes
Decentralized
Governance &
Data Management
Infrastructure
Automation
Design for
Failure
Evolutionary
Design
What did we Learn Today?
109
110. @arafkarsh arafkarsh 110
100s Microservices
1,000s Releases / Day
10,000s Virtual Machines
100K+ User actions / Second
81 M Customers Globally
1 B Time series Metrics
10 B Hours of video streaming
every quarter
Source: NetFlix: : https://www.youtube.com/watch?v=UTKIT6STSVM
10s OPs Engineers
0 NOC
0 Data Centers
So what do NetFlix think about DevOps?
No DevOps
Don’t do lot of Process / Procedures
Freedom for Developers & be Accountable
Trust people you Hire
No Controls / Silos / Walls / Fences
Ownership – You Build it, You Run it.
111. @arafkarsh arafkarsh 111
Design Patterns are
solutions to general
problems that
software developers
faced during software
development.
Design Patterns
115. @arafkarsh arafkarsh
References
1. July 15, 2015 – Agile is Dead : GoTo 2015 By Dave Thomas
2. Apr 7, 2016 - Agile Project Management with Kanban | Eric Brechner | Talks at Google
3. Sep 27, 2017 - Scrum vs Kanban - Two Agile Teams Go Head-to-Head
4. Feb 17, 2019 - Lean vs Agile vs Design Thinking
5. Dec 17, 2020 - Scrum vs Kanban | Differences & Similarities Between Scrum & Kanban
6. Feb 24, 2021 - Agile Methodology Tutorial for Beginners | Jira Tutorial | Agile Methodology Explained.
Agile Methodologies
115
116. @arafkarsh arafkarsh
References
1. Vmware: What is Cloud Architecture?
2. Redhat: What is Cloud Architecture?
3. Cloud Computing Architecture
4. Cloud Adoption Essentials:
5. Google: Hybrid and Multi Cloud
6. IBM: Hybrid Cloud Architecture Intro
7. IBM: Hybrid Cloud Architecture: Part 1
8. IBM: Hybrid Cloud Architecture: Part 2
9. Cloud Computing Basics: IaaS, PaaS, SaaS
116
1. IBM: IaaS Explained
2. IBM: PaaS Explained
3. IBM: SaaS Explained
4. IBM: FaaS Explained
5. IBM: What is Hypervisor?
Cloud Architecture
117. @arafkarsh arafkarsh
References
Microservices
1. Microservices Definition by Martin Fowler
2. When to use Microservices By Martin Fowler
3. GoTo: Sep 3, 2020: When to use Microservices By Martin Fowler
4. GoTo: Feb 26, 2020: Monolith Decomposition Pattern
5. Thought Works: Microservices in a Nutshell
6. Microservices Prerequisites
7. What do you mean by Event Driven?
8. Understanding Event Driven Design Patterns for Microservices
117
118. @arafkarsh arafkarsh
References – Microservices – Videos
118
1. Martin Fowler – Micro Services : https://www.youtube.com/watch?v=2yko4TbC8cI&feature=youtu.be&t=15m53s
2. GOTO 2016 – Microservices at NetFlix Scale: Principles, Tradeoffs & Lessons Learned. By R Meshenberg
3. Mastering Chaos – A NetFlix Guide to Microservices. By Josh Evans
4. GOTO 2015 – Challenges Implementing Micro Services By Fred George
5. GOTO 2016 – From Monolith to Microservices at Zalando. By Rodrigue Scaefer
6. GOTO 2015 – Microservices @ Spotify. By Kevin Goldsmith
7. Modelling Microservices @ Spotify : https://www.youtube.com/watch?v=7XDA044tl8k
8. GOTO 2015 – DDD & Microservices: At last, Some Boundaries By Eric Evans
9. GOTO 2016 – What I wish I had known before Scaling Uber to 1000 Services. By Matt Ranney
10. DDD Europe – Tackling Complexity in the Heart of Software By Eric Evans, April 11, 2016
11. AWS re:Invent 2016 – From Monolithic to Microservices: Evolving Architecture Patterns. By Emerson L, Gilt D. Chiles
12. AWS 2017 – An overview of designing Microservices based Applications on AWS. By Peter Dalbhanjan
13. GOTO Jun, 2017 – Effective Microservices in a Data Centric World. By Randy Shoup.
14. GOTO July, 2017 – The Seven (more) Deadly Sins of Microservices. By Daniel Bryant
15. Sept, 2017 – Airbnb, From Monolith to Microservices: How to scale your Architecture. By Melanie Cubula
16. GOTO Sept, 2017 – Rethinking Microservices with Stateful Streams. By Ben Stopford.
17. GOTO 2017 – Microservices without Servers. By Glynn Bird.
119. @arafkarsh arafkarsh
References
119
Domain Driven Design
1. Oct 27, 2012 What I have learned about DDD Since the book. By Eric Evans
2. Mar 19, 2013 Domain Driven Design By Eric Evans
3. Jun 02, 2015 Applied DDD in Java EE 7 and Open Source World
4. Aug 23, 2016 Domain Driven Design the Good Parts By Jimmy Bogard
5. Sep 22, 2016 GOTO 2015 – DDD & REST Domain Driven API’s for the Web. By Oliver Gierke
6. Jan 24, 2017 Spring Developer – Developing Micro Services with Aggregates. By Chris Richardson
7. May 17. 2017 DEVOXX – The Art of Discovering Bounded Contexts. By Nick Tune
8. Dec 21, 2019 What is DDD - Eric Evans - DDD Europe 2019. By Eric Evans
9. Oct 2, 2020 - Bounded Contexts - Eric Evans - DDD Europe 2020. By. Eric Evans
10. Oct 2, 2020 - DDD By Example - Paul Rayner - DDD Europe 2020. By Paul Rayner
120. @arafkarsh arafkarsh
References
Event Sourcing and CQRS
1. IBM: Event Driven Architecture – Mar 21, 2021
2. Martin Fowler: Event Driven Architecture – GOTO 2017
3. Greg Young: A Decade of DDD, Event Sourcing & CQRS – April 11, 2016
4. Nov 13, 2014 GOTO 2014 – Event Sourcing. By Greg Young
5. Mar 22, 2016 Building Micro Services with Event Sourcing and CQRS
6. Apr 15, 2016 YOW! Nights – Event Sourcing. By Martin Fowler
7. May 08, 2017 When Micro Services Meet Event Sourcing. By Vinicius Gomes
120
121. @arafkarsh arafkarsh
References
121
Kafka
1. Understanding Kafka
2. Understanding RabbitMQ
3. IBM: Apache Kafka – Sept 18, 2020
4. Confluent: Apache Kafka Fundamentals – April 25, 2020
5. Confluent: How Kafka Works – Aug 25, 2020
6. Confluent: How to integrate Kafka into your environment – Aug 25, 2020
7. Kafka Streams – Sept 4, 2021
8. Kafka: Processing Streaming Data with KSQL – Jul 16, 2018
9. Kafka: Processing Streaming Data with KSQL – Nov 28, 2019
122. @arafkarsh arafkarsh
References
Databases: Big Data / Cloud Databases
1. Google: How to Choose the right database?
2. AWS: Choosing the right Database
3. IBM: NoSQL Vs. SQL
4. A Guide to NoSQL Databases
5. How does NoSQL Databases Work?
6. What is Better? SQL or NoSQL?
7. What is DBaaS?
8. NoSQL Concepts
9. Key Value Databases
10. Document Databases
11. Jun 29, 2012 – Google I/O 2012 - SQL vs NoSQL: Battle of the Backends
12. Feb 19, 2013 - Introduction to NoSQL • Martin Fowler • GOTO 2012
13. Jul 25, 2018 - SQL vs NoSQL or MySQL vs MongoDB
14. Oct 30, 2020 - Column vs Row Oriented Databases Explained
15. Dec 9, 2020 - How do NoSQL databases work? Simply Explained!
1. Graph Databases
2. Column Databases
3. Row Vs. Column Oriented Databases
4. Database Indexing Explained
5. MongoDB Indexing
6. AWS: DynamoDB Global Indexing
7. AWS: DynamoDB Local Indexing
8. Google Cloud Spanner
9. AWS: DynamoDB Design Patterns
10. Cloud Provider Database Comparisons
11. CockroachDB: When to use a Cloud DB?
122
123. @arafkarsh arafkarsh
References
Docker / Kubernetes / Istio
1. IBM: Virtual Machines and Containers
2. IBM: What is a Hypervisor?
3. IBM: Docker Vs. Kubernetes
4. IBM: Containerization Explained
5. IBM: Kubernetes Explained
6. IBM: Kubernetes Ingress in 5 Minutes
7. Microsoft: How Service Mesh works in Kubernetes
8. IBM: Istio Service Mesh Explained
9. IBM: Kubernetes and OpenShift
10. IBM: Kubernetes Operators
11. 10 Consideration for Kubernetes Deployments
Istio – Metrics
1. Istio – Metrics
2. Monitoring Istio Mesh with Grafana
3. Visualize your Istio Service Mesh
4. Security and Monitoring with Istio
5. Observing Services using Prometheus, Grafana, Kiali
6. Istio Cookbook: Kiali Recipe
7. Kubernetes: Open Telemetry
8. Open Telemetry
9. How Prometheus works
10. IBM: Observability vs. Monitoring
123
124. @arafkarsh arafkarsh
References
124
1. Feb 6, 2020 – An introduction to TDD
2. Aug 14, 2019 – Component Software Testing
3. May 30, 2020 – What is Component Testing?
4. Apr 23, 2013 – Component Test By Martin Fowler
5. Jan 12, 2011 – Contract Testing By Martin Fowler
6. Jan 16, 2018 – Integration Testing By Martin Fowler
7. Testing Strategies in Microservices Architecture
8. Practical Test Pyramid By Ham Vocke
Testing – TDD / BDD
125. @arafkarsh arafkarsh 125
1. Simoorg : LinkedIn’s own failure inducer framework. It was designed to be easy to extend and
most of the important components are plug‐ gable.
2. Pumba : A chaos testing and network emulation tool for Docker.
3. Chaos Lemur : Self-hostable application to randomly destroy virtual machines in a BOSH-
managed environment, as an aid to resilience testing of high-availability systems.
4. Chaos Lambda : Randomly terminate AWS ASG instances during business hours.
5. Blockade : Docker-based utility for testing network failures and partitions in distributed
applications.
6. Chaos-http-proxy : Introduces failures into HTTP requests via a proxy server.
7. Monkey-ops : Monkey-Ops is a simple service implemented in Go, which is deployed into an
OpenShift V3.X and generates some chaos within it. Monkey-Ops seeks some OpenShift
components like Pods or Deployment Configs and randomly terminates them.
8. Chaos Dingo : Chaos Dingo currently supports performing operations on Azure VMs and VMSS
deployed to an Azure Resource Manager-based resource group.
9. Tugbot : Testing in Production (TiP) framework for Docker.
Testing tools
126. @arafkarsh arafkarsh
References
CI / CD
1. What is Continuous Integration?
2. What is Continuous Delivery?
3. CI / CD Pipeline
4. What is CI / CD Pipeline?
5. CI / CD Explained
6. CI / CD Pipeline using Java Example Part 1
7. CI / CD Pipeline using Ansible Part 2
8. Declarative Pipeline vs Scripted Pipeline
9. Complete Jenkins Pipeline Tutorial
10. Common Pipeline Mistakes
11. CI / CD for a Docker Application
126
127. @arafkarsh arafkarsh
References
127
DevOps
1. IBM: What is DevOps?
2. IBM: Cloud Native DevOps Explained
3. IBM: Application Transformation
4. IBM: Virtualization Explained
5. What is DevOps? Easy Way
6. DevOps?! How to become a DevOps Engineer???
7. Amazon: https://www.youtube.com/watch?v=mBU3AJ3j1rg
8. NetFlix: https://www.youtube.com/watch?v=UTKIT6STSVM
9. DevOps and SRE: https://www.youtube.com/watch?v=uTEL8Ff1Zvk
10. SLI, SLO, SLA : https://www.youtube.com/watch?v=tEylFyxbDLE
11. DevOps and SRE : Risks and Budgets : https://www.youtube.com/watch?v=y2ILKr8kCJU
12. SRE @ Google: https://www.youtube.com/watch?v=d2wn_E1jxn4
128. @arafkarsh arafkarsh
References
128
1. Lewis, James, and Martin Fowler. “Microservices: A Definition of This New Architectural Term”, March 25, 2014.
2. Miller, Matt. “Innovate or Die: The Rise of Microservices”. e Wall Street Journal, October 5, 2015.
3. Newman, Sam. Building Microservices. O’Reilly Media, 2015.
4. Alagarasan, Vijay. “Seven Microservices Anti-patterns”, August 24, 2015.
5. Cockcroft, Adrian. “State of the Art in Microservices”, December 4, 2014.
6. Fowler, Martin. “Microservice Prerequisites”, August 28, 2014.
7. Fowler, Martin. “Microservice Tradeoffs”, July 1, 2015.
8. Humble, Jez. “Four Principles of Low-Risk Software Release”, February 16, 2012.
9. Zuul Edge Server, Ketan Gote, May 22, 2017
10. Ribbon, Hysterix using Spring Feign, Ketan Gote, May 22, 2017
11. Eureka Server with Spring Cloud, Ketan Gote, May 22, 2017
12. Apache Kafka, A Distributed Streaming Platform, Ketan Gote, May 20, 2017
13. Functional Reactive Programming, Araf Karsh Hamid, August 7, 2016
14. Enterprise Software Architectures, Araf Karsh Hamid, July 30, 2016
15. Docker and Linux Containers, Araf Karsh Hamid, April 28, 2015
129. @arafkarsh arafkarsh
References
129
16. MSDN – Microsoft https://msdn.microsoft.com/en-us/library/dn568103.aspx
17. Martin Fowler : CQRS – http://martinfowler.com/bliki/CQRS.html
18. Udi Dahan : CQRS – http://www.udidahan.com/2009/12/09/clarified-cqrs/
19. Greg Young : CQRS - https://www.youtube.com/watch?v=JHGkaShoyNs
20. Bertrand Meyer – CQS - http://en.wikipedia.org/wiki/Bertrand_Meyer
21. CQS : http://en.wikipedia.org/wiki/Command–query_separation
22. CAP Theorem : http://en.wikipedia.org/wiki/CAP_theorem
23. CAP Theorem : http://www.julianbrowne.com/article/viewer/brewers-cap-theorem
24. CAP 12 years how the rules have changed
25. EBay Scalability Best Practices : http://www.infoq.com/articles/ebay-scalability-best-practices
26. Pat Helland (Amazon) : Life beyond distributed transactions
27. Stanford University: Rx https://www.youtube.com/watch?v=y9xudo3C1Cw
28. Princeton University: SAGAS (1987) Hector Garcia Molina / Kenneth Salem
29. Rx Observable : https://dzone.com/articles/using-rx-java-observable
Editor's Notes
DevOpsAmazon: https://www.youtube.com/watch?v=mBU3AJ3j1rg
NetFlix: https://www.youtube.com/watch?v=UTKIT6STSVM
DevOps and SRE: https://www.youtube.com/watch?v=uTEL8Ff1Zvk
SLI, SLO, SLA : https://www.youtube.com/watch?v=tEylFyxbDLE
DevOps and SRE : Risks and Budgets : https://www.youtube.com/watch?v=y2ILKr8kCJU
Memory
You can limit the amount of RAM and swap space that can be used by a group of processes.It accounts for the memory used by the processes for their private use (their Resident Set Size, or RSS), but also for the memory used for caching purposes.
This is actually quite powerful, because traditional tools (ps, analysis of /proc, etc.) have no way to find out the cache memory usage incurred by specific processes. This can make a big difference, for instance, with databases.
A database will typically use very little memory for its processes (unless you do complex queries, but let’s pretend you don’t!), but can be a huge consumer of cache memory: after all, to perform optimally, your whole database (or at least, your “active set” of data that you refer to the most often) should fit into memory.
Limiting the memory available to the processes inside a cgroup is as easy as echo1000000000 > /cgroup/polkadot/memory.limit_in_bytes (it will be rounded to a page size).
To check the current usage for a cgroup, inspect the pseudo-filememory.usage_in_bytes in the cgroup directory. You can gather very detailed (and very useful) information into memory.stat; the data contained in this file could justify a whole blog post by itself!
CPU
You might already be familiar with scheduler priorities, and with the nice and renice commands. Once again, control groups will let you define the amount of CPU, that should be shared by a group of processes, instead of a single one. You can give each cgroup a relative number of CPU shares, and the kernel will make sure that each group of process gets access to the CPU in proportion of the number of shares you gave it.
Setting the number of shares is as simple as echo 250 > /cgroup/polkadot/cpu.shares. Remember that those shares are just relative numbers: if you multiply everyone’s share by 10, the end result will be exactly the same. This control group also gives statistics incpu.stat.
CPU sets
This is different from the cpu controller.In systems with multiple CPUs (i.e., the vast majority of servers, desktop & laptop computers, and even phones today!), the cpuset control group lets you define which processes can use which CPU.
This can be useful to reserve a full CPU to a given process or group of processes. Those processes will receive a fixed amount of CPU cycles, and they might also run faster because there will be less thrashing at the level of the CPU cache.
On systems with Non Uniform Memory Access (NUMA), the memory is split in multiple banks, and each bank is tied to a specific CPU (or set of CPUs); so binding a process (or group of processes) to a specific CPU (or a specific group of CPUs) can also reduce the overhead happening when a process is scheduled to run on a CPU, but accessing RAM tied to another CPU.
Block I/O
The blkio controller gives a lot of information about the disk accesses (technically, block devices requests) performed by a group of processes. This is very useful, because I/O resources are much harder to share than CPU or RAM.
A system has a given, known, and fixed amount of RAM. It has a fixed number of CPU cycles every second – and even on systems where the number of CPU cycles can change (tickless systems, or virtual machines), it is not an issue, because the kernel will slice the CPU time in shares of e.g. 1 millisecond, and there is a given, known, and fixed number of milliseconds every second (doh!). I/O bandwidth, however, is quite unpredictable. Or rather, as we will see, it is predictable, but the prediction isn’t very useful.
A hard disk with a 10ms average seek time will be able to do about 100 requests of 4 KB per second; but if the requests are sequential, typical desktop hard drives can easily sustain 80 MB/s transfer rates – which means 20000 requests of 4 kB per second.
The average throughput (measured in IOPS, I/O Operations Per Second) will be somewhere between those two extremes. But as soon as some application performs a task requiring a lot of scattered, random I/O operations, the performance will drop – dramatically. The system does give you some guaranteed performance, but this guaranteed performance is so low, that it doesn’t help much (that’s exactly the problem of AWS EBS, by the way). It’s like a highway with an anti-traffic jam system that would guarantee that you can always go above a given speed, except that this speed is 5 mph. Not very helpful, is it?
That’s why SSD storage is becoming increasingly popular. SSD has virtually no seek time, and can therefore sustain random I/O as fast as sequential I/O. The available throughput is therefore predictably good, under any given load.
Actually, there are some workloads that can cause problems; for instance, if you continuously write and rewrite a whole disk, you will find that the performance will drop dramatically. This is because read and write operations are fast, but erase, which must be performed at some point before write, is slow. This won’t be a problem in most situations.
An example use-case which could exhibit the issue would be to use SSD to do catch-up TV for 100 HD channels simultaneously: the disk will sustain the write throughput until it has written every block once; then it will need to erase, and performance will drop below acceptable levels.)
To get back to the topic – what’s the purpose of the blkio controller in a PaaS environment like dotCloud?
The blkio controller metrics will help detecting applications that are putting an excessive strain on the I/O subsystem. Then, the controller lets you set limits, which can be expressed in number of operations and/or bytes per second. It also allows for different limits for read and write operations. It allows to set some safeguard limits (to make sure that a single app won’t significantly degrade performance for everyone). Furthermore, once a I/O-hungry app has been identified, its quota can be adapted to reduce impact on other apps.
more
The pid namespace
This is probably the most useful for basic isolation.
Each pid namespace has its own process numbering. Different pid namespaces form a hierarchy: the kernel keeps track of which namespace created which other. A “parent” namespace can see its children namespaces, and it can affect them (for instance, with signals); but a child namespace cannot do anything to its parent namespace. As a consequence:
each pid namespace has its own “PID 1” init-like process;
processes living in a namespace cannot affect processes living in parent or sibling namespaces with system calls like kill or ptrace, since process ids are meaningful only inside a given namespace;
if a pseudo-filesystem like proc is mounted by a process within a pid namespace, it will only show the processes belonging to the namespace;
since the numbering is different in each namespace, it means that a process in a child namespace will have multiple PIDs: one in its own namespace, and a different PID in its parent namespace.
The last item means that from the top-level pid namespace, you will be able to see all processes running in all namespaces, but with different PIDs. Of course, a process can have more than 2 PIDs if there are more than two levels of hierarchy in the namespaces.
The net namespace
With the pid namespace, you can start processes in multiple isolated environments (let’s bite the bullet and call them “containers” once and for all). But if you want to run e.g. a different Apache in each container, you will have a problem: there can be only one process listening to port 80/tcp at a time. You could configure your instances of Apache to listen on different ports… or you could use the net namespace.
As its name implies, the net namespace is about networking. Each different net namespace can have different network interfaces. Even lo, the loopback interface supporting 127.0.0.1, will be different in each different net namespace.
It is possible to create pairs of special interfaces, which will appear in two different net namespaces, and allow a net namespace to talk to the outside world.
A typical container will have its own loopback interface (lo), as well as one end of such a special interface, generally named eth0. The other end of the special interface will be in the “original” namespace, and will bear a poetic name like veth42xyz0. It is then possible to put those special interfaces together within an Ethernet bridge (to achieve switching between containers), or route packets between them, etc. (If you are familiar with the Xen networking model, this is probably no news to you!)
Note that each net namespace has its own meaning for INADDR_ANY, a.k.a. 0.0.0.0; so when your Apache process binds to *:80 within its namespace, it will only receive connections directed to the IP addresses and interfaces of its namespace – thus allowing you, at the end of the day, to run multiple Apache instances, with their default configuration listening on port 80.
In case you were wondering: each net namespace has its own routing table, but also its own iptables chains and rules.
The ipc namespace
This one won’t appeal a lot to you; unless you passed your UNIX 101 a long time ago, when they still taught about IPC (InterProcess Communication)!
IPC provides semaphores, message queues, and shared memory segments.
While still supported by virtually all UNIX flavors, those features are considered by many people as obsolete, and superseded by POSIX semaphores, POSIX message queues, and mmap. Nonetheless, some programs – including PostgreSQL – still use IPC.
What’s the connection with namespaces? Well, each IPC resources are accessed through a unique 32-bits ID. IPC implement permissions on resources, but nonetheless, one application could be surprised if it failed to access a given resource because it has already been claimed by another process in a different container.
Introduce the ipc namespace: processes within a given ipc namespace cannot access (or even see at all) IPC resources living in other ipc namespaces. And now you can safely run a PostgreSQL instance in each container without fearing IPC key collisions!
The mnt namespace
You might already be familiar with chroot, a mechanism allowing to sandbox a process (and its children) within a given directory. The mnt namespace takes that concept one step further.
As its name implies, the mnt namespace deals with mountpoints.
Processes living in different mnt namespaces can see different sets of mounted filesystems – and different root directories. If a filesystem is mounted in a mnt namespace, it will be accessible only to those processes within that namespace; it will remain invisible for processes in other namespaces.
At first, it sounds useful, since it allows to sandbox each container within its own directory, hiding other containers.
At a second glance, is it really that useful? After all, if each container is chroot‘ed in a different directory, container C1 won’t be able to access or see the filesystem of container C2, right? Well, that’s right, but there are side effects.
Inspecting /proc/mounts in a container will show the mountpoints of all containers. Also, those mountpoints will be relative to the original namespace, which can give some hints about the layout of your system – and maybe confuse some applications which would rely on the paths in /proc/mounts.
The mnt namespace makes the situation much cleaner, allowing each container to have its own mountpoints, and see only those mountpoints, with their path correctly translated to the actual root of the namespace.
The uts namespace
Finally, the uts namespace deals with one little detail: the hostname that will be “seen” by a group of processes.
Each uts namespace will hold a different hostname, and changing the hostname (through the sethostname system call) will only change it for processes running in the same namespace.
Unique IP Address of the Pod: https://kubernetes.io/docs/tutorials/kubernetes-basics/expose/expose-intro/
A reverse proxy server is a type of proxy server that typically sits behind the firewall in a private network and directs client requests to the appropriate backend server. A reverse proxy provides an additional level of abstraction and control to ensure the smooth flow of network traffic between clients and servers. A reverse proxy server is a type of proxy server that typically sits behind the firewall in a private network and directs client requests to the appropriate backend server. A reverse proxy provides an additional level of abstraction and control to ensure the smooth flow of network traffic between clients and servers.