Araf Karsh Hamid, profile picture
Uploaded byAraf Karsh Hamid
PPSX, PPTX3,118 views

Microservices, Containers, Kubernetes, Kafka, Kanban

The document discusses microservices architecture, highlighting their benefits such as independent deployment, technology diversity, and strong module boundaries, while also addressing challenges like operational complexity and determining the appropriate size of a microservice. It outlines best practices for implementing microservices, including using containers and Kubernetes, and emphasizes the importance of business capabilities and agile methodologies. Additionally, the document provides insights into Kafka and its advantages for handling data streams in scalable and fault-tolerant systems.

Embed presentation

Download as PPSX, PPTX
ARAF KARSH HAMID Co-Founder / CTO MetaMagic Global Inc., NJ, USA @arafkarsh arafkarsh Microservices Architecture Containers, Saga, Testing, Kanban World API & DevOps Summit Bangalore, February 15, 2019
2 Microservices Containers & Kubernetes, Kafka Capability Centric Design DDD / ES & CQRS Agile: Kanban Continuous Delivery Testing Strategies Security Best Practices
MICROSERVICES • CONCEPTS • WHEN SHOULD YOU USE MICROSERVICES? • WHAT’S THE RIGHT SIZE? • ARCHITECTURE (INFRATRUCTURE AND SOFTWRE) 2/19/2019 3
Pioneers in Microservices Implementation 19-02-2019 4 New Entrants
Microservices Characteristics 5 By James Lewis and Martin Fowler We can scale our operation independently, maintain unparalleled system availability, and introduce new services quickly without the need for massive reconfiguration. — Werner Vogels, CTO, Amazon Web Services Modularity ... is to a technological economy what the division of labor is to a manufacturing one. W. Brian Arthur, author of e Nature of Technology The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be. Alan Kay, 1998 email to the Squeak-dev list Components via Services Organized around Business Capabilities Products NOT Projects Smart Endpoints & Dumb Pipes Decentralized Governance & Data Management Infrastructure Automation Design for Failure Evolutionary Design
When should I use them (Microservices)? 19-02-2019 6 • Strong Module Boundaries: Microservices reinforce modular structure, which is particularly important for larger teams. • Independent Deployment: Simple services are easier to deploy, and since they are autonomous, are less likely to cause system failures when they go wrong. • Technology Diversity: With microservices you can mix multiple languages, development frameworks and data- storage technologies. When you have What’s the Cost Distribution: Distributed systems are harder to program, since remote calls are slow and are always at risk of failure. Eventual Consistency: Maintaining strong consistency is extremely difficult for a distributed system, which means everyone has to manage eventual consistency. Operational Complexity: You need a mature operations team to manage lots of services, which are being redeployed regularly. Source: https://www.martinfowler.com/microservices/
What is the right size for a Microservice? 19-02-2019 7 • Rather than the size what matters is the Business Function / Domain of the service. • One Microservice may have half a dozen entities and other a couple of dozen entities. What’s more important is the role Microservices plays. • Bounded Context from DDD helps you to decompose a large multi domain Monolith into a Microservice for each Bounded Context. • Focusing on User stories will help you clearly define the boundaries of the Business Domain.
Microservices Architecture 19-02-2019 8 • API Gateway • Service Discovery • Config Service • Event Bus / Streams • Service Mesh • Hexagonal Architecture • Domain Driven Design • Event Sourcing & CQRS • Functional Reactive Programming Infrastructure Architecture
19-02-20199 Monolithic vs. Microservices Example Traditional Monolithic App using Single Technology Stack Micro Services with Multiple Technology Stack This 3 tier model is obsolete now. Source: Gartner Market Guide for Application Platforms Nov 23, 2016 Event Stream API Gateway (Zuul Edge Server) Load Balancer (Ribbon) Circuit Breaker (Hystrix) Service Discovery (Eureka) Load Balancer (Ribbon) Circuit Breaker (Hystrix) Load Balancer (Ribbon) Circuit Breaker (Hystrix) UI Layer Web Services Business Logic Database Layer Micro Service 4 EE 7 Inventory UI Layer Web Services Business Logic Database Layer Micro Service 1 Customer SE 8 UI Layer Web Services Business Logic Database Layer Micro Service 3 ShoppingCart UI Layer Web Services Business Logic Database Layer Micro Service 2 Order UI Layer WS BL DL Database ShoppingCart Order Customer Inventory
19-02-2019 10 SOA vs. Microservices Example Traditional Monolithic App with SOA Micro Services with Multiple Technology Stack Event Stream UI Layer Web Services Business Logic Database Layer Micro Service 1 Customer SE 8 UI Layer Web Services Business Logic Database Layer Micro Service 3 ShoppingCart UI Layer Web Services Business Logic Database Layer Micro Service 2 Order API Gateway Load Balancer Circuit Breaker Service Discovery Load Balancer Circuit Breaker Load Balancer Circuit Breaker UI Layer Database ShoppingCart Order Customer Inventory Enterprise Service Bus Messaging REST / SOAP HTTP MOM JMS ODBC / JDBC Translation Web Services XML WSDL Addressing Security Registry Management Producers Shared Database Consumers3rd Party Apps Smart Pipes Lot of Business logic resides in the Pipe
Microservices Deployment Model Micro Services with Multiple Technology Stack Event Stream Users Service Discovery (Eureka) Config Server (Spring) API (Zuul) Gateway UI Layer Web Services Business Logic Database Layer Micro Service 2 ShoppingCart SE 8 LB = Ribbon CB = Hystrix LB = Ribbon CB = Hystrix UI Layer Web Services Business Logic Database Layer Product SE 8 Micro Service 1 With 4 node cluster LB = Ribbon CB = Hystrix UI Layer Web Services Business Logic Database Layer Order SE 8 Micro Service 3 With 2 node Cluster LB = Ribbon CB = Hystrix UI Layer Web Services Business Logic Database Layer Customer Micro Service 4 With 2 node cluster HTTP Server All UI Code is bundled Virtual Private Network 19-02-2019 11
Pros 1. Adds Complexity 2. Skillset shortage 3. Confusion on getting the right size 4. Team need to manage end-to-end of the Service (From UI to Backend to Running in Production). 19-02-2019 12 1. Robust 2. Scalable 3. Testable (Local) 4. Easy to Change and Replace 5. Easy to Deploy 6. Technology Agnostic Cons Microservices Pros and Cons
Monolithic > Microservices • FORRESTER RESEARCH • MODERNIZATION JOURNEY • ASSESS AND CLASSIFY YOUR APP PORTFOLIO • PLAN AND PRIORITIZE 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 13
* For IT Services : They can do one more project of same size with ZERO COST in Platform Licensing (Based on 20 developer pack USD $50K per month for 3 months License cost = $150K) 1419-02-2019
Modernization Journey 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 15  Start new features as Microservices Incrementally establish the success early.  Expose Legacy On-Premise Apps API’s If legacy Apps cant be shifted to Cloud  Refactor Monolithic features to Microservices Breakdown and Deploy Feature by Feature  Containerize the Microservice Reduce costs, simplifies the operations and consistent environment between Dev, QA and Production  Monolith De-commission Plan Incrementally sunset the Monolith Velocity as you transform Increase your Delivery Velocity along the Journey High FuturePresent Low
Assess and Classify your App Portfolio 2/19/2019 16  Take inventory of your Apps Classify the Apps based on technology, complexity.  Align Apps to your Business Priorities Identify the Apps that are critical for Modernization.  Identify Business Modernization Requirements Create a Roadmap with faster go to market strategy  Understand the effort and Cost Evaluate all possible Modernization options Container Refactor Expose APIsLift & Shift BusinessValueCostComplexity Product Catalogue Product Review Inventory Shopping Cart Customer Profile Order Management
Plan and Prioritize 2/19/2019 17 Complexity Cost Value Score Rank Weightage 35% 25% 40% Product Reviews Med 3 High 5 Med 3 3.50 11 3 Shopping Cart High 5 Med 3 Med 3 3.70 11 4 Customer Med 3 Med 3 Low 1 2.20 7 6 Product Catalogue Med 3 Med 3 High 5 4.80 11 1 Order Very High 7 Med 3 High 5 5.20 15 2 Inventory Very High 7 High 5 Med 3 4.90 15 5  Prioritize Low Priority projects are good test cases but does not bring much business value.  Quick Wins Identify a feature / project which has good business value and low in complexity.  Project Duration Focus on shorter duration projects with high Business Value. Shopping Portal Features Prioritization
Containers & Kubernetes • DOCKER CONTAINERS • KUBERNETES – CONTAINER ORCHESTRATION • ISTIO – TRAFFIC MANAGEMENT / NETWORK POLICIES 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 18
Servers / Virtual Machines / Containers Hardware OS BINS / LIB App 1 App 2 App 3 Server Hardware Host OS HYPERVISOR App 1 App 2 App 3 Guest OS BINS / LIB Guest OS BINS / LIB Guest OS BINS / LIB Type 1 Hypervisor Hardware Host OS App 1 App 2 App 3 BINS / LIB BINS / LIB BINS / LIB Container Hardware HYPERVISOR App 1 App 2 App 3 Guest OS BINS / LIB Guest OS BINS / LIB Guest OS BINS / LIB Type 2 Hypervisor19-02-2019 19
Linux Kernel 20 HOST OS (Ubuntu) Client Docker Daemon Cent OS Alpine Debian HostLinuxKernel Host Kernel Host Kernel Host Kernel All the containers will have the same Host OS Kernel If you require a specific Kernel version then Host Kernel needs to be updated HOST OS (Windows 10) Client Docker Daemon Nano Server Server Core Nano Server WindowsKernel Host Kernel Host Kernel Host Kernel Windows Kernel
Docker DaemonDocker Client How Docker works…. $ docker search …. $ docker build …. $ docker container create .. Docker Hub Images Containers $ docker container run .. $ docker container start .. $ docker container stop .. $ docker container ls .. $ docker push …. $ docker swarm .. 19-02-2019 21 21 34 1. Search for the Container 2. Docker Daemon Sends the request to Hub 3. Downloads the image 4. Run the Container from the image
Kubernetes Key Concepts 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 22 o Declarative Model Infrastructure as a code o Desired State Minimum state of your App / Microservice o Current State Current State of the due to error or load factor Pods Replicas Deploy Service
Deployment – Updates and rollbacks, Canary Release D ReplicaSet – Self Healing, Scalability, Desired State R Worker Node 1 Master Node (Control Plane) Kubernetes Architecture POD POD itself is a Linux Container, Docker container will run inside the POD. PODs with single or multiple containers (Sidecar Pattern) will share Cgroup, Volumes, Namespaces of the POD. (Cgroup / Namespaces) Scheduler Controller Manager Using yaml or json declare the desired state of the app. State is stored in the Cluster store. Self healing is done by Kubernetes using watch loops if the desired state is changed. POD POD POD BE 1.210.1.2.34 BE 1.210.1.2.35 BE 1.210.1.2.36 BE 15.1.2.100 DNS: a.b.com 1.2 Service Pod IP Address is dynamic, communication should be based on Service which will have routable IP and DNS Name. Labels (BE, 1.2) play a critical role in ReplicaSet, Deployment, & Services etc. Cluster Store etcd Key Value Store Pod Pod Pod Label Selector selects pods based on the Labels. Label Selector Label Selector Label Selector Node Controller End Point Controller Deployment Controller Pod Controller …. Labels Internet Firewall K8s Cluster Cloud Controller For the cloud providers to manage nodes, services, routes, volumes etc. Kubelet Node Manager Container Runtime Interface Port 10255 gRPC ProtoBuf Kube-Proxy Network Proxy TCP / UDP Forwarding IPTABLES / IPVS Allows multiple implementation of containers from v1.7 RESTful yaml / json $ kubectl …. Port 443API Server Pod IP ...34 ...35 ...36EP • Declarative Model • Desired State Key Aspects Namespace1Namespace2 • Pods • ReplicaSet • Deployment • Service • Endpoints • StatefulSet • Namespace • Resource Quota • Limit Range • Persistent Volume Kind Secrets Kind • apiVersion: • kind: • metadata: • spec: Declarative Model • Pod • ReplicaSet • Service • Deployment • Virtual Service • Gateway, SE, DR • Policy, MeshPolicy • RbaConfig • Prometheus, Rule, • ListChekcer … @ @ Annotations Names Cluster IP Node Port Load Balancer External Name @ Ingress 19-02-2019 23
Service Mesh – Sidecar Design Pattern 19-02-2019 24 CB – Circuit Breaker LB – Load Balancer SD – Service Discovery Microservice Process1Process2 Service Mesh Control Plane Service Discovery Routing Rules Control Plane will have all the rules for Routing and Service Discovery. Local Service Mesh will download the rules from the Control pane will have a local copy. Service Discovery Calls Service Mesh Calls Customer Microservice Application Localhost calls http://localhost/api/order/ Router Network Stack LBCB SD ServiceMesh Sidecar UI Layer Web Services Business Logic Order Microservice Application Localhost calls http://localhost/api/payment/ Router Network Stack LBCB SD ServiceMesh Sidecar UI Layer Web Services Business Logic Data Plane
Shopping Portal /ui /productms /productreview Gateway Virtual Service UI Pod UI Pod UI Pod UI Service Product Pod Product Pod Product Pod Product Service Review Pod Review Pod Review Pod Review Service Deployment / Replica / Pod N1 N2 N2 MySQL Pod N4 N3 N1 N4 N3 NodesIstio Sidecar - Envoy Destination Rule Destination Rule Destination Rule Load Balancer Kubernetes Objects Istio Objects Firewall P M CIstio Control Plane UI Pod N5 v1 v2 Stable / v1 Canary v2User X = Canary Others = Stable A / B Testing using Canary Deployment Service Call Kube DNS EndPoints EndPoints EndPoints Internal Load Balancers 25 Source:https://github.com/meta-magic/kubernetes_workshop
Shopping Portal /ui /productms /productreview Gateway Virtual Service UI Pod UI Pod UI Pod UI Service Product Pod Product Pod Product Pod Product Service Review Pod Review Pod Review Pod Review Service Deployment / Replica / Pod N1 N2 N2 MySQL Pod N4 N3 N1 N4 N3 NodesIstio Sidecar - Envoy Destination Rule Destination Rule Destination Rule Load Balancer Kubernetes Objects Istio Objects Firewall P M CIstio Control Plane UI Pod N5 v1 v2 Stable / v1 Canary v210% = Canary 90% = Stable Traffic Shifting Canary Deployment Service Call Kube DNS EndPoints EndPoints EndPoints Internal Load Balancers 26 Source:https://github.com/meta-magic/kubernetes_workshop
27 Shopping Portal /ui /productms /productreview Gateway Virtual Service UI Pod UI Pod UI Pod UI Service Review Pod Review Pod Review Pod Review Service Deployment / Replica / Pod N1 N2 N2 MySQL Pod N4 N3 N1 N4 N3 NodesIstio Sidecar - Envoy Destination Rule Destination Rule Destination Rule Load Balancer Kubernetes Objects Istio Objects Firewall P M CIstio Control Plane v1 Fault Injection Delay = 2 Sec Abort = 10% Fault Injection Product Pod Product Pod Product Pod Product Service Service Call Kube DNS EndPoints EndPoints EndPoints Internal Load Balancers 27 Source:https://github.com/meta-magic/kubernetes_workshop
Kafka • CONCEPTS : QUEUES / PUB – SUB / EVENT STREAMING • WHY IS IT DIFFERENT FROM TRADITIONAL MESSAGE QUEUES? • DATA STORAGE / CLUSTER / DURABILITY • PERFORMANCE 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 28
Kafka Core Concepts 19-02-2019 29 Publish & Subscribe Read and write streams of data like a messaging system Process Write scalable stream processing apps that react to events in real- time. Store Store streams of data safely in a distributed, replicated, fault tolerant cluster.
Traditional Queue / Pub-Sub Vs. Kafka 19-02-2019 30 0 1 2 3 4 5 6 7 8 9 8 7 9 Consumer 1 Consumer 2 Consumer 3 Queues Data Data can be partitioned for scalability for parallel processing by same type of consumers Pros: Cons: Queues are not multi subscribers. Once a Consumer reads the data, its gone from the queue. Ordering of records will be lost in asynchronous parallel processing. 0 1 2 3 4 5 6 7 8 9 9 9 9 Consumer 1 Consumer 2 Consumer 3 Pub – Sub Data Multiple subscribers can get the same data.Pros: Scaling is difficult as every message goes to every subscriber. Cons: Kafka generalizes these two concepts.  As with a queue the consumer group allows you to divide up processing over a collection of processes (the members of the consumer group).  As with publish-subscribe, Kafka allows you to broadcast messages to multiple consumer groups.
Anatomy of a Topic 19-02-2019 31 Source : https://kafka.apache.org/intro • A Topic is a category or feed name to which records are published. • Topics in Kafka are always multi subscriber. • That is, a Topic can have zero, one, or many consumers that subscribe to the data written to it. • Each Partition is an ordered, immutable sequence of records that is continually appended to—a structured commit log. • A Partition is nothing but a directory of Log Files • The records in the partitions are each assigned a sequential id number called the offset that uniquely identifies each record within the partition.
19-02-2019 32 Partition Log Segment • Partition (Kafka’s Storage unit) is Directory of Log Files. • A partition cannot be split across multiple brokers or even multiple disks • Partitions are split into Segments • Segments are two files: 000.log & 000.index • Segments are named by their base offset. The base offset of a segment is an offset greater than offsets in previous segments and less than or equal to offsets in that segment. • Indexes store offsets relative to its segments base offset • Indexes map each offset to their message position in the log and they are used to look up messages. • Purging of data is based on oldest segment and one segment at a time. 0 1 2 3 4 5 6 7 8 9 Partition Data 6 3 0 Segment 0 Segment 3 Segment 6 9 Segment 9 - Active $ tree kafka-logs | head -n 6 kafka-logs |──── SigmaHawk-2 | |──── 00000000006109871597.index | |──── 00000000006109871597.log | |──── 00000000007306321253.index | |──── 00000000007306321253.log Topic / Partition Segment 1 Segment 2 Rel.Offset, Position Offset, Position, Size, Payload 0000.index 0000.log 0 0 0 0 7 ABCDEFG 1 7 1 7 4 ABCD 2 11 2 11 9 ABCDEFGIJ 4 Bytes 4 Bytes
19-02-2019 33 Kafka Cluster – Topics & Partitions • The partitions of the log are distributed over the servers in the Kafka cluster with each server handling data and requests for a share of the partitions. Source : https://kafka.apache.org/intro Broker 1 Leader Broker 2 Follower Broker 3 Follower Broker 4 Follower Broker 5 Leader Partition 1 Partition 0 Topic ABC • Each server acts as a leader for some of its partitions and a follower for others so load is well balanced within the cluster. • Each partition has one server which acts as the "leader" and zero or more servers which act as "followers".
19-02-2019 34 Record Commit Process Broker 1 Leader Topic 1 Broker 2 Follower Producer Consumer 2 2 Commit 3 ack • Each partition is replicated across a configurable number of servers for fault tolerance. • The leader handles all read and write requests for the partition while the followers passively replicate the leader. • If the leader fails, one of the followers will automatically become the new leader.1 Message with Offset 4 777743 Broker 3 Follower Data Durability From Kafka v0.8.0 onwards acks Acknowledgement Description 0 If set to zero then the producer will NOT wait for any acknowledgment from the server at all. The record will be immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received the record in this case, and the retries configuration will not take effect (as the client won't generally know of any failures). The offset given back for each record will always be set to -1. 1 This will mean the leader will write the record to its local log but will respond without awaiting full acknowledgement from all followers. In this case should the leader fail immediately after acknowledging the record but before the followers have replicated it then the record will be lost. All / -1 This means the leader will wait for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost as long as at least one in-sync replica remains alive. This is the strongest available guarantee. This is equivalent to the acks=-1 setting. Source: https://kafka.apache.org/documentation/#topicconfigs acks Steps 0 1 1 1,3 -1 1,2,3 Producer Configuration
19-02-2019 35 Replication 6 3.2 m1 m2 m3 L(A) m1 m2 F(B) m1 F(C)ISR = (A, B, C) Leader A commits Message m1. Message m2 & m3 not yet committed. 1 m1 m2 F(C) m1 m2 L(B) m1 m2 m3 L(A) ISR = (B,C) A fails and B is the new Leader. B commits m22 m1 m2 m3 L(A) m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 ISR = (B,C) B commits new messages m4 and m5 3 m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 m1 F(A) ISR = (A, B,C) A comes back, restores to last commit and catches up to latest messages. 4 m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 m1 m2 F(A) m4 m5 ISR – In-sync Replica • Instead of majority vote, Kafka dynamically maintains a set of in-sync replicas (ISR) that are caught-up to the leader. • Only members of this set are eligible for election as leader. • A write to a Kafka partition is not considered committed until all in-sync replicas have received the write. • This ISR set is persisted to ZooKeeper whenever it changes. Because of this, any replica in the ISR is eligible to be elected leader.
LinkedIn Kafka Cluster 19-02-2019 36 Brokers60 Partitions50K Messages / Second800K MB / Second inbound300 MB / Second Outbound1024 The tuning looks fairly aggressive, but all of the brokers in that cluster have a 90% GC pause time of about 21ms, and they’re doing less than 1 young GC per second.
Uber Kafka Cluster 19-02-2019 37 Topics10K+ Events / Second11M Petabyte of Data1PB+
12 Factor App Methodology 19-02-2019 38 4 Backing Services Treat Backing services like DB, Cache as attached resources 5 Build, Release, Run Separate Build and Run Stages 6 Process Execute App as One or more Stateless Process 7 Port Binding Export Services with Specific Port Binding 8 Concurrency Scale out via the process Model 9 Disposability Maximize robustness with fast startup and graceful exit 10 Dev / Prod Parity Keep Development, Staging and Production as similar as possible 11 Logs Treat logs as Event Streams 12 Admin Process Run Admin Tasks as one of Process Source:https://12factor.net/ Factors Description 1 Codebase One Code base tracked in revision control 2 Dependencies Explicitly declare dependencies 3 Configuration Configuration driven Apps
CAPABILITY CENTRIC DESIGN • BUSINESS FUNCTIONS • BUSINESS PROCESS • TEAM STRUCTURE 2/19/2019 39
Business Solution & Business Process 2/19/2019 40  Business Solution focuses the entire Journey of the User which can run across multiple Micro Services.  Business Solution comprises a set of Business Processes.  A specific Micro Service functionality will be focused on a Business Process / Concern Business Solution: Customer Dining Experience Order PaymentFood Menu KitchenDining Browse Menu Order Dinner Dinner Served Get Bill Make Payment
2/19/2019 41 Capability Centric Design Vertically sliced Product Team Business Centric Development • Focus on Business Capabilities • Entire team is aligned towards Business Capability. • From Specs to Operations – The team handles the entire spectrum of Software development. • Every vertical will have it’s own Code Pipeline Front-End-Team Back-End-Team Database-Team In a typical Monolithic way the team is divided based on technology / skill set rather than business functions. This leads to not only bottlenecks but also lack of understanding of the Business Domain. QA / QC Team Front-End Back-End Database Business Capability 1 QA/QCTeam Front-End Back-End Database Business Capability 2 QA/QCTeam Front-End Back-End Database Business Capability 3 QA/QCTeam
2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 42 Architecture / Design • DOMAIN DRIVEN DESIGN • EVENT SOURCING AND CQRS • FUNCTIONAL REACTIVE PROGRAMMING • DISTRIBUTED TRANSACTIONS • CASE STUDY
2/19/2019 43 Ubiquitous Language Vocabulary shared by all involved parties Used in all forms of spoken / written communication Ubiquitous Language Domain Expert Analyst Developers QA Design Docs Test Cases Code Restaurant Context – Food Item : Eg. Food Item (Navrathnakurma) can have different meaning or properties depends on the context. • In the Menu Context it’s a Veg Dish. • In the Kitchen Context it’s is recipe. • And in the Dining Context it will have more info related to user feed back etc. DDD: Ubiquitous Language: Strategic Design As an Restaurant Owner I want to know who my Customers are So that I can serve them better Role-Feature-Reason Matrix BDD – Behavior Driven Development Given Customer John Doe exists When Customer orders food Then Assign customer preferences as Veg or Non Veg customer BDD Construct
2/19/2019 44 Domain Driven Design – Tactical Design Source: Domain-Driven Design Reference by Eric Evans
2/19/2019 45 Shopping Portal Modules – Code Packaging Auth Products Cart OrderCustomer Domain Layer • Models • Repo • Services • Factories Adapters • Repo • Services • Web Services Domain Layer • Models • Repo • Services • Factories Adapters • Repo • Services • Web Services Domain Layer • Models • Repo • Services • Factories Adapters • Repo • Services • Web Services Packaging Structure Bounded Context Implementation (Repositories, Business Services, Web Services) Domain Models (Entities, Value Objects, DTOs) (Repositories, Business Services, Web Services) Entity Factories Interfaces (Ports)
2/19/2019 46 DDD: Use Case Order Module Models Value Object • Currency • Item Value • Order Status • Payment Type • Record State • Audit Log Entity • Order (Aggregate Root) • Order Item • Shipping Address • Payment DTO • Order • Order Item • Shipping Address • Payment Domain Layer Adapters • Order Repository • Order Service • Order Web Service • Order Query Web Service • Shipping Address Web Service • Payment Web Service Adapters Consists of Actual Implementation of the Ports like Database Access, Web Services API etc. Converters are used to convert an Enum value to a proper Integer value in the Database. For Example Order Status Complete is mapped to integer value 100 in the database. Services / Ports • Order Repository • Order Service • Order Web Service Utils • Order Factory • Order Status Converter • Record State Converter • Order Query Web Service • Shipping Address Web Service • Payment Web Service Shopping Portal
2/19/2019 47 Event Sourcing Intro Standard CRUD Operations – Customer Profile – Aggregate Root Profile Created Title Updated New Address added Derived Notes Removed Time T1 T2 T4T3 Event Sourcing and Derived Aggregate Root Commands 1. Create Profile 2. Update Title 3. Add Address 4. Delete Notes 2 Events 1. Profile Created Event 2. Title Updated Event 3. Address Added Event 4. Notes Deleted Event 3 Current State of the Customer Profile 4 Event store Single Source of Truth Greg Young
2/19/2019 48 Functional Reactive Programming: 4 Building Blocks of RxJava Source of Data Stream [ Sender ]Observable1 Listens for emitted values [ Receiver ]Observer2 Source: http://reactivex.io/ Schedulers3 Schedulers are used to manage and control concurrency. 1. observeOn: Thread Observable is executed 2. subscribeOn: Thread subscribe is executed 4 Operators Content Filtering Time Filtering Transformation Operators that let you Transform, Combine, Manipulate, and work with the sequence of items emitted by Observables
2/19/2019 49 Comparison : Iterable / Streams / Observable 1Building Block First Class Visitor (Consumer) Serial Operations Parallel Streams (10x Speed) Still On Next, On Complete and On Error are Serial Operations Completely Asynchronous Operations Java 8 – Blocking CallJava 6 – Blocking Call Rx Java - Freedom Source Code: https://github.com/meta-magic/rxjava
2/19/2019 50 Rx 2 Java Operator : Filter / Sort / FlatMap 4Building Block Objective: toSortedList() returns an Observable with a single List containing Fruits. Using FlatMap to Transform Observable <List> to Observable <Fruit> Rx Example 2 SourceCodeGitHub:https://github.com/meta-magic/Rx-Java-2 • Merge • Filter • Sort • Take
2/19/2019 51 Distributed Tx: SAGA Design Pattern instead of 2PC Long Lived Transactions (LLTs) hold on to DB resources for relatively long periods of time, significantly delaying the termination of shorter and more common transactions. Source: SAGAS (1987) Hector Garcia Molina / Kenneth Salem, Dept. of Computer Science, Princeton University, NJ, USA T1 T2 Tn Local Transactions C1 C2 Cn-1 Compensating Transaction Divide long–lived, distributed transactions into quick local ones with compensating actions for recovery. Travel : Flight Ticket & Hotel Booking Example BASE (Basic Availability, Soft State, Eventual Consistency) Room ReservedT1 Room PaymentT2 Seat ReservedT3 Ticket PaymentT4 Cancelled Room Reservation C1 Cancelled Room Payment C2 Cancelled Ticket Reservation C3
2/19/2019 52 Case Study: Restaurant Dining – Event Sourcing and CQRS Order Payment • Add Drinks • Add Food • Update Food Commands • Open Table • Add Juice • Add Soda • Add Appetizer 1 • Add Appetizer 2 • Serve Drinks • Prepare Food • Serve Food Events • Drinks Added • Food Added • Food Updated • Food Discontinued • Table Opened • Juice Added • Soda Added • Appetizer 1 Added • Appetizer 2 Added • Juice Served • Soda Served • Appetizer Served • Food Prepared • Food Served • Prepare Bill • Process Payment • Bill Prepared • Payment Processed • Payment Approved • Payment Declined • Cash Paid When people arrive at the Restaurant and take a table, a Table is opened. They may then order drinks and food. Drinks are served immediately by the table staff, however food must be cooked by a chef. Once the chef prepared the food it can then be served. Table is closed then the bill is prepared. Microservices • Dinning Order • Billable Order Customer Journey thru Dinning Processes Processes Food Menu KitchenDining • Remove Soda • Add Food 1 • Add Food 2 • Place Order • Close Table • Remove Soda • Food 1 Added • Food 2 Added • Order Placed • Table Closed ES Aggregate 3 2 4 1
2/19/2019 53 RESTful APIs • STANDARDS • API VERSIONING STANDARDS
2/19/2019 54 RESTful Guidelines 1. Endpoints as nouns, NOT verbs Ex. /catalogues /orders /catalogues/products and NOT /getProducts/ /updateProducts/ 2. Use plurals Ex. /catalogues/{catalogueId} and NOT /catalogue/{catalogueId} 3. Documenting 4. Paging 5. Use SSL 6. HTTP Methods GET / POST / PUT / DELETE / OPTIONS / HEAD 7. HTTP Status Codes (Effective usage) 8. Versioning Media Type Version GET /account/5555 HTTP/1.1 Accept: application/vnd.catalogues.v1+json URL path version https://domain/v1/catalogues/products
2/19/2019 55 RESTful Guidelines – Query Examples Search All Products Search Products By Catalogue ID Search Products By Catalogue ID & Product ID
2/19/2019 56 RESTful Guidelines – Query Examples Two different implementation of same query
2/19/2019 57 # Name * Who Uses Pros Cons 1 Media Type Versioning Accept: Application/vnd.api.article+xml; version=1.0 Med GitHub • Version Directly @ resource level • Preserve URI • Close to RESTful Specs • Harder to Test • Distort HTTP Headers purpose • Tools required for testing 2 Custom Headers Versioning X-API-Version: 2. Med Microsoft • Preservers URI • Harder to Test • Tools required for testing 3 URI Versioning api.example.com/v1/resource High Google Twitter Amazon • Most common method • Versions can be explored using Browser • Easy to use • Disrupts RESTful Compliance. URI should represent resource and not versions 4 Domain Versioning apiv1.example.com/resource Low Facebook • Same as are URI Versioning • Same as URI Versioning 5 Request Parameter Versioning GET /something/?version=0.1 High Pivotal NetFlix • Similar to URI versioning • It can get messy 6 Date Versioning First request saves the date. Low Clearbit • New APIs can be shipped without changing the end points • Complex to implement • Traceability is difficult. API Versioning
2/19/2019 58 Testing Strategies • UNIT TESTING • COMPONENT TESTING • INTEGRATION CONTRACT TESTING • INTEGRATION TESTING
2/19/2019 59 Microservices Testing Strategies Ubiquitous Language Domain Expert Analyst Developers QA Design Docs Test Cases Code E2E Testing Integration Testing Contract Testing Component Testing Unit Testing Number of Tests Speed Cost Time Mike Cohen’s Testing Pyramid Test Pyramid: https://martinfowler.com/bliki/TestPyramid.html 70% 20% 10%
2/19/2019 60 Microservices Testing Strategy Unit Testing A unit test exercises the smallest piece of testable software in the application to determine whether it behaves as expected. Source: https://martinfowler.com/articles/microservice-testing/#agenda Component Testing A component test limits the scope of the exercised software to a portion of the system under test, manipulating the system through internal code interfaces and using test doubles to isolate the code under test from other components. Integration Testing An integration test verifies the communication paths and interactions between components to detect interface defects Integration Contract Testing An Integration Contract test is a test at the boundary of an external service verifying that it meets the contract expected by a consuming service. End 2 End Testing An end-to-end test verifies that a system meets external requirements and achieves its goals, testing the entire system, from end to end Say NO to End 2 End Tests - Mike Walker April 22, 2015. Google Test Blog
2/19/2019 61 Testing ToolsMicroservices Testing Scenarios / Tools Contract Testing Scope Integration Testing Verifies the communication paths and interactions between components to detect interface defects Contract Testing It is a test at the boundary of an external service verifying that it meets the contract expected by a consuming service. Payment Mock IntegrationContractTestingScope Test Double Montebank Cart Component Testing Unit Testing IntegrationTestingScope Order REST / HTTP or Events / Kafka Item ID, Quantity, Address.. Mock Order Component Testing A component test limits the scope of the exercised software to a portion of the system under test. Order Payment Unit Testing Firewall Integration Testing Scope REST / HTTP Payment Sandbox Component Testing
2/19/2019 62 Security Best Practices • DOCKER • KUBERNETES
2/19/2019 63 Build Small Container Images • Simple Java Web Apps with Ubuntu & Tomcat can have a size of 700 MB • Use Alpine Image as your base Linux OS • Alpine images are 10x smaller than base Ubuntu images • Smaller Image size reduce the Container vulnerabilities. • Ensure that only Runtime Environments are there in your container. For Example your Alpine + Java + Tomcat image should contain only the JRE and NOT JDK. • Log the App output to Container Std out and Std error. 1
2/19/2019 64 Docker: To Root or Not to Root! • Create Multiple layers of Images • Create a User account • Add Runtime software’s based on the User Account. • Run the App under the user account • This gives added security to the container. • Add Security module SELinux or AppArmour to increase the security, Alpine JRE 8 Tomcat 8 My App 1 2
2/19/2019 65 Docker: Container Security 1. Secure your HOST OS! Containers runs on Host Kernel. 2. No Runtime software downloads inside the container. Declare the software requirements at the build time itself. 3. Download Docker base images from Authentic site. 4. Limit the resource utilization using Container orchestrators like Kubernetes. 5. Don’t run anything on Super privileged mode. 3
2/19/2019 66 Kubernetes: Naked Pods • Never use a Naked Pod, that is Pod without any ReplicaSet or Deployments. Naked pods will never get re-scheduled if the Pod goes down. • Never access a Pod directly from another Pod. Always use a Service to access a Pod. • User labels to select the pods { app: myapp, tier: frontend, phase: test, deployment: v3 }. • Never use :latest tag in the image in the production scenario. 4
2/19/2019 67 Kubernetes: Namespace default Kube system Kube public Kubernetes Cluster• Group your Services / Pods / Traffic Rules based on Specific Namespace. • This helps you apply specific Network Policies for that Namespace with increase in Security and Performance. • Handle specific Resource Allocations for a Namespace. • If you have more than a dozen Microservices then it’s time to bring in Namespaces. Service-Name.Namespace.svc.cluster.local $ kubectl config set-context $(kubectl config current-context) --namespace=your-ns The above command will let you switch the namespace to your namespace (your-ns). 5
2/19/2019 68 Kubernetes: Pod Health Check • Pod Health check is critical to increase the overall resiliency of the network. • Readiness • Liveness • Ensure that all your Pods have Readiness and Liveness Probes. • Choose the Protocol wisely (HTTP, Command & TCP) 6
2/19/2019 69 Kubernetes: Resource Utilization • For the Best Quality define the requests and limits for your Pods. • You can set specific resource requests for a Dev Namespace to ensure that developers don’t create pods with a very large resource or a very small resource. • Limit Range can be set to ensure that containers were create with too low resource or too large resource. 7
2/19/2019 70 Kubernetes: Pod Termination Lifecycle • Make sure that the Application to Handle SIGTERM message. • You can use preStop Hook • Set the terminationGracePeriodSeconds: 60 • Ensure that you clean up the connections or any other artefacts and ready for clean shutdown of the App (Microservice). • If the Container is still running after the grace period, Kubernetes sends a SIGKILL event to shutdown the Pod. 8
2/19/2019 71 Kubernetes: External Services • There are systems that can be outside the Kubernetes cluster like • Databases or • external services in the cloud. • You can create an Endpoint with Specific IP Address and Port with the same name as Service. • You can create a Service with an External Name (URL) which does a CNAME redirection at the Kernel level. 9
2/19/2019 72 Kubernetes: Upgrade Cluster • Make sure that the Master behind a Load Balancer. • Upgrade Master • Scale up the Node with an extra Node • Drain the Node and • Upgrade Node • Cluster will be running even if the master is not working. Only Kubectl and any master specific functions will be down until the master is up. 10
2/19/2019 73 Kanban • BOARDS AND CARDS • KANBAN VS SCRUM • BENEFITS OF KANBAN
2/19/2019 74 What is Kanban Kanban is a method for managing the creation of products with an emphasis on • continual delivery while • not overburdening the development team. Like Scrum, Kanban is a process designed to help teams work together more effectively. Kanban is a visual management method that was developed by Hirotaka Takeuchi and Ikujiro Nonaka
2/19/2019 75 Three Principles of Kanban Source: https://resources.collab.net/agile-101/what-is-kanban • Visualize what you do today (workflow): seeing all the items in context of each other can be very informative • Limit the amount of work in progress (WIP): this helps balance the flow-based approach so teams don’t start and commit to too much work at once • Enhance flow: when something is finished, the next highest thing from the backlog is pulled into play
2/19/2019 76 Kanban Release Cycles Preparation Requirements Design Development Testing Release 1 – 4 Weeks Cycle
2/19/2019 77 Kanban Boards and Cards
2/19/2019 78 Kanban vs. Scrum Kanban Scrum Roles & Responsibilities No prescribed roles Pre-defined roles of Scrum master, Product owner and team member Delivery Time Lines Continuous Delivery Time boxed sprints Delegation & Prioritization Work is pulled through the system (single piece flow) Work is pulled through the system in batches (the sprint backlog) Modifications Changes can be made at any time No changes allowed mid-sprint Measurement of Productivity Cycle time Velocity When to Use? More appropriate in operational environments with a high degree of variability in priority More appropriate in situations where work can be prioritized in batches that can be left alone Source: https://leankit.com/learn/kanban/kanban-vs-scrum/
2/19/2019 79 Benefits of Kanban • Shorter cycle times can deliver features faster. • Responsiveness to Change: • When priorities change very frequently, Kanban is ideal. • Balancing demand against throughput guarantees that most the customer-centric features are always being worked. • Requires fewer organization / room set-up changes to get started • Reducing waste and removing activities that don’t add value to the team/department/organization • Rapid feedback loops improve the chances of more motivated, empowered and higher-performing team members
2/19/2019 80 Design Patterns are solutions to general problems that software developers faced during software development. Design Patterns
2/19/2019 81 Araf Karsh Hamid : Co-Founder / CTO araf.karsh@metamagic.in USA: +1 (973) 969-2921 India: +91.999.545.8627 Skype / LinkedIn / Twitter / Slideshare : arafkarsh http://www.slideshare.net/arafkarsh https://www.linkedin.com/in/arafkarsh/ http://www.arafkarsh.com/ DREAM | AUTOMATE | EMPOWER
2/19/2019 82 UI DesignPatterns MVC/ MVP/ MVVM View Controller Model Passes calls To Fire Events Manipulates • The Controller is responsible to process incoming requests. It receives input from users via the View, then process the user's data with the help of Model and passing the results back to the View. • Typically, it acts as the coordinator between the View and the Model. Model View Controller 1 * • The View Model is responsible for exposing methods, commands, and other properties that helps to maintain the state of the view, manipulate the model as the result of actions on the view, and trigger events in the view itself. • There is many-to-one relationship between View and ViewModel means many View can be mapped to one ViewModel. • Supports two-way data binding between View and ViewModel. View ViewModel Model Passes calls To Manipulates Updates Fire Events Model View ViewModel • The Presenter is responsible for handling all UI events on behalf of the view. This receive input from users via the View, then process the user's data with the help of Model and passing the results back to the View. • Unlike view and controller, view and presenter are completely decoupled from each other’s and communicate to each other’s by an interface. Also, presenter does not manage the incoming request traffic as controller. • Supports two-way data binding. Model View Presenter View Presenter Model Passes calls To Fire Events Manipulates Updates1 1
2/19/2019 83 UI Design Patterns Flux / Redux ViewDispatcher Every action is sent to all Stores via callbacks the stores register with the Dispatcher Store Action Action 1 * Controller-Views • Listens to Store changes • Emit Actions to Dispatcher Dispatcher • Single Dispatcher per Application • Manages the Data Flow View to Model • Receives Actions and dispatch them to Stores Stores • Contains state for a Domain (Vs. Specific Component) • In Charge of modifying the Data • Inform the views when the Data is changed by emitting the Changed Event. Flux Core Concepts 1. One way Data Flow 2. No Event Chaining 3. Entire App State is resolved in store before Views Update 4. Data Manipulation ONLY happen in one place (Store). Actions • Simple JS Objects • Contains Name of the Action and Data (Payload) • Action represent something that has happened. • Has No Business Logic
2/19/2019 84 UI Design Patterns Redux Actions • Simple JS Objects • Contains Name of the Action and Data (Payload) • Has NO Business Logic • Action represent something that has happened. Store • Multiple View layers can Subscribe • View layer to Dispatch actions • Single Store for the Entire Application • Data manipulation logic moves out of store to Reducers Reducer • Pure JS Functions • No External calls • Can combine multiple reducers • A function that specifies how the state changes in response to an Action. • Reducer does NOT modify the state. It returns the NEW State. Redux Core Concepts 1. One way Data Flow 2. No Dispatcher compared to Flux 3. Immutable Store Available for React & Angular View Action State Dispatcher Reducer R R R Store Middleware Middleware Middleware • Handles External calls • Multiple Middleware's can be chained.

More Related Content

PPTX
Microservices Architecture - Bangkok 2018
byAraf Karsh Hamid
 
PPTX
Micro services Architecture
byAraf Karsh Hamid
 
PPSX
Containers Docker Kind Kubernetes Istio
byAraf Karsh Hamid
 
PPSX
Microservices, DevOps & SRE
byAraf Karsh Hamid
 
PPSX
Big Data Redis Mongodb Dynamodb Sharding
byAraf Karsh Hamid
 
PPSX
Agile, User Stories, Domain Driven Design
byAraf Karsh Hamid
 
PPSX
Microservices Testing Strategies JUnit Cucumber Mockito Pact
byAraf Karsh Hamid
 
PPSX
Event Sourcing & CQRS, Kafka, Rabbit MQ
byAraf Karsh Hamid
 
Microservices Architecture - Bangkok 2018
byAraf Karsh Hamid
 
Micro services Architecture
byAraf Karsh Hamid
 
Containers Docker Kind Kubernetes Istio
byAraf Karsh Hamid
 
Microservices, DevOps & SRE
byAraf Karsh Hamid
 
Big Data Redis Mongodb Dynamodb Sharding
byAraf Karsh Hamid
 
Agile, User Stories, Domain Driven Design
byAraf Karsh Hamid
 
Microservices Testing Strategies JUnit Cucumber Mockito Pact
byAraf Karsh Hamid
 
Event Sourcing & CQRS, Kafka, Rabbit MQ
byAraf Karsh Hamid
 

What's hot

PPSX
Microservices Docker Kubernetes Istio Kanban DevOps SRE
byAraf Karsh Hamid
 
PPTX
Microservices Architecture & Testing Strategies
byAraf Karsh Hamid
 
PDF
A Pattern Language for Microservices
byChris Richardson
 
PPSX
Domain Driven Design
byAraf Karsh Hamid
 
PDF
Implementing Domain Events with Kafka
byAndrei Rugina
 
PDF
Kubernetes 101
byWinton Winton
 
PPTX
Microservices Part 3 Service Mesh and Kafka
byAraf Karsh Hamid
 
PPTX
Service mesh
byArnab Mitra
 
PDF
Istio service mesh introduction
byKyohei Mizumoto
 
PPTX
AKS
bygirish goudar
 
PPTX
Microservices Architecture Part 2 Event Sourcing and Saga
byAraf Karsh Hamid
 
PDF
Evolution of containers to kubernetes
byKrishna-Kumar
 
PDF
Kubernetes Basics
byEueung Mulyana
 
PPSX
Microservices Architecture - Cloud Native Apps
byAraf Karsh Hamid
 
PDF
Microservices architecture
byAbdelghani Azri
 
PPSX
CI-CD Jenkins, GitHub Actions, Tekton
byAraf Karsh Hamid
 
PPTX
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
bySimplilearn
 
PDF
Introduction to kubernetes
byGabriel Carro
 
PPSX
Service Mesh - Observability
byAraf Karsh Hamid
 
PPTX
Microservice architecture design principles
bySanjoy Kumar Roy
 
Microservices Docker Kubernetes Istio Kanban DevOps SRE
byAraf Karsh Hamid
 
Microservices Architecture & Testing Strategies
byAraf Karsh Hamid
 
A Pattern Language for Microservices
byChris Richardson
 
Domain Driven Design
byAraf Karsh Hamid
 
Implementing Domain Events with Kafka
byAndrei Rugina
 
Kubernetes 101
byWinton Winton
 
Microservices Part 3 Service Mesh and Kafka
byAraf Karsh Hamid
 
Service mesh
byArnab Mitra
 
Istio service mesh introduction
byKyohei Mizumoto
 
AKS
bygirish goudar
 
Microservices Architecture Part 2 Event Sourcing and Saga
byAraf Karsh Hamid
 
Evolution of containers to kubernetes
byKrishna-Kumar
 
Kubernetes Basics
byEueung Mulyana
 
Microservices Architecture - Cloud Native Apps
byAraf Karsh Hamid
 
Microservices architecture
byAbdelghani Azri
 
CI-CD Jenkins, GitHub Actions, Tekton
byAraf Karsh Hamid
 
What Is A Docker Container? | Docker Container Tutorial For Beginners| Docker...
bySimplilearn
 
Introduction to kubernetes
byGabriel Carro
 
Service Mesh - Observability
byAraf Karsh Hamid
 
Microservice architecture design principles
bySanjoy Kumar Roy
 

Similar to Microservices, Containers, Kubernetes, Kafka, Kanban

PPSX
Microservices Architecture, Monolith Migration Patterns
byAraf Karsh Hamid
 
PPTX
Microservices why?
bySascha Düpre
 
PDF
Everything you want to know about microservices
byYouness Lasmak
 
PPT
Webinar_102317_Heffner.ppt
byssuserc04f7b
 
PDF
Production-Ready_Microservices_excerpt.pdf
byajcob123
 
PDF
Microservices for Application Modernisation
byAjay Kumar Uppal
 
PDF
20141210 - Microservice Container
byJamie (Taka) Wang
 
PPTX
Architecting for speed: How agile innovators accelerate growth through micros...
byJesper Nordström
 
PPTX
Architecting for speed - how agile innovators accelerate growth through micro...
by3gamma
 
PDF
Overcoming Ongoing Digital Transformational Challenges with a Microservices A...
byCognizant
 
PDF
Building Microservices Software practics
bymuhammed84essa
 
PDF
Migrating from a monolith to microservices – is it worth it?
byKatherine Golovinova
 
PDF
Practical Microservice Architecture (edition 2022).pdf
byAhmed Misbah
 
PDF
Platforms and Microservices - Is There a Middle Ground for Engineers and Tech...
byDialexa
 
PPTX
Webinar : Microservices and Containerization
byNewt Global Consulting LLC
 
PDF
Microservices Interview Questions and Answers PDF By ScholarHat
byScholarhat
 
PPTX
Think Small To Go Big - Introduction To Microservices
byRyan Baxter
 
PPTX
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
byCloud Native Day Tel Aviv
 
PPTX
Microservice intro
byramesh_sharma
 
PPTX
Microservices: Lessons from the Trenches
byGregory Beamer
 
Microservices Architecture, Monolith Migration Patterns
byAraf Karsh Hamid
 
Microservices why?
bySascha Düpre
 
Everything you want to know about microservices
byYouness Lasmak
 
Webinar_102317_Heffner.ppt
byssuserc04f7b
 
Production-Ready_Microservices_excerpt.pdf
byajcob123
 
Microservices for Application Modernisation
byAjay Kumar Uppal
 
20141210 - Microservice Container
byJamie (Taka) Wang
 
Architecting for speed: How agile innovators accelerate growth through micros...
byJesper Nordström
 
Architecting for speed - how agile innovators accelerate growth through micro...
by3gamma
 
Overcoming Ongoing Digital Transformational Challenges with a Microservices A...
byCognizant
 
Building Microservices Software practics
bymuhammed84essa
 
Migrating from a monolith to microservices – is it worth it?
byKatherine Golovinova
 
Practical Microservice Architecture (edition 2022).pdf
byAhmed Misbah
 
Platforms and Microservices - Is There a Middle Ground for Engineers and Tech...
byDialexa
 
Webinar : Microservices and Containerization
byNewt Global Consulting LLC
 
Microservices Interview Questions and Answers PDF By ScholarHat
byScholarhat
 
Think Small To Go Big - Introduction To Microservices
byRyan Baxter
 
Microservices: Why and When? - Alon Fliess, CodeValue - Cloud Native Day Tel ...
byCloud Native Day Tel Aviv
 
Microservice intro
byramesh_sharma
 
Microservices: Lessons from the Trenches
byGregory Beamer
 

More from Araf Karsh Hamid

PPSX
Elastic-Engineering
byAraf Karsh Hamid
 
PPSX
Zero-Trust SASE DevSecOps
byAraf Karsh Hamid
 
PPSX
Cloud Architecture - Multi Cloud, Edge, On-Premise
byAraf Karsh Hamid
 
PPSX
Apache Flink, AWS Kinesis, Analytics
byAraf Karsh Hamid
 
PPTX
Docker Kubernetes Istio
byAraf Karsh Hamid
 
PPSX
Blockchain HyperLedger Fabric Internals - Clavent
byAraf Karsh Hamid
 
PPTX
Blockchain Intro to Hyperledger Fabric
byAraf Karsh Hamid
 
PPSX
Docker Kubernetes Istio
byAraf Karsh Hamid
 
PPTX
Domain Driven Design
byAraf Karsh Hamid
 
PPTX
Microservices Architecture & Testing Strategies
byAraf Karsh Hamid
 
PPTX
Microservices Part 4: Functional Reactive Programming
byAraf Karsh Hamid
 
PPTX
Blockchain Hyper Ledger Fabric : Bangkok Conference
byAraf Karsh Hamid
 
PPTX
Blockchain - HyperLedger Fabric
byAraf Karsh Hamid
 
PDF
Event Storming and Saga
byAraf Karsh Hamid
 
Elastic-Engineering
byAraf Karsh Hamid
 
Zero-Trust SASE DevSecOps
byAraf Karsh Hamid
 
Cloud Architecture - Multi Cloud, Edge, On-Premise
byAraf Karsh Hamid
 
Apache Flink, AWS Kinesis, Analytics
byAraf Karsh Hamid
 
Docker Kubernetes Istio
byAraf Karsh Hamid
 
Blockchain HyperLedger Fabric Internals - Clavent
byAraf Karsh Hamid
 
Blockchain Intro to Hyperledger Fabric
byAraf Karsh Hamid
 
Docker Kubernetes Istio
byAraf Karsh Hamid
 
Domain Driven Design
byAraf Karsh Hamid
 
Microservices Architecture & Testing Strategies
byAraf Karsh Hamid
 
Microservices Part 4: Functional Reactive Programming
byAraf Karsh Hamid
 
Blockchain Hyper Ledger Fabric : Bangkok Conference
byAraf Karsh Hamid
 
Blockchain - HyperLedger Fabric
byAraf Karsh Hamid
 
Event Storming and Saga
byAraf Karsh Hamid
 

Recently uploaded

PDF
Mesh WiFi Router: The Smart Solution for Fast, Seamless, Whole-Home Internet
byAeroMesh Systems
 
PDF
BEP-20 Token on BNB Chain: From Concept to Deployment.pdf
byimoliviabennett
 
PDF
Rustici Software: eLearning standards in the age of AI
byRustici Software
 
PDF
GenerationAI Paris 2025 | How Agentic AI is Reinventing Organizations
byapidays
 
PDF
Making Search Less Taxing: Leveraging Semantics and Keywords in Hybrid Search
byEnterprise Knowledge
 
PDF
Chapter 5 Security Mechanisms iin computer security.pdf
byGetnet Tigabie Askale -(GM)
 
PPTX
Celonis Optimizing your future with process
bydevjeremyappleyard
 
PPTX
Retrieval Augmented Generation- The Synergistic Power of Prompt Engineering
bySemantic SEO BD
 
PDF
TravelTech Paris 2025 | The EU Digital Identity Wallet and it’s impact on Tra...
byapidays
 
PPTX
Tech Trends 2026: AI Agents, Quantum Computing, Robotics & Cybersecurity
byridwansassman
 
PDF
Explaining specific examples of purpose-specific BOM
byakipii ogaoga
 
PDF
Designing a Blog Using Wordpress
bymarkzubi50
 
PDF
Analyze and Preserve Logs - RHCSA (RH134).pdf
byLinuxCert Guru
 
PDF
Transcript: Escape from the Forbidden Zone: Smuggling green and inclusive tec...
byBookNet Canada
 
PPTX
Do You Control the AI, or Does the AI Control You?
bymedhateltoukhy
 
PPTX
Computer architecture, Computer architecture ,Computer architecture
bywaheedqazi123
 
PPTX
apidays Paris 2025 | Integration is Feminist: Building Peace in Distributed S...
byapidays
 
PDF
final.pdf
byomarbishtawi04
 
PDF
Startup Formation Collapses While Acquisition Activity Hits New High!
byMemoori
 
PDF
Configure and Manage Systemd Timers- RHCSA (RH134).pdf
byLinuxCert Guru
 
Mesh WiFi Router: The Smart Solution for Fast, Seamless, Whole-Home Internet
byAeroMesh Systems
 
BEP-20 Token on BNB Chain: From Concept to Deployment.pdf
byimoliviabennett
 
Rustici Software: eLearning standards in the age of AI
byRustici Software
 
GenerationAI Paris 2025 | How Agentic AI is Reinventing Organizations
byapidays
 
Making Search Less Taxing: Leveraging Semantics and Keywords in Hybrid Search
byEnterprise Knowledge
 
Chapter 5 Security Mechanisms iin computer security.pdf
byGetnet Tigabie Askale -(GM)
 
Celonis Optimizing your future with process
bydevjeremyappleyard
 
Retrieval Augmented Generation- The Synergistic Power of Prompt Engineering
bySemantic SEO BD
 
TravelTech Paris 2025 | The EU Digital Identity Wallet and it’s impact on Tra...
byapidays
 
Tech Trends 2026: AI Agents, Quantum Computing, Robotics & Cybersecurity
byridwansassman
 
Explaining specific examples of purpose-specific BOM
byakipii ogaoga
 
Designing a Blog Using Wordpress
bymarkzubi50
 
Analyze and Preserve Logs - RHCSA (RH134).pdf
byLinuxCert Guru
 
Transcript: Escape from the Forbidden Zone: Smuggling green and inclusive tec...
byBookNet Canada
 
Do You Control the AI, or Does the AI Control You?
bymedhateltoukhy
 
Computer architecture, Computer architecture ,Computer architecture
bywaheedqazi123
 
apidays Paris 2025 | Integration is Feminist: Building Peace in Distributed S...
byapidays
 
final.pdf
byomarbishtawi04
 
Startup Formation Collapses While Acquisition Activity Hits New High!
byMemoori
 
Configure and Manage Systemd Timers- RHCSA (RH134).pdf
byLinuxCert Guru
 

Microservices, Containers, Kubernetes, Kafka, Kanban

  • 1.
    ARAF KARSH HAMID Co-Founder/ CTO MetaMagic Global Inc., NJ, USA @arafkarsh arafkarsh Microservices Architecture Containers, Saga, Testing, Kanban World API & DevOps Summit Bangalore, February 15, 2019
  • 2.
    2 Microservices Containers & Kubernetes, Kafka CapabilityCentric Design DDD / ES & CQRS Agile: Kanban Continuous Delivery Testing Strategies Security Best Practices
  • 3.
    MICROSERVICES • CONCEPTS • WHENSHOULD YOU USE MICROSERVICES? • WHAT’S THE RIGHT SIZE? • ARCHITECTURE (INFRATRUCTURE AND SOFTWRE) 2/19/2019 3
  • 4.
    Pioneers in MicroservicesImplementation 19-02-2019 4 New Entrants
  • 5.
    Microservices Characteristics 5 By JamesLewis and Martin Fowler We can scale our operation independently, maintain unparalleled system availability, and introduce new services quickly without the need for massive reconfiguration. — Werner Vogels, CTO, Amazon Web Services Modularity ... is to a technological economy what the division of labor is to a manufacturing one. W. Brian Arthur, author of e Nature of Technology The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be. Alan Kay, 1998 email to the Squeak-dev list Components via Services Organized around Business Capabilities Products NOT Projects Smart Endpoints & Dumb Pipes Decentralized Governance & Data Management Infrastructure Automation Design for Failure Evolutionary Design
  • 6.
    When should Iuse them (Microservices)? 19-02-2019 6 • Strong Module Boundaries: Microservices reinforce modular structure, which is particularly important for larger teams. • Independent Deployment: Simple services are easier to deploy, and since they are autonomous, are less likely to cause system failures when they go wrong. • Technology Diversity: With microservices you can mix multiple languages, development frameworks and data- storage technologies. When you have What’s the Cost Distribution: Distributed systems are harder to program, since remote calls are slow and are always at risk of failure. Eventual Consistency: Maintaining strong consistency is extremely difficult for a distributed system, which means everyone has to manage eventual consistency. Operational Complexity: You need a mature operations team to manage lots of services, which are being redeployed regularly. Source: https://www.martinfowler.com/microservices/
  • 7.
    What is theright size for a Microservice? 19-02-2019 7 • Rather than the size what matters is the Business Function / Domain of the service. • One Microservice may have half a dozen entities and other a couple of dozen entities. What’s more important is the role Microservices plays. • Bounded Context from DDD helps you to decompose a large multi domain Monolith into a Microservice for each Bounded Context. • Focusing on User stories will help you clearly define the boundaries of the Business Domain.
  • 8.
    Microservices Architecture 19-02-2019 8 •API Gateway • Service Discovery • Config Service • Event Bus / Streams • Service Mesh • Hexagonal Architecture • Domain Driven Design • Event Sourcing & CQRS • Functional Reactive Programming Infrastructure Architecture
  • 9.
    19-02-20199 Monolithic vs. MicroservicesExample Traditional Monolithic App using Single Technology Stack Micro Services with Multiple Technology Stack This 3 tier model is obsolete now. Source: Gartner Market Guide for Application Platforms Nov 23, 2016 Event Stream API Gateway (Zuul Edge Server) Load Balancer (Ribbon) Circuit Breaker (Hystrix) Service Discovery (Eureka) Load Balancer (Ribbon) Circuit Breaker (Hystrix) Load Balancer (Ribbon) Circuit Breaker (Hystrix) UI Layer Web Services Business Logic Database Layer Micro Service 4 EE 7 Inventory UI Layer Web Services Business Logic Database Layer Micro Service 1 Customer SE 8 UI Layer Web Services Business Logic Database Layer Micro Service 3 ShoppingCart UI Layer Web Services Business Logic Database Layer Micro Service 2 Order UI Layer WS BL DL Database ShoppingCart Order Customer Inventory
  • 10.
    19-02-2019 10 SOA vs. MicroservicesExample Traditional Monolithic App with SOA Micro Services with Multiple Technology Stack Event Stream UI Layer Web Services Business Logic Database Layer Micro Service 1 Customer SE 8 UI Layer Web Services Business Logic Database Layer Micro Service 3 ShoppingCart UI Layer Web Services Business Logic Database Layer Micro Service 2 Order API Gateway Load Balancer Circuit Breaker Service Discovery Load Balancer Circuit Breaker Load Balancer Circuit Breaker UI Layer Database ShoppingCart Order Customer Inventory Enterprise Service Bus Messaging REST / SOAP HTTP MOM JMS ODBC / JDBC Translation Web Services XML WSDL Addressing Security Registry Management Producers Shared Database Consumers3rd Party Apps Smart Pipes Lot of Business logic resides in the Pipe
  • 11.
    Microservices Deployment Model MicroServices with Multiple Technology Stack Event Stream Users Service Discovery (Eureka) Config Server (Spring) API (Zuul) Gateway UI Layer Web Services Business Logic Database Layer Micro Service 2 ShoppingCart SE 8 LB = Ribbon CB = Hystrix LB = Ribbon CB = Hystrix UI Layer Web Services Business Logic Database Layer Product SE 8 Micro Service 1 With 4 node cluster LB = Ribbon CB = Hystrix UI Layer Web Services Business Logic Database Layer Order SE 8 Micro Service 3 With 2 node Cluster LB = Ribbon CB = Hystrix UI Layer Web Services Business Logic Database Layer Customer Micro Service 4 With 2 node cluster HTTP Server All UI Code is bundled Virtual Private Network 19-02-2019 11
  • 12.
    Pros 1. Adds Complexity 2.Skillset shortage 3. Confusion on getting the right size 4. Team need to manage end-to-end of the Service (From UI to Backend to Running in Production). 19-02-2019 12 1. Robust 2. Scalable 3. Testable (Local) 4. Easy to Change and Replace 5. Easy to Deploy 6. Technology Agnostic Cons Microservices Pros and Cons
  • 13.
    Monolithic > Microservices •FORRESTER RESEARCH • MODERNIZATION JOURNEY • ASSESS AND CLASSIFY YOUR APP PORTFOLIO • PLAN AND PRIORITIZE 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 13
  • 14.
    * For ITServices : They can do one more project of same size with ZERO COST in Platform Licensing (Based on 20 developer pack USD $50K per month for 3 months License cost = $150K) 1419-02-2019
  • 15.
    Modernization Journey 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 15  Startnew features as Microservices Incrementally establish the success early.  Expose Legacy On-Premise Apps API’s If legacy Apps cant be shifted to Cloud  Refactor Monolithic features to Microservices Breakdown and Deploy Feature by Feature  Containerize the Microservice Reduce costs, simplifies the operations and consistent environment between Dev, QA and Production  Monolith De-commission Plan Incrementally sunset the Monolith Velocity as you transform Increase your Delivery Velocity along the Journey High FuturePresent Low
  • 16.
    Assess and Classifyyour App Portfolio 2/19/2019 16  Take inventory of your Apps Classify the Apps based on technology, complexity.  Align Apps to your Business Priorities Identify the Apps that are critical for Modernization.  Identify Business Modernization Requirements Create a Roadmap with faster go to market strategy  Understand the effort and Cost Evaluate all possible Modernization options Container Refactor Expose APIsLift & Shift BusinessValueCostComplexity Product Catalogue Product Review Inventory Shopping Cart Customer Profile Order Management
  • 17.
    Plan and Prioritize 2/19/201917 Complexity Cost Value Score Rank Weightage 35% 25% 40% Product Reviews Med 3 High 5 Med 3 3.50 11 3 Shopping Cart High 5 Med 3 Med 3 3.70 11 4 Customer Med 3 Med 3 Low 1 2.20 7 6 Product Catalogue Med 3 Med 3 High 5 4.80 11 1 Order Very High 7 Med 3 High 5 5.20 15 2 Inventory Very High 7 High 5 Med 3 4.90 15 5  Prioritize Low Priority projects are good test cases but does not bring much business value.  Quick Wins Identify a feature / project which has good business value and low in complexity.  Project Duration Focus on shorter duration projects with high Business Value. Shopping Portal Features Prioritization
  • 18.
    Containers & Kubernetes •DOCKER CONTAINERS • KUBERNETES – CONTAINER ORCHESTRATION • ISTIO – TRAFFIC MANAGEMENT / NETWORK POLICIES 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 18
  • 19.
    Servers / VirtualMachines / Containers Hardware OS BINS / LIB App 1 App 2 App 3 Server Hardware Host OS HYPERVISOR App 1 App 2 App 3 Guest OS BINS / LIB Guest OS BINS / LIB Guest OS BINS / LIB Type 1 Hypervisor Hardware Host OS App 1 App 2 App 3 BINS / LIB BINS / LIB BINS / LIB Container Hardware HYPERVISOR App 1 App 2 App 3 Guest OS BINS / LIB Guest OS BINS / LIB Guest OS BINS / LIB Type 2 Hypervisor19-02-2019 19
  • 20.
    Linux Kernel 20 HOST OS(Ubuntu) Client Docker Daemon Cent OS Alpine Debian HostLinuxKernel Host Kernel Host Kernel Host Kernel All the containers will have the same Host OS Kernel If you require a specific Kernel version then Host Kernel needs to be updated HOST OS (Windows 10) Client Docker Daemon Nano Server Server Core Nano Server WindowsKernel Host Kernel Host Kernel Host Kernel Windows Kernel
  • 21.
    Docker DaemonDocker Client HowDocker works…. $ docker search …. $ docker build …. $ docker container create .. Docker Hub Images Containers $ docker container run .. $ docker container start .. $ docker container stop .. $ docker container ls .. $ docker push …. $ docker swarm .. 19-02-2019 21 21 34 1. Search for the Container 2. Docker Daemon Sends the request to Hub 3. Downloads the image 4. Run the Container from the image
  • 22.
    Kubernetes Key Concepts 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 22 oDeclarative Model Infrastructure as a code o Desired State Minimum state of your App / Microservice o Current State Current State of the due to error or load factor Pods Replicas Deploy Service
  • 23.
    Deployment – Updatesand rollbacks, Canary Release D ReplicaSet – Self Healing, Scalability, Desired State R Worker Node 1 Master Node (Control Plane) Kubernetes Architecture POD POD itself is a Linux Container, Docker container will run inside the POD. PODs with single or multiple containers (Sidecar Pattern) will share Cgroup, Volumes, Namespaces of the POD. (Cgroup / Namespaces) Scheduler Controller Manager Using yaml or json declare the desired state of the app. State is stored in the Cluster store. Self healing is done by Kubernetes using watch loops if the desired state is changed. POD POD POD BE 1.210.1.2.34 BE 1.210.1.2.35 BE 1.210.1.2.36 BE 15.1.2.100 DNS: a.b.com 1.2 Service Pod IP Address is dynamic, communication should be based on Service which will have routable IP and DNS Name. Labels (BE, 1.2) play a critical role in ReplicaSet, Deployment, & Services etc. Cluster Store etcd Key Value Store Pod Pod Pod Label Selector selects pods based on the Labels. Label Selector Label Selector Label Selector Node Controller End Point Controller Deployment Controller Pod Controller …. Labels Internet Firewall K8s Cluster Cloud Controller For the cloud providers to manage nodes, services, routes, volumes etc. Kubelet Node Manager Container Runtime Interface Port 10255 gRPC ProtoBuf Kube-Proxy Network Proxy TCP / UDP Forwarding IPTABLES / IPVS Allows multiple implementation of containers from v1.7 RESTful yaml / json $ kubectl …. Port 443API Server Pod IP ...34 ...35 ...36EP • Declarative Model • Desired State Key Aspects Namespace1Namespace2 • Pods • ReplicaSet • Deployment • Service • Endpoints • StatefulSet • Namespace • Resource Quota • Limit Range • Persistent Volume Kind Secrets Kind • apiVersion: • kind: • metadata: • spec: Declarative Model • Pod • ReplicaSet • Service • Deployment • Virtual Service • Gateway, SE, DR • Policy, MeshPolicy • RbaConfig • Prometheus, Rule, • ListChekcer … @ @ Annotations Names Cluster IP Node Port Load Balancer External Name @ Ingress 19-02-2019 23
  • 24.
    Service Mesh –Sidecar Design Pattern 19-02-2019 24 CB – Circuit Breaker LB – Load Balancer SD – Service Discovery Microservice Process1Process2 Service Mesh Control Plane Service Discovery Routing Rules Control Plane will have all the rules for Routing and Service Discovery. Local Service Mesh will download the rules from the Control pane will have a local copy. Service Discovery Calls Service Mesh Calls Customer Microservice Application Localhost calls http://localhost/api/order/ Router Network Stack LBCB SD ServiceMesh Sidecar UI Layer Web Services Business Logic Order Microservice Application Localhost calls http://localhost/api/payment/ Router Network Stack LBCB SD ServiceMesh Sidecar UI Layer Web Services Business Logic Data Plane
  • 25.
    Shopping Portal /ui /productms /productreview Gateway Virtual Service UIPod UI Pod UI Pod UI Service Product Pod Product Pod Product Pod Product Service Review Pod Review Pod Review Pod Review Service Deployment / Replica / Pod N1 N2 N2 MySQL Pod N4 N3 N1 N4 N3 NodesIstio Sidecar - Envoy Destination Rule Destination Rule Destination Rule Load Balancer Kubernetes Objects Istio Objects Firewall P M CIstio Control Plane UI Pod N5 v1 v2 Stable / v1 Canary v2User X = Canary Others = Stable A / B Testing using Canary Deployment Service Call Kube DNS EndPoints EndPoints EndPoints Internal Load Balancers 25 Source:https://github.com/meta-magic/kubernetes_workshop
  • 26.
    Shopping Portal /ui /productms /productreview Gateway Virtual Service UIPod UI Pod UI Pod UI Service Product Pod Product Pod Product Pod Product Service Review Pod Review Pod Review Pod Review Service Deployment / Replica / Pod N1 N2 N2 MySQL Pod N4 N3 N1 N4 N3 NodesIstio Sidecar - Envoy Destination Rule Destination Rule Destination Rule Load Balancer Kubernetes Objects Istio Objects Firewall P M CIstio Control Plane UI Pod N5 v1 v2 Stable / v1 Canary v210% = Canary 90% = Stable Traffic Shifting Canary Deployment Service Call Kube DNS EndPoints EndPoints EndPoints Internal Load Balancers 26 Source:https://github.com/meta-magic/kubernetes_workshop
  • 27.
    27 Shopping Portal /ui /productms /productreview Gateway Virtual Service UIPod UI Pod UI Pod UI Service Review Pod Review Pod Review Pod Review Service Deployment / Replica / Pod N1 N2 N2 MySQL Pod N4 N3 N1 N4 N3 NodesIstio Sidecar - Envoy Destination Rule Destination Rule Destination Rule Load Balancer Kubernetes Objects Istio Objects Firewall P M CIstio Control Plane v1 Fault Injection Delay = 2 Sec Abort = 10% Fault Injection Product Pod Product Pod Product Pod Product Service Service Call Kube DNS EndPoints EndPoints EndPoints Internal Load Balancers 27 Source:https://github.com/meta-magic/kubernetes_workshop
  • 28.
    Kafka • CONCEPTS :QUEUES / PUB – SUB / EVENT STREAMING • WHY IS IT DIFFERENT FROM TRADITIONAL MESSAGE QUEUES? • DATA STORAGE / CLUSTER / DURABILITY • PERFORMANCE 2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 28
  • 29.
    Kafka Core Concepts 19-02-201929 Publish & Subscribe Read and write streams of data like a messaging system Process Write scalable stream processing apps that react to events in real- time. Store Store streams of data safely in a distributed, replicated, fault tolerant cluster.
  • 30.
    Traditional Queue /Pub-Sub Vs. Kafka 19-02-2019 30 0 1 2 3 4 5 6 7 8 9 8 7 9 Consumer 1 Consumer 2 Consumer 3 Queues Data Data can be partitioned for scalability for parallel processing by same type of consumers Pros: Cons: Queues are not multi subscribers. Once a Consumer reads the data, its gone from the queue. Ordering of records will be lost in asynchronous parallel processing. 0 1 2 3 4 5 6 7 8 9 9 9 9 Consumer 1 Consumer 2 Consumer 3 Pub – Sub Data Multiple subscribers can get the same data.Pros: Scaling is difficult as every message goes to every subscriber. Cons: Kafka generalizes these two concepts.  As with a queue the consumer group allows you to divide up processing over a collection of processes (the members of the consumer group).  As with publish-subscribe, Kafka allows you to broadcast messages to multiple consumer groups.
  • 31.
    Anatomy of aTopic 19-02-2019 31 Source : https://kafka.apache.org/intro • A Topic is a category or feed name to which records are published. • Topics in Kafka are always multi subscriber. • That is, a Topic can have zero, one, or many consumers that subscribe to the data written to it. • Each Partition is an ordered, immutable sequence of records that is continually appended to—a structured commit log. • A Partition is nothing but a directory of Log Files • The records in the partitions are each assigned a sequential id number called the offset that uniquely identifies each record within the partition.
  • 32.
    19-02-2019 32 Partition LogSegment • Partition (Kafka’s Storage unit) is Directory of Log Files. • A partition cannot be split across multiple brokers or even multiple disks • Partitions are split into Segments • Segments are two files: 000.log & 000.index • Segments are named by their base offset. The base offset of a segment is an offset greater than offsets in previous segments and less than or equal to offsets in that segment. • Indexes store offsets relative to its segments base offset • Indexes map each offset to their message position in the log and they are used to look up messages. • Purging of data is based on oldest segment and one segment at a time. 0 1 2 3 4 5 6 7 8 9 Partition Data 6 3 0 Segment 0 Segment 3 Segment 6 9 Segment 9 - Active $ tree kafka-logs | head -n 6 kafka-logs |──── SigmaHawk-2 | |──── 00000000006109871597.index | |──── 00000000006109871597.log | |──── 00000000007306321253.index | |──── 00000000007306321253.log Topic / Partition Segment 1 Segment 2 Rel.Offset, Position Offset, Position, Size, Payload 0000.index 0000.log 0 0 0 0 7 ABCDEFG 1 7 1 7 4 ABCD 2 11 2 11 9 ABCDEFGIJ 4 Bytes 4 Bytes
  • 33.
    19-02-2019 33 Kafka Cluster– Topics & Partitions • The partitions of the log are distributed over the servers in the Kafka cluster with each server handling data and requests for a share of the partitions. Source : https://kafka.apache.org/intro Broker 1 Leader Broker 2 Follower Broker 3 Follower Broker 4 Follower Broker 5 Leader Partition 1 Partition 0 Topic ABC • Each server acts as a leader for some of its partitions and a follower for others so load is well balanced within the cluster. • Each partition has one server which acts as the "leader" and zero or more servers which act as "followers".
  • 34.
    19-02-2019 34 Record Commit Process Broker1 Leader Topic 1 Broker 2 Follower Producer Consumer 2 2 Commit 3 ack • Each partition is replicated across a configurable number of servers for fault tolerance. • The leader handles all read and write requests for the partition while the followers passively replicate the leader. • If the leader fails, one of the followers will automatically become the new leader.1 Message with Offset 4 777743 Broker 3 Follower Data Durability From Kafka v0.8.0 onwards acks Acknowledgement Description 0 If set to zero then the producer will NOT wait for any acknowledgment from the server at all. The record will be immediately added to the socket buffer and considered sent. No guarantee can be made that the server has received the record in this case, and the retries configuration will not take effect (as the client won't generally know of any failures). The offset given back for each record will always be set to -1. 1 This will mean the leader will write the record to its local log but will respond without awaiting full acknowledgement from all followers. In this case should the leader fail immediately after acknowledging the record but before the followers have replicated it then the record will be lost. All / -1 This means the leader will wait for the full set of in-sync replicas to acknowledge the record. This guarantees that the record will not be lost as long as at least one in-sync replica remains alive. This is the strongest available guarantee. This is equivalent to the acks=-1 setting. Source: https://kafka.apache.org/documentation/#topicconfigs acks Steps 0 1 1 1,3 -1 1,2,3 Producer Configuration
  • 35.
    19-02-2019 35 Replication 6 3.2 m1 m2 m3 L(A) m1 m2 F(B) m1 F(C)ISR= (A, B, C) Leader A commits Message m1. Message m2 & m3 not yet committed. 1 m1 m2 F(C) m1 m2 L(B) m1 m2 m3 L(A) ISR = (B,C) A fails and B is the new Leader. B commits m22 m1 m2 m3 L(A) m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 ISR = (B,C) B commits new messages m4 and m5 3 m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 m1 F(A) ISR = (A, B,C) A comes back, restores to last commit and catches up to latest messages. 4 m1 m2 L(B) m4 m5 m1 m2 F(C) m4 m5 m1 m2 F(A) m4 m5 ISR – In-sync Replica • Instead of majority vote, Kafka dynamically maintains a set of in-sync replicas (ISR) that are caught-up to the leader. • Only members of this set are eligible for election as leader. • A write to a Kafka partition is not considered committed until all in-sync replicas have received the write. • This ISR set is persisted to ZooKeeper whenever it changes. Because of this, any replica in the ISR is eligible to be elected leader.
  • 36.
    LinkedIn Kafka Cluster 19-02-201936 Brokers60 Partitions50K Messages / Second800K MB / Second inbound300 MB / Second Outbound1024 The tuning looks fairly aggressive, but all of the brokers in that cluster have a 90% GC pause time of about 21ms, and they’re doing less than 1 young GC per second.
  • 37.
    Uber Kafka Cluster 19-02-201937 Topics10K+ Events / Second11M Petabyte of Data1PB+
  • 38.
    12 Factor AppMethodology 19-02-2019 38 4 Backing Services Treat Backing services like DB, Cache as attached resources 5 Build, Release, Run Separate Build and Run Stages 6 Process Execute App as One or more Stateless Process 7 Port Binding Export Services with Specific Port Binding 8 Concurrency Scale out via the process Model 9 Disposability Maximize robustness with fast startup and graceful exit 10 Dev / Prod Parity Keep Development, Staging and Production as similar as possible 11 Logs Treat logs as Event Streams 12 Admin Process Run Admin Tasks as one of Process Source:https://12factor.net/ Factors Description 1 Codebase One Code base tracked in revision control 2 Dependencies Explicitly declare dependencies 3 Configuration Configuration driven Apps
  • 39.
    CAPABILITY CENTRIC DESIGN •BUSINESS FUNCTIONS • BUSINESS PROCESS • TEAM STRUCTURE 2/19/2019 39
  • 40.
    Business Solution &Business Process 2/19/2019 40  Business Solution focuses the entire Journey of the User which can run across multiple Micro Services.  Business Solution comprises a set of Business Processes.  A specific Micro Service functionality will be focused on a Business Process / Concern Business Solution: Customer Dining Experience Order PaymentFood Menu KitchenDining Browse Menu Order Dinner Dinner Served Get Bill Make Payment
  • 41.
    2/19/2019 41 Capability CentricDesign Vertically sliced Product Team Business Centric Development • Focus on Business Capabilities • Entire team is aligned towards Business Capability. • From Specs to Operations – The team handles the entire spectrum of Software development. • Every vertical will have it’s own Code Pipeline Front-End-Team Back-End-Team Database-Team In a typical Monolithic way the team is divided based on technology / skill set rather than business functions. This leads to not only bottlenecks but also lack of understanding of the Business Domain. QA / QC Team Front-End Back-End Database Business Capability 1 QA/QCTeam Front-End Back-End Database Business Capability 2 QA/QCTeam Front-End Back-End Database Business Capability 3 QA/QCTeam
  • 42.
    2/19/2019 (C)COPYRIGHTMETAMAGICGLOBALINC.,NEWJERSEY,USA 42 Architecture / Design •DOMAIN DRIVEN DESIGN • EVENT SOURCING AND CQRS • FUNCTIONAL REACTIVE PROGRAMMING • DISTRIBUTED TRANSACTIONS • CASE STUDY
  • 43.
    2/19/2019 43 Ubiquitous Language Vocabulary sharedby all involved parties Used in all forms of spoken / written communication Ubiquitous Language Domain Expert Analyst Developers QA Design Docs Test Cases Code Restaurant Context – Food Item : Eg. Food Item (Navrathnakurma) can have different meaning or properties depends on the context. • In the Menu Context it’s a Veg Dish. • In the Kitchen Context it’s is recipe. • And in the Dining Context it will have more info related to user feed back etc. DDD: Ubiquitous Language: Strategic Design As an Restaurant Owner I want to know who my Customers are So that I can serve them better Role-Feature-Reason Matrix BDD – Behavior Driven Development Given Customer John Doe exists When Customer orders food Then Assign customer preferences as Veg or Non Veg customer BDD Construct
  • 44.
    2/19/2019 44 Domain DrivenDesign – Tactical Design Source: Domain-Driven Design Reference by Eric Evans
  • 45.
    2/19/2019 45 Shopping PortalModules – Code Packaging Auth Products Cart OrderCustomer Domain Layer • Models • Repo • Services • Factories Adapters • Repo • Services • Web Services Domain Layer • Models • Repo • Services • Factories Adapters • Repo • Services • Web Services Domain Layer • Models • Repo • Services • Factories Adapters • Repo • Services • Web Services Packaging Structure Bounded Context Implementation (Repositories, Business Services, Web Services) Domain Models (Entities, Value Objects, DTOs) (Repositories, Business Services, Web Services) Entity Factories Interfaces (Ports)
  • 46.
    2/19/2019 46 DDD: UseCase Order Module Models Value Object • Currency • Item Value • Order Status • Payment Type • Record State • Audit Log Entity • Order (Aggregate Root) • Order Item • Shipping Address • Payment DTO • Order • Order Item • Shipping Address • Payment Domain Layer Adapters • Order Repository • Order Service • Order Web Service • Order Query Web Service • Shipping Address Web Service • Payment Web Service Adapters Consists of Actual Implementation of the Ports like Database Access, Web Services API etc. Converters are used to convert an Enum value to a proper Integer value in the Database. For Example Order Status Complete is mapped to integer value 100 in the database. Services / Ports • Order Repository • Order Service • Order Web Service Utils • Order Factory • Order Status Converter • Record State Converter • Order Query Web Service • Shipping Address Web Service • Payment Web Service Shopping Portal
  • 47.
    2/19/2019 47 Event SourcingIntro Standard CRUD Operations – Customer Profile – Aggregate Root Profile Created Title Updated New Address added Derived Notes Removed Time T1 T2 T4T3 Event Sourcing and Derived Aggregate Root Commands 1. Create Profile 2. Update Title 3. Add Address 4. Delete Notes 2 Events 1. Profile Created Event 2. Title Updated Event 3. Address Added Event 4. Notes Deleted Event 3 Current State of the Customer Profile 4 Event store Single Source of Truth Greg Young
  • 48.
    2/19/2019 48 Functional ReactiveProgramming: 4 Building Blocks of RxJava Source of Data Stream [ Sender ]Observable1 Listens for emitted values [ Receiver ]Observer2 Source: http://reactivex.io/ Schedulers3 Schedulers are used to manage and control concurrency. 1. observeOn: Thread Observable is executed 2. subscribeOn: Thread subscribe is executed 4 Operators Content Filtering Time Filtering Transformation Operators that let you Transform, Combine, Manipulate, and work with the sequence of items emitted by Observables
  • 49.
    2/19/2019 49 Comparison :Iterable / Streams / Observable 1Building Block First Class Visitor (Consumer) Serial Operations Parallel Streams (10x Speed) Still On Next, On Complete and On Error are Serial Operations Completely Asynchronous Operations Java 8 – Blocking CallJava 6 – Blocking Call Rx Java - Freedom Source Code: https://github.com/meta-magic/rxjava
  • 50.
    2/19/2019 50 Rx 2Java Operator : Filter / Sort / FlatMap 4Building Block Objective: toSortedList() returns an Observable with a single List containing Fruits. Using FlatMap to Transform Observable <List> to Observable <Fruit> Rx Example 2 SourceCodeGitHub:https://github.com/meta-magic/Rx-Java-2 • Merge • Filter • Sort • Take
  • 51.
    2/19/2019 51 Distributed Tx:SAGA Design Pattern instead of 2PC Long Lived Transactions (LLTs) hold on to DB resources for relatively long periods of time, significantly delaying the termination of shorter and more common transactions. Source: SAGAS (1987) Hector Garcia Molina / Kenneth Salem, Dept. of Computer Science, Princeton University, NJ, USA T1 T2 Tn Local Transactions C1 C2 Cn-1 Compensating Transaction Divide long–lived, distributed transactions into quick local ones with compensating actions for recovery. Travel : Flight Ticket & Hotel Booking Example BASE (Basic Availability, Soft State, Eventual Consistency) Room ReservedT1 Room PaymentT2 Seat ReservedT3 Ticket PaymentT4 Cancelled Room Reservation C1 Cancelled Room Payment C2 Cancelled Ticket Reservation C3
  • 52.
    2/19/2019 52 Case Study:Restaurant Dining – Event Sourcing and CQRS Order Payment • Add Drinks • Add Food • Update Food Commands • Open Table • Add Juice • Add Soda • Add Appetizer 1 • Add Appetizer 2 • Serve Drinks • Prepare Food • Serve Food Events • Drinks Added • Food Added • Food Updated • Food Discontinued • Table Opened • Juice Added • Soda Added • Appetizer 1 Added • Appetizer 2 Added • Juice Served • Soda Served • Appetizer Served • Food Prepared • Food Served • Prepare Bill • Process Payment • Bill Prepared • Payment Processed • Payment Approved • Payment Declined • Cash Paid When people arrive at the Restaurant and take a table, a Table is opened. They may then order drinks and food. Drinks are served immediately by the table staff, however food must be cooked by a chef. Once the chef prepared the food it can then be served. Table is closed then the bill is prepared. Microservices • Dinning Order • Billable Order Customer Journey thru Dinning Processes Processes Food Menu KitchenDining • Remove Soda • Add Food 1 • Add Food 2 • Place Order • Close Table • Remove Soda • Food 1 Added • Food 2 Added • Order Placed • Table Closed ES Aggregate 3 2 4 1
  • 53.
    2/19/2019 53 RESTful APIs •STANDARDS • API VERSIONING STANDARDS
  • 54.
    2/19/2019 54 RESTful Guidelines 1.Endpoints as nouns, NOT verbs Ex. /catalogues /orders /catalogues/products and NOT /getProducts/ /updateProducts/ 2. Use plurals Ex. /catalogues/{catalogueId} and NOT /catalogue/{catalogueId} 3. Documenting 4. Paging 5. Use SSL 6. HTTP Methods GET / POST / PUT / DELETE / OPTIONS / HEAD 7. HTTP Status Codes (Effective usage) 8. Versioning Media Type Version GET /account/5555 HTTP/1.1 Accept: application/vnd.catalogues.v1+json URL path version https://domain/v1/catalogues/products
  • 55.
    2/19/2019 55 RESTful Guidelines– Query Examples Search All Products Search Products By Catalogue ID Search Products By Catalogue ID & Product ID
  • 56.
    2/19/2019 56 RESTful Guidelines– Query Examples Two different implementation of same query
  • 57.
    2/19/2019 57 # Name* Who Uses Pros Cons 1 Media Type Versioning Accept: Application/vnd.api.article+xml; version=1.0 Med GitHub • Version Directly @ resource level • Preserve URI • Close to RESTful Specs • Harder to Test • Distort HTTP Headers purpose • Tools required for testing 2 Custom Headers Versioning X-API-Version: 2. Med Microsoft • Preservers URI • Harder to Test • Tools required for testing 3 URI Versioning api.example.com/v1/resource High Google Twitter Amazon • Most common method • Versions can be explored using Browser • Easy to use • Disrupts RESTful Compliance. URI should represent resource and not versions 4 Domain Versioning apiv1.example.com/resource Low Facebook • Same as are URI Versioning • Same as URI Versioning 5 Request Parameter Versioning GET /something/?version=0.1 High Pivotal NetFlix • Similar to URI versioning • It can get messy 6 Date Versioning First request saves the date. Low Clearbit • New APIs can be shipped without changing the end points • Complex to implement • Traceability is difficult. API Versioning
  • 58.
    2/19/2019 58 Testing Strategies •UNIT TESTING • COMPONENT TESTING • INTEGRATION CONTRACT TESTING • INTEGRATION TESTING
  • 59.
    2/19/2019 59 Microservices TestingStrategies Ubiquitous Language Domain Expert Analyst Developers QA Design Docs Test Cases Code E2E Testing Integration Testing Contract Testing Component Testing Unit Testing Number of Tests Speed Cost Time Mike Cohen’s Testing Pyramid Test Pyramid: https://martinfowler.com/bliki/TestPyramid.html 70% 20% 10%
  • 60.
    2/19/2019 60 Microservices TestingStrategy Unit Testing A unit test exercises the smallest piece of testable software in the application to determine whether it behaves as expected. Source: https://martinfowler.com/articles/microservice-testing/#agenda Component Testing A component test limits the scope of the exercised software to a portion of the system under test, manipulating the system through internal code interfaces and using test doubles to isolate the code under test from other components. Integration Testing An integration test verifies the communication paths and interactions between components to detect interface defects Integration Contract Testing An Integration Contract test is a test at the boundary of an external service verifying that it meets the contract expected by a consuming service. End 2 End Testing An end-to-end test verifies that a system meets external requirements and achieves its goals, testing the entire system, from end to end Say NO to End 2 End Tests - Mike Walker April 22, 2015. Google Test Blog
  • 61.
    2/19/2019 61 Testing ToolsMicroservicesTesting Scenarios / Tools Contract Testing Scope Integration Testing Verifies the communication paths and interactions between components to detect interface defects Contract Testing It is a test at the boundary of an external service verifying that it meets the contract expected by a consuming service. Payment Mock IntegrationContractTestingScope Test Double Montebank Cart Component Testing Unit Testing IntegrationTestingScope Order REST / HTTP or Events / Kafka Item ID, Quantity, Address.. Mock Order Component Testing A component test limits the scope of the exercised software to a portion of the system under test. Order Payment Unit Testing Firewall Integration Testing Scope REST / HTTP Payment Sandbox Component Testing
  • 62.
    2/19/2019 62 Security BestPractices • DOCKER • KUBERNETES
  • 63.
    2/19/2019 63 Build SmallContainer Images • Simple Java Web Apps with Ubuntu & Tomcat can have a size of 700 MB • Use Alpine Image as your base Linux OS • Alpine images are 10x smaller than base Ubuntu images • Smaller Image size reduce the Container vulnerabilities. • Ensure that only Runtime Environments are there in your container. For Example your Alpine + Java + Tomcat image should contain only the JRE and NOT JDK. • Log the App output to Container Std out and Std error. 1
  • 64.
    2/19/2019 64 Docker: ToRoot or Not to Root! • Create Multiple layers of Images • Create a User account • Add Runtime software’s based on the User Account. • Run the App under the user account • This gives added security to the container. • Add Security module SELinux or AppArmour to increase the security, Alpine JRE 8 Tomcat 8 My App 1 2
  • 65.
    2/19/2019 65 Docker: ContainerSecurity 1. Secure your HOST OS! Containers runs on Host Kernel. 2. No Runtime software downloads inside the container. Declare the software requirements at the build time itself. 3. Download Docker base images from Authentic site. 4. Limit the resource utilization using Container orchestrators like Kubernetes. 5. Don’t run anything on Super privileged mode. 3
  • 66.
    2/19/2019 66 Kubernetes: NakedPods • Never use a Naked Pod, that is Pod without any ReplicaSet or Deployments. Naked pods will never get re-scheduled if the Pod goes down. • Never access a Pod directly from another Pod. Always use a Service to access a Pod. • User labels to select the pods { app: myapp, tier: frontend, phase: test, deployment: v3 }. • Never use :latest tag in the image in the production scenario. 4
  • 67.
    2/19/2019 67 Kubernetes: Namespace default Kubesystem Kube public Kubernetes Cluster• Group your Services / Pods / Traffic Rules based on Specific Namespace. • This helps you apply specific Network Policies for that Namespace with increase in Security and Performance. • Handle specific Resource Allocations for a Namespace. • If you have more than a dozen Microservices then it’s time to bring in Namespaces. Service-Name.Namespace.svc.cluster.local $ kubectl config set-context $(kubectl config current-context) --namespace=your-ns The above command will let you switch the namespace to your namespace (your-ns). 5
  • 68.
    2/19/2019 68 Kubernetes: PodHealth Check • Pod Health check is critical to increase the overall resiliency of the network. • Readiness • Liveness • Ensure that all your Pods have Readiness and Liveness Probes. • Choose the Protocol wisely (HTTP, Command & TCP) 6
  • 69.
    2/19/2019 69 Kubernetes: ResourceUtilization • For the Best Quality define the requests and limits for your Pods. • You can set specific resource requests for a Dev Namespace to ensure that developers don’t create pods with a very large resource or a very small resource. • Limit Range can be set to ensure that containers were create with too low resource or too large resource. 7
  • 70.
    2/19/2019 70 Kubernetes: PodTermination Lifecycle • Make sure that the Application to Handle SIGTERM message. • You can use preStop Hook • Set the terminationGracePeriodSeconds: 60 • Ensure that you clean up the connections or any other artefacts and ready for clean shutdown of the App (Microservice). • If the Container is still running after the grace period, Kubernetes sends a SIGKILL event to shutdown the Pod. 8
  • 71.
    2/19/2019 71 Kubernetes: ExternalServices • There are systems that can be outside the Kubernetes cluster like • Databases or • external services in the cloud. • You can create an Endpoint with Specific IP Address and Port with the same name as Service. • You can create a Service with an External Name (URL) which does a CNAME redirection at the Kernel level. 9
  • 72.
    2/19/2019 72 Kubernetes: UpgradeCluster • Make sure that the Master behind a Load Balancer. • Upgrade Master • Scale up the Node with an extra Node • Drain the Node and • Upgrade Node • Cluster will be running even if the master is not working. Only Kubectl and any master specific functions will be down until the master is up. 10
  • 73.
    2/19/2019 73 Kanban • BOARDSAND CARDS • KANBAN VS SCRUM • BENEFITS OF KANBAN
  • 74.
    2/19/2019 74 What isKanban Kanban is a method for managing the creation of products with an emphasis on • continual delivery while • not overburdening the development team. Like Scrum, Kanban is a process designed to help teams work together more effectively. Kanban is a visual management method that was developed by Hirotaka Takeuchi and Ikujiro Nonaka
  • 75.
    2/19/2019 75 Three Principlesof Kanban Source: https://resources.collab.net/agile-101/what-is-kanban • Visualize what you do today (workflow): seeing all the items in context of each other can be very informative • Limit the amount of work in progress (WIP): this helps balance the flow-based approach so teams don’t start and commit to too much work at once • Enhance flow: when something is finished, the next highest thing from the backlog is pulled into play
  • 76.
    2/19/2019 76 Kanban ReleaseCycles Preparation Requirements Design Development Testing Release 1 – 4 Weeks Cycle
  • 77.
  • 78.
    2/19/2019 78 Kanban vs.Scrum Kanban Scrum Roles & Responsibilities No prescribed roles Pre-defined roles of Scrum master, Product owner and team member Delivery Time Lines Continuous Delivery Time boxed sprints Delegation & Prioritization Work is pulled through the system (single piece flow) Work is pulled through the system in batches (the sprint backlog) Modifications Changes can be made at any time No changes allowed mid-sprint Measurement of Productivity Cycle time Velocity When to Use? More appropriate in operational environments with a high degree of variability in priority More appropriate in situations where work can be prioritized in batches that can be left alone Source: https://leankit.com/learn/kanban/kanban-vs-scrum/
  • 79.
    2/19/2019 79 Benefits ofKanban • Shorter cycle times can deliver features faster. • Responsiveness to Change: • When priorities change very frequently, Kanban is ideal. • Balancing demand against throughput guarantees that most the customer-centric features are always being worked. • Requires fewer organization / room set-up changes to get started • Reducing waste and removing activities that don’t add value to the team/department/organization • Rapid feedback loops improve the chances of more motivated, empowered and higher-performing team members
  • 80.
    2/19/2019 80 Design Patternsare solutions to general problems that software developers faced during software development. Design Patterns
  • 81.
    2/19/2019 81 Araf KarshHamid : Co-Founder / CTO araf.karsh@metamagic.in USA: +1 (973) 969-2921 India: +91.999.545.8627 Skype / LinkedIn / Twitter / Slideshare : arafkarsh http://www.slideshare.net/arafkarsh https://www.linkedin.com/in/arafkarsh/ http://www.arafkarsh.com/ DREAM | AUTOMATE | EMPOWER
  • 82.
    2/19/2019 82 UI DesignPatterns MVC/MVP/ MVVM View Controller Model Passes calls To Fire Events Manipulates • The Controller is responsible to process incoming requests. It receives input from users via the View, then process the user's data with the help of Model and passing the results back to the View. • Typically, it acts as the coordinator between the View and the Model. Model View Controller 1 * • The View Model is responsible for exposing methods, commands, and other properties that helps to maintain the state of the view, manipulate the model as the result of actions on the view, and trigger events in the view itself. • There is many-to-one relationship between View and ViewModel means many View can be mapped to one ViewModel. • Supports two-way data binding between View and ViewModel. View ViewModel Model Passes calls To Manipulates Updates Fire Events Model View ViewModel • The Presenter is responsible for handling all UI events on behalf of the view. This receive input from users via the View, then process the user's data with the help of Model and passing the results back to the View. • Unlike view and controller, view and presenter are completely decoupled from each other’s and communicate to each other’s by an interface. Also, presenter does not manage the incoming request traffic as controller. • Supports two-way data binding. Model View Presenter View Presenter Model Passes calls To Fire Events Manipulates Updates1 1
  • 83.
    2/19/2019 83 UI DesignPatterns Flux / Redux ViewDispatcher Every action is sent to all Stores via callbacks the stores register with the Dispatcher Store Action Action 1 * Controller-Views • Listens to Store changes • Emit Actions to Dispatcher Dispatcher • Single Dispatcher per Application • Manages the Data Flow View to Model • Receives Actions and dispatch them to Stores Stores • Contains state for a Domain (Vs. Specific Component) • In Charge of modifying the Data • Inform the views when the Data is changed by emitting the Changed Event. Flux Core Concepts 1. One way Data Flow 2. No Event Chaining 3. Entire App State is resolved in store before Views Update 4. Data Manipulation ONLY happen in one place (Store). Actions • Simple JS Objects • Contains Name of the Action and Data (Payload) • Action represent something that has happened. • Has No Business Logic
  • 84.
    2/19/2019 84 UI DesignPatterns Redux Actions • Simple JS Objects • Contains Name of the Action and Data (Payload) • Has NO Business Logic • Action represent something that has happened. Store • Multiple View layers can Subscribe • View layer to Dispatch actions • Single Store for the Entire Application • Data manipulation logic moves out of store to Reducers Reducer • Pure JS Functions • No External calls • Can combine multiple reducers • A function that specifies how the state changes in response to an Action. • Reducer does NOT modify the state. It returns the NEW State. Redux Core Concepts 1. One way Data Flow 2. No Dispatcher compared to Flux 3. Immutable Store Available for React & Angular View Action State Dispatcher Reducer R R R Store Middleware Middleware Middleware • Handles External calls • Multiple Middleware's can be chained.

Editor's Notes

  • #10 https://dzone.com/articles/microservices-vs-soa-2
  • #11 https://dzone.com/articles/microservices-vs-soa-2
  • #15 https://www.forrester.com/report/Application+Modernization+Service+By+Microservice/-/E-RES122550 https://www.forrester.com/report/Microservices+And+External+APIs+Underpin+Digital+Business/-/E-RES137951 e(l(44)/3) = 3.5
  • #18 Analytics = Score = 3 + 5 + 3 = 11 WS = 1.05 + 1.25 + 1.2 = 3.50 Shopping Cart Score = 5 + 3 + 3 = 11 WS = 1.75 + 0.75 + 1.2 = 3.70 Customer Score = 3 + 3 + 1 = 7 WS = 1.05 + 0.75 + 0.40 = 2.20 Catalogue Score = 3 + 3 + 5 = 11 WS = 1.05 + 0.75 + 2 = 4.80 Order Score = 7 + 3 + 5 = 15 WS = 2.45 + 0.75 + 2 = 5.20 Inventory Score = 7 + 5 + 3 = 15 WS = 2.45 +
  • #25 https://buoyant.io/2017/04/25/whats-a-service-mesh-and-why-do-i-need-one/
  • #33 The index file is made up of 8 byte entries, 4 bytes to store the offset relative to the base offset and 4 bytes to store the position. The offset is relative to the base offset so that only 4 bytes is needed to store the offset. For example: let’s say the base offset is 10000000000000000000, rather than having to store subsequent offsets 10000000000000000001 and 10000000000000000002 they are just 1 and 2. Kafka wraps compressed messages together Producers sending compressed messages will compress the batch together and send it as the payload of a wrapped message. And as before, the data on disk is exactly the same as what the broker receives from the producer over the network and sends to its consumers. https://thehoard.blog/how-kafkas-storage-internals-work-3a29b02e026
  • #35 Durability - the ability to withstand wear, pressure, or damage.
  • #37 https://docs.confluent.io/current/kafka/deployment.html
  • #38 https://docs.confluent.io/current/kafka/deployment.html