SlideShare a Scribd company logo
Microservices:
Drinking the Kool-Aid
Matt West Kyle McIntyre
1
Review of Microservice Architectures
This is everything you need to know.
2
3
Review of Microservice Architectures
Microservice Characteristics
● Like SOA but “smaller”
● Many small services
● Versioned independently
● Deployed independently
● Scale independently
Possible Microservice Benefits
● Resilience (through clustered services)
● Zero downtime
● Horizontal scaling
● Independent scaling of components
● Ease of making changes
● Ease of uptaking new technology
● Canary testing
● Independent release cycles
“Building Microservices” - Sam Newman
4
The taste of the kool-aid (part 1)
● It’s all about the contract! That’s not exactly revolutionary
● Microservices mess with everything. Including testing strategies.
● Hard things become easier (releasing changes, scaling)
● Easy things become hard (e.g. multi-lingual support, referential integrity checks)
● Service boundaries don’t necessarily follow the data model.
● Some ‘scary’ stuff like a repository/db per service really isn’t a big deal. Seriously.
5
The taste of the kool-aid (part 2)
● Communication and state exchange between microservices gets complicated (more to follow)
● Zero-downtime upgrades: Not actually that bad once you get the hang of it
● Horizontal scalability: Not a simple yes or no question but a spectrum. Also, not free.
● Canaries are essential for both quality and velocity (more to follow)
● Polyglot stack: More compelling at the database level than at the language/runtime level
● As a developer, you’re empowered. With power comes responsibility
6
Deep-Dive #1
Continuous Deployment and Canary Testing….
7
“Monolithic” Deployment
OperationsDevelopment
Customers
Months
Support
8
“Monolithic” Deployment (Hotfix)
OperationsDevelopment
Customers
Minutes
Support
9
Continuous Deployment
OperationsDevelopment
Customers
Days/Hours
Support
10
Continuous Deployment - ala Microservices
Customers
Days/Hours
11
Continuous Deployment - ala Microservices
Customers
Days/Hours
12
“Monolithic” Deployment
OperationsDevelopment
Customers
Months
Support
13
Guiding Principles
1. You can’t “contrive” production in a
laboratory
○ It’s probably impossible or at a minimum
not worth the cost
2. Reality is our friend
○ Time doesn’t make everything “stronger”
○ Are devs thinking about production?
3. Build “crisis” reflexes
○ Do everyday what you will do in a crisis
14
How?
Pop Quiz Hotshot:
You MUST push a change out to
production in the next hour - what do
you do?
15
Continuous Deployment - ala Microservices
Customers
Days/Hours
16
Canary Testing
?
17
● Code level
○ Easy to do
○ Basically some “if checks” and a global registry per tenant or %
○ Larger blast radius
● Binary Level
○ Harder to do
○ Utilize service routing techniques (per tenant/% based)
○ Smaller blast radius
● Example (Chat change)
○ Reverse works on this as well
Canary Testing -- It’s not that hard
18
Guiding Principles Played Out @ Quiq
1. You can’t “contrive” production in a
laboratory
○ It’s probably impossible or at a minimum
not worth the cost
2. Reality is our friend
○ Time doesn’t make everything “stronger”
○ Are devs thinking about production?
3. Build “crisis” reflexes
○ Do everyday what you will do in a crisis
19
Deep-Dive #2 - Microservice Communication
20
21
More Protocols, More Problems
● Achieving the benefits of microservices is non-trivial
● Using both HTTP & AMQP made each more daunting
○ HTTP necessitates service discovery
○ Canary testing - must cutover & direct two traffic types
○ Resilience - Need clustered/redundant AMQP
○ Horizontal scalability - AMQP server a possible bottleneck
● AMQP worked directly against some of the hallmarks
○ Becomes a point of centralization, like a shared database
○ Polyglot stack - Not as ubiquitous as HTTP
But here’s the worst part…
We were actually afraid to horizontally scale some of our services!
22
1. What if Downstream2 finishes first?
a. Even DB locking does not help
because the race condition is in
the queue itself
2. The more instances of Downstream1,
the more likely the corruption
3. We need an AMQP extension, TCP-
style message ordering, or merge
strategies, or...
User A Offline
User A Online
Downstream2Downstream1
SessionService1
23
Knock, knock: race condition (who’s there?)
Our Solution
● All inter-service communication over HTTP
● Store the transactionally ordered side effects in the same database that mutated records
during the transaction (e.g. in redis)
● Ordered executor in (e.g. SessionService) publishes side effect via HTTP
○ Ordered executor leverages locking mechanisms of host service’s primary DB
● Downstream1 receives side effect, pushes it into its own queue, kicks off a future &
immediately returns (still async!)
● Now we have ordered, durable downstream state updates and/or side effects!
24
25
Some Observations
1. Achieving the benefits of microservices is challenging. Do yourself a favor and limit
the number of communication protocols
2. Don’t use producer/consumer queues for general communication
3. In the event of problems like process death, always err on the side of sending the
same message twice instead of dropping it. Idempotency is king.
26
Questions?
27

More Related Content

What's hot

Pub/Sub Messaging
Pub/Sub MessagingPub/Sub Messaging
Pub/Sub Messaging
Peter Hanzlik
 
Deep Into Isolation Levels
Deep Into Isolation LevelsDeep Into Isolation Levels
Deep Into Isolation Levels
Boris Hristov
 
KubeCon + CloudNative Con NA 2021 | A New Generation of NATS
KubeCon + CloudNative Con NA 2021 | A New Generation of NATSKubeCon + CloudNative Con NA 2021 | A New Generation of NATS
KubeCon + CloudNative Con NA 2021 | A New Generation of NATS
NATS
 
“Microservices: how to avoid making the same mistake twice.”
“Microservices: how to avoid making the same mistake twice.”“Microservices: how to avoid making the same mistake twice.”
“Microservices: how to avoid making the same mistake twice.”
HYS Enterprise
 
Microservices Meetup San Francisco - August 2017 Talk on NATS
Microservices Meetup San Francisco - August 2017 Talk on NATSMicroservices Meetup San Francisco - August 2017 Talk on NATS
Microservices Meetup San Francisco - August 2017 Talk on NATS
NATS
 
Taking Your Enterprise to the Next Level with WSO2 Message Broker and WSO2 En...
Taking Your Enterprise to the Next Level with WSO2 Message Broker and WSO2 En...Taking Your Enterprise to the Next Level with WSO2 Message Broker and WSO2 En...
Taking Your Enterprise to the Next Level with WSO2 Message Broker and WSO2 En...
WSO2
 
Simple Solutions for Complex Problems
Simple Solutions for Complex Problems Simple Solutions for Complex Problems
Simple Solutions for Complex Problems
Apcera
 
Implementing Microservices with NATS
Implementing Microservices with NATSImplementing Microservices with NATS
Implementing Microservices with NATS
Apcera
 
NATS: Simple, Secure and Scalable Messaging For the Cloud Native Era
NATS: Simple, Secure and Scalable Messaging For the Cloud Native EraNATS: Simple, Secure and Scalable Messaging For the Cloud Native Era
NATS: Simple, Secure and Scalable Messaging For the Cloud Native Era
wallyqs
 
NATS + Docker meetup talk Oct - 2016
NATS + Docker meetup talk Oct - 2016NATS + Docker meetup talk Oct - 2016
NATS + Docker meetup talk Oct - 2016
wallyqs
 
Deep Dive into Building a Secure & Multi-tenant SaaS Solution with NATS
Deep Dive into Building a Secure & Multi-tenant SaaS Solution with NATSDeep Dive into Building a Secure & Multi-tenant SaaS Solution with NATS
Deep Dive into Building a Secure & Multi-tenant SaaS Solution with NATS
NATS
 
How Clarifai uses NATS and Kubernetes for Machine Learning
How Clarifai uses NATS and Kubernetes for Machine LearningHow Clarifai uses NATS and Kubernetes for Machine Learning
How Clarifai uses NATS and Kubernetes for Machine Learning
Apcera
 
Improved Business Availability and Scalability with WSO2 ESB 4.9’s New Coordi...
Improved Business Availability and Scalability with WSO2 ESB 4.9’s New Coordi...Improved Business Availability and Scalability with WSO2 ESB 4.9’s New Coordi...
Improved Business Availability and Scalability with WSO2 ESB 4.9’s New Coordi...
WSO2
 

What's hot (13)

Pub/Sub Messaging
Pub/Sub MessagingPub/Sub Messaging
Pub/Sub Messaging
 
Deep Into Isolation Levels
Deep Into Isolation LevelsDeep Into Isolation Levels
Deep Into Isolation Levels
 
KubeCon + CloudNative Con NA 2021 | A New Generation of NATS
KubeCon + CloudNative Con NA 2021 | A New Generation of NATSKubeCon + CloudNative Con NA 2021 | A New Generation of NATS
KubeCon + CloudNative Con NA 2021 | A New Generation of NATS
 
“Microservices: how to avoid making the same mistake twice.”
“Microservices: how to avoid making the same mistake twice.”“Microservices: how to avoid making the same mistake twice.”
“Microservices: how to avoid making the same mistake twice.”
 
Microservices Meetup San Francisco - August 2017 Talk on NATS
Microservices Meetup San Francisco - August 2017 Talk on NATSMicroservices Meetup San Francisco - August 2017 Talk on NATS
Microservices Meetup San Francisco - August 2017 Talk on NATS
 
Taking Your Enterprise to the Next Level with WSO2 Message Broker and WSO2 En...
Taking Your Enterprise to the Next Level with WSO2 Message Broker and WSO2 En...Taking Your Enterprise to the Next Level with WSO2 Message Broker and WSO2 En...
Taking Your Enterprise to the Next Level with WSO2 Message Broker and WSO2 En...
 
Simple Solutions for Complex Problems
Simple Solutions for Complex Problems Simple Solutions for Complex Problems
Simple Solutions for Complex Problems
 
Implementing Microservices with NATS
Implementing Microservices with NATSImplementing Microservices with NATS
Implementing Microservices with NATS
 
NATS: Simple, Secure and Scalable Messaging For the Cloud Native Era
NATS: Simple, Secure and Scalable Messaging For the Cloud Native EraNATS: Simple, Secure and Scalable Messaging For the Cloud Native Era
NATS: Simple, Secure and Scalable Messaging For the Cloud Native Era
 
NATS + Docker meetup talk Oct - 2016
NATS + Docker meetup talk Oct - 2016NATS + Docker meetup talk Oct - 2016
NATS + Docker meetup talk Oct - 2016
 
Deep Dive into Building a Secure & Multi-tenant SaaS Solution with NATS
Deep Dive into Building a Secure & Multi-tenant SaaS Solution with NATSDeep Dive into Building a Secure & Multi-tenant SaaS Solution with NATS
Deep Dive into Building a Secure & Multi-tenant SaaS Solution with NATS
 
How Clarifai uses NATS and Kubernetes for Machine Learning
How Clarifai uses NATS and Kubernetes for Machine LearningHow Clarifai uses NATS and Kubernetes for Machine Learning
How Clarifai uses NATS and Kubernetes for Machine Learning
 
Improved Business Availability and Scalability with WSO2 ESB 4.9’s New Coordi...
Improved Business Availability and Scalability with WSO2 ESB 4.9’s New Coordi...Improved Business Availability and Scalability with WSO2 ESB 4.9’s New Coordi...
Improved Business Availability and Scalability with WSO2 ESB 4.9’s New Coordi...
 

Similar to 2 years into drinking the Microservice kool-aid (Fact and Fiction)

Monolithic to Microservices Architecture - STM 6
Monolithic to Microservices Architecture - STM 6Monolithic to Microservices Architecture - STM 6
Monolithic to Microservices Architecture - STM 6
Tricode (part of Dept)
 
Breaking down a monolith
Breaking down a monolithBreaking down a monolith
Breaking down a monolith
GeekNightHyderabad
 
Microservices
MicroservicesMicroservices
Microservices
NewsCred Dhaka
 
Ledingkart Meetup #1: Monolithic to microservices in action
Ledingkart Meetup #1: Monolithic to microservices in actionLedingkart Meetup #1: Monolithic to microservices in action
Ledingkart Meetup #1: Monolithic to microservices in action
Mukesh Singh
 
Introduction to Microservices Patterns
Introduction to Microservices PatternsIntroduction to Microservices Patterns
Introduction to Microservices Patterns
Dimosthenis Botsaris
 
Introduction to Microservices Patterns
Introduction to Microservices PatternsIntroduction to Microservices Patterns
Introduction to Microservices Patterns
arconsis
 
Introduction to Microservices Architecture - SECCOMP 2020
Introduction to Microservices Architecture - SECCOMP 2020Introduction to Microservices Architecture - SECCOMP 2020
Introduction to Microservices Architecture - SECCOMP 2020
Rodrigo Antonialli
 
High-Speed Reactive Microservices
High-Speed Reactive MicroservicesHigh-Speed Reactive Microservices
High-Speed Reactive Microservices
Rick Hightower
 
Microservice architecture
Microservice architectureMicroservice architecture
Microservice architecture
Žilvinas Kuusas
 
Micro Front-End & Microservices - Plansoft
Micro Front-End & Microservices - PlansoftMicro Front-End & Microservices - Plansoft
Micro Front-End & Microservices - Plansoft
Miki Lombardi
 
WTF is a Microservice - Rafael Schloming, Datawire
WTF is a Microservice - Rafael Schloming, DatawireWTF is a Microservice - Rafael Schloming, Datawire
WTF is a Microservice - Rafael Schloming, Datawire
Ambassador Labs
 
Microservices patterns
Microservices patternsMicroservices patterns
Microservices patterns
Vikram Babu Kuruguntla
 
Surviving microservices
Surviving microservicesSurviving microservices
Surviving microservices
Francesco Degrassi
 
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
InfluxData
 
Reactive Micro Services with Java seminar
Reactive Micro Services with Java seminarReactive Micro Services with Java seminar
Reactive Micro Services with Java seminar
Gal Marder
 
3 Keys to Performance Testing at the Speed of Agile
3 Keys to Performance Testing at the Speed of Agile3 Keys to Performance Testing at the Speed of Agile
3 Keys to Performance Testing at the Speed of Agile
Neotys
 
The working architecture of NodeJS applications, Виктор Турский
The working architecture of NodeJS applications, Виктор ТурскийThe working architecture of NodeJS applications, Виктор Турский
The working architecture of NodeJS applications, Виктор Турский
Sigma Software
 
The working architecture of node js applications open tech week javascript ...
The working architecture of node js applications   open tech week javascript ...The working architecture of node js applications   open tech week javascript ...
The working architecture of node js applications open tech week javascript ...
Viktor Turskyi
 
9 patterns of microservices
9 patterns of microservices9 patterns of microservices
9 patterns of microservices
Al Sayed Gamal
 
Building a Small DC
Building a Small DCBuilding a Small DC
Building a Small DC
APNIC
 

Similar to 2 years into drinking the Microservice kool-aid (Fact and Fiction) (20)

Monolithic to Microservices Architecture - STM 6
Monolithic to Microservices Architecture - STM 6Monolithic to Microservices Architecture - STM 6
Monolithic to Microservices Architecture - STM 6
 
Breaking down a monolith
Breaking down a monolithBreaking down a monolith
Breaking down a monolith
 
Microservices
MicroservicesMicroservices
Microservices
 
Ledingkart Meetup #1: Monolithic to microservices in action
Ledingkart Meetup #1: Monolithic to microservices in actionLedingkart Meetup #1: Monolithic to microservices in action
Ledingkart Meetup #1: Monolithic to microservices in action
 
Introduction to Microservices Patterns
Introduction to Microservices PatternsIntroduction to Microservices Patterns
Introduction to Microservices Patterns
 
Introduction to Microservices Patterns
Introduction to Microservices PatternsIntroduction to Microservices Patterns
Introduction to Microservices Patterns
 
Introduction to Microservices Architecture - SECCOMP 2020
Introduction to Microservices Architecture - SECCOMP 2020Introduction to Microservices Architecture - SECCOMP 2020
Introduction to Microservices Architecture - SECCOMP 2020
 
High-Speed Reactive Microservices
High-Speed Reactive MicroservicesHigh-Speed Reactive Microservices
High-Speed Reactive Microservices
 
Microservice architecture
Microservice architectureMicroservice architecture
Microservice architecture
 
Micro Front-End & Microservices - Plansoft
Micro Front-End & Microservices - PlansoftMicro Front-End & Microservices - Plansoft
Micro Front-End & Microservices - Plansoft
 
WTF is a Microservice - Rafael Schloming, Datawire
WTF is a Microservice - Rafael Schloming, DatawireWTF is a Microservice - Rafael Schloming, Datawire
WTF is a Microservice - Rafael Schloming, Datawire
 
Microservices patterns
Microservices patternsMicroservices patterns
Microservices patterns
 
Surviving microservices
Surviving microservicesSurviving microservices
Surviving microservices
 
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
Wayfair Storefront Performance Monitoring with InfluxEnterprise by Richard La...
 
Reactive Micro Services with Java seminar
Reactive Micro Services with Java seminarReactive Micro Services with Java seminar
Reactive Micro Services with Java seminar
 
3 Keys to Performance Testing at the Speed of Agile
3 Keys to Performance Testing at the Speed of Agile3 Keys to Performance Testing at the Speed of Agile
3 Keys to Performance Testing at the Speed of Agile
 
The working architecture of NodeJS applications, Виктор Турский
The working architecture of NodeJS applications, Виктор ТурскийThe working architecture of NodeJS applications, Виктор Турский
The working architecture of NodeJS applications, Виктор Турский
 
The working architecture of node js applications open tech week javascript ...
The working architecture of node js applications   open tech week javascript ...The working architecture of node js applications   open tech week javascript ...
The working architecture of node js applications open tech week javascript ...
 
9 patterns of microservices
9 patterns of microservices9 patterns of microservices
9 patterns of microservices
 
Building a Small DC
Building a Small DCBuilding a Small DC
Building a Small DC
 

More from roblund

Adventures in Real-World Data Science
Adventures in Real-World Data ScienceAdventures in Real-World Data Science
Adventures in Real-World Data Science
roblund
 
ES6 Primer
ES6 PrimerES6 Primer
ES6 Primer
roblund
 
Jason Moore - Why releasing 50 features are less than 1 solution - BSDC 2016
Jason Moore - Why releasing 50 features are less than 1 solution - BSDC 2016Jason Moore - Why releasing 50 features are less than 1 solution - BSDC 2016
Jason Moore - Why releasing 50 features are less than 1 solution - BSDC 2016
roblund
 
Will Price - Venture Capital in Montana - BSDC 2016
Will Price - Venture Capital in Montana - BSDC 2016Will Price - Venture Capital in Montana - BSDC 2016
Will Price - Venture Capital in Montana - BSDC 2016
roblund
 
Pete Sveen - How to Build, Grow, and Monetize Your Online Platform - BSDC 2016
Pete Sveen - How to Build, Grow, and Monetize Your Online Platform - BSDC 2016Pete Sveen - How to Build, Grow, and Monetize Your Online Platform - BSDC 2016
Pete Sveen - How to Build, Grow, and Monetize Your Online Platform - BSDC 2016
roblund
 
Chris Omland - AWS Code Deploy - BSDC 2016
Chris Omland - AWS Code Deploy - BSDC 2016Chris Omland - AWS Code Deploy - BSDC 2016
Chris Omland - AWS Code Deploy - BSDC 2016
roblund
 
Josef Verbanac - Voice is (a) Best Practice
Josef Verbanac - Voice is (a) Best PracticeJosef Verbanac - Voice is (a) Best Practice
Josef Verbanac - Voice is (a) Best Practice
roblund
 
Emergence Of Code Schools
Emergence Of Code SchoolsEmergence Of Code Schools
Emergence Of Code Schools
roblund
 
Nora McDougall-Collins - I Can Do That
Nora McDougall-Collins - I Can Do ThatNora McDougall-Collins - I Can Do That
Nora McDougall-Collins - I Can Do That
roblund
 
Better tests automagically (big sky dev con 2015)
Better tests automagically (big sky dev con 2015)Better tests automagically (big sky dev con 2015)
Better tests automagically (big sky dev con 2015)
roblund
 
Ben Werner - Mountains and startups
Ben Werner - Mountains and startupsBen Werner - Mountains and startups
Ben Werner - Mountains and startups
roblund
 
Jason Moore - Interaction design in enterprise teams
Jason Moore - Interaction design in enterprise teamsJason Moore - Interaction design in enterprise teams
Jason Moore - Interaction design in enterprise teams
roblund
 

More from roblund (12)

Adventures in Real-World Data Science
Adventures in Real-World Data ScienceAdventures in Real-World Data Science
Adventures in Real-World Data Science
 
ES6 Primer
ES6 PrimerES6 Primer
ES6 Primer
 
Jason Moore - Why releasing 50 features are less than 1 solution - BSDC 2016
Jason Moore - Why releasing 50 features are less than 1 solution - BSDC 2016Jason Moore - Why releasing 50 features are less than 1 solution - BSDC 2016
Jason Moore - Why releasing 50 features are less than 1 solution - BSDC 2016
 
Will Price - Venture Capital in Montana - BSDC 2016
Will Price - Venture Capital in Montana - BSDC 2016Will Price - Venture Capital in Montana - BSDC 2016
Will Price - Venture Capital in Montana - BSDC 2016
 
Pete Sveen - How to Build, Grow, and Monetize Your Online Platform - BSDC 2016
Pete Sveen - How to Build, Grow, and Monetize Your Online Platform - BSDC 2016Pete Sveen - How to Build, Grow, and Monetize Your Online Platform - BSDC 2016
Pete Sveen - How to Build, Grow, and Monetize Your Online Platform - BSDC 2016
 
Chris Omland - AWS Code Deploy - BSDC 2016
Chris Omland - AWS Code Deploy - BSDC 2016Chris Omland - AWS Code Deploy - BSDC 2016
Chris Omland - AWS Code Deploy - BSDC 2016
 
Josef Verbanac - Voice is (a) Best Practice
Josef Verbanac - Voice is (a) Best PracticeJosef Verbanac - Voice is (a) Best Practice
Josef Verbanac - Voice is (a) Best Practice
 
Emergence Of Code Schools
Emergence Of Code SchoolsEmergence Of Code Schools
Emergence Of Code Schools
 
Nora McDougall-Collins - I Can Do That
Nora McDougall-Collins - I Can Do ThatNora McDougall-Collins - I Can Do That
Nora McDougall-Collins - I Can Do That
 
Better tests automagically (big sky dev con 2015)
Better tests automagically (big sky dev con 2015)Better tests automagically (big sky dev con 2015)
Better tests automagically (big sky dev con 2015)
 
Ben Werner - Mountains and startups
Ben Werner - Mountains and startupsBen Werner - Mountains and startups
Ben Werner - Mountains and startups
 
Jason Moore - Interaction design in enterprise teams
Jason Moore - Interaction design in enterprise teamsJason Moore - Interaction design in enterprise teams
Jason Moore - Interaction design in enterprise teams
 

Recently uploaded

Introduction to IP address concept - Computer Networking
Introduction to IP address concept - Computer NetworkingIntroduction to IP address concept - Computer Networking
Introduction to IP address concept - Computer Networking
Md.Shohel Rana ( M.Sc in CSE Khulna University of Engineering & Technology (KUET))
 
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
sunnuchadda
 
21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY
21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY
21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY
PradeepKumarSK3
 
readers writers Problem in operating system
readers writers Problem in operating systemreaders writers Problem in operating system
readers writers Problem in operating system
VADAPALLYPRAVEENKUMA1
 
FINE-TUNING OF SMALL/MEDIUM LLMS FOR BUSINESS QA ON STRUCTURED DATA
FINE-TUNING OF SMALL/MEDIUM LLMS FOR BUSINESS QA ON STRUCTURED DATAFINE-TUNING OF SMALL/MEDIUM LLMS FOR BUSINESS QA ON STRUCTURED DATA
FINE-TUNING OF SMALL/MEDIUM LLMS FOR BUSINESS QA ON STRUCTURED DATA
kevig
 
Software Engineering and Project Management - Activity Planning
Software Engineering and Project Management - Activity PlanningSoftware Engineering and Project Management - Activity Planning
Software Engineering and Project Management - Activity Planning
Prakhyath Rai
 
The world of Technology Management MEM 814.pptx
The world of Technology Management MEM 814.pptxThe world of Technology Management MEM 814.pptx
The world of Technology Management MEM 814.pptx
engrasjadshahzad
 
Metrology Book, Bachelors in Mechanical Engineering
Metrology Book, Bachelors in Mechanical EngineeringMetrology Book, Bachelors in Mechanical Engineering
Metrology Book, Bachelors in Mechanical Engineering
leakingvideo
 
ARITMETICO.pdf xxxxxxxxxxxxxxxxxxxxxxxx
ARITMETICO.pdf  xxxxxxxxxxxxxxxxxxxxxxxxARITMETICO.pdf  xxxxxxxxxxxxxxxxxxxxxxxx
ARITMETICO.pdf xxxxxxxxxxxxxxxxxxxxxxxx
alemaro1123
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
rawankhanlove256
 
Presentation python programming vtu 6th sem
Presentation python programming vtu 6th semPresentation python programming vtu 6th sem
Presentation python programming vtu 6th sem
ssuser8f6b1d1
 
Thermodynamics Digital Material basics subject
Thermodynamics Digital Material basics subjectThermodynamics Digital Material basics subject
Thermodynamics Digital Material basics subject
JigneshChhatbar1
 
Top EPC companies in India - Best EPC Contractor
Top EPC companies in India - Best EPC  ContractorTop EPC companies in India - Best EPC  Contractor
Top EPC companies in India - Best EPC Contractor
MangeshK6
 
IE-469-Lecture-Notes-3IE-469-Lecture-Notes-3.pptx
IE-469-Lecture-Notes-3IE-469-Lecture-Notes-3.pptxIE-469-Lecture-Notes-3IE-469-Lecture-Notes-3.pptx
IE-469-Lecture-Notes-3IE-469-Lecture-Notes-3.pptx
BehairyAhmed2
 
Introduction And Differences Between File System And Dbms.pptx
Introduction And Differences Between File System And Dbms.pptxIntroduction And Differences Between File System And Dbms.pptx
Introduction And Differences Between File System And Dbms.pptx
SerendipityYoon
 
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
YanKing2
 
Python programming Introduction about Python
Python programming Introduction about PythonPython programming Introduction about Python
Python programming Introduction about Python
Senthil Vit
 
RECENT DEVELOPMENTS IN RING SPINNING.pptx
RECENT DEVELOPMENTS IN RING SPINNING.pptxRECENT DEVELOPMENTS IN RING SPINNING.pptx
RECENT DEVELOPMENTS IN RING SPINNING.pptx
peacesoul123
 
ANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
ANATOMY OF SOA - Thomas Erl - Service Oriented ArchitectureANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
ANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
Divya Rajasekar
 
Adv. Digital Signal Processing LAB MANUAL.pdf
Adv. Digital Signal Processing LAB MANUAL.pdfAdv. Digital Signal Processing LAB MANUAL.pdf
Adv. Digital Signal Processing LAB MANUAL.pdf
T.D. Shashikala
 

Recently uploaded (20)

Introduction to IP address concept - Computer Networking
Introduction to IP address concept - Computer NetworkingIntroduction to IP address concept - Computer Networking
Introduction to IP address concept - Computer Networking
 
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 in City
 
21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY
21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY
21EC63_Module1B.pptx VLSI design 21ec63 MOS TRANSISTOR THEORY
 
readers writers Problem in operating system
readers writers Problem in operating systemreaders writers Problem in operating system
readers writers Problem in operating system
 
FINE-TUNING OF SMALL/MEDIUM LLMS FOR BUSINESS QA ON STRUCTURED DATA
FINE-TUNING OF SMALL/MEDIUM LLMS FOR BUSINESS QA ON STRUCTURED DATAFINE-TUNING OF SMALL/MEDIUM LLMS FOR BUSINESS QA ON STRUCTURED DATA
FINE-TUNING OF SMALL/MEDIUM LLMS FOR BUSINESS QA ON STRUCTURED DATA
 
Software Engineering and Project Management - Activity Planning
Software Engineering and Project Management - Activity PlanningSoftware Engineering and Project Management - Activity Planning
Software Engineering and Project Management - Activity Planning
 
The world of Technology Management MEM 814.pptx
The world of Technology Management MEM 814.pptxThe world of Technology Management MEM 814.pptx
The world of Technology Management MEM 814.pptx
 
Metrology Book, Bachelors in Mechanical Engineering
Metrology Book, Bachelors in Mechanical EngineeringMetrology Book, Bachelors in Mechanical Engineering
Metrology Book, Bachelors in Mechanical Engineering
 
ARITMETICO.pdf xxxxxxxxxxxxxxxxxxxxxxxx
ARITMETICO.pdf  xxxxxxxxxxxxxxxxxxxxxxxxARITMETICO.pdf  xxxxxxxxxxxxxxxxxxxxxxxx
ARITMETICO.pdf xxxxxxxxxxxxxxxxxxxxxxxx
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Presentation python programming vtu 6th sem
Presentation python programming vtu 6th semPresentation python programming vtu 6th sem
Presentation python programming vtu 6th sem
 
Thermodynamics Digital Material basics subject
Thermodynamics Digital Material basics subjectThermodynamics Digital Material basics subject
Thermodynamics Digital Material basics subject
 
Top EPC companies in India - Best EPC Contractor
Top EPC companies in India - Best EPC  ContractorTop EPC companies in India - Best EPC  Contractor
Top EPC companies in India - Best EPC Contractor
 
IE-469-Lecture-Notes-3IE-469-Lecture-Notes-3.pptx
IE-469-Lecture-Notes-3IE-469-Lecture-Notes-3.pptxIE-469-Lecture-Notes-3IE-469-Lecture-Notes-3.pptx
IE-469-Lecture-Notes-3IE-469-Lecture-Notes-3.pptx
 
Introduction And Differences Between File System And Dbms.pptx
Introduction And Differences Between File System And Dbms.pptxIntroduction And Differences Between File System And Dbms.pptx
Introduction And Differences Between File System And Dbms.pptx
 
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large...
 
Python programming Introduction about Python
Python programming Introduction about PythonPython programming Introduction about Python
Python programming Introduction about Python
 
RECENT DEVELOPMENTS IN RING SPINNING.pptx
RECENT DEVELOPMENTS IN RING SPINNING.pptxRECENT DEVELOPMENTS IN RING SPINNING.pptx
RECENT DEVELOPMENTS IN RING SPINNING.pptx
 
ANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
ANATOMY OF SOA - Thomas Erl - Service Oriented ArchitectureANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
ANATOMY OF SOA - Thomas Erl - Service Oriented Architecture
 
Adv. Digital Signal Processing LAB MANUAL.pdf
Adv. Digital Signal Processing LAB MANUAL.pdfAdv. Digital Signal Processing LAB MANUAL.pdf
Adv. Digital Signal Processing LAB MANUAL.pdf
 

2 years into drinking the Microservice kool-aid (Fact and Fiction)

  • 2. Review of Microservice Architectures This is everything you need to know. 2
  • 3. 3
  • 4. Review of Microservice Architectures Microservice Characteristics ● Like SOA but “smaller” ● Many small services ● Versioned independently ● Deployed independently ● Scale independently Possible Microservice Benefits ● Resilience (through clustered services) ● Zero downtime ● Horizontal scaling ● Independent scaling of components ● Ease of making changes ● Ease of uptaking new technology ● Canary testing ● Independent release cycles “Building Microservices” - Sam Newman 4
  • 5. The taste of the kool-aid (part 1) ● It’s all about the contract! That’s not exactly revolutionary ● Microservices mess with everything. Including testing strategies. ● Hard things become easier (releasing changes, scaling) ● Easy things become hard (e.g. multi-lingual support, referential integrity checks) ● Service boundaries don’t necessarily follow the data model. ● Some ‘scary’ stuff like a repository/db per service really isn’t a big deal. Seriously. 5
  • 6. The taste of the kool-aid (part 2) ● Communication and state exchange between microservices gets complicated (more to follow) ● Zero-downtime upgrades: Not actually that bad once you get the hang of it ● Horizontal scalability: Not a simple yes or no question but a spectrum. Also, not free. ● Canaries are essential for both quality and velocity (more to follow) ● Polyglot stack: More compelling at the database level than at the language/runtime level ● As a developer, you’re empowered. With power comes responsibility 6
  • 7. Deep-Dive #1 Continuous Deployment and Canary Testing…. 7
  • 11. Continuous Deployment - ala Microservices Customers Days/Hours 11
  • 12. Continuous Deployment - ala Microservices Customers Days/Hours 12
  • 14. Guiding Principles 1. You can’t “contrive” production in a laboratory ○ It’s probably impossible or at a minimum not worth the cost 2. Reality is our friend ○ Time doesn’t make everything “stronger” ○ Are devs thinking about production? 3. Build “crisis” reflexes ○ Do everyday what you will do in a crisis 14
  • 15. How? Pop Quiz Hotshot: You MUST push a change out to production in the next hour - what do you do? 15
  • 16. Continuous Deployment - ala Microservices Customers Days/Hours 16
  • 18. ● Code level ○ Easy to do ○ Basically some “if checks” and a global registry per tenant or % ○ Larger blast radius ● Binary Level ○ Harder to do ○ Utilize service routing techniques (per tenant/% based) ○ Smaller blast radius ● Example (Chat change) ○ Reverse works on this as well Canary Testing -- It’s not that hard 18
  • 19. Guiding Principles Played Out @ Quiq 1. You can’t “contrive” production in a laboratory ○ It’s probably impossible or at a minimum not worth the cost 2. Reality is our friend ○ Time doesn’t make everything “stronger” ○ Are devs thinking about production? 3. Build “crisis” reflexes ○ Do everyday what you will do in a crisis 19
  • 20. Deep-Dive #2 - Microservice Communication 20
  • 21. 21
  • 22. More Protocols, More Problems ● Achieving the benefits of microservices is non-trivial ● Using both HTTP & AMQP made each more daunting ○ HTTP necessitates service discovery ○ Canary testing - must cutover & direct two traffic types ○ Resilience - Need clustered/redundant AMQP ○ Horizontal scalability - AMQP server a possible bottleneck ● AMQP worked directly against some of the hallmarks ○ Becomes a point of centralization, like a shared database ○ Polyglot stack - Not as ubiquitous as HTTP But here’s the worst part… We were actually afraid to horizontally scale some of our services! 22
  • 23. 1. What if Downstream2 finishes first? a. Even DB locking does not help because the race condition is in the queue itself 2. The more instances of Downstream1, the more likely the corruption 3. We need an AMQP extension, TCP- style message ordering, or merge strategies, or... User A Offline User A Online Downstream2Downstream1 SessionService1 23 Knock, knock: race condition (who’s there?)
  • 24. Our Solution ● All inter-service communication over HTTP ● Store the transactionally ordered side effects in the same database that mutated records during the transaction (e.g. in redis) ● Ordered executor in (e.g. SessionService) publishes side effect via HTTP ○ Ordered executor leverages locking mechanisms of host service’s primary DB ● Downstream1 receives side effect, pushes it into its own queue, kicks off a future & immediately returns (still async!) ● Now we have ordered, durable downstream state updates and/or side effects! 24
  • 25. 25
  • 26. Some Observations 1. Achieving the benefits of microservices is challenging. Do yourself a favor and limit the number of communication protocols 2. Don’t use producer/consumer queues for general communication 3. In the event of problems like process death, always err on the side of sending the same message twice instead of dropping it. Idempotency is king. 26

Editor's Notes

  1. Characteristics enable but don’t guarantee benefits Takes hard work… you can evolve into these (even in greenfield)
  2. Different gates along the way
  3. Whatever gates were there before are ignored or a different “policy”
  4. Different gates along the way
  5. Developers check in line of code - not thinking about production
  6. Every line of code the developer is thinking about production (quality, scalability, performance?) Pair program
  7. Key: Small enough chunks of work Includes: Size of service and how often release What about bake time? 2 Extremes: Data migration code vs Low level library
  8. Customer/Tenant
  9. Customer/Tenant
  10. A service must be able to read from another service while processing requests Must support a service registering a side effect in another service in a durable, transactional way E.g. sending an email to the user iff their account is created Must support a service emitting partial state updates to one or more downstream services in a durable, transactional way E.g. user online/offline events It seemed odd/heavy to use anything other than HTTP for 1, so we did that. 2 & 3 are inherently asynchronous, so we employed AMQP. Makes sense, right?
  11. Service A manages sessions and produces online/offline events Service B consumes those events to maintain a denormalized list of online users (so that A doesn’t have to scale w/ B) This is microservices . . There are multiple instances of service B. What if B2 finishes first? Even DB locking does not help because the race condition is in the AMQP queue itself The more instances of B, the more likely the corruption We need an AMQP extension, TCP-style message ordering, or merge strategies, or...
  12. The “Process” block uses the databases locking mechanisms to insure a single worker on pipelineA We solved our problems by moving the queue(s) into the services, where we’re in control There’s an implicit retry daemon