© 2018 Magento, An Adobe Company Page | 1
A long way from
Monolith to Micro-
Service Architecture
Yesterday, Today and Tomorrow
© 2018 Magento, An Adobe Company Page | 2
Igor Miniailo
Magento Architect,
Community Engineering Team
© 2018 Magento, An Adobe Company Page | 3
Agenda
© 2018 Magento, An Adobe Company Page | 3
• Magento 1. Big ball of Mud
• SOLID and Single Responsibility
• Performance vs Scalability
• Coupling vs Cohesion
• Bounded contexts and their Boundaries
• CAP Theorem
• ACID vs BASE
• Modularity as a first step towards micro-services
• Headless Magento
• Service Isolation (Code and DB split)
• Conway's law
• Microservice architecture
© 2018 Magento, An Adobe Company Page | 4© 2018 Magento, An Adobe Company Page | 4
Magento 1
And all the legacy we got starting Magento 2 development
© 2018 Magento, An Adobe Company Page | 5
Classical N-layered architecture
• Compromise in the design of Data Model
• Not possible to scale separately Read out of
Write operations
• Tend to Anemic Data Model
• Tend to Monolithic Architecture
• An application has to be deployed as a
whole
• An application has to be scaled as a
whole
© 2018 Magento, An Adobe Company Page | 6
Big Ball of Mud
Even though a modular code
structure groups related behavior, it
is easy to introduce an undesired
dependency between application
services, because services are not
deployed and tested independently.
© 2018 Magento, An Adobe Company Page | 7
SOLID comes to rescue
© 2018 Magento, An Adobe Company Page | 8© 2018 Magento, An Adobe Company Page | 8
Modularity as a first step towards
micro-services
© 2018 Magento, An Adobe Company Page | 9
• Every sophisticated business
domain consists of a bunch of
Bounded Contexts
• Each Bounded Context
contains models and maybe
other contexts
• The Bounded Context is also a
boundary for the meaning of a
given model
Bounded Context
© 2018 Magento, An Adobe Company Page | 10
Bounded Context
© 2018 Magento, An Adobe Company Page | 11
© 2018 Magento, An Adobe Company Page | 12
CAP Theorem
States that it is impossible for a distributed
data store to simultaneously provide more
than two out of the following three
guarantees:
• Consistency: Every read receives the
most recent write or an error
• Availability: Every request receives a
(non-error) response – without the
guarantee that it contains the most
recent write
• Partition tolerance: The system
continues to operate despite an arbitrary
number of messages being dropped (or
delayed) by the network between nodes
© 2018 Magento, An Adobe Company Page | 13
Asynchronous processing
http://www.enterpriseintegrationpatterns.com/docs/IEEE_Software_Design_2PC.pdf
© 2018 Magento, An Adobe Company Page | 14
Conversation pattern
The interaction between two parties
(customer and coffee shop)
consists of:
• a short synchronous interaction (ordering
and paying)
• and a longer, asynchronous interaction
(making and receiving the drink)
This type of conversation pattern is quite
common in purchasing scenarios.
© 2018 Magento, An Adobe Company Page | 15
Error-handling strategies for loosely coupled systems
(a) Write-off
(b) Retry
(c) Compensating action
(d) Transaction coordinator (two phase commit)
© 2018 Magento, An Adobe Company Page | 16
Implement business transaction that spans multiple
services as a saga
A saga is a sequence of local
transactions
Each local transaction updates
the database and publishes a
message or event to trigger the
next local transaction in the
saga
© 2018 Magento, An Adobe Company Page | 17
Event Sourcing
© 2018 Magento, An Adobe Company Page | 18
*The property of certain operations whereby they can be
applied multiple times without changing the result beyond
the initial application.
Idempotence of operation and at-least-once delivery
Network communication is unreliable.
Retries will be required, and some
messages will be delivered more than
once
© 2018 Magento, An Adobe Company Page | 19
• All operations MUST BE idempotent.
• Sagas SHOULD BE used for consistency of
distributed operations
• All new service contracts SHOULD expose
asynchronous APIs
• All new state modifying operations SHOULD expose
bulk APIs
• All service operations MUST BE stateless
• There MUST BE NO data dependencies between
services
Design Principles to follow
© 2018 Magento, An Adobe Company Page | 20
• Every module defines its APIs (service contracts),
which are PHP interfaces that can be called either
from within a PHP process or remotely through
REST, SOAP or AMQP APIs.
• Every module can call other modules only through
their service contracts.
Service Layer
© 2018 Magento, An Adobe Company Page | 21
Service Layer. Current State
Legacy undesired
dependencies direct
inter-module
- model-to-model
- presentation-to-model
still exist.
© 2018 Magento, An Adobe Company Page | 22
Modules split
© 2018 Magento, An Adobe Company Page | 23
Modules Split. Inventory
• API – service contracts and
extension points
• Implementation (private) -
should not be called directly
from other modules
• Admin UI
• Frontend UI
© 2018 Magento, An Adobe Company Page | 24
Headless Multi-Source Inventory
UI modules are removable if merchant
wants to use external ERP system as a
source of truth where he will manage
inventory
© 2018 Magento, An Adobe Company Page | 25
Desirable State
© 2018 Magento, An Adobe Company Page | 26
Database Split
A service must not talk to other service
database directly, only through service
contracts
TODO:
- Minimize data relations
- Replace the required inter-service db-
level data relations, joins, and transactions
with application-level relations, joins, and
transactions
© 2018 Magento, An Adobe Company Page | 27
Desirable state
Isolated ServicesMonolith
© 2018 Magento, An Adobe Company Page | 28
Monoliths vs Microservices. Scalability
© 2018 Magento, An Adobe Company Page | 29
© 2018 Magento, An Adobe Company Page | 30
Conway’s Law
Any organization that designs a system
(defined broadly) will produce a design
whose structure is a copy of the
organization's communication structure.
-- Melvyn Conway, 1967
© 2018 Magento, An Adobe Company Page | 31
2.3 Stage #1 Stage #2 Stage #3 Stage #4 Stage #5
Inventory (MSI) App Framework Order Management CMS Shipping Marketing
Pricing Tax Reports Promotions Risk
Checkout Customer Payments Wishlist
Rewards
Company
Service decomposition
© 2018 Magento, An Adobe Company Page | 32
• Magento Service Isolation Design Document by
Anton Kril
• Magento administrative tool
• Magento MSI
Useful Links
© 2018 Magento, An Adobe Company Page | 33
Q&AFollow me in Twitter
@iminyaylo
© 2018 Magento, An Adobe Company Page | 34
Thank You
© 2018 Magento, An Adobe Company Page | 34

The long way from Monolith to Microservices

  • 1.
    © 2018 Magento,An Adobe Company Page | 1 A long way from Monolith to Micro- Service Architecture Yesterday, Today and Tomorrow
  • 2.
    © 2018 Magento,An Adobe Company Page | 2 Igor Miniailo Magento Architect, Community Engineering Team
  • 3.
    © 2018 Magento,An Adobe Company Page | 3 Agenda © 2018 Magento, An Adobe Company Page | 3 • Magento 1. Big ball of Mud • SOLID and Single Responsibility • Performance vs Scalability • Coupling vs Cohesion • Bounded contexts and their Boundaries • CAP Theorem • ACID vs BASE • Modularity as a first step towards micro-services • Headless Magento • Service Isolation (Code and DB split) • Conway's law • Microservice architecture
  • 4.
    © 2018 Magento,An Adobe Company Page | 4© 2018 Magento, An Adobe Company Page | 4 Magento 1 And all the legacy we got starting Magento 2 development
  • 5.
    © 2018 Magento,An Adobe Company Page | 5 Classical N-layered architecture • Compromise in the design of Data Model • Not possible to scale separately Read out of Write operations • Tend to Anemic Data Model • Tend to Monolithic Architecture • An application has to be deployed as a whole • An application has to be scaled as a whole
  • 6.
    © 2018 Magento,An Adobe Company Page | 6 Big Ball of Mud Even though a modular code structure groups related behavior, it is easy to introduce an undesired dependency between application services, because services are not deployed and tested independently.
  • 7.
    © 2018 Magento,An Adobe Company Page | 7 SOLID comes to rescue
  • 8.
    © 2018 Magento,An Adobe Company Page | 8© 2018 Magento, An Adobe Company Page | 8 Modularity as a first step towards micro-services
  • 9.
    © 2018 Magento,An Adobe Company Page | 9 • Every sophisticated business domain consists of a bunch of Bounded Contexts • Each Bounded Context contains models and maybe other contexts • The Bounded Context is also a boundary for the meaning of a given model Bounded Context
  • 10.
    © 2018 Magento,An Adobe Company Page | 10 Bounded Context
  • 11.
    © 2018 Magento,An Adobe Company Page | 11
  • 12.
    © 2018 Magento,An Adobe Company Page | 12 CAP Theorem States that it is impossible for a distributed data store to simultaneously provide more than two out of the following three guarantees: • Consistency: Every read receives the most recent write or an error • Availability: Every request receives a (non-error) response – without the guarantee that it contains the most recent write • Partition tolerance: The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes
  • 13.
    © 2018 Magento,An Adobe Company Page | 13 Asynchronous processing http://www.enterpriseintegrationpatterns.com/docs/IEEE_Software_Design_2PC.pdf
  • 14.
    © 2018 Magento,An Adobe Company Page | 14 Conversation pattern The interaction between two parties (customer and coffee shop) consists of: • a short synchronous interaction (ordering and paying) • and a longer, asynchronous interaction (making and receiving the drink) This type of conversation pattern is quite common in purchasing scenarios.
  • 15.
    © 2018 Magento,An Adobe Company Page | 15 Error-handling strategies for loosely coupled systems (a) Write-off (b) Retry (c) Compensating action (d) Transaction coordinator (two phase commit)
  • 16.
    © 2018 Magento,An Adobe Company Page | 16 Implement business transaction that spans multiple services as a saga A saga is a sequence of local transactions Each local transaction updates the database and publishes a message or event to trigger the next local transaction in the saga
  • 17.
    © 2018 Magento,An Adobe Company Page | 17 Event Sourcing
  • 18.
    © 2018 Magento,An Adobe Company Page | 18 *The property of certain operations whereby they can be applied multiple times without changing the result beyond the initial application. Idempotence of operation and at-least-once delivery Network communication is unreliable. Retries will be required, and some messages will be delivered more than once
  • 19.
    © 2018 Magento,An Adobe Company Page | 19 • All operations MUST BE idempotent. • Sagas SHOULD BE used for consistency of distributed operations • All new service contracts SHOULD expose asynchronous APIs • All new state modifying operations SHOULD expose bulk APIs • All service operations MUST BE stateless • There MUST BE NO data dependencies between services Design Principles to follow
  • 20.
    © 2018 Magento,An Adobe Company Page | 20 • Every module defines its APIs (service contracts), which are PHP interfaces that can be called either from within a PHP process or remotely through REST, SOAP or AMQP APIs. • Every module can call other modules only through their service contracts. Service Layer
  • 21.
    © 2018 Magento,An Adobe Company Page | 21 Service Layer. Current State Legacy undesired dependencies direct inter-module - model-to-model - presentation-to-model still exist.
  • 22.
    © 2018 Magento,An Adobe Company Page | 22 Modules split
  • 23.
    © 2018 Magento,An Adobe Company Page | 23 Modules Split. Inventory • API – service contracts and extension points • Implementation (private) - should not be called directly from other modules • Admin UI • Frontend UI
  • 24.
    © 2018 Magento,An Adobe Company Page | 24 Headless Multi-Source Inventory UI modules are removable if merchant wants to use external ERP system as a source of truth where he will manage inventory
  • 25.
    © 2018 Magento,An Adobe Company Page | 25 Desirable State
  • 26.
    © 2018 Magento,An Adobe Company Page | 26 Database Split A service must not talk to other service database directly, only through service contracts TODO: - Minimize data relations - Replace the required inter-service db- level data relations, joins, and transactions with application-level relations, joins, and transactions
  • 27.
    © 2018 Magento,An Adobe Company Page | 27 Desirable state Isolated ServicesMonolith
  • 28.
    © 2018 Magento,An Adobe Company Page | 28 Monoliths vs Microservices. Scalability
  • 29.
    © 2018 Magento,An Adobe Company Page | 29
  • 30.
    © 2018 Magento,An Adobe Company Page | 30 Conway’s Law Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. -- Melvyn Conway, 1967
  • 31.
    © 2018 Magento,An Adobe Company Page | 31 2.3 Stage #1 Stage #2 Stage #3 Stage #4 Stage #5 Inventory (MSI) App Framework Order Management CMS Shipping Marketing Pricing Tax Reports Promotions Risk Checkout Customer Payments Wishlist Rewards Company Service decomposition
  • 32.
    © 2018 Magento,An Adobe Company Page | 32 • Magento Service Isolation Design Document by Anton Kril • Magento administrative tool • Magento MSI Useful Links
  • 33.
    © 2018 Magento,An Adobe Company Page | 33 Q&AFollow me in Twitter @iminyaylo
  • 34.
    © 2018 Magento,An Adobe Company Page | 34 Thank You © 2018 Magento, An Adobe Company Page | 34

Editor's Notes

  • #15 For example, when you place an order on Amazon.com, a short synchronous interaction assigns a unique order number first. All subsequent steps (charging your credit card and packaging and shipping the product) are performed asynchronously: you’re notified via (asynchronous) email as additional processing steps complete. If anything goes wrong, Amazon usually uses similar compensation strategies—refunding your credit card or retrying the action by resending the goods.
  • #27 Unlike current implementation (OS & Commerce split), the data relation decoupling should be performed in Open Source edition of Magento to simplify extension development.