Developing applications with a microservice architecture (SVforum, microservices meetup)

Microservices: Decomposing
@crichardson
Applications for
Deployability and Scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@crichardson
chris@chrisrichardson.net
http://plainoldobjects.com

Presentation goal
How decomposing applications into
@crichardson
microservices
improves deployability and scalability
and
simplifies the adoption of new
technologies

@crichardson
About Chris
Founder of a buzzword compliant (stealthy, social, mobile,
big data, machine learning, ...) startup
Consultant helping organizations improve how they
architect and deploy applications using cloud computing,
micro services, polyglot applications, NoSQL, ...

@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Client ⇔ service interaction design
Decentralized data management

@crichardson
Let’s imagine you are
building an online store

@crichardson
Traditional application
architecture
StoreFrontUI
Product Info
Service
Recommendation
Service
Tomcat
Browser/
Client
WAR/EAR
MySQL
Database
Review Service
Simple to
develop
test
deploy
Load
balancer
scale
Spring MVC
Spring
Hibernate
Order Service
HTML
REST/JSON

But large, complex, monolithic
@crichardson
applications
⇒
problems

@crichardson
Intimidates developers

@crichardson
Obstacle to frequent
deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often - really long QA cycles
e.g. Makes A/B testing UI really difficult
Eggs in
one basket

@crichardson
Overloads your IDE and
container
Slows down development

@crichardson
Obstacle to scaling
development
I want
to update the UI
But
the backend is not working
yet!
Lots of coordination and
communication required

Requires long-term commitment
to a technology stack
@crichardson

@crichardson
The scale cube
X axis
- horizontal duplication
Z axis - data partitioning
Y axis -
functional
decomposition
similar
splitting by things
Scale Scale by
splitting
different things

@crichardson
Y-axis scaling - application level
WAR
Storefront UI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service

@crichardson
Storefront UI
Product Info
Service
Recommendation
Service
Review
Service
Order
Service

@crichardson
Product Info
Service
Product Info
Recommendation
Service
Review
Service
Order
Service
Browse Products
UI
Checkout UI
Order management
UI
Account
management UI
Apply X-axis and Z-axis scaling
to each service independently

Service deployment options
@crichardson
Isolation, manageability
VM or Physical Machine
Docker/Linux container
JVM
JAR/WAR/OSGI bundle/...
Density/efficiency

@crichardson
Partitioning strategies...
Partition by noun, e.g. product info service
Partition by verb, e.g. Checkout UI
Single Responsibility Principle
Unix utilities - do one focussed thing well

@crichardson
Partitioning strategies
Too few
Drawbacks of the monolithic architecture
Too many - a.k.a. Nano-service anti-pattern
Runtime overhead
Potential risk of excessive network hops
Potentially difficult to understand system
Something of an art

@crichardson
Example micro-service

@crichardson
More service, less micro
But more realistically...
Focus on building services that make
development and deployment easier
- not just tiny services

@crichardson
Real world examples
http://techblog.netflix.com/
~600 services
http://highscalability.com/amazon-architecture
100-150 services to build a page
http://www.addsimplicity.com/downloads/
eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128

@crichardson
There are many benefits

@crichardson
Smaller, simpler apps
Easier to understand and develop
Less jar/classpath hell - who needs OSGI?
Faster to build and deploy
Reduced startup time - important for GAE

Scales development:
develop, deploy and scale
each service independently
@crichardson

@crichardson
Improves fault isolation

Eliminates long-term commitment
to a single technology stack
@crichardson
Modular, polyglot, multi-framework
applications

@crichardson
Two levels of architecture
System-level
Services
Inter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each service
Each service could use a different technology stack
Pick the best tool for the job
Rapidly evolving

Easily try other technologies
@crichardson
... and fail safely

But there are drawbacks
@crichardson

Complexity of developing a
@crichardson
distributed system
http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html

@crichardson
Multiple databases
&
Transaction management
e.g. Fun with eventual consistency

@crichardson
Complexity of testing a
distributed system

Complexity of deploying and
operating a distributed
@crichardson
system
You need a lot of automation

@crichardson
Developing and deploying
features that span multiple
services requires careful
coordination

@crichardson
When to use it?
In the beginning:
•You don’t need it
•It will slow you down
Later on:
•You need it
•Refactoring is painful

Let’s imagine that you want to
display a product’s details...
Product
Info Reviews
Recommendations
@crichardson

Directly connecting the front-end to the backend
@crichardson
View Controller
Model
Product Info
service
Recommendation
Service
Review
service
REST
REST
AMQP
Traditional server-side
web application
View Controller
Model
Browser/Native App
Chatty API
Web unfriendly
protocols

@crichardson
Use an API gateway
View Controller
Model
Product Info
service
Recommendation
Service
Review
service
REST
REST
AMQP
API
Gateway
View Controller
Model
Browser/Native App
Single entry point
Client
specific APIs
Protocol
translation
Traditional server-side
web application

@crichardson
Optimized client-specific
APIs
Web
application
Mobile
App
NodeJS
API
Gateway
REST
proxy
Event
publishing
Product Info
service
Recommendation
Service
Review
service
REST
REST
AMQP
getProductInfo()
getRecomm...()
getReviews()
getProductDetails()

@crichardson
Netflix API Gateway
http://techblog.netflix.com/2013/01/optimizing-netflix-api.html
Device specific
end points

@crichardson
API gateway design
challenges
Performance and scalability
Non-blocking I/O
Asynchronous, concurrent code
Handling partial failures
....
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

@crichardson
Useful frameworks for
building an API gateway
JVM:
Netty, Vertex
Netflix Hystrix
...
Other:
NodeJS

How does a browser
interact with the partitioned
@crichardson
web application?

@crichardson
Partitioned web app ⇒ no
longer a single base URL
/products
Browse Products
UI
/checkout
Checkout UI
/orders
Order management
UI
/account
Account
management UI
Browser ?

The solution: single entry point
that routes based on URL
@crichardson
/products
Browse Products
UI
/checkout
Checkout UI
/orders
Order management
UI
/account
Account
management UI
http://acme.com/<service>/...
Content
Browser Router
Hidden from
browser
Single entry
point

@crichardson
How do the services
communicate?

Inter-service communication
options
@crichardson
Synchronous HTTP ⇔ asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Asynchronous is preferred
JSON is fashionable but binary format
is more efficient

Pros and cons of messaging
Pros
Cons
Decouples client from
Additional complexity of
server
message broker
Message broker buffers
Request/reply-style
messages
communication is more
Supports a variety of
complex
communication patterns

Pros and cons of HTTP
Pros
Simple and familiar
Request/reply is easy
Firewall friendly
No intermediate broker
Cons
Only supports request/
reply
Server must be
available
Client needs to
discover URL(s) of
server(s)

@crichardson
Discovery option #1:
Internal load balancer
Load
Balancer
Product Info
Service
Product Info
Service
Product Info
Service
Product Info
Service
Client/
API gateway
Services register
with load balancer
Client talks to
load balancer
Has a well-known
location
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/USVPC_creating_basic_lb.html

@crichardson
Discovery option #2:
client-side load balancing
REST
Client
Product Info
Service
Product Info
Service
Product Info
Service
Product Info
Service
Client
Service
Registry
Services register
with registry
Client polls
registry
http://techblog.netflix.com/2013/01/announcing-ribbon-tying-netflix-mid.html
http://techblog.netflix.com/2012/09/eureka.html

@crichardson
Lots of moving parts!
PProrodducutc tIn Ifnofo
Service
Recommendation
Service
Review
Service
Order
Service
Browse Products UI
Checkout UI
Order management
UI
Account
management UI
API
Gate
way
Service registry
Content
Router
HTML
Browser
REST
Client
Ext.
LB
Ext.
LB

@crichardson
Decomposed services ⇒
decomposed databases
Order management Customer management
Order
Database
Customer
Database
Separate databases ⇒ less coupling

@crichardson
Decomposed databases ⇒
polyglot persistence
IEEE Software Sept/October 2010 - Debasish Ghosh / Twitter @debasishg

Untangling orders and customers
Customer management
@crichardson
Order Service
placeOrder()
Order management
Customer Service
availableCredit()
updateCustomer()
Customer
creditLimit
...
Order belongs to has orders
total
Invariant:
sum(order.total) <= creditLimit
available credit= creditLimit -
Trouble! sum(order.total)

@crichardson
Problems
Reads
Service A needs to read data owned by service B
Updates
Transaction must update data owned by multiple
services

Handling reads: requesting
credit limit
@crichardson
placeOrder()
Order management
getCreditLimit()
Customer management

@crichardson
Pulling data
Benefits
Simple to implement
Ensures data is fresh
Drawbacks
Reduces availability
Increases response time

Customer management
@crichardson
Handling reads: replicating the
credit limit
Order Service
placeOrder()
sum(order.total) <=
Order management
Customer
creditLimit
...
Order
total
Customer’
creditLimit
creditLimit
changeCreditLimit()
Customer Service
updateCustomer()
Simplified

Useful idea: Bounded context
@crichardson
Different services have a different view of a
domain object, e.g.
User Management = complex view of user
Rest of application: User = PK + ACL + Name
Different services can have a different domain
model

@crichardson
Replicating data
Benefits
Improved availability for reads
Improves latency
Drawbacks
Additional complexity of replication mechanism

How to update distributed
(and replicated) data?
@crichardson

Use distributed transactions
@crichardson
Benefits
Simple development
Guarantees consistency
Drawbacks
Operationally complex
Reduced availability

@crichardson
Use event-driven
architecture
How
Services publish events when data changes
Subscribing services update their data
Benefits:
Simpler
Better availability
Drawbacks:
Application has to handle eventually consistent data
Application has to handle duplicate events

How do services publish
@crichardson
events?

@crichardson
To maintain consistency the
application must
atomically publish an event
whenever
a domain object changes

@crichardson
How to generate events?
Database triggers
Hibernate event listener
Ad hoc event publishing code mixed into business logic
Domain events - “formal” modeling of events
Event Sourcing

Atomically publishing events
Use distributed transactions to update database and publish
to message
Database and message broker must support 2PC
2PC is best avoided
Use two step eventual consistency mechanism:
1. Update database: new entity state & intent to publish event
2. Publish event & mark event as published
• Difficult to implement when using a NoSQL database :-(
@crichardson

@crichardson
Event sourcing
An event-centric approach to designing domain models
Request becomes a command that is sent to an Aggregate
Aggregates handle commands by generating events
Apply events to an aggregate to update state
Persist events NOT state
Replay events to recreate the current state of an aggregate
Event Store ≃ database + message broker

Request handling in an event-sourced application
@crichardson
HTTP
Handler
Event
Store
pastEvents = findEvents(entityId)
Account
new()
applyEvents(pastEvents)
newEvents = processCmd(SomeCmd)
saveEvents(newEvents)
Microservice A

@crichardson
Event Store publishes events -
consumed by other services
Event
Store
Microservice B
Event
Subscriber
subscribe(EventTypes)
publish(event)
publish(event)
Aggregate
NoSQL
materialized
view
update()
update()

UpdateCreditLimitCommand
@crichardson
Using event sourcing
Customer Service
updateCustomer()
CustomerCreditLimitUpdatedEvent
Event Store
Order management
Order
total
Customer’
creditLimit
CustomerCreditLimitUpdatedEvent(...)
Customer management
Customer
creditLimit
...

@crichardson
Customer aggregate
case class Customer(customerId: String, creditLimit: BigDecimal)
extends ValidatingAggregate[Customer, CustomerCommands.CustomerCommand] {
def this() = this(null, null)
override def processCommand = {
case CreateCustomerCommand(customerId, creditLimit) =>
Seq(CustomerCreatedEvent(customerId, creditLimit))
case UpdateCreditLimitCommand(newLimit) if newLimit >= 0 =>
Seq(CustomerCreditLimitUpdatedEvent(newLimit))
}
override def applyEvent = {
case CustomerCreatedEvent(customerId, creditLimit) =>
copy(customerId=customerId, creditLimit=creditLimit)
case CustomerCreditLimitUpdatedEvent(newLimit) =>
copy(creditLimit=newLimit)
}
}
Command
⇒
Events
Event
⇒
Updated
state

@crichardson
EventStore API
trait EventStore {
def save[T](entityId: Id, events: Seq[Event]): T
def update[T](entityId: Id,
version: EntityVersion, events: Seq[Event]): T
def load[T](entityType: Class[T], entityId: EntityId): T
def subscribe(...) : ...
..
}

Unfamiliar but it solves many
problems
@crichardson
Eliminates O/R mapping problem
Supports both SQL and NoSQL databases
Publishes events reliably
Reliable eventual consistency framework
...

But let’s imagine that you
want to display an account
and it’s recent transactions...
@crichardson

Displaying balance + recent
transactions
We need to do a “join: between the Account and the corresponding
TransferTransactions
(Assuming Debit/Credit events don’t include other account, ...)
@crichardson
BUT
Event Store = primary key lookup of individual aggregates, ...
⇒
Use Command Query Responsibility Separation
Define separate “materialized” query-side views that implement
those queries

HTTP GET
Request
View Query
Service
@crichardson
Query-side microservices
Updater - microservice
View Updater
Service
Events
Event Store
Reader - microservice
View
Store
e.g.
MongoDB
Neo4J
CloudSearch
update query

@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
Client ⇔ service interaction design
Decentralized data management
Bonus feature: microservices in action

API gateway implemented
@crichardson
using NodeJS

Spring Boot provides opinionated Spring application
architecture
Simplifies applications using Convention over
Configuration
Packages application as standalone, executable jar
Provides production-ready features externalized
configuration and health checks
@crichardson
Microservices built with
Spring Boot
For more on Spring Boot + Microservices see
http://bit.ly/bootmicro1

@crichardson
Event-driven architecture
Business logic implemented using event sourcing-based
domain model
Views implemented using
DynamoDB
Cloud Search (text search)

@crichardson
Jenkins-based deployment
pipeline
Build & Test
micro-service
Build & Test
Docker
image
Deploy
Docker
image
to
Repository
One pipeline per micro-service

@crichardson
Deployed on Amazon EC2

Monolithic applications are
simple to develop and deploy
@crichardson
BUT have significant
drawbacks

@crichardson
Apply the scale cube
Modular, polyglot, and
scalable applications
Services developed,
deployed and scaled
independently

Use a modular, polyglot architecture
View Controller Product Info
@crichardson
Model
service
Recommendation
Service
Review
service
REST
REST
AMQP
API
Gateway
Server-side web
application
View Controller
Model
Browser/Native
application

@crichardson
Start refactoring your
monolith
Monolith Anti-corruption Service
layer
Glue code
Pristine

@crichardson chris@chrisrichardson.net
@crichardson
Questions?
http://plainoldobjects.com http://microservices.io

Developing applications with a microservice architecture (SVforum, microservices meetup)

More Related Content

What's hot

Similar to Developing applications with a microservice architecture (SVforum, microservices meetup)

More from Chris Richardson

Recently uploaded

Developing applications with a microservice architecture (SVforum, microservices meetup)