Microservice Architecture

Microservice Architecture
Viraj Brian Wijesuriya
vbw@ucsc.cmb.ac.lk
1
SCS 4120 - Software Engineering IV
BACHELOR OF SCIENCE HONOURS IN COMPUTER SCIENCE
BACHELOR OF SCIENCE HONOURS IN SOFTWARE ENGINEERING
All in One Place Lecture Notes
Distribution Among Friends Only
All copyrights belong to their respective owners

• Definition:
– In short, the microservice architectural style is an approach to developing a
single application as a suite of small services, each running in its own process
and communicating with lightweight mechanisms, often an HTTP resource
API. These services are built around business capabilities and independently
deployable by fully automated deployment machinery. There is a bare
minimum of centralized management of these services, which may be
written in different programming languages and use different data storage
technologies [martinfowler.com]
• On the logical level, microservice architectures are defined by a
– functional system decomposition into manageable and independently
deployable components
SEIV – Microservices – Introduction
2
What is Microservices Architecture?

• 2011: First discussions using this term at a
software architecture workshop near Venice
• May 2012: microservices settled as the most
appropriate term
• March 2012: “Java, the Unix Way” at 33rd
degree by James Lewis
• September 2012: “μService Architecture“ at
Baruco by Fred George
• All along, Adrian Cockroft pioneered this
style at Netflix as “fine grained SOA”
3
History
James Lewis
Fred George
Adrian Crockcroft

• Area of consideration
– Web systems
– Built collaboratively by several development teams
– With traffic load that requires horizontal scaling (i.e. load
balancing across multiple copies of the system)
• Observation
– Such systems are often built as monoliths or layered systems
4
The Problem

• One build and deployment unit.
• One code base.
• One technology stack (Linux, JVM, Tomcat, Libraries).
• Benefits
– Simple mental model for developers
• one unit of access for coding, building, and deploying
– Simple scaling model for operations
• just run multiple copies behind a load balancer
5
Software Monolith

• Huge and intimidating code base for developers.
• Development tools get overburdened
– refactorings take minutes
– builds take hours
– testing in continuous integration takes days
• Scaling is limited
– Running a copy of the whole system is resource intense
– It doesn’t scale with the data volume out of the box
• Deployment frequency is limited
– Redeploying means halting the whole system
– Redeployments will fail and increase the perceived risk of deployment
6
Problems with Software Monolith

7
Monolith Architecture
Load Balancer
Monolithic App
Account
Component
Catalog
Component
Recommendation
Component
Customer Service
Component
Database
Monolithic applications can be successful, but increasingly people are feeling
frustrations with them - especially as more applications are being deployed to
the cloud .

8
Monolith vs Microservices

• A layered system decomposes a monolith into
layers.
• Usually: presentation, logic, data access.
• At most one technology stack per layer
– Presentation: Linux, JVM, Tomcat, Libs, EJB client,
JavaScript
– Logic: Linux, JVM, EJB container, Libs
– Data Access: Linux, JVM, EJB JPA, EJB container, Libs
• Benefits
– Simple mental model, simple dependencies
– Simple deployment and scaling model
9
Layered Systems

• Still huge codebases (one per layer).
• ... with the same impact on development, building, and
deployment.
• Scaling works better, but still limited.
• Staff growth is limited: roughly speaking, one team per layer works
well
– Developers become specialists on their layer.
– Communication between teams is biased by layer experience (or lack
thereof).
10
Problems with Layered Systems

Service Oriented Architecture
• Maximum reuse [GOOD]
• Maximum canonicality [GOOD] 11

• Incremental change [BAD]
• Operationally complex [BAD] 12
Problems with Service Oriented Architecture

13
Lot of Services!!!

• Applications and teams need to grow beyond the limits imposed by
monoliths and layered systems, and they do in an uncontrolled way.
• Large companies end up with landscapes of layered systems that
often interoperate in undocumented ways.
• These landscapes then often break in unexpected ways.
• How can a company grow and still have a working IT architecture
and vision?
– Observing and documenting successful companies (e.g. Amazon, Netflix) lead
to the definition of microservice architecture principles.
14
Growing Beyond Limits

15
Need of an Architectural Pattern

Yesterday’s best practice is tomorrow’s
anti-pattern.
We inadvertently build architectures to
solve outdated problems.
16
Problem with Architectural Patterns

“organizations which design systems ...
are constrained to produce designs which
are copies of the communication
structures of these organizations”
—Melvin Conway
17
Conway’s Law

18
Valued Business Drivers
Better
• Resilient
• Technology
choice
Cheaper
• Test effort
• Cost of
maintaining
Faster
• Change
• Deployment
• Execution

19
Trends in Software Development
Platform as
a Service
Autonomous
teams
Continuous
Delivery
Agile
Organization
Reactive
manifesto

20
Availability
A single missing “;” brought down the Netflix website for many
hours (~2008)

• The term “micro” refers to the sizing: a microservice must be
manageable by a single development team (5-9 developers).
• Functional system decomposition means vertical slicing (in contrast
to horizontal slicing through layers).
• Independent deployability implies no shared state and inter-
process communication (often via HTTP RESTish interfaces).
• Microservice is the ﬁrst architectural style developed for post-
Continuous Delivery.
21
What is Microservices Architecture?

22
Other definitions
• Small and
focused on
doing one
thing well
• Autonomous
“Loosely coupled service
oriented architecture with
bounded contexts”
Adrian Cockcroft (Netflix)
“SOA done right”
Anonymous
“… services are independently deployable and
scalable, each service also provides a firm
module boundary, even allowing for different
services to be written in different programming
languages.”
Martin Fowler (Thoughtworks)

• Each microservice is functionally complete with
– Resource representation
– Data management
• Each microservice handles one resource (or verb), e.g. Clients,
Shop Items, Carts, Checkout
• Microservices are fun-sized services, as in “still fun to develop
and deploy”
• Independent Deployability is key
• It enables separation and independent evolution of
– code base
– technology stacks
– scaling
– and features, too
23
Points about Microservices Architecture

• Each service has its own software repository.
• Codebase is maintainable for developers – it fits into their
brain.
• Tools work fast – building, testing, refactoring code takes
seconds.
• Service startup only takes seconds.
• No accidental cross-dependencies between code bases.
24
Independent Codebase

• Each service is implemented on its own technology stacks.
• The technology stack can be selected to fit the task best.
• Teams can also experiment with new technologies within a
single microservice.
• No system-wide standardized technology stack also means
– No struggle to get your technology introduced to the canon
– No piggy-pack dependencies to unnecessary technologies or libraries
– It‘s only your own dependency hell you need to struggle with
• Selected technology stacks are often very lightweight
– A microservice is often just a single process that is started via
command line, and not code and configuration that is deployed to a
container.
25
Independent Technology Stack

• Each microservice can be scaled independently
• Identified bottlenecks can be addressed directly
• Data sharding can be applied to microservices as needed
• Parts of the system that do not represent bottlenecks can
remain simple and un-scaled
• Microservices can be extended without affecting other
services
– For example, you can deploy a new version of (a part of) the UI
without re-deploying the whole system
– You can also go so far as to replace the service by a complete rewrite
– But you have to ensure that the service interface remains stable
26
Independent Scaling

• Communication between microservices is often standardized
using
• HTTP(S) – battle-tested and broadly available transport
protocol
• REST – uniform interfaces on data as resources with known
manipulation means
• JSON – simple data representation format
REST and JSON are convenient because they simplify interface
evolution
27
Inter-communication

• Bounded Context is a central pattern in Domain-Driven Design.
• It is the focus of DDD's strategic design section which is all about dealing with
large models and teams.
• DDD deals with large models by dividing them into different Bounded Contexts
and being explicit about their interrelationships.
28
Bounded Context

• In computing, a polyglot is a computer program or script written in a valid form
of multiple programming languages, which performs the same operations or
output independent of the programming language used to compile or interpret it
• Generally polyglots are written in a combination of C (which allows redefinition
of tokens with a preprocessor) and a scripting language such as Lisp, Perl or sh.
• Complex applications combine different types of problems, so picking the right
language for the job may be more productive than trying to fit all aspects into a
single language.
• One of the interesting consequences of this is that we are gearing up for a shift
to polyglot persistence- where any decent sized enterprise will have a variety of
different data storage technologies for different kinds of data.
• This polyglot affect will be apparent even within a single application, a complex
enterprise application uses different kinds of data, and already usually integrates
information from different sources.
29
Polyglot Persistence

• Features:
– Distributed architecture
– separately deployed components
– Service components
– Bounded context
– Service orchestration
30
Microservice Architecture (1)

31
Microservice Architecture (2)
Load Balancer
Account Service Catalog
Service
Recommendation
Service
Customer Service
Service
Catalog
DB
API Gateway
Customer
DB

32
Simple and Lightweight

33
Independent Processes

34
Language Agnostic APIs and Decoupled Services
"Language agnostic" describes a software development paradigm where a
particular language is chosen because of its appropriateness for a particular task
(taking into consideration all factors, including ecosystem, developer skill-sets,
performance, etc.), and not purely because of the skill-set available within a
development team.

35
Size matters (not)
• It’s not about
lines of code used
• Small enough to
– Be handled by a
single team
– Not feel “big”

36
Slicing
Change your approach to vertical partitions
• Modeled after organization’s domains
• Owned by team
• Top to bottom
• Isolated from each other as much as possible

37
From monolithic to microservices

SEIV – Microservices – SOA vs Microservices
ACID vs BASE
38

Multiple services work together as a system
39

SOA vs Microservices
• What “Traditional” SOA got Right
– Breaking monoliths into services
– Focus on integration over internal coupling
– Prefer BASE to ACID
• What “Traditional” SOA Missed
– Architecture is abstract until operationalized.
– Impact of Conway’s Law
– Didn’t support easy change (ESB pattern)
– It is often thought as decomposition within tiers, and introducing
another tier – the service orchestration tier
40

SOA vs Microservices
41
Microservices Focus on Change
The API is
the
contract
…And the
product
“This is
what I
need…”
“Here is
what I
have to
offer…”
WSDL is
the
Contract
Backend
App is the
Product
APIManagement
Public APIs:
Better Customer
Experience
SOA:
Better Apps +
Integration
Microservices:
Improved Developer
Experience

SEIV – Microservices – Characteristics
Characteristics
• Componentization via Services
• Organized around Business Capabilities
• Products not Projects
• Smart endpoints and dumb pipes
• Decentralized Governance
• Decentralized Data Management
• Infrastructure Automation
• Design for failure
• Evolutionary Design
42

Componentization via Services
• What is a component? We think about how it relates to software
instead of software itself
– Independently Replaceable, Independently Updatable
• How we can do this: Libraries or Services?  We do with services
• Interaction mode: share-nothing, cross-process communication
• Independently deployable (with all the benefits), Explicit, REST-based
public interface
• Sized and designed for replaceability
– Upgrading technologies should not happen big-bang, all-or-nothing-style
• Downsides
– Communication is more expensive than in-process
– Interfaces need to be coarser-grained
– Re-allocation of responsibilities between services is harder 43

Organized around Business Capabilities (1)
• The microservice approach to division is different,
splitting up into services organized around business
capability.
• Such services take a broad-stack implementation of
software for that business area, including user-interface,
persistant storage, and any external collaborations.
• Consequently the teams are cross-functional, including
the full range of skills required for the development:
user-experience, database, and project management.
• Line of separation is along functional boundaries, not
along tiers.
44

45
Will this work?

46

Products not Projects
• Most application development efforts that we see use a project
model: where the aim is to deliver some piece of software which is
then considered to be completed.
• On completion the software is handed over to a maintenance
organization and the project team that built it is disbanded.
• Microservice proponents tend to avoid this model, preferring instead
the notion that a team should own a product over its full lifetime.
• A common inspiration for this is Amazon's notion of "you build, you
run it" where a development team takes full responsibility for the
software in production.
• This brings developers into day-to-day contact with how their
software behaves in production and increases contact with their
users, as they have to take on at least some of the support burden.
47

Smart endpoints and dumb pipes (1)
48
What’s the problem here?

Smart endpoints and dumb pipes (2)
• Applications built from microservices aim to be as decoupled and as
cohesive as possible - they own their own domain logic and act more as
filters in the classical Unix sense - receiving a request, applying logic as
appropriate and producing a response.
• These are choreographed using simple RESTish protocols rather than
complex protocols such as WS-Choreography or BPEL or orchestration by a
central tool.
• Microservice teams use the principles and protocols that the world wide
web (and to a large extent, Unix) is built on.
• Often used resources can be cached with very little effort on the part of
developers or operations folk.
• Uses HTTP request-response with resource API.
• Messaging over a lightweight message bus. The infrastructure chosen is
typically dumb (dumb as in acts as a message router only) - simple
implementations such as RabbitMQ or ZeroMQ don't do much more than
provide a reliable asynchronous fabric - the smarts still live in the end points
that are producing and consuming messages; in the services. 49

Decentralized Governance
• Principle: focus on standardizing the relevant parts, and leverage battle-
tested standards and infrastructure
• Rather than use a set of defined standards written down somewhere on
paper, prefer the idea of producing useful tools that other developers can
use to solve similar problems to the ones they are facing.
• These tools are usually harvested from implementations and shared with a
wider group, sometimes, but not exclusively using an internal open source
model.
• What needs to be standardized
– Communication protocol (HTTP)
– Message format (JSON)
• What should be standardized
– Communication patterns (REST)
• What doesn‘t need to be standardized
– Application technology stack
50

Decentralized Data Management
• Each service can choose the persistence solution that
fits best its
– Data access patterns
– Scaling and data sharding requirements
• Only few services really need enterprisey persistence
• Structured programming – shared global state is a bad thing!!!
• Microservices prefer letting each service manage its own database, either different
instances of the same database technology, or entirely different database systems -
an approach called Polyglot Persistence. You can use polyglot persistence in a
monolith, but it appears more frequently with microservices.
51

Infrastructure Automation• Having to deploy significant number of services
forces operations to automate the infrastructure for
– Deployment (Continuous Delivery)
– Monitoring (Automated failure detection)
– Managing (Automated failure recovery)
• Consider that:
– Amazon AWS is primarily an internal service
– Netflix uses Chaos Monkey to further enforce
infrastructure resilience
52

Infrastructure as a code practices
1. Use definition files
2. Self-documented systems and processes
3. Version all things
4. Continuously test systems and processes
5. Small changes rather than batches
6. Keep services available continuously
53

Design for Failure• A consequence of using services as components, is that applications
need to be designed so that they can tolerate the failure of services.
• Since services can fail at any time, it's important to be able to detect
the failures quickly and, if possible, automatically restore service.
• Microservice applications put a lot of emphasis on real-time
monitoring of the application, checking both architectural elements
(how many requests per second is the database getting) and business
relevant metrics (such as how many orders per minute are received).
• Semantic monitoring can provide an early warning system of
something going wrong that triggers development teams to follow up
and investigate.
• Microservice teams would expect to see sophisticated monitoring and
logging setups for each individual service such as dashboards showing
up/down status and a variety of operational and business relevant
metrics 54

Evolutionary Design
• The key property of a component is the notion of independent replacement
and upgradeability - which implies we look for points where we can imagine
rewriting a component without affecting its collaborators.
• Indeed many microservice groups take this further by explicitly expecting
many services to be scrapped rather than evolved in the longer term.
• The Guardian website is a good example of an application that was designed
and built as a monolith, but has been evolving in a microservice direction.
• The monolith still is the core of the website, but they prefer to add new
features by building microservices that use the monolith's API.
• This approach is particularly handy for features that are inherently
temporary, such as specialized pages to handle a sporting event.
• Such a part of the website can quickly be put together using rapid
development languages, and removed once the event is over.
• Similar approaches at financial institutions where new services are added
for a market opportunity and discarded after a few months or even weeks.
55

SEIV – Microservices – Why and When
Why use Microservices Architecture
• Faster and simpler deployments and rollbacks
– Independent Speed of Delivery (by different teams)
• Right framework/tool/language for each domain
• Greater Resiliency
– Fault Isolation
• Better Scaling
– A microservices architecture puts each element of functionality
into a separate service and scales by distributing these services
across servers, replicating as needed.
• Better Availability
– If architected right
56

When to use Microservices Architecture (1)
• When you need to support Desktop, web , mobile, Smart TVs,
Wearable, etc... or you don't know in future which kind of devices you
need to support.
• You are developing a server-side enterprise application. It must
support a variety of different clients including desktop browsers,
mobile browsers and native mobile applications. The application might
also expose an API for 3rd parties to consume. It might also integrate
with other applications via either web services or a message broker.
The application handles requests (HTTP requests and messages) by
executing business logic; accessing a database; exchanging messages
with other systems; and returning a HTML/JSON/XML response.
• You have lot of people in your organization and organize them in way
that make sense/break teams into useful group.
57

When to use Microservices Architecture (2)
58
microservices approach without good cause - Microservice Envy

Microservice Premium
59
Microservice Premium

SEIV – Microservices – Challenges
Microservices Challenges
60
Can lead to chaos if not designed right …

Complexity
• Distributed Systems are inherently Complex
– N/W Latency, Fault Tolerance, Retry storms ..
• Operational Overhead
– TIP: Embrace DevOps Model
61

Service Discovery (1)
• 100s of MicroServices
– Need a Service Metadata Registry (Discovery
Service)
62
Account Service Catalog
Service
Recommendation
Service
Customer Service
Service
X
Service Y
Service
Z
Service
Registry
Service
(e.g. Netflix Eureka)

Service Discovery (2)
63

Chattiness (and Fan Out)
• The biggest issue in changing a monolith into microservices lies in
changing the communication pattern.
• A naive conversion from in-memory method calls to RPC leads to chatty
communications which don't perform well.
• Instead you need to replace the fine-grained communication with a
coarser -grained approach.
64
2 Billion Requests per day on Edge Service
Results in ~20 Billion Fan out requests in ~100 MicroServices
1 Request 1 Request
Monolithic App MicroServices

Microservice Silo Architecture
65

Testing
• A single microservice isn‘t the whole system.
• A clear picture of upstream and downstream
services is needed for integration testing
• Refer to deck: http://martinfowler.com/articles/microservice-testing/
66

Prerequisites (1)
• Before you go into production with a microservices system,
you need to ensure that you have key prerequisites in place
– Rapid Provisioning
• Dev teams should be able to automatically provision new
infrastructure
– Basic Monitoring
• Essential to detect problems in the complex system
landscape
– Rapid Application Deployment
– Devops Culture
67microservices approach without good cause - Microservice Envy

Prerequisites (2)
• These are optional things will improve your microservices
system
– Traceable Business Transactions
– Continuous Delivery
– Product-centered teams
– Multi-Dev Environment
– Infrastructure as code
– Have experiences and Knowledgeable about Domain-Driven
Design
68

Versioning (1)
• With a monolith any changes require a full build and
deployment of the entire application.
• With microservices, however, you only need to redeploy the
service(s) you modified.
• This can simplify and speed up the release process.
• The downside is that you have to worry about changes to one
service breaking its consumers.
• The traditional integration approach is to try to deal with this
problem using versioning, but the preference in the
microservice world is to only use versioning as a last resort. 69

Versioning (2)
• Microservice architectures enable independent evolution of
services – but how is this done without breaking existing
clients?
• There are two answers
– Version service APIs on incompatible API changes
– Using JSON and REST limits versioning needs of service APIs
• Versioning is key
– Service interfaces are like programmer APIs – you need to know which
version you program is in.
– As service provider, you need to keep old versions of your interface
operational while delivering new versions.
70

API compatibility
• There are two types of compatibility
• Forward Compatibility
– Upgrading the service in the future will not break existing clients
– Requires some agreements on future design features, and the design
of new versions to respect old interfaces
– imagine processing new messages arriving to the service with new
data.
– Postel’s law: ‘be conservative in what you do, be
liberal in what you accept from others’
• Backward Compatibility
– Newly created service is compatible with old clients
– Requires the design of new versions to respect old interfaces
The hard type of compatibility is forward compatibility!
71

Further Challenges
• Transactions
– Instead of distributed transactions, compensations are used (as in
SOA)
• Authentication
– Is often offloaded to reverse proxies making use of authentication
(micro)services
• Request logging
– Pass along request tokens
– Add them to the log
– Perform log aggregation
72

SEIV – Microservices – Benefits
Benefits of Microservices
• Asynchronicity
• Integration & Disintegration
• Complected (intertwined)Deployments
• Evolutionary Architecture
• Reduction of release risk
• Backends for Frontend
73

SEIV – Microservices – Recommendations
Inverse Conway Maneuver
74
Build teams that look like the architecture you want
(and it will follow).

Efferent Coupling
• Strive for low efferent coupling for your team.
• Efferent Coupling (Ce) is also known as Outgoing
Dependencies or the Number of Types inside a Package that
Depend on Types of other Packages. Ce is a metric for
packages. In short, this measure includes all the types within
the source of the measured package referring to the types not
in the measured package.
• Teams with low efferent coupling deliver relatively
independently into a common integration pipeline (without
fearing breaking each others builds.
75

Do choreography instead orchestration
• What is service orchestration? It's the process of managing
different services and the dependencies between them
such that we promote the principles of loose coupling.
• What is choreography? Choreography is the process of
agreeing, between the dancers, how a dance will be
performed before it is performed.
• One thing with micro-services that people get caught up on
is how to do orchestration. The answer is don't. Instead of
doing orchestration, do choreography.
76

Low coupling and high cohesion
• Low coupling and high cohesion.
• Coupling: Measurement of dependency between
two components. Expressed as sensitivity
towards propagation of changes and errors.
• Cohesion: Degree to which parts of a module
belong together.
77
Coincidental
cohesion
(worst)
• random
Logical
cohesion
• same in nature
Temporal
cohesion
• executed at a
particular time
Procedural
cohesion
• part of fixed
execution
sequence
Informational
cohesion
• operating on the
same data
Sequential
cohesion
• Output of one is
used as input by
another
Functional
cohesion
(best)
• contributing to a
single task

Monolith First
• Teams using a microservices architecture have a common
pattern.
• Almost all the successful microservice stories have started with a
monolith that got too big and was broken up
• Almost all the cases a system that was built as a microservice
system from scratch, it has ended up in serious trouble.
• This pattern has led many of people to argue that you shouldn't
start a new project with microservices, even if you're sure your
application will be big enough to make it worthwhile.
• Instead begin with a monolith, keep it modular, and split it into
microservices once the monolith becomes a problem.
78

Another solution to monoliths – Cookie cutter scaling
• Microservices is an antidote to monoliths
• Cookie cutter scaling is another antidote to monoliths, with
the right build infrastructure and DevOps.
79

SEIV – Microservices – Epilogue
Advantages of Microservices
• Each micro service is small and focused on a specific feature / business
requirement.
• Microservice can be developed independently by small team of
developers (normally 2 to 5 developers).
• Microservice is loosely coupled, means services are independent, in terms
of development and deployment both.
• Microservice can be developed using different programming language
(Personally I don't suggest to do it).
• Microservice allows easy and flexible way to integrate automatic
deployment with Continuous Integration tools (for e.g: Jenkins, Hudson,
bamboo etc..).
• The productivity of a new team member will be quick enough.
• Microservice is easy to understand, modify and maintain for a developer
because separation of code,small team and focused work.
80

Disadvantages of Microservices
• Microservice architecture brings a lot of operations overhead.
• DevOps Skill required (http://en.wikipedia.org/wiki/DevOps).
• Duplication of Effort.
• Distributed System is complicated to manage .
• Default to trace problem because of distributed deployment.
• Complicated to manage whole products when number of
services increases.
81

Who uses Microservices?
• Most large scale web sites including Twitter, Netflix, Amazon
and eBay have evolved from a monolithic architecture to a
microservices architecture.
82
Microservices at Netflix

Building Microservices
83
Microservices Resource Guide: http://martinfowler.com/microservices/
Microservices Articles: http://martinfowler.com/tags/microservices.html

Microservice Architecture

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Microservice Architecture

Similar to Microservice Architecture (20)

Recently uploaded

Recently uploaded (20)

Microservice Architecture