Advanced Microservices - Greach 2015

THIRDCHANNEL
Advanced Microservice
Concerns
Steve Pember
CTO, ThirdChannel
@svpember
Greach, 2015
Let’s talk about Microservices

THIRDCHANNEL
* Before I begin, I should probably point out there’s not much in this talk that speciﬁcally relates to Groovy

* In fact, there’s no code in this talk at all. Mostly just theory.

* Still, I believe the information is useful. Microservices are all the rage.

* Incidentally, At 3C we love microservices, and Groovy is our most used language in our services

THIRDCHANNEL
Agenda
• Microservices: Overview
• Microservices: Advantages
• Service Design
• Organizational Structure
• Dev Ops
The agenda for this presentation ﬁrst is to brieﬂy cover the microservice architectural pattern, some of it’s advantages,

and then get into some of the advanced concerns, where I’ve grouped them by overall topic… Service Design, Organizational Structure, and Dev ops!

THIRDCHANNEL
Agenda
* To begin:

* Who here has heard of ‘Microservices’? Just trying to measure how to talk about this

* Who here is currently working with or has worked on a Microservices platform?

THIRDCHANNEL
Microservices
• Fight the Monolith
So let’s get started with a quick overview of what Microservices actually are.

I believe microservices are an attempt to ﬁght against a distressing pattern in software architecture: the monolithic application

THIRDCHANNEL
Building a monolith has become, I think, the natural response for many developers or companies when they ﬁrst start out on a project

THIRDCHANNEL
-A monolithic app is: Single, logical executable unit, often comprised of 3 components - at least for a web app: the View layer (CSS/HTML/JS), the middle
tier (contains business logic) and the data storage.

- here’s an example of an e-comm app’s middle tier: product browsing, catalog management, order processing, shopping cart

- as I add features to my application, I simply tack them on to the existing code base. Feature upon Feature upon feature

Feels Natural
The danger with the monolith I think, is that this feels like the natural thing to do.

You start a new project, choose a framework and begin building.

start out easy

quick wins as you add new features

“Oh, I need a shopping cart now. Great, let’s shove in a Cart model and a Controller. Maybe service”

…in the long term, though, you’ll run into some problems

NEED TO GO BACK AND ADD BITS ABOUT HOW MONOLITHS GROW

complexity grows quickly as you keep adding new features!

Even if you’re very, very careful… application will still inevitably become a mess

–Johnny Appleseed
“Type a quote here.”
To the point where it leads to Developer Misery.

Open up your IDE, only to be bogged down by the countless controllers or View objects popping out at you. Quick hacks and other Technical Debt that
promised yourself you would go back and fix … and you never do… make the system hard to develop.

When adding new features or fixing bugs, developers will spend huge amounts of time figuring out how something works before they can even start fixing a
problem. May require dragging in other people who worked on it before to help.

Microservices to the Rescue!
This frustration - of working on large, monstrous applications - , I think, is why Microservices have become so popular.

THIRDCHANNEL
Microservices
• Distributed
So, when we use that term “Microservice”, what exactly are we talking about?

Well, fundamentally what it means is that we’re building a distributed application

THIRDCHANNEL
So, instead of the monolith I showed you before, we do this:

- describe break up

What About ‘Service Oriented
Architecture’?
Now you may be saying:

“Wait, isn’t this just Service Oriented Architecture? People have done this before.”

* It’s true, this has been done before. SOA has been around for years now

* Microservices, though, are a more focused and disciplined version of SOA.

* REASON BEING: Involves technical and organizational changes not deﬁned in SOA. Let’s go over a few of them:

THIRDCHANNEL
Microservices
• Distributed
• Independent
First: Independence.

In a Microservice system, Each service should be completely independent of each other

THIRDCHANNEL
Independently developed. Each with their own separate code repository.

Independently deployed.

- allows for independent history

- results in isolated code which is great for adhering to design principles like separation of concerns.

-

THIRDCHANNEL
Microservices
• Distributed
• Independent
• Single Context
Next.

Each Service contains a single context…. Or rather, each service is responsible for One thing.

This one thing could be a particular set of data or a particular process.

This approach naturally leads to smaller, less complicated codebases

THIRDCHANNEL
Let’s say I was building an e-commerce application. I may have… <name services>

Each one responsible for a single object or a single process

Each one of them, will be developed separately from each other and will be each far less complicated than if we tried to jam each service together into a
single monolith

THIRDCHANNEL
Microservices
• Distributed
• Independent
• Single Context
• Simple, “Dumb”, Communication Technology
Your services should communicate with each other using very simplistic channels, but be very smart about how they do so.

<slow down>

No Enterprise Service Buses
Which basically means “No enterprise Service Buses” or ESBs.

You would often ﬁnd ESBs in Service Oriented Architecture or SOA implementations. They are complicated services that are responsible for ﬁguring out
how to route messages to the various services.

In practice, they Would often be very complex with custom logic; another thing to maintain

Instead, Use HTTP or Message Queues
Instead, use direct HTTP calls or Message Queues

* Http: Already in the web, so why not use a web technology? HTTP allows for Direct communication between services, but it’s synchronous… so, while
the request is active, both services have resources that locked in the communication

* Message Queues on the other hand: Asynchronous, but require some additional code or library for messaging formats

* Regardless, Each approach does require a lightweight program to help with the communication.

THIRDCHANNEL
So with HTTP based comms and a handful of services, you may end up with something like this:

web of inter service communications. Each service needs to know the ‘address’ of the other services, along with when it needs to talk to the others.

So, if a user places an order, that code needs to know to talk to the order history service and the billing service in a particular order.

THIRDCHANNEL
* With HTTP, you’ll need a service to help you discover other services, otherwise each service would also have to maintain a huge internal map of the
locations of the other services.

* A “Service Discovery Service”

* Netﬁx has open sourced one, called “Eureka” which is very useful.

* With all of that, HTTP is very commonly used in Microservice Architectures.

THIRDCHANNEL
With a message Oriented approach on the other hand..

- services don’t know where each other are, nor do they care where the other services are.

- communicate entirely through the the Message Broker

- When an event needs to go out, it is simply generated. The services do not need to individually worry about which other service needs this information.

- Personally, I prefer this approach for a variety of reasons, which I’d be happy to talk about after.

THIRDCHANNEL
Regardless, With MQ you’ll need a Message Broker to quickly and eﬃciently route your messages.

I recommend either:

THIRDCHANNEL
Microservices
• Distributed
• Independent
• Single Context
• REST-ful Communications
Your services should communicate with each other using very simplistic channels, but be very smart about how they do so.

REST != CRUD != HTTP
I feel that when many people think of REST or say that they have a “RESTful api”, they’re thinking of performing CRUD operations (create / read / update /
delete) over HTTP, using some of the HTTP verbs.

THIRDCHANNEL
There are many more aspects to REST than that.
Primarily, I think, no State should be shared across services.
Attempting to keep shared state across distributed services WILL drive you crazy.

Caching, Layering, and Uniform
Interface
the REST speciﬁcation also calls for caching your data, layering your services, and ensuring they have a uniform interface.

- Layering: when your services ask for or push information, they shouldn’t care whether who they’re talking to is the actual service or some intermediary
that’s then passing it on. As long as the data is handled, the services should trust that the handlers know what to do.

- Uniform interface implies many things… I’m going to take a shortcut and just say that each of your services should communicate with each other via the
same technique. If anyone wants to argue about HATEOAS later, I’d be happy to.

THIRDCHANNEL
Microservices
• Distributed
• Independent
• Single Context
• REST-ful Communications
• Dedicated, Cross-Purpose Teams
lastly,

… from an organizational standpoint, the teams building the service should be cross-purpose.

THIRDCHANNEL
Teams in charge of a service should come from a variety of disciplines. Engineering, QA, Art & Design, Project Management, perhaps a deployment
engineer, etc.

THIRDCHANNEL
Agenda
* Next on the agenda: Microservice architectures have been gaining huge followings in the past few years due to the beneﬁts they bring.

THIRDCHANNEL
Microservices: Advantages
• Loose Coupling!
Because our systems are independent of each other, this promotes natural Loose Coupling. Individual teams can develop the services independently,
which keeps complexity down.

THIRDCHANNEL
• Loose Coupling!
• Right Tool for the Job
* Each service can be built according to the technology best suited to it.

* Not locked into one language or framework

Always use Groovy, naturally
But of course, Groovy is always the right tool for the job!

THIRDCHANNEL
• Loose Coupling!
• Easy to Scale Services
It’s easy to scale Microservice applications to meet the particular user demand

–Johnny Appleseed
Borrowed this from Martin Fowler’s website

As demand on the application grows, I can scale just the components that are needed, rather than needing to replicate the whole monolith
* targeted growth results in your team using fewer resources
* which saves $$$ and makes your team more efﬁcient

THIRDCHANNEL
• Loose Coupling!
• Easy to Scale Development
Speaking of scaling…

Scaling development in general is very diﬃcult. With a microservices architecture, you have an advantage where it’s relatively easy for developers.

Because the code base is smaller, your developers can hold the context of a service in their head at once (something impossible with a monolith). This
allows them to ﬁnd bugs quickly and identify exactly where new features should go.

THIRDCHANNEL
• Loose Coupling!
• Fast to Fail / Start Over
Microservices allow us to fail fast.

Find out quickly of a technology or a service isn’t working.

Easy to refactor or replace a service

…

THIRDCHANNEL
• Loose Coupling!
• Fast to Fail / Start Over
• Allows for Continuous Delivery
Finally, Adopting a Microservices approach allows for your team to Continuously Deliver your code.

THIRDCHANNEL
Which, I think, is the absolute ultimate goal of any development team. I think it’s the main reason to use microservices to begin with.

The ability to basically make a change or implement a feature and then immediately have it running in production? It’s thrilling, and powerful, and liberating
for your developers.

Furthermore, It essentially eliminates the notion of a Sprint and dramatically changes Agile Development.

Goodbye, Release Windows
Who here has worked in teams where you have a development release window or a ‘release train’? I have. Who here was worked in companies where you
release a new build every 3 or 4 weeks… or longer? I certainly

It was miserable, it was depressing, to stack up a bunch of features for release. Everyone was a little unsure if the changes they made broke other people’s
code. If a bug was detected in the wild, it wouldn’t be ﬁxed until the next release. Madness.

Microservices Have
Disadvantages
That all being said, Microservices do have their downsides. Like any new buzzword or new hot thing in tech, people will complain about it or point out any
problems it has.

Microservices do have a few complaints, although I don’t agree with all of these.

Some of the complaints I’ve seen have been:

THIRDCHANNEL
Microservices: Problems
• Distributed Systems are Hard
Working with distributed systems is diﬃcult. Or really, it’s diﬃcult to wrap your mind around a distributed system.

And yeah, it can be hard to worry about multiple small services.

THIRDCHANNEL
• Added Complexity
Splitting your code into services can add additional operational complexity, as now we have more moving parts in our platform.

I disagree with this one, breaking your code down reduces programming & computational complexity

THIRDCHANNEL
• “What if a service goes down?”
You may here someone in your organization say something like “What if service X goes down? It’s central to our business!”

Well, That person is not ready for microservices.

THIRDCHANNEL
• “What if a service goes down?”
• Signiﬁcant Operations / Dev Ops Work
Microservices will require some additional focus on dev ops related work, like streamlining deployments.

These Can All Be Mitigated
These points, while valid, can be addressed, reduced, or handled pretty well. These next sections cover some of these ‘Advanced’ concerns when working
with Microservices. These are things that everyone should be aware of.

THIRDCHANNEL
Agenda
• Service Design
* First up: Service design!

* As we’re building our individual services, what should we be aware of?

THIRDCHANNEL
Service Design
• Single Bounded Context
I’ll reiterate here that each service needs to be a single bounded context.

It’s a bit tricky, but this point basically states how large a service should be… which is: a service should be small enough that a single developer can hold
the entire functionality of it within their head.

THIRDCHANNEL
Service Design
• Resilient
your services should be resilient

they should embrace and welcome that errors will occur within the platform… and they should know how to handle any potential error.

For example, if our order billing system goes down… instead of shutting down the whole website, we simply accumulate orders, allow our users to go on
as normal, and hope the system comes back up eventually.

Independent Things Fail
Independently
The overal rule I think, is that your services should maintain their independence… and fail independently.

THIRDCHANNEL
Service Design
• Resilient
• Fast and Eﬃcient
Design your services to be fast and eﬃcient.

–Johnny Appleseed
You should strive to return a result to your Request as fast as you can. In the US we have an idiom called “hot potato” which means to get read of
something as fast as you can.

It’s important that your back end services return a result as quickly as possible in order to

Speed is king.

Several years ago, an engineer with Amazon named Greg Linden revealed that the company ran experiment where they intentionally slowed down site
performance

- 100ms delay resulted in 1 percent drop in sales.

THIRDCHANNEL
Service Design
• Resilient
• Each Service is an Authority
Each service should be responsible for one particular thing within your application. This relates to single bounded context.

Each service should be an Authority on some data… or an authority on a particular process. For example, I may have a ‘User Service’ that is the source of
information on Users within my system… or I could have an Email Notiﬁcation Service which is responsible for sending emails

THIRDCHANNEL
Service Design
• Resilient
• Embrace Eventual Consistency
Embrace Eventual consistency. Does everyone know about this? basically, in a distributed system, you must accept that an individual client or service may
not have the most recent view of the data… but that eventually, every service will have data consistent with the authority service.

This is a very large subject and I’d love to talk about some of the nuances about it later.

THIRDCHANNEL
Service Design
• Resilient
• CAP Theorem
These next two topics are perhaps the most diﬃcult in this talk. At least, very diﬃcult to explain.

First up, the CAP Theorem

THIRDCHANNEL
CAP Theorem
• (C)onsistency
• (A)vailability
• (P)artition Tolerance
The CAP Theorem states that there are 3 three attributes or guarantees that a Distributed System can have:

- consistency: when an update occurs in our system, all services know about it at the same time (which is the opposite of eventual consistency)

- availability: when information is asked of the service, it responds as quickly as it can

- partition tolerance: a service is able to handle or deal with failures occurring in other services

THIRDCHANNEL
CAP Theorem
• (C)onsistency
• (A)vailability
• Pick 2!
Well, the CAP Theorem says that’s great, and in fact you can only have two of these. A distributed system can only meet two of these guarantees.

THIRDCHANNEL
CAP Theorem
• (C)onsistency
• (A)vailability
• Pick 2!
• Theorem from Paper which proved “Brewer’s Conjecture”
Originally called Brewer’s Conjecture

THIRDCHANNEL
In 2012, this paper contained a formal proof of the Conjecture and it graduated to full-on theorem.

THIRDCHANNEL
CAP Theorem
• (C)onsistency
• (A)vailability
• Pick 2!
• Theorem from Paper which proved “Brewer’s Conjecture”
• CAP Decisions can be Service-Speciﬁc

Web Apps -> A & P
Banks / Money -> C

THIRDCHANNEL
Service Design
• Single Bounded Design
• Resilient
• CAP Theorem
• Locality of Reference (Data Locality)
Last… let’s talk about Data Locality.

You should be very concerned about Data Locality

THIRDCHANNEL
Data Locality
• Spatial: “How far away or how distant is our data?”
There are two aspects of data locality:

1. Spatial. How proximal is our data? In a single data base, data can be considered highly spatial if it’s in the same table. Data which is reachable by joins is
less spatial and perhaps less eﬃcient to reach

THIRDCHANNEL
Data Locality
• Temporal: “How often is our data accessed?”
2. Temporal. Being highly temporal means that data is read frequently. Highly temporal data - is an excellent candidate for caching.

These terms take on slightly diﬀerent meanings in a distributed system, though.

THIRDCHANNEL
In an ideal distributed environment, each system would be completely separate.
-However, there will always be some conceptual overlap between domain objects. For example, let’s say we have an ecommerce system where
<describe system>.
-Red lines are anchors between data localities
- Here, both the user service and the O history service maintain a concept of user. But, users’ uuid is a shared natural key which acts as each
services’ anchor to understanding the concept of a User.
- Our user service is the authoritative source on a User
- Our order history system also understands the User, but simply maintains the order history along with the user uuid of the user placing the order

THIRDCHANNEL
Same chart from before, but another service, communication via email. This service also conceptualizes the user, but with a diﬀerent subset of
information. Namely, the email address.
- Email address is stored in the user information service, but is *also* used extensively by the communication service. Which one should be the
authoritative source of the email address? The communication service uses it more (and thus has higher Temporal locality), but email is a key piece
of information which should be managed by the user service thus keeping it Spatial close to the other user data.
- Tough Choice
- One Compromise is to (unfortunately) synchronize the email address across multiple services
- Or, communication service has no concept of User’s email address, but simply blindly sends emails out. The email addresses involved must be
placed into each message received by the Communication service
- Or, third option, perhaps we cache the email address on the communication service and wait for an event or notiﬁcation from the user service
when it changes (Eventual consistency, eh? When the user changes their email address, the communication service eventually, busts the cache and
resets the email address. The user may miss an email until that happens, and that may be ok).

THIRDCHANNEL
Data Locality
• Embrace Eventual Consistency, Do not sync data!
Remember the point from earlier though… embrace eventual consistency

My recommendation is to never sync data. It will likely be inevitable that you’ll eventually have to synchronize something

Do NOT Synchronize
I believe that synchronizing data will eventually always break down and cause no end of problems. Inevitably data will become out of sync; you will have to
create some jobs or processes to re-sync any data which has ‘drifted’.

We’ve tried synchronizing data at ThirdChannel in the past, and data discrepancies were the the primary cause of bug reports. It was terrible.

If you overhear anyone on your team say “Well, we *could* just synchronize these two things”, roll up a newspaper and whack them on the nose like a dog.

THIRDCHANNEL
Data Locality
• Embrace Eventual Consistency, Do not sync data!
• Goal: Have Highly Spatial Data & Eﬃciently Handle Temporal Data
With a MS, your goal is to have..

THIRDCHANNEL
Agenda
• Service Design
* Next,

THIRDCHANNEL
Organizational Changes
• Cross-Discipline Teams

THIRDCHANNEL
DBA
Engineers
QA
UX
In many companies, it’s a common pattern to have your staﬀ broken up into massive departments: a QA department, a UX team, your engineers, and the
dreaded DBAs…

Instead, with microservices, it is encouraged to break up that thinking. Build teams around your services such that it’s comprised of multiple people of
multiple disciplines.

One of the best working experiences I had was with an awesome QA person that sat right next to me… ﬁnish story

THIRDCHANNEL
• Dedicated & Responsible
With that, each team is ultimately responsible for their individual service(s).

When I say I responsible, I mean it. If the service malfunctions over the weekend, the people on that team should be receiving notifications about it. There’s
not a team of junior engineers or interns around to fix it. If you commit code, you’re responsible for fixing it.

What ends up happening, is that the engineers end up writing a hell of a lot of unit and integration tests in order to avoid the dreaded early am phone call.

THIRDCHANNEL
• Autonomous
With all that, the engineering teams should be autonomous…

–Johnny Appleseed
-But, They also need to be aligned well with the business. Essentially, provide the end goal, and let the team ﬁgure out how best to get there.

-I borrowed this drawing from a Spotify blog post on their engineering culture, and it shows the concept of a highly Aligned, highly Autonomous team.

Describe the top right

THIRDCHANNEL
• Autonomous
• Microservice Champion(s)
You will need some architects or team leads to champion the micro service vision.

* These people will need to have the vision to direct people. To guide them. This leadership team will need to have a shared vision and design from which
they can guide the rest of the company.

*If you start down the Microservices path, you will have some people in house who disagree with it. They’ll complain for various reasons I mentioned earlier.

*What they’re really saying is that this new approach is too hard and they’re afraid. The leadership team must be there to guide people forward and to
educate.

* And to encourage communication

THIRDCHANNEL
• Autonomous
• Microservice Champion(s)
• Conway’s Law
which brings us to Conway’s Law.
I talked about Conway’s law last year.. I think at Great.
I was at a conference a couple weeks ago, and it was the subject of several talks.
So… I just want to tell you all I’m a bit of a trendsetter, leading the pack.

-Melvin Conway, 1968
http://en.wikipedia.org/wiki/Conway's_law
“organizations which design systems ... are constrained to
produce designs which are copies of the communication
structures of these organizations”
Somewhat tongue and cheek, but basically says that the structure of any given computer system will begin to reﬂect the social structures of the
company that built it.
While yes, teams should be organized around services, these services still need to communicate with one another.

If Conway’s Law is not observed, some interesting things can happen.

You may end up with situations where:

- intra service APIs not matching up at all

- teams expecting data to be in services, that may not be there. A misunderstanding of what a service provides

- team A waiting for Team B to ﬁnish service without bothering to help them

- eventually you may get a point where teams start blaming or attacking each other for problems in the system. There can be arguments. “Man, the team
working on the Shopping Cart Service is terrible. Their code is always broken”

Hard Problem to Solve
Avoiding Conway’s Law is not easy, and there’s no clear way to do it. Try your best to encourage communication and collaboration between teams.

Rotate people between service teams

Get everyone on a shared communication channel (Slack is a popular choice)

Ensure that each service’s API is published and highly visible to other teams.

THIRDCHANNEL
Agenda
• Service Design
• Dev Ops
* Next up, Dev ops

Because each team is responsible for their deployments, nearly everyone in your team is going to get some Dev Ops experience.

Whether they like it or not….

However, there are several things you can do to make the Dev Ops experience better, and make it easier to manage your application

THIRDCHANNEL
Dev Ops
• Centralized Service Monitoring & Logging

THIRDCHANNEL
Receives Metrics, Logs and Health checks
One absolute requirement of a successful microservices deployment is to have detailed monitoring of your services.

This monitoring service is responsible for performing heartbeat checks on your services and gathering metrics about them. It should also be responsible
acting as a centralized logging platform for each of your services.

If you think this may end up being an absolute ton of data… well, you’re right.

THIRDCHANNEL
There’s many applications or paid services available to store and visualize all of this data. However, many people I speak to recommend the Open Source
‘ELK stack’. Which is LogStash for parsing your logs, elastic search for enabling search on your data, and Kibana for visualization

No SSH == You’re doing it right
If you can get data from services and be aware of what they’re doing… and you *never* have to ssh into your services… well, then, you’re doing it right.

I read somewhere that some companies will actually block the ssh ports (22?) on their instances or their Amazon VPC to prove this point

THIRDCHANNEL
Dev Ops
• Standardize your Deployment Process
Even if your services use diﬀerent languages or frameworks, do your best to make sure everyone is using the same general approach to deployment.

THIRDCHANNEL
Core OS’s rocket

THIRDCHANNEL
Dev Ops
• Small, Fast Deployments

THIRDCHANNEL
Dev Ops
• Small, Fast Deployments
• Continuous Deployment!
With continuous Deployment, your engineers should be releasing as soon as things are ready. Push a feature when it’s complete.

Beware “I have to deploy Service A before
Service C or else Service B will break”
Wasn’t sure how to diagram this.

The only thing that should concern your team are the moments where a release has dependencies on other services… E.g. I have to deploy Service C
before I deploy service A or else service B will break.

However, these cases should be RARE

THIRDCHANNEL
The day to day deployments, though, will be amazing.

It may be a lot of work to get to the point where you can continuous deploy, but once you get there and are able to do it… well, it feels great.

In Summary:
So that’s really all I have, but before I stop, here is a brief summary

THIRDCHANNEL
Laws of the Microservice
1. Be Independent
2. Communicate with your coworkers
3. Keep Data Close and Cache it Well
4. Do Not Sync
5. Monitor and Visualize
6. Work towards Continuous Deployment
Be Independent, but at the same time, Communicate with your coworkers. A well designed system requires constant communication

Keep your data close and cache it … but do not sync any data

Get some excellent monitoring in place

Reach for Continuous Deployment!

Any questions?
Steve Pember
@svpember
steve@thirdchannel.com

• Sad Dog: https://www.youtube.com/watch?v=Xw1C5T-fH2Y
• Monolith: http://commons.wikimedia.org/wiki/File:Monolith_swfc_shanghai.jpg
• Morpheus: www.desktopexchange.net/movie-pictures/matrix-wallpapers/
• Enterprise Crew: www.desktopexchange.net/movie-pictures/matrix-wallpapers/
• Mountain Climber: http://www.freshstartministries.com/just-one-more-step/
• Simpson Dev Ops: http://devops.com/blogs/devops-lessons-learned-the-devops-engineer/#!prettyPhoto
• rocky stairs: http://pixgood.com/rocky-stairs.html
• ELK Stack: https://www.elastic.co/products
• Docker deployment Example Diagram: http://blog.risingstack.com/content/images/2015/01/codeship_ansible_docker.png
Images

Advanced Microservices - Greach 2015

More Related Content

Viewers also liked

Similar to Advanced Microservices - Greach 2015

More from Steve Pember

Recently uploaded

Advanced Microservices - Greach 2015