This document discusses patterns and practices for building real-world, event-driven microservices. It recommends that microservices be built using the F# programming language to gain benefits like increased productivity, expanded feature sets with discriminated unions and type providers, and more readable and concise code. It provides guidelines for building microservices functionally by avoiding state changes and side effects, isolating side effects, and using an external service to control the microservices' lifecycles.
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Patterns & Practices for Cloud-based Microservices
1. Patterns and Practices for
Real-world, Event-driven
Microservices
Rachel Reese | @rachelreese | rachelree.se
Jet Technology | @JetTechnology | tech.jet.com
2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
jet-microservices-cloud
3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
5. We plan to be the new Amazon.com
Launched July 22, 2015
• Both Apple & Android named our app
as one of their tops for 2015
• Over 20k orders per day
• Over 10.5 million SKUs
• #4 marketplace worldwide
• 700 microservices
Stop by the power-up lounge!
6. Azure Web sites
Cloud
services VMs Service bus
queues
Services
bus topics
Blob storage
Table
storage Queues Hadoop DNS Active
directory
SQL Azure R
F# Paket FSharp.Data Chessie Unquote SQLProvider Python
Deedle FAKE FSharp.Async React Node Angular SAS
Storm Elastic
Search
Xamarin Microservices Consul Kafka PDW
Splunk Redis SQL Puppet Jenkins
Apache
Hive
Apache
Tez
8. Microservices
An application of the single responsibility principle at the service level.
Has an input, produces an output.
Easy scalability
Independent releasability
More even distribution of complexity
Benefits
“A class should have one, and only one, reason to change.”
11. Events
Any significant change in
state that has happened
in your domain
Past tense
Immutable
Contains only relevant
data to transaction
All events should be represented as
verbs in the past tense such as
CustomerRelocated,
CargoShipped, or
InventoryLossageRecorded.
For those who speak French, it should be
passé composé, they are things that have
completed in the past.
Greg Young
14. Compare to relational
model which captures
only the latest state
change. These sets
are then related to
each other.
Event-sourced
• Event-sourced is about how you model the domain.
• An append-only sequence of events as data store.
• Keep track of all state changes.
• Can REPLAY these event streams.
22. The F# solution offers us an order of magnitude
increase in productivity and allows one developer to
perform the work [of] a team of dedicated
developers…
Yan Cui
Lead Server Engineer, Gamesys
“
“ “
25. Expanded feature set: Discriminated unions
public abstract class Transport{ }
public abstract class Car : Transport {
public string Make { get; private set; }
public string Model { get; private set; }
public Car (string make, string model) {
this.Make = make;
this.Model = model;
}
}
public abstract class Bus : Transport {
public int Route { get; private set; }
public Bus (int route) {
this.Route = route;
}
}
public class Bicycle: Transport {
public Bicycle() {
}
}
type Transport =
| Car of Make:string * Model:string
| Bus of Route:int
| Bicycle
C# F#
Trivial to pattern match on!
27. Concise & powerful code
public abstract class Transport{ }
public abstract class Car : Transport {
public string Make { get; private set; }
public string Model { get; private set; }
public Car (string make, string model) {
this.Make = make;
this.Model = model;
}
}
public abstract class Bus : Transport {
public int Route { get; private set; }
public Bus (int route) {
this.Route = route;
}
}
public class Bicycle: Transport {
public Bicycle() {
}
}
type Transport =
| Car of Make:string * Model:string
| Bus of Route:int
| Bicycle
| Train of Line:int
let getThereVia (transport:Transport) =
match transport with
| Car (make,model) -> ...
| Bus route -> ...
| Bicycle -> ...
Warning FS0025: Incomplete pattern
matches on this expression. For example,
the value ’Train' may indicate a case not
covered by the pattern(s)
C# F#
32. type Booking =
| Basic of Plane
| Combo of Combo
| FullPack of Plane * Hotel * Car
and Plane = {Outbound: DateTime; Return: DateTime; Destination: Country}
and Combo =
| ``With Hotel`` of Plane * Hotel
| ``With Car`` of Plane * Car
and Hotel = {Arrival: DateTime; Departure: DateTime; Location: Country}
and Car = {From: DateTime; To: DateTime; Location: Country}
and Country = {Name: String; ``ISO 3166-1``: char*char}
36. F# way 37
type Year = int
type [<Measure>] percent
type Customer = Simple | Valuable | MostValuable
type AccountStatus =
| Registered of Customer * since:Year
| Unregistered
let customerDiscount = function
| Simple -> 1<percent>
| Valuable -> 3<percent>
| MostValuable -> 5<percent>
let yearsDiscount = function
| years when years > 5 -> 5<percent>
| years -> 1<percent> * years
let accountDiscount = function
| Registered (customer, years) ->
customerDiscount customer, yearsDiscount years
| Unregistered -> 0<percent>, 0<percent>
let asPercent p = decimal p / 100m
let reducePriceBy discount price =
price - price * (asPercent discount)
let calculateDiscountedPrice price account =
let custDiscount, yrsDiscount =
accountDiscount account
price
|> reducePriceBy custDiscount
|> reducePriceBy yrsDiscount
41. Be functional!
Prefer immutability
Avoid state changes,
side effects, and
mutable data
Use data in data out
transformations
Think about mapping inputs
to outputs.
Look at problems
recursively
Consider successively
smaller chunks of the
same problem
Treat functions as
unit of work
Higher-order functions
42. Don’t abstract
This one magical service could write to ALL of the following:
…badly.
Event
store
Nservice
Bus
MSMQ 0MQ
SQL
Server
43. Isolate side effects
Submit order
microservice
Insert order to
SQL microservice
Send thank you
email microservice
Updates SQL
Sends “Thank you for
ordering” Email
Updates SQL
Sends “Thank you for
ordering” Email
44. Use a backup service to replay events
Service 1 runs in production as normal
Backup service 1 replays events until up-to-date.
Switch over. Instantly live with changes!
Also stage a copy of any aggregate/data store
until stream has completed replaying!
45. type Input =
| Product of Product
type Output =
| ProductPriceNile of Product * decimal
| ProductPriceCheckFailed of PriceCheckFailed
let handle (input:Input) =
async {
return Some(ProductPriceNile({Sku="343434"; ProductId = 17; ProductDescription = "My
amazing product"; CostPer=1.96M}, 3.96M))
}
let interpret id output =
match output with
| Some (Output.ProductPriceNile (e, price)) -> async {()} // write to event store
| Some (Output.ProductPriceCheckFailed e) -> async {()} // log failure
| None -> async {{ }} // log failure
let consume = EventStoreQueue.consume (decodeT Input.Product) handle interpret
What do our services look like?
Define inputs
& outputs
Define how input
transforms to output
Define what to do
with output
Read events,
handle, & interpret
46. Microservices should not control own lifecycle
Execution
runtime
Deployment Configuration Restarting
Versioning Scaling Availability
Grouping by
subsystem
Scheduling
Input-output
static analysis
Dashboard
Think
IoC!
47. Torch YAML files
torchVer: 2.0.0
subSystem: PriceCheck
name: PriceCheck
description: checks prices on nile
ver: 0.0.1
autoStart: always
compile: true
ha: aa ##active-active. Could be ap for active-passive
scriptPath: PriceCheckNilePriceCheckNile.fsx
libPath: binrelease
args: --jsonConfig=PriceCheckNile.json
It used to mean “Yet Another Markup Language” but was backronymed to clarify its focus as data-oriented.
YAML = “YAML Ain’t Markup Language”