This is my presentation from #hacksummit 2016
This presentation looks at the importance of events and the role that they play in applications. We describe how events are a key application integration mechanism and how they are used by applications to communicate with the outside world. You will learn how the microservices inside a system can use events to maintain data consistency. We discuss how easy it is to implement both of these mechanisms by developing your core business logic using an event-centric approach known as event sourcing.
#hacksummit 2016 - event-driven microservices – Events on the outside, on the inside and at the core
1. @crichardson
Events on the outside, on
the inside and at the core
Chris Richardson
Founder of Eventuate.io
Founder of the original CloudFoundry.com
Author of POJOs in Action
@crichardson
chris@chrisrichardson.net
http://microservices.io
http://eventuate.io
http://plainoldobjects.com
5. @crichardson
About Chris
Founder of a startup that is
creating
a platform that makes it easy for
application developers to write
microservices
http://eventuate.io
14. @crichardson
Who consumes an event?
Order
Application
Human
Notification
Service
Dashboard/
Monitoring
Shipping
Application
OrderCreated
Email
SMS
…
21. @crichardson
Polling for events
HTTP
Periodically poll for events
Atom Publishing Protocol (AtomPub)
Based on HTTP
Head is constantly changing
Tail is immutable and can be efficiently cached
High-latency, inefficient
23. @crichardson
Webhooks = user-defined
HTTP callback
Client Service
register(events, callbackUrl)
POST callbackUrl
POST callbackUrl
…
https://en.wikipedia.org/wiki/Webhook
Low latency, more efficient, but what
about past events?
26. @crichardson
Twilio - Telephony and SMS as
a service
Twilio Your
Application
TwiML doc
HTTP GET/
POST
REST API
Manage resources
Send SMS
Initiate voice calls
Webhooks handle incoming SMS and
voice calls
Voice
SMS
Phone number
SMS URL +VOICE URL
27. @crichardson
Integration hubs - Zapier,
IFTTT
Application abstraction:
Triggers - events published by application: polling or
Webhooks
Action - operation supported by application, e.g. REST
API end points
28. @crichardson
The event-driven enterprise
App A App BApp X
App Y
Firewall
Message Broker
webhooks
WebSockets
AtomPub
App Z App C
Msg
Broker
Client
Msg
Broker
Client
Msg
Broker
Client
App D
Msg
Broker
Client
Inside the firewallOutside the firewall
29. @crichardson
Agenda
Events on the outside
Events on the inside
Events at the core with event sourcing
Designing event-centric domain model
30. @crichardson
Events on the inside
Application
Service A
Service B
Service C
Event X
Event YEvent Z
Event X
Event Y
33. @crichardson
Today: use a microservice, polyglot
architecture
Orders
Customers
…
Shopping UI
Mobile Client
Browser
API Gateway
Order
management
REST
API
Customer
Management
REST
API
….
REST
API
Order
Database
(MongoDB)
MongoDB
Adapter
Customer
Database
(Sharded
MySQL)
MySQL
Adapter
35. @crichardson
Example: placing an order
Order Service Customer Service
Order
Database
Customer Database
Order #1
Customer #1
No 2PC
No
ACID
NoSQL SQL
36. @crichardson
Customer management
How to maintain invariants?
Order management
Order Service
placeOrder()
Customer Service
updateCreditLimit()
Customer
creditLimit
...
has ordersbelongs toOrder
total
Invariant:
sum(open order.total) <= customer.creditLimit
?
37. @crichardson
Use an event-driven
architecture
Services publish events when something important happens,
e.g. state changes
Services subscribe to events and update their state
Maintain eventual consistency across multiple aggregates
(in multiple datastores)
Synchronize replicated data
39. @crichardson
Order Management
Order
id : 4567
total: 343
state = CREATED
Customer Management
Customer
creditLimit : 12000
creditReservations: {}
Customer
creditLimit : 12000
creditReservations: { 4567 -> 343}
Order
id : 4567
total: 343
state = OPEN
Eventually consistent credit checking
Message Bus
createOrder()
Publishes:
Subscribes to:
Subscribes to:
publishes:
OrderCreatedEvent
CreditReservedEvent
OrderCreatedEvent CreditReservedEvent
42. @crichardson
Problem #2: How to atomically update
database and publish an event
Order Service
Order
Database
Message Broker
insert Order
publish
OrderCreatedEvent
dual write problem
?
49. @crichardson
Persists events
NOT current state
Event table
Entity type
Event
id
Entity
id
Event
data
Order 902101 …OrderApproved
Order 903101 …OrderShipped
Event
type
Order 901101 …OrderCreated
50. @crichardson
Replay events to recreate
state
Order
state
OrderCreated(…)
OrderAccepted(…)
OrderShipped(…)
Events
Periodically snapshot to avoid loading all events
52. @crichardson
Events at the core
Aggregate A
Aggregate B
Aggregate C
Event X
Event YEvent Z
Event Z
Event Y
Service
53. @crichardson
Domain logic = event-driven
aggregates
Customer
AggregateOrder
Aggregate
Order Created
Event Check Credit
Command
Credit Reserved
Event
Approve Order
Command
Create Order
Command
External
request
Emits events Event Command
54. @crichardson
Request handling in an event sourced application
HTTP
Handler
Event
Store
pastEvents = findEvents(entityId)
Order
new()
applyEvents(pastEvents)
newEvents = processCmd(SomeCmd)
saveEvents(newEvents) (optimistic locking)
Order Service
apply(newEvents)
55. @crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
publish(event)
publish(event)
Customer
update()
Customer Service
56. @crichardson
Event Store publishes events
consumed by other services
Event
Store
Event
Subscriber
subscribe(EventTypes)
publish(event)
publish(event)
CQRS View
update()
Service Xyz
send notifications
…
57. Event store = database + message
broker
Hybrid database and
message broker
Implementations:
Home-grown/DIY
geteventstore.com by
Greg Young
http://eventuate.io
(mine)
Event Store
Save
aggregate
events
Get
aggregate
events
Subscribe
to events
58. @crichardson
Benefits of event sourcing
Solves data consistency issues in a Microservice/NoSQL based
architecture
Reliable event publishing: publishes events needed by predictive
analytics etc, user notifications,…
Eliminates O/R mapping problem (mostly)
Reifies state changes:
Built in, reliable audit log,
temporal queries
Preserved history More easily implement future requirements
59. @crichardson
Drawbacks of event
sourcing…
Requires application rewrite
Weird and unfamiliar style of programming
Events = a historical record of your bad design decisions
Must detect and ignore duplicate events
Idempotent event handlers
Track most recent event and ignore older ones
…
60. @crichardson
… Drawbacks of event
sourcing
Querying the event store can be challenging
Some queries might be complex/inefficient, e.g. accounts with
a balance > X
Event store might only support lookup of events by entity id
Must use Command Query Responsibility Segregation (CQRS)
to handle queries application must handle eventually
consistent data
61. @crichardson
Agenda
Events on the outside
Events on the inside
Events at the core with event sourcing
Designing event-centric domain model
62. @crichardson
Use the familiar building
blocks of DDD
Entity
Value object
Services
Repositories
Aggregates ⟸ essential
63. About Aggregates
Graph consisting of a root
entity and one or more other
entities and value objects
Each core business entity =
Aggregate: e.g. customer,
Account, Order, Product, ….
Reference other aggregate
roots via primary key
Often contains partial copy
of other aggregates’ data
Order
OrderLine
Item
quantity
productId
productName
productPrice
customerId
Address
street
city
…
64. @crichardson
Domain model = collection of
loosely connected aggregates
Order
OrderLine
Item
quantity
…
Address
street
city
…
Customer
Product
name
price
66. @crichardson
Transaction = processing one
command by one aggregate
No opportunity to update multiple aggregates within a
transaction event driven eventual consistency between
aggregates
If an update must be atomic (i.e. no compensating
transaction) then it must be handled by a single aggregate
Therefore, aggregate granularity is important
69. Designing domain events
Record state changes for an
aggregate
Part of the public API of the domain
model ProductAddedToCart
id : TimeUUID
senderId: UUID
productId
productName
productPrice
…
Required by
aggregate
Enrichment:
Useful for
consumers
Event
metadata
71. @crichardson
Designing commands
Created by a service from incoming request
Processed by an aggregate
Immutable
Contains value objects for
Validating request
Creating event
Auditing user activity
81. @crichardson
Summary
Events are a central to modern applications
Events integrate applications
Events maintain data consistency in a microservices
architecture
Build events into the core of your application using event
sourcing