Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Opportunities & Pitfalls
of
Event-Driven Utopia
@berndruecker
Why this talk
Why this talk
Service 1
Agenda
Service 2
Events on the inside
Events on the outside
1
3
Service 1
Agenda
Service 2
Events on the inside
Events on the outside
1
3
Events inside out2
Service 1
Agenda
Service 2
Events on the inside1
Events on the outside3
Events inside out2
Once upon a time…
RDMS
Application
Client
BBC architecture
(box - arrow – box – arrow - cylinder)
Every architecture diagr...
The great thing about this architecture
RDMS
Application
DB gurantees
(e.g. ACID)
The problem
not webscale
resiliency is expensive
RDMS
Application
RDMS
Pat Helland
https://vimeo.com/52831373
Append-only
Log
…
bank
account
created
+2,500 $
transfered
-14.99$
paid by
credit card
…
Persistent change
RDMS
Account # ...
Append-only
Log
…
bank
account
created
+2,500 $
transfered
-14.99$
paid by
credit card
…
Persistent change Event
Bank Acco...
Event Sourcing in a nutshell
Customer
Business
Logic
Save
event
2. Read events
Replay
…
Internal State
3. Build
internal s...
Working without distributed transactions
Customer
Business
Logic
Save
event
2. Read events
Replay
…
Internal State
3. Buil...
Traditional Architecture
Customer
1. Create Customer
Account
RDMS
Business
Logic
Open
Account
2. Persist state
changes
3. ...
Pat Helland
“
Distributed Systems Guru
Worked at Amazon,
Microsoft & Salesforce
@berndruecker
Pat Helland
Grown-Ups Don’t Use
Distributed Transactions
“
Distributed Systems Guru
Worked at Amazon,
Microsoft & Salesfor...
Open
Account
Outbox pattern in traditional architectures
Customer
1. Create Customer
Account
RDMS
Business
Logic
Job:
„Ope...
Outbox pattern – Implementation Approaches
Scheduler
Database
Transaction Log
Workflow
Automation
Idempotency
Customer
1. Create
Customer
Account
RDMS
Business
Logic
Job:
„Open
Account“
2. Persist state
changes
3. Remote...
Idempotency
Customer
1. Create
Customer
Account
RDMS
Business
Logic
Job:
„Open
Account“
2. Persist state
changes
3. Remote...
Working without distributed transactions
Customer
Business
Logic
Save
event
2. Read events
Replay
…
Internal State
3. Buil...
Events on the inside.
An example from my world
mail@berndruecker.io
@berndruecker
Bernd Ruecker
Co-founder and
Chief Technologist of
Camunda
We offer two different workflow engines. Why?
Camunda ZeebePersistent
State
Persistent
change
Workflow
Instance
Id
Current
Activity
State
2 RetrievePayment running
Workflow
Instance
Id
Current
Activity
State
2 ShipGo...
Append-only
Log1.
create
workflow
instance
workflow
instance
created
start
event
occured
sequence
flow taken
activity
acti...
Event Handling, Replication & Single Writer
Follower
Follower
complete task
command
task completed
event
1 send
2 append c...
What we do different
Follower
Follower
complete task
command
task completed
event
1 send
2 append command
Leader
Stream
Pr...
Consistency
Availability
Partition
Zeebe is CP
Follower
Follower
complete task
command
task completed
event
1 send
2 append command
Leader
Stream
Processor
4...
Horizontal scalability by partitioning
Partition 1
Partition 2
Partition 3
Partition 4
Every workflow instance is
exactly ...
Queries and
read models
Zeebe
Broker
Zeebe
Broker
Streaming
Exporter
ask
ask
Recap 1 – Events on the inside
# Natural mechanism to build scalable services in distributed
systems (with Outbox & co inc...
Service 1
Agenda
Service 2
Events on the inside
Events on the outside
1
3
Events inside out2
Event Store and Messaging
Customer
…
1. Create Customer
Customer Event Store
Merge Messaging and Event Store
Customer
…
1. Create Customer
Customer Event Store
Merge messaging and event store
Customer
…
1. Create Customer
Shared Event Store
Enter the world of Kafka…
Merge messaging and event store
Customer
…
1. Create Customer
Shared Event Store
Kafka as transport
Customer
…
1. Create Customer
Used as queue (but persistent!)
Service 1
Agenda
Service 2
Events on the inside
Events on the outside
1
3
Events inside out2
Once upon a time
Billing
Customer
Change
Address
Event Notification
Address
changed
Billing
Customer
Event Notification
Address
changed
Billing
Customer
Billing
Customer
Reverse direction
of dependency
direction
of dependen...
Event Notification
Address
changed
Billing
Customer AdressChanged
{
customerId: 42
}
Ask for
details
Event-carried State Transfer
Address
changed
Billing
Customer
AddressChanged
{
customerId: 42,
address: ...
}
CustomerChan...
This decision is complex
Address
changed
Billing
Customer
Billing
Customer
Reverse direction
of dependency
direction
of de...
Example
Change Address
Address
Submit
From bla@company.com
Date 2019-04-23 09.05
To confirm your address change please cli...
Example
Customer
Notification
Address change
confirmed
Change Adress
Address change
requested
http://company.com/confirm?i...
Example
Customer
Notification
‚Confirmation‘
approved
Change Adress
http://company.com/confirm?id=82e97
d49-166c-4862-9973...
Challenge:
Command vs. Event
Event
Command
vs
It is NOTabout communication protocols
Address
changed
Billing
Customer
Billing
Customer
Change
Address
It can be messagin...
Manifold ways of transport
…
Manifold ways of transport
…
?
Event Command Query
Message Record Event
Fact,
happened in the past,
immutable
Intend,
Want s.th. to happen,
The intenti...
?
Event Command Query
Message Record Event
Commands in disguise
The Customer Needs To Be
Sent A Message To Confirm
Address Change
Event
Send
Message
Wording of
Sende...
Examples
PaymentOrder
Subscription
Retrieve
Payment
More general,
does not need to know
who is retrieving payments
Custome...
Distributed Monoliths
Authorization
Service
Document Context Page
Context
Document
attached
Page
created
Document
moved
Pa...
Define stable contract/API instead
Authorization
Service
Document Context Page
Context
Add
auth
…
Next challenge:
Event chains
Event Chains
Adress
Check
Credit Check
Registration
@berndruecker
Customer
Event Bus
Registration
requested
Credit
checked...
Event Chains
Adress
Check
Credit Check
Registration
@berndruecker
Customer
Event Bus
Registration
requested
Credit
checked...
The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
...
The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
...
The danger is that it's very easy to make
nicely decoupled systems with event
notification, without realizing that you're
...
Monitoring Workflows Across Microservices
https://www.infoq.com/articles/monitor-workflow-collaborating-microservices
@ber...
Typical approachesDistributed Tracing
Data Lake / Event Monitoring
Process Mining
Process Tracking
@berndruecker
What we currently build with customers…
Camunda
Optimize
Elastic
Registration
requested
Credit
checked
Address
checked
Cus...
All great – until you have to move…
Changes required for an additional check
Adress
Check
Credit Check
Registration
Criminal
Check
@berndruecker
Customer
Even...
Changes required for an additional check
Adress
Check
Credit Check
Registration
Criminal
Check
@berndruecker
Customer
Even...
Alternative flow
Adress
Check
Credit Check
Registration
Criminal
Check
@berndruecker
Customer
Kafka
Customer
registered
Re...
Alternative flow
Adress
Check
Credit Check
Registration
Criminal
Check
@berndruecker
Customer
Kafka
Customer
registered
Re...
Keep it stable, just move
sticks with yellow color to the
top.
How hard
can it be?
What we wanted
Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0
@bernd...
Orchestration
Adress
CheckCredit Check
Registration
Customer
Kafka
Registration
requested
Credit
checked
Address
checked
C...
Changes
Adress
CheckCredit Check
Registration
Customer
Kafka
Registration
requested
Credit
checked
Address
checked
Custome...
Comparison
2 changes
criminal check can be deployed first
2 changes,
criminal check can be deployed first
See also https:/...
In my world…
Customer
On-boarding
Leverage Workflow Engine & BPMN within Service
Customer On-boarding
Local Orchestration
Central
Orchestration
Service
Recap 2
# Commands vs. Events: Decide about the direction of
dependencies
# Beware of event-chains and avoid losing sight
...
Service 2Service 1
Recap
Events on the inside
Events on the outside
1
3
Persistent state vs persistent change
Event sourci...
Want to see code?
Nothing for the faint of heart…
Events on the inside
Events on the outside
Nothing for the faint of heart…
…but doable…
…and worth it
Thank you!
@berndruecker
mail@berndruecker.io
@berndruecker
https://berndruecker.io
https://medium.com/berndruecker
https://github.com/berndruecker...
QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia
Upcoming SlideShare
Loading in …5
×

of

QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 1 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 2 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 3 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 4 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 5 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 6 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 7 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 8 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 9 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 10 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 11 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 12 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 13 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 14 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 15 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 16 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 17 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 18 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 19 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 20 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 21 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 22 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 23 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 24 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 25 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 26 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 27 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 28 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 29 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 30 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 31 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 32 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 33 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 34 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 35 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 36 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 37 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 38 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 39 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 40 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 41 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 42 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 43 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 44 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 45 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 46 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 47 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 48 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 49 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 50 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 51 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 52 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 53 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 54 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 55 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 56 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 57 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 58 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 59 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 60 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 61 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 62 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 63 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 64 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 65 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 66 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 67 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 68 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 69 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 70 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 71 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 72 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 73 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 74 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 75 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 76 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 77 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 78 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 79 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 80 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 81 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 82 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 83 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 84 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 85 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 86 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 87 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 88 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 89 QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia Slide 90
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

6 Likes

Share

Download to read offline

QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia

Download to read offline

Slides from my talk at QCon NYC on 25th of June 2019 in the Microservices and Serverless track

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

QCon 2019 - Opportunities and Pitfalls of Event-Driven Utopia

  1. 1. Opportunities & Pitfalls of Event-Driven Utopia @berndruecker
  2. 2. Why this talk
  3. 3. Why this talk
  4. 4. Service 1 Agenda Service 2 Events on the inside Events on the outside 1 3
  5. 5. Service 1 Agenda Service 2 Events on the inside Events on the outside 1 3 Events inside out2
  6. 6. Service 1 Agenda Service 2 Events on the inside1 Events on the outside3 Events inside out2
  7. 7. Once upon a time… RDMS Application Client BBC architecture (box - arrow – box – arrow - cylinder) Every architecture diagram you'll ever need
  8. 8. The great thing about this architecture RDMS Application DB gurantees (e.g. ACID)
  9. 9. The problem not webscale resiliency is expensive RDMS Application RDMS
  10. 10. Pat Helland https://vimeo.com/52831373
  11. 11. Append-only Log … bank account created +2,500 $ transfered -14.99$ paid by credit card … Persistent change RDMS Account # Balance 12345 2,500$ Persistent state Current Balance = 2,485.01 $
  12. 12. Append-only Log … bank account created +2,500 $ transfered -14.99$ paid by credit card … Persistent change Event Bank Account Created 2019/04/16 11:00 # 12345 Event Money Transfer Received 2,500$ 2019/04/16 11:00 # 12345
  13. 13. Event Sourcing in a nutshell Customer Business Logic Save event 2. Read events Replay … Internal State 3. Build internal state 5. Add Customer Created Event 1. Create Customer 4. Validation and Invariant Checks e.g. Customer Created, Customer Credit Limit Approved, …Customer Event Store 6. Async publish Customer Created Domain Event
  14. 14. Working without distributed transactions Customer Business Logic Save event 2. Read events Replay … Internal State 3. Build internal state 5. Add Customer Created Event 1. Create Customer 4. Validation and Invariant Checks e.g. Customer Created, Customer Credit Limit Approved, …Customer Event Store 6. Async publish Customer Created Domain Event This is the only atomic operation required
  15. 15. Traditional Architecture Customer 1. Create Customer Account RDMS Business Logic Open Account 2. Persist state changes 3. Remote Communication
  16. 16. Pat Helland “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce @berndruecker
  17. 17. Pat Helland Grown-Ups Don’t Use Distributed Transactions “ Distributed Systems Guru Worked at Amazon, Microsoft & Salesforce
  18. 18. Open Account Outbox pattern in traditional architectures Customer 1. Create Customer Account RDMS Business Logic Job: „Open Account“ 2. Persist state changes 3. Remote CommunicationExecute: „Open Account“ TX 1 TX 2 Async after first transaction!
  19. 19. Outbox pattern – Implementation Approaches Scheduler Database Transaction Log Workflow Automation
  20. 20. Idempotency Customer 1. Create Customer Account RDMS Business Logic Job: „Open Account“ 2. Persist state changes 3. Remote Communication Async after transaction! Execute: „Open Account“ Capture Request TX 1 TX 2
  21. 21. Idempotency Customer 1. Create Customer Account RDMS Business Logic Job: „Open Account“ 2. Persist state changes 3. Remote Communication Async after transaction! Execute: „Open Account“ Capture Request TX 1 TX 3TX 2
  22. 22. Working without distributed transactions Customer Business Logic Save event 2. Read events Replay … Internal State 3. Build internal state 5. Add Customer Created Event 1. Create Customer 4. Validation and Invariant Checks e.g. Customer Created, Customer Credit Limit Approved, …Customer Event Store 6. Async publish Customer Created Domain Event This is the only atomic operation required
  23. 23. Events on the inside. An example from my world
  24. 24. mail@berndruecker.io @berndruecker Bernd Ruecker Co-founder and Chief Technologist of Camunda
  25. 25. We offer two different workflow engines. Why? Camunda ZeebePersistent State Persistent change
  26. 26. Workflow Instance Id Current Activity State 2 RetrievePayment running Workflow Instance Id Current Activity State 2 ShipGoods running Workflow Instance Id Current Activity State 2 OrderDelivered ended 2. 3. UPDATE 2. UPDATE 1. INSERT 3.1. RDMS Workflow Engine
  27. 27. Append-only Log1. create workflow instance workflow instance created start event occured sequence flow taken activity activated task created lock created task locked complete task task completed activity completed sequence flow taken … … 2. 2. 1. Workflow Engine
  28. 28. Event Handling, Replication & Single Writer Follower Follower complete task command task completed event 1 send 2 append command Broker (Leader) Stream Processor 4 process 7 store & replicate event 6 append event 3 store & replicate command 5 respond Single Writer (single thread)
  29. 29. What we do different Follower Follower complete task command task completed event 1 send 2 append command Leader Stream Processor 4 process 7 store & replicate event 6 append event 3 store & replicate command 5 respond Single Writer (single thread) Store and replay commands Delete records that are fully processed Persist & replicate internal state
  30. 30. Consistency Availability Partition
  31. 31. Zeebe is CP Follower Follower complete task command task completed event 1 send 2 append command Leader Stream Processor 4 process 7 store & replicate event 6 append event 3 store & replicate command 5 respond Single Writer (single thread)
  32. 32. Horizontal scalability by partitioning Partition 1 Partition 2 Partition 3 Partition 4 Every workflow instance is exactly handled by one partition instance id: 2-42 instance id: 3-66 Stream Processor Single Writer (single thread)
  33. 33. Queries and read models Zeebe Broker Zeebe Broker Streaming Exporter ask ask
  34. 34. Recap 1 – Events on the inside # Natural mechanism to build scalable services in distributed systems (with Outbox & co included) But # You have to think about reads, queries & eventual consistency # Much less industry experience available @berndruecker
  35. 35. Service 1 Agenda Service 2 Events on the inside Events on the outside 1 3 Events inside out2
  36. 36. Event Store and Messaging Customer … 1. Create Customer Customer Event Store
  37. 37. Merge Messaging and Event Store Customer … 1. Create Customer Customer Event Store
  38. 38. Merge messaging and event store Customer … 1. Create Customer Shared Event Store
  39. 39. Enter the world of Kafka…
  40. 40. Merge messaging and event store Customer … 1. Create Customer Shared Event Store
  41. 41. Kafka as transport Customer … 1. Create Customer Used as queue (but persistent!)
  42. 42. Service 1 Agenda Service 2 Events on the inside Events on the outside 1 3 Events inside out2
  43. 43. Once upon a time Billing Customer Change Address
  44. 44. Event Notification Address changed Billing Customer
  45. 45. Event Notification Address changed Billing Customer Billing Customer Reverse direction of dependency direction of dependency Change Address
  46. 46. Event Notification Address changed Billing Customer AdressChanged { customerId: 42 } Ask for details
  47. 47. Event-carried State Transfer Address changed Billing Customer AddressChanged { customerId: 42, address: ... } CustomerChanged { customerId: 42, status: A, address: ..., } AddressChanged { customerId: 42, oldAddress: ... newAddress: ... } CustomerMoved { ...,
  48. 48. This decision is complex Address changed Billing Customer Billing Customer Reverse direction of dependency direction of dependency Change Address
  49. 49. Example Change Address Address Submit From bla@company.com Date 2019-04-23 09.05 To confirm your address change please click on this link: http://company.com/confirm?id=82e97d49-166c-4862- 9973-4db348e6225d Incoming Email
  50. 50. Example Customer Notification Address change confirmed Change Adress Address change requested http://company.com/confirm?id=82e97 d49-166c-4862-9973-4db348e6225d direction of dependency
  51. 51. Example Customer Notification ‚Confirmation‘ approved Change Adress http://company.com/confirm?id=82e97 d49-166c-4862-9973-4db348e6225d Send mail ‚Confirmation‘ Address changed direction of dependency
  52. 52. Challenge: Command vs. Event Event Command vs
  53. 53. It is NOTabout communication protocols Address changed Billing Customer Billing Customer Change Address It can be messaging, REST, whatever, ….
  54. 54. Manifold ways of transport …
  55. 55. Manifold ways of transport …
  56. 56. ? Event Command Query Message Record Event Fact, happened in the past, immutable Intend, Want s.th. to happen, The intention itself is a fact
  57. 57. ? Event Command Query Message Record Event
  58. 58. Commands in disguise The Customer Needs To Be Sent A Message To Confirm Address Change Event Send Message Wording of Sender Wording of recipient
  59. 59. Examples PaymentOrder Subscription Retrieve Payment More general, does not need to know who is retrieving payments Customer Order Address Changed Billing More general, does not need to care about who is interessted in address changes Notification Send Mail Global service Order Notification Order Placed Payment Received Goods Shipped Service that can handle notifications for orders autonomously
  60. 60. Distributed Monoliths Authorization Service Document Context Page Context Document attached Page created Document moved Page moved …
  61. 61. Define stable contract/API instead Authorization Service Document Context Page Context Add auth …
  62. 62. Next challenge: Event chains
  63. 63. Event Chains Adress Check Credit Check Registration @berndruecker Customer Event Bus Registration requested Credit checked Address checked Customer registered
  64. 64. Event Chains Adress Check Credit Check Registration @berndruecker Customer Event Bus Registration requested Credit checked Address checked Customer registered How does customer registration work?
  65. 65. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
  66. 66. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
  67. 67. The danger is that it's very easy to make nicely decoupled systems with event notification, without realizing that you're losing sight of that larger-scale flow, and thus set yourself up for trouble in future years. https://martinfowler.com/articles/201701-event-driven.html @berndruecker
  68. 68. Monitoring Workflows Across Microservices https://www.infoq.com/articles/monitor-workflow-collaborating-microservices @berndruecker
  69. 69. Typical approachesDistributed Tracing Data Lake / Event Monitoring Process Mining Process Tracking @berndruecker
  70. 70. What we currently build with customers… Camunda Optimize Elastic Registration requested Credit checked Address checked Customer registered @berndruecker
  71. 71. All great – until you have to move…
  72. 72. Changes required for an additional check Adress Check Credit Check Registration Criminal Check @berndruecker Customer Event Bus Registration requested Credit checked Customer registered Address checked
  73. 73. Changes required for an additional check Adress Check Credit Check Registration Criminal Check @berndruecker Customer Event Bus Registration requested Credit checked Customer registered Address checked Criminal checked
  74. 74. Alternative flow Adress Check Credit Check Registration Criminal Check @berndruecker Customer Kafka Customer registered Registration requested Address checked Credit checked Criminal checked
  75. 75. Alternative flow Adress Check Credit Check Registration Criminal Check @berndruecker Customer Kafka Customer registered Registration requested Address checked Credit checked Criminal checked „Credit checks got more expensive, do that only if all other checks succeed“
  76. 76. Keep it stable, just move sticks with yellow color to the top. How hard can it be?
  77. 77. What we wanted Photo by Lijian Zhang, available under Creative Commons SA 2.0 License and Pedobear19 / CC BY-SA 4.0 @berndruecker
  78. 78. Orchestration Adress CheckCredit Check Registration Customer Kafka Registration requested Credit checked Address checked Customer registeredCheck credit Check address Customer checked Customer On-boarding Of course these two services could be merged
  79. 79. Changes Adress CheckCredit Check Registration Customer Kafka Registration requested Credit checked Address checked Customer registeredCheck credit Check address Customer checked Customer On-boarding Criminal Check Crimes checked Check crimes
  80. 80. Comparison 2 changes criminal check can be deployed first 2 changes, criminal check can be deployed first See also https://www.infoworld.com/article/3391592/ how-to-tame-event-driven-microservices.html
  81. 81. In my world… Customer On-boarding Leverage Workflow Engine & BPMN within Service
  82. 82. Customer On-boarding Local Orchestration Central Orchestration Service
  83. 83. Recap 2 # Commands vs. Events: Decide about the direction of dependencies # Beware of event-chains and avoid losing sight # Balance choreography and orchestration @berndruecker
  84. 84. Service 2Service 1 Recap Events on the inside Events on the outside 1 3 Persistent state vs persistent change Event sourcing & Event Store Consistency & CAP Read Models & CQRS Events as API Event vs Command Event chains & visibility Orchestration vs Choreography Shared Event Store Events inside out2
  85. 85. Want to see code?
  86. 86. Nothing for the faint of heart… Events on the inside Events on the outside
  87. 87. Nothing for the faint of heart… …but doable… …and worth it
  88. 88. Thank you! @berndruecker
  89. 89. mail@berndruecker.io @berndruecker https://berndruecker.io https://medium.com/berndruecker https://github.com/berndruecker https://www.infoq.com/articles/events- workflow-automation Contact: Slides: Blog: Code: https://www.infoworld.com/article/3254777/ application-development/ 3-common-pitfalls-of-microservices- integrationand-how-to-avoid-them.html https://thenewstack.io/5-workflow-automation- use-cases-you-might-not-have-considered/
  • guidofischermanns

    Oct. 3, 2021
  • tuxinoo

    Mar. 2, 2021
  • powerirs

    Mar. 5, 2020
  • AudriusPliknys

    Mar. 4, 2020
  • JulianRuppel1

    Feb. 25, 2020
  • MarcusFinck

    Sep. 12, 2019

Slides from my talk at QCon NYC on 25th of June 2019 in the Microservices and Serverless track

Views

Total views

19,370

On Slideshare

0

From embeds

0

Number of embeds

17,533

Actions

Downloads

59

Shares

0

Comments

0

Likes

6

×