Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Events & Microservices

1,071 views

Published on

An ode to the underrepresented and underused pattern of events and asynchrony in the design and development of Microservices.

Prepared by Saul Caganoff, and delivered by Saul at Melbourne Microservices, and by Yamen Sader at Sydney Microservices.

Published in: Technology
  • Be the first to comment

Events & Microservices

  1. 1. Events & Microservices Saul Caganoff, CTO Sixtree @scaganoff Yamen Sader @yaamehn
  2. 2. Introduction • Asynchrony is considered a microservice characteristic – But not often discussed • In the top 20 Google results – 7 mentioned async/choreography or events – 3 of those by Chris Richardson or Martin Fowler • Mostly mentioned in passing • Honourable Mentions: – http://www.infoq.com/articles/microservices-intro – http://highscalability.com/blog/2014/4/8/microservices-not-a-free- lunch.html
  3. 3. WHY EVENTS?
  4. 4. Bounded Contexts Finance Sales Fulfilment Product Online bookstore is the “Hello World” of enterprise systems http://maribyrnong.com.au
  5. 5. Sales Context Wishlist Book ISBN Title Author Price Order Status Timestamp Sub-total GST Total Item Quantity History Customer Name Age Loyalty Status
  6. 6. Finance Context Wishlist Book ISBN Title Author Price Order Status Timestamp Sub-total GST Total Item Quantity History Customer Name Age Loyalty Status Credit Card Holder Name Number Expiry Code
  7. 7. Fulfilment Context Book ISBN Title Customer Name Receipt Order Number Sub-total GST Total Address Street Number Street Name Suburb Postcode Courier Depot Location Driver Shipment Dispatch Time ETA Location
  8. 8. Product Context Book Description Abstract Picture Review Author Name Biography Inventory Location Quantity ISBN Title Price
  9. 9. Coupling & Cohesion • Cohesion – how well a subsystem forms a uniting whole – How well a bounded context does its job • Coupling – the propensity for change in one part of a system to disrupt another – how badly “stuff” leaks across the boundaries • Coupling is evil – it sneaks into solutions in all kinds of insidious ways – But… distributed monolith is the “worst kind of evil”
  10. 10. Coupling & Cohesion • Microservices strive to eliminate coupling • Insidious sources of coupling – Shared databases – Single Common Data Model (vs bounded contexts) – DRY – shared code – APIs generated from code – Poor client code – Misunderstood ‘transactions’ – Assumptions about business processes – Organizational structure – Services…
  11. 11. Business Processes • End-to-end business processes require coordination across bounded contexts Finance Sales Ful- filment Product
  12. 12. Business Processes • Browse the inventory • Create order • Enter Shipping Details • Quote shipping charge • Make the payment • Update inventory • Create Shipment • Pick & Pack • Courier • Accept delivery GET /products POST /order PUT /order/:id/address GET /3pl/charges?zip=… POST /finance/payment?order= POST /products POST /3pl/shipment GET /3pl/shipment/:id GET /3pl/shipment/:id/address PUT /3pl/shipment/:id?status=
  13. 13. Business Processes • Orchestration – Centralized coordination – E.g. the user-agent, BPM engine, service facade – the “god” object • Hypermedia – HATEOAS – But what about at the edges? – How much should one BC know about another BC? “microservices architecture will never be SOA done right unless it means building hypermedia systems of microservices” – Jason Bloomberg http://intellyx.com/2015/07/20/are-microservices-soa-done-right/
  14. 14. Some Perspective 0 2 4 6 8 10 12 DB Triggers Monolith Orchestration Hypermedia EDA Coupling
  15. 15. Data Synchronization • How to handle data “owned” by multiple domains – Glibness: “don’t” have multiple data owners – …in reality it is impossible to eliminate – …especially if legacy or COTS systems involved. • Some data synchronization is naturally conveyed by the business process • …some is not • Another topic!
  16. 16. Choreography • Use events to coordinate activity across bounded contexts • Publishers & Subscribers are highly decoupled – Event subscribers can come & go minimal or no code or configuration changes • Publishers & Subscribers are more resilient to change – either planned or unplanned – E.g. events redelivered in the event of subscriber failure – either to the restored subscriber, or a replacement
  17. 17. A Dependency View Finance Sales Sales Tells Finance Finance Sales Finance Asks Sales Sales Sales Announces
  18. 18. Choreography • But trade-offs • Asynchronous events lie at the heart of some of the microservices challenges – Centralised orchestration is harder to change, but easier to control/monitor – Event-based choreography is easier to change but harder to control/monitor • Distributed logging/monitoring systems are needed to provide a view into asynchronous interactions
  19. 19. An Aside • No global view may seem like a major downside • But very rare for anyone to know this anyway • When X happens, we do A and B • When A happens, we do F • Etc • The more ‘inter-domain’ concerns can be solved by just depending on events, the more likely you are to have the ‘correct’ boundaries
  20. 20. Coordination • Think about hypermedia inside boundaries • Events outside the boundaries Finance Sales Ful- filment Product
  21. 21. MODELLING EVENTS
  22. 22. Commands & Events • Duality between commands & events – Commands cause state transitions – Events reflect state transitions • An external observer can infer the state of a resource from its event-stream – This is the essence of event-sourcing & CQRS
  23. 23. State Modelling Empty Cart Full Cart Quoted Paid POST /cart PUT /cart/:id { book: … } PUT /cart/:id { book: … } DELETE /cart/:id/books/:id PUT /cart/:id { address: … } POST /payments { cart: …, card: {…}} Commands (HATEOAS) {create: { cart: … } … } {add: { cart: …, book: {…}} ..} {add: { cart: …, book: {…}} ..} {del: { cart: …, book: {…}} ..} {quoted: { cart: … } … } {paid: { cart: … } … } Events (EATNOAS)
  24. 24. Commands & Events Finance Sales Ful- filment Product Inventory Marketing Commands Events Reporting Logistics
  25. 25. Event Structure • Immutable structures • Typical properties – Timestamp – Source – Entity URI – Action (state transition) • The MVP is to notify that a resource has changed and allow subscribers to pull the update if needed • Don’t forget Postel’s Law, versioning etc
  26. 26. Event Granularity • Customer details changed? • Or email address changed? • Some considerations: – Event storm with the business or the technical specialists, they will call out ‘special’ events – ‘Eye of the Beholder’ ie the subscriber determines deltas important to them – Expect to get this wrong once or twice
  27. 27. Concurrency • Race Conditions • The importance of order • Single Master vs Multi-Master • Deltas vs Copies • Action indicators • CRDTs – conflict-free replicated data types • …think in terms of events rather than copies
  28. 28. EVENT TRANSPORTS
  29. 29. Pull vs Push • Pull = subscribers beat – Subscriber never get overwhelmed – Messages pile up on publisher • Push = publishers beat – Publisher never gets backed up – Subscribers can be overwhelmed • Answer: use an intermediary tuned to this problem
  30. 30. Intermediary
  31. 31. Known Quantity • Events are highly regular – Append only series – Generic attributes (topics, metadata) – Timestamped – Consumed in sequence – Like a very simple database • Common to standardise on an intermediary – Like standardising on a database – Usually exposed directly, a platform service
  32. 32. Syndication (Pull) • Service exposes an RSS/atom feed • Consumers do all the hard work: – Subscription & Concurrency – State Management (where am I up to?) – Retries • Positives – Very easy…well-known pattern – Web-scale • Negatives – Polling – High latency – Consumer concurrency very difficult
  33. 33. Web Hooks (Push) • Service POSTs events to subscribers • Publishers do all the hard work: – Manage subscribers – Managed guaranteed delivery/ retries • Positives – Not polling, relatively lower latency – Web technology (HTTP endpoints and load balancers) • Negatives – Consumer concurrency a little harder – Retries and history difficult
  34. 34. Message Queues (Push) • Events published to queues/topics – E.g JMS, AMQP, ZMQ, SQS/SNS, IronMQ • Positives – Mature, well-known technologies – Good open-source implementations available – Competing consumer pattern • Negatives – Extra infrastructure – Reliability depends on reliable file-systems • SANs, Distributed file-lock managers – No history or replay – Slow consumer problem
  35. 35. Pull vs Push (redux) • Pull – Subscriber tracks state – Intermediary simplified – Consumer concurrency more difficult – Natural history is kept • Push – Intermediary tracks consumer state – Can create a bottleneck and contention – Consumer concurrency simpler – Generally no history or replay
  36. 36. Apache Kafka • Distributed log – Consumers tail the logs • Consumers tail this log, but: – Server manages consumer position (if desired) – Server manages consumer concurrency – Subscription is very simple – Pull is very low latency (push-like) – Replay and history is native – Application level replication • Others (eg Event Store) provide some similar features (history, pull + push), but none as rich in capability
  37. 37. Event Processing • Simple events are very useful for data synchronization or process coordination • Continuum of complexity • Some tools: Drools, Riemann.io, Apache Storm, Apache Spark (Streaming), Samza, etc • This is a whole ‘nother topic Cardinality State Tools Single event Simple Seriously? Multiple events/stream Simple Stream processor Multiple events/stream Complex / Rules Complex Event Processor
  38. 38. Wrap-Up • Bounded contexts are important – they promote cohesion and reduce coupling • You need to decide how to coordinate data & processes across boundaries – Orchestration (relatively higher coupling) – Hypermedia (couples across boundaries) – Events • Prefer Hypermedia within bounded contexts & events across bounded contexts Events have many benefits but also some downsides - they are an important tool in your toolbox

×