Your SlideShare is downloading. ×
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Event Driven-Architecture from a Scalability perspective
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Event Driven-Architecture from a Scalability perspective

10,694

Published on

Event Driven-Architecture from a Scalability perspective

Event Driven-Architecture from a Scalability perspective

Published in: News & Politics, Technology
1 Comment
54 Likes
Statistics
Notes
No Downloads
Views
Total Views
10,694
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
461
Comments
1
Likes
54
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Building loosely coupled and scalable systems using Event-Driven Architecture Jonas Bonér Patrik Nordwall Andreas Källberg
  • 2. Why is EDA Important for Scalability?
  • 3. What building blocks does EDA consists of?
  • 4. OutlineConcepts Patterns ChallengesHighly Scalable Web Sites
  • 5. Concepts
  • 6. MessagingPublish-Subscribe Point-to-Point Store-forward Request-Reply
  • 7. Publish-Subscribe
  • 8. Point-to-Point
  • 9. Store-Forward
  • 10. Request-Reply
  • 11. StandardsAMQP JMS
  • 12. Some Products ...
  • 13. Domain Events“Its really become clear to me in the lastcouple of years that we need a new buildingblock and that is the Domain Events” -- Eric Evans, 2009
  • 14. Domain Events“State transitions are an important part of ourproblem space and should be modeled withinour domain.” -- Greg Young, 2008
  • 15. Domain Events Something that hashappened in the past CustomerRelocated CargoShippedInventoryLossageRecorded
  • 16. Domain EventsUniquely identifiable Self containedObservable   Time relevant
  • 17. Patterns
  • 18. Event Stream Processingselect  *  from  Withdrawal(amount>=200).win:length(5)
  • 19. Actors• Share NOTHING• Isolated lightweight processes• Communicates through messages• Asynchronous and non-blocking• No shared state … hence, nothing to synchronize.• Each actor has a mailbox (message queue)
  • 20. ActorsEasier to reason aboutRaised abstraction level Easier to avoid Race conditions Deadlocks Starvation Live locks
  • 21. ActorsTransparent remoting • Client-managed • Server-managedPub-Sub • Redis • ZeroMQGuaranteed deliveryPersistent mailbox • File-based • Network-based
  • 22. Command and Query Responsibility Segregation“A single model cannot be appropriatefor reporting, searching andtransactional behavior.” -- Greg Young, 2008
  • 23. CQRS• Aggregate roots receive Commands and publish Events• All state changes are represented by Domain Events• Reporting module is updated as a result of the published Events• All Queries go directly to the Reporting, the Domain is not involved
  • 24. The traditional way... DB Data access Domain Service Update Query DTO DTO Client
  • 25. The CQRS way... DB DBData access publish subscribe Domain Data access Service Service QueryCommand DTO Client
  • 26. The CQRS way... DB DB Data access Service DBData access publish Data access Domain Service DB Data access Service Service Client
  • 27. CQRS Benefits• Separation of concern• Fully encapsulated domain that only exposes behavior• Queries do not use the domain model• Easy integration with external systems• Performance and scalability• Testability
  • 28. Event Sourcing• Every state change is materialized in an Event• All events are stored in an Event Log• System can be reset and Event Log replayed• Many different Listeners can be added
  • 29. Storing Structure * Line Item Purchase Order Shipping Information
  • 30. Event Sourcing - Storing Deltas Shipping Cart Added 2 Added 2 InfoCreated Socks Shirts Added
  • 31. Aggregates are tracking events as to what has changed within them Current state is constructed by replaying all eventsData is not persisted in a structure but as a series of transactions No ORM is needed
  • 32. Event Sourcing - Replaying Events1 2 3 4 5 6 7
  • 33. Event Sourcing - Rolling Snapshot Snapshot1 2 ... 100 101 102 103 104
  • 34. Event Sourcing - Benefits• No object-relational impedance mismatch• Bullet-proof auditing and historical tracing• Support future ways of looking at data• Performance and scalability• Testability• Reconstruct production scenarios
  • 35. Simple CQRS Samplehttp://github.com/patriknw/ sculptor-simplecqrs/
  • 36. Challenges
  • 37. Clustering of BrokersActiveMQ• Master-Slave• Store and Forward Network of BrokersRabbitMQ• Cluster of Erlang nodesZeroMQ• Brokerless - point-to-point or pub-sub
  • 38. ZeroMQNetwork protocol - thin layer above TCP/IPTransports Text• INPROC• IPC• MULTICAST• TCP
  • 39. ZeroMQForwarding devices• QUEUE• FORWARDER• STREAMER
  • 40. ZeroMQPatternsREQUEST/REPLY (load-balanced)PUB/SUBUPSTREAM/DOWNSTREAM (pipelining)PAIR (exclusive)
  • 41. Wire FormatsJava serialization (binary, schema, runtime)Protobuf (binary, schema, compiled)Avro (binary, schema, compiled & runtime)Thrift (binary, schema, compiled) TextMsgPack (binary, schema, compiled)Protostuff (binary, schema, compiled)Kryo (binary, schema-less, runtime)BERT (binary, schema-less, runtime)Hessian (binary, schema-less, compiled)XML (text, schema)JSON (text, schema-less)
  • 42. Guaranteed DeliveryDo I really need it?Persistence increasesreliability at the expenseof performance  
  • 43. Competing Consumers Pattern for solving: • Load balancing • Concurrency • Failover Only works with Point-to-Point ChannelChallenge• ordering• duplicates (idempotent receiver)
  • 44. Duplicate Messages What do I need? • Once-and-only-once • At-least-once • At-most-once QOS keep history of processed idsUnique message identifier Business semantics
  • 45. How to get back on track?Point-to-point: no problem, justmake the queue persistentPub/sub: well, not so straightforward Problem: only active subscribers Solution: durable subscriber Problem: failover and load balancing
  • 46. Producer Flow ControlWhat to do when producers flood youwith messages?Running low on broker resources, slowconsumersGraceful degradation • caller run (in process only)• block• abort• discard
  • 47. Behind the Scenes of Highly Scalable Web Sites
  • 48. caching is important, but also...
  • 49. Minimize latencyFlickr: Do The Essential Work Up-Front And Queue The Rest Amazon: ~99% of content is staticReddit: Precompute everythingand cache it
  • 50. Changes - pull or pushFacebook: Pull on Demand Digg: Push on ChangeTwitter: Push tweets
  • 51. Truly event-driven web clientsRequest-response doesntfit collaborative systems WebSockets enable real event- driven web
  • 52. Why is EDA Important for Scalability?• Scale out and up• Load balance• Parallel execution• Non-blocking• Loosely coupled components can scale more independent of each other
  • 53. ?
  • 54. thanksfor listening

×