Event Driven Architecture

945 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
945
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
18
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • RabbitMQ: 3-5k persistent msg/s vs >10k transient msg/s\nQpid: 1k persistent msg/s vs > 5k transient msg/s\nActiveMQ: ???\n
  • MDB analogi, pool av MDB instanser\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Event Driven Architecture

    1. 1. Building loosely coupled and scalable systems using Event-Driven Architecture Jonas Bonér Patrik Nordwall Andreas Källberg
    2. 2. Why is EDA Important for Scalability?
    3. 3. What building blocks does EDA consists of?
    4. 4. OutlineConcepts Patterns ChallengesHighly Scalable Web Sites
    5. 5. Concepts
    6. 6. MessagingPublish-Subscribe Point-to-Point Store-forward Request-Reply
    7. 7. Publish-Subscribe
    8. 8. Point-to-Point
    9. 9. Store-Forward
    10. 10. Request-Reply
    11. 11. StandardsAMQP JMS
    12. 12. Some Products ...
    13. 13. Domain Events“Its really become clear to me in the lastcouple of years that we need a new buildingblock and that is the Domain Events” -- Eric Evans, 2009
    14. 14. Domain Events“State transitions are an important part of ourproblem space and should be modeled withinour domain.” -- Greg Young, 2008
    15. 15. Domain Events Something that hashappened in the past CustomerRelocated CargoShippedInventoryLossageRecorded
    16. 16. Domain EventsUniquely identifiable Self containedObservable   Time relevant
    17. 17. Patterns
    18. 18. Event Stream Processingselect
*
from
Withdrawal(amount>=200).win:length(5)
    19. 19. Actors• Share NOTHING• Isolated lightweight processes• Communicates through messages• Asynchronous and non-blocking• No shared state … hence, nothing to synchronize.• Each actor has a mailbox (message queue)
    20. 20. ActorsEasier to reason aboutRaised abstraction level Easier to avoid Race conditions Deadlocks Starvation Live locks
    21. 21. ActorsTransparent remoting • Client-managed • Server-managedPub-Sub • Redis • ZeroMQGuaranteed deliveryPersistent mailbox • File-based • Network-based
    22. 22. Command and Query Responsibility Segregation“A single model cannot be appropriatefor reporting, searching andtransactional behavior.” -- Greg Young, 2008
    23. 23. CQRS• Aggregate roots receive Commands and publish Events• All state changes are represented by Domain Events• Reporting module is updated as a result of the published Events• All Queries go directly to the Reporting, the Domain is not involved
    24. 24. The traditional way... DB Data access Domain Service Update Query DTO DTO Client
    25. 25. The CQRS way... DB DBData access publish subscribe Domain Data access Service Service QueryCommand DTO Client
    26. 26. The CQRS way... DB DB Data access Service DBData access publish Data access Domain Service DB Data access Service Service Client
    27. 27. CQRS Benefits• Separation of concern• Fully encapsulated domain that only exposes behavior• Queries do not use the domain model• Easy integration with external systems• Performance and scalability• Testability
    28. 28. Event Sourcing• Every state change is materialized in an Event• All events are stored in an Event Log• System can be reset and Event Log replayed• Many different Listeners can be added
    29. 29. Storing Structure * Line Item Purchase Order Shipping Information
    30. 30. Event Sourcing - Storing Deltas Shipping Cart Added 2 Added 2 InfoCreated Socks Shirts Added
    31. 31. Aggregates are tracking events as to what has changed within them Current state is constructed by replaying all eventsData is not persisted in a structure but as a series of transactions No ORM is needed
    32. 32. Event Sourcing - Replaying Events1 2 3 4 5 6 7
    33. 33. Event Sourcing - Rolling Snapshot Snapshot1 2 ... 100 101 102 103 104
    34. 34. Event Sourcing - Benefits• No object-relational impedance mismatch• Bullet-proof auditing and historical tracing• Support future ways of looking at data• Performance and scalability• Testability• Reconstruct production scenarios
    35. 35. Simple CQRS Samplehttp://github.com/patriknw/ sculptor-simplecqrs/
    36. 36. Challenges
    37. 37. Clustering of BrokersActiveMQ• Master-Slave• Store and Forward Network of BrokersRabbitMQ• Cluster of Erlang nodesZeroMQ• Brokerless - point-to-point or pub-sub
    38. 38. ZeroMQNetwork protocol - thin layer above TCP/IPTransports Text• INPROC• IPC• MULTICAST• TCP
    39. 39. ZeroMQForwarding devices• QUEUE• FORWARDER• STREAMER
    40. 40. ZeroMQPatternsREQUEST/REPLY (load-balanced)PUB/SUBUPSTREAM/DOWNSTREAM (pipelining)PAIR (exclusive)
    41. 41. Wire FormatsJava serialization (binary, schema, runtime)Protobuf (binary, schema, compiled)Avro (binary, schema, compiled & runtime)Thrift (binary, schema, compiled) TextMsgPack (binary, schema, compiled)Protostuff (binary, schema, compiled)Kryo (binary, schema-less, runtime)BERT (binary, schema-less, runtime)Hessian (binary, schema-less, compiled)XML (text, schema)JSON (text, schema-less)
    42. 42. Guaranteed DeliveryDo I really need it?Persistence increasesreliability at the expenseof performance  
    43. 43. Competing Consumers Pattern for solving: • Load balancing • Concurrency • Failover Only works with Point-to-Point ChannelChallenge• ordering• duplicates (idempotent receiver)
    44. 44. Duplicate Messages What do I need? • Once-and-only-once • At-least-once • At-most-once QOS keep history of processed idsUnique message identifier Business semantics
    45. 45. How to get back on track?Point-to-point: no problem, justmake the queue persistentPub/sub: well, not so straightforward Problem: only active subscribers Solution: durable subscriber Problem: failover and load balancing
    46. 46. Producer Flow ControlWhat to do when producers flood youwith messages?Running low on broker resources, slowconsumersGraceful degradation • caller run (in process only)• block• abort• discard
    47. 47. Behind the Scenes of Highly Scalable Web Sites
    48. 48. caching is important, but also...
    49. 49. Minimize latencyFlickr: Do The Essential Work Up-Front And Queue The Rest Amazon: ~99% of content is staticReddit: Precompute everythingand cache it
    50. 50. Changes - pull or pushFacebook: Pull on Demand Digg: Push on ChangeTwitter: Push tweets
    51. 51. Truly event-driven web clientsRequest-response doesntfit collaborative systems WebSockets enable real event- driven web
    52. 52. Why is EDA Important for Scalability?• Scale out and up• Load balance• Parallel execution• Non-blocking• Loosely coupled components can scale more independent of each other
    53. 53. ?
    54. 54. thanksfor listening

    ×