Messaging for modern applications


Published on

Published in: Technology, Business
  • Be the first to comment

Messaging for modern applications

  1. 1. Taking the next step shouldn’t be scary … Messaging for Modern Applications CONFIDENTIAL 1
  2. 2. Messaging for Modern ApplicationsPronam Chatterjee, Staff Systems Engineer, VMwareEmail: pronamc@vmware.comTwitter: pronamc CONFIDENTIAL © 2010 SpringSource, A division of VMware. All rights reserved
  3. 3. Session Agenda• Modern Applications – Trends and Impact on Messaging – Impact on Application Architecture• Spring Integration – Messaging Use Cases – Messaging DSL for Spring Applications• AMQP – Why AMQP? – AMQP Architecture• RabbitMQ – Why RabbitMQ? – RabbitMQ Architecture CONFIDENTIAL 3
  4. 4. Modern Application Trends• Written with frameworks – Developer productivity and innovation• New Application Types – Mobile, SaaS, Social – Apps released early and often• Data intensive – Emerging requirements: elasticity, multi-cloud – Web orientation drives exponential data volumes• Deployed on virtual and cloud infrastructure – Trending toward hybrid (public and private) infrastructure CONFIDENTIAL 4
  5. 5. Today’s Middleware and Databases Not Keeping Up Rise of Modern Developer Frameworks Service Loose Simplicity Polyglot Consumption Coupling Today’s Middleware and Databases Service Elastic Elastic Virtualization Delivery Deployment Scaling Aware Rise of Virtual and Cloud Infrastructure CONFIDENTIAL 5
  6. 6. Modern Applications need Modern Messaging Traditional approaches to messaging imply predictable, static interaction between well-known end-points Modern approaches to messaging embrace the unpredictable, dynamic, and transient nature of the web CONFIDENTIAL 6
  7. 7. Synchronous Architecture is Brittle An exception occurred, Manage please resend Inventory Place Fulfill Order Order X X X X X X X X X Schedule Delivery If any component is unavailable within this synchronous flow, all in-flight messages will be affected, regardless of what component they are currently in -- and since state is maintained across the components, the system can be complex to reset CONFIDENTIAL 7
  8. 8. Asynchronous Architecture Rules the Web The status of your order is I can fulfill … part of that Order I have an Order I can fulfill part of Your order will that Order be delivered next Tuesday I’ll schedule delivery of that order In this asynchronous flow: • all state is kept in the simple messages being passed • each stateless component interacts only with the broker cloud • if a component is lost, only the in-flight messages within that component must be re-delivered • components can be quickly provisioned on-demand CONFIDENTIAL 8
  9. 9. How is Messaging Used within Modern Apps?  Event I need to know when to do something  Fragment I need to only give you a piece of this data  Route Control who gets which message, without changing sender or receiver  Batch Producer and consumer can run at independent times  Publish Tell everyone who wants to know about this  Modularity Deploy, Scale, and Version a distributed system CONFIDENTIAL 9
  10. 10. Spring Integration CONFIDENTIAL 10
  11. 11. What is Spring Integration?• At the core, an embedded Message Bus – Inspired by Gregor Hohpe and Bobby Woolfs Enterprise Integration Patterns (2004) – Runs within any Spring ApplicationContext – All components are Spring-managed objects• Also, an Application Integration Framework – Connects to other systems via adapters – Unidirectional Channel Adapters – Bidirectional Messaging Gateways CONFIDENTIAL 11
  12. 12. EventI need to know when to do something … <service-activator input-channel="hotDrinks” ref="barista" method="prepareHotDrink" output-channel="preparedDrinks” /> An Event-Driven Consumer is one that is automatically handed messages as they’re delivered on the channel. The receiver acts like the message delivery is an event that triggers the receiver into action. CONFIDENTIAL 12
  13. 13. FragmentI need to only give you this piece of data … <int:splitter input-channel="orders” expression="payload.items” output-channel="drinks” /> Use a Splitter to break out the composite message into a series of individual messages, each containing data related to one item. CONFIDENTIAL 13
  14. 14. RouteControl who gets what message <int:router input-channel="drinks” expression="payload.iced ? coldDrinks : hotDrinks” />A Router consumes a Message from one Channel and republishes it to a different Channel channel depending on a set of conditions. CONFIDENTIAL 14
  15. 15. BatchProducer and Consumer can run at independent times <int:channel id="preparedDrinks"/> <int:aggregator input-channel="preparedDrinks" method="prepareDelivery" output-channel="deliveries” > <bean class=""/> </int:aggregator> Use a stateful filter, an Aggregator, to collect and store individual messages until a complete set of related messages has been received, then publish a single message distilled from the individual messages. CONFIDENTIAL 15
  16. 16. PublishTell everyone who wants to know about this <int-event:outbound-channel-adapter channel="eventChannel"/> All messages sent to the eventChannel channel will be published as ApplicationEvents to any relevant ApplicationListener instances that are registered within the same Spring ApplicationContext. CONFIDENTIAL 16
  17. 17. ModularityDeploy, Scale, Version a distributed system Local Distributed<service-activator <int-amqp:outbound-gateway input-channel="coldDrinks” id="coldDrinksBarista" ref="barista” request-channel="coldDrinks” method="prepareColdDrink” reply-channel="preparedDrinks” output-channel="preparedDrinks” /> routing-key="ordered.drinks.cold” /><service-activator <int-amqp:outbound-gateway input-channel="hotDrinks” id="hotDrinksBarista" ref="barista" request-channel="hotDrinks” method="prepareHotDrink" reply-channel="preparedDrinks” output-channel="preparedDrinks” /> routing-key="” /> Spring Integration doesn’t force you to finalize decisions about deployment before testing begins. Individual modules can be versioned, deployed, and scaled independently at any time through configuration, not code. CONFIDENTIAL 17
  18. 18. Advanced Message Queuing Protocol (AMQP) CONFIDENTIAL 18
  19. 19. Advanced Message Queuing Protocol “Like email, but you can send money over it” Open, Ubiquitous, and Adaptable async SMTP AMQP sync HTTP IIOP unreliable reliable CONFIDENTIAL 19
  20. 20. Why AMQP? A Protocol, not an API • A defined set of messaging capabilities called the AMQ model • A network wire-level protocol, AMQP On commodity hardware • 10-25 thousand messages per second is routine * • The NIC is usually the bottleneck * Non-persistent messages CONFIDENTIAL 20
  21. 21. Why AMQP?AMQP security• Brokers support separate virtual hosts• Three levels of permissions• AMQP over SSL is supported Designed to scale horizontally • Dozens of clustered brokers are common • JPMorgan sends 1 billion AMQP messages per day CONFIDENTIAL 21
  22. 22. AMQP Architecture new.order new.order new.order routing key binding M1 M2 M3 queue CONFIDENTIAL 22
  23. 23. AMQP Architecture café deliveries M1 M2 M3 queue café NA deliveries M1 M2 M3 queue café WW deliveries M1 M2 M3 queue CONFIDENTIAL 23
  24. 24. AMQP Architecture all_drinks 1 2 3 queue drink.cold cold_drinks 2 queue hot_drinks 1 Message Routing Keys: queue 1. 2. drink.cold 3. drink.warm CONFIDENTIAL 24
  25. 25. RabbitMQ CONFIDENTIAL 25
  26. 26. Legacy Middleware not suited for Modern Apps Apps & Middleware Existing middleware impacts agility by tightly coupling applications to underlying application server and other middleware components Costly to Scale Up Complex to Scale Out Licensed by Peak Physical Servers CONFIDENTIAL 26
  27. 27. RabbitMQ – Messaging that Just Works design goal: no bloat - “it just works” mission: “be everywhere” AMQP LEADER CONFIDENTIAL 27
  28. 28. Incredible Community Lowers Your Cost of Use ! "##$ (0$ % #G% 0(#K(C 4. % $ G9(% (E8 4-C . % . -$ , (@/ KE, "i ! "##$ & `j <>("/ 0(& $ -4. 4R(2 % : g7(E"-% , -. +$ / E A , (4^, -(E": T"?, . (F4-(C "/ K(4% -(= Q "i 4-C . +, (E8 ;8 4G0("/ 0(W 8 ! G/ H "/ "?, (* $8 3, (C 4-, (. "8 . G$0H & 8(0-$ , CONFIDENTIAL 28
  29. 29. Enterprise Users, such as this Tier-1 Telco CONFIDENTIAL 29
  30. 30. Enterprise Features, such as Clustering RMQ Cluster1 queue-1 RMQ1 1 queue-2 RMQ2 2 queue-3 RMQ3 3 1. queue-1 2. queue-3 3. queue-2 CONFIDENTIAL 30
  31. 31. Enterprise Features, such as Fault Tolerance ESX1 VM1 VM2 RMQ1 RMQ2 hot-drinks hot-drinks hot-drinks cold-drinks cold-drinks cold-drinks master slave CONFIDENTIAL 31
  32. 32. Summary• Modern Applications – Use frameworks, less emphasis on servers – The explosion of connected mobile devices requires a new solution for messaging• Spring Integration – A DSL for introducing messaging abstractions into your spring application – Externalizes messaging concepts from code• AMQP – A TCP/IP protocol not an API – Fast, Reliable, Open, Secure, Scalable• RabbitMQ – AMQP Leader – Backed by VMware CONFIDENTIAL 32
  33. 33. Messaging for Modern ApplicationsPronam Chatterjee, Staff Systems Engineer, VMwareEmail: pronamc@vmware.comTwitter: pronamc CONFIDENTIAL © 2010 SpringSource, A division of VMware. All rights reserved