0
Beyond Horizontal Scalability:    Concurrency & Messaging Using Spring    Bruce Snyder, Senior Software Engineer, SpringSo...
Problem Statement    • Software systems are growing larger than ever before           –   Virtual Machines           –   P...
Typical Application Layers                                 3Friday, July 8, 2011
Typical Code Construction    • Command-and-control scheme           – One method calls another           – Blocking calls ...
Typical Component Interaction    • Single memory space    • Using the call stack    • Sequential execution    • Assumption...
Typical Assumptions    • All work must take place in a single call stack    • Example:           –   Receive request      ...
• The higher the number of assumptions, the more tightly      coupled the system                                          ...
Typical Client Request                             8Friday, July 8, 2011
Typical Request Growth                             9Friday, July 8, 2011
Typical Strategy For Horizontal Scaling                                              10Friday, July 8, 2011
Typical Assumptions    • All work must take place on a single machine    • Example:           –   Receive request         ...
Treating the Symptoms    • Horizontal scale is good to a point           – Treat the symptoms instead of fixing the cause ...
The Problem With Distributed Systems    • Most developers don’t understand them           – Call stack provides specific i...
Is There a Better Solution?    • Insert a level of indirection           – Remove direct interaction between components   ...
Concurrency    • Concurrency is a style of asynchronous execution                                                         ...
Concurrency With Spring    • Spring provides the TaskExecutor interface           – Abstracts the execution of a Runnable,...
JDK ExecutorService   ExecutorService executorService = Executors.newSingleThreadExecutor();   executorService.execute(tas...
Spring TaskExecutor     <bean id="myTask" class="org.bsnyder.spring.concurrency.MyTask" />    <bean id="springConcurrencyE...
Spring TaskExecutor   public class SpringConcurrencyExample {   !    @Autowired   !    private Runnable task;   !    @Auto...
Messaging    • Messaging is a style of communication    • Often used for integration purposes                             ...
Messaging With Spring    • JMS Support           – JmsTemplate for sync send and receive           – DefaultMessageListene...
Spring JmsTemplate                                                                      Synchronous   <bean id="connection...
Spring JmsTemplate                                                                           Synchronous         // Use th...
DefaultMessageListenerContainer                                                                          Asynchronous    •...
Spring RabbitTemplate                                                                        Synchronous     <bean id="rab...
Spring RabbitTemplate                                                                           Synchronous      // Use th...
SimpleMessageListenerContainer                                                                              Asynchronous  ...
Best of Both Worlds    • What if concurrency and messaging was provided      together?                                    ...
Spring Integration    • A framework for integration    • Styles of integration           – Intra-application integration  ...
Enterprise Integration Patterns (EIP)                       http://enterpriseintegrationpatterns.com/                     ...
Spring Integration    • Provides both concurrency and messaging           – Message Endpoints                 • Connection...
Spring Integration    • Supports           –   AMQP           –   Email           –   File system           –   Gemfire   ...
Spring Integration                         33Friday, July 8, 2011
Spring Integration                         34Friday, July 8, 2011
Spring Integration                         35Friday, July 8, 2011
Spring Integration                         36Friday, July 8, 2011
Spring Integration                         37Friday, July 8, 2011
Spring Integration                         38Friday, July 8, 2011
Spring Integration                         39Friday, July 8, 2011
Spring Integration                         40Friday, July 8, 2011
Spring Integration                         41Friday, July 8, 2011
Spring Integration                         42Friday, July 8, 2011
Your Coffee Shop Does Not Use 2PC    •    Order is accepted    •    Cup is labeled and placed in the queue    •    Money i...
Conversation Pattern    • Interaction between two parties           – Short synchronous interaction           – Longer asy...
Cafe Demo                       45Friday, July 8, 2011
Thank You!           Q&AFriday, July 8, 2011
Upcoming SlideShare
Loading in...5
×

Beyond Horizontal Scalability: Concurrency and Messaging Using Spring

3,678

Published on

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,678
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
63
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

Transcript of "Beyond Horizontal Scalability: Concurrency and Messaging Using Spring"

  1. 1. Beyond Horizontal Scalability: Concurrency & Messaging Using Spring Bruce Snyder, Senior Software Engineer, SpringSource/VMwareFriday, July 8, 2011
  2. 2. Problem Statement • Software systems are growing larger than ever before – Virtual Machines – Physical Machines – More CPUs – Multi-core CPUs – More requests – More data • With so many hardware changes, shouldn’t our software practices change too?Friday, July 8, 2011
  3. 3. Typical Application Layers 3Friday, July 8, 2011
  4. 4. Typical Code Construction • Command-and-control scheme – One method calls another – Blocking calls • Assumptions – One task takes place at a time – The order of operations in known – The provider of a particular function is known – All execution happens in a single JVM 4Friday, July 8, 2011
  5. 5. Typical Component Interaction • Single memory space • Using the call stack • Sequential execution • Assumptions – No separation between: • Knowing what needs to happen next • Knowing which method to invoke – Interactions are always known at compile time 5Friday, July 8, 2011
  6. 6. Typical Assumptions • All work must take place in a single call stack • Example: – Receive request – Verify data – Save data – Query data – Generate PDF – Send email – Render response 6Friday, July 8, 2011
  7. 7. • The higher the number of assumptions, the more tightly coupled the system 7Friday, July 8, 2011
  8. 8. Typical Client Request 8Friday, July 8, 2011
  9. 9. Typical Request Growth 9Friday, July 8, 2011
  10. 10. Typical Strategy For Horizontal Scaling 10Friday, July 8, 2011
  11. 11. Typical Assumptions • All work must take place on a single machine • Example: – Receive request – Verify data – Save data – Query data – Generate PDF – Send email – Render response 11Friday, July 8, 2011
  12. 12. Treating the Symptoms • Horizontal scale is good to a point – Treat the symptoms instead of fixing the cause of the illness • Throughput of functions should not equal response time! • This is not the case at forward-thinking companies – Amazon.com 12Friday, July 8, 2011
  13. 13. The Problem With Distributed Systems • Most developers don’t understand them – Call stack provides specific interaction style – Most object-oriented systems focus on structure vs. interaction • Interaction becomes more important than structure – Mediator pattern - encapsulates how objects interact – Observer pattern - notifies dependent objects of state change • The real point of service-orientated design – Composition rules are rather simple – Loosely coupled interaction becomes very important 13Friday, July 8, 2011
  14. 14. Is There a Better Solution? • Insert a level of indirection – Remove direct interaction between components – Extract the interaction into a separate element – Enterprise Integration Patterns (EIP) • http://enterpriseintegrationpatterns.com/ • Simplify the rules of component interaction – Remove the coordination and continuation style of interaction – Just send data 14Friday, July 8, 2011
  15. 15. Concurrency • Concurrency is a style of asynchronous execution 15Friday, July 8, 2011
  16. 16. Concurrency With Spring • Spring provides the TaskExecutor interface – Abstracts the execution of a Runnable, i.e., asynchronous work – SimpleAsyncTaskExecutor • Creates a new thread for each task – ThreadPoolTaskExecutor • Uses a JDK 1.5 ThreadPoolExecutor to create a pool of threads – org.springframework.scheduling.concurrent package • Many scheduling related utility classes – WorkManagerTaskExecutor • Delegates to a JCA 1.5 CommonJ WorkManager 16Friday, July 8, 2011
  17. 17. JDK ExecutorService ExecutorService executorService = Executors.newSingleThreadExecutor(); executorService.execute(task); executorService.shutdown(); executorService = Executors.newFixedThreadPool(10); executorService.execute(task); executorService.shutdown(); int corePoolSize = 5; int maxPoolSize = 10; long keepAliveTime = 5000; executorService = new ThreadPoolExecutor(corePoolSize, maxPoolSize, keepAliveTime, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>()); executorService.execute(task); executorService.shutdown(); 17Friday, July 8, 2011
  18. 18. Spring TaskExecutor <bean id="myTask" class="org.bsnyder.spring.concurrency.MyTask" /> <bean id="springConcurrencyExample" class="org.bsnyder.spring.concurrency.SpringConcurrencyExample" /> <bean id="simpleAsyncTaskExecutor" class="org.springframework.core.task.SimpleAsyncTaskExecutor" p:daemon="false" /> <bean id="taskExecutor" class="org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor" p:corePoolSize="5" p:maxPoolSize="10" p:queueCapacity="30" /> 18Friday, July 8, 2011
  19. 19. Spring TaskExecutor public class SpringConcurrencyExample { ! @Autowired ! private Runnable task; ! @Autowired ! private SimpleAsyncTaskExecutor simpleAsyncTaskExecutor; ! @Autowired ! private ThreadPoolTaskExecutor threadPoolTaskExecutor; ! ! public void runExamples() { ! ! simpleAsyncTaskExecutor.execute(task); ! ! threadPoolTaskExecutor.execute(task); ! } ! ! public static void main(String[] args) { ! ! ApplicationContext context = ! ! ! new ClassPathXmlApplicationContext("/META-INF/spring/executor-context.xml", SpringConcurrencyExample.class); ! ! SpringConcurrencyExample example = context.getBean(SpringConcurrencyExample.class); ! ! example.runExamples(); ! } } 19Friday, July 8, 2011
  20. 20. Messaging • Messaging is a style of communication • Often used for integration purposes 20Friday, July 8, 2011
  21. 21. Messaging With Spring • JMS Support – JmsTemplate for sync send and receive – DefaultMessageListenerContainer and SimpleMessageListenerContainer for async receive • javax.jms.MessageListener • org.springframework.jms.listener.SessionAwareMessageListener • org.springframework.jms.listener.adapter.MessageListenerAdapter • AMQP Support – RabbitTemplate for sync send and receive – SimpleMessageListenerContainer for async receive • org.springframework.amqp.core.MessageListener • org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter 21Friday, July 8, 2011
  22. 22. Spring JmsTemplate Synchronous <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" p:brokerURL="tcp://localhost:61616" /> <bean id="destination" class="org.apache.activemq.command.ActiveMQQueue"> <constructor-arg value="FOO.TEST" /> </bean> <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate" p:connectionFactory-ref="connectionFactory" p:defaultDestination-ref="destination" /> <bean id="messageProducer" class="org.bsnyder.spring.jms.producer.SimpleMessageProducer" p:jmsTemplate-ref="jmsTemplate" /> 22Friday, July 8, 2011
  23. 23. Spring JmsTemplate Synchronous // Use the default destination jmsTemplate.convertAndSend("Hello World!"); // Use a different destination jmsTemplate.convertAndSend(“TEST.BAR”, “Hello World!”); // Use a different destination String textMessage1 = (String) jmsTemplate.receiveAndConvert(); // Use a different destination String textMessage2 = (String) jmsTemplate.receiveAndConvert(“TEST.BAR”); 23Friday, July 8, 2011
  24. 24. DefaultMessageListenerContainer Asynchronous • XML configuration <bean id="connectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory" p:brokerURL="tcp://localhost:61616" /> <bean id="messageListener" class="org.bsnyder.spring.jms.listener.SimpleMessageListener" /> <jms:listener-container concurrency="5-10"> <jms:listener destination="FOO.TEST" ref="messageListener"/> </jms:listener-container> 24Friday, July 8, 2011
  25. 25. Spring RabbitTemplate Synchronous <bean id="rabbitAdmin" class="org.springframework.amqp.rabbit.core.RabbitAdmin"> <constructor-arg ref="connectionFactory"/> </bean> <bean id="connectionFactory" class="org.springframework.amqp.rabbit.connection.SingleConnectionFactory" p:host="localhost" p:port="5672" p:username="guest" p:password="guest"/> <bean id="rabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate" p:connectionFactory-ref="connectionFactory" p:routingKey="TEST.FOO" p:queue="TEST.FOO" /> <bean id="messageProducer" class="org.bsnyder.spring.amqp.producer.SimpleMessageProducer" p:rabbitAdmin-ref="rabbitAdmin" p:rabbitTemplate-ref="rabbitTemplate"/> 25Friday, July 8, 2011
  26. 26. Spring RabbitTemplate Synchronous // Use the default destination rabbitTemplate.convertAndSend("Hello World!"); // Use a different destination rabbitTemplate.send(“TEST.FOO”, “TEST.FOO”, message); // Use a different destination Message message1 = rabbitTemplate.receiveAndConvert(); // Use a different destination String textMessage2 = (String) rabbitTemplate.receive(“TEST.FOO”); 26Friday, July 8, 2011
  27. 27. SimpleMessageListenerContainer Asynchronous <bean id="connectionFactory" class="org.springframework.amqp.rabbit.connection.SingleConnectionFactory" p:host="localhost" p:port="5672" p:username="guest" p:password="guest"/> <bean id="rabbitAdmin" class="org.springframework.amqp.rabbit.core.RabbitAdmin"> <constructor-arg ref="connectionFactory"/> </bean> <bean id="rabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate" p:connectionFactory-ref="connectionFactory" p:routingKey="TEST.FOO" p:queue="TEST.FOO" /> <bean id="messageListener" class="org.bsnyder.spring.amqp.listener.SimpleMessageListener" p:rabbitTemplate-ref="rabbitTemplate"/> <bean id="listenerContainer" class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer" p:connectionFactory-ref="connectionFactory" p:queueName="TEST.FOO" p:concurrentConsumers="5" p:listener="messageListener" /> 27Friday, July 8, 2011
  28. 28. Best of Both Worlds • What if concurrency and messaging was provided together? 28Friday, July 8, 2011
  29. 29. Spring Integration • A framework for integration • Styles of integration – Intra-application integration – Inter-application integration – External system integration 29Friday, July 8, 2011
  30. 30. Enterprise Integration Patterns (EIP) http://enterpriseintegrationpatterns.com/ 30Friday, July 8, 2011
  31. 31. Spring Integration • Provides both concurrency and messaging – Message Endpoints • Connections between services – Channel Adapters • Adapter between application and message broker – Messaging Gateways • Provides uni-directional or bi-directional messaging – Service Activators • Invokes a services based on an incoming message – Routers • Determines where to dispatch a message – Splitters and Aggregators • Breaks up a message and reassembles it after processing 31Friday, July 8, 2011
  32. 32. Spring Integration • Supports – AMQP – Email – File system – Gemfire – JMS – JMX – MongoDB – Redis – Spring Batch – Testing – Web Services 32Friday, July 8, 2011
  33. 33. Spring Integration 33Friday, July 8, 2011
  34. 34. Spring Integration 34Friday, July 8, 2011
  35. 35. Spring Integration 35Friday, July 8, 2011
  36. 36. Spring Integration 36Friday, July 8, 2011
  37. 37. Spring Integration 37Friday, July 8, 2011
  38. 38. Spring Integration 38Friday, July 8, 2011
  39. 39. Spring Integration 39Friday, July 8, 2011
  40. 40. Spring Integration 40Friday, July 8, 2011
  41. 41. Spring Integration 41Friday, July 8, 2011
  42. 42. Spring Integration 42Friday, July 8, 2011
  43. 43. Your Coffee Shop Does Not Use 2PC • Order is accepted • Cup is labeled and placed in the queue • Money is exchanged • Coffee drink is processed • Multiple baristas = competing consumers • Drinks are processed out of order = correlation id (label) • Cannot pay for drink = discard (write-off) • Erroneous drink = retry • Drink machine fails = compensating action (refund, etc.) 43Friday, July 8, 2011
  44. 44. Conversation Pattern • Interaction between two parties – Short synchronous interaction – Longer asynchronous interaction • Other examples – Amazon.com 44Friday, July 8, 2011
  45. 45. Cafe Demo 45Friday, July 8, 2011
  46. 46. Thank You! Q&AFriday, July 8, 2011
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×