Chicago, October 19 - 22, 2010
Concurrent Programming
and Distributed Applications
Mark Fisher, Dave Syer - SpringSource
Goal
●
Demystify concurrent and distributed programming.
●
Identify and help avoid the pitfalls
●
Show how Spring makes it...
Agenda
• Concurrency
• Asynchronous execution
• Tasks, schedules and triggers
• Events, Messaging and intra-process commun...
Concurrency
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Why?
- Performance
- Responsive...
When threads were more esoteric,
concurrency was an "advanced" topic;
now, mainstream developers must be
aware of thread-s...
Thread Safe?
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
public class Counter {
private...
Thread Safe?
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
public class Counter {
private...
Thread Safe?
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
public class Service {
private...
Thread Safety
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
What?
Properly managing concu...
Immutable?
public class StringHolder {
private final int id;
private final String value;
public StringHolder(int id, Strin...
Immutable?
public class StringList {
private final int id;
private final List<String> strings;
public StringList(int id, L...
Immutable?
public class MapHolder {
private final Map<Integer, StringValue> map;
public MapHolder(Map<Integer, StringValue...
Immutable?
private final Map<Integer, StringValue> map;
public MapHolder(Map<Integer, StringValue> map) {
this.map = new H...
Immutable?
private final Map<Integer, StringValue> map;
public MapHolder(Map<Integer, StringValue> map) {
this.map = new H...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Thread Safety: Confinement
When immutability is not an option, consider not sharing
mutable state
• Stack Confinement
– Me...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Synchronization
• The 'synchronized' keyword
• Locks
• wait/notify
• Atomic variables
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Lazy Initialization (recap)
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
public class Se...
Dependency Injection and Singletons
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
public ...
Concurrency
• Background processing and performance optimization
• Java Memory Model: final, volatile, synchronized
• Lock...
Executor and ExecutorService
JDK Functionality
• Configurable thread pools
• Execute Runnables asynchronously
• Submit Cal...
Async Execution Considerations
• Blocking and timeouts
• Cancellation
• Interruption
• Thread Pool Rejection Policies
• Ex...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Asynchronous Execution
• Futures, executors, completion service and thread pools
• @Async
• Lifecycle and SmartLifecycle, ...
Spring Support for Task Management
• TaskExecutor
• TaskScheduler and Trigger
• @Async
• @Scheduled
• File-Polling adapter...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Events and Intraprocess Communication
• The observer pattern: application and framework design
• ApplicationEvent and Appl...
Observer Pattern
Publisher
invoke(“input”)
confirmation
Multicaster
fire(“input”)
observe(“input”)
Observer
Event
ApplicationListener
// This is a listener
public class TransferListener implements
ApplicationListener<TransferEvent> {
pu...
ApplicationEventPublisher
public class TransferService implements
ApplicationEventPublisherAware {
public setApplicationEv...
The Observer Pattern
Publisher
Multicaster
ObserverObserverObserverObserver
ApplicationListener
ApplicationContext
Applica...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Messaging
• Decouple Producers and Consumers
• Observer Pattern but often with extra semantics
– header and payload
– publ...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Staged Event Driven Architecture (SEDA)
Staged Event Driven Architecture (SEDA)
Staged Event Driven Architecture (SEDA)
Staged Event Driven Architecture (SEDA)
Dispatcher
Queue
Ordering
• Multi-threaded applications in general do not preserve
order
• E.g. Messaging
– Producer sends two widgets 1, 2...
Stateful Messaging Patterns
• Scatter-Gather
• Composed Message Processor
• Claim Check
• Publish-Subscribe Channel
• Reci...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Quality of Service, Transactions
Asynchronous hand off =
potential lost messages
Quality of Service, Transactions
Transactional (receive,
send) = no lost messages
Distributed Applications
• RPC, Spring Remoting
• Messaging
• Middleware: HTTP, JDBC, JMS, AMQP
• Latency
• QoS, transacti...
Distributed Application or System
• Application:
– Multiple JVM process nodes
– Same binaries
– Same release schedule
• Sy...
Patterns of Distributed Transactions
• Full XA with 2PC
• XA with the 1PC Optimisation
• XA and the Last Resource Gambit
•...
Remote Procedure Call (RPC)
• RPC is synchronous inter-process communication
• Only evil if the system is not an Applicati...
Messaging
• Messaging is asynchronous inter-process communication
• Can use Messaging to implement RPC
• Transports (some ...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
Latency
• All distributed systems and applications suffer from
latency
– EU-US = 6000km, min network latency 6e6/3e8 = 20m...
Distributed Application and Data
Nosql & Spring Data
• Key value stores
– redis, gemfire
– coherence
– riak, voldemort, memcached
• Document stores
– couch...
SpringSource Gemfire
• Distributed data cache, datastore and compute grid
• Java (so embeddable)
• Low-level API is java.u...
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Demo
SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
Q&A
Source code for demos:
http://git.sprin...
Distributed Transaction: Sunny Day
1. Start messaging transaction
2. Receive message
3. Start database transaction
4. Upda...
Distributed Transaction: Rollback
1. Start messaging transaction
2. Receive message
3. Start database transaction
4. Updat...
Distributed Transaction: Partial Failure
1. Start messaging transaction
2. Receive message
3. Start database transaction
4...
Distributed Data: Shared Database
Application
Cache
Distributed Data: Cache Overflow
Cache
Distributed Cache: Database Meltdown
Distributed Data
Distributed Data
Upcoming SlideShare
Loading in …5
×

Concurrency (Fisher Syer S2GX 2010)

4,236 views

Published on

Concurrent and Distributed Applications with Spring (from SpringOne2GX 2010)

Published in: Technology, Business
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,236
On SlideShare
0
From Embeds
0
Number of Embeds
31
Actions
Shares
0
Downloads
196
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Concurrency (Fisher Syer S2GX 2010)

  1. 1. Chicago, October 19 - 22, 2010 Concurrent Programming and Distributed Applications Mark Fisher, Dave Syer - SpringSource
  2. 2. Goal ● Demystify concurrent and distributed programming. ● Identify and help avoid the pitfalls ● Show how Spring makes it easier to write multi- threaded and multi-process applications
  3. 3. Agenda • Concurrency • Asynchronous execution • Tasks, schedules and triggers • Events, Messaging and intra-process communication • Distributed systems
  4. 4. Concurrency SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Why? - Performance - Responsiveness - Scalability Where? - Event-driven Architecture - Scheduling Tasks
  5. 5. When threads were more esoteric, concurrency was an "advanced" topic; now, mainstream developers must be aware of thread-safety issues. --Brian Goetz
  6. 6. Thread Safe? SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. public class Counter { private boolean active = false; public boolean isActive() { return active; } public void setActive(boolean active) { this.active = active; } }
  7. 7. Thread Safe? SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. public class Counter { private volatile int count = 0; public int increment() { return count++; } }
  8. 8. Thread Safe? SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. public class Service { private volatile Resource resource; public void process(String data) { if (this.resource == null) { this.resource = new Resource(); } this.resource.process(data); } }
  9. 9. Thread Safety SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. What? Properly managing concurrent access to mutable state. How? a) avoid mutability b) don't share c) synchronize
  10. 10. Immutable? public class StringHolder { private final int id; private final String value; public StringHolder(int id, String value) { this.id = id; this.value = value; } public void display() { Sysem.out.println(id + ": " + this.value); } } SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
  11. 11. Immutable? public class StringList { private final int id; private final List<String> strings; public StringList(int id, List<String> strings) { this.id = id; this.strings = strings; } public void display() { Sysem.out.println(id + ": " + this.strings); } } SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
  12. 12. Immutable? public class MapHolder { private final Map<Integer, StringValue> map; public MapHolder(Map<Integer, StringValue> map) { this.map = new HashMap<Integer, StringValue>(map); } public void display() { System.out.println(this.map); } } SpringOne 2GX 2010. All rights reserved. Do not distribute without permission.
  13. 13. Immutable? private final Map<Integer, StringValue> map; public MapHolder(Map<Integer, StringValue> map) { this.map = new HashMap<Integer, StringValue>(map); } SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. public static class StringValue { private String string; public StringValue(String string) { this.string = string; } public void setString(String string) { this.string = string; } } public static class StringValue { private String string; public StringValue(String string) { this.string = string; } public void setString(String string) { this.string = string; } }
  14. 14. Immutable? private final Map<Integer, StringValue> map; public MapHolder(Map<Integer, StringValue> map) { this.map = new HashMap<Integer, StringValue>(map); } SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. public static class StringValue { private final String string; public StringValue(String string) { this.string = string; } } public static class StringValue { private final String string; public StringValue(String string) { this.string = string; } }
  15. 15. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  16. 16. Thread Safety: Confinement When immutability is not an option, consider not sharing mutable state • Stack Confinement – Method Parameters – Local Variables • Thread Confinement – ThreadLocal – Custom Scopes in Spring
  17. 17. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  18. 18. Synchronization • The 'synchronized' keyword • Locks • wait/notify • Atomic variables
  19. 19. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  20. 20. Lazy Initialization (recap) SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. public class Service { private volatile Resource resource; public void process(String data) { if (this.resource == null) { this.resource = new Resource(); } this.resource.process(data); } }
  21. 21. Dependency Injection and Singletons SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. public class Service { private volatile Resource resource; public void setResource(Resources resource) { this.resource = resource; } public void process(String data) { this.resource.process(data); } }
  22. 22. Concurrency • Background processing and performance optimization • Java Memory Model: final, volatile, synchronized • Locks: wait, notify, utility wrappers • Concurrent collections, immutable wrappers • Queue, BlockingQueue, DelayQueue • Deadlock, livelock, starvation • Immutability, stateless components, "hidden" state, thread safety • Typical Spring application concerns • Stateful components in Spring
  23. 23. Executor and ExecutorService JDK Functionality • Configurable thread pools • Execute Runnables asynchronously • Submit Callable<V> that returns Future<V>
  24. 24. Async Execution Considerations • Blocking and timeouts • Cancellation • Interruption • Thread Pool Rejection Policies • Excessive thread creation • Context-switching overhead
  25. 25. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  26. 26. Asynchronous Execution • Futures, executors, completion service and thread pools • @Async • Lifecycle and SmartLifecycle, cf. InitializingBean • Gateways with Futures • Timeouts
  27. 27. Spring Support for Task Management • TaskExecutor • TaskScheduler and Trigger • @Async • @Scheduled • File-Polling adapter in Spring Integration • Message listener containers – JMS – AMQP – Spring Integration – GemFire • Spring Batch
  28. 28. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  29. 29. Events and Intraprocess Communication • The observer pattern: application and framework design • ApplicationEvent and ApplicationListener • Messaging • SEDA • Ordering • Stateful Messaging patterns
  30. 30. Observer Pattern Publisher invoke(“input”) confirmation Multicaster fire(“input”) observe(“input”) Observer Event
  31. 31. ApplicationListener // This is a listener public class TransferListener implements ApplicationListener<TransferEvent> { public onApplicationEvent(TransferEvent event) { this.auditLogger.log(event.getTransfer()); } … } ApplicationEvent java.util.Event
  32. 32. ApplicationEventPublisher public class TransferService implements ApplicationEventPublisherAware { public setApplicationEventPublisher( ApplicationEventPublisher publisher) { this.publisher = publisher; } public void transfer(Transfer transfer) { ... this.publisher.publish(new TransferEvent(transfer)); }
  33. 33. The Observer Pattern Publisher Multicaster ObserverObserverObserverObserver ApplicationListener ApplicationContext ApplicationEventPublisher
  34. 34. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  35. 35. Messaging • Decouple Producers and Consumers • Observer Pattern but often with extra semantics – header and payload – publish-subscribe or point-to-point – acks and nacks and other protocol details – persistence and quality of service – once-only or at-least-once delivery widgets
  36. 36. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  37. 37. Staged Event Driven Architecture (SEDA)
  38. 38. Staged Event Driven Architecture (SEDA)
  39. 39. Staged Event Driven Architecture (SEDA)
  40. 40. Staged Event Driven Architecture (SEDA) Dispatcher Queue
  41. 41. Ordering • Multi-threaded applications in general do not preserve order • E.g. Messaging – Producer sends two widgets 1, 2 – Consumer receives 2, 1 • To preserve order expect some overhead – Storage (stateful patterns) – Processing time (wait for out of order messages)
  42. 42. Stateful Messaging Patterns • Scatter-Gather • Composed Message Processor • Claim Check • Publish-Subscribe Channel • Recipient List Router • Splitter • Multi-valued Router • Aggregator • Resequencer
  43. 43. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  44. 44. Quality of Service, Transactions Asynchronous hand off = potential lost messages
  45. 45. Quality of Service, Transactions Transactional (receive, send) = no lost messages
  46. 46. Distributed Applications • RPC, Spring Remoting • Messaging • Middleware: HTTP, JDBC, JMS, AMQP • Latency • QoS, transactions, durability and guaranteed delivery • Distributed data: partitioning and eventual consistency • Gemfire • Spring Integration adapters
  47. 47. Distributed Application or System • Application: – Multiple JVM process nodes – Same binaries – Same release schedule • System: – Multiple JVM process nodes – Different binaries – Different release schedule per node • Blurred boundary if Application has to have a rolling upgrade with no downtime
  48. 48. Patterns of Distributed Transactions • Full XA with 2PC • XA with the 1PC Optimisation • XA and the Last Resource Gambit • Shared Transaction Resource • Best Efforts 1PC • Non-transactional Access • Wing and a Prayer (anti-pattern) http://www.javaworld.com/javaworld/jw-01-2009/jw-01- spring-transactions.html JTA + Spring Spring Data + Oracle Spring JMS
  49. 49. Remote Procedure Call (RPC) • RPC is synchronous inter-process communication • Only evil if the system is not an Application • Transports: – RMI – HTTP – JMS – JDBC – ... • Spring Remoting helps strategise and configure: defers important architectural choices • Spring Integration provides additional options
  50. 50. Messaging • Messaging is asynchronous inter-process communication • Can use Messaging to implement RPC • Transports (some point-to-point only): – RMI – HTTP – JMS – JDBC – AMQP – ... • Spring Integration: helps strategise and configure: defers important architectural choices
  51. 51. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  52. 52. Latency • All distributed systems and applications suffer from latency – EU-US = 6000km, min network latency 6e6/3e8 = 20ms – Add marshal/unmarshal overhead (variable) – Transaction (min 50ms) • Often much worse, e.g. >100ms even on local network • Request-reply patterns double the problem • Over-modularization: each inter-process hop is expensive • Abstractions (e.g. Spring) are dangerous • Measure and analyse
  53. 53. Distributed Application and Data
  54. 54. Nosql & Spring Data • Key value stores – redis, gemfire – coherence – riak, voldemort, memcached • Document stores – couchdb, mongodb • Sparse table or column stores – cassandra, bigtable • Graph or object stores – neo4j • Distributed filesystem – hdf
  55. 55. SpringSource Gemfire • Distributed data cache, datastore and compute grid • Java (so embeddable) • Low-level API is java.util.Map • Many high-level abstractions – Transactions – Functions and node affinity – Events, continuous queries – Replication • Spring Gemfire: http://git.springsource.org/spring-gemfire
  56. 56. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Demo
  57. 57. SpringOne 2GX 2010. All rights reserved. Do not distribute without permission. Q&A Source code for demos: http://git.springsource.org/s2gx-2010/concurrent-programming- distributed-applications
  58. 58. Distributed Transaction: Sunny Day 1. Start messaging transaction 2. Receive message 3. Start database transaction 4. Update database 5. Commit database transaction 6. Commit messaging transaction
  59. 59. Distributed Transaction: Rollback 1. Start messaging transaction 2. Receive message 3. Start database transaction 4. Update database, fail! 5. Rollback database transaction 6. Rollback messaging transaction
  60. 60. Distributed Transaction: Partial Failure 1. Start messaging transaction 2. Receive message 3. Start database transaction 4. Update database 5. Commit database transaction 6. Commit messaging transaction, fail!
  61. 61. Distributed Data: Shared Database Application Cache
  62. 62. Distributed Data: Cache Overflow Cache
  63. 63. Distributed Cache: Database Meltdown
  64. 64. Distributed Data
  65. 65. Distributed Data

×