Concurrency and Multithreading Demistified - Reversim Summit 2014


Published on

Life as a software engineer is so exciting! Computing power continue to rise exponentially, software demands continue to rise exponentially as well, so far so good. The bad news are that in the last decade the computing power of single threaded application remains almost flat.
If you decide to continue ignoring concurrency and multi-threading the gap between the problems you are able to solve and your hardware capabilities will continue to rise. In this session we will discuss different approaches for taming the concurrency beast, such as shared mutability,shared immutability and isolated mutability actors, STM, etc we will discuss the shortcomings and the dangers of each approach and we will compare different programming languages and how they choose to tackle/ignore concurrency.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Concurrency and Multithreading Demistified - Reversim Summit 2014

  1. 1. Reversim  Summit  2014     Concurrency  and  Multi-­‐Threading   Demystified   ! Haim Yadid - Performize-IT
  2. 2. About  Me:  Haim  Yadid •21 Years of SW development experience •Performance Expert •Consulting R&D Groups •Training: Java Performance Optimization
  3. 3. Moore’s  Law •Number of transistors doubles constantly •CPU frequency à stalled. •Performance boost through parallelism
  4. 4. Yes….  But •Performance is not a problem anymore •We prefer commodity hardware •We have Hadoop and Big Data!!! •Hardware is cheap. •Several processes will do •My programming language
 will protect me
  5. 5. ! Concurrency
  6. 6. Concurrency •Decomposition of your program
 into independently executing processes •About structure •About design •It is not something you code it is something you architect
  7. 7. Parallelism •Simultaneous execution of (possibly related) computations •On different cores •About execution and scheduling •Not about architecture
  8. 8. Worker •Someone who is capable of doing an efficient job •Hard working •Can execute •Will do it with pleasure Sid
  9. 9. State •What in Sid’s mind ? •Whats in the environment •In a certain point in time Pile  of  sand
  10. 10. Task •A Unit of work scheduled to sid •Possibly code •Data •Means to communicate with Sid •E.g. Java: Callable Task
  11. 11. Our  World
  12. 12. Liveliness  Problems •Deadlock •Starvation •Waiting for a train that will never come func count() {
 for i := 0; i < 1000; i++ {
 time.Sleep(10 * time.Millisecond)
 } func main() {
 go count()
 time.Sleep(3000 * time.Millisecond)
 for {} 
  13. 13. Data  Races •Inopportune interleaving •Stale values •Loosing updates •Infinite loops class Foo { private HashSet h = new HashSet(); ! } ! boolean introduceNewVal(Object v) { if (!h.contains(v)) {h.add(v); return true; } return false; } Java
  14. 14. Performance •Communication overhead •Contention •Imbalanced task distribution •False sharing val v = Vector(…) + 1) Scala
  15. 15. Whats  wrong  here? •IncrementX accessed from ThreadX •IncrementY accessed from Thread Y •An aggregator thread will read both Class  T  {
      volatile  int  x  =  0;  
      volatile  int  y=0;
 ! long incrementX() { x++; } long incrementY() { y++; }
 } False  sharing:  cache  coherency  -­‐>  hitting  same  cache  line Java
  16. 16. Sid  Life  Cycle •Creation •Destruction
  17. 17. Heavy  Weight  Sid •Threads •Thread Pools/Executors: Single Threaded/Fixed size/Bounded min..max/ Unbounded •Fork Join pools (Work stealing) •Storm Bolts
  18. 18. Lightweight  Sid •Sid is not a thread rather Scheduled to a thread •Green Threads •Actors (Scala/Akka) •Agents(Closures) •Go Routines
  19. 19. Communication •BlockingQueues •Futures and promises CompletableFuture •Dequeues •<- Go Channels •! (Scala actors)
  20. 20. Serial  Execution •Three queries to DB executed •In Serial •Long response time doGet(req,resp) {
 rs1 = runQuery1()
 rs2 = runQuery2()
 rs3 = runQuery3() resp.write(mergeResults(rs1,rs2,rs3))
 } Can  be  parallelized Pseudo(Java)
  21. 21. Create  Threads •Run three queries in parallel….. •but: doGet(req,resp) {
 q1 = new Query();
 new Thread(q1).start();
 q2 = new Query();
 new Thread(q2).start();
 q3 = new Query();
 new Thread(q3).start();
 ! } rs1 = q1.getRs();
 rs2 = q2.getRs();
 rs3 = q3.getRs();
 Thread  leak  +  Thread  creation  overhead  +  data  races Pseudo(Java)
  22. 22. Thread  Pool doGet(req,resp) {
 ExecutorService e = Executors.newFixedThreadPool(3); 
 ArrayList tasks = …;
 tasks.add(new QueryTask(Query1)) …. 3 List<Future<Integer>> fs;
 fs = e.invokeAll(tasks); // invoke all in parallel 
 ArrayList results = …
 for (Future<Integer> f : l) { // collect results 
 if (f.isDone()) {
 } resp.write(mergeResults(results));
 } Thread  pool  leak  +  Thread  pool  creation  overhead Pseudo(Java)
  23. 23. Thread  Pool  2 static ExecutorService e = Executors.newFixedThreadPool(3); ! doGet(req,resp) { ArrayList tasks = …; tasks.add(new QueryTask(Query1)) …. 3 List<Future<Integer>> fs; fs = e.invokeAll(tasks); // invoke all in parallel ! } ArrayList results = … for (Future<Integer> f : l) { // collect results if (f.isDone()) { res.add(f.get()); } } resp.write(mergeResults(results)); Size  ?/  share  thread  pool?  /  name  thread  pool  threads Pseudo(Java)
  24. 24. Same  Example  With  Go func  execQuery(query  string,  c  chan  *Row)  {
          c  <-­‐  db.Query(query)
 ! func  doGet(req,resp) {
 c := make(chan *Row) go execQuery(query1,c)
 go execQuery(query2,c)
 go execQuery(query3,c)
 for i := 0; i<3 ; i++ combineRs(rs <-c) 
 } Pseudo(Go)
  25. 25. State  Management •Eventually we need to have state •It is easy to deal with state when we have one Sid •But what happens when there are several •We have three approaches •Most are familiar with only one
  26. 26. Handling  State •Shared Mutability •Shared Immutability •Isolated Mutability
  27. 27. Shared  Mutability •Multiple Sids access the same data •Mutate the state •Easily exposed to concurrency hazards
  28. 28. Visibility •Change made by sid1 is visible to sid2? •Solutions •volatile keyword •Memory Model(Happens before) Memory L3  cache L2  cache L1  cache •compiler reordering Registers •Caches CPU •Not so simple
  29. 29. Atomicity •What can be done in a single step ? •CAS constructs (Compare and swap) •AtomicInteger •AtomicLong •ConcurrenctHashMap putIfAbsent
  30. 30. Atomicity  is  not  Viral •An (almost) real example •A non transactional database •Balance per user •Use atomicity to solve the problem class  User  {        private  AtomicLong  balance  =  …..   !      int updateBalance(int diff) { long temp = balance.addAndGet(diff); setToDB(temp); } } Java
  31. 31. Locking  (Pessimistic) •Synchronized •Sync •Mutex •Reentrant lock •Semaphores •Synchronized Collections
  32. 32. Beware  Sync  Collections •Two synchronized operations •are not synchronized if  (syncedHashMap.get(“b”)  !=  null)  {      syncedHashMap.put(“b”,2);   } Java
  33. 33. Hazards   •Fine grained •—> Deadlocks •Coarsed Grained •—> contention
  34. 34. STM  (Optimistic) •Software Transactional Memory •Transactional semantics ACI: (not Durable) •Atomic, •Consistent and •Isolated •No deadlocks - when collision retry! •Clojure refs , Akka refs STM  performance  problem  when  there  are  too  many  mutations.
  35. 35. STM •Clojure refs and dosync •Scala Refs and atomic •Multiverse Java
  36. 36. Mutliverse  STM  Example import org.multiverse.api.references.*; import static org.multiverse.api.StmUtils.*; ! ! ! public class Account{ private final TxnRef<Date> lastUpdate; private final TxnInteger balance; public Account(int balance){ this.lastUpdate = newTxnRef<Date>(new Date()); this.balance = newTxnInteger(balance); } Java
  37. 37. Mutliverse  STM  Example public void incBalance(final int amount, final Date date){ atomic(() ->{; lastUpdate.set(date); ! if(balance.get()<0){ throw new IllegalStateException("Not enough money"); } }); } } Java8
  38. 38. Mutliverse  STM  Example ! public static void transfer(final Account from, final Account to, final int amount){ Java8 atomic(()->{ Date date = new Date(); from.incBalance(-amount, date); to.incBalance(amount, date); }); } Retry  ahead  beware  of  side  effects  
  39. 39. Pure  Immutability   •We have shared state •But Shared state is read only (after construction) •No concurrency issues •No deadlocks •No race conditions •No stale values •Optimal for cache
  40. 40. Support  From  Languages •Functional Languages favour immutability •vals in scala clojure •final keyword in java •freeze method in ruby
  41. 41. Immutable  Object  Example Object cannot be changed after construction all fields are final public final Class MySet {
 private final Set<String> vals = new HashSet<String>();
 public MySet(String names[]) {
 for(name:names) vals.add(name);
 public boolean containsVal(String name);…..
 } Java
  42. 42. CopyOnWrite  Collections   •Any changes to it will create a new copy •Safe •Fast read, read without synchronisation •Iteration is fast •do not support remove() set() add() Bad  performance  when  mutation  rate  is  high  
  43. 43. Persistent  collections •Immutable collections •Which are efficient •Preserve the same O(_) characteristic of a mutable collection. •Shared structure •Recursive definition
  44. 44. Persistent  Trie root 0 A 1 2 C E 1 D F 2,1
  45. 45. Persistent  Trie root root 0 A 1 2 C E E 1 D 1 F
  46. 46. Example  Customization  Cache •A Web application server •Serving Complicated and customisable UI •Each user has it’s own customization (potentially) •Classic for immutable collection •Low mutation rate •High read rate
  47. 47. Example  Customization  Cache •Customization Data is immutable •Customization Data HashMap is a Persistent Map •Cache is represented by a single STM reference •Update will fail if two are performing it at once
  48. 48. Immutability  and  GC •Immutability is great •But: Generates of a lot of objects •When done for short lived objects GC can cope with it •Long lived immutable objects/collections which change frequently may cause GC to have low throughput and high pause times
  49. 49. Isolated  Mutability •No shared state •Each Sid has its own pile of sand •Message passing between Sids •Prefer passing immutable objects
  50. 50. Isolated  Mutability •Javascript Workers •Ruby/NodeJS multi process •Actors (Scala Erlang) •Agents (Clojure) •Go routines/ channels - Go
  51. 51. Actor Actor Isolated   Mutable   State Actor Actor Actor
  52. 52. Building  a  Monitoring  System •Monitoring System (e.g. Nagios) •~100k of monitors •running periodically •Each one has a state. •Consumers are able to query state. •Some monitors may affect other monitor state
  53. 53. Components •MonitorActor (two actors) •HostActor •MonitorCache •SchedulerActor •QueryActor •UpdateActor
  54. 54. Monitor  Actors •MonitorStateActor(MSA) •Alway readys to be queried •State updated by message from MRA •Stateless •MonitorRecalculateActor(MRA) •Maybe recalculating and not responsive •Stateful •Supervises MSA
  55. 55. MonitorsCache •An immutable cache - holds all actor refs •Single view of the world. •Used by SchedulerActor and Query Actor •May have several objects managed By STM
  56. 56. Actor MonitorsCache Status/   state Query   Actor Scheduler MSA MRA
  57. 57. Further  Reading •Java Concurrency In Practice /Brian Goetz •Effective Akka / Jamie Allen •Clojure High Performance Programming / Shantanu Kumar •Programming Concurrency on the JVM: Mastering Synchronization, STM, and Actors /Subramaniam, Venkat
  58. 58. Thanks + Q&A + Contact Me @lifeyx © Copyright Performize-IT LTD.