Operating Systems - Concurrency

2,592 views
2,452 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,592
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
71
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Operating Systems - Concurrency

  1. 1. Operating Systems CMPSCI 377 Concurrency Patterns Emery Berger University of Massachusetts Amherst UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
  2. 2. Finishing Up From Last Time Avoiding deadlock: is this ok?  lock (a); lock (b); lock (b); lock (a); unlock (b); unlock (a); unlock (a); unlock (b); UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 2
  3. 3. Finishing Up From Last Time Not ok – may deadlock.  lock (a); lock (b); lock (b); lock (a); unlock (b); unlock (a); unlock (a); unlock (b); Solution: impose canonical order (acyclic)  lock (a); lock (a); lock (b); lock (b); unlock (b); unlock (b); unlock (a); unlock (a); UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 3
  4. 4. Motivating Example: Web Server web server Client (browser)  Requests HTML, images  not found Server  Caches requests  http://server/Easter-bunny/ Sends to client  200x100/75.jpg client UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
  5. 5. Possible Implementation while (true) { wait for connection; read from socket & parse URL; look up URL contents in cache; if (!in cache) { fetch from disk / execute CGI; put in cache; } send data to client; } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 5
  6. 6. Possible Implementation while (true) { wait for connection; // net read from socket & parse URL; // cpu look up URL contents in cache; // cpu if (!in cache) { fetch from disk / execute CGI;//disk put in cache; // cpu } send data to client; // net } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 6
  7. 7. Problem: Concurrency web Sequential fine until:  server More clients  Bigger server  Multicores, multiprocessors  Goals:  Hide latency of net & disk I/O  Don’t keep clients waiting  Improve throughput  Serve up more pages  clients UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science
  8. 8. Building Concurrent Apps Patterns / Architectures  Thread pools  Producer-consumer  “Bag of tasks”  Worker threads (work stealing)  Goals:  Minimize latency  Maximize parallelism  Keep progs. simple to program & maintain  UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 8
  9. 9. Thread Pools Thread invocation & destruction relatively  expensive Instead: use pool of threads  When new task arrives, get thread from pool  to work on it; block if pool empty Faster with many tasks  Limits max threads  ( ThreadPoolExecutor class in Java)  UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 9
  10. 10. Producer-Consumer Can get pipeline parallelism:  One thread (producer) does work  E.g., I/O  and hands it off to other thread (consumer)  producer consumer UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 10
  11. 11. Producer-Consumer Can get pipeline parallelism:  One thread (producer) does work  E.g., I/O  and hands it off to other thread (consumer)  producer consumer UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 11
  12. 12. Producer-Consumer Can get pipeline parallelism:  One thread (producer) does work  E.g., I/O  and hands it off to other thread (consumer)  producer consumer UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 12
  13. 13. Producer-Consumer Can get pipeline parallelism:  One thread (producer) does work  E.g., I/O  and hands it off to other thread (consumer)  producer consumer UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 13
  14. 14. Producer-Consumer Can get pipeline parallelism:  One thread (producer) does work  E.g., I/O  and hands it off to other thread (consumer)  producer consumer LinkedBlockingQueue Blocks on put() if full, poll() if empty UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 14
  15. 15. Producer-Consumer Web Server Use 2 threads: producer & consumer  queue.put(x) and x = queue.poll();  while (true) { while (true) { wait for connection; do something… read from socket & parse URL; queue.put (x); look up URL contents in cache; } if (!in cache) { fetch from disk / execute CGI; put in cache; } while (true) { send data to client; x = queue.poll(); } do something… } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 15
  16. 16. Producer-Consumer Web Server Pair of threads – one reads, one writes  while (true) { while (true) { wait for connection; URL = queue.poll(); read from socket & parse URL; look up URL contents in cache; queue.put (URL); if (!in cache) { } fetch from disk / execute CGI; put in cache; } send data to client; } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 16
  17. 17. Producer-Consumer Web Server More parallelism –  optimizes common case (cache hit) while (true) { while (true) { wait for connection; URL = queue1.poll(); read from socket & parse URL; look up URL contents in cache; queue1.put (URL); if (!in cache) { 1 } queue2.put (URL); return; } send data to client; } 2 while (true) { URL = queue2.poll(); fetch from disk / execute CGI; put in cache; send data to client; } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 17
  18. 18. When to Use Producer-Consumer Works well for pairs of threads  Best if producer & consumer are symmetric  Proceed roughly at same rate  Order of operations matters  Not as good for  Many threads  Order doesn’t matter  Different rates of progress  UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 18
  19. 19. Producer-Consumer Web Server Have to be careful to balance load across  threads while (true) { while (true) { wait for connection; URL = queue1.poll(); read from socket & parse URL; look up URL contents in cache; queue1.put (URL); if (!in cache) { 1 } queue2.put (URL); } send data to client; } 2 while (true) { URL = queue2.poll(); fetch from disk / execute CGI; put in cache; send data to client; } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 19
  20. 20. Bag of Tasks Collection of mostly independent tasks  worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 20
  21. 21. Bag of Tasks Collection of mostly independent tasks  worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 21
  22. 22. Bag of Tasks Collection of mostly independent tasks  worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 22
  23. 23. Bag of Tasks Collection of mostly independent tasks  worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 23
  24. 24. Bag of Tasks Collection of mostly independent tasks  worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 24
  25. 25. Bag of Tasks Collection of mostly independent tasks  worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 25
  26. 26. Bag of Tasks Collection of mostly independent tasks  addWork worker worker worker worker Bag could also be LinkedBlockingQueue  (put, poll) UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 26
  27. 27. Bag of Tasks Web Server Re-structure this into bag of tasks:  addWork & worker threads  t = bag.poll() or bag.put(t)  while (true) { wait for connection; read from socket & parse URL; look up URL contents in cache; if (!in cache) { fetch from disk / execute CGI; put in cache; } send data to client; } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 27
  28. 28. Bag of Tasks Web Server Re-structure this into bag of tasks:  addWork & worker  t = bag.poll() or bag.put(t)  addWork: Worker: while (true) { while (true) { wait for connection; URL = bag.poll(); bag.put (URL); look up URL contents in cache; } if (!in cache) { fetch from disk / execute CGI; put in cache; } send data to client; } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 28
  29. 29. Bag of Tasks Web Server Re-structure this into bag of tasks:  t = bag.poll() or bag.put(t)  addWork: while (true){ wait for connection; bag.put (URL); } worker worker: while (true) { addWork URL = bag.poll(); look up URL contents in cache; if (!in cache) { fetch from disk / execute CGI; put in cache; worker worker } send data to client; } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 29
  30. 30. Bag of Tasks vs. Prod/Consumer Exploits more parallelism  Even with coarse-grained threads  Don’t have to break up tasks too finely  Easy to change or add new functionality  But: one major performance problem…  UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 30
  31. 31. What’s the Problem? addWork worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 31
  32. 32. What’s the Problem? Contention – single lock on structure  Bottleneck to scalability  addWork worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 32
  33. 33. Work Queues Each thread has own work queue (deque)  No single point of contention  executor executor executor executor Threads now generic “executors”  Tasks (balls): blue = parse, yellow = connect…  UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 33
  34. 34. Work Queues Each thread has own work queue (deque)  No single point of contention  executor executor executor executor UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 34
  35. 35. Work Queues Each thread has own work queue (deque)  No single point of contention  executor executor executor executor UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 35
  36. 36. Work Queues Each thread has own work queue (deque)  No single point of contention  executor executor executor executor UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 36
  37. 37. Work Queues Each thread has own work queue  No single point of contention  executor executor executor executor Now what?  UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 37
  38. 38. Work Stealing When thread runs out of work,  steal work from random other thread worker worker worker worker UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 38
  39. 39. Work Stealing When thread runs out of work,  steal work from top of random deque worker worker worker worker Optimal load balancing algorithm  UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 39
  40. 40. Work Stealing Web Server Re-structure:  readURL, lookUp, addToCache, output myQueue.put(new readURL (url))  while (true) { wait for connection; read from socket & parse URL; look up URL contents in cache; if (!in cache) { fetch from disk / execute CGI; put in cache; } send data to client; } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 40
  41. 41. Work Stealing Web Server Re-structure:  readURL, lookUp, addToCache, output myQueue.put(new readURL (url))  readURL(url) { wait for connection; read from socket & parse URL; myQueue.put (new lookUp (URL)); } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 41
  42. 42. Work Stealing Web Server Re-structure:  readURL, lookUp, addToCache, output myQueue.put(new readURL (url))  readURL(url) { lookUp(url) { wait for connection; look up URL contents in cache; read from socket & parse URL; if (!in cache) { myQueue.put (new lookUp (URL)); myQueue.put (new addToCache (URL)); } } else { myQueue.put (new output(contents)); } addToCache(URL) { } fetch from disk / execute CGI; put in cache; myQueue.put (new output(contents)); } UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 42
  43. 43. Work Stealing Works great for heterogeneous tasks  Convert addWork and worker into units of  work (different colors) Flexible: can easily re-define tasks  Coarse, fine-grained, anything in-between  Automatic load balancing  Separates thread logic from functionality  Popular model for structuring servers  UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 43
  44. 44. The End UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 44

×