Multithreading, Blocking IO and Async IO

3,393 views

Published on

Published in: Technology, Spiritual
2 Comments
6 Likes
Statistics
Notes
No Downloads
Views
Total views
3,393
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
0
Comments
2
Likes
6
Embeds 0
No embeds

No notes for slide

Multithreading, Blocking IO and Async IO

  1. 1. Multithreading, Blocking IO and Async IO By Bhavin Turakhia CEO, Directi [email_address]
  2. 2. Agenda <ul><li>Multithreading </li></ul><ul><li>Blocking IO </li></ul><ul><li>Async Blocking IO </li></ul><ul><li>Async Non Blocking IO </li></ul>
  3. 3. Introduction <ul><li>A program performs the following activities – </li></ul><ul><ul><li>Requests Input </li></ul></ul><ul><ul><li>Performs Computations </li></ul></ul><ul><ul><li>Publishes Output </li></ul></ul><ul><li>A program requires the following resources </li></ul><ul><ul><li>CPU </li></ul></ul><ul><ul><li>Memory </li></ul></ul><ul><li>A CPU can only do one thing at a time </li></ul>
  4. 4. Analogy = Task = Process = CPU GOD
  5. 5. Scenario 1 – Computational Task <ul><li>Person => Process </li></ul><ul><li>God => CPU </li></ul><ul><li>Task </li></ul><ul><ul><li>Inspect the Bucket (purely computational) </li></ul></ul><ul><li>God is busy all the time doing exactly what we want ie computing </li></ul>GOD
  6. 6. <ul><li>Rule 1 – We always want to keep God Busy </li></ul>Rule 1 – We always want to keep the CPU Busy
  7. 7. <ul><li>Will adding one more person make a difference? </li></ul>
  8. 8. Scenario 2 – Same Task – Multi-Process <ul><li>Persons => Processes </li></ul><ul><li>God => CPU </li></ul><ul><li>Task </li></ul><ul><ul><li>Inspect the Bucket (purely computational) </li></ul></ul><ul><li>Now God is busy all the time but not doing what we want </li></ul><ul><ul><li>Spends time picking up person A </li></ul></ul><ul><ul><li>Spends time computing </li></ul></ul><ul><ul><li>Spends time putting person A down </li></ul></ul><ul><ul><li>Repeat with person B </li></ul></ul>GOD
  9. 9. <ul><li>Rule 2 – We want to keep God Busy doing important stuff. Switching between Persons is not the best utilization of God’s time </li></ul>Rule 2 – We want to keep the CPU Busy doing important stuff. Switching between processes is not the best utilization of the CPUs time
  10. 10. <ul><li>Corollary – Multiple processes reduce performance for tasks that are CPU-bound </li></ul>GOD
  11. 11. Scenario 2 - IO <ul><li>Person => Process </li></ul><ul><li>God => CPU </li></ul><ul><li>Bucket => Input </li></ul><ul><li>Task </li></ul><ul><ul><li>Wait for Bucket to be filled (Input) </li></ul></ul><ul><ul><li>Inspect Bucket (Compute) </li></ul></ul>GOD
  12. 12. <ul><li>But God is twiddling his thumbs while the bucket is filling!!! </li></ul>
  13. 13. <ul><li>Rule 1 – We always want to keep God Busy </li></ul>
  14. 14. Scenario 3 – Multiple Processes <ul><li>Persons => Processes </li></ul><ul><li>God => CPU </li></ul><ul><li>Bucket => Input </li></ul><ul><li>God can now switch between Persons while they are “blocked” on Input </li></ul>GOD
  15. 15. <ul><li>Rule 3 – If a person is waiting for his bucket to be filled, God can drop him and pick up another person </li></ul>Rule 3 – If a process is waiting for IO, the CPU can switch its attention to another Process (context switching)
  16. 16. <ul><li>But Persons are Heavy!!! </li></ul>
  17. 17. Scenario 4 – Multi-threading <ul><li>Person => Process </li></ul><ul><li>Hands => Threads </li></ul><ul><li>God => CPU </li></ul><ul><li>Bucket => Input </li></ul><ul><li>One Hand per bucket </li></ul><ul><li>God can now switch between Hands while they are “blocked” on Input </li></ul><ul><li>If God picks a hand whose bucket is full, God begins computation </li></ul><ul><li>Switching between hands is faster than switching between persons </li></ul>GOD
  18. 18. <ul><li>Rule 4 – God can switch between hands, faster than switching between persons </li></ul>Rule 4 – The CPU can switch between threads, faster than switching between processes
  19. 19. <ul><li>Threads take up lesser memory -> lesser context switching time -> more efficient CPU utilization </li></ul>Threads vs Processes
  20. 20. Is there a scenario where I would want to use processes instead of threads anyways?
  21. 21. Scenario 5 – Async IO <ul><li>Person => Process </li></ul><ul><li>Hands => Threads </li></ul><ul><li>God => CPU </li></ul><ul><li>Bucket => Input </li></ul><ul><li>The bucket notifies God that I am done </li></ul><ul><li>Number of hands required = 1 </li></ul>GOD
  22. 22. Scenario 5 – Async Blocking IO <ul><li>epoll() and Kqueue() </li></ul>
  23. 23. Advantages of Async Non-blocking IO <ul><li>Removes requirement of threads -> eliminates context switching </li></ul>
  24. 24. Is there a scenario where I would want multiple threads even if I use Async I/O ??
  25. 25. Scenario 6 – More than 1 GOD <ul><li>Each God can only do one thing at a time </li></ul><ul><li>With Async IO, if I have two Gods, I should have two hands </li></ul><ul><li>This applies to CPUs and CPU Cores </li></ul><ul><ul><li>Eg Dual Core Dual CPUs => 4 threads </li></ul></ul>GOD GOD
  26. 26. Async IO Success Story <ul><li>Tomcat 6.0 – 16000 simultaneous connections </li></ul><ul><li>Apache MINA + Async Web </li></ul>
  27. 27. http://www.directi.com | http://careers.directi.com Questions? Email me: [email_address] Follow me: http://twitter.com/bhavintu Facebook: http://www.facebook.com/bhavin.t Blog: http://bhavin.directi.com Website: http://directi.com Download Slides: http://wiki.directi.com

×