Scheduler Activations - Effective Kernel Support for the User-Level Management of Parallelism

1. SCHEDULER ACTIVATIONS Effective Kernel Support for the User-Level Management of Parallelism Kasun Gajasinghe Nisansa de Silva University of Moratuwa Based on the paper “Scheduler Activations - Effective Kernel Support for the User-Level Management of Parallelism” by Thomas E. Anderson et.al.

2. Scheduler Activations <ul><ul><li>Introduction to Threads </li></ul></ul><ul><li> </li></ul><ul><ul><li>User-Level and Kernel-Level Threads </li></ul></ul><ul><ul><ul><li>Implementation </li></ul></ul></ul><ul><ul><ul><li>Pros and Cons </li></ul></ul></ul><ul><li> </li></ul><ul><ul><li>Effective Kernel Support for User-Level Management of Parallelism </li></ul></ul><ul><ul><ul><li>Scheduler Activations </li></ul></ul></ul><ul><ul><ul><ul><li>Design </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Implementation </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Performance </li></ul></ul></ul></ul><ul><li> </li></ul><ul><ul><li>Conclusion </li></ul></ul>

3. What is a Thread? <ul><ul><li>Smallest unit of processing that can be scheduled by the operating system </li></ul></ul><ul><li> </li></ul><ul><ul><li>Similar to a process </li></ul></ul><ul><li> </li></ul><ul><ul><li>Separate “thread of execution” </li></ul></ul><ul><li> </li></ul><ul><li> </li></ul><ul><ul><li>Each thread has a separate stack, program counter, and run state </li></ul></ul>

4. Why Threads? <ul><ul><li>Perform operations in parallel on the same data </li></ul></ul><ul><li> </li></ul><ul><ul><li>Avoid complex explicit scheduling by applications </li></ul></ul><ul><li> </li></ul><ul><ul><li>Often more efficient than separate processes </li></ul></ul><ul><li> </li></ul><ul><ul><li>Remain responsive for user inputs </li></ul></ul>

5. Implementing Threads <ul><li>Three basic strategies </li></ul><ul><ul><li>User-level: possible on most operating systems, even ancient ones </li></ul></ul><ul><li> </li></ul><ul><ul><li>Kernel-level: looks almost like a process (i.e., Linux, Solaris) </li></ul></ul><ul><li> </li></ul><ul><ul><li>Scheduler activations (Digital Tru Unix 64, NetBSD, some Mach) </li></ul></ul>

6. <ul><li> </li></ul> User Thread Kernel Thread

7. Thread System - User Level <ul><ul><li>Thread library’s code + data structures in user space </li></ul></ul><ul><li> </li></ul><ul><ul><li>Invocation of library function = local function call. Not system call </li></ul></ul><ul><li> </li></ul><ul><ul><li>Pros - </li></ul></ul><ul><ul><ul><li>Good performance </li></ul></ul></ul><ul><ul><ul><ul><li>Fast thread context switching </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Better Scaling </li></ul></ul></ul></ul><ul><ul><ul><li>Highly flexible </li></ul></ul></ul><ul><ul><ul><ul><li>Custom scheduling algorithms </li></ul></ul></ul></ul><ul><ul><li>Cons - </li></ul></ul><ul><ul><ul><li>Poor concurrency support </li></ul></ul></ul><ul><ul><ul><ul><li>blocking system calls starve sibling threads </li></ul></ul></ul></ul>

8. Thread System - System Level <ul><ul><li>Thread library’s code + data structures in kernel space </li></ul></ul><ul><li> </li></ul><ul><ul><li>Invocation of library function = system call </li></ul></ul><ul><li> </li></ul><ul><ul><li>Pros - </li></ul></ul><ul><ul><ul><li>Good concurrency support </li></ul></ul></ul><ul><ul><ul><ul><li>Blocking system calls do not starve sibling threads </li></ul></ul></ul></ul><ul><li> </li></ul><ul><ul><li>Cons - </li></ul></ul><ul><ul><ul><li>Poor performance </li></ul></ul></ul><ul><ul><ul><ul><li>Operations involve system calls </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Full context switch to perform thread switches </li></ul></ul></ul></ul><ul><ul><ul><li>Less Flexible </li></ul></ul></ul><ul><ul><ul><ul><li>Generic scheduling Algorithm </li></ul></ul></ul></ul><ul><li> </li></ul>

9. <ul><li>User Threads </li></ul><ul><ul><li>Excellent performance </li></ul></ul><ul><li> </li></ul><ul><ul><li>No system calls to perform thread operations </li></ul></ul><ul><li> </li></ul><ul><ul><li>More flexible =>Can use domain specific scheduling algorithm (‘customized’ thread library) </li></ul></ul><ul><li> </li></ul><ul><ul><li>Blocking system calls such as I/O problematic; Starvation of sibling threads </li></ul></ul><ul><li>Kernel Threads </li></ul><ul><ul><li>Bad performance </li></ul></ul><ul><li> </li></ul><ul><ul><li>System calls needed to perform thread operations </li></ul></ul><ul><li> </li></ul><ul><ul><li>Generic scheduling algorithm( scheduled by kernel) </li></ul></ul><ul><li> </li></ul><ul><ul><li>Good integration with system services – </li></ul></ul><ul><ul><ul><li>blocking calls do not prevent other user threads from being scheduled. </li></ul></ul></ul><ul><ul><ul><li>Less likelihood of starvation </li></ul></ul></ul>

10. <ul><li>Fig. Thread Operation Latencies (micro seconds) </li></ul><ul><li>Topaz - A highly tuned kernel. </li></ul><ul><li>FastThreads - A user-level thread library </li></ul><ul><li>Ultrix - BSD family unix-like OS </li></ul>

11. SCHEDULER ACTIVATIONS

12. Scheduler Activations <ul><ul><li>Get the best of both worlds — </li></ul></ul><ul><ul><ul><li>The efficiency and flexibility of user-level threads </li></ul></ul></ul><ul><ul><ul><li>The non-blocking ability of kernel threads (Poor integration with system services) </li></ul></ul></ul><ul><li> </li></ul><ul><ul><li>Relies on upcalls </li></ul></ul><ul><li> </li></ul><ul><ul><li>The term “scheduler activation” was selected because each vectored event causes the user-level thread system to reconsider its scheduling decision of which threads to run on which processors. </li></ul></ul>

13. Scheduler Activations <ul><li>What’s an Upcall? </li></ul><ul><ul><li>Normally, user programs call functions in the kernel. Which call as system calls </li></ul></ul><ul><li> </li></ul><ul><ul><li>Sometimes, though, the kernel calls a user-level process to report an event </li></ul></ul><ul><li> </li></ul><ul><ul><li>Sometimes considered unclean — violates usual layering </li></ul></ul>

14. Scheduler Activations <ul><ul><li>Thread creation and scheduling is done at user level </li></ul></ul><ul><li> </li></ul><ul><ul><li>When a system call from a thread blocks, the kernel does an upcall to the thread manager </li></ul></ul><ul><li> </li></ul><ul><ul><li>The thread manager marks that thread as blocked, and starts running another thread </li></ul></ul><ul><li> </li></ul><ul><ul><li>When a kernel interrupt occurs that’s relevant to the thread, another upcall is done to unblock it </li></ul></ul>

15. Scheduler Activations <ul><li> </li></ul>

16. Scheduler Activations <ul><li>It serves as a vessel, or execution context, for running user-level threads, in exactly the same way that a kernel thread does. </li></ul><ul><li>It notifies the user-level thread system of a kernel event. </li></ul><ul><li>It provides space in the kernel for saving the processor context of the activation’s current user-level thread, when the thread is stopped by the kernel (e.g., because the thread blocks in the kernel on I/0 or the kernel preempts its processor). </li></ul>

17. Scheduler Activations <ul><li>Two execution stacks </li></ul><ul><li>One mapped into the kernel </li></ul><ul><li>one mapped into the application address space. </li></ul>Kernel Application Address space

18. Scheduler Activations <ul><li>kernel stack is used whenever the user-level thread running in the scheduler activation’s context executes in the kernel (e.g. system call) </li></ul><ul><li>The kernel also maintains a control block for each activation (akin to a thread control block) to record the state of the scheduler activation when its thread blocks in the kernel or is preempted. </li></ul>Kernel Application Address space

19. Scheduler Activations <ul><li>The user-level thread scheduler runs on the activation’s user-level stack and maintains a record of which user-level thread is running in which scheduler activation. </li></ul><ul><li>Each user-level thread is allocated its own stack when it starts running </li></ul>Kernel Application Address space

20. Scheduler Activations - Programs Kernel Application 1 Address space

21. Scheduler Activations - Programs Kernel Application 1 Address space

22. Scheduler Activations - Upcalls Kernel Application 1 Address space

23. Scheduler Activations – thread stopping <ul><li>The crucial distinction between scheduler activations and kernel threads is that once an activation’s user-level thread is stopped by the kernel, the thread is never directly resumed by the kernel. </li></ul>

24. Scheduler Activations – thread stopping <ul><li>A new scheduler activation is created to notify the user-level thread system that the thread has been stopped. </li></ul><ul><li>The user-level thread system removes the state of the thread from the old activation, </li></ul><ul><li>Tells the kernel that the old activation can be re-used (Explained later) </li></ul><ul><li>Decides which thread to run on the processor. </li></ul>

25. Scheduler Activations – thread stopping <ul><li>The kernel is able to maintain the invariant that there are always exactly as many running scheduler activations (vessels for running user-level threads) as there are processors assigned to the address space. </li></ul>

26. Scheduler Activations -Blocking Kernel Application 1 Address space

28. Scheduler Activations -Blocking Kernel Application 1 Address space 

30. Scheduler Activations -Blocking Kernel Application 1 Address space 

31. Scheduler Activations -Blocking Kernel Application 1 Address space  P

32. Scheduler Activations -Multiprogramming Application 2 Address space Kernel Application 1 Address space

33. Scheduler Activations -Multiprogramming Application 2 Address space Kernel Application 1 Address space P

34. Scheduler Activations - Additional <ul><li>if threads have priorities, and there exists running thread with a lower priority than both the unblocked and the preempted thread. </li></ul><ul><li>Application concurrency model </li></ul>

35. Scheduler Activations – Upcall Points

36. Scheduler Activations – Address space to Kernel

37. Scheduler Activations - Enhancements <ul><li>The processor allocator can favour address spaces that use fewer processors and penalize those that use more. </li></ul><ul><li>If overall the system has fewer threads than processors, the idle processors should be left in the address spaces most likely to create work in the near future. </li></ul>

38. Scheduler Activations – Critical sections <ul><li>A user-level thread could be executing in a critical section at the instant when it is blocked or preempted. </li></ul><ul><ul><li>Poor performance </li></ul></ul><ul><ul><li>Deadlock </li></ul></ul><ul><li>Use “Recovery” instead of “Prevention” </li></ul>

39. Scheduler Activations –Critical section Kernel Application 1 Address space P User level context switch!

40. Implementation <ul><li>Topaz kernel thread management system </li></ul><ul><ul><li>Where Topaz formerly blocked, resumed, or preempted a thread, it now performs upcalls to allow the user level to take these actions </li></ul></ul><ul><ul><li>Do explicit allocation of processors to address spaces </li></ul></ul><ul><li>Fast threads </li></ul><ul><ul><li>Process upcalls </li></ul></ul><ul><ul><li>Resume interrupted critical sections </li></ul></ul><ul><ul><li>Provide Topaz with needed infomation </li></ul></ul>

41. Implementation <ul><li>Topaz kernel thread management system </li></ul><ul><ul><li>Add 1200 lines (on 4000) </li></ul></ul><ul><ul><li>Mostly processor allocation policy </li></ul></ul><ul><li>Fast threads </li></ul><ul><ul><li>Few hundred lines </li></ul></ul><ul><li>NB: design is “neutral” on the choice of policies for allocating processors to address spaces and for scheduling threads onto processors. </li></ul>

42. Implementation– Processor Allocation Policy <ul><li>Processors are divided evenly among address spaces </li></ul><ul><li>if some address spaces do not need all of the processors in their share, those processors are divided evenly among the remainder. </li></ul><ul><li>processors are time-sliced only if the number of available processors is not an integer multiple of the number of address spaces. </li></ul><ul><li>The kernel processor allocator only needs to know whether each address space could use more processors or has some processors that are idle. </li></ul>

43. Implementation– Thread Scheduling Policy <ul><li>kernel has no knowledge of an application’s concurrency model or scheduling policy, or of the data structures used to manage parallelism at the user level. </li></ul><ul><li>Each application is completely free to choose these as appropriate; they can be tuned to fit the application’s needs. </li></ul>

44. Performance Enhancements– Critical sections <ul><li>Have a duplicate code section of the critical section </li></ul><ul><li>At the end of the copy (Only!) put code to yield the processor back to the resumer. </li></ul>

45. Performance Enhancements– Critical sections <ul><li>Normal execution uses the original code. </li></ul><ul><li>When a preemption occurs, the kernel starts a new scheduler activation to notify the user-level thread system </li></ul><ul><li>This activation checks the preempted thread’s program counter to see if it was in one of these critical sections </li></ul><ul><li>If so, continue the thread at the corresponding place in the copy of the critical section. The copy relinquishes control back to the original upcall at the end of the critical section. </li></ul>

46. Performance Enhancements– UPCalls <ul><li>Logically, a new scheduler activation is created for each upcall. </li></ul><ul><li>Creating a new scheduler activation is not free, however, because it requires data structures to be allocated and initialized. </li></ul><ul><li>Instead, discarded scheduler activations can be cached for eventual re-use. (Remember?  ) </li></ul>

47. Performance Enhancements– Debugging <ul><li>Not very significant to note </li></ul><ul><li>The point is that you have to use a debugger which have as little effect as possible on the sequence of instructions being debugged. </li></ul>

48. Performance <ul><ul><li>Thread performance - without kernel involvement quite similar to FastThreads before the changes. </li></ul></ul><ul><li> </li></ul><ul><ul><li>Upcall performance - significantly worse than Topaz threads. </li></ul></ul><ul><ul><ul><li>Untuned implementation. </li></ul></ul></ul><ul><ul><ul><li>Topaz in assembler, this system in Modula-2+. </li></ul></ul></ul><ul><li> </li></ul><ul><ul><li>Application performance - </li></ul></ul><ul><ul><ul><li>Negligible I/O: As quick as original FastThreads. </li></ul></ul></ul><ul><ul><ul><li>With I/O: Performs better than either FastThreads or Topaz threads. </li></ul></ul></ul>

49. Performance <ul><li> </li></ul>

52. Summary <ul><ul><li>Processor allocation (the allocation of processors to address spaces) is done by the kernel. </li></ul></ul><ul><li> </li></ul><ul><ul><li>Thread scheduling (the assignment of an address space’ threads to its processors) is done by each address space. </li></ul></ul><ul><li> </li></ul><ul><ul><li>The kernel notifies the address space thread scheduler of every event affecting the address space. </li></ul></ul><ul><li> </li></ul><ul><ul><li>The address space notifies the kernel of the subset of user-level events that can affect processor allocation decisions. </li></ul></ul>

53. THANK YOU!