http://rust-class.org Smarter scheduling strategies Priority Inversion Lottery Scheduling and Stride Scheduling History os the Linux scheduler

2. Plan for Today
• Course Schedule Update
• Norvig Numbers
• Priority Pre-Emptive Scheduling
• Lottery Scheduling
• Stride Scheduling
• What Linux Does (“Completely Fair
Scheduler”)
3 October 2013 University of Virginia cs4414 1

3. Course Schedule
Original (Syllabus)
PS1: Simple server (10 Sept)
PS2: Shell (24 Sept)
PS3: Web server (10 Oct)
Midterm (10 Oct)
PS4: IceBox (5 Nov)
Project: Open (5 Dec)
Revised
PS1: zhttpto (10 Sept)
PS2: gash (30 Sept + demos)
3 October 2013 University of Virginia cs4414 2
Today is 3 Oct and PS3 is not yet out!

4. Course Schedule
Original (Syllabus)
PS1: Simple server (10 Sept)
PS2: Shell (24 Sept)
PS3: Web server (10 Oct)
Midterm (10 Oct)
PS4: IceBox (5 Nov)
Project: Open (5 Dec)
Revised
PS1: zhttpto (10 Sept)
PS2: gash (30 Sept + demos)
Midterm (due 14 Oct)
PS3: zhtta (due 28 Oct)
Project: open (due 5 Dec)
3 October 2013 University of Virginia cs4414 3
Note: if you already have a great idea for a project that will be more
worthwhile for you than PS3, you can make a case for doing that
instead of PS3 also.

5. Oh no…what can we possibly do
with no Problem Set to work on?
3 October 2013 University of Virginia cs4414 4

6. 3 October 2013 University of Virginia cs4414 5
From Class 3…

7. 3 October 2013 University of Virginia cs4414 6

8. Updating the “Norvig Numbers”
• We don’t just want the numbers, we want programs
that can produce the number.
• Combined efforts and talents of this class should be
able to make a really useful list (and learn a lot)
• Post in Piazza forum with details
– Everyone should contribute something
– Can work in teams of any size (1-60+, but larger teams
should make more interesting contributions)
– Claim and justify a number you will contribute (and then,
start working on the program to do it) – First Come, First
Serve
3 October 2013 University of Virginia cs4414 7

9. Example
Kernel Timer Interval (Wil Thomason has dibs
on this one!)
Why programmers should care about this
Program that estimates its value
Result for some machine today
3 October 2013 University of Virginia cs4414 8
Another example (also not on Norvig’s list):
Time to Switch Tasks
https://github.com/brson/rust-sched-bench/blob/master/coroutine-call.rs

10. Scheduling
3 October 2013 University of Virginia cs4414 9

11. Pre-emptive Priority Scheduling
• Always run the highest priority process that is
ready to run
• Round-robin schedule among equally high,
ready to run, highest-priority processes
3 October 2013 University of Virginia cs4414 10
P 629 P 124Priority 0:
P 528Priority 1:
P 44Priority 2: P 815 P 516
Waiting:
Memory Read P 131
Network Data P 221
Shared Bus P 1209

12. 3 October 2013 University of Virginia cs4414 11
Mars Curiosity (2012)

13. 3 October 2013 University of Virginia cs4414 12
Mars Pathfinder (1997)

14. Pathfinder OS:
Pre-emptive Priority Scheduling
• Always run the highest priority process that is
ready to run
• Round-robin schedule among equally high,
ready to run, highest-priority processes
3 October 2013 University of Virginia cs4414 13
CPU
SharedBus
Radio
Camera
Flash
MemoryActuators

15. Priority Inversion
3 October 2013 University of Virginia cs4414 14
Task 1 (scheduler) – highest priority (Priority = 1)
Task 2 (send data) – (Priority = 4)
Task 3 (science analysis) – lowest priority (Priority = 97)
CPU
SharedBus
Radio
Camera
Flash
MemoryActuators

16. How should we solve
priority inversion?
3 October 2013 University of Virginia cs4414 15

17. 3 October 2013 University of Virginia cs4414 16
Priority 0:
P 528Priority 1:
P 44Priority 2: P 815 P 516
Waiting:
Memory Read P 131
Network Data P 221
Shared Bus P 1209 PRI: 0
Holds Bus Lock

18. Priority
3 October 2013 University of Virginia cs4414 17
Should my MacBook use a priority pre-emptive scheduler
with priority inheritance?

19. Kinds of Processes
“Compute-Bound”
3 October 2013 University of Virginia cs4414 18
P1
“I/O-Bound”
P2 wait for disk… P2 wait for network… P2 wait for user…
“Real Time”
P3
need frame ^ need frame ^ need frame ^ need frame ^
P3 P3 P3

20. 3 October 2013 University of Virginia cs4414 19
Carl Waldspurger

21. Lottery Scheduling
3 October 2013 University of Virginia cs4414 20

22. Lottery Scheduling
• Each user (process) gets a share of the “tickets”
– e.g., 1000 total tickets, 20 processes each get 50
tickets (or more/less weighted by priority)
• User/process can distribute tickets however it
wants
– Among its own threads, can “loan” to other processes’
threads
• Scheduler: randomly picks a ticket
– Associated thread gets to run for that time slice
3 October 2013 University of Virginia cs4414 21
Silly aside: what does “A/B” mean in English?

23. Do not use “/” except to mean divide
3 October 2013 University of Virginia cs4414 22
“The University of Virginia is an
equal opportunity/affirmative action
employer.”
On 10/2/13 3:56 PM, [name removed] wrote:
> Mr. Evans,
> The forward slash is commonly used to represent "and" in this context.
“…the accommodation the employee and/or the employee’s
doctor/medical professional believe will enable the employee to
perform the essential functions of the position. The employee
should provide his/her doctor/medical professional with the
definition of a disability…”

24. Priority Pre-Emptive Lottery Scheduling
3 October 2013 University of Virginia cs4414 23

25. 3 October 2013 University of Virginia cs4414 24

26. 3 October 2013 University of Virginia cs4414 25

27. 3 October 2013 University of Virginia cs4414 26
What is the running time?
> uname -a
Linux power2 3.2.0-49-generic #75-Ubuntu SMP Tue Jun 18 17:39:32
UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
> sysctl kernel.pid_max
kernel.pid_max = 32768

28. Linux Scheduler before V2.6 (2002)
• Three types of processes:
#define SCHED_OTHER 0
#define SCHED_FIFO 1
#define SCHED_RR 2
• Not pre-emptive: only user-level processes
could be pre-empted
• Select next process according to “goodness”
function
3 October 2013 University of Virginia cs4414 27
Normal user processes
Non-prementable
Real-time round-robin

29. 3 October 2013 University of Virginia cs4414 28
/* linux/kernel/sched.c
* This is the function that decides how desirable a process is.
* You can weigh different processes against each other depending
* on what CPU they've run on lately etc to try to handle cache
* and TLB miss penalties.
*
* Return values:
* -1000: never select this
* 0: out of time, recalculate counters (but it might still
* be
* selected)
* +ve: "goodness" value (the larger, the better)
* +1000: realtime process, select this.
*/
static inline int goodness(struct task_struct * p, int this_cpu, struct
mm_struct *this_mm)
{
int weight;
/*
* Realtime process, select the first one on the
* runqueue (taking priorities within processes
* into account).
*/
if (p->policy != SCHED_OTHER) {
weight = 1000 + p->rt_priority;
goto out;
}
/*
* Give the process a first-approximation goodness value
* according to the number of clock-ticks it has left.
*
* Don't do any other calculations if the time slice is
* over..
*/
weight = p->counter;
if (!weight)
goto out;
#ifdef __SMP__
/* Give a largish advantage to the same processor... */
/* (this is equivalent to penalizing other processors) */
if (p->processor == this_cpu)
weight += PROC_CHANGE_PENALTY;
#endif
/* .. and a slight advantage to the current MM */
if (p->mm == this_mm)
weight += 1;
weight += p->priority;
out:
return weight;
}
/* linux/kernel/sched.c
* This is the function that decides how desirable a process is.
* You can weigh different processes against each other depending
* on what CPU they've run on lately etc to try to handle cache
* and TLB miss penalties.
*
* Return values:
* -1000: never select this
* 0: out of time, recalculate counters (but it might still
* be
* selected)
* +ve: "goodness" value (the larger, the better)
* +1000: realtime process, select this.
*/
static inline int goodness(struct task_struct * p, int this_cpu,
struct mm_struct *this_mm)
{
…

30. 3 October 2013 University of Virginia cs4414 29
static inline int goodness(struct task_struct * p, int this_cpu, struct mm_struct *this_mm)
{
int weight;
/* Realtime process, select the first one on the runqueue (taking priorities into account). */
if (p->policy != SCHED_OTHER) {
weight = 1000 + p->rt_priority;
goto out;
}
/* Give the process a first-approximation goodness value according to the number of clock-
ticks it has left. Don't do any other calculations if the time slice is over.. */
weight = p->counter;
if (!weight)
goto out;
#ifdef __SMP__
/* Give a largish advantage to the same processor... (equivalent to penalizing other
processors) */
if (p->processor == this_cpu)
weight += PROC_CHANGE_PENALTY;
#endif
/* .. and a slight advantage to the current MM (memory segment) */
if (p->mm == this_mm) weight += 1;
weight += p->priority;
out:
return weight;
}

31. What is the running time of the
Linux 2.2-2.5 Scheduler?
3 October 2013 University of Virginia cs4414 30

32. 3 October 2013 University of Virginia cs4414 31
It was called the “O(n) scheduler”

33. Linux 2.6 Scheduler (2003-2007)
• 140 different queues (for each processor)
– 0-99 for “real time” processes
– 100-139 for “normal” processes
• Bit vector keeps track of which queues have
ready to run process
• Scheduler picks first process from highest
priority queue with a ready process
– Give it time quantum that scales with priority
3 October 2013 University of Virginia cs4414 32

34. 3 October 2013 University of Virginia cs4414 33
struct runqueue {
struct prioarray *active;
struct prioarray *expired;
struct prioarray arrays[2];
};
struct prioarray
{
int nr_active; /* # Runnable */
unsigned long bitmap[5];
struct list_head queue[140];
};

35. What is the running time of the
Linux 2.6 Scheduler?
3 October 2013 University of Virginia cs4414 34

36. 3 October 2013 University of Virginia cs4414 35
(Sadly, O(1) scheduler has no Facebook page.)

37. Linux V2.6.23+ Scheduler
3 October 2013 University of Virginia cs4414 36

38. 3 October 2013 University of Virginia cs4414 37
This is exactly stride scheduling (but with different terminology)!
Rotating Staircase Deadline Scheduler

39. 3 October 2013 University of Virginia cs4414 38
What is the running time of the
Linux 2.6.23+ Scheduler?
Not called the O(log N) scheduler – by Linux 2.6.23 marketing
matters: “Completely Fair Scheduler”

40. 3 October 2013 University of Virginia cs4414 39
What is log2 1 000 000?

41. Charge
• Claim your “Norvig Number” early!
– No penalty if you decide to switch later
• Stride scheduling works!
– Use it to manage your real life: much smarter than
priority pre-emptive (never finish anything) or
first-come first-served or earliest-deadline-first
– Unless you like to live serendipitously: then you
should use lottery scheduling
3 October 2013 University of Virginia cs4414 40