This paper provides an open implementation and an experimental evaluation of an adaptive partitioning approach for scheduling real-time tasks on symmetric multicore systems. The proposed technique is based on combining partitioned EDF scheduling with an adaptive migration policy that moves tasks across processors only when strictly needed to respect their temporal constraints. The implementation of the technique within the Linux kernel, via modifications to the SCHED DEADLINE code base, is presented. An extensive experimentation has been conducted by applying the technique on a real multi-core platform with several randomly generated synthetic task sets. The obtained experimental results highlight that the approach exhibits a promising performance to schedule real-time workloads on a real system, with a greatly reduced number of migrations compared to the original global EDF available in SCHED DEADLINE. More information about the paper is available at: http://retis.sssup.it/~tommaso/papers/isorc21.php

1. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna
An Evaluation of Adaptive Partitioning
An Evaluation of Adaptive Partitioning
of Real-Time Workloads on Linux
of Real-Time Workloads on Linux
IEEE ISORC 2021
IEEE ISORC 2021
A. Stevanato, T. Cucinotta, L. Abeni
Real-Time Systems Laboratory
Scuola Superiore Sant’Anna
Pisa, Italy
D. B. de Oliveira
Red Hat, Inc.
Pisa, Italy

2. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 2
Background and
Background and
motivations #1/2
motivations #1/2
Relentless growth of computing demand
●
satisfied for years by the “frequency-race”
●
up to the impossibility of handling heat dissipation
– Intel’s cancelled plans for 5GHz-10GHz CPUs in 2004
●
=> we’ve been living a “multi-core race” in the last decade
Multi-core/multi-processing pervasive
●
not only in High-Performance Computing (HPC)
●
but also in embedded and real-time systems
●
noteworthy modern use-cases:
high-performance embedded scenarios
– intelligent real-time driving assistants
– real-time video processing and object recognition
– real-time time-series forecasting
●
Need for a multitude of “acceleration” technologies:
– Multi-core, GP-GPU, FPGA

3. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 3
Background and
Background and
motivations #2/2
motivations #2/2
●
Open real-time systems
– dynamic real-time task sets on multi-core platforms
●
new RT tasks may ask to be admitted at any time
●
running RT tasks may terminate and leave the system
– requirement of minimizing energy consumption
(mobile devices)
●
Frequent migration of RT tasks among cores
– when using utilization-based heuristics for task/core
assignment

4. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 4
Focus of the paper
●
Adaptively Partitioned EDF (apEDF) on multi-cores
●
Originally appeared in
– “Adaptive Partitioning of Real-Time Tasks on Multiple Processors,” ACM SAC 2020
– “EDF scheduling of real-time tasks on multiple cores: adaptive partitioning vs. global
scheduling,” ACM SIGAPP Applied Computing Review, 2020
●
Contributions of this paper
– Prototype implementation in the Linux kernel (SCHED_DEADLINE)
– Experimental evaluation
apEDF
●
RT tasks scheduled using CBS/EDF reservations
– each task with parameters (WCET Ci, min inter-arrival period Ti)
has associated a dedicated reservation with (Qi, Pi)
●
normally Qi = Ci and Pi = Ti
– at any time tasks (and their reservations) are partitioned among cores
●
reservations {(Qi, Pi)}i deployed onto each core respect
●
CBS handles a current budget qi(t) and abs deadline di(t)C
Proposed Approach #1/2
Proposed Approach #1/2

5. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 5
Proposed Approach #2/2
Proposed Approach #2/2
●
apEDF
– utilization-based heuristics for task/core assignment
– First-Fit
●
good: simple utilization-based overall test
●
bad: for energy-management
●
on DVFS frequencies scale up/down
=> would need to migrate tasks to restore FF invariant
– Worst-Fit
●
good: load spread across cores
– allows for keeping lower DVFS frequencies
●
bad: if a new big U arrives
=> would need to migrate tasks to make room

6. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 6
Adaptively Partitioned EDF
Adaptively Partitioned EDF
●
apEDF
– placement logic called on
●
real-time task arrival
●
task wake-up
– placement algorithm for new task τi = (Ci, Pi)
1.if task on non-overloaded core => keep it where it is
2.find a core j that does not overload once accepting the task
– Uj + Ci/Pi <= Ulub
– using first-fit (FF)
3.if no such core exists, then fallback to global EDF
●
apEDF general properties
– no more migrations once a schedulable partitioning is found
– if a schedulable partitioning does not exist, or cannot be found by FF,
then fallback to global EDF (rule 3.)
– apEDF is not work-conserving
(cores may be idle with tasks ready to run)

7. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 7
Adaptively Partitioned EDF
Adaptively Partitioned EDF
(variant with adaptive pull)
(variant with adaptive pull)
●
a2
pEDF
– when a core goes idle, pull the non-executing task from
overloaded cores with the most urgent deadline
●
a task is pulled only if its core j has Uj > Ulub
– a2
pEDF general properties
●
recovers a minimum of work-conserving property
(of all Linux schedulers when not under rt-throttling)
– but it may still leave cores idle with tasks ready to run
●
no more migrations once a schedulable partitioning is found

8. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 8
apEDF
apEDF
Schedulability Analysis
Schedulability Analysis
●
apEDF schedulability can be analyzed applying
– Lopez et al., “Worst-case utilization bound for EDF scheduling on
real-time multiprocessor systems,” ECRTS 2000
●
given m tasks to be placed on n identical CPUs
●
worst-case of
●
However, the FF invariant is not guaranteed (e.g., task
death or change of scheduling policy)

9. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 9
●
Original SCHED_DEADLINE (kernel 5.7.0)
Implementation #1/2
Implementation #1/2
sched/core.c
try_to_wakeup()
wake_up_new_task()
sched_exec()
sched/deadline.c
select_task_rq()
__schedule()
schedule_tail()
__sched_setscheduler()
select_task_rq_dl()
balance_callback()
push_dl_tasks() push_dl_task()
activate_task()
enqueue_task_dl()
enqueue_task()
dequeue_task()
deactivate_task()
dequeue_task_dl()
select_task_rq_dl()
cpudl_find()
pick_next_task() put_prev_balance() balance_dl() pull_dl_task()
find_later_rq()

10. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 10
Implementation #2/2
Implementation #2/2
●
SCHED_DEADLINE+apEDF (kernel 5.7.0)
sched/deadline.c
push_dl_task()
find_xf_suitable_rq() first_fit_cpu_find()
worst_fit_cpu_find()
find_later_rq() cpudl_find()
select_task_rq_dl()
push_dl_task_gedf()
push_dl_task_xf()
Scheduler sysctl/proc tunables
/proc/sys/kernel/sched_dl_policy
/proc/sys/kernel/sched_dl_xf_pull
/proc/sys/kernel/sched_dl_fallback_to_gedf
pull_dl_task() pick_earliest_pushable_dl_task()
pull_dl_task_gedf()
pull_dl_task_ff()
pull_dl_task_wf()

11. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 11
Experimental Evaluation
Experimental Evaluation
vs Simulations (SAC’20)
vs Simulations (SAC’20)
Simulations from
SAC’20
Experiments on Linux
(patched rt-app)
Note: total utilization
beyond maximum
theoretical value
(5/2=2.5)

12. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 12
Experimental Evaluation
Experimental Evaluation
vs Simulations (SAC’20)
vs Simulations (SAC’20)

13. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 13
Conclusions & Future Work
Conclusions & Future Work
●
Proposed Linux implementation of apEDF/a2
pEDF
●
Presented experimental results
– 4 cores (up to 3.8 U and 15 tasks)
– 8 core s (up to 7.4 U and 25 tasks)
●
Obtained results
– deadline misses not too different from global EDF
– deadline misses better with a2
pEDF at high utilizations
– migrations tremendously better than global EDF
●
Future Work
– improve patch-set
– experiment with data-intensive workload
– integrate with Energy Aware Scheduling (EAS)
– deeper comparison between FF and WF with DVFS
– evaluate effectiveness on ARM big.LITTLE / DynamIQ

14. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 14
Our related publications
Our related publications
Journals
●
A. Mascitti, T. Cucinotta, M. Marinoni, L. Abeni. "Dynamic Partitioned Scheduling of Real-Time Tasks on ARM big.LITTLE
Architectures," Elsevier Journal of Systems and Software (JSS), Vol. 173, March 2021
●
D. B. De Oliveira, R. S. De Oliveira, T. Cucinotta. "A thread synchronization model for the PREEMPT_RT Linux kernel,"
Elsevier Journal of Systems Architecture (JSA), Vol. 107, August 2020
●
L. Abeni, T. Cucinotta. "EDF scheduling of real-time tasks on multiple cores: adaptive partitioning vs. global scheduling,"
ACM SIGAPP Applied Computing Review, Vol. 20, Issue 2 (June 2020), pp. 5-18, July 2020
●
A. Balsini, T. Cucinotta, L. Abeni, J. Fernandes, P. Burk, P. Bellasi, M. Rasmussen. "Energy-Efficient Low-latency Audio on
Android," Elsevier Journal of Systems and Software (JSS), Vol. 152, pp. 182-195, June 2019
●
Improving Responsiveness of Time-Sensitive Applications by Exploiting Dynamic Task Dependencies, Software: Practice
and Experience, April 2018
Conferences
●
T. Cucinotta, L. Abeni. "Migrating Constant Bandwidth Servers on Multi-Cores," 29th International Conference on Real-Time
Networks and Systems (RTNS 2021), April 7-9, 2021, Nantes, France
●
A. Mascitti, T. Cucinotta. "Dynamic Partitioned Scheduling of Real-Time DAG Tasks on ARM big.LITTLE Architectures,"
29th International Conference on Real-Time Networks and Systems (RTNS 2021), April 7-9, 2021, Nantes, France.
●
D. B. de Oliveira, D. Casini, R. S. de Oliveira, T. Cucinotta. "Demystifying the Real-Time Linux Scheduling Latency,"
32nd Euromicro Conference on Real-Time Systems (ECRTS 2020), July 7-10, 2020, Modena, Italy.
●
E. Quiñones, S. Royuela, C. Scordino, L. M. Pinho, T. Cucinotta, B. Forsberg, A. Hamann, D. Ziegenbein, P. Gai, A. Biondi, L.
Benini, J. Rollo, H. Saoud, R. Soulat, G. Mando, L. Rucher, L. Nogueira. "The AMPERE Project: A Model-driven development
framework for highly Parallel and EneRgy-Efficient computation supporting multi-criteria optimization,"
23rd IEEE International Symposium on Real-Time Distributed Computing (IEEE ISORC 2020), May 19-21, 2020, Nashville,
Tennessee, USA.
●
A. Mascitti, T. Cucinotta, L. Abeni. "Heuristic partitioning of real-time tasks on multi-processors," 23rd IEEE International
Symposium on Real-Time Distributed Computing (IEEE ISORC 2020), May 19-21, 2020, Nashville, Tennessee, USA.
●
G. Serra, G. Ara, P. Fara, T. Cucinotta. "An Architecture for Declarative Real-Time Scheduling on Linux," 23rd IEEE
International Symposium on Real-Time Distributed Computing (IEEE ISORC 2020), May 19-21, 2020, Nashville, Tennessee, USA.

15. T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 15
Questions ?
tommaso.cucinotta@santannapisa.it
http://retis.santannapisa.it/~tommaso
Thanks!
Thanks!