T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna
An Evaluation of Adaptive Partitioning
An Evaluation of Adaptive Partitioning
of Real-Time Workloads on Linux
of Real-Time Workloads on Linux
IEEE ISORC 2021
IEEE ISORC 2021
A. Stevanato, T. Cucinotta, L. Abeni
Real-Time Systems Laboratory
Scuola Superiore Sant’Anna
Pisa, Italy
D. B. de Oliveira
Red Hat, Inc.
Pisa, Italy

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 2
Background and
Background and
motivations #1/2
motivations #1/2
Relentless growth of computing demand
●
satisfied for years by the “frequency-race”
●
up to the impossibility of handling heat dissipation
– Intel’s cancelled plans for 5GHz-10GHz CPUs in 2004
●
=> we’ve been living a “multi-core race” in the last decade
Multi-core/multi-processing pervasive
●
not only in High-Performance Computing (HPC)
●
but also in embedded and real-time systems
●
noteworthy modern use-cases:
high-performance embedded scenarios
– intelligent real-time driving assistants
– real-time video processing and object recognition
– real-time time-series forecasting
●
Need for a multitude of “acceleration” technologies:
– Multi-core, GP-GPU, FPGA

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 3
Background and
Background and
motivations #2/2
motivations #2/2
●
Open real-time systems
– dynamic real-time task sets on multi-core platforms
●
new RT tasks may ask to be admitted at any time
●
running RT tasks may terminate and leave the system
– requirement of minimizing energy consumption
(mobile devices)
●
Frequent migration of RT tasks among cores
– when using utilization-based heuristics for task/core
assignment

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 4
Focus of the paper
●
Adaptively Partitioned EDF (apEDF) on multi-cores
●
Originally appeared in
– “Adaptive Partitioning of Real-Time Tasks on Multiple Processors,” ACM SAC 2020
– “EDF scheduling of real-time tasks on multiple cores: adaptive partitioning vs. global
scheduling,” ACM SIGAPP Applied Computing Review, 2020
●
Contributions of this paper
– Prototype implementation in the Linux kernel (SCHED_DEADLINE)
– Experimental evaluation
apEDF
●
RT tasks scheduled using CBS/EDF reservations
– each task with parameters (WCET Ci, min inter-arrival period Ti)
has associated a dedicated reservation with (Qi, Pi)
●
normally Qi = Ci and Pi = Ti
– at any time tasks (and their reservations) are partitioned among cores
●
reservations {(Qi, Pi)}i deployed onto each core respect
●
CBS handles a current budget qi(t) and abs deadline di(t)C
Proposed Approach #1/2
Proposed Approach #1/2

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 5
Proposed Approach #2/2
Proposed Approach #2/2
●
apEDF
– utilization-based heuristics for task/core assignment
– First-Fit
●
good: simple utilization-based overall test
●
bad: for energy-management
●
on DVFS frequencies scale up/down
=> would need to migrate tasks to restore FF invariant
– Worst-Fit
●
good: load spread across cores
– allows for keeping lower DVFS frequencies
●
bad: if a new big U arrives
=> would need to migrate tasks to make room

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 6
Adaptively Partitioned EDF
Adaptively Partitioned EDF
●
apEDF
– placement logic called on
●
real-time task arrival
●
task wake-up
– placement algorithm for new task τi = (Ci, Pi)
1.if task on non-overloaded core => keep it where it is
2.find a core j that does not overload once accepting the task
– Uj + Ci/Pi <= Ulub
– using first-fit (FF)
3.if no such core exists, then fallback to global EDF
●
apEDF general properties
– no more migrations once a schedulable partitioning is found
– if a schedulable partitioning does not exist, or cannot be found by FF,
then fallback to global EDF (rule 3.)
– apEDF is not work-conserving
(cores may be idle with tasks ready to run)

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 7
Adaptively Partitioned EDF
Adaptively Partitioned EDF
(variant with adaptive pull)
(variant with adaptive pull)
●
a2
pEDF
– when a core goes idle, pull the non-executing task from
overloaded cores with the most urgent deadline
●
a task is pulled only if its core j has Uj > Ulub
– a2
pEDF general properties
●
recovers a minimum of work-conserving property
(of all Linux schedulers when not under rt-throttling)
– but it may still leave cores idle with tasks ready to run
●
no more migrations once a schedulable partitioning is found

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 8
apEDF
apEDF
Schedulability Analysis
Schedulability Analysis
●
apEDF schedulability can be analyzed applying
– Lopez et al., “Worst-case utilization bound for EDF scheduling on
real-time multiprocessor systems,” ECRTS 2000
●
given m tasks to be placed on n identical CPUs
●
worst-case of
●
However, the FF invariant is not guaranteed (e.g., task
death or change of scheduling policy)

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 9
●
Original SCHED_DEADLINE (kernel 5.7.0)
Implementation #1/2
Implementation #1/2
sched/core.c
try_to_wakeup()
wake_up_new_task()
sched_exec()
sched/deadline.c
select_task_rq()
__schedule()
schedule_tail()
__sched_setscheduler()
select_task_rq_dl()
balance_callback()
push_dl_tasks() push_dl_task()
activate_task()
enqueue_task_dl()
enqueue_task()
dequeue_task()
deactivate_task()
dequeue_task_dl()
select_task_rq_dl()
cpudl_find()
pick_next_task() put_prev_balance() balance_dl() pull_dl_task()
find_later_rq()

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 10
Implementation #2/2
Implementation #2/2
●
SCHED_DEADLINE+apEDF (kernel 5.7.0)
sched/deadline.c
push_dl_task()
find_xf_suitable_rq() first_fit_cpu_find()
worst_fit_cpu_find()
find_later_rq() cpudl_find()
select_task_rq_dl()
push_dl_task_gedf()
push_dl_task_xf()
Scheduler sysctl/proc tunables
/proc/sys/kernel/sched_dl_policy
/proc/sys/kernel/sched_dl_xf_pull
/proc/sys/kernel/sched_dl_fallback_to_gedf
pull_dl_task() pick_earliest_pushable_dl_task()
pull_dl_task_gedf()
pull_dl_task_ff()
pull_dl_task_wf()

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 11
Experimental Evaluation
Experimental Evaluation
vs Simulations (SAC’20)
vs Simulations (SAC’20)
Simulations from
SAC’20
Experiments on Linux
(patched rt-app)
Note: total utilization
beyond maximum
theoretical value
(5/2=2.5)

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 12
Experimental Evaluation
Experimental Evaluation
vs Simulations (SAC’20)
vs Simulations (SAC’20)

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 13
Conclusions & Future Work
Conclusions & Future Work
●
Proposed Linux implementation of apEDF/a2
pEDF
●
Presented experimental results
– 4 cores (up to 3.8 U and 15 tasks)
– 8 core s (up to 7.4 U and 25 tasks)
●
Obtained results
– deadline misses not too different from global EDF
– deadline misses better with a2
pEDF at high utilizations
– migrations tremendously better than global EDF
●
Future Work
– improve patch-set
– experiment with data-intensive workload
– integrate with Energy Aware Scheduling (EAS)
– deeper comparison between FF and WF with DVFS
– evaluate effectiveness on ARM big.LITTLE / DynamIQ

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 14
Our related publications
Our related publications
Journals
●
A. Mascitti, T. Cucinotta, M. Marinoni, L. Abeni. "Dynamic Partitioned Scheduling of Real-Time Tasks on ARM big.LITTLE
Architectures," Elsevier Journal of Systems and Software (JSS), Vol. 173, March 2021
●
D. B. De Oliveira, R. S. De Oliveira, T. Cucinotta. "A thread synchronization model for the PREEMPT_RT Linux kernel,"
Elsevier Journal of Systems Architecture (JSA), Vol. 107, August 2020
●
L. Abeni, T. Cucinotta. "EDF scheduling of real-time tasks on multiple cores: adaptive partitioning vs. global scheduling,"
ACM SIGAPP Applied Computing Review, Vol. 20, Issue 2 (June 2020), pp. 5-18, July 2020
●
A. Balsini, T. Cucinotta, L. Abeni, J. Fernandes, P. Burk, P. Bellasi, M. Rasmussen. "Energy-Efficient Low-latency Audio on
Android," Elsevier Journal of Systems and Software (JSS), Vol. 152, pp. 182-195, June 2019
●
Improving Responsiveness of Time-Sensitive Applications by Exploiting Dynamic Task Dependencies, Software: Practice
and Experience, April 2018
Conferences
●
T. Cucinotta, L. Abeni. "Migrating Constant Bandwidth Servers on Multi-Cores," 29th International Conference on Real-Time
Networks and Systems (RTNS 2021), April 7-9, 2021, Nantes, France
●
A. Mascitti, T. Cucinotta. "Dynamic Partitioned Scheduling of Real-Time DAG Tasks on ARM big.LITTLE Architectures,"
29th International Conference on Real-Time Networks and Systems (RTNS 2021), April 7-9, 2021, Nantes, France.
●
D. B. de Oliveira, D. Casini, R. S. de Oliveira, T. Cucinotta. "Demystifying the Real-Time Linux Scheduling Latency,"
32nd Euromicro Conference on Real-Time Systems (ECRTS 2020), July 7-10, 2020, Modena, Italy.
●
E. Quiñones, S. Royuela, C. Scordino, L. M. Pinho, T. Cucinotta, B. Forsberg, A. Hamann, D. Ziegenbein, P. Gai, A. Biondi, L.
Benini, J. Rollo, H. Saoud, R. Soulat, G. Mando, L. Rucher, L. Nogueira. "The AMPERE Project: A Model-driven development
framework for highly Parallel and EneRgy-Efficient computation supporting multi-criteria optimization,"
23rd IEEE International Symposium on Real-Time Distributed Computing (IEEE ISORC 2020), May 19-21, 2020, Nashville,
Tennessee, USA.
●
A. Mascitti, T. Cucinotta, L. Abeni. "Heuristic partitioning of real-time tasks on multi-processors," 23rd IEEE International
Symposium on Real-Time Distributed Computing (IEEE ISORC 2020), May 19-21, 2020, Nashville, Tennessee, USA.
●
G. Serra, G. Ara, P. Fara, T. Cucinotta. "An Architecture for Declarative Real-Time Scheduling on Linux," 23rd IEEE
International Symposium on Real-Time Distributed Computing (IEEE ISORC 2020), May 19-21, 2020, Nashville, Tennessee, USA.

T. Cucinotta – Real-Time Systems Laboratory (ReTiS) – Scuola Superiore Sant’Anna 15
Questions ?
tommaso.cucinotta@santannapisa.it
http://retis.santannapisa.it/~tommaso
Thanks!
Thanks!