The document discusses the Linux kernel's interrupt handling, specifically the concept of bottom halves, which include softirqs and tasklets, and explores their evolution to workqueues for better concurrency management. It highlights the importance of low latency in interrupt processing and the necessity for non-blocking behavior, elaborating on the implications of threaded interrupts and scheduling models. Additionally, it presents potential issues such as deadlocks and performance considerations, along with references for further reading on these topics.

1. BottomBottom
HalvesHalves
ON LINUXON LINUX
 Interrupts / Hard-IRQs
 Why we need them?
 Bottom Halves - The concept
 Bottom Halves - The original implementation
 Bottom Halves 2.0 - SoftIRQs and Tasklets
 Bottom Halves today – Workqueues
 Concurrency Managed Workqueues
 Preempt-RT goodies
 Better control over prioritisation = Predictable execution.
 IRQ_FORCED_THREADING - Threaded Hard-IRQs
 PREEMPT_SOFTIRQS - Threaded Soft-IRQs.
 Few open Qs
 References / Recommended Reading​ ​ ​ ​
45
Mins
left
This work is licensed under a Creative Commons Attribution 4.0 International License.
by CVS
JUNE 2018

2. Imagine a world where…
 each time a single character is received on UART
 >8000 lines of code are executed
 before anything else can run
45
Mins
left

3. IRQ vs. Entire Execution Context
46 lines
vs.
>8K lines
175 times more!
44
Mins
left

4. Interrupts can’t sleep
Why?
Because no process / thread context *
42
Mins
left

5. Why?
Because no process / thread context*
Why?
Because designed to be instantaneous
Interrupts can’t sleep
41
Mins
left

6. Interrupts can’t sleep
Why?
Because no process / thread context*
Why?
Because designed to be instantaneous
Why?
Because low latency required at first level of triage
41
Mins
left

7. Interrupts can’t sleep
Why?
Because no process / thread context*
Why?
Because designed to be instantaneous
Why?
Because low latency required at first level of triage
Why?
Because Bottom halves exist to process the larger time consuming logic.
41
Mins
left

8. Interrupts can’t sleep
Why?
Because no process / thread context*
Why?
Because designed to be instantaneous
Why?
Because low latency required at first level of triage
Why?
Because Bottom halves exist to process larger time consuming logic.
Why?
I wrote the Linux kernel. That’s why
Note: Photo captured in a different context.
41
Mins
left

9. Interrupts can’t sleep
Why?
Because no process / thread context*
Why?
Because designed to be instantaneous
Why?
Because low latency required at first level of triage
Why?
Because Bottom halves exist to process larger time consuming logic.
Decision Logs = Important part of
Requirements / Design Documentation
41
Mins
left

10. What happens if an interrupt sleeps?
 msleep()
`--> schedule()
`--> __schedule()
`--> schedule_debug()
40
Mins
left

11. 2018 : What are Bottom Halves?
 Bottom Halves (deprecated)
 SoftIRQs
 Tasklets
 Workqueues (deprecated)
 Concurrency Managed Workqueues
 Threaded interrupts
38
Mins
left

12. What are Bottom Halves?
were 37
Mins
left

13. SoftIRQs and Tasklets
 Softirqs and Tasklets were introduced
during the Linux kernel 2.3 development series
to completely replace the BH interface.
 Softirqs are a set of statically defined bottom halves
that can run simultaneously on any processor;
even two of the same type can run concurrently.
 Tasklets are statically/dynamically created
bottom halves built on top of softirqs.
 Two different tasklets
can run concurrently on different processors.
 But two of the same type of tasklet cannot run simultaneously.
This is
the only slide
in this slide-deck
without a picture
Promise!
35
Mins
left

14. Default SoftIRQ scheduling on Linux
 Variable priority of softirq
 Very high priority
 when softirq context lightly loaded
 preempt everything except hard-irq
 Very low priority
 when running in ksoftirqd kernel thread
 The goal is to provide low latency to softirq tasks
without starving other tasks
 The downside is that there is no consistent priority model!!
30
Mins
left

15. A note about preempt-rt patch-set
 Effective Latency =
 Interrupt-latency
+
 Scheduling-latency
 Preempt-RT
aims to reduce Scheduling-latency
25
Mins
left

16. Deterministic softirq scheduling on Linux
 A lesser known feature in RT_PREEMPT patch-set
 Called “preempt softirq” in Kconfig
 Running softirqs always in thread context.
 With this change, traditional softirq no longer exists
 softirq tasks always run in ksoftirqd
24
Mins
left

17. Tasklets
Gone since v4.1
Tasklets.
Not going anywhere,
anytime soon.
19
Mins
left

18. Workqueues
16
Mins
left

19. Why CMWQ? – Scaling with CPUs++
 16 traditional multi-threaded WQs
 on a 64 CPU core system
 =1024 threads!
 Cross 32K PID space on modern servers
 Avoid this!!
13
Mins
left

20. Why CMWQ? – Avoid deadlocks
 WQ_MEM_RECLAIM
 Potential for a deadlock
exists between:
 creation of a new
worker thread
and
 work items waiting in
the workqueue.
 Avoid this!!
11
Mins
left

21. Threaded Interrupts
 Not all interrupts are triggers for the real-time tasks.
 But all interrupts steal cycles from the real-time task.
 Threaded interrupt handlers:
 allow an independent priority to be associated with the interrupt.
 and for IRQ handler to be scheduled at an appropriate time.
 request_threaded_irq() instead of the usual request_irq().
 You can make threaded IRQs the default by configuring the
kernel with CONFIG_IRQ_FORCED_THREADING=Y which makes all
handlers into threads unless they have explicitly prevented
this by setting the IRQF_NO_THREAD flag.
 When the PREEMPT_RT patches are applied,
interrupts are configured as threads this way by default.
09
Mins
left

22. Bottom Halves on Linux
05
Mins
left
…for now 

23. Question(s)
 What is the IRQ Stack on Linux kernel?
 arm64: Introduce IRQ stack - lwn.net/Articles/657969
 stackoverflow.com/a/22500017/319204
 stackoverflow.com/a/28767868/319204
 What are the various pre-defined SoftIRQs used for?
 How often they are called?
 How much CPU do they use?
 How to profile top and bottom-halves on our system?
 What is the default CPU usage on an “idle” system?
05
Mins
left

24. REFERENCES / RECOMMENDED READING
 IRQF_DISABLED - lwn.net/Articles/380931
 Top and Bottom Halves - static.lwn.net/images/pdf/LDD3/ch10.pdf
 RT_PREEMPT to solve real-world problems (SoftIRQs) - elinux.org/images/7/72/Elc2011_xi_rt.pdf
 CMWQ - kernel.org/doc/html/latest/core-api/workqueue.html
 CMWQ - events.static.linuxfound.org/sites/events/files/slides/Async%20execution%20with%20wqs.pdf
 SoftIRQs vs. Tasklets - stackoverflow.com/a/31706431/319204
 Eliminating Tasklets - lwn.net/Articles/239633
 IRQs - events.static.linuxfound.org/sites/events/files/slides/Chaiken_ELCE2016.pdf
 Moving interrupts to threads - lwn.net/Articles/302043
120
Mins