"Controlling a laser with Linux is crazy, but everyone in this room is crazy in his own way. So if you want to use Linux to control an industrial welding laser, I have no problem with your using PREEMPT_RT." -- Linus Torvalds

1. Missing points from last presentation
• Which modifications done
• Detailed configuration options
• Change realtime support run time or statically?
• Statically, you need to compile the realtime patched kernel with CONFIG_PREEMPT_RT_FULL=y to support realtime.
• Linux kernel API documents
• https://www.kernel.org/doc/htmldocs/kernel-api/
• Operating system design
• Scheduling
• Other approaches to Linux realtime
• Xenomai
• RTLinux
• RTAI
• extra: does other mobile operating systems provide realtime capability?
• android os
• firefox os
1
• will be covered later

2. Preempt_rt research
Emre Can Kucukoglu
eckucukoglu@gmail.com
28.08.14

3. When does real-time Linux come into
embedded development?
• Hard real time requirements
• missing a deadline is a total system failure.
• strict deadline
• to control something that will end up
killing people if something goes wrong.
• eg. nuclear systems, pacemakers, avionics
3

4. When does real-time Linux come into
embedded development?
• Soft real time requirements
• maximizing the number of deadlines met,
• minimizing the lateness of tasks and
• maximizing the number of high priority tasks meeting their deadlines.
• violation of constraints results in degraded quality, but the system can
continue to operate.
• eg. Live audio-video systems
4

5. When does real-time Linux come into
embedded development?
• Firm real time requirements
• infrequent deadline misses are tolerable
• usefulness of a result is zero after its deadline
• eg. forecast systems
5

6. Preempt_rt: Hard real time
• "Hard" real-time software
• for robotics, stock exchanges (imkb)...
• has been used on computers that have gone into space.
6

7. Preempt_rt: Hard realtime
• faster response times
• removes all unbounded latencies
• linux kernel is filled with unbounded latencies
• non-deterministic behavior
• eg. unfair reader writer locks
• readers can continually take the lock
• bounded latency: eg. fair reader writer lock
• new readers will block if there's a writer waiting
7

8. Advantage over other rt linux
implementations
• preempt_rt makes linux itself real-time,
• others create a small ‘microkernel’ that runs like a hypervisor
• linux kernel runs as a task
• not really linux:
• rt tasks must be modified to communicate with ‘microkernel’
8

9. Main configuration
• No Preemption
• little scheduling overhead
• never schedule unless a function
explicitly calls schedule()
• eg. Servers
9

10. Main configuration
• Voluntary Preemption
• schedule only at “preemption points”
• reduce the maximum latency of rescheduling
• providing faster application reaction at the cost of slightly lower throughput.
• Implementation: might_sleep()
• calls might_resched();
• calls _cond_resched()
10

11. Main configuration
• Preemptible Kernel
• except within spin_locks and
• preempt_disable();
/* preemption is disabled */
preempt_enable();
• every spin_lock acts like a single “global lock” WRT preemption.
11

12. Main configuration
• Preemptible Kernel (Basic RT)
• to debug PREEMPT_RT_FULL
• to learn that problem is mutexes or not
• enables parts of the PREEMPT_RT options,
without sleeping spin_locks
• it will probably go away
12

13. • Fully Preemptible Kernel
• interrupts run as threads
• ability to modify the priorities of interrupts
• user space tasks can run at even a higher priority than interrupts.
• remove disabling of interrupts
• needed for kernel preemption
• conversion of spin_locks into mutexes
• spin_locks side effect
• disabling preemption
• priority inheritance for all locks
linux kernel
Main configuration
13

14. Sleeping spin_lock
• if task is blocked go to sleep
• mutexes
• threaded interrupts needed
• otherwise miss interrupt
• if not-threaded (not-prioritized) interrupt’s spin_locks’ changed to mutexes
• must not be in atomic paths
• preempt_disable()
• local_irq_save(flags): disable interrupt on the current processor and prior to
which it saves current interrupt state into flags.
14

15. raw_spin_lock
• creation of non-preemptible sections
• same as current mainline spin_locks
• should only be used for scheduler, rtmutex implementation,
debugging/tracing infrastructure and for timer interrupts
15

16. Non-Thread IRQs
• timer interrupt
• IRQF_TIMER flag
• flag for timer interrupts
• IRQF_NO_THREAD flag
• Explicitly marking interrupt as not be a thread
16

17. Threaded Interrupts
• driver wants handler as thread
• request_threaded_irq(irq, handler, thread_fn, irqflags, devname, dev_id)
• same as request_irq() with the addition of the thread_fn
• handler
• called in hard interrupt context
• check whether the interrupt originates from the device. If yes it needs to disable the interrupt on the
device and return IRQ_WAKE_THREAD
• IRQ_WAKE_THREAD will wake up the handler thread and run thread_fn
• if null, must have thread_fn
• threadirqs
• commandline parameter
• forces all interrupts to run threaded
• except IRQF_NO_THREAD
• mostly a debug option to allow retrieving better debug data from crashing interrupt
handlers.
17

18. Critical sections & disabling interrupts
• local_irq_disable()
• disables local interrupt delivery
• should not be used since:
• what it's protecting?
• spin_lock_irqsave(lock, flags)
• disables interrupts before taking the spinlock; the previous interrupt state is
stored in flags. If you are absolutely sure nothing else might have already
disabled interrupts, you can use spin_lock_irq instead
• preempt_rt does not disable interrupts
• should be used since:
• spin_lock_irqsave labeling what it’s protecting
18

19. Critical sections & disabling interrupts
• Avoid using
• local_irq_save()
• preempt_disable()
• get_cpu_var() since it calls preempt_disable() for per_cpu variables
• per_cpu variables can be manipulated without explicit locking
• rwlocks
• Writes must wait for unknown amount of readers
• Use
• get_cpu_light()
• local_lock[_irq[save]](var)
• get_local_var(var)
19

20. preempt_rt primitives and methods
• http://lwn.net/Articles/146861/
• Locking Primitives
• Per-CPU Variables
• Interrupt Handlers
• Miscellaneous methods
20

21. Understanding files
• https://www.kernel.org/pub/linux/kernel/projects/rt/
• patch-x.xx.xx-rty.patch
• x: linux kernel, y: patch. x-y pairs are one-to-one.
• patches-x.xx.xx-rty
• ~ 300 patches
• eg. ata-Do-not-disable-interrupts-in-ide-code-for-preemp.patch
• ide-Do-not-disable-interrupts-for-PREEMPT-RT.patch
• core-Do-not-disable-interrupts-on-RT-in-kernel-users.patch
• comments in code: Use the local_irq_*_nort variants to reduce latencies in RT. The codeis
serialized by the locks. No need to disable interrupts.
• ...
21