Introduction to Real-Time Operating Systems
Upcoming SlideShare
Loading in...5
×
 

Introduction to Real-Time Operating Systems

on

  • 6,628 views

shared by Mansoor Mirza...

shared by Mansoor Mirza
Understanding Real-Time Operating Systems
Types of Real-Time Operating System
Requirements for Real-Time Operating System
Difference between General Purpose Operating System (GPOS) and Real-Time Operating System (RTOS)

Conversion Linux kernel to support Real-Time operations
Patching the linux kernel
Major changes in patched kernel
Hands-on labs
Conversion of Linux kernel to support real time
Code a real time application (Audio Feedback removal)

Statistics

Views

Total Views
6,628
Views on SlideShare
6,628
Embed Views
0

Actions

Likes
2
Downloads
284
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Introduction to Real-Time Operating Systems Introduction to Real-Time Operating Systems Presentation Transcript

  • Introduction to Real-Time Operating Systems
  • Objectives
    • Understanding Real-Time Operating Systems
      • Types of Real-Time Operating System
      • Requirements for Real-Time Operating System
      • Difference between General Purpose Operating System (GPOS) and Real-Time Operating System (RTOS)
  • Objectives
    • Conversion Linux kernel to support Real-Time operations
      • Patching the linux kernel
      • Major changes in patched kernel
    • Hands-on labs
      • Conversion of Linux kernel to support real time
      • Code a real time application (Audio Feedback removal)
  • Introduction
    • Operating System (OS)
      • Abstraction layer over the raw hardware
      • Manages the hardware
      • Multiplexes the resources using scheduling policies
    • Types of operating systems
      • General purpose OS(GPOS)
        • Optimized for throughput and fairness
      • Real-Time OS (RTOS)
        • Optimized for handling tasks within timing limits
  • Types of RTOS
    • Hard RTOS
      • Strict timing requirements
      • Optimized w.r.t. predictability and timing limits
      • Does not share critical resources such as CPU
      • Missing a single deadline cause system failure
      • Examples of application under hard real time
        • Defense applications
        • Fuel ignition controller for automobile engine
        • Life supporting medical equipment
        • Industrial control system
  • Types of RTOS
    • Soft RTOS
      • Relative tolerable timing deadlines
      • Tolerance is defined as a part of policy
      • Occasional deadline missing may not cause system failure
      • Examples
        • Audio processing
        • Real-time video processing
        • Network interface subsystem
  • GPOS and RTOS comparison
    • Scheduling
    • Locking mechanism
    • Hardware and software support
  • Scheduling
    • Makes multitasking feasible
    • Shares resources between tasks
    • Follows a scheduling policy
    • Task priority
      • A number assigned to task according to its importance
    • A higher priority task is preferred over a low priority task
  • Scheduling
    • Preemption
      • Shift of resource control from low to high priority task
      • Responsibility of kernel
      • Preemption limitations
        • A code segment can not be preempted when it has acquire some lock which is acquired disabling interrupts
        • A process can only be preempted after a specified task expiration.
          • Tradeoffs between coarse and fine grained timer
  • Scheduling Preemption process
  • Scheduling
    • Yielding
      • Process itself give up the resources
      • Responsibility of process
      • Program designer has to take care of yielding
      • Disadvantages
        • Unable to use available resources efficiently
        • Real-time systems can not rely on voluntary giving up of the CPU
    • Priority inversion
      • A low priority task might cause high priority task to wait for execution
  • Kernel Locks
    • Locks
      • Avoid a critical code section to be accessed by two active threads simultaneously
      • Especially important in multicore machines
    • Effect of locks on real-time response
      • Unpredictable delays for waiting threads
      • Might cause priority inversion
      • Some types of lock disables code preemptions
  • Summary of comparison RTOS
    • Unfair scheduling
      • Scheduling based on priority
    • Kernel is preemptive either completely or up to maximum degree
    • Priority inversion is a major issue
    • Predictable behavior
    GPOS
    • Fair scheduling
      • Scheduling can be adjusted dynamically for optimized throughput
    • Kernel is non-preemptive or have long non-preemptive code sections
    • Priority inversion usually remain unnoticed
    • No predictability guarantees
  • Limitations for RT applications
    • System management interrupts
    • DMA bus mastering
    • On-demand CPU frequency scaling
    • VGA text console
    • Page faults
    • Context switching
  • Limitations for RT applications
    • System Management Mode (SMM)
      • SMI cause system to enter in SMM
      • Debugs the hardware
      • Protects system
        • Shutting down the system if CPU temperature exceeds
      • Emulates hardware
        • Emulates USB keyboard/mouse to ps2 keyboard/mouse
      • CPU jumps to hardwired memory location to service SMI
      • SMI has highest priority
  • Limitations for RT applications
    • System Management Mode (SMM)
      • OS has no control to preempt SMI handler
      • Causes unacceptable delay for RT applications
        • Delay ranges from tens to hundreds of microseconds
    • DMA bus mastering
      • Caused by devices using DMA
        • SATA/PATA/SCSI devices, network adapters, video cards etc.
      • Device drivers can increase latency
      • Common issue for all RTOS
  • Limitations for RT applications
    • On-demand CPU frequency scaling
      • CPU is put in low power state after a period of inactivity
      • Can cause unpredictable or longer delays
  • Limitations for RT applications
    • Page faults
      • Occurs when requested data is not available or its reference is absent from a TLB
      • Types of page faults
        • Major/Hard page faults
        • Minor/Soft page faults
      • Major page faults
        • Occurs when requested data has to be fetched from disk
        • Can cause very large latencies
        • RT application should be written to minimize Major page faults
  • Limitations for RT applications
    • Page faults
      • Minor page faults
        • Occurs when requested data resides in the main memory but missing from TLB
        • It does not involve IO operation to fetch data from disk
        • Has negligible impact in RT performance
      • Tips to avoid page faults
        • Use mlockall() to load all the address space of the program and then lock it such that it cannot be swapped out
  • Limitations for RT applications
    • Page faults
      • Tips to avoid page faults
        • Create all threads at startup time of the application because run time thread creation can cause latencies
        • Avoid dynamic allocation and freeing of memory
        • Minimize the use system calls that are known to generate page faults
  • Limitations for RT applications
    • Context switching
      • Flushed pipeline and branch prediction counters
      • Can cause invalidation of cashes
      • Changes the entries in Instruction and data TLBs
      • Hence context switching might cause unacceptable behavior for some real-time application
  • Conversion of GPOS to RTOS
    • RT support in stock Linux kernel
      • Stock Linux kernel supports soft RT response
      • Two RT scheduling policies in stock Linux kernel
        • SCHED_FIFO
        • SCHED_RR
      • Non-RT scheduling policy
        • SCHED_NORMAL
      • Static priority is implemented in RT scheduling
      • sched_setschedular() can be used to manage scheduling policies
  • Conversion of GPOS to RTOS
    • RT support in stock Linux kernel
      • Scheduling policies definition in /include/linux/sched.h
  • Conversion of GPOS to RTOS
    • SCHED_FIFO
      • First-in-first-out scheduling policy
      • Preempts all SCHED_NORMAL tasks
      • Tasks under SCHED_FIFO cannot be preempted
      • Task itself can yield the resources
      • Scheduling is not based on time slots
      • Two SCHED_FIFO process are scheduled as ‘first come first served’ fashion
  • Conversion of GPOS to RTOS
    • SCHED_RR
      • Same as SCHED_FIFO but involves time slots
      • Process are scheduled in round-robin fashion
      • Share resources in allocated time slots
        • Lower priority task cannot preempt higher priority task
      • SCHED_RR process preempts SCHED_FIFO process
  • Conversion of GPOS to RTOS
    • RT patch for stock Linux kernel
      • CONFIG_PREEMPT_RT
      • Adds support of hard RT in kernel
      • Managed by Ingo Molnar
      • Constantly developing patch
      • A suitable patch is required to be selected corresponding to chosen kernel depending on the kernel version
  • Conversion of GPOS to RTOS
    • Applying the patch
      • Download the real-time patch
      • Download the kernel
      • Kernel and patch should be of same version
      • Commands in these slides are for version 2.6.33.7
  • Conversion of GPOS to RTOS
    • Applying the patch
      • Place both kernel and patch in same directory
      • Unpack the kernel
      • Change the directory
  • Conversion of GPOS to RTOS
    • Applying the patch
      • Dry-run the patch
      • For correct patch output should be like this
  • Conversion of GPOS to RTOS
    • Applying the patch
      • In case of some error check the versions of downloaded kernel and patch
      • Apply the patch if dry-run is succeeded
      • For uncompressed patch following set of commands can be used from uncompressed kernel directory
  • Conversion of GPOS to RTOS
    • Configuration of Linux kernel
      • Open configuration dialog
      • xconfig or gconfig options can also be used
      • Apply following changes
        • Activate High Resolution Timer
        • Enable Complete Preemption (Real-Time)
        • Apply power management settings according to the hardware
  • Conversion of GPOS to RTOS
    • Configuration of Linux kernel
      • Activate High Resolution Timer
  • Conversion of GPOS to RTOS
    • Configuration of Linux kernel
      • Enable Complete Preemption (Real-Time)
  • Conversion of GPOS to RTOS
    • Configuration of Linux kernel
      • Power management configuration
        • Some power management options can raise SMI
        • Disabling some options can cause system damage
        • Read every option carefully
        • Disable optional or unnecessary options
        • Disable CPU frequency scaling
        • Disable USB mouse/keyboard from BIOS
        • Use ps/2 keyboard/mouse
        • Disable TCO timers
  • Conversion of GPOS to RTOS
    • Configuration of Linux kernel
        • Disable CPU frequency scaling (Power Management and ACPI Options ->CPU Frequency scaling)
  • Conversion of GPOS to RTOS
    • Configuration of Linux kernel
        • Disable TCO timers (Device Drivers---> watchdog timer support)
  • Conversion of GPOS to RTOS
    • Building the Linux kernel
      • Make the kernel
      • Make the modules
      • Install the modules
      • Install the kernel
  • Real-Time patch effect on kernel
    • Locking mechanism
    • Priority inheritance
    • Interrupt handler replaced with kernel threads
    • High resolution timers
  • Locking mechanism
    • Locks
      • Prevent the shared resources to be accessed by two active processes simultaneously.
      • Simultaneous access to unprotected shared resources will cause races and Heisen bugs
    • Types of locks
      • Spin locks
      • Semaphores
      • Readers/writer locks
  • Locking mechanism
    • Spinlocks
      • Waiting threads keep on trying until the lock is acquired
      • Waiting threads does not sleep
      • Local interrupts are disabled when spinlock is acquired to avoid deadlocks
      • Used in interrupt handlers
      • Lock should be acquired for short period of time
        • Interrupts are disabled
        • Waiting threads do not sleep and utilize CPU resources
  • Locking mechanism
    • Spinlocks
      • Acquiring a spinlock raw_spin_lock() (/include/linux/spinlock_api_smp.h)
      • Releasing spinlock raw_spin_unlock() (/include/linux/spinlock_api_smp.h)
  • Locking mechanism
    • Spinlocks
      • Advantages of spinlocks
        • Low locking overhead
        • Does not require time to sleep and awake the waiting threads
      • Disadvantages of spinlocks
        • Interrupts are disables
        • Waiting threads utilize the CPU
        • Lock have to released as early as possible
        • Thread acquiring spinlock cannot sleep
  • Locking mechanism
    • Semaphores
      • Sleeping locks
      • Waiting threads sleep and wait in wait queue
      • Waiting task has to be awaken before acquiring the lock
      • Advantages
        • Waiting task frees the CPU for other workload
        • Lock can be acquired for long time
      • Disadvantages
        • Sleeping and awakening overhead
        • Spinlock acquired thread cannot acquire semaphore
  • Locking mechanism
    • Semaphores
      • Sleeping locks
      • Waiting threads sleep and wait in wait queue
      • Waiting task has to be awaken before acquiring the lock
      • Advantages
        • Waiting task frees the CPU for other workload
        • Lock can be acquired for long time
      • Disadvantages
        • Sleeping and awakening overhead
        • Spinlock acquired thread cannot acquire semaphore
  • Locking mechanism
    • Spinlock for RT application – A potential problem
      • Unpredictable in nature
        • Time to acquire lock for waiting thread is dependent on the thread that currently holds the lock
      • OS cannot force the lock holder thread to release the lock
      • Non-preemptive sections of Linux kernel
      • Spinlocks disables local interrupts
  • Locking mechanism
    • Solution for RT behavior
      • Replace spinlock with mutex if possible
    • Advantages
      • Mutex is a binary semaphore
      • Deterministic in nature
      • Can be preempted by high priority task
  • Locking mechanism
    • Spinlocks are unavoidable at low level
      • Example: Implementation of mutex
    • Spinlock usage in RT kernel is just a fraction to that of stock kernel
    • Reduction in usage reduces probability to contend the same spinlock by two different threads
  • Locking mechanism
    • Real-Time implementation of spinlocks
      • rt_spin_lock()
      • rt_spin_lock_fastlock()
  • Locking mechanism
    • Real-Time implementation of spinlocks
      • rt_spin_unlock()
      • rt_spin_lock_fastlock()
  • Priority inheritance
    • Priority inversion
    • L and H require same lock but M requires another lock
    http://book.opensourceproject.org.cn/embedded/oreillyembed/opensource/0596009836/id-i_0596009836_chp_10_sect_4.html
  • Priority inheritance
    • Priority inversion
      • Priority inversion may not be noticeable in GPOS
      • High priority tasks may get starved of resources in GPOS due to priority inversion
      • High priority task should be executed as early as possible
  • Priority inversion
    • Solution of priority inversion
      • Priority inheritance (PI)
    • Priority inversion Priority Inheritance
    http://www.linuxjournal.com/article/9361?page=0,3
  • Priority inversion
    • Priority inversion non-mutual exclusion Locks
      • PI works fine for mutual exclusion locks
      • Problematic in case of other types of locks
        • Example readers/writer locks
      • PI can cause unacceptable response
    • Solution
      • Only one task at a time can read
      • Use read-copy-update if necessary
  • Priority inversion
    • Read-copy-update
      • Technique to share read/write resources
      • Low cost reading
        • Reading is carried out from local copy of data
        • Does not involve conventional locking for reading
      • High cost writing
        • Updates a global pointer to new data
        • Maintains copy of old data until all reading processes are finished
      • Suitable for situation where there are fewer writes and many read operations
  • Threaded interrupt handler
    • Interrupt handler
      • Function that runs whenever a particular interrupt occurs
    • Two parts
      • Top half
      • Bottom half
    • Divided into two parts to reduce interrupt disabled time
  • Threaded interrupt handler
    • Top half
      • Performs essential services to hardware
      • Works with interrupts disabled
      • Gathers raw data
      • Performs critical time sensitive work
    • Bottom half
      • Performs time consuming tasks
      • Processes raw data
  • Threaded interrupt handler
    • Interrupt handler in GOPS
  • Threaded interrupt handler
    • Interrupt handler in GOPS
      • Kernel identifies the interrupt
      • Calls interrupt handler
      • CPU jumps to a particular location
      • Executes top half
      • Raise softirq and returns to pre-interrupt position
    • This behavior is not acceptable in RTOS
      • Cause unpredictability and long latencies
      • Local interrupts are disabled for longer time
  • Threaded interrupt handler
    • Interrupt handler in RTOS
      • Interrupt handler runs as kernel thread
    Building Embedded Linux Systems ISBN-10 = 0596529686, chapter 14,section: Interrupts as threads
  • Threaded interrupt handler
    • Interrupt handler in RTOS
      • Top half identifies the interrupt
      • Sends acknowledgement
      • Initializes kernel thread
      • Rest of task is performed as kernel thread
    • A few interrupts are still handled conventionally
    • Threaded interrupt handler are also available in stock Linux kernel
  • Threaded interrupt handler
    • Advantages of conversion of handler to kernel thread
      • Preemptable interrupt handler
      • Reduces interrupt blocked time
      • Reduces probability of priority inversion
      • Handlers scheduling can be controlled by assigning a priority number to it
      • Reduces unpredictable latencies
  • Threaded interrupt handler
    • Kernel code snippet
      • request_threaded_irq() is to register threaded interrupt handler
  • Threaded interrupt handler
    • request_threaded_irq() code snippet
      • Initializes pointers and calls __setup_irq()
  • Threaded interrupt handler
    • __setup_irq() code snippet
    • preempt_hardirq_setup() code snippet
  • Threaded interrupt handler
    • irq_default_primary_handler() code snippet
      • This function raise the interrupt handler
  • High resolution timers
    • Timer triggers an event at a specified time
    • Shorter the tick of timer clock, higher the timer resolution
    • Jiffy: Software ticking unit
    • Granularity of time is reciprocal of jiffy
    • Increasing the jiffy value:
      • improves the resolution
      • Increases the timer overhead
        • Timer overhead is increased as jiffy is required to be updated more frequently
  • High resolution timers
    • Improving the timer resolution
      • Shift ticking source from jiffy to hardware clock
    • Advantages
      • Improves granularity of clock
      • Reduces timer overhead to update software entity
      • Frees resources that handle jiffy
  • High resolution timers
    • Timer wheel
      • Timer as wheel and delays as bucket in the wheel
      • 1 st set of buckets can provide 256 units delay
        • If an event is to be triggered after 10 units it is placed in 10 th bucket
      • Next level of buckets are used for delay greater than 256 units
      • Each bucket in 2 nd level represents 265 unit delay
      • An event is placed in next layer of buckets if delay is greater than 65536 (256 X 256)
  • High resolution timers
    • Issues with timer wheel implementation in RTOS
      • Unpredictable behavior
      • Rehashing is required for each shift from 2 nd level to 1 st level
      • Rehashing is a function of no. of events in the wheel
        • Delays increases as no of events increase (Rehashing takes O(n), where n is no. of events in the wheel)
  • High resolution timers
    • High resolution(HR) timers for real-time kernel
      • Introduced by Thomas Gleixner
      • Division of timers in two types
        • Action timers
        • Timeout timers
      • Action timers
        • Measures elapsed time using timestamps
      • Timeout timers
        • Trigger an event after a specific time
  • High resolution timers
    • Action timers in timer wheel
      • Rehashing is required for each event entered in wheel
      • Rehashing takes time order O(n)
      • Entering or removal of an entry is of order O(1)
      • Rehashing delay is not constant and hence not predictable and might cause high latency
  • High resolution timers
    • Timeout timers in wheel
      • Does not require rehashing
      • Operating is of order O(1)
      • Efficient and predictable
    • HR timers majorly solves action timer problems
  • High resolution timers
    • HR timers
      • Implements timer in red/black trees
      • Does not involve hash data structure
    • Advantages
      • Add or removes the node in order O(logn)
      • Nodes in trees are already sorted
      • performance improves form O(n) to O(logn) where n is usually a large value
  • Examples of RT application
    • Time bound is the basic characteristic for RT application
    • Example
      • Control systems in industry
      • Position and speed control system for synthetic aperture radars (SARs)
      • Operating system for ebook reader e.g. Kindle
    • Audio feedback cancellation is selected for detailed discussion
  • Audio feedback cancellation
    • Output signal is fed back to input and amplified again
    • Amplified signal includes both input and feedback resulting in noise and suppressing the original signal
    • Usually undesirable and required to eliminate as feedback development is in process
  • Audio feedback cancellation
    • Audio feedback block diagram representation
    • Condition for audio feedback
      • |H(f)H f (f)| < 1
      • phase( H(f)H f (f)) = n360 o where n is integer
  • Audio feedback cancellation
    • Factors effecting feedback
      • Distance
      • Amplification
      • Non constant gain of amplifier
      • Feed-forward path
      • Feed-back path
      • Transfer function of surrounding environment
  • Audio feedback cancellation
    • Audio feedback cause the system to oscillate at single frequency
    • Feedback starts with a number of frequencies
    • Frequency with higher magnitude grow faster
    • This frequency consumes greater power
    • Causes the die out of other feedback frequencies
  • Audio feedback cancellation Frequency spectrum for a voice sample
  • Audio feedback cancellation Frequency spectrum for a voice sample with feedback
  • Audio feedback cancellation
    • Methods of audio feedback cancellation
      • Two major categories
        • Manual feedback cancellation
        • Automatic feedback cancellation
    • Manual feedback methods are useful in static environment
    • In dynamic environments automatic feedback cancellation methods are preffered
  • Audio feedback cancellation
    • Manual feedback cancellation
      • Move the position of microphone
        • Changes phase of the signal fed back to input
        • Reduces magnitude of signal if mic is moved away
      • Tuning the sound source (musical instruments) according to environment
      • Feedback frequencies attenuators can be sued if feedback frequencies are known
  • Audio feedback cancellation
    • Automatic feedback cancellation
      • Two major categories for automatic feedback cancellation
        • Automatic equalization
        • Frequency shifting
  • Audio feedback cancellation
    • Automatic equalization
      • Signal spectrum is observed all the time
      • Adaptive filters are used to suppress the frequency buildup
      • If feedback causes a particular frequency to increase, adaptive filters adopt themselves to suppress that typical frequency
      • Filter response is critical
  • Audio feedback cancellation
    • Frequency shifting
      • Shifts the input signal by a small frequency
      • Breaks feedback path for a particular frequency
      • Very effective method to remove feedback
      • Disturbs the harmonic relationship between the signal components
      • Frequency shift should be selected with care
  • Audio feedback cancellation
    • Frequency shifting
    http://www.alango.com/contents/products/technologies/afr/papers/alango_afc.pdf
  • Audio feedback cancellation
    • Implemented feedback removal technique
      • Human voice is highly uncorrelated when sampled after reasonable time interval
      • Feedback signal is a single tone present in the voice signal
      • High value of correlation of two non-overlapping signals taken from voice sample indicates presence of feedback
  • Audio feedback cancellation
    • Implemented feedback removal technique
      • In case of feedback, remove problematic frequency component form the signal
    • Block diagram for feedback cancellation algorithm
  • Audio feedback cancellation
    • Removing the feedback frequency
      • Adaptive filters are used for removing the feedback
    • Filter properties
      • Adaptive
      • Quick to adopt to new frequency
      • Very high Q as to remove a single frequency
  • Audio feedback cancellation
    • Filter implementation
      • Implementation of such a filter is very difficult
        • Very large number of taps to get required response
      • Low Q filter completely distort the signal
      • Frequency domain filter
    • Procedure for frequency rectification
      • Convert the signal to frequency domain
      • Detect the frequency that is to be removed
      • Remove that frequency
      • Convert the signal back to time domain
  • Audio feedback cancellation
    • Block diagram for frequency domain filter
    • Removing frequency component can be seen as multiplying the signal with a specific vector
  • Audio feedback cancellation
    • Filter implementation
      • Initially signal is converted to frequency domain using FFT
      • Required frequency is removed by zeroing the appropriate elements of FFT output
      • Signal is converted back to time domain using IFFT
  • Audio feedback cancellation
    • Results of frequency domain filtering
    • MATLAB plots for testing audio signal
  • Audio feedback cancellation
    • Parallelization of filter
      • Major workload in filter is FFT and IFFT
      • Parallelization of filter means to implement the FFT and IFFT in parallel
        • In the sample code FFT and IFFT is carried out using FFTW library which is open source library that supports parallel implementation of FFT and IFFT.
      • Parallelization improves the throughput of the system
  • Audio feedback cancellation
    • Result of parallelization ( Data is gathered using intel Xeon 2 GHz, Quad core based system )
    • Increase in number of threads my lower the throughput due to synchronization overhead of threads and it might be overcome by increasing the amount of workload.
    *Throughput is for 100 second workload Threads Execution Time* (ms) 1 1294.873 2 836.96 3 645.57 4 578.07 5 510.23 6 487.7 7 462.05 8 682.008
  • Labs
    • Conversion of stock kernel to real time kernel
      • Procedure to patch the kernel and to install it
    • Audio feedback cancellation
      • Algorithm and C implementation of removal of audio feedback