Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
E.s unit 4 and 5


Real Time Systems

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Real Time Systems

  1. 1. Real Time Systems© 2010 Anil Kumar Pugalia <> All Rights Reserved.
  2. 2. What to Expect?Ws of Real Time & RTOS?Design Methodologies of RTOSLinux → Real Time Linux Various Latencies & Schedulers The “Real Time Patch” Path Real Time Applications Real Time Kernel DebuggingPeek into a Co-Kernel © 2010 Anil Kumar Pugalia <> 2 All Rights Reserved.
  3. 3. Real Time DefinitionCorrectness depends on Functional Accuracy, and Timings of the Result producedExamples MPEG Decoder in your DVD player Call Response in your Mobile Phone © 2010 Anil Kumar Pugalia <> 3 All Rights Reserved.
  4. 4. Real Time TypesHard Real Time Guaranteed to meet the response reqs Examples: Defense Systems, Vehicle Control Systems, Satellite SystemsSoft Real Time Once in a while may not meet the response reqs Examples: Multimedia Devices, VoIP, CE devicesAn intermediate one: Firm Real TimeCheck: What is our Desktop? © 2010 Anil Kumar Pugalia <> 4 All Rights Reserved.
  5. 5. Is Real Time needed?Deadline of Applications Reduces Quality or UnacceptableExternal Device Interactions with Apps Handling of Messages – Time-bound or notApplication Task Priority Could it be needed higher than OS servicesExpressing delays & timeouts Is it needed to be fine-grained, say in usecs © 2010 Anil Kumar Pugalia <> 5 All Rights Reserved.
  6. 6. Real Time OSRTOS Phylum Non Real-time Soft Real-time Hard Real-timeConflicting Requirements Real Time vs Rich Features of GPOS © 2010 Anil Kumar Pugalia <> 6 All Rights Reserved.
  7. 7. Expected Features of an RTOSRTOS Requirements Strictly Enforced Task Priorities Fine-grained Preemption External Event Handling in Bounded time frameWhich entails Predictable Low Interrupt Latency Predictable Low Scheduler Latency Priority InheritanceBut wanted with Rich Features of Multi Tasking / Processing Interprocess Communication & Synchronization © 2010 Anil Kumar Pugalia <> 7 All Rights Reserved.
  8. 8. RTOS Design MethodologiesExpand an RTOSMake a GPOS Real Time capable Linux → Real Time Linux (Separate Session)The Co-Kernel Approach RT Apps in User Space Open Source Xenomai (Separate Session) Open Source RTAI RT Apps in Kernel Space RTLinux from Windriver © 2010 Anil Kumar Pugalia <> 8 All Rights Reserved.
  9. 9. Expand an RTOSDifficult to ensure Determinism As all core capabilities must be fully preemptiveDrivers for Hardware becomes very complexExisting Software cannot be used without changesHigh Maintenance Costs for Developers and CustomersSo, lets understand the other one w.r.t. Linux © 2010 Anil Kumar Pugalia <> 9 All Rights Reserved.
  10. 10. Linux → Real Time Linux© 2010 Anil Kumar Pugalia <> 10 All Rights Reserved.
  11. 11. Why Linux is non Real Time?High Interrupt LatencyHigh Scheduler LatencyOS services with indeterministic timing behaviour IPC, Memory Allocation, ...Linux is in general undeterministic in its response Virtual Memory, System Calls, …Later two are more of a better Coding PrinciplesHowever, the first two are major Design ChangesLets take a closer look © 2010 Anil Kumar Pugalia <> 11 All Rights Reserved.
  12. 12. Real Time related ConceptsLatency Interrupt SchedulingBoth are related to Preemption And Critical Sections © 2010 Anil Kumar Pugalia <> 12 All Rights Reserved.
  13. 13. Latency VisualizationInterrupt ISR Runs ISR Signals RT ProcessEvent RT Process Runs Time t0 t1 t2 t3 Interrupt Interrupt Scheduling Latency Scheduling Latency Interrupt to Process Latency © 2010 Anil Kumar Pugalia <> 13 All Rights Reserved.
  14. 14. Latency DetailsWhy Interrupt Latency?Attributes of ISRReal Time Process Typically, high priority Otherwise, anyway it is a problemAll 3 could potentially pose problems For Hard Real Time © 2010 Anil Kumar Pugalia <> 14 All Rights Reserved.
  15. 15. Possible SolutionsKey Requirement Schedule RT Process, within a given timePossible Helpers Remove / Reduce interrupt-disabled code (critical sections) Write ISRs really really minimal in execution time Real Time Apps with the highest priority May be higher than interrupts, softirqs, etc © 2010 Anil Kumar Pugalia <> 15 All Rights Reserved.
  16. 16. Preemption Visualization Time User Space System Call Interface Kernel Space Process A Process B © 2010 Anil Kumar Pugalia <> 16 All Rights Reserved.
  17. 17. Preemption DetailsHigher Preemption makes it more RealTime feasibleHigher Priority (RT Tasks) could bescheduled in a more predictable timeSources of Preemption Latency Critical Section Processing Interrupt Context Processing © 2010 Anil Kumar Pugalia <> 17 All Rights Reserved.
  18. 18. Preemption & Critical SectionsHigher Preemption at the cost of reducing Critical Sections Isolate & Minimize the critical sections Protect them by disabling preemption spin_lock / spin_unlock; preempt_disable / preempt_enablePreemption Models for reducing Critical Sections Preempt only safe locations Entry & Exit of system calls, Return from interrupt processing, … Put explicit “allow preempt” code at these places Overall not an excellent approach Preempt everywhere except only the Critical Sections lock-breaking mechanism Getting achieved by instrumenting kernel for latency measurements And by fixing the longest latency code paths © 2010 Anil Kumar Pugalia <> 18 All Rights Reserved.
  19. 19. SMP World & Critical SectionsUniprocessor case needs Protection from Interrupt Processing Exception ProcessingMultiprocessor with additional Multiple Kernel ThreadsPosed a bigger need than Real Time forreduced Critical SectionsReal Time has gained benefits from SMPdevelopment, as well © 2010 Anil Kumar Pugalia <> 19 All Rights Reserved.
  20. 20. Real Time Kernel PatchCritical Section Control is a major stepWe have achieved good Soft Real Time Performance Since 2.6.12, in single-digit milliseconds On reasonably fast x86 processorBut without solving the Interrupt Latency Hard Real Time is not possibleRecent Developments have majorly focused on this aspectA major Real Time Kernel Patch evolved Maintained by Ingo Molnar Available @ Could be downloaded using say, ketchup -G And takes care of many more issues © 2010 Anil Kumar Pugalia <> 20 All Rights Reserved.
  21. 21. Ws & Hows of RT Kernel Patch? Lots of Questions What all does it contain? How to configure it? What should be used & when? And many more Before answering these, lets understand Schedulers in main-stream Kernel Preemption Support in main-stream Kernel © 2010 Anil Kumar Pugalia <> 21 All Rights Reserved.
  22. 22. Linux SchedulersProvides multi-tasking capabilities by Time Slicing Preemption Based on various task priorities Specified by its scheduling policiesUnderstands the following execution instances Kernel Thread User Process User ThreadLinux Basic Scheduling Normal (SCHED_OTHER) – Fairness SchedulingOther Advanced Scheduling supported Round Robin (SCHED_RR) FIFO (SCHED_FIFO)All Schedulers are O(1) © 2010 Anil Kumar Pugalia <> 22 All Rights Reserved.
  23. 23. Linux Kernel Preemption LevelsNone (PREEMPT_NONE) No forced preemption Overall throughput, on average, is goodVoluntary (PREEMPT_VOLUNTARY) First stage of latency reduction Explicit preemption points are placed at strategic locationsStandard (PREEMPT_DESKTOP) Preemption enabled everywhere except within critical sections Good for soft real-time applications, like audio, multimedia, …Kernel Parameter: preempt © 2010 Anil Kumar Pugalia <> 23 All Rights Reserved.
  24. 24. Preemption Level from RT Patch Complete (PREEMPT_RT) Spin locks replaced with preembtable mutexes Preemption enabled everywhere except where protected by preempt_disable() Smoothes out variation in latency Allows a low & predictable latency for time- critical real-time applications © 2010 Anil Kumar Pugalia <> 24 All Rights Reserved.
  25. 25. Additional Real Time FeaturesISRs as Kernel Tasks CONFIG_PREEMPT_HARDIRQ Schedulable & Preemptible Priority Assignment as Required Control Window: /proc/sys/kernel/hardirq_preemption Kernel Parameter: hardirq-preemptPreemptable Softirqs CONFIG_PREEMPT_SOFTIRQ All ran in the context of ksoftirqd → Now as individual threads A proper Linux task configured for real time Control Window: /proc/sys/kernel/softirq_preemption Kernel Parameter: softirq-preemptCertain Read-Copy-Update (RCU) sections preemptible CONFIG_PREEMPT_RCU RCU is synchronization primitive for frequently read data © 2010 Anil Kumar Pugalia <> 25 All Rights Reserved.
  26. 26. Creating a Real Time Process#include <sched.h>#define MY_RT_PRIO 1int main(...) { int old_policy; struct sched_param params = { .sched_priority = MY_RT_PRIO }; ... old_policy = sched_getscheduler(0); if (sched_setscheduler(0, SCHED_RR, &params) == -1) handle_error(); …} © 2010 Anil Kumar Pugalia <> 26 All Rights Reserved.
  27. 27. RT Application: Dos & DontsHave very few Real Time ProcessesKeep the RT Applications Short & EfficientDo not do heavy duty operations Like Memory Allocation, etc If essential, move to Initialization SectionsCheck for the working of the needed OSservices Especially, if any RT Application has priority higher than that OS ServiceTest for the Desired Results © 2010 Anil Kumar Pugalia <> 27 All Rights Reserved.
  28. 28. Debugging Real Time KernelSoft Lockup CONFIG_DETECT_SOFTLOCKUPUnsafe Preemption Use Logging CONFIG_DEBUG_PREEMPTDeadlock Conditions Detection & Reporting CONFIG_DEBUG_DEADLOCK Deadlocks due to semaphores & spinlocksRuntime Control of Locking Mode CONFIG_DEBUG_RT_LOCKING_MODE Mutex back to Spinlock © 2010 Anil Kumar Pugalia <> 28 All Rights Reserved.
  29. 29. Tracing Real Time KernelLatency Tracing CONFIG_LATENCY_TRACE For data associated with last maximum preemption latency Kernel Window: /proc/latency_traceFunction Call Tracing Inserts hooks to every function via gcc Gives function level details through Kernel Window: /proc/latency_traceSoon getting outdated by Ftrace © 2010 Anil Kumar Pugalia <> 29 All Rights Reserved.
  30. 30. Further Tracing RT KernelHigh Priority Process Wakeup CONFIG_WAKEUP_TIMING CONFIG_WAKEUP_LATENCY_HISTOr, Interrupt Off Timing CONFIG_CRITICAL_IRQSOFF_TIMING CONFIG_INTERRUPT_OFF_HISTKernel Windows /proc/sys/kernel/preempt_max_latency /proc/latency_hist/interrupt_off_latency/CPU0 (0-10000us)Time spent with Preemption disabled (in critical sections) CONFIG_CRITICAL_PREEMPT_TIMING CONFIG_PREEMPT_OFF_HIST © 2010 Anil Kumar Pugalia <> 30 All Rights Reserved.
  31. 31. Real Time Linux Myths & RealitiesWith all these understood Is Linux really Hard Real Time? Can it be used in all Real Time Embedded activities? Can it be used in a Nuclear Reactor?Answers Can be used in most Real Time Embedded Applications, such as Robotics But cannot be “one-size-fits-it-all” approach to Real Time Is still a patch and hence not that extensibly tested Not all Device Drivers have yet been verified as Real Time safe More & more users (such as multimedia users), using the patch would verify more drivers, as they are the users noticing longer latency delays caused by these devices But even if they are, it is after all a GPOS Harder to verify a lack of bugs Could have potential bugs (from real time perspective) © 2010 Anil Kumar Pugalia <> 31 All Rights Reserved.
  32. 32. Co-Kernel Approach© 2010 Anil Kumar Pugalia <> 32 All Rights Reserved.
  33. 33. Ws of Co-Kernel ApproachDo not convert the GPOS into an RTOSPlace a small Real Time Kernel besides it On the same HardwareThis would be the Co-Kernel Taking care of all Real Time needsNon-RT tasks are done by the GPOSSo, © 2010 Anil Kumar Pugalia <> 33 All Rights Reserved.
  34. 34. What is a Co-Kernel? Co-Kernel is a SubsystemIntegrated with the main Kernel Handling the Real Time part © 2010 Anil Kumar Pugalia <> 34 All Rights Reserved.
  35. 35. Types of Co-Kernel ApproachesTwo Prevalent Approaches Support running of RT Apps in User Space RT Apps to be embodied into Kernel SpaceOur Focus of Discussion is the First One With Xenomai in consideration © 2010 Anil Kumar Pugalia <> 35 All Rights Reserved.
  36. 36. Benefits of XenomaiCo-Kernel or Core running under Kernel, licensed under GPL 2Kernel Support Both 2.4 & 2.6 for with & without MMU systemsProvides generic building blocks to implement RT APIs (skins)RT API Support for VxWorks, pSOS+, VRTX, uITRON, POSIX 1003.1bUser Space i/f libraries: LGPL 2.1Architecture Support PowerPC*, Blackfin, ARM, x86*Excellent Resources Documentation, Technical Articles, …Website: © 2010 Anil Kumar Pugalia <> 36 All Rights Reserved.
  37. 37. Xenomai Architecture User Space User Space User Space User Space Application Application Application Application Linux syscall interfaceVxWorks pSOS VRTX POSIX ... Kernel-based Skin Skin Skin Skin Applications Abstract RTOS Core SAL / HAL Portability Layers I-Pipe © 2010 Anil Kumar Pugalia <> 37 All Rights Reserved.
  38. 38. Xenomai ComponentsInterrupt PipelineHardware & System Abstraction LayersXenomai Core & NucleusXenomai Skins © 2010 Anil Kumar Pugalia <> 38 All Rights Reserved.
  39. 39. How Xenomai Works?Real Time ShadowNew Sets of System CallsSharing Kernel Features By Domain MigrationReal Time Driver Model Mediation © 2010 Anil Kumar Pugalia <> 39 All Rights Reserved.
  40. 40. What all have we learnt?Ws of Real Time & RTOS?Design Methodologies of RTOSLinux → Real Time Linux Various Latencies & Schedulers The “Real Time Patch” Path Real Time Applications Real Time Kernel DebuggingPeek into a Co-Kernel © 2010 Anil Kumar Pugalia <> 40 All Rights Reserved.
  41. 41. Any Queries?© 2010 Anil Kumar Pugalia <> 41 All Rights Reserved.
  • kschoudhuryChoudhury

    Mar. 24, 2020
  • wyh991131

    Jun. 13, 2019
  • AksharaRSankar

    Jun. 6, 2018

    Jul. 23, 2017
  • AmanPurwar

    May. 14, 2017
  • kritikapurohit98

    Sep. 13, 2016
  • superdudern

    Aug. 25, 2015
  • flylinux

    Jul. 25, 2015

    Jul. 3, 2015
  • shubhamvermainform

    Apr. 16, 2015
  • RamachandranRadhakri

    Mar. 29, 2015
  • VikasTiwari18

    Jan. 23, 2015
  • piliwilliam

    Aug. 16, 2014
  • mjrajeshwari

    Jul. 11, 2014
  • DiluBiki

    Oct. 24, 2012
  • yarahebara

    Nov. 29, 2011
  • Hiba_maitah

    Mar. 5, 2011


Total views


On Slideshare


From embeds


Number of embeds