SlideShare a Scribd company logo
1 of 10
FAULT TOLERANT
ARCHITECTURES
Presented by
D.S.MYDHEESWARAN
Fault in real time operating
system
A fault is an unexpected and
exceptional condition or event that can
disrupt the normal operation of the
system.
Faults in an RTOS can lead to system
failures, unpredictable behavior, or
reduced real-time performance, which is
critical in applications where timing and
reliability are paramount
Common types of faults in RTOS
 Hardware Faults
 Deadlock
 Software Faults
 Memory Leaks
 Interrupt Handling Faults
 Stack Overflow
 Priority Inversion
 Timing Violations
Fault tolerance mechanisms
 Fault tolerance mechanisms are strategies
and techniques used to ensure that the
RTOS and the embedded systems running
on it can continue to function reliably in
the presence of faults, errors, or
unexpected failures.
 The goal is to maintain the system's real-
time behavior and meet its timing
constraints even when faults occur.
Fault injection testing
 Fault injection testing in a real time operating
system is a controlled and deliberate process
of introducing simulated faults, errors, or
abnormal conditions into the RTOS and the
embedded system it manages.
 The primary goal of this testing technique is
to assess how the RTOS and the entire
system respond to these injected faults,
evaluate their fault tolerance mechanisms,
and determine the system's resilience under
adverse conditions.
Interrupt handling faults
 Interrupt handling faults are refer to
situations where the RTOS encounters
errors or issues while processing
hardware or software interrupts.
 Interrupts are mechanisms used by the
operating system to respond to external
events, such as hardware device signals,
timers, or software-triggered events.
Fault masking
 Fault masking is refers to a situation
where a fault or error occurs within the
system but is not immediately detected or
reported because of the RTOS's ability to
continue normal operation without
apparent disruption.
 This can give the illusion of a fault-
tolerant system, but it may hide
underlying issues that can lead to
problems later on.
Fault tolerant architectures
 Fault tolerant architectures in real time
operating systems are design strategies and
mechanisms implemented to ensure that an
embedded system can continue to operate
correctly and reliably in the presence of
hardware or software faults, errors, or
failures.
 These architectures are crucial for systems
where reliability and availability are
paramount, such as safety critical systems,
aerospace applications, medical devices, and
industrial control systems.
Hardware Redundancy
 This approach involves duplicating critical
hardware components (e.g., CPUs,
memory, power supplies) to create a
redundant system.
Software Redundancy
 Redundancy can also be applied at the
software level, where multiple copies of
the same application or task run
concurrently on separate processors.
 In the event of a fault in one instance,
another can take over.

More Related Content

Similar to Fault-tolerant architectures in real-time operating systems

Developing fault tolerance integrity protocol for distributed real time systems
Developing fault tolerance integrity protocol for distributed real time systemsDeveloping fault tolerance integrity protocol for distributed real time systems
Developing fault tolerance integrity protocol for distributed real time systemsDr Amira Bibo
 
Proposed Algorithm for Surveillance Applications
Proposed Algorithm for Surveillance ApplicationsProposed Algorithm for Surveillance Applications
Proposed Algorithm for Surveillance ApplicationsEditor IJCATR
 
There are many operating systemsReal-Time Operating SystemReal-t.pdf
There are many operating systemsReal-Time Operating SystemReal-t.pdfThere are many operating systemsReal-Time Operating SystemReal-t.pdf
There are many operating systemsReal-Time Operating SystemReal-t.pdfankitmobileshop235
 
UNIT-I-RTOS and Concepts
UNIT-I-RTOS and ConceptsUNIT-I-RTOS and Concepts
UNIT-I-RTOS and ConceptsDr.YNM
 
Software rejuvenation
Software rejuvenationSoftware rejuvenation
Software rejuvenationRVCE
 
Software rejuvenation
Software rejuvenationSoftware rejuvenation
Software rejuvenationRVCE2
 
Software rejuvenation
Software rejuvenationSoftware rejuvenation
Software rejuvenationRVCE
 
Critical System Specification in Software Engineering SE17
Critical System Specification in Software Engineering SE17Critical System Specification in Software Engineering SE17
Critical System Specification in Software Engineering SE17koolkampus
 
FMEA - Failure Mode and Effects Analysis
FMEA - Failure Mode and Effects AnalysisFMEA - Failure Mode and Effects Analysis
FMEA - Failure Mode and Effects Analysisマルセロ 白井
 
Embedded system software
Embedded system softwareEmbedded system software
Embedded system softwareJamia Hamdard
 
Real time os(suga)
Real time os(suga) Real time os(suga)
Real time os(suga) Nagarajan
 
DevOps_SelfHealing
DevOps_SelfHealingDevOps_SelfHealing
DevOps_SelfHealingAtul Dhingra
 
Real Time Operating System ,Structures of Operating System (Monolithic, Micro...
Real Time Operating System ,Structures of Operating System(Monolithic, Micro...Real Time Operating System ,Structures of Operating System(Monolithic, Micro...
Real Time Operating System ,Structures of Operating System (Monolithic, Micro...VIJETHAK2
 
Fault Tolerance System
Fault Tolerance SystemFault Tolerance System
Fault Tolerance Systemprakashjjaya
 
The Comprehensive Guide to Embedded Systems Architecture: Building Blocks, De...
The Comprehensive Guide to Embedded Systems Architecture: Building Blocks, De...The Comprehensive Guide to Embedded Systems Architecture: Building Blocks, De...
The Comprehensive Guide to Embedded Systems Architecture: Building Blocks, De...rohithraj3101
 
Real Time Software Design in Software Engineering SE13
Real Time Software Design in Software Engineering SE13Real Time Software Design in Software Engineering SE13
Real Time Software Design in Software Engineering SE13koolkampus
 

Similar to Fault-tolerant architectures in real-time operating systems (20)

Developing fault tolerance integrity protocol for distributed real time systems
Developing fault tolerance integrity protocol for distributed real time systemsDeveloping fault tolerance integrity protocol for distributed real time systems
Developing fault tolerance integrity protocol for distributed real time systems
 
Proposed Algorithm for Surveillance Applications
Proposed Algorithm for Surveillance ApplicationsProposed Algorithm for Surveillance Applications
Proposed Algorithm for Surveillance Applications
 
There are many operating systemsReal-Time Operating SystemReal-t.pdf
There are many operating systemsReal-Time Operating SystemReal-t.pdfThere are many operating systemsReal-Time Operating SystemReal-t.pdf
There are many operating systemsReal-Time Operating SystemReal-t.pdf
 
UNIT-I-RTOS and Concepts
UNIT-I-RTOS and ConceptsUNIT-I-RTOS and Concepts
UNIT-I-RTOS and Concepts
 
Software rejuvenation
Software rejuvenationSoftware rejuvenation
Software rejuvenation
 
Software rejuvenation
Software rejuvenationSoftware rejuvenation
Software rejuvenation
 
Software rejuvenation
Software rejuvenationSoftware rejuvenation
Software rejuvenation
 
Ch15
Ch15Ch15
Ch15
 
Testability
TestabilityTestability
Testability
 
Critical System Specification in Software Engineering SE17
Critical System Specification in Software Engineering SE17Critical System Specification in Software Engineering SE17
Critical System Specification in Software Engineering SE17
 
FMEA - Failure Mode and Effects Analysis
FMEA - Failure Mode and Effects AnalysisFMEA - Failure Mode and Effects Analysis
FMEA - Failure Mode and Effects Analysis
 
Embedded system software
Embedded system softwareEmbedded system software
Embedded system software
 
Real time os(suga)
Real time os(suga) Real time os(suga)
Real time os(suga)
 
DevOps_SelfHealing
DevOps_SelfHealingDevOps_SelfHealing
DevOps_SelfHealing
 
Real Time Operating System ,Structures of Operating System (Monolithic, Micro...
Real Time Operating System ,Structures of Operating System(Monolithic, Micro...Real Time Operating System ,Structures of Operating System(Monolithic, Micro...
Real Time Operating System ,Structures of Operating System (Monolithic, Micro...
 
Real timedata
Real timedataReal timedata
Real timedata
 
Fault Tolerance System
Fault Tolerance SystemFault Tolerance System
Fault Tolerance System
 
The Comprehensive Guide to Embedded Systems Architecture: Building Blocks, De...
The Comprehensive Guide to Embedded Systems Architecture: Building Blocks, De...The Comprehensive Guide to Embedded Systems Architecture: Building Blocks, De...
The Comprehensive Guide to Embedded Systems Architecture: Building Blocks, De...
 
Ch10
Ch10Ch10
Ch10
 
Real Time Software Design in Software Engineering SE13
Real Time Software Design in Software Engineering SE13Real Time Software Design in Software Engineering SE13
Real Time Software Design in Software Engineering SE13
 

More from mydheeswarandseec

OPERATING SYSTEMS FOR SENSOR NETWORKSpptx
OPERATING SYSTEMS FOR SENSOR NETWORKSpptxOPERATING SYSTEMS FOR SENSOR NETWORKSpptx
OPERATING SYSTEMS FOR SENSOR NETWORKSpptxmydheeswarandseec
 
SENSOR TASKING AND CONTROL in WSN .pptx
SENSOR TASKING AND CONTROL in WSN  .pptxSENSOR TASKING AND CONTROL in WSN  .pptx
SENSOR TASKING AND CONTROL in WSN .pptxmydheeswarandseec
 
WIRELESS TRANSMISSION in wireless sensor network architecture.pptx
WIRELESS TRANSMISSION in wireless sensor network architecture.pptxWIRELESS TRANSMISSION in wireless sensor network architecture.pptx
WIRELESS TRANSMISSION in wireless sensor network architecture.pptxmydheeswarandseec
 
SINGLE NODE ARCHITECTURE Hardware components of a sensor node.pptx
SINGLE NODE ARCHITECTURE Hardware components of a sensor node.pptxSINGLE NODE ARCHITECTURE Hardware components of a sensor node.pptx
SINGLE NODE ARCHITECTURE Hardware components of a sensor node.pptxmydheeswarandseec
 
Introduction to Wireless Sensor Network .pptx
Introduction to  Wireless Sensor Network .pptxIntroduction to  Wireless Sensor Network .pptx
Introduction to Wireless Sensor Network .pptxmydheeswarandseec
 
Memory Management in real time operating system
Memory Management in real time operating systemMemory Management in real time operating system
Memory Management in real time operating systemmydheeswarandseec
 
Design Techniques in the Real Time System
Design Techniques in the Real Time SystemDesign Techniques in the Real Time System
Design Techniques in the Real Time Systemmydheeswarandseec
 

More from mydheeswarandseec (7)

OPERATING SYSTEMS FOR SENSOR NETWORKSpptx
OPERATING SYSTEMS FOR SENSOR NETWORKSpptxOPERATING SYSTEMS FOR SENSOR NETWORKSpptx
OPERATING SYSTEMS FOR SENSOR NETWORKSpptx
 
SENSOR TASKING AND CONTROL in WSN .pptx
SENSOR TASKING AND CONTROL in WSN  .pptxSENSOR TASKING AND CONTROL in WSN  .pptx
SENSOR TASKING AND CONTROL in WSN .pptx
 
WIRELESS TRANSMISSION in wireless sensor network architecture.pptx
WIRELESS TRANSMISSION in wireless sensor network architecture.pptxWIRELESS TRANSMISSION in wireless sensor network architecture.pptx
WIRELESS TRANSMISSION in wireless sensor network architecture.pptx
 
SINGLE NODE ARCHITECTURE Hardware components of a sensor node.pptx
SINGLE NODE ARCHITECTURE Hardware components of a sensor node.pptxSINGLE NODE ARCHITECTURE Hardware components of a sensor node.pptx
SINGLE NODE ARCHITECTURE Hardware components of a sensor node.pptx
 
Introduction to Wireless Sensor Network .pptx
Introduction to  Wireless Sensor Network .pptxIntroduction to  Wireless Sensor Network .pptx
Introduction to Wireless Sensor Network .pptx
 
Memory Management in real time operating system
Memory Management in real time operating systemMemory Management in real time operating system
Memory Management in real time operating system
 
Design Techniques in the Real Time System
Design Techniques in the Real Time SystemDesign Techniques in the Real Time System
Design Techniques in the Real Time System
 

Recently uploaded

How to Manage Call for Tendor in Odoo 17
How to Manage Call for Tendor in Odoo 17How to Manage Call for Tendor in Odoo 17
How to Manage Call for Tendor in Odoo 17Celine George
 
How to Add New Custom Addons Path in Odoo 17
How to Add New Custom Addons Path in Odoo 17How to Add New Custom Addons Path in Odoo 17
How to Add New Custom Addons Path in Odoo 17Celine George
 
Introduction to TechSoup’s Digital Marketing Services and Use Cases
Introduction to TechSoup’s Digital Marketing  Services and Use CasesIntroduction to TechSoup’s Digital Marketing  Services and Use Cases
Introduction to TechSoup’s Digital Marketing Services and Use CasesTechSoup
 
REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxDr. Ravikiran H M Gowda
 
Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jisc
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...Nguyen Thanh Tu Collection
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxRamakrishna Reddy Bijjam
 
21st_Century_Skills_Framework_Final_Presentation_2.pptx
21st_Century_Skills_Framework_Final_Presentation_2.pptx21st_Century_Skills_Framework_Final_Presentation_2.pptx
21st_Century_Skills_Framework_Final_Presentation_2.pptxJoelynRubio1
 
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...EADTU
 
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdf
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdfFICTIONAL SALESMAN/SALESMAN SNSW 2024.pdf
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdfPondicherry University
 
How to Add a Tool Tip to a Field in Odoo 17
How to Add a Tool Tip to a Field in Odoo 17How to Add a Tool Tip to a Field in Odoo 17
How to Add a Tool Tip to a Field in Odoo 17Celine George
 
Graduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - EnglishGraduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - Englishneillewis46
 
The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxheathfieldcps1
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17Celine George
 
Towards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxTowards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxJisc
 
Details on CBSE Compartment Exam.pptx1111
Details on CBSE Compartment Exam.pptx1111Details on CBSE Compartment Exam.pptx1111
Details on CBSE Compartment Exam.pptx1111GangaMaiya1
 
Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)Jisc
 
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...Amil baba
 

Recently uploaded (20)

OS-operating systems- ch05 (CPU Scheduling) ...
OS-operating systems- ch05 (CPU Scheduling) ...OS-operating systems- ch05 (CPU Scheduling) ...
OS-operating systems- ch05 (CPU Scheduling) ...
 
How to Manage Call for Tendor in Odoo 17
How to Manage Call for Tendor in Odoo 17How to Manage Call for Tendor in Odoo 17
How to Manage Call for Tendor in Odoo 17
 
How to Add New Custom Addons Path in Odoo 17
How to Add New Custom Addons Path in Odoo 17How to Add New Custom Addons Path in Odoo 17
How to Add New Custom Addons Path in Odoo 17
 
Introduction to TechSoup’s Digital Marketing Services and Use Cases
Introduction to TechSoup’s Digital Marketing  Services and Use CasesIntroduction to TechSoup’s Digital Marketing  Services and Use Cases
Introduction to TechSoup’s Digital Marketing Services and Use Cases
 
REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptx
 
Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docx
 
21st_Century_Skills_Framework_Final_Presentation_2.pptx
21st_Century_Skills_Framework_Final_Presentation_2.pptx21st_Century_Skills_Framework_Final_Presentation_2.pptx
21st_Century_Skills_Framework_Final_Presentation_2.pptx
 
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
Transparency, Recognition and the role of eSealing - Ildiko Mazar and Koen No...
 
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdf
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdfFICTIONAL SALESMAN/SALESMAN SNSW 2024.pdf
FICTIONAL SALESMAN/SALESMAN SNSW 2024.pdf
 
How to Add a Tool Tip to a Field in Odoo 17
How to Add a Tool Tip to a Field in Odoo 17How to Add a Tool Tip to a Field in Odoo 17
How to Add a Tool Tip to a Field in Odoo 17
 
Graduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - EnglishGraduate Outcomes Presentation Slides - English
Graduate Outcomes Presentation Slides - English
 
The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptx
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17
 
Towards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxTowards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptx
 
Details on CBSE Compartment Exam.pptx1111
Details on CBSE Compartment Exam.pptx1111Details on CBSE Compartment Exam.pptx1111
Details on CBSE Compartment Exam.pptx1111
 
Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)
 
VAMOS CUIDAR DO NOSSO PLANETA! .
VAMOS CUIDAR DO NOSSO PLANETA!                    .VAMOS CUIDAR DO NOSSO PLANETA!                    .
VAMOS CUIDAR DO NOSSO PLANETA! .
 
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...
 

Fault-tolerant architectures in real-time operating systems

  • 2. Fault in real time operating system A fault is an unexpected and exceptional condition or event that can disrupt the normal operation of the system. Faults in an RTOS can lead to system failures, unpredictable behavior, or reduced real-time performance, which is critical in applications where timing and reliability are paramount
  • 3. Common types of faults in RTOS  Hardware Faults  Deadlock  Software Faults  Memory Leaks  Interrupt Handling Faults  Stack Overflow  Priority Inversion  Timing Violations
  • 4. Fault tolerance mechanisms  Fault tolerance mechanisms are strategies and techniques used to ensure that the RTOS and the embedded systems running on it can continue to function reliably in the presence of faults, errors, or unexpected failures.  The goal is to maintain the system's real- time behavior and meet its timing constraints even when faults occur.
  • 5. Fault injection testing  Fault injection testing in a real time operating system is a controlled and deliberate process of introducing simulated faults, errors, or abnormal conditions into the RTOS and the embedded system it manages.  The primary goal of this testing technique is to assess how the RTOS and the entire system respond to these injected faults, evaluate their fault tolerance mechanisms, and determine the system's resilience under adverse conditions.
  • 6. Interrupt handling faults  Interrupt handling faults are refer to situations where the RTOS encounters errors or issues while processing hardware or software interrupts.  Interrupts are mechanisms used by the operating system to respond to external events, such as hardware device signals, timers, or software-triggered events.
  • 7. Fault masking  Fault masking is refers to a situation where a fault or error occurs within the system but is not immediately detected or reported because of the RTOS's ability to continue normal operation without apparent disruption.  This can give the illusion of a fault- tolerant system, but it may hide underlying issues that can lead to problems later on.
  • 8. Fault tolerant architectures  Fault tolerant architectures in real time operating systems are design strategies and mechanisms implemented to ensure that an embedded system can continue to operate correctly and reliably in the presence of hardware or software faults, errors, or failures.  These architectures are crucial for systems where reliability and availability are paramount, such as safety critical systems, aerospace applications, medical devices, and industrial control systems.
  • 9. Hardware Redundancy  This approach involves duplicating critical hardware components (e.g., CPUs, memory, power supplies) to create a redundant system.
  • 10. Software Redundancy  Redundancy can also be applied at the software level, where multiple copies of the same application or task run concurrently on separate processors.  In the event of a fault in one instance, another can take over.