The document discusses validation of embedded control systems through hardware-in-the-loop testing. It notes that embedded control systems are becoming increasingly complex due to growing software and hardware capabilities. This complexity poses risks that must be addressed through systematic testing of functionality, performance, and failure handling. The document outlines hardware-in-the-loop testing as an important method for validating embedded control system software.
Complying with New Functional Safety StandardsDesign World
The document is a presentation on complying with new functional safety standards. It discusses what functional safety is, what is happening in the functional safety market, what standards should be used for machines, and how to determine safety levels and perform calculations according to standards like ISO 13849-1 and IEC 62061. It provides an example of applying the standards to a dual channel emergency stop application and calculating the resulting safety integrity level.
The document discusses a new fieldbus barrier product from MTA called the 9370-FB Series Fieldbus Barrier. It establishes some key benefits over existing fieldbus barrier implementations, including lower cost, safer operation, and higher reliability over the lifecycle of a fieldbus network. Some key features of the 9370-FB Series mentioned are that it allows for live pluggable modules, pluggable trunk and spur surge protectors, and screw-secured pluggable spur terminals. Overall, the new barrier aims to provide value to plant operators and those involved in the design and installation of fieldbus networks in hazardous areas.
This document discusses simulation-based software development for time-triggered communication systems like FlexRay, which are commonly used in automotive applications. It introduces an approach using the SIDERA simulation system to develop and test application software on simulated communication controllers. This allows accelerating the software development process by eliminating delays from compiling and loading code onto hardware and easing debugging in distributed real-time systems. The goal is to enable executing host applications on simulated FlexRay controllers without requiring actual hardware or modifying the original code.
The document discusses burner management systems (BMS) and how programmable electronic systems (PES) can be used for burner control while ensuring safety. It outlines several key requirements for PES-based BMS to be certified, including using redundant safety-related PES, obtaining independent safety certification, and the designer demonstrating proper development and testing practices. The document also describes various safety features that can be designed into BMS, such as input/output monitoring, guarded outputs, processor watchdog timers, and power monitoring. It discusses architectures for safety programmable logic controllers (PLCs) including 1oo1D (one out of one with diagnostics) and 1oo2D (one out of two with diagnostics).
Testability refers to the design parameter which makes it relatively easy to identify and isolate faults in the system. Testability can be considered to be a subset of maintainability, because fault detection and isolation are important drivers in the maintainability of a system
This document discusses employee and network monitoring software. It outlines the team members developing the software. It then discusses what employee monitoring is, the existing system's disadvantages, and why monitoring is important. The software allows remote access and administration of employee computers from the server. It can capture screenshots, control computers remotely, send messages, end tasks, and monitor internet access. The monitoring software has advantages like operating stealthily, being easy to use, having low resource usage, and flexibility.
Safety Verification and Software aspects of Automotive SoCPankaj Singh
IP-SoC Conference 2017 Grenoble
Automotive industry has evolved over last 100 years. Electronic systems were
introduced into the automotive industry in 1960. Since then the complexity has grown
many fold and today’s automobiles have as many as 150 programmable computing
elements or Electronic Control Units(ECUs) with several wiring connections.
The software content has also increased significantly with today’s car having more than
100 million of lines of software code.
This increased hardware and software complexity increases the risk of failure that could
impact negatively on vehicle safety. This has led to concerns regarding the validation of
failure modes and the detection mechanisms. Car maker and suppliers need to prove
that, despite increasing complexity, their electronic systems will deliver the required
functionality safely and reliably.
This presentation describes the challenges and methodology related to Safety
verification and Software development aspects of Automotive Microcontroller SoC.
This document discusses principles of software safety for clinical information systems and electronic medical records (EMRs). It provides background on software safety incidents in other industries. Key concepts discussed include adjusting the software development methodology based on risk level, and that no software is completely safe. The document advocates analyzing EMR software to understand how defects could contribute to patient safety risk scenarios from minor to catastrophic. It suggests increased rigor for software that controls computerized protocols, clinical data posting and updating, and overall EMR performance and availability.
Complying with New Functional Safety StandardsDesign World
The document is a presentation on complying with new functional safety standards. It discusses what functional safety is, what is happening in the functional safety market, what standards should be used for machines, and how to determine safety levels and perform calculations according to standards like ISO 13849-1 and IEC 62061. It provides an example of applying the standards to a dual channel emergency stop application and calculating the resulting safety integrity level.
The document discusses a new fieldbus barrier product from MTA called the 9370-FB Series Fieldbus Barrier. It establishes some key benefits over existing fieldbus barrier implementations, including lower cost, safer operation, and higher reliability over the lifecycle of a fieldbus network. Some key features of the 9370-FB Series mentioned are that it allows for live pluggable modules, pluggable trunk and spur surge protectors, and screw-secured pluggable spur terminals. Overall, the new barrier aims to provide value to plant operators and those involved in the design and installation of fieldbus networks in hazardous areas.
This document discusses simulation-based software development for time-triggered communication systems like FlexRay, which are commonly used in automotive applications. It introduces an approach using the SIDERA simulation system to develop and test application software on simulated communication controllers. This allows accelerating the software development process by eliminating delays from compiling and loading code onto hardware and easing debugging in distributed real-time systems. The goal is to enable executing host applications on simulated FlexRay controllers without requiring actual hardware or modifying the original code.
The document discusses burner management systems (BMS) and how programmable electronic systems (PES) can be used for burner control while ensuring safety. It outlines several key requirements for PES-based BMS to be certified, including using redundant safety-related PES, obtaining independent safety certification, and the designer demonstrating proper development and testing practices. The document also describes various safety features that can be designed into BMS, such as input/output monitoring, guarded outputs, processor watchdog timers, and power monitoring. It discusses architectures for safety programmable logic controllers (PLCs) including 1oo1D (one out of one with diagnostics) and 1oo2D (one out of two with diagnostics).
Testability refers to the design parameter which makes it relatively easy to identify and isolate faults in the system. Testability can be considered to be a subset of maintainability, because fault detection and isolation are important drivers in the maintainability of a system
This document discusses employee and network monitoring software. It outlines the team members developing the software. It then discusses what employee monitoring is, the existing system's disadvantages, and why monitoring is important. The software allows remote access and administration of employee computers from the server. It can capture screenshots, control computers remotely, send messages, end tasks, and monitor internet access. The monitoring software has advantages like operating stealthily, being easy to use, having low resource usage, and flexibility.
Safety Verification and Software aspects of Automotive SoCPankaj Singh
IP-SoC Conference 2017 Grenoble
Automotive industry has evolved over last 100 years. Electronic systems were
introduced into the automotive industry in 1960. Since then the complexity has grown
many fold and today’s automobiles have as many as 150 programmable computing
elements or Electronic Control Units(ECUs) with several wiring connections.
The software content has also increased significantly with today’s car having more than
100 million of lines of software code.
This increased hardware and software complexity increases the risk of failure that could
impact negatively on vehicle safety. This has led to concerns regarding the validation of
failure modes and the detection mechanisms. Car maker and suppliers need to prove
that, despite increasing complexity, their electronic systems will deliver the required
functionality safely and reliably.
This presentation describes the challenges and methodology related to Safety
verification and Software development aspects of Automotive Microcontroller SoC.
This document discusses principles of software safety for clinical information systems and electronic medical records (EMRs). It provides background on software safety incidents in other industries. Key concepts discussed include adjusting the software development methodology based on risk level, and that no software is completely safe. The document advocates analyzing EMR software to understand how defects could contribute to patient safety risk scenarios from minor to catastrophic. It suggests increased rigor for software that controls computerized protocols, clinical data posting and updating, and overall EMR performance and availability.
Foundation level testing Concepts,Non function testing ,Non-Functional testing ,Selenium Tool,
What is Software Testing Software Testing is an activity in software development.
It is an investigation performed against a software to provide information about the quality of the software to stakeholders.
Software testing is associated with the two terms.
Validation: Are we doing the right job?
Verification: Are we doing the job right?
Case study "Virtual Show Room" – VSR,water fall model,General Principles of Testing,
The General V-Model
Unit Testing
Component Testing
Integration Testing
System Testing
Acceptance Testing
1. The document discusses various types of software testing including unit testing, integration testing, system testing, and acceptance testing. It explains that unit testing focuses on individual program units in isolation while integration testing tests modules assembled into subsystems.
2. The document then provides examples of different integration testing strategies like incremental, bottom-up, top-down, and discusses regression testing. It also defines smoke testing and explains its purpose in integration, system and acceptance testing levels.
3. Finally, the document emphasizes the importance of system and acceptance testing to verify functional and non-functional requirements and ensure the system can operate as intended in a real environment.
This document provides an overview of software quality assurance. It defines SQA as procedures, techniques, and tools to ensure a product meets pre-specified standards during development. SQA is an umbrella activity that includes standards and procedures, metrics and measurement, configuration management, testing, and reviews. The document discusses principles of SQA and emphasizes that quality must be planned, measured, tracked, and used to improve future products.
This document provides an overview of different types of software testing, including functional testing and non-functional testing. It describes various testing techniques such as white-box testing, black-box testing, unit testing, integration testing, regression testing, and performance testing. It explains the steps and goals of software testing processes like verification, validation, and different testing phases. Key aspects like test planning, test case design, test execution and reporting are also summarized.
FCT is used to detect failed components on finished PCBs before they are shipped. The functional test verifies PCB’s functionality to emphasize its function, development, and procedure to distinguish from PCB to PCB and system to system. Depending on the PCB, we have a wide range of test probes, for instance, multi-purpose probes, interface probes, fixture customizing probe, high-current probe, and a lot more. For more information visit https://bit.ly/3RfmPhU
This document discusses testing and evaluating software design and development. It covers various types of testing including unit testing, program testing, system testing, live testing, and quality assurance. Documentation and communication are important parts of reporting the testing process. The goal of testing is to ensure the software meets specifications, finds errors, and prevents problems.
Software QA Fundamentals by Prabhath DarshanaShamain Peiris
The document provides an overview of software quality assurance (QA) fundamentals. It defines software QA as a process to ensure software matches expected results in terms of correctness, completeness, and quality. It discusses why QA is needed to ensure reliability, usability, and that software meets requirements and works as intended. The document also describes different types of testing methods like black box, white box, and grey box testing and explains unit testing, integration testing, system testing, and other testing types part of the software development life cycle.
System testing validates the fully integrated software product by evaluating end-to-end system specifications. The software is developed in units and interfaced with other software and hardware to create a complete computer system. Regression testing confirms that new modifications have not caused new bugs and that old bugs have not reappeared from added modules over time.
This document discusses verification of redundant safety system architectures for embedded systems. It presents background on using redundant systems for safety and outlines experiments testing redundant software systems using fault injection. The experiments showed redundant systems improved safety in a state transition system for a pot and in a robot obstacle detection system, with the robot system not exhibiting unsafe behavior in 136 test cases. The paper concludes redundant software systems can effectively improve safety, and combining hardware and software redundancy provides even higher reliability. Future work could include testing more fault cases and designing watchdogs to detect and report unsafe conditions.
This document discusses verification of redundant safety system architectures for embedded systems. It presents background on using redundant systems for safety and outlines experiments testing redundant software systems using fault injection. The experiments showed redundant systems improved safety in a state transition system for a pot and in a robot obstacle detection system, with the robot system not exhibiting any unsafe behavior across 136 test cases. The paper concludes redundant software systems can effectively improve safety, and combining hardware and software redundancy provides even higher reliability. Future work could include testing more fault cases and designing watchdogs to detect and respond to unsafe conditions.
System testing is conducted to detect errors before the system is handed over to users for acceptance testing. It examines system performance and whether it meets acceptance standards. System testing checks turnaround time, backups, file protection, and human factors like workspace ergonomics. There are different types of testing like unit testing of individual programs, integration testing of program interactions, validation testing against requirements, and system testing of the fully integrated system. The goal is to identify errors at each stage of development to deliver a robust and user-ready system.
The document discusses software validation techniques. It begins with an overview of reliability, failures, faults, and approaches to fault avoidance and tolerance. It then covers verification and validation methods, including static techniques like inspections and dynamic techniques like testing. It details the testing process from unit to system testing and describes regression testing. It also explains black box and white box testing strategies and coverage criteria for generating test cases.
This document discusses a webinar on risk-based software verification and compliance with ISO14971 and IEC62304 regulations. The webinar will cover identifying residual risk, risk-based verification techniques, objective-based verification planning, documenting verification activities, and practical tips for verifying requirements. Attendees will learn how to define residual risk, streamline verification using a risk-based approach, and ensure compliance with relevant regulations.
Using Syncade Workflow and AMS Device Manager for SIF Proof Testing on a Delt...Emerson Exchange
The document discusses how Syncade Workflow and AMS Device Manager can be used for SIS proof testing on a DeltaV SMART SIS system. It describes how SMART instruments and logic solvers enable automated testing and documentation to satisfy IEC 61511 standards. Syncade workflow guides technicians through tests, documents results electronically, and ensures tasks are done correctly. This facilitates faster commissioning and proof testing while reducing costs.
The document introduces the SafeEx software solution for more efficient inspections and maintenance. SafeEx uses RFID technology and handheld devices to document inspections electronically, creating an audit trail and proving inspectors' presence on site. It saves at least 30% of man-hours through automated processes and uploads. SafeEx provides full transparency and traceability of inspection and maintenance data for both offshore and onshore management.
Multipoint Conferencing Unit Comparative StudyVideoguy
The Polycom MCU outperformed the other MCUs in the comparative study. It passed 63 out of 63 test cases, while the RADVISION viaIP 400 MCU passed 14 cases, the TANDBERG MCU 16+16 passed 5 cases, and the TANDBERG MPS passed 7 cases. The Polycom MCU demonstrated the most complete set of security, versatility, and administration features. It provided the largest and most flexible set of options to successfully complete three example use cases.
The document discusses software fault tolerance techniques. It begins by explaining why fault tolerant software is needed, particularly for safety critical systems. It then covers single version techniques like checkpointing and process pairs. Next it discusses multi-version techniques using multiple variants of software. It also covers software fault injection testing. Finally it provides examples of fault tolerant systems used in aircraft like the Airbus and Boeing.
Different Software Testing Types and CMM StandardDhrumil Panchal
This document discusses software engineering concepts including the CMM standard and different types of testing. It defines the five levels of the CMM standard for process maturity. It also describes various types of testing such as unit testing, integration testing, validation testing, system testing, and acceptance testing. For each type of testing it provides details about the goals, steps, and techniques involved.
Foundation level testing Concepts,Non function testing ,Non-Functional testing ,Selenium Tool,
What is Software Testing Software Testing is an activity in software development.
It is an investigation performed against a software to provide information about the quality of the software to stakeholders.
Software testing is associated with the two terms.
Validation: Are we doing the right job?
Verification: Are we doing the job right?
Case study "Virtual Show Room" – VSR,water fall model,General Principles of Testing,
The General V-Model
Unit Testing
Component Testing
Integration Testing
System Testing
Acceptance Testing
1. The document discusses various types of software testing including unit testing, integration testing, system testing, and acceptance testing. It explains that unit testing focuses on individual program units in isolation while integration testing tests modules assembled into subsystems.
2. The document then provides examples of different integration testing strategies like incremental, bottom-up, top-down, and discusses regression testing. It also defines smoke testing and explains its purpose in integration, system and acceptance testing levels.
3. Finally, the document emphasizes the importance of system and acceptance testing to verify functional and non-functional requirements and ensure the system can operate as intended in a real environment.
This document provides an overview of software quality assurance. It defines SQA as procedures, techniques, and tools to ensure a product meets pre-specified standards during development. SQA is an umbrella activity that includes standards and procedures, metrics and measurement, configuration management, testing, and reviews. The document discusses principles of SQA and emphasizes that quality must be planned, measured, tracked, and used to improve future products.
This document provides an overview of different types of software testing, including functional testing and non-functional testing. It describes various testing techniques such as white-box testing, black-box testing, unit testing, integration testing, regression testing, and performance testing. It explains the steps and goals of software testing processes like verification, validation, and different testing phases. Key aspects like test planning, test case design, test execution and reporting are also summarized.
FCT is used to detect failed components on finished PCBs before they are shipped. The functional test verifies PCB’s functionality to emphasize its function, development, and procedure to distinguish from PCB to PCB and system to system. Depending on the PCB, we have a wide range of test probes, for instance, multi-purpose probes, interface probes, fixture customizing probe, high-current probe, and a lot more. For more information visit https://bit.ly/3RfmPhU
This document discusses testing and evaluating software design and development. It covers various types of testing including unit testing, program testing, system testing, live testing, and quality assurance. Documentation and communication are important parts of reporting the testing process. The goal of testing is to ensure the software meets specifications, finds errors, and prevents problems.
Software QA Fundamentals by Prabhath DarshanaShamain Peiris
The document provides an overview of software quality assurance (QA) fundamentals. It defines software QA as a process to ensure software matches expected results in terms of correctness, completeness, and quality. It discusses why QA is needed to ensure reliability, usability, and that software meets requirements and works as intended. The document also describes different types of testing methods like black box, white box, and grey box testing and explains unit testing, integration testing, system testing, and other testing types part of the software development life cycle.
System testing validates the fully integrated software product by evaluating end-to-end system specifications. The software is developed in units and interfaced with other software and hardware to create a complete computer system. Regression testing confirms that new modifications have not caused new bugs and that old bugs have not reappeared from added modules over time.
This document discusses verification of redundant safety system architectures for embedded systems. It presents background on using redundant systems for safety and outlines experiments testing redundant software systems using fault injection. The experiments showed redundant systems improved safety in a state transition system for a pot and in a robot obstacle detection system, with the robot system not exhibiting unsafe behavior in 136 test cases. The paper concludes redundant software systems can effectively improve safety, and combining hardware and software redundancy provides even higher reliability. Future work could include testing more fault cases and designing watchdogs to detect and report unsafe conditions.
This document discusses verification of redundant safety system architectures for embedded systems. It presents background on using redundant systems for safety and outlines experiments testing redundant software systems using fault injection. The experiments showed redundant systems improved safety in a state transition system for a pot and in a robot obstacle detection system, with the robot system not exhibiting any unsafe behavior across 136 test cases. The paper concludes redundant software systems can effectively improve safety, and combining hardware and software redundancy provides even higher reliability. Future work could include testing more fault cases and designing watchdogs to detect and respond to unsafe conditions.
System testing is conducted to detect errors before the system is handed over to users for acceptance testing. It examines system performance and whether it meets acceptance standards. System testing checks turnaround time, backups, file protection, and human factors like workspace ergonomics. There are different types of testing like unit testing of individual programs, integration testing of program interactions, validation testing against requirements, and system testing of the fully integrated system. The goal is to identify errors at each stage of development to deliver a robust and user-ready system.
The document discusses software validation techniques. It begins with an overview of reliability, failures, faults, and approaches to fault avoidance and tolerance. It then covers verification and validation methods, including static techniques like inspections and dynamic techniques like testing. It details the testing process from unit to system testing and describes regression testing. It also explains black box and white box testing strategies and coverage criteria for generating test cases.
This document discusses a webinar on risk-based software verification and compliance with ISO14971 and IEC62304 regulations. The webinar will cover identifying residual risk, risk-based verification techniques, objective-based verification planning, documenting verification activities, and practical tips for verifying requirements. Attendees will learn how to define residual risk, streamline verification using a risk-based approach, and ensure compliance with relevant regulations.
Using Syncade Workflow and AMS Device Manager for SIF Proof Testing on a Delt...Emerson Exchange
The document discusses how Syncade Workflow and AMS Device Manager can be used for SIS proof testing on a DeltaV SMART SIS system. It describes how SMART instruments and logic solvers enable automated testing and documentation to satisfy IEC 61511 standards. Syncade workflow guides technicians through tests, documents results electronically, and ensures tasks are done correctly. This facilitates faster commissioning and proof testing while reducing costs.
The document introduces the SafeEx software solution for more efficient inspections and maintenance. SafeEx uses RFID technology and handheld devices to document inspections electronically, creating an audit trail and proving inspectors' presence on site. It saves at least 30% of man-hours through automated processes and uploads. SafeEx provides full transparency and traceability of inspection and maintenance data for both offshore and onshore management.
Multipoint Conferencing Unit Comparative StudyVideoguy
The Polycom MCU outperformed the other MCUs in the comparative study. It passed 63 out of 63 test cases, while the RADVISION viaIP 400 MCU passed 14 cases, the TANDBERG MCU 16+16 passed 5 cases, and the TANDBERG MPS passed 7 cases. The Polycom MCU demonstrated the most complete set of security, versatility, and administration features. It provided the largest and most flexible set of options to successfully complete three example use cases.
The document discusses software fault tolerance techniques. It begins by explaining why fault tolerant software is needed, particularly for safety critical systems. It then covers single version techniques like checkpointing and process pairs. Next it discusses multi-version techniques using multiple variants of software. It also covers software fault injection testing. Finally it provides examples of fault tolerant systems used in aircraft like the Airbus and Boeing.
Different Software Testing Types and CMM StandardDhrumil Panchal
This document discusses software engineering concepts including the CMM standard and different types of testing. It defines the five levels of the CMM standard for process maturity. It also describes various types of testing such as unit testing, integration testing, validation testing, system testing, and acceptance testing. For each type of testing it provides details about the goals, steps, and techniques involved.
1. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Methods for Hardware-in-the-loop testing for UAV
Ehsan Peymani
Norwegian University of Science and Technology
1 August, 2013 Trondheim, Norway
Ehsan Peymani Validation of Embedded Control Systems
2. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Main Statement
Systematic testing of functionality, performance, and failure
handling of embedded control systems is increasingly important
due to the growing complexity of the software and hardware of
systems.
Ehsan Peymani Validation of Embedded Control Systems
3. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Main Statement
Systematic testing of functionality, performance, and failure
handling of embedded control systems is increasingly important
due to the growing complexity of the software and hardware of
systems.
Ehsan Peymani Validation of Embedded Control Systems
4. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Main Statement
Systematic testing of functionality, performance, and failure
handling of embedded control systems is increasingly important
due to the growing complexity of the software and hardware of
systems.
Ehsan Peymani Validation of Embedded Control Systems
5. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Main Statement
Systematic testing of functionality, performance, and failure
handling of embedded control systems is increasingly important
due to the growing complexity of the software and hardware of
systems.
Ehsan Peymani Validation of Embedded Control Systems
6. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Main Statement
Systematic testing of functionality, performance, and failure
handling of embedded control systems is increasingly important
due to the growing complexity of the software and hardware of
systems.
Ehsan Peymani Validation of Embedded Control Systems
7. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Main Statement
Systematic testing of functionality, performance, and failure
handling of embedded control systems is increasingly important
due to the growing complexity of the software and hardware of
systems.
Ehsan Peymani Validation of Embedded Control Systems
8. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Main Statement
Systematic testing of functionality, performance, and failure
handling of embedded control systems is increasingly important
due to the growing complexity of the software and hardware of
systems.
Ehsan Peymani Validation of Embedded Control Systems
9. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Outline
1 Embedded Control Systems
2 Validation of Control Software
3 Hardware-in-the-loop testing
4 Closing Remarks
Ehsan Peymani Validation of Embedded Control Systems
10. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Outline
1 Embedded Control Systems
Definitions
Complexity
2 Validation of Control Software
3 Hardware-in-the-loop testing
4 Closing Remarks
Ehsan Peymani Validation of Embedded Control Systems
11. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Control Systems
Control System: all systems and components, hardware, software,
and user interfaces, necessary to perform the required
control action
Actuator systems
Sensor systems
Power systems
Control computer system
processor, I/O interface, user interface, operator stations, network
& communication (cabling, ...), and SW/HW platforms
Ehsan Peymani Validation of Embedded Control Systems
12. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Control Systems
Control System: all systems and components, hardware, software,
and user interfaces, necessary to perform the required
control action
Actuator systems
Sensor systems
Power systems
Control computer system
processor, I/O interface, user interface, operator stations, network
& communication (cabling, ...), and SW/HW platforms
Ehsan Peymani Validation of Embedded Control Systems
13. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems
Embedded System: a microprocessor used as a component in
another piece of technology
According to a report (2000),
150 million microprocessors used in traditional computers
8 billion microprocessors used in embedded applications
General Types of Embedded Systems
1 General (PDA’s, portable games)
2 Communications (cell phones)
3 Signal processing (video and audio)
4 Control
real time feedback control
automotive
aerospace
appliances
Ehsan Peymani Validation of Embedded Control Systems
14. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems
Embedded System: a microprocessor used as a component in
another piece of technology
According to a report (2000),
150 million microprocessors used in traditional computers
8 billion microprocessors used in embedded applications
General Types of Embedded Systems
1 General (PDA’s, portable games)
2 Communications (cell phones)
3 Signal processing (video and audio)
4 Control
real time feedback control
automotive
aerospace
appliances
Ehsan Peymani Validation of Embedded Control Systems
15. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems
Embedded System: a microprocessor used as a component in
another piece of technology
According to a report (2000),
150 million microprocessors used in traditional computers
8 billion microprocessors used in embedded applications
General Types of Embedded Systems
1 General (PDA’s, portable games)
2 Communications (cell phones)
3 Signal processing (video and audio)
4 Control
real time feedback control
automotive
aerospace
appliances
Ehsan Peymani Validation of Embedded Control Systems
16. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems
Embedded System: a microprocessor used as a component in
another piece of technology
According to a report (2000),
150 million microprocessors used in traditional computers
8 billion microprocessors used in embedded applications
General Types of Embedded Systems
1 General (PDA’s, portable games)
2 Communications (cell phones)
3 Signal processing (video and audio)
4 Control
real time feedback control
automotive
aerospace
appliances
Ehsan Peymani Validation of Embedded Control Systems
17. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Embedded control systems are ubiquitous.
The complexity of embedded control systems is amazingly
growing.
Complexity: how hard something is to understand or verify.
The main consequence of complexity is risk.
Humans role: understandability can be enhanced through
education and training;
thus, the level of complexity can be reduced.
Ehsan Peymani Validation of Embedded Control Systems
18. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Embedded control systems are ubiquitous.
The complexity of embedded control systems is amazingly
growing.
Complexity: how hard something is to understand or verify.
The main consequence of complexity is risk.
Humans role: understandability can be enhanced through
education and training;
thus, the level of complexity can be reduced.
Ehsan Peymani Validation of Embedded Control Systems
19. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Embedded control systems are ubiquitous.
The complexity of embedded control systems is amazingly
growing.
Complexity: how hard something is to understand or verify.
The main consequence of complexity is risk.
Humans role: understandability can be enhanced through
education and training;
thus, the level of complexity can be reduced.
Ehsan Peymani Validation of Embedded Control Systems
20. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Embedded control systems are ubiquitous.
The complexity of embedded control systems is amazingly
growing.
Complexity: how hard something is to understand or verify.
The main consequence of complexity is risk.
Humans role: understandability can be enhanced through
education and training;
thus, the level of complexity can be reduced.
Ehsan Peymani Validation of Embedded Control Systems
21. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Embedded control systems are ubiquitous.
The complexity of embedded control systems is amazingly
growing.
Complexity: how hard something is to understand or verify.
The main consequence of complexity is risk.
Humans role: understandability can be enhanced through
education and training;
thus, the level of complexity can be reduced.
Ehsan Peymani Validation of Embedded Control Systems
22. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Complexity means how hard something is to understand or verify. The
main consequence of complexity is risk.
Types of complexity:
essential functionality which comes from vetted requirements and
is unavoidable.
incidental complexity which arises from decisions about
architecture, design, and implementation.
This part can be reduced by making wise
decisions.
Ehsan Peymani Validation of Embedded Control Systems
23. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Complexity means how hard something is to understand or verify. The
main consequence of complexity is risk.
Source of [unnecessary] complexity:
1 requirements and design (advanced control algorithms)
2 software development
3 testing and operations
Requirements
Analysis &
Design
Software Testing Operations
N
O
W
!
nge
ack.c
o
m
w
w
Figure: The scope of this talk is on software complexity included both
upstream and downstream activities in the engineering lifecycle.
Ehsan Peymani Validation of Embedded Control Systems
24. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Complexity means how hard something is to understand or verify. The
main consequence of complexity is risk.
Source of [unnecessary] complexity:
1 requirements and design (advanced control algorithms)
2 software development
3 testing and operations
Requirements
Analysis &
Design
Software Testing Operations
N
O
W
!
nge
ack.c
o
m
w
w
Figure: The scope of this talk is on software complexity included both
upstream and downstream activities in the engineering lifecycle.
Ehsan Peymani Validation of Embedded Control Systems
25. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Complexity
Complexity means how hard something is to understand or verify. The
main consequence of complexity is risk.
Source of [unnecessary] complexity:
1 requirements and design (advanced control algorithms)
2 software development
3 testing and operations
Requirements
Analysis &
Design
Software Testing Operations
N
O
W
!
nge
ack.c
o
m
w
w
Figure: The scope of this talk is on software complexity included both
upstream and downstream activities in the engineering lifecycle.
Ehsan Peymani Validation of Embedded Control Systems
26. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
NASA report on Flight Software Complexity (2009)
Flight software executes onboard a spacecraft/aircraft.
◦ mission-level capabilities:
◦ platform/framework/infrastructure
Ehsan Peymani Validation of Embedded Control Systems
27. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
NASA report on Flight Software Complexity (2009)
Flight software executes onboard a spacecraft/aircraft.
◦ mission-level capabilities:
guidance, navigation, and control
entry, descent, and landing
surface mobility
environmental control
instrument control
communications
◦ platform/framework/infrastructure
Ehsan Peymani Validation of Embedded Control Systems
28. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
NASA report on Flight Software Complexity (2009)
Flight software executes onboard a spacecraft/aircraft.
◦ mission-level capabilities:
◦ platform/framework/infrastructure
computer boot-up and initialization
time management
hardware interface control
command processing
telemetry processing
data storage management
flight software patch and load
fault protection
Ehsan Peymani Validation of Embedded Control Systems
29. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
NASA report on Flight Software Complexity (2009)
Exponential growth rate of a factor of 10 approximately every
10 years.
Flight Software Complexity
Growth in Code Size for Human and Robotic Missions
1
10
100
1000
10000
100000
1000000
10000000
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
Year of Mission
NCSL(logscale)
unmanned
manned
Expon. (unmanned)
Expon. (manned)
Non-CommentSourceLines(logscale)
1969 Mariner-6 (30)
1975 Viking (5K)
1977 Voyager (3K)
1989 Galileo (8K)
1990 Cassini (120K)
1997 Pathfinder (175K)
1999 DS1 (349K)
2003 SIRTF/Spitzer (554K)
2004 MER (555K)
2005 MRO (545K)
1968 Apollo (8.5K)
1980 Shuttle(470K)
1989 ISS (1.5M)
Robotic
Human
Figure 3. History of flight software growth in human and robotic missions.
In the realm of human spaceflight, there are only three NASA data points (Apollo, Space Shuttle,
Figure: Growth in flight software size for human missions (in red) and
robotic missions (in blue)
Ehsan Peymani Validation of Embedded Control Systems
30. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
NASA report on Flight Software Complexity (2009)
Exponential growth rate of a factor of 10 approximately every
10 years.
Flight Software Complexity
Growth in Code Size for Human and Robotic Missions
1
10
100
1000
10000
100000
1000000
10000000
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
Year of Mission
NCSL(logscale)
unmanned
manned
Expon. (unmanned)
Expon. (manned)
Non-CommentSourceLines(logscale)
1969 Mariner-6 (30)
1975 Viking (5K)
1977 Voyager (3K)
1989 Galileo (8K)
1990 Cassini (120K)
1997 Pathfinder (175K)
1999 DS1 (349K)
2003 SIRTF/Spitzer (554K)
2004 MER (555K)
2005 MRO (545K)
1968 Apollo (8.5K)
1980 Shuttle(470K)
1989 ISS (1.5M)
Robotic
Human
Figure 3. History of flight software growth in human and robotic missions.
In the realm of human spaceflight, there are only three NASA data points (Apollo, Space Shuttle,
Figure: Growth in flight software size for human missions (in red) and
robotic missions (in blue)
Ehsan Peymani Validation of Embedded Control Systems
31. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
NASA report on Flight Software Complexity (2009)
The F-22A is reported to have
2.5 million lines of code.
The F-35 Joint Strike Fighter,
will, according to one source,
have 5.7 M lines of code.
Spacecraft, aircraft, and automobiles are all examples of embedded real-time
and all have witnessed major growth in software size and complexity over the
The closely related aeronautics industry has reported exponential growth in bo
signals to be processed and source lines of code (SLOC) required for civil air
year period [Gillette 1998]. In a period of forty years, the percent of function
pilots of military aircraft has risen from 8% in 1960 (F-4 Phantom) to 80
Raptor), as shown in Figure 7. The F-22A is reported to have 2.5 million lines o
Software in Military Aircraft
0
10
20
30
40
50
60
70
80
90
1960
(F-4)
1964
(A-7)
1970
(F-
111)
1975
(F-15)
1982
(F-16)
1990
(B-2)
2000
(F-22)
Year of Introduction
PercentofFunctionalityProvided
bySoftware
Figure 7. Growth in functionality provided by software to pilots of military aircr
The reasons for such growth are easy to understand. The exponentially increa
microprocessors (Moore’s Law) along with their decreasing mass and power r
made them the tool of choice for adding functionality in the most cost-effective
An exponential growth in both the number of signals to be
processed and source lines of code (SLOC) required for civil
aircraft is observed over a thirty year period.
Ehsan Peymani Validation of Embedded Control Systems
32. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
NASA report on Flight Software Complexity (2009)
The F-22A is reported to have
2.5 million lines of code.
The F-35 Joint Strike Fighter,
will, according to one source,
have 5.7 M lines of code.
Spacecraft, aircraft, and automobiles are all examples of embedded real-time
and all have witnessed major growth in software size and complexity over the
The closely related aeronautics industry has reported exponential growth in bo
signals to be processed and source lines of code (SLOC) required for civil air
year period [Gillette 1998]. In a period of forty years, the percent of function
pilots of military aircraft has risen from 8% in 1960 (F-4 Phantom) to 80
Raptor), as shown in Figure 7. The F-22A is reported to have 2.5 million lines o
Software in Military Aircraft
0
10
20
30
40
50
60
70
80
90
1960
(F-4)
1964
(A-7)
1970
(F-
111)
1975
(F-15)
1982
(F-16)
1990
(B-2)
2000
(F-22)
Year of Introduction
PercentofFunctionalityProvided
bySoftware
Figure 7. Growth in functionality provided by software to pilots of military aircr
The reasons for such growth are easy to understand. The exponentially increa
microprocessors (Moore’s Law) along with their decreasing mass and power r
made them the tool of choice for adding functionality in the most cost-effective
An exponential growth in both the number of signals to be
processed and source lines of code (SLOC) required for civil
aircraft is observed over a thirty year period.
Ehsan Peymani Validation of Embedded Control Systems
33. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
NASA report on Flight Software Complexity (2009)
In 1990 the typical GM car had one million lines of code,
growing to 100M by 2010.
Flight Software Complexity
shows, the average General Motors automobile had a million lines of code in 1
projected to have 100 million lines of code by the year 2010; while all of the code is
to the car’s operation, it is indicative of the continuing trend with embedded systems.
Lines of Code in Typical GM Car
1
10
100
1000
10000
100000
1970 1990 2010
Model Year
KLOC
Figure 8. In 1990 the typical GM car had one million lines of code, growing to 100M by 201
Ehsan Peymani Validation of Embedded Control Systems
34. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definitions
Complexity
Embedded Control Systems: Software Complexity
National Institute of Standards and Technology (NIST) Report (2002)
Software bugs cost the U.S. economy $59.5 billion annually.
More than a third of this cost could be avoided if a better
testing was performed.
◦ The earlier a defect/error/bug is found, the cheaper it is to fix.
Fixing a bug during operation may cost 20-100 times more than
cost to correct during the requirement definition phase.
Ehsan Peymani Validation of Embedded Control Systems
35. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Outline
1 Embedded Control Systems
2 Validation of Control Software
Definition and Objective
Methods of Testing
Development Stages
3 Hardware-in-the-loop testing
4 Closing Remarks
Ehsan Peymani Validation of Embedded Control Systems
36. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Software Testing
Software testing is an investigation to provide stakeholders with
information about the ::::::
quality of the product;
an independent view to appreciate and understand the ::::
risks of
software implementation.
According to Miller (1981),
“The general aim of testing is to affirm the
quality of software systems by
systematically exercising the software in
::::::::
carefully :::::::::
controlled::::::::::::::
circumstances.”
Dijkstras criticism: “Program testing can be used to show the
presence of bugs, but never to show their absence”.
Ehsan Peymani Validation of Embedded Control Systems
37. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Practical Issues
Testing intends to find errors.
a good test is one that has a high probability of finding an
undiscovered error.
a successful test is one that uncovers an undiscovered error.
How to test a product?
Testing under all combinations of inputs and initial states is
not feasible.
infrequent errors are difficult to find.
non-functional requirements (such as quality, performance,
scalability, usability, compatibility, reliability) are subjective.
How to validate control software?
Ehsan Peymani Validation of Embedded Control Systems
38. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Methods of Software Testing
Static testing vs. Dynamic testing
Functional testing vs. Structural testing
Ehsan Peymani Validation of Embedded Control Systems
39. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Static testing and Dynamic testing
Based on whether the actual execution of software is needed or
not, there are two major categories of quality assurance activities:
Static testing: (without reference to actual execution)
methods: code inspection, program analysis, symbolic analysis,
and model checking
verification: (to test compliance of the system to the
requirements)
Dynamic testing: (with actual execution)
methods: boundary value analysis, state transition tables,
decision table testing, model-based testing, specification-based
testing
validation: (additionally questions the correctness and
feasibility of requirements with respect to philosophy and
intended use, rules and regulations, · · · .
Ehsan Peymani Validation of Embedded Control Systems
40. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Static testing and Dynamic testing
Based on whether the actual execution of software is needed or
not, there are two major categories of quality assurance activities:
Static testing: (without reference to actual execution)
methods: code inspection, program analysis, symbolic analysis,
and model checking
verification: (to test compliance of the system to the
requirements)
Dynamic testing: (with actual execution)
methods: boundary value analysis, state transition tables,
decision table testing, model-based testing, specification-based
testing
validation: (additionally questions the correctness and
feasibility of requirements with respect to philosophy and
intended use, rules and regulations, · · · .
Ehsan Peymani Validation of Embedded Control Systems
41. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Static testing and Dynamic testing
Based on whether the actual execution of software is needed or
not, there are two major categories of quality assurance activities:
Static testing: (without reference to actual execution)
methods: code inspection, program analysis, symbolic analysis,
and model checking
verification: (to test compliance of the system to the
requirements)
Dynamic testing: (with actual execution)
methods: boundary value analysis, state transition tables,
decision table testing, model-based testing, specification-based
testing
validation: (additionally questions the correctness and
feasibility of requirements with respect to philosophy and
intended use, rules and regulations, · · · .
Ehsan Peymani Validation of Embedded Control Systems
42. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Functional testing and Structural testing
According to the point of view that the test engineer takes when
setting objectives for test cases, there are two major categories for
quality assurance activities:
Functional testing (to test the functionality exposed to the
end-user without any knowledge of internal implementation)
∗ emphasizes on the external behavior of the software entity.
∗ the system is viewed as a black box.
∗ the selection of test cases is based on the::::::::::
requirement or
:::::
design:::::::::::
specification.
Structural testing (to test internal structures or workings; the
aim is to cause execution of a specific spot)
∗ emphasizes on the internal structure of the software entity.
∗ the system is viewed as a white box.
∗ the selection of test cases is based on the:::::::::::::
implementation.
Ehsan Peymani Validation of Embedded Control Systems
43. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Functional testing and Structural testing
According to the point of view that the test engineer takes when
setting objectives for test cases, there are two major categories for
quality assurance activities:
Functional testing (to test the functionality exposed to the
end-user without any knowledge of internal implementation)
∗ emphasizes on the external behavior of the software entity.
∗ the system is viewed as a black box.
∗ the selection of test cases is based on the::::::::::
requirement or
:::::
design:::::::::::
specification.
Structural testing (to test internal structures or workings; the
aim is to cause execution of a specific spot)
∗ emphasizes on the internal structure of the software entity.
∗ the system is viewed as a white box.
∗ the selection of test cases is based on the:::::::::::::
implementation.
Ehsan Peymani Validation of Embedded Control Systems
44. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Functional testing and Structural testing
According to the point of view that the test engineer takes when
setting objectives for test cases, there are two major categories for
quality assurance activities:
Functional testing (to test the functionality exposed to the
end-user without any knowledge of internal implementation)
∗ emphasizes on the external behavior of the software entity.
∗ the system is viewed as a black box.
∗ the selection of test cases is based on the::::::::::
requirement or
:::::
design:::::::::::
specification.
Structural testing (to test internal structures or workings; the
aim is to cause execution of a specific spot)
∗ emphasizes on the internal structure of the software entity.
∗ the system is viewed as a white box.
∗ the selection of test cases is based on the:::::::::::::
implementation.
Ehsan Peymani Validation of Embedded Control Systems
45. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Functional testing and Structural testing
According to the point of view that the test engineer takes when
setting objectives for test cases, there are two major categories for
quality assurance activities:
Functional testing (to test the functionality exposed to the
end-user without any knowledge of internal implementation)
∗ emphasizes on the external behavior of the software entity.
∗ the system is viewed as a black box.
∗ the selection of test cases is based on the::::::::::
requirement or
:::::
design:::::::::::
specification.
Structural testing (to test internal structures or workings; the
aim is to cause execution of a specific spot)
∗ emphasizes on the internal structure of the software entity.
∗ the system is viewed as a white box.
∗ the selection of test cases is based on the:::::::::::::
implementation.
Ehsan Peymani Validation of Embedded Control Systems
46. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
V-model for control system development
R. Skjetne and O. Egeland246
Figure 5. V-model for control system development and installation
Ehsan Peymani Validation of Embedded Control Systems
47. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Typical Stages in Software Testing
as part of the static testing effort.9
Once the code ● Belief in the malleabili
Figure 3 Typical stages of testing within IBM
UNIT TEST/CODE
INSPECTION
FUNCTION/
COMPONENT
TEST
M = MODULE F = FUNCTION P = PRODUCT
MODULE
M2
M3 M1
F2
F1
SYSTEM/
INTEGRATION OR
PRODUCT TEST
F4
F3
Unit Testing Component Testing Software Integration
Module Testing Module Integration Testing Testing
Ehsan Peymani Validation of Embedded Control Systems
48. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Typical Stages in Software Development
Figure 2 The activities that involve debugging, testing, and verification in a typical software development process
DEBUGGING
VERIFICATION
Ehsan Peymani Validation of Embedded Control Systems
49. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Testing vs. Debugging
Testing Debugging
to discover a defect to find and remove the cause of the defect
carried out by testing team carried out by development team
to find as many defects as possible to remove those defects
Ehsan Peymani Validation of Embedded Control Systems
50. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Definition and Objective
Methods of Testing
Development Stages
Development and Validation Efficiency
1 Cost
2 Duration (time-to-market)
3 Safety
4 Feasibility
Ehsan Peymani Validation of Embedded Control Systems
51. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Outline
1 Embedded Control Systems
2 Validation of Control Software
3 Hardware-in-the-loop testing
Motivation and Concept
Components
Benefits
Recent Issues
4 Closing Remarks
Ehsan Peymani Validation of Embedded Control Systems
52. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Design
Mechanical
System
Actuators Sensors
Disturbances Noise
Control System implemented on software and hardware platforms
functions to achieve the intended purpose and characteristic
action;
safety-related functions
1 to ensure system integrity, failure detection, failure handling,
fail safe actions, ...
Ehsan Peymani Validation of Embedded Control Systems
53. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Design
Mechanical
System
Actuators Sensors
Disturbances Noise
Process
Variable
Controller
SW/HW
Control
Signal
Reference
Control System implemented on software and hardware platforms
functions to achieve the intended purpose and characteristic
action;
safety-related functions
1 to ensure system integrity, failure detection, failure handling,
fail safe actions, ...
Ehsan Peymani Validation of Embedded Control Systems
54. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Design
Mechanical
System
Actuators Sensors
Disturbances Noise
Process
Variable
Controller
SW/HW
Control
Signal
Reference
Control System implemented on software and hardware platforms
functions to achieve the intended purpose and characteristic
action;
safety-related functions
1 to ensure system integrity, failure detection, failure handling,
fail safe actions, ...
Ehsan Peymani Validation of Embedded Control Systems
55. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 1: System-level testing
Process
Variable
Mechanical
System
Actuators Sensors
Disturbances Noise
Controller
SW/HW
Control
Signal
Reference
N
O
W
!
ange
ack.c
o
m
Ehsan Peymani Validation of Embedded Control Systems
56. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 1: System-level testing
Process
Variable
Mechanical
System
Actuators Sensors
Disturbances Noise
Controller
SW/HW
Control
Signal
Reference
N
O
W
!
ange
ack.c
o
m
Ehsan Peymani Validation of Embedded Control Systems
57. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 1: System-level testing
Process
Variable
Mechanical
System
Actuators Sensors
Disturbances Noise
Controller
SW/HW
Control
Signal
Reference
Test Program
Test Profile Logging/Analysis
N
O
W
!
ange
ack.c
o
m
Ehsan Peymani Validation of Embedded Control Systems
58. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 1: System-level testing - Challenges
1 Cost to test
2 Cost of failure
3 Availability
4 System variations
5 Repeatability
Ehsan Peymani Validation of Embedded Control Systems
59. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 1: System-level testing - Challenges
1 Cost to test
2 Cost of failure
3 Availability
4 System variations
5 Repeatability
Ehsan Peymani Validation of Embedded Control Systems
60. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 1: System-level testing - Challenges
1 Cost to test
2 Cost of failure
3 Availability
4 System variations
5 Repeatability
Ehsan Peymani Validation of Embedded Control Systems
61. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 1: System-level testing - Challenges
1 Cost to test
2 Cost of failure
3 Availability
4 System variations
5 Repeatability
Ehsan Peymani Validation of Embedded Control Systems
62. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 1: System-level testing - Challenges
1 Cost to test
2 Cost of failure
3 Availability
4 System variations
5 Repeatability
Ehsan Peymani Validation of Embedded Control Systems
63. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 2: Component-level testing
Process
Variable
Mechanical
System
Actuators Sensors
Disturbances Noise
Controller
SW/HW
Control
Signal
Reference
System
c
o
m
Ehsan Peymani Validation of Embedded Control Systems
64. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 2: Component-level testing
Simulated Process Variable
Controller
SW/HW
Control SignalReference
Test Program
Test Profile Logging/Analysis
c
o
m
Ehsan Peymani Validation of Embedded Control Systems
65. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 2: Component-level testing - Challenges
Simulated Process Variable
Controller
SW/HW
Control SignalReference
Test Program
Test Profile Logging/Analysis
c
o
m
Ehsan Peymani Validation of Embedded Control Systems
66. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 2: Component-level testing - Challenges
Simulated Process Variable
Controller
SW/HW
Control SignalReference
Test Program
Test Profile Logging/Analysis
c
o
m
Ehsan Peymani Validation of Embedded Control Systems
67. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Control System Validation
Approach 2: Component-level testing - Challenges
Simulated Process Variable
Controller
SW/HW
Control SignalReference
Test Program
Test Profile Logging/Analysis
Disturbance
c
o
m
Ehsan Peymani Validation of Embedded Control Systems
68. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Concept
An effective approach for Validation of Embedded Control Systems
As a result,
We need to create a system-level testing to include the
complexity of the plant under control.
We want to do testing with only the embedded control system.
Mechanical
Actuators Sensors
Disturbances Noise
ProcessController ControlReference
Real-Time Simulator
Mechanical
System
Actuators Sensors
Disturbances Noise
Process
Variable
Controller
SW/HW
Control
Signal
Reference
C
lick
to
bu
w
w
w
.docu-track.c
o
m
◦ This real-time simulator is called “hardware-in-the-loop
simulator”.
Ehsan Peymani Validation of Embedded Control Systems
69. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Concept
An effective approach for Validation of Embedded Control Systems
As a result,
We need to create a system-level testing to include the
complexity of the plant under control.
We want to do testing with only the embedded control system.
Mechanical
System
Actuators Sensors
Disturbances Noise
Process
Variable
Controller
SW/HW
Control
Signal
Reference
Real-Time Simulator
Mechanical
System
Actuators Sensors
Disturbances Noise
Process
Variable
Controller
SW/HW
Control
Signal
Reference
◦ This real-time simulator is called “hardware-in-the-loop
simulator”.
Ehsan Peymani Validation of Embedded Control Systems
70. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Concept
An effective approach for Validation of Embedded Control Systems
Test Program
Mechanical
System
Actuators Sensors
Disturbances Noise
Process
Variable
Controller
SW/HW
Control
Signal
Reference
Real-time Simulator
Test Plan
Test Requirements
Test Analysis
Test Documentation
Test Signals
Result (simulated signals)Stimuli
Hardware-in-the-loop testing is a system-level testing, with only
the embedded control system, by creating a system model.
Hardware-in-the-loop testing: virtual system-level testingEhsan Peymani Validation of Embedded Control Systems
71. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Components
The hardware-in-the-loop testing has two main components:
1 Hardware-in-the-loop simulator
2 Hardware-in-the-loop test program
Ehsan Peymani Validation of Embedded Control Systems
72. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop simulator
HIL simulator is a mathematical representation of the plant.
+ the mechatronic system
+ actuators and sensors
+ environment
HIL Simulator
HIL testing concept
Simulated
actuators
Simulated
sensors
Simulated
environment
Process Under Control
Actuators
Dynamic
systems
Sensors
Environment
Control signals
Control signals
Measurements
Simulated
Measurements
Control
system
I/O
MMI
Man
Simulated
dynamic
systems
Control
system
I/O
MMI
Man
C
lick
to
buy
N
O
W
!
PDF-XChange
w
w
w
.docu-track.c
o
m
C
lick
to
buy
N
O
W
!
PDF-XChange
w
w
w
.docu-track.c
o
m
Ehsan Peymani Validation of Embedded Control Systems
73. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop simulator
HIL simulator is a mathematical representation of the plant.
+ the mechatronic system
+ actuators and sensors
+ environment
The simulator should be capable of simulating the necessary
range of failures.
Interfacing to the embedded control system through
1 the normal I/O interface
2 a dedicated (HIL) test I/O interface
! The communication delay in the interface should be less than
1/10 of the main sampling time in the control system.
Fault insertion unit to create signal faults
Ehsan Peymani Validation of Embedded Control Systems
74. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop simulator
HIL simulator is a mathematical representation of the plant.
+ the mechatronic system
+ actuators and sensors
+ environment
The simulator should be capable of simulating the necessary
range of failures.
Interfacing to the embedded control system through
1 the normal I/O interface
2 a dedicated (HIL) test I/O interface
! The communication delay in the interface should be less than
1/10 of the main sampling time in the control system.
Fault insertion unit to create signal faults
Ehsan Peymani Validation of Embedded Control Systems
75. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop simulator
HIL simulator is a mathematical representation of the plant.
+ the mechatronic system
+ actuators and sensors
+ environment
The simulator should be capable of simulating the necessary
range of failures.
Interfacing to the embedded control system through
1 the normal I/O interface
2 a dedicated (HIL) test I/O interface
! The communication delay in the interface should be less than
1/10 of the main sampling time in the control system.
Fault insertion unit to create signal faults
Figure 1. An HIL test system consists of three primarycomponents: an operator interface, a real-time process
Hardware Fault Insertion
Many HIL test systems use hardware fault insertion to create signal faults between the ECU and the rest of the system to test, ch
device under these conditions. To accomplish this, you can insert fault insertion units (FIUs) between the I/O interfaces and the E
switch the interface signals between normal operation and fault conditions such as a short-to-ground or open circuit.
Figure 2. You can use hardware fault insertion to test the behavior of the ECU during signal fa
C
lick
to
buy
N
O
W
!
PDF-XChange
w
w
w
.docu-track.c
o
m
Ehsan Peymani Validation of Embedded Control Systems
76. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop simulator
Simulator dilemma:
sufficiently accurate simulator
sufficiently small time step
synchronization between simulated part and physical part
data must be exchanged at every time that hardware is
sampled.
The HIL simulator must find the state at the next time step in
less time than it takes the step to occur in reality.
Ehsan Peymani Validation of Embedded Control Systems
77. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop simulator
Real-time simulator
consistent amount of computations at every time step
Ehsan Peymani Validation of Embedded Control Systems
78. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop simulator
Real-time simulator
consistent amount of computations at every time step
Ehsan Peymani Validation of Embedded Control Systems
79. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop simulator
Real-time simulator
inconsistent amount of computations time steps
Figure 2.3: Simulation Requirements for Hardware-in-the-Loop ApplicationsEhsan Peymani Validation of Embedded Control Systems
80. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop test program
The HIL test program includes:
1 test requirements
2 test plan
3 test profile
4 user interface
5 test assessment and analysis
6 test documentation
Ehsan Peymani Validation of Embedded Control Systems
81. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop test program
The HIL test plan should consider these test scenarios:
1 Functional testing: to test a function to verify that the
specified functional requirements are met.
2 Failure testing: to test the functionality of the ECS for failure
handling.
3 Performance testing: to test and quantify the performance
(how good) of the ECS under all feasible operating conditions.
Ehsan Peymani Validation of Embedded Control Systems
82. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Software-to-Hardware ratio
Simulator
Control Software
Embedded
Control System
Simulator
C
lick
to
buy
N
O
W
!
PDF-XChange
w
w
w
.docu-track.c
o
m
C
lick
to
PDF-XC
w
w
w
.docu
Figure: These are not hardware-in-the-loop simulators.
The HIL simulator and the embedded control system under
test are implemented in two separate platforms.
Ehsan Peymani Validation of Embedded Control Systems
83. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Benefits
1 Enhancing the quality of testing by increasing test coverage.
2 Tight development schedules by testing the control system in
parallel with the development of the plant.
3 High-burden-rate plant: Simulators are much cheaper than
actual systems (e.g. jet engines).
4 Early process human factors development by providing
human-in-the-loop testing to evaluate ::::::::
usability::::
and:::::::
system
::::::::::
consistency.
− Optimal selection of parameters in fly-by-wire flight controls.
− If HIL testing is not used, field trials should be performed for
parameter selection.
Not good: cost, duration, safety, feasibility.
Ehsan Peymani Validation of Embedded Control Systems
84. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Benefits
1 Enhancing the quality of testing by increasing test coverage.
2 Tight development schedules by testing the control system in
parallel with the development of the plant.
3 High-burden-rate plant: Simulators are much cheaper than
actual systems (e.g. jet engines).
4 Early process human factors development by providing
human-in-the-loop testing to evaluate ::::::::
usability::::
and:::::::
system
::::::::::
consistency.
− Optimal selection of parameters in fly-by-wire flight controls.
− If HIL testing is not used, field trials should be performed for
parameter selection.
Not good: cost, duration, safety, feasibility.
Ehsan Peymani Validation of Embedded Control Systems
85. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Benefits
1 Enhancing the quality of testing by increasing test coverage.
2 Tight development schedules by testing the control system in
parallel with the development of the plant.
3 High-burden-rate plant: Simulators are much cheaper than
actual systems (e.g. jet engines).
4 Early process human factors development by providing
human-in-the-loop testing to evaluate ::::::::
usability::::
and:::::::
system
::::::::::
consistency.
− Optimal selection of parameters in fly-by-wire flight controls.
− If HIL testing is not used, field trials should be performed for
parameter selection.
Not good: cost, duration, safety, feasibility.
Ehsan Peymani Validation of Embedded Control Systems
86. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Benefits
1 Enhancing the quality of testing by increasing test coverage.
2 Tight development schedules by testing the control system in
parallel with the development of the plant.
3 High-burden-rate plant: Simulators are much cheaper than
actual systems (e.g. jet engines).
4 Early process human factors development by providing
human-in-the-loop testing to evaluate ::::::::
usability::::
and:::::::
system
::::::::::
consistency.
− Optimal selection of parameters in fly-by-wire flight controls.
− If HIL testing is not used, field trials should be performed for
parameter selection.
Not good: cost, duration, safety, feasibility.
Ehsan Peymani Validation of Embedded Control Systems
87. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Benefits
5 Reducing the validity process time
6 Achieving much greater test repeatability
7 Reducing the number of actual system-level testings
Ehsan Peymani Validation of Embedded Control Systems
88. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Benefits
5 Reducing the validity process time
6 Achieving much greater test repeatability
7 Reducing the number of actual system-level testings
Ehsan Peymani Validation of Embedded Control Systems
89. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing: Benefits
5 Reducing the validity process time
6 Achieving much greater test repeatability
7 Reducing the number of actual system-level testings
Figure: Aircraft Arrestor Control System
Ehsan Peymani Validation of Embedded Control Systems
90. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Hardware-in-the-loop testing in the V-model
R. Skjetne and O. Egeland246
Figure 5. V-model for control system development and installation
Ehsan Peymani Validation of Embedded Control Systems
91. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Recent Issues: Test Automation
There are four major parts to any testing effort:
1 test case design
2 test case creation
3 test case execution
4 test case analysis and debugging
+ test documentation
+ test assessment
+ requirement tracing
Ehsan Peymani Validation of Embedded Control Systems
92. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Motivation and Concept
Components
Benefits
Recent Issues
Recent Issues: Test Automation
There are four major parts to any testing effort:
1 test case design
2 test case creation
3 test case execution
4 test case analysis and debugging
+ test documentation
+ test assessment
+ requirement tracing
Ehsan Peymani Validation of Embedded Control Systems
93. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Outline
1 Embedded Control Systems
2 Validation of Control Software
3 Hardware-in-the-loop testing
4 Closing Remarks
Ehsan Peymani Validation of Embedded Control Systems
94. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Closing Remarks
The software growth trend is expected to continue because of
increasingly ambitious requirements and because of the
advantages of situating new functionality in software or
firmware rather than hardware.
“Engineers and scientists often do not realize the downstream
complexity and cost entailed by their local decisions. [· · · ] A
lack of consideration for testability can complicate verification
efforts”, NASA report.
Ehsan Peymani Validation of Embedded Control Systems
95. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Closing Remarks
Hardware-in-the-loop testing is a fundamental part of the
development of a plant, especially at aviation industry.
Hardware-in-the-loop testing is useful, but it never replaces
actual system testing (field trials are inevitable).
A hardware-in-the-loop simulator, not only, can be used for
testing an embedded control system, but also it can be used
for
control system design and tuning
system design revisions and optimization
quality monitoring
training of personnel (pilots, operators)
The simulator can be made a multi-purpose simulator,
specially by adding 3D visualization capability.
Ehsan Peymani Validation of Embedded Control Systems
96. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Closing Remarks
Hardware-in-the-loop testing is a fundamental part of the
development of a plant, especially at aviation industry.
Hardware-in-the-loop testing is useful, but it never replaces
actual system testing (field trials are inevitable).
A hardware-in-the-loop simulator, not only, can be used for
testing an embedded control system, but also it can be used
for
control system design and tuning
system design revisions and optimization
quality monitoring
training of personnel (pilots, operators)
The simulator can be made a multi-purpose simulator,
specially by adding 3D visualization capability.
Ehsan Peymani Validation of Embedded Control Systems
97. Embedded Control Systems
Validation of Control Software
Hardware-in-the-loop testing
Closing Remarks
Thank you for your participation!
Ehsan Peymani Validation of Embedded Control Systems