Towards Target-Level Testing and Debugging Tools For Embedded Software Harry Koehnemann, Arizona State University Dr. Timo...
<ul><li>Goal: Identify the problems associated with test case execution for Embedded Systems </li></ul><ul><li>To propose ...
Introduction <ul><li>Currently: Testing and Debugging of Embedded Real Time Software:  A “black  art” - ad hoc methods and...
Software Testing and Debugging Process <ul><li>Testing: Executing a piece of software in order to reveal errors- A substan...
Embedded Systems <ul><li>Correct execution of embedded applications absolutely critical. </li></ul><ul><li>Testing and Deb...
Software Testing <ul><li>Software testing for embedded systems:4 basic stages </li></ul><ul><li>Module level testing </li>...
Testing Concurrent Systems <ul><li>Concurrency increases the difficulty of s/w testing. </li></ul><ul><li>-Unmanageably la...
Embedded Testing <ul><li>Embedded systems have critical issues/concerns as discussed on an earlier slide. </li></ul><ul><l...
Current Solutions <ul><li>Hardware Solutions: Attempt to gain execution visibility and program control. </li></ul><ul><li>...
Problems with Embedded Testing <ul><li>1. Expense of testing process -  Little reuse, expensive custom validation faciliti...
Increasing Target Functionality <ul><li>-Tool support for embedded systems: Lacking. </li></ul><ul><li>-Current approaches...
Model Debugging System
The model debugging system continued.. <ul><li>The data path from the debugging/testing tool represents symbol table infor...
Architectural Additions <ul><li>1-Hardware partitioning of memory:  Each process has its own protected address space, prov...
Architectural Additions Continued… <ul><li>4-Architectural Support for Abstractions:  Abstractions to be incorporated into...
Run-Time System Additions <ul><li>RTS requirements describe an interface between a tool and the underlying system: logical...
Run-Time System Additions continued.. <ul><li>The CIFO/MRTSI provide abstractions for: </li></ul><ul><li>Processes </li></...
Conclusions <ul><li>With the RTS and Architectural additions, a solution to the problems in embedded system testing can be...
Strengths <ul><li>Emphasis on the requirement of detection of the most difficult to fix errors as early as possible during...
Weakness <ul><li>The costs of the additions have been neglected. The authors have identified the need to study the same fo...
Upcoming SlideShare
Loading in …5
×

Towards Target-Level Testing and Debugging Tools For Embedded ...

609 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
609
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Towards Target-Level Testing and Debugging Tools For Embedded ...

  1. 1. Towards Target-Level Testing and Debugging Tools For Embedded Software Harry Koehnemann, Arizona State University Dr. Timothy Lindquist, Arizona State University Presented by: Soubhagya Kumar Dash
  2. 2. <ul><li>Goal: Identify the problems associated with test case execution for Embedded Systems </li></ul><ul><li>To propose solutions for making embedded testing more effective at revealing errors. </li></ul>
  3. 3. Introduction <ul><li>Currently: Testing and Debugging of Embedded Real Time Software: A “black art” - ad hoc methods and techniques. </li></ul><ul><li>- Ineffective and inadequate. </li></ul><ul><li>Huge costs associated with validation of embedded applications. Despite this, most difficult errors are discovered extremely late in the testing process, making them even more costly to repair. </li></ul><ul><li>Requirement: Formal methods, development of architectural and software capabilities which support testing and debugging with minimal intrusion on the executing system. This is critical for testing Embedded applications. </li></ul><ul><li>Test cases must take care of the real time and environmental factors as well. Not just check for correct input-output (functional behavior) mapping. </li></ul>
  4. 4. Software Testing and Debugging Process <ul><li>Testing: Executing a piece of software in order to reveal errors- A substantial portion of the validation process. </li></ul><ul><li>Development of test procedures, generation and execution of test cases. </li></ul><ul><li>Debugging: This is concerned with locating and correcting the cause of an error once it has been revealed. </li></ul><ul><li>Developer must recreate exact execution scenario. </li></ul><ul><li>Same instruction sequences </li></ul><ul><li>All environmental variants must be accounted for. </li></ul>
  5. 5. Embedded Systems <ul><li>Correct execution of embedded applications absolutely critical. </li></ul><ul><li>Testing and Debugging: Greatly restricted by embedded systems, with constraints such as: </li></ul><ul><li>Concurrent Designs </li></ul><ul><li>Real-time constraints </li></ul><ul><li>Embedded target environments </li></ul><ul><li>Distributed hardware architectures </li></ul><ul><li>Device control dependencies </li></ul><ul><li>These restrict execution visibility and control. </li></ul><ul><li>Target environment: grossly inadequate computing resources. </li></ul>
  6. 6. Software Testing <ul><li>Software testing for embedded systems:4 basic stages </li></ul><ul><li>Module level testing </li></ul><ul><li>Integration testing </li></ul><ul><li>System testing </li></ul><ul><li>Hardware/Software Integration testing – This is unique to embedded systems. </li></ul><ul><li>Techniques of particular interest to us: </li></ul><ul><li>Testing Concurrent Systems </li></ul><ul><li>Non-intrusive testing </li></ul>
  7. 7. Testing Concurrent Systems <ul><li>Concurrency increases the difficulty of s/w testing. </li></ul><ul><li>-Unmanageably large set of legal execution sequences that a concurrent program may take. </li></ul><ul><li>-Subsequent execution could lead to different-yet correct results. </li></ul><ul><li>-Dealing with abstraction </li></ul><ul><li>-Static and Dynamic testing </li></ul><ul><li>Non-intrusive testing </li></ul><ul><li>For host based- intrusion is acceptable. </li></ul><ul><li>Embedded applications have strict timing requirements. Absolutely imperative that there be no intrusions on a test execution. </li></ul><ul><li>Embedded tools: ROM/Bus monitors, Emulators </li></ul>
  8. 8. Embedded Testing <ul><li>Embedded systems have critical issues/concerns as discussed on an earlier slide. </li></ul><ul><li>-Typically developed on custom h/w configurations, each would require own set of tools and techniques. </li></ul><ul><li>Errors discovered during H/S integration testing are most difficult of all. Often require significant modifications to the s/w system. </li></ul><ul><li>2 environments: Host and the Target. Target has little support for s/w development tools. </li></ul>
  9. 9. Current Solutions <ul><li>Hardware Solutions: Attempt to gain execution visibility and program control. </li></ul><ul><li>Bus monitors, ROM monitors, in-circuit emulators. </li></ul><ul><li>Minimal effectiveness for s/w development, since aid only in gathering low level data, which have to be subsequently mapped- difficult, cumbersome. </li></ul><ul><li>Software Solutions: Attempt to reduce cost & time spent testing on target. </li></ul><ul><li>Factors: Level of criticality, Test platform availability, test classification etc. </li></ul><ul><li>- Will likely lead to extensive modifications, and hence extensive retesting. </li></ul>
  10. 10. Problems with Embedded Testing <ul><li>1. Expense of testing process - Little reuse, expensive custom validation facilities required for every project. Retests extremely costly. </li></ul><ul><li>2. Level of functionality on target </li></ul><ul><li>Very little, hence a lot of effort to discover errors. </li></ul><ul><li>3. Late discovery of errors - s/w modified to rectify h/w errors, delays error discovery. </li></ul><ul><li>4. Poor test selection criteria </li></ul><ul><li>Test case selection rarely on theoretical criteria </li></ul><ul><li>5. Potential use in advancing architectures </li></ul><ul><li>Little chance of current methods remaining applicable to future architectures. </li></ul>
  11. 11. Increasing Target Functionality <ul><li>-Tool support for embedded systems: Lacking. </li></ul><ul><li>-Current approaches will soon be rendered obsolete. </li></ul><ul><li>-Proposal: Adding facilities to the underlying system to better support testing and debugging tools. </li></ul><ul><li>Underlying system: Hardware architecture and the run-time system. </li></ul>
  12. 12. Model Debugging System
  13. 13. The model debugging system continued.. <ul><li>The data path from the debugging/testing tool represents symbol table information that allows the mapping of machine level information to source level constructs. ASIS toolkit provides easy implementation. </li></ul><ul><li>Any debugging system has at least two processes executing: test program and the debugger. </li></ul><ul><li>Part of the debugger runs on the host, and the other on the target machine. </li></ul><ul><li>To make this non-intrusive: </li></ul><ul><li>1-Execute code only at break point OR </li></ul><ul><li>2-Run debugger as a separate process OR </li></ul><ul><li>3-Provide separate execution unit to execute debugger </li></ul>
  14. 14. Architectural Additions <ul><li>1-Hardware partitioning of memory: Each process has its own protected address space, provision for access control and shared memory. </li></ul><ul><li>2-Computational facilities for Debugger: The internal debugger requires computational facilities, it could be modeled as a regular process on the processor, or architecture provisions may be made. </li></ul><ul><li>3-Hardware Break Points: S/w break points are intrusive. Architecture has to provide for a set of hardware registers- for data and instruction, and the logic to compare them with the operands of the current instruction. </li></ul>
  15. 15. Architectural Additions Continued… <ul><li>4-Architectural Support for Abstractions: Abstractions to be incorporated into the instruction sets. The processor would hence fewer and more meaningful messages, and therefore lesser mapping will be required. </li></ul><ul><li>5-Dedicated Bus: Interface that allows the processor to communicate with the external world without interfering with the system under test. Should be detachable, required for deployment. </li></ul><ul><li>Architectural Additions are costly in both time and space. </li></ul>
  16. 16. Run-Time System Additions <ul><li>RTS requirements describe an interface between a tool and the underlying system: logical interface requiring substantial hardware support. </li></ul><ul><li>Goal is to minimize the data and computational requirements of the internal debugger as well as the required communication between the internal and external debuggers. </li></ul><ul><li>MRTSI and CIFO implements the abstractions as per a standard. </li></ul>
  17. 17. Run-Time System Additions continued.. <ul><li>The CIFO/MRTSI provide abstractions for: </li></ul><ul><li>Processes </li></ul><ul><li>Interrupt Management </li></ul><ul><li>Time Management </li></ul><ul><li>Memory Management </li></ul><ul><li>Exception/Fault Handling </li></ul><ul><li>RTS additions are minimal. </li></ul>
  18. 18. Conclusions <ul><li>With the RTS and Architectural additions, a solution to the problems in embedded system testing can be addressed. </li></ul><ul><li>Testing and Debugging of embedded, real time software remains a “black art”, with ad hoc methods and techniques. </li></ul><ul><li>Evaluation of the additions and determination of their feasibility is to be done. </li></ul>
  19. 19. Strengths <ul><li>Emphasis on the requirement of detection of the most difficult to fix errors as early as possible during the testing phases. </li></ul><ul><li>Appropriate Caution while using software solutions, has been advised. </li></ul><ul><li>The foresight on the effects of certification on test case selection is thought provoking. </li></ul><ul><li>An Elegant model for debugging suggested. </li></ul>
  20. 20. Weakness <ul><li>The costs of the additions have been neglected. The authors have identified the need to study the same for the feasibility of the model, though. </li></ul><ul><li>It is not very clear as to how the Hardware break points would not require computation cycles as the software break points do. The same applies to the internal debugger. </li></ul>

×