Robustness testing

5,259 views

Published on

NASA interns exploring robustness testing

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
5,259
On SlideShare
0
From Embeds
0
Number of Embeds
32
Actions
Shares
0
Downloads
77
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Robustness testing

    1. 1. Robustness Testing & Research Infusion IV&V Facility Robert Ball Fairmont State University
    2. 2. Robustness Testing with Fault Injection <ul><li>Robustness - the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions </li></ul><ul><li>The goal of robustness testing is to develop test cases and test environments where a system's robustness can be assessed. </li></ul>
    3. 3. Process Overview <ul><li>Fault Injection </li></ul><ul><ul><li>Preparation </li></ul></ul><ul><ul><li>Test Execution </li></ul></ul><ul><ul><li>Log Analysis </li></ul></ul><ul><li>Documentation </li></ul><ul><ul><li>Describes scope of test campaign </li></ul></ul><ul><ul><li>Lists directives/routines tested </li></ul></ul><ul><ul><li>Describes test results </li></ul></ul>
    4. 4. Flight Software Traces RTOS API Test Results RTOS Directives found in Project FSW Directives found in API Test Results and Project FSW Source Navigator Source Code
    5. 5. Test Execution
    6. 6. Documentation Sample Fault Table Error ID #: 0x3d001 Error Description: S_objLib_OBJ_ID_ERROR Analysis: Since the range of priority levels are limited to be between 0 and 255, anything out of this range is expected to return this error message. Results set to a value outside of range 0 to 255 Test Case 1 C:TornProjTest2usrAppInit.c line: 186 Fault Location int taskSpawn ( char * name, int priority, int options, int stackSize, FUNCPTR entryPt, int arg1, int arg2, int arg3, int arg4, int arg5, int arg6, int arg7, int arg8, int arg9, int arg10 ) Declaration
    7. 7. Results VxWorks <ul><li>HIGHLY AUTOMATED using scripts and support programs </li></ul><ul><li>Testing VxWorks 5.5 </li></ul><ul><li>Testing POSIX and Classic APIs </li></ul><ul><li>GLAST Flight Software Traces – Complete nearly 300 functions used </li></ul><ul><li>VxWorks Libraries </li></ul><ul><ul><li>Task Management, Task Hook, Task Information, Task Show, Task Variables Support, Timer, Timex, Clock Tick Support, Watchdog Timer </li></ul></ul><ul><ul><li>Over 70 routines </li></ul></ul><ul><li>Progress </li></ul><ul><ul><li>S:NASA SharedResearch and Development Fisher FacilityResearchProgramRobustness TestingVXWORKS GLASTProgress.xls </li></ul></ul><ul><li>Documents </li></ul><ul><ul><li>GLAST/VxWorks Robustness Testing </li></ul></ul>
    8. 8. Automation Tools <ul><li>Test Case Generator </li></ul><ul><li>Test Script Generator </li></ul><ul><li>Various Scripts (Batch Files) </li></ul><ul><li>Xception </li></ul><ul><li>Robustness testing lends itself to automation, saving time and money </li></ul><ul><li>S:NASA SharedResearch and Development - FisherFacilityResearchProgramRobustness TestingVXWORKS </li></ul>
    9. 9. Applicability <ul><li>Any project that utilizes a Real-Time Operating System with an Application Programmable Interface (API) </li></ul><ul><ul><li>VxWorks </li></ul></ul><ul><ul><li>RTEMS </li></ul></ul><ul><li>Both developers and analysts may find the tool useful </li></ul>
    10. 10. Research Infusion Project <ul><li>This project’s goal is to transition mature software engineering research-derived technologies into practice by NASA and its contractors. </li></ul><ul><li>Our task was to evaluate the tools, create a lessons learned document, and support future users of the tools. </li></ul>
    11. 11. Tool Evaluation Criteria <ul><li>Is it a burden to process? </li></ul><ul><li>Are features organized intuitively? </li></ul><ul><li>Was good documentation included? </li></ul><ul><li>Is it useful? </li></ul><ul><li>What errors were encountered? </li></ul>

    ×