Breakout Session: How to Test Flight Software


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Breakout Session: How to Test Flight Software

  1. 1. 2007 Workshop on Spacecraft Flight Software (FSW-07) November 5-6, 2007
  2. 2. Standard Flight Software Architectures <ul><li>Various agencies and organizations have been seeking a standard for flight software. Standards would enable software to be reused across organizations and would reduce software costs from mission to mission. However, where should standardization occur? At the hardware level? The OS (if so, which OS? And does Linux have a place?)? At a messaging layer? Is it even possible to standardize software? One of the goals of this breakout session is answer some of these questions and to also judge the feasibility of applying a standard to organizations that already have a large amount invested in their current architectures. </li></ul>
  3. 3. Preventing and Responding to Flight Software Errors <ul><li>All developers have war stories of how they had a bad index in an array or made a particular programming error. With flight software, that can be a significant event. This session delves into some of the gotchas in flight software and what can be done when they aren’t caught before launch. For instance, what do we need to focus when developing code or conducting code walkthroughs? And once on-orbit, there are questions of how to handle updates or anomalies. Are there benefits to patching code as opposed to doing full builds? Should processor resets be avoided at all costs or are they acceptable in some cases? </li></ul>
  4. 4. How to Test Flight Software <ul><li>Flight software testing is a key element in the development process. This session focuses on defining the best techniques for delivering solid code. Potential discussion areas include: how should stress testing be performed? How should code from other organizations or auto-generated code be tested? What is the developers role in I&T and FSW testing? With limited resources and testing seemingly always pushed to the end of the process, what provides the best bang for the buck? </li></ul>
  5. 5. Stress testing Is . . . <ul><li>Test SW until it breaks (although customers don’t require this – but we think they probably should) </li></ul><ul><li>Spacecraft – ensure it degrades gracefully </li></ul><ul><li>Instrument – know when to do a reset </li></ul><ul><li>Understand what will break it (and write operational procedures to preclude Ops breaking it) </li></ul><ul><li>What is stress? </li></ul><ul><ul><li>Use as much of the processor, I/O, TLM as possible </li></ul></ul><ul><ul><li>Stress using all possible loads (worst case scenarios) – then go beyond </li></ul></ul><ul><li>Stress definition evolves as you proceed through development </li></ul>
  6. 6. How Should Stress Testing Be Done? <ul><li>Build a worst-on-worst test case </li></ul><ul><li>Capture each “oops” in the lab/test bed to add to the suite of stress tests </li></ul>
  7. 7. How Do We Test Autocode? <ul><li>Black Box </li></ul><ul><ul><li>Test against requirements, in such a way as to ensure the software is robust </li></ul></ul><ul><ul><li>Test for critical timing </li></ul></ul><ul><li>White/Clear Box </li></ul><ul><ul><li>So many paths through autogenerated code that complete path testing may not be practical </li></ul></ul><ul><ul><li>Issue with combination of GFE/reused code combined with autogenerated code </li></ul></ul><ul><ul><li>Mostly rely on testing at the model level (especially for algorithms and functionality) </li></ul></ul><ul><li>For “dead code” from OTS, reuse or autogenerated, just use hand analysis to assure it is benign </li></ul>
  8. 8. Role of Developers in Testing <ul><li>Testers participate in design reviews </li></ul><ul><li>Developers do subsystem testing </li></ul><ul><li>Independent testers for qualification testing </li></ul><ul><li>Challenge folks to “break” the software </li></ul><ul><ul><li>Ability to both “break” the software/system AND give this info as a “gift” to the developers that they cherish </li></ul></ul>
  9. 9. Biggest “Bang for the Buck” in Testing <ul><li>Make sure all requirements are tested </li></ul><ul><ul><li>Use model-based requirements testing wherever possible </li></ul></ul><ul><li>Desktop software simulation/testbed </li></ul><ul><li>Robust testbed with flight-like hardware </li></ul><ul><li>Trade-off between making fixes quickly and doing lots of regression testing </li></ul><ul><li>Do system testing and stress testing on mature version of the software </li></ul><ul><li>Robust regression testing </li></ul><ul><li>Patch testing (like you fly) </li></ul><ul><li>Day-in-the-life testing </li></ul><ul><li>Week (or month)-in-the-life characterization </li></ul><ul><li>Get all changes in at least 1 release before the final release </li></ul>
  10. 10. Autonomous Flight Software <ul><li>Modern spacecraft are becoming increasingly reliant upon autonomous systems for operations. This includes command and control as well as data processing and fault protection. However, a price is paid in computing costs with this technology and some argue that it hasn’t been needed in the past, so why do we need it now. Questions for to answer for this session can include: What functionality is appropriate for flight and for the ground? How does fault protection differ between mission objectives? Is a “black box” a solution for spacecraft fault protection? What is the impact of more onboard processing? </li></ul>