Automated Testing of NASA Software

1,445 views

Published on

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,445
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
19
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Automated Testing of NASA Software

  1. 1. CESE Automated Testing of Large Multi-language SW Systems using Cloud Computing Executive Status Report Principal Investigator (PI): Dr. Mikael Lindvall, CESE NASA POC: Markland Benson, White Sands Team members: Dharma Ganesan, Dr. Chris Ackermann(CESE) GMSEC, CFS, Space Network, MSL © 2011 Fraunhofer USA, Inc. 1 Center for Experimental Software Engineering
  2. 2. CESE Motivation• Bugs can lead to deaths, injuries, or financial loss• Software testing is necessary to find the presence of bugs• Software testing consumes 50% - 75% of the development effort• Many NASA projects could benefit from automated test generation• In fact, several skeptical engineers noted that the proposed work is illuminating! © 2011 Fraunhofer USA, Inc. Center for Experimental Software Engineering
  3. 3. CESE Technical Context• The system(s) under test (SUT) are complex• Multiple programming languages – Ada, C, C++, Fortran, Java• Distributed legacy systems – More than 20 yrs old – Runs on multiple machines and involves various OSes,e.g. OpenVMS• GUI intensive – Web, Java Swing, AWT, X Windows, Motif © 2011 Fraunhofer USA, Inc. 3 Center for Experimental Software Engineering
  4. 4. CESE Problems• Test cases are developed manually• Some test execution is automated (e.g., JUnit)• Test cases miss valid “corner” cases• Test cases are also programs: not easy for non- technical stakeholders to understand• Difficult to summarize what was tested Approach: Lightweight Model-based Test Generation and Execution © 2011 Fraunhofer USA, Inc. 4 Center for Experimental Software Engineering
  5. 5. CESE Approach• Develop a model of the system under test• The model contains actions and expected results• Automatically derive test cases from the model• Execute the test cases• Decide when to stop testing based on model coverage! © 2011 Fraunhofer USA, Inc. Center for Experimental Software Engineering
  6. 6. CESE An Example Model APIs of the module under test public interface IConnection { public Status Connect(); public Status Disconnect(); © 2011 Fraunhofer USA, Inc. } Center for Experimental Software Engineering
  7. 7. CESE Currently Targeted Projects• GMSEC – Reusable framework for ground systems – Modeled the Core API and Applications – Generated executable test cases from the model – Confirmed defects/violations reported and fixed• Core Flight Software – Reusable framework for flight systems – Modeled the OS abstraction layer (OSAL) – Generated executable test cases from the model – Confirmed defects/violations reported and fixed – Test cases delivered to the CFS team © 2011 Fraunhofer USA, Inc. 7 Center for Experimental Software Engineering
  8. 8. CESE Currently Targeted Projects• Space Network – White Sands – Evaluating different test execution tools suitable for Open VMS environment – E.g. Image-based test execution tools: EggPlant and Sikuli – Challenges in this project: • SUT not possible to run at Fraunhofer • Must visit Goddard with an escort (secured building) • Not possible to connect to Internet © 2011 Fraunhofer USA, Inc. 8 Center for Experimental Software Engineering
  9. 9. CESE Currently Targeted Projects• Mars Science Laboratory (MSL) – JPL – Can current test cases be modeled and used to generate new test cases? • New set cover scenarios not previously covered? • New approach more cost effective? – Challenges in this project: • SUT not possible to run at Fraunhofer • Software is sensitive (ITAR) © 2011 Fraunhofer USA, Inc. 9 Center for Experimental Software Engineering
  10. 10. CESE Current Results• An end-to-end approach for test generation – Successfully used on the GMSEC and the CFS and detected bugs• Approach works well on different levels: – API (Module interface) level testing – GUI testing• Approach transferred to an intern at GSFC – The intern found new bugs using the approach – Demonstrates the approach is lightweight © 2011 Fraunhofer USA, Inc. 10 Center for Experimental Software Engineering
  11. 11. CESE Summary and Next Steps• We’re building a new approach that – Helps in test automation using lightweight models• Prototype approach successfully applied to GMSEC and CFS/cFE• Next steps are to apply to the Space Network and the MSL projects• Examples and technical details in technical presentation! © 2011 Fraunhofer USA, Inc. 11 Center for Experimental Software Engineering

×