Automated Testing of NASA Software
Upcoming SlideShare
Loading in...5
×
 

Automated Testing of NASA Software

on

  • 883 views

 

Statistics

Views

Total Views
883
Slideshare-icon Views on SlideShare
883
Embed Views
0

Actions

Likes
2
Downloads
11
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Automated Testing of NASA Software Automated Testing of NASA Software Presentation Transcript

    • CESE Automated Testing of Large Multi-language SW Systems using Cloud Computing Executive Status Report Principal Investigator (PI): Dr. Mikael Lindvall, CESE NASA POC: Markland Benson, White Sands Team members: Dharma Ganesan, Dr. Chris Ackermann(CESE) GMSEC, CFS, Space Network, MSL © 2011 Fraunhofer USA, Inc. 1 Center for Experimental Software Engineering
    • CESE Motivation• Bugs can lead to deaths, injuries, or financial loss• Software testing is necessary to find the presence of bugs• Software testing consumes 50% - 75% of the development effort• Many NASA projects could benefit from automated test generation• In fact, several skeptical engineers noted that the proposed work is illuminating! © 2011 Fraunhofer USA, Inc. Center for Experimental Software Engineering
    • CESE Technical Context• The system(s) under test (SUT) are complex• Multiple programming languages – Ada, C, C++, Fortran, Java• Distributed legacy systems – More than 20 yrs old – Runs on multiple machines and involves various OSes,e.g. OpenVMS• GUI intensive – Web, Java Swing, AWT, X Windows, Motif © 2011 Fraunhofer USA, Inc. 3 Center for Experimental Software Engineering
    • CESE Problems• Test cases are developed manually• Some test execution is automated (e.g., JUnit)• Test cases miss valid “corner” cases• Test cases are also programs: not easy for non- technical stakeholders to understand• Difficult to summarize what was tested Approach: Lightweight Model-based Test Generation and Execution © 2011 Fraunhofer USA, Inc. 4 Center for Experimental Software Engineering
    • CESE Approach• Develop a model of the system under test• The model contains actions and expected results• Automatically derive test cases from the model• Execute the test cases• Decide when to stop testing based on model coverage! © 2011 Fraunhofer USA, Inc. Center for Experimental Software Engineering
    • CESE An Example Model APIs of the module under test public interface IConnection { public Status Connect(); public Status Disconnect(); © 2011 Fraunhofer USA, Inc. } Center for Experimental Software Engineering
    • CESE Currently Targeted Projects• GMSEC – Reusable framework for ground systems – Modeled the Core API and Applications – Generated executable test cases from the model – Confirmed defects/violations reported and fixed• Core Flight Software – Reusable framework for flight systems – Modeled the OS abstraction layer (OSAL) – Generated executable test cases from the model – Confirmed defects/violations reported and fixed – Test cases delivered to the CFS team © 2011 Fraunhofer USA, Inc. 7 Center for Experimental Software Engineering
    • CESE Currently Targeted Projects• Space Network – White Sands – Evaluating different test execution tools suitable for Open VMS environment – E.g. Image-based test execution tools: EggPlant and Sikuli – Challenges in this project: • SUT not possible to run at Fraunhofer • Must visit Goddard with an escort (secured building) • Not possible to connect to Internet © 2011 Fraunhofer USA, Inc. 8 Center for Experimental Software Engineering
    • CESE Currently Targeted Projects• Mars Science Laboratory (MSL) – JPL – Can current test cases be modeled and used to generate new test cases? • New set cover scenarios not previously covered? • New approach more cost effective? – Challenges in this project: • SUT not possible to run at Fraunhofer • Software is sensitive (ITAR) © 2011 Fraunhofer USA, Inc. 9 Center for Experimental Software Engineering
    • CESE Current Results• An end-to-end approach for test generation – Successfully used on the GMSEC and the CFS and detected bugs• Approach works well on different levels: – API (Module interface) level testing – GUI testing• Approach transferred to an intern at GSFC – The intern found new bugs using the approach – Demonstrates the approach is lightweight © 2011 Fraunhofer USA, Inc. 10 Center for Experimental Software Engineering
    • CESE Summary and Next Steps• We’re building a new approach that – Helps in test automation using lightweight models• Prototype approach successfully applied to GMSEC and CFS/cFE• Next steps are to apply to the Space Network and the MSL projects• Examples and technical details in technical presentation! © 2011 Fraunhofer USA, Inc. 11 Center for Experimental Software Engineering