Model Based Software Testing


Published on

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Model Based Software Testing

  1. 1. Esin Karaman SWE550 Boğaziçi University 1
  2. 2. Agenda Software testing process evalution What is model-based testing The process of model-based testing MBT (Model-based testing) Tools Prerequisites for MBT Benefits of MBT Limitations of MBT 2
  3. 3. «Life is too short for manual testing.» Harry Robinson, test architect for Microsoft’s Engineering Excellence Group and a driving force behind Microsoft’s model-based testing initiative, 1999-2004 3
  4. 4. Software testing process evalution 4 Manual Testing Capture/Replay Script Base Testing Keyword-Driven Automated Testing Model-Based
  5. 5. What is model-based testing? Model-based testing is a variant of testing for automatic generation of test cases using models describing some aspects of the software under test (SUT). It can automate the complete test design process, given a suitable model, and produces complete test sequences that can be transformed into executable tests cripts. Model-based testing is the automation of the design of black-box tests 5
  6. 6. What is model-based testing? The difference from the usual black-box testing is that rather than manually writing tests based on the requirements documentation, we instead create a model of the expected SUT behaviour, which captures some of the requirements. Then the model based testing tools are used to automatically generate tests from that model. 6
  7. 7. The process of model-based testing Create a model detailed enough to describe the characteristics that we want to test Automatically generate abstract test cases from the model by using a model-based testing tool Transform abstract test cases into into executable test scripts Execute test scripts by a test execution tool 7
  8. 8. 8 Model-Based Testing Process
  9. 9. Modelling A model of the SUT is built on the grounds of requirements Includes expected behaviour Must be concise and precise 9
  10. 10. Modelling (2) Model-based testing uses models of the SUT and its environment. Abstraction - Function - Data M1-M3 is suitable 10
  11. 11. Modelling (3) Modelling notations State based (or Pre/Post) Notations Transition-based Notations Functional Notations Operational Notations Stochastic Notations Data-Flow Notations 11
  12. 12. Test requirements selection Used to control the generation of tests Steps: Test selection criteria are defined Test selection criteria are then transformed into test case specifications 12
  13. 13. Test selection criteria Defining «best» test selection criteria is not possible. Test engineer chooses adequate test selection criteria. Criteria types: Structural model coverage criteria Data coverage criteria Requirements-based coverage criteria Ad-Hoc test case specifications 13
  14. 14. Test generation Once the model and the test case specification are defined, an abstract test suite is generated. Technology: Random Graph-search algorithms Model-checking Symbolic execution Theorem proving 14
  15. 15. Test concretization(transformation) Concretize abstract test suite to executable level (test script). This can be done by a MBT Tool using some translation tables supplied by the test engineer. The resulting executable tests may be Junit in Java, or in a dynamic language such as Tcl or Python, or in a dedicated test scripting language. 15
  16. 16. Test execution Test scripts are executed and outcomes are compared to expected ones. Tests are marked as pass, fail or inconclusive Two approaches: Online Offline 16
  17. 17. Test execution(2) Online Testing Tests are executed as they are generated MBT tool is tightly coupled to the SUT Slow Offline Testing Tests are generated first, executed afterwards Decouples the generation and execution phases It can be used repeatedly for regression purposes 17
  18. 18. MBT Tools TorX LTG (LEIRIOS Test Generator) Matlab Simulink V&V JUMBL AETG 18
  19. 19. Prerequisites for MBT Someone enthusiastic Test execution maturity Modelling skills Access to training, tools, and experts The right kind of application 19
  20. 20. Benefits of MBT Fault detection : MBT founds roughly the same number of errors as manually designed test suits Reduced testing cost and time Improved test quality Requirements defect detection Requirements evolution 20
  21. 21. Limitations of MBT The model designers must be expert in the application area. This may require some training costs. Usually used only for functional testing, little experience for other kinds of testing. Sophisticated approach to testing, so it requires a reosonable level of test maturity. It is unwise to start using MBT without some experience with automated test execution. 21
  22. 22. References Mark Utting, Alexander Pretschner and Bruno Legeard , A Taxonomy of Model-Based Testing, 2006 Bruno Legeard, Mark Utting, Practical Model-Based Testing A Tools Approach, 2007 A. Pretschner, W. Prenninger, S. Wagner, C. Kuhnel, M. Baumgartner, B. Sostawa, One Evaluation of Model-Based Testing and its Automation, 2005 22