'Model Based Test Design' by Mattias Armholt

403 views
328 views

Published on

MBT (Model Based Testing) has been used within my department in Ericsson since 2007. As an MBT tool we have been using Conformiq Modeler, which is a commercially available tool. This has been a great success, and is now our main way of working when verifying functional requirements.



Until now, MBT has neither within Ericsson nor outside, only been used very rarely for verification of non-functional requirements, such as performance testing, load testing, stability and robustness tests and characteristics measurements.



This presentation covers the work of two Master Students, who in 2010 performed a study of the possibilities to use MBT for verifying non-functional requirements. One of the results of this study was a new method, inspired by MPDE (Model Driven Performance Engineering), where non-functional requirements can be covered by test models describing the functional behavior. Test Cases can then be generated from these models with an MBT tool.



The proposed method provides different possibilities to handle the non-functional requirements. The requirements can, for example, be introduced with new dedicated states in the behavioral model, or be introduced by extending the existing state model. Another possibility is to implement the non-functional requirements in the test harness, and by that keeping the model simple. The most realistic scenario, however, is a combination of all the above. The grouping and allocation of both functional and non-functional requirements should be considered already in the early test analysis phase.



The new method has been tried out and evaluated. It has been proved useful and fully applicable, and there are clear indications that it is beneficial, and that project lead time can be reduced by using it. We have therefore now started to apply this method in our new development projects.



The presentation includes examples of real cases where MBT has been used for verifying non-functional requirements.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
403
On SlideShare
0
From Embeds
0
Number of Embeds
87
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

'Model Based Test Design' by Mattias Armholt

  1. 1. MODEL BASED TEST DESIGN FORPERFORMANCE TESTING AND OTHERNON-FUNCTIONAL REQUIREMENTSMATTIAS ARMHOLTERICSSON AB
  2. 2. AgendaIntroductionModel Design TechniquesConclusionsModel Based Test Design for Performance Testingand other Non-Functional Requirements
  3. 3. IntroductionMy work› Function tester at Ericsson AB› Testing IP functionality in a middleware platform› Working with MBT and automation for 3 yearsEnvironment› Conformiq Modeler – Model Design› Conformiq Designer – Test generator› Java and TCL/Expect – Test automation framework
  4. 4. Model Based Test› Model based testing is a processused to generate abstract test casesfrom a model of the system undertest› The abstract test cases can then beconverted into executable test cases› We use state diagram based todesign test models from costumerrequirements› Test model reflects the expectedbehaviour of the product
  5. 5. IntroductionModel Based Test Design
  6. 6. WorkflowInput:Customer requirements,Function Specifications,Function Descriptions,Interwork Descriptions,etc.Modeldesign(UML) Modeling tool MBT tool(Conformiq)InputTestcasesGenerateGenerateTest documentationAutomatedtest casesConvertImportTest ManagementtoolTestresultsExecute
  7. 7. Non-Functional Requirements (NFR)› Capacity› Performance Requirements– Response time– Throughput– Processor-utilization› Interoperability– IP Standards› Robustness› Stability› And more …
  8. 8. Problems With NFRLookahead depth– Tool algorithm do not want torepeat the same transitionsmultiple times without fulfillingnew requirements or coveringnew states/transitions.Parameterisation– Requirements need transitionswith several parameters to firemany times.– The number of parametercombinations becomes unableto handle.– Would generate thousands oftest cases.100 times3 param5 param
  9. 9. Problems With NFRRobustness/Stability– Nothing new should happenduring test– No clear boundary value to testwith.Measurements– No transition available in SUTfor measurements– No clear boundary value to testwith.? times
  10. 10. Method For MBT Of NFR› Group the Non-Functional requirements based on similarities› Evaluate if the group is possible to include in the model› Design a test model including non-functional requirements› Generate Test Cases
  11. 11. NFR Testability› NFR requirements logic can be included in the testharness/environment– Use test applications for iterations– External equipment for interoperability– Add functionality to test harness– Increase SUT testability with test commands› Testability is one criteria for MBT of NFR
  12. 12. NFR Test Logic SupportSUTTESTEQUIPMENTTEST APPTEST HARNESSPERFORMANCEMANUALINSTRUCTIONSSTABILITYMBT MODELITERATIONS
  13. 13. NFR Model Design Techniques› Design the non-functional requirements in the model with– Requirement keyword– Ad hoc requirements– States– Transitions– Parameters
  14. 14. NFR Model Design Techniques› Group iterations together– Don’t create 1 host 100 times, create 100 hosts at 1 time– Removes risk of parametersation and lookahead depth– Reduces test case length makes it easier to read– Add logic in test harness or test environment
  15. 15. NFR Model Design Techniques
  16. 16. NFR Model Design Techniques› Use different abstraction levels– Focus the transitions to the parameters that counts for NFR– Use precondition when modeling NFR– Reduces the risk of unnecessary parameter combination testing
  17. 17. NFR Model Design TechniquesMODELPARAMETER
  18. 18. Conclusions› In order to develop a good model covering non-functionalrequirements, you need to practice and learn how the toolgenerate test cases› Support for testing of NFR must be possible to include inthe test harness or test environment› In general NFR increases logic and complexity in testharness and test environment
  19. 19. Conclusions› Model cost of NFR the same compared to functionalrequirements› Test harness/environment support development for NFRcost more compared to functional requirements› Most valuable when NFR and functional requirements aremodeled together› Gain maintenance cost by MBT for all requirements– Cost less to maintain model + test harness compared to separatetest scripts

×