16-12-2009 English presentation about a pilot on Model-Based Testing using Smartesting Test Designer.

1. Pilot Model Based Testingusing Smartesting Test DesignerTM3/12/08 - 5/12/08

2. Agenda Background Objective of the pilot Findings Conclusions & Recommendations

3. Background (1/2) Expectations of model based testing... Increased speed in test case specification Excellent first step towards automated test execution Transferable and reusable testware opportunity in combination with India route? Improved involvement in requirements review Shift up in the V-model a possible new service offering to customers and/or in testlines

4. Background (2/2) Activities Modelling by Smartesting Turn around time: 3 days FEBO Front end application for .... Requirements used : release 2.1 (2.2 currently being built) .NET application, back end simulated by stubs Vista based

5. Objective of the pilot (1/2) In general To gain hands on experience with Model Based Testing (MBT) Specifically To assess the learning curve of the UML models used Class diagram State diagram Object diagram OCL

6. Objective of the pilot (2/2) To assess maintenance effort testware To assess alignment with Test Strategy decisions How can test design techniques be utilized; What coverage can be achieved; How can coverage be influenced. To compare effort and outcome with ‘normal’ test specification Time spent on preparation Time spent on creating test cases Number of test cases created

7. Findings (1/5) and 2. Hands on experience;Learning curve Due to reduced scope and turn around time: no real result Maintenance Requirement change incorporated in model Quick change

8. Findings (2/5) Alignment with Test Strategy Only possible to tailor (automated) coverage with test depth in PCT Test design techniques that can be used are: PCT, UCT, DCoT, SEM, SYN, DTT, ECT To combine MBT with more test design techniques the maintenance is under pressure Two separate activities

9. Findings (3/5) Comparison to manual test specification Time (hours)

10. Findings (4/5) Comparison to manual test specification Test cases

11. Findings (5/5) Extra: Infrastructure Windows Vista not supported by Smartesting Test Designer cannot be installed on Windows Vista Sogeti Need license for IBM Rational Software Modeler Export to HP Quality Center: OK Export to IBM Raitional Quality Manager: not yet available

12. Conclusions & Recommendations (1/5) Overall: Positive introduction to small scale pilot: no statistical metrics possible Good and clear interaction with analyst/designer Same level of knowledge Increases results tester Increases acceptance designer Find defects in requirements

13. Test Design Techniques PCT: very suitable (adjustable test depth) DCoT/SEM/SYN: auditable with adjustments in test data More analyses/experience is needed to optimize the use of test design techniques with MBT Manual execution of model bases test cases is easy Recommendation to use meaningful variables focus not only on automated test execution but also manually Extra service testers Sogeti? Conclusions & Recommendations (2/5)

14. Current infrastructure not suitable for Test Designer Windows Vista Sogeti Support Windows Vista No export to Rational Quality Manager OCL looks a lot like programming Not every tester will have the required skills Extra skill required/extra role Suitable for outsourcing/offshoring Create models internally Create and execute test cases externally Conclusions & Recommendations (3/5)

15. No discussion about who creates model In test model there are items not used in design model Booleans Test data Customizing model takes longer Creating model filters defects in design Fully independent Conclusions & Recommendations (4/5)

16. Conclusions & Recommendations (5/5) Prolong the pilot by course in MBT Creating models and test cases Teach this knowledge Use this knowledge in a real testline Small project Add to services Software Control Integrate in STaaS

17. Review Models (UML) Req’s Risks Review Test Repository Test data TestManagementtool TestAutomation Test Cases Testbasis Testgevallen