Pilot Model Based Testing


Published on

English presentation about a pilot on Model-Based Testing using Smartesting Test Designer.

Published in: Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Pilot Model Based Testing

  1. 1. Pilot Model Based Testingusing Smartesting Test DesignerTM3/12/08 - 5/12/08<br />
  2. 2. Agenda<br />Background<br />Objective of the pilot<br />Findings<br />Conclusions & Recommendations<br />
  3. 3. Background (1/2)<br />Expectations of model based testing...<br />Increased speed in test case specification<br />Excellent first step towards automated test execution<br />Transferable and reusable testware<br />opportunity in combination with India route?<br />Improved involvement in requirements review<br />Shift up in the V-model<br /> a possible new service offering to customers and/or in testlines<br />
  4. 4. Background (2/2)<br />Activities<br />Modelling by Smartesting<br />Turn around time: 3 days<br />FEBO<br />Front end application for ....<br />Requirements used : release 2.1 (2.2 currently being built)<br />.NET application, back end simulated by stubs<br />Vista based<br />
  5. 5. Objective of the pilot (1/2)<br />In general<br />To gain hands on experience with Model Based Testing (MBT)<br />Specifically<br />To assess the learning curve of the UML models used<br />Class diagram<br />State diagram<br />Object diagram<br />OCL<br />
  6. 6. Objective of the pilot (2/2)<br />To assess maintenance effort testware<br />To assess alignment with Test Strategy decisions<br />How can test design techniques be utilized;<br />What coverage can be achieved;<br />How can coverage be influenced.<br />To compare effort and outcome with ‘normal’ test specification<br />Time spent on preparation <br />Time spent on creating test cases<br />Number of test cases created<br />
  7. 7. Findings (1/5)<br />and 2. Hands on experience;Learning curve<br />Due to reduced scope and turn around time: no real result<br />Maintenance<br />Requirement change incorporated in model<br />Quick change<br />
  8. 8. Findings (2/5)<br />Alignment with Test Strategy<br />Only possible to tailor (automated) coverage with test depth in PCT<br />Test design techniques that can be used are:<br /> PCT, UCT, DCoT, SEM, SYN, DTT, ECT<br />To combine MBT with more test design techniques the maintenance is under pressure<br /> Two separate activities<br />
  9. 9. Findings (3/5)<br />Comparison to manual test specification<br />Time (hours)<br />
  10. 10. Findings (4/5)<br />Comparison to manual test specification<br />Test cases<br />
  11. 11. Findings (5/5)<br />Extra: Infrastructure<br />Windows Vista not supported by Smartesting<br /> Test Designer cannot be installed on Windows Vista Sogeti<br />Need license for IBM Rational Software Modeler<br />Export to HP Quality Center: OK<br />Export to IBM Raitional Quality Manager: not yet available<br />
  12. 12. Conclusions & Recommendations (1/5)<br />Overall:<br />Positive introduction<br />to small scale pilot: no statistical metrics possible<br />Good and clear interaction with analyst/designer<br />Same level of knowledge<br />Increases results tester<br />Increases acceptance designer<br />Find defects in requirements<br />
  13. 13. Test Design Techniques<br />PCT: very suitable (adjustable test depth)<br />DCoT/SEM/SYN: auditable with adjustments in test data<br />More analyses/experience is needed to optimize the use of test design techniques with MBT<br />Manual execution of model bases test cases is easy<br />Recommendation to use meaningful variables<br />focus not only on automated test execution but also manually<br />Extra service testers Sogeti?<br />Conclusions & Recommendations (2/5)<br />
  14. 14. Current infrastructure not suitable for Test Designer<br />Windows Vista Sogeti<br />Support Windows Vista<br />No export to Rational Quality Manager<br />OCL looks a lot like programming<br />Not every tester will have the required skills<br />Extra skill required/extra role<br />Suitable for outsourcing/offshoring<br />Create models internally<br />Create and execute test cases externally<br />Conclusions & Recommendations (3/5)<br />
  15. 15. No discussion about who creates model<br />In test model there are items not used in design model<br />Booleans<br />Test data<br />Customizing model takes longer<br />Creating model filters defects in design<br />Fully independent<br />Conclusions & Recommendations (4/5)<br />
  16. 16. Conclusions & Recommendations (5/5)<br />Prolong the pilot by course in MBT<br />Creating models and test cases<br />Teach this knowledge<br />Use this knowledge in a real testline<br />Small project<br />Add to services Software Control<br />Integrate in STaaS<br />
  17. 17. Review<br />Models<br /> (UML)<br />Req’s<br />Risks<br />Review<br />Test<br />Repository<br />Test data<br />TestManagementtool<br />TestAutomation<br />Test Cases<br />Testbasis<br />Testgevallen<br />