Your SlideShare is downloading. ×
0
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Test design techniques: Structured and Experienced-based techniques
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Test design techniques: Structured and Experienced-based techniques

12,472

Published on

Test Design techniques: Structured and Experienced-based techniques

Test Design techniques: Structured and Experienced-based techniques

Published in: Technology
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,472
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
10
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • what is test case coverage? What is code coverage? How can you ensure that your test case covers all requirements? could you name some coverage items? + coverage items may be requirements, function calls, modules, objects, menu options, screens, buttons, transition steps, dropdown items, business transactions… + EP: percentage of equivalence partitions exercised + BVA: percentage of boundaries exercised + Decision tables: percentage of business rules or decision table columns tested + State transition testing: Percentage of states visited Percentage of (valid) transitions exercised Percentage of pairs of valid transitions exercised (1-switch coverage, 2-switch coverage) Percentage of invalid transitions exercised (state table)- What is the coverage interested by business analysts, testers or users?- What is the coverage interested by developers?
  • In reality, 100% statement coverage is time-consuming and could be achieved only by strict requirement from client. Normally, 80% of code coverage is good enough.
  • - To reach 100%SC, choose what test data? How about 1, 7? Test data? How many percentage? How about 1, 2, 6, 7? Test data? How many percentage?
  • Answer: aP = 40, Q = 70, %SC = ?  7/8P = 60, Q = 60, %SC = ?  8/8
  • Answer: b1, 2, 3, 4, 51, 2, 3, 5Outlook does not disappear, %DC = 50%
  • Reference BS7925-2, Software Testing Foundations (ISTQB)
  • a= (b+c) counts as 1 predicate
  • Answer: P = 4, C = 5
  • Answer: P = 5, C = 6
  • Answer: P = 5, C = 6
  • One drawback of code coverage measurement is that it measures coverage of what has been written, i.e. the code itself; it cannot say anything about the software that has not been written. If a specified function has not been implemented, specification-based testing techniques will reveal this. If a function was omitted from the specification, then experience-based techniques may find it. But structure-based techniques can only look at a structure which is already there. Best practice is to use all those techniques: spec-based and structure-based followed by exp-based testing techniques. Keep in the mind of objective of each technique.
  • Test objective is the most important
  • Transcript

    • 1. Test Design techniques: Structured and Experienced-based techniques<br />Author: Khuong Nguyen<br />
    • 2. Structured based testing techniques<br />Cyclomatic complexity<br />Experienced based testing techniques<br />Choosing appropriate testing techniques<br />Sample exam<br />Agenda<br />
    • 3. Understand and apply common Structured-Based Testing techniques<br />Know how to calculate Cyclomatic Complexity<br />Understand Experienced-Based Testing techniques<br />Learn to choose appropriate testing techniques in specific circumstances<br />Objectives<br />
    • 4. Software testing foundations<br />Certified Tester Foundation Level Syllabus 2007<br />Software Testing Dictionary<br />BS7925-2: Component Testing<br />References<br />
    • 5. Structure-based techniques<br />
    • 6. Structure-based testing/white-box testing is based on an identified structure of the software or system<br />It is a method for writing a set of white-box test cases that exercise the paths in the code <br />How to measure thoroughly the test cases exercise the code<br />Introduction<br />
    • 7. Test coverage measures in some specific way the amount of testing performed by a set of tests.<br />Wherever we can count things and can tell whether or not each of those things has been tested by some test, then we can measure coverage<br />What is test coverage?<br />
    • 8. Statement testing and coverage<br />Decision testing and coverage <br />Other structure-based techniques<br />Basic white box test case design techniques<br />
    • 9. In component testing, statement coverage is the assessment of the percentage of executable statements that have been exercised by a test case suite. (TRUE OR FALSE)<br />Statement testing derives test cases to execute specific statements, normally to increase statement coverage.<br />Every statement is requested to be executed at least one <br />Definition of statement coverage<br />
    • 10. Translate the source code to control flow graph<br />Defines nodes(Statement) and control flow between statements(edges)<br />Conditional statements(IF, CASE) and loops(WHILE, FOR) have more than one edges going out<br />Verify that each statement have been executed<br />Steps<br />
    • 11. Examples<br />IF<br />1<br />IF<br />2<br />WHILE<br />3<br />4<br />6<br />ENDIF<br />7<br />ENDIF<br />1 If (a&gt;0) then <br />2 If (b&lt;1) then<br />3 While (c&lt;2) <br />4 c++;<br />5 End while<br />End IF<br />7 End IF<br />5<br />Node, statement<br />Edge, Control flow<br />
    • 12. Statement coverage<br />IF<br />1<br />IF<br />2<br />WHILE<br />3<br />4<br />6<br />ENDIF<br />7<br />ENDIF<br />100%SC: All nodes can be reached by a single test case<br />1, 2, 3, 4, 5, 6, 7<br />5<br />
    • 13. It is a very weak criterion<br />Value of the technique:<br />Unreachable source code (dead code) can be detected<br />Empty ELSE-parts are not considered<br />Statement Coverage<br />Coverage is measured by using test tools<br />
    • 14. What is minimum number of test cases required for full statement coverage?<br />1 test for statement coverage<br />2 tests for statement coverage<br />3 tests for statement coverage<br />Read P<br />Read Q<br />IF P+Q &gt; 100 THEN<br /> Print “Large”<br />ENDIF<br />If P &gt; 50 THEN<br /> Print “P Large”<br />ENDIF<br />Exercise<br />
    • 15. Decision coverage, related to branch coverage, is the assessment of the percentage of decision outcomes (TRUE and FALSE)<br />Require every decision outcome: all possible cases-statement<br />Decision testing is a form of control flow testing as it generates a specific flow of control through the decision points<br />Definition of decision/branch coverage<br />
    • 16. Translate the source code to control flow graph<br />Defines nodes(Statement) and control flow between statements(edges)<br />Conditional statements(IF, CASE) and loops(WHILE, FOR) have more than one edges going out<br />Verify that all possible branches of the control flow are tested<br />Steps<br />
    • 17. It is a stronger criterion than statement coverage<br />Be able to detect missing statement in empty branches<br />Decision coverage is stronger than statement coverage: 100% decision coverage guarantees 100% statement coverage, but not vice versa.<br />Decision coverage requires test of every decision outcome:<br /> both THEN and ELSE in the IF-statement<br /> all possibilities for the CASE-statement and fall-through case<br /> both execution of the FOR loop and the bypass<br />Decision/branch Coverage<br />
    • 18. Examples<br />IF<br />1<br />IF<br />2<br />WHILE<br />3<br />4<br />6<br />ENDIF<br />7<br />ENDIF<br />1 If (a&gt;0) then <br />2 If (b&lt;1) then<br />3 While (c&lt;2) <br />4 c++;<br />5 End while<br />6 End IF<br />7 End IF<br />5<br />Node, statement<br />Edge, Control flow<br />
    • 19. Decision coverage<br />IF<br />1<br />IF<br />2<br />WHILE<br />3<br />4<br />6<br />ENDIF<br />7<br />ENDIF<br />100%DC: All branches are executed<br />Some edges are executed more than one<br />5<br />1, 2, 3, 5, 3, 4, 5, 6, 7<br />1, 2, 6, 7<br />1, 7<br /><ul><li>Branch coverage = (Number of executed branches / total number of branches) * 100%</li></li></ul><li>What is minimum number of test cases required for full decision coverage?<br />1 test for decision coverage<br />2 tests for decision coverage<br />3 tests for decision coverage<br />Switch PC on<br />Start “outlook”<br />IF outlook appears THEN<br /> Send an email<br /> Close outlook<br />Endif<br />Exercise<br />
    • 20. For object-oriented system, statement as well as decision coverage is inadequate.<br /><ul><li>Additional coverage criteria are necessary</li></ul> Tools often support determining coverage, coverage data can be used to detect uncalled methods or program parts<br />Limitations of the techniques<br />
    • 21. Data flow Testing<br />Branch Condition Testing<br />Branch Condition Combination Testing<br />Modified Condition Decision Testing<br />LCSAJ Testing<br />Other structure-based test techniques<br />
    • 22. There are stronger levels of structural coverage beyond decision coverage (condition coverage and multiple condition coverage)<br />The concept of coverage can also be applied at other test levels (e.g. at integration level) where the percentage of modules, components or classes that have been exercised by a test case suite could be expressed as module, component or class coverage<br />Tool support is useful for the structural testing of code<br />The basis of white box techniques is the source code. Adequate test case design techniques chosen depending on the complexity of the program structure<br />Other structure-based test techniques (cont.)<br />
    • 23. Testing contains the combination of different techniques<br />The expected risk guides to select testing techniques and the intensity of execution<br />Basis of the selection of the white box technique is the structure of the test object<br />Which techniques should be chosen?<br />
    • 24. Cyclomatic Complexity<br />
    • 25. Cyclomatic complexity or conditional complexity:<br />Was developed by McCabe <br />Is the software metric (measurement) to measure the complexity of the program’s source code<br />Definition<br />
    • 26. M = E − N + 2P <br />M = cyclomatic complexity<br />E = the number of edges (links) of the graph<br />N = the number of nodes of the graph<br />P = the number of connected components<br />Number of predicates + 1<br />How to calculate<br />
    • 27. Examples of cyclomatic complexity<br />E=1, N=2, P=1<br />M=1-2+2=1<br />E=4, N=4, P=1<br />M=4-4+2=2<br />E=2, N=4, P=2<br />M=2-4+4=2<br />E=4, N=5, P=1<br />M=4-5+2=1<br />
    • 28. Cyclomatic complexity of programming constructs<br />1<br />1 if E then<br />2 A<br /> else <br /> B<br />4 C<br />2<br />3<br />4<br />E = 4, N = 4, P =1 <br />M = 2<br />
    • 29. C<br />
    • 30. C<br />
    • 31. Predicate notes P = ?<br />Complexity C = ?<br />
    • 32. Predicate notes P = ?<br />Complexity C = ?<br />
    • 33. Predicate notes P = ?<br />Complexity C = ?<br />
    • 34. Complex systems are<br />hard to understand<br />hard to change<br />hard to reuse<br />McCabe found that modules with a cyclomatic complexity greater than 10 were hard to test and error prone.<br />Look for procedures with high cyclomatic complexity and rewrite them, focus testing on them, or focus reviewing on them.<br />Metrics view<br />
    • 35. Experience-based techniques<br />
    • 36. Experienced-based test design technique: Procedure to derive and/or select test cases based on the tester’s experience, knowledge and intuition<br />The techniques can be useful in identifying special tests not easily captured by formal techniques, especially when applied after more formal approaches<br />Definition<br />
    • 37. Tester select test cases to uncover expected problems based on the skills, experienced and knowledge of the tester<br />The term of error guessing is used very often in practice<br />A commonly used experienced-based technique is error guessing<br />Introduction<br />
    • 38. Exploratory testing<br />Error guessing<br />Basic experience-based test techniques<br />
    • 39. Error guessing is a test design technique where the experience of the tester is used to anticipate what defect might be present in the system under test.<br />Testers design tests specifically to expose errors<br />Error Guessing - Definition<br />
    • 40. Error guessing is a technique that should be used as a complement to other formal techniques<br />Testers anticipate defects based on their experience, skills<br />Knowledge in developing similar applications and using similar technologies are also used when designing test cases.<br />“Guess” where errors happened based on intuition and experience of tester.<br />Error guessing<br />
    • 41. These defect and failure lists can be built based on experience, available defect and failure data, and from common knowledge about why software fails.<br />Error guessing<br />
    • 42. Exploratory testing is test design technique where the tester actively controls the design of the test as those tests are performed and uses information gained while testing to design new and better test<br />Exploratory testing is exploring, finding out about the software:<br />What it does<br />What it doesn’t do<br />What works<br />What doesn’t work<br />Definition of Exploratory testing<br />
    • 43. Exploratory testing is a hands-on approach in which testers are involved in minimum planning and maximum test execution<br />Activities in exploratory testing is concurrent:<br />Test planning <br />Test design<br />Test execution<br />Test logging<br />Learning<br />Main features of exploratory testing <br />
    • 44. Result of one test case influence the design and execution of further test cases<br />During testing, a “mental” model of program under test is created. It contains:<br />How the program works<br />How it behaves <br />How it should behave<br />Focus on finding further aspect/behavior of the program.<br />Main features of exploratory testing<br />
    • 45. Planning: involve the creation of the test charter that includes:<br />Test charter<br />Short declaration of scope<br />Time-boxed of effort<br />Objectives and possible approaches to be used<br />Test design and test execution are performed parallel:<br />Do not have formal documents: test conditions, test cases, test scripts…<br />Some notes written during exploratory testing session<br />Test logging is undertaken as test execution is performed.<br />The key aspect of exploratory testing is learning about the software, its strengths and weaknesses<br />How to implement<br />
    • 46. Approach<br />Execute few test cases<br />Analyze results<br />Consider “special” to next test cases<br />Knowledge about the test object under test is collected<br />
    • 47. Possible elements of the test object are “explored” to decide which parts will be tested<br />Result of Exploratory testing helps to determine which test techniques can be applied if there is time left<br />The test of “test charter” should not take more than one or two hours of uninterrupted test time<br />Benefit<br />
    • 48. The technique is applicable when:<br />There no poor specification<br />The time is severely restricted because it uses much less time than other techniques<br />It can serves to complement formal testing<br />It can serve to check on the test process, to help ensure that the most serious defects are found.<br />When we use exploratory testing<br />
    • 49. Why? With which goal is the test run?<br />What is to be tested?<br />How? Which testing method should be used?<br />Which problems should be found?<br />Common questions for executing test charter<br />
    • 50. Choosing test techniques<br />
    • 51. What are the pros and cons of:<br />Specification-based testing techniques<br />Structure-based testing techniques<br />Experience-based testing techniques<br />What is the best practice?<br />When to use what?<br />
    • 52. The choice of which test techniques to use depends on a number of factors:<br />the type of system<br />regulatory standards<br />customer or contractual requirements<br />level of risk, type of risk<br />test objective, documentation available<br />knowledge of the testers<br />time and budget<br />development life cycle, use case models<br />previous experience of types of defects found. <br />Which techniques should be chosen?<br />
    • 53. Testing contains the combination of different techniques<br />The critically and the expected risk guide to select testing techniques and the intensity of execution<br />Basis of the selection of the white box technique is the structure of the test object<br />Which techniques should be chosen?<br />
    • 54. Thank You<br />KhuongNguyen, Senior Quality Control Engineer<br />Post graduate student, Australian Business School, Funds Management, UNSW, 2032, AustraliaEmail: khuong0602@gmail.com; op_khuong_nguyen@yahoo.com<br />

    ×