Mc Dc In A Nutshell


Published on

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Mc Dc In A Nutshell

    1. 1. MC/DC in a nutshell Christopher Ackermann
    2. 2. What is MC/DC? <ul><li>Unrelated to AC/DC! </li></ul><ul><li>MC/DC stands for M odified </li></ul><ul><li>C ondition / D ecision C overage </li></ul><ul><li>Some kind of Predicate Coverage technique </li></ul><ul><li>Some terms: </li></ul><ul><ul><li>Condition: Leaf level Boolean expression. </li></ul></ul><ul><ul><li>Decision: Controls the program flow. </li></ul></ul><ul><li>Main idea: Each condition must be shown to independently affect the outcome of a decision, i.e. the outcome of a decision changes as a result of changing a single condition. </li></ul>
    3. 3. 1. Requirement <ul><li>The decision has taken all possible outcomes at least once. </li></ul><ul><li>We could also say we cover both the true and the false branch (like we do in Branch Testing). </li></ul>If ((a) & (b) | (c)) then… true false
    4. 4. 2. Requirement <ul><li>Every condition in the decision has taken all possible outcomes at least once. </li></ul><ul><li>Aims to cover compound conditions (like Condition/Decision Coverage) </li></ul>If ((a) & (b) | (c)) then… true false true false true false
    5. 5. 3. Requirement <ul><li>Every condition in the decision independently affects the decision’s outcome. </li></ul><ul><li>Not like anything we have talked about, so let’s elaborate… </li></ul>If ((a) & (b) | (c)) then… true false true false Change the value of each condition individually while keeping all other conditions constant.
    6. 6. Creating MC/DC test cases <ul><li>If (A and B) then… </li></ul><ul><li>(1) create truth table for conditions. </li></ul><ul><li>(2) Extend truth table so that it indicated which test cases can be used to show the independence of each condition. </li></ul>
    7. 7. Creating test cases cont’d <ul><li>Show independence of A : </li></ul><ul><ul><li>Take 1 + 3 </li></ul></ul><ul><li>Show independence of B : </li></ul><ul><ul><li>Take 1 + 2 </li></ul></ul><ul><li>Resulting test cases are </li></ul><ul><ul><li>1 + 2 + 3 </li></ul></ul><ul><ul><li>(T , T) + (T , F) + (F , T) </li></ul></ul>
    8. 8. More advanced example <ul><li>If (A and (B or C)) then… </li></ul><ul><li>Note: We want to determine the MINIMAL set of test cases </li></ul><ul><li>Here: </li></ul><ul><li>{2,3,4,6} </li></ul><ul><li>{2,3,4,7} </li></ul><ul><li>Non-minimal set is: </li></ul><ul><li>{1,2,3,4,5} </li></ul>
    9. 9. Where does it fit in? <ul><li>The MC/DC criterion is much stronger than the condition/decision coverage criterion, but the number of test cases to achieve the MC/DC criterions still varies linearly with the number of conditions n in the decisions. </li></ul><ul><ul><li>Much more complete coverage than condition/decision coverage, but </li></ul></ul><ul><ul><li>at the same time it is not terribly costly in terms of number of test cases. </li></ul></ul>
    10. 10. Object Code Coverage <ul><li>In some situations the compiler performs a translation of each conditions into a single decision (conditional jump) in the object code. </li></ul><ul><li>The minimal test set for condition/decision coverage does not provide decision coverage of the generated object code. ({1,8},{2,7},{3,6}) </li></ul><ul><li>MC/DC ensures decision coverage of the underlying object code. ({2,3,4,6} or {2,3,4,7}) </li></ul>
    11. 11. Advantages of MC/DC <ul><li>Linear growth in required tests: </li></ul><ul><ul><li>For an expression with N uncoupled conditions, the modified condition/decision criterion can be met with a minimum of N + 1 tests by varying exactly one condition of each of the first N tests. </li></ul></ul><ul><li>Object code coverage: </li></ul><ul><ul><li>The MC/DC criterion ensures a much higher level of decision coverage of the object code than either decision or condition/decision coverage. </li></ul></ul><ul><li>Operand sensitivity: </li></ul><ul><ul><li>Extraneous operands that cannot independently affect the outcome are found during construction of the test set </li></ul></ul><ul><li>Sensitivity to non-equivalent functions: </li></ul>
    12. 12. Efficiency <ul><li>For a given function of N operands and any M distinct tests, the probability P (N,M) of detecting an error in an incorrect implementation of a Boolean expression with N conditions is given by </li></ul><ul><li>Probability of finding an error increases as N increases! </li></ul>
    13. 13. Efficiency cont’d <ul><li>This suggests that breaking down compound conditions has a negative effect on the results of MC/DC based testing. </li></ul>
    14. 14. Argument in the software industry <ul><li>Federal Aviation Administration’s requirement that test suites be MC/DC adequate (DO-178B). </li></ul><ul><li>In the avionics domain, complex Boolean expressions are much more common. </li></ul>
    15. 15. Argument in the software industry <ul><li>Federal Aviation Administration’s requirement that test suites be MC/DC adequate. </li></ul><ul><li>Argument – Too expensive </li></ul><ul><ul><li>“ For example, one of our industrial partners reports that for one of its products of about 20,000 lines of code, the MC/DC-adequate test suite requires seven weeks to run.” </li></ul></ul><ul><li>Counter Argument – Significantly more effective </li></ul><ul><ul><li>A recent empirical study performed during the real testing of the attitude-control software for the HETE-2 (High Energy Transient Explorer) found that the test cases generated to satisfy the MC/DC coverage requirement detected important errors not detectable by functional testing. </li></ul></ul>
    16. 16. Conclusion <ul><li>Critics lead to research work that tries to reduce testing effort through </li></ul><ul><ul><li>Test suite reduction </li></ul></ul><ul><ul><li>Test suite prioritization </li></ul></ul><ul><li>Question: What is the difference between MC/DC and the technique discussed in the Predicat3e Paper? </li></ul><ul><li>Paper: “ Much of the benefit of the MC/DC criterion lies not so much in the specific tests selected as in the requirement that at least N + 1 tests are executed ” </li></ul><ul><ul><li>Question: Why not just randomly pick N+1 test cases then? </li></ul></ul>
    17. 17. References <ul><li>J.J. Chilenski and S.P. Miller, “ Applicability of Modified Condition/Decision Coverage to Software Testing ,” Software Eng. J., vol. 9, no. 5, pp. 193-200, 1994. </li></ul><ul><li>D. Richard Kuhn. “ Fault classes and error detection capability of specification-based testing ” ACM Transactions on Software Engineering and Methodology, 8(4):411--424, October 1999. </li></ul><ul><li>Jones, J. and Harrold, M. “ Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage ”, Proceedings of the IEEE International Conference on Software Maintenance ( ICSM'01), Florence, Italy, 7-9 November 2001, pp. 92--101. </li></ul><ul><li>Dupuy, A. and Leveson, N. “ An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software ”, Proceedings of the Digital Aviation Systems Conference (DASC), Philadelphia, USA, October 2000. </li></ul>