Upcoming SlideShare
×

# Muffler a tool using mutation to facilitate fault localization 2.0

358
-1

Published on

Published in: Technology
0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total Views
358
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
2
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Muffler a tool using mutation to facilitate fault localization 2.0

1. 1. Muffler: An Approach Using Mutationto Facilitate Fault Localization Tao He elfinhe@gmail.com Software Engineering Laboratory Department of Computer Science, Sun Yat-Sen University Department of Computer Science and Engineering, HKUST November 2011 Sun Yat-Sen University, Guangzhou, China 1/34
2. 2. Key hypothesis Mutating the faulty statement tends to maintain the outcome of passed test cases.  Only one statement is faulty. By contrast, mutating a correct statement tends to toggle the outcome of passed test cases (from passed to failed).  Two statements are faulty. Intuition  Two faulty statements can trigger more failures than one faulty statement. 2/34
3. 3. Intuition In branches F M M: Mutant point F: Fault point 3/34
4. 4. Intuition In branches M F M: Mutant point F: Fault point 4/34
5. 5. Intuition In branches F +M M: Mutant point F: Fault point 5/34
6. 6. Intuition In series M F M: Mutant point F: Fault point 6/34
7. 7. Intuition In series F +M M: Mutant point F: Fault point 7/34
8. 8. A motivating example Siemens – tcas v27 8/34
9. 9. A motivating example  Siemens – tcas v27  Golden version – line 118  enabled = High_Confidence && (Own_Tracked_Alt_Rate <= OLEV) && (Cur_Vertical_Sep > MAXALTDIFF);  Fault version – line 118  enabled = High_Confidence && (Own_Tracked_Alt_Rate <= OLEV) ;  // missing ‘(Cur_Vertical_Sep > MAXALTDIFF)’enabled = High_Confidence && (Own_Tracked_Alt_Rate <= OLEV) && (Cur_Vertical_Sep > MAXALTDIFF); enabled = High_Confidence && (Own_Tracked_Alt_Rate <= OLEV) ; 9/34
10. 10. A motivating example  Suspiciousness by OchiaiBest rank Worst rank 10/34
11. 11. A motivating example  Result changes by MufflerVioplot from tcas v27 Vioplot from tcas v27Y-axis: # of changes from passed to failed Y-axis: proportion of changes from passed to failedRed line: fault line’s change Red line: fault line’s change 11/34
12. 12. A motivating example Formula  Susp(s) = Failed(s) * TotalPassed - Passed(s) – PassedToFailed(s) Improve fault’s worst rank 12/34
13. 13. Formula Susp(s) = Failed(s) * TotalPassed - Passed(s) – PassedToFailed(s) Primary Key Secondary Key Additional Key (imprecise when (invalid when (inclined multiple faults coincidental to handle occurs) correctness% coincidental is high) correctness) 13/34
14. 14. Formula Susp(s) = Failed(s) * TotalPassed - Passed(s) – PassedToFailed(s) Coincidental PtoF (f) correctness: i.e. P(f) Passed PtoF (c) After mutating each executable statement Failed Total Tests PtoF (c) is empirically greater than PtoF(f) 14/34
15. 15. Model of our approach - Muffler Simple and clear target of each input and step List all possible research questions  We first focus on the key issue, and leave the others as future work. 15/34
16. 16. Model of our approach – Muffler 16/34
17. 17. Input variables Program Test suite Mutant operators (No more) 17/34
18. 18. Program Program  Number of faults  Failure-triggering ability by covering this statement (Is it easy to cause coincidental correctness?)  Fault type  Statement type of fault  Fault position  Program structure  Program size  Etc. 18/34
19. 19. Test suite Number of total test cases Number of passed/failed test cases Number of passed test cases cover each statement to be mutated  test cases that may alter testing results Number of statements covered by each failed test case  Help us select suspicious statements. Can we locate faults without test oracle? 19/34
20. 20. Mutant operators Types of mutation operators Applicable condition for each operator The probability of failure-triggering by covering mutant point  The ability for mutant to kill test cases 20/34
21. 21. Steps I list steps for separation of concerns  This can help us locate where our hypothesis is. 21/34
22. 22. Step I: execute test cases Get testing results Get coverage  Of failed test cases  For the selection of suspicious statements  Of passed test cases  For the selection of test cases to re-run mutants 22/34
23. 23. Step II: select statements to mutate Input  Coverage of failed test cases Output  Statements to be mutated  Covered by least-coverage failed test case Discussion  Multi-fault scenarios  Our approach depend more on passed test cases  Practical situation (E.g., in gcc bug reports, most faults are reported with one failure)  Precision of failure clustering is … 23/34
24. 24. Step III: mutate selected statements Input  Program  Statements to be mutated  Mutant operators Output  Mutants by mutating each suspicious statement Discussion  How many mutants to take from each suspicious statement?  What if there is no applicable mutant operator for this statement? (I.e., no mutant is generated.)  What if all mutants of a statement do not change the passed testing results to failed? (I.e., mutation impact is 0.0.) Two possible reasons:  Equivalent mutants  Coincidental correctness 24/34
25. 25. Step IV: select passed test cases Select passed test cases that cover the mutant point Select passed test cases that cover less statements Reduce the number of passed test cases by discarding similar test cases 25/34
26. 26. Step V: run mutants against passed testcases that cover the mutated statement Input  Mutants for each suspicious statement  Selected passed test cases on original program Output  Number of failed test cases from running on each mutant 26/34
27. 27. Step VI: weight statements By dynamic impact By clustering mutant statements by analyzing program structure 27/34
28. 28. Step VII: compute mutation impactand rank the suspicious Discussion  Why not use failed test cases?  How to design the formula of mutation impact?  What if the number of passed test cases that cover the mutant point is 0? 28/34
29. 29. Robust with potential issues in CBFL? Coincidental Correctness Multi-fault Coverage equivalence 29/34
30. 30. Coincidental Correctness When coincidental correctness frequently occurs, our approach can work. When coincidental correctness rarely occurs, our approach is as good as CBFL.  This is why we use ‘– PassedToFailed(s)’, rather than ‘– PassedToFailed(s)/Passed(s)’ 30/34
31. 31. Multi-fault CBFL techniques mainly rely on the coverage of passed test cases and failed test cases caused by the same fault. (More on the failed test cases) When applying to multi-fault scenarios, the different coverage between failed test cases caused by different fault will affect the effectiveness. Our approach more depend on passed tests. 31/34
32. 32. Equivalent coverage Inside a basic block, different statements have different def-use pair impact on statements outside the basic block. 32/34
33. 33. Q&A 33/34
34. 34. Thank you!Contact me via elfinhe@gmail.com 34/34
35. 35. 35/34
36. 36. Guideline of our approach Simple and clear target of each step List all possible research questions  We first focus on the key issue, and leave the others as future work. 36/34
37. 37. 37/34
1. #### A particular slide catching your eye?

Clipping is a handy way to collect important slides you want to go back to later.