Testing, object-oriented programs, anti-patterns

1. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Outline
Motivations
Goal
Empirical Study
Results
Other Findings
Conclusion
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
1 / 26

2. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Motivations
What are Antipatterns?
Identify poor solutions to recurring design problems[Brown et al.]
Some causes
Time-market pressure
Insufficient knowledge and or experience
Misapplication of some design patterns
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
2 / 26

3. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Motivations
Impact of Antipatterns...
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
3 / 26

4. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Motivations
What about Testability and Testing?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
4 / 26

5. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Goal
Study the impact of antipatterns on class testability
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
5 / 26

6. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
Class Testability Measure
What is Testability?
Degree to which a system or component facilitates the
establishment of test criteria and performance of tests to
determine whether those criteria have been met [IEEE
Glossary]
Testing effort [Bache et al.]
How to Measure Testability?
Metric-Based approaches [Bruntink et al.]
number of test cases required to satisfy a given coverage
criterion [Bache et al.]
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
6 / 26

7. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
MaDUM
What is it?
Unit testing technique for object oriented programs [Bashir et
al.]
Based on data slices
Core idea: The correctness of a class is equivalent to the
correctness of all its slices
Focus on the test of methods interactions
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
7 / 26

8. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
MaDUM
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
8 / 26

9. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
MaDUM
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
9 / 26

10. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
Why MaDUM?
Why MaDUM?
Specific to OO programs
Not require specific design documentation
Possibility to compute the number of required test cases
Upper-bound for testing cost
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
10 / 26

11. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
Objects
4 systems: Ant, ArgoUML, Checkstyle & JFreechart
Systems belong to different application domains, have
bug-fixing data available and have been used in previous
studies [Abbes et al.], [Bruntink et al.]
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
11 / 26

12. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
Subjects
13 antipatterns
Studied in previous studies [Khomh et al.], [Abbes et al.],
Detection tool: Decor [Moha et al.]
Name (Abbr)
Antipattern classes
No Antipattern (None)
Ant
452
297
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
ArgoUML
901
376
CheckStyle
161
99
JFreeChart
245
233
March 07th, 2013
12 / 26

13. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
RQ1: Assess AP class impact on testing effort?
How large is the MaDUM test suite for classes participating in APs
compared to that of other classes?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
13 / 26

14. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
RQ2: Which APs are the most expensive
How does the size of the MaDUM test suite vary among classes
participating in different kinds of APs?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
14 / 26

15. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Class Testability Measure
MaDUM
Objects
Subjects
RQ1: Assess AP class impact on testing effort?
RQ2: Which APs are the most expensive
RQ3: Cost-effectiveness of APs testing
RQ3: Cost-effectiveness of APs testing
What is the potential cost-benefit achieved when focusing testing
on APs, as opposed to other classes?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
15 / 26

16. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
RQ1
RQ2
RQ3
Results
Results...
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
16 / 26

17. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
RQ1
RQ2
RQ3
RQ1: How large is the MaDUM test suite for classes
participating in APs compared to that of other classes?
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
0
# of Test Cases (<=100)
20
40
60
80
q
Classes
System
AP
NAP
Ant
AP
NAP
ArgoUML
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
AP
NAP
Checkstyle
AP
NAP
JFreeChart
March 07th, 2013
17 / 26

18. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
RQ1
RQ2
RQ3
RQ1: How large is the MaDUM test suite for classes
participating in APs compared to that of other classes?
System
Ant
ArgoUML
CheckStyle
JFreeChart
Mean TCs AP
18
10
9
26
Mean TCs NAP
9
3
6
13
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
p-Value
< 0.01
< 0.01
= 0.01
< 0.01
Cliff’s d
0.23 (Small)
0.35 (Medium)
NA
0.22 (Small)
March 07th, 2013
18 / 26

19. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
RQ1
RQ2
RQ3
100
RQ2: How does the size of the MaDUM test suite vary
among classes participating in different kinds of APs?
80
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
60
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
20
40
q
q
q
q
q
q
q
q
q
0
# of Test Cases (<=100)
q
AS BCSBA B CDSBP CC
LzC
LM
LPL
MC
RPB
SC
SG
SAK NONE
Kind of Antipattern
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
19 / 26

20. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
RQ1
RQ2
RQ3
100
RQ2: How does the size of the MaDUM test suite vary
among classes participating in different kinds of APs?
80
60
q
q
40
q
q
20
# of Test Cases (<=100)
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
0
q
AS BCSBA B CDSBP CC
LzC
LM
LPL
MC
RPB
SC
SG
SAK NONE
Kind of Antipattern
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
20 / 26

21. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
RQ1
RQ2
RQ3
80 100
60
40
20
0
Defects in tested classes (cum.)
RQ3: What is the potential cost-benefit achieved when
focusing testing on APs, as opposed to other classes?
0
1000
2000
3000
4000
5000
6000
7000
# of T
est Case (cum.)
It is cost-effective to analyze/test APs classes with a higher priority
than other classes
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
21 / 26

22. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Other Findings: Refactoring for Reducing Testing Cost
Effect of traditional refactoring on testing cost?
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
22 / 26

23. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Other Findings: Refactoring for Reducing Testing Cost
Before
refactoring
Class (system)
Type TRS
TokenFilter (Ant)
CDSBP
5
PropPanel (ArgoUML)
Blob
5
BooleanExpressionComplexityCheck
LPL
6
(Checkstyle)
AxisState (JFreeChart)
NAP
5
DynamicTimeSeriesCollection Blob
4
(JFreeChart)
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
After
refactoring
TCs
TRS
263
2
271
3
TCs
27
43
732
5
132
248
208
1
2
11
122
March 07th, 2013
23 / 26

24. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Conclusion
Findings
Most of APs classes are expensive for unit testing
APs classes with excess of responsibilities require more testing
effort than other
Prioritize the testing of APs classes can be more cost-effective
Specific refactoring can help to reduce testing cost
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
24 / 26

25. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Conclusion
Future Work
Study more systems
Investigate the symptoms that make APs classes more
expensive
Study the impact of APs on testing effort from other aspects
Investigate specific refactoring for testability
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
25 / 26

26. Motivations
Goal
Empirical Study
Results
Other Findings: Refactoring for Reducing Testing Cost
Conclusion
Thank you very much for your attention!
A. Saban´, M. Di Penta, G. Antoniol, Y-G. Gu´h´neuc
e
e e
March 07th, 2013
26 / 26