CSMR11a.ppt

289 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
289
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CSMR11a.ppt

  1. 1. Antipatterns and Comprehension Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction An Empirical Study of the Impact of TwoRelated Work Antipatterns on Program ComprehensionExperimentalStudyStudy Results Marwen Abbes, Foutse Khomh,Discussions Yann-Ga¨l Gu´h´neuc, and Giuliano Antoniol e e eConclusion andFuture WorkQuestions ´ DGIGL, Ecole Polytechnique de Montr´al, Qu´bec, Canada e eReferences E-mails: {marwen.abbes, yann-gael.gueheneuc, giuliano.antoniol}@polymtl.ca, foutse.khomh@queens.ca CSMR 2011 February 3, 2011 Pattern Trace Identification, Detection, and Enhancement in Java SOftware Cost-effective Change and Evolution Research Lab
  2. 2. Antipatterns and Comprehension Outline Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction IntroductionRelated WorkExperimental Related WorkStudyStudy ResultsDiscussions Experimental StudyConclusion andFuture Work Study ResultsQuestionsReferences Discussions Conclusion and Future Work Questions 2 / 37
  3. 3. Antipatterns and Comprehension Introduction Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudy OO ParadigmStudy Results There is many benefits in using OO for analysis, design,Discussions coding.Conclusion andFuture Work These benefits include better understandability, easierQuestions modifications, greater productivity, and simpler reuse.References These benefits rely mainly on intuition instead of observations or experimental results. 3 / 37
  4. 4. Antipatterns and Comprehension Introduction Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudy Empirical StudiesStudy Results There exist few empirical studies confirming the benefitsDiscussions of OO.Conclusion andFuture Work It is only recently that researchers started to study theQuestions “OO claims” empirically.References Obtained results show that OO does not always bring the expected benefits. 4 / 37
  5. 5. Antipatterns and Comprehension Introduction Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction Empirical StudiesRelated Work The literature suggests many explanations for theExperimentalStudy apparent lack of benefits of OO:Study Results OO notions (objects, polymorphism, inheritance,Discussions patterns, and so on) are not easy to learn and useConclusion and adequately.Future Work The use of OO impacts negatively the understandabilityQuestions of systems.References Antipatterns deteriorate the quality of OO systems and them harder to maintain. Therefore, the benefits of OO require further empirical studies, in particular to show its effectiveness and efficacy or lack thereof. 5 / 37
  6. 6. Antipatterns and Comprehension Introduction Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction GoalRelated Work Antipatterns are conjectured in the literature toExperimentalStudy decrease the quality of OO systems.Study Results Yet, despite the many studies on antipatterns, fewDiscussions studies empirically investigated the impact ofConclusion andFuture Work antipatterns on program comprehension.Questions We want to gather quantitative evidence on the relationReferences between antipatterns and program comprehension (or lack thereof). We perform a quantitative analysis of the impact of two antipatterns on developers’ performance during comprehension and maintenance tasks. 6 / 37
  7. 7. Antipatterns and Comprehension Related Work Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimental Antipatterns and QualityStudy Webster’s first book on quality-assurance problems [1].Study ResultsDiscussions Brown’s 40 antipatterns [2].Conclusion and Antipatterns are bad practices when solving designFuture Work problems.Questions Antipatterns are conjectured in the literature toReferences decrease the quality of systems. Fowler’s 22 code smells and refactorings [3] to improve the design of existing code. 7 / 37
  8. 8. Antipatterns and Comprehension Related Work Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudy Antipatterns and DetectionStudy Results Many approaches exist for specifying and detectingDiscussions code smells and antipatterns, e.g., [4, 5, 6].Conclusion andFuture Work We detect antipatterns using DEX, which stems fromQuestions our method DECOR (recall of 100% and an averageReferences precision greater than 60%) to locate and remove them in the systems that we study. 8 / 37
  9. 9. Antipatterns and Comprehension Related Work Abbes, Khomh, Gu´h´neuc, and e e Antoniol Antipatterns and Software Evolution and MaintenanceIntroductionRelated Work Several works have studied the impact of antipatternsExperimental on evolution phenomena.Study Vaucher et al. [7] studied in the versions of twoStudy Results different systems the evolution of Blobs:Discussions They tracked the occurrences of the Blob.Conclusion andFuture Work They observed that Blobs are, in few cases, created by design as the best solution to a particular problem.QuestionsReferences Olbrich et al. [8] studied occurrences of the Blob and Shotgun Surgery: They tracked the occurrences of these antipattersn. They observed that they are more change-prone than other classes. These works suggest that antipatterns impact program evolution and maintenance. 9 / 37
  10. 10. Antipatterns and Comprehension Related Work Abbes, Khomh, Gu´h´neuc, and e e Antoniol Antipatterns and UnderstandabilityIntroduction Several works have studied the impact of antipatternsRelated Work on evolution phenomena.ExperimentalStudy Deligiannis et al. [9] conducted a controlled experimentStudy Results on the impact of Blob on the maintainability of twoDiscussions systems.Conclusion and They concluded that the maintainability OO systemsFuture Work depends on good practices and design.Questions They did not assess the impact of Blob on theReferences developers’ ease of understanding. Du Bois et al. [10] studied the impact on understandability of the decomposition of a Blob. They found that students had more difficulties understanding the original Blob than other decompositions. Their study did not reveal any objective notion of “optimal” understandability. 10 / 37
  11. 11. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction In a NutshellRelated WorkExperimental We perform 3 experiments to assess subjects’Study comprehension of source code in the presence of twoStudy Results antipatterns:Discussions Experiment 1 includes one and only one Blob.Conclusion and Experiment 2 includes one and only one Spaghetti Code.Future Work Experiment 3 includes both antipatterns.QuestionsReferences In each experiment, we assign two systems to each subject: one containing one occurrence of one (or both) antipattern(s) and one without any occurrence. We then measure and compare the subjects’ performances for both systems. 11 / 37
  12. 12. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e Antoniol HypothesesIntroduction H0Blob : There is no statistically significant differenceRelated Work between the subjects’ average performance whenExperimental executing comprehension tasks on the source code ofStudy systems containing one occurrence of the antipatternStudy ResultsDiscussions Blob and their average performance with source codeConclusion and without any antipattern.Future Work If we reject the previous null hypothesis, then:Questions Either, E1Blob : the subjects’ average performance isReferences better when executing comprehension tasks on systems containing no occurrence of the Blob; Or, E2Blob : the subjects’ average performance is better when executing comprehension tasks on systems containing one occurrence of the Blob; We have two similar null hypotheses H0SpaghettiCode and H0Blob+SpaghettiCode and sets of alternative hypotheses for the other two experiments. 12 / 37
  13. 13. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e Antoniol ObjectsIntroduction We choose three systems for each experiment, allRelated Work developed in Java.ExperimentalStudy Numbers of ReleaseStudy Results Experiments Systems Classes SLOCs datesDiscussions YAMM 0.9.1 64 11,272 1999Conclusion and 1 JVerFileSystem 167 38,480 2008Future Work AURA 95 10,629 2008Questions GanttProject 2.0.6 527 68,545 2008 2 and 3 JFreeChart 1.0.13 989 302,844 2009References Xerces 2.7.0 740 233,331 2008 Table: Object Systems We perform each experiment on 3 systems, because a single system could be intrinsically easier/more complex to understand. 13 / 37
  14. 14. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction ObjectsRelated Work We use DECOR to ensure that each system has at leastExperimental one occurrence of antipattern.StudyStudy Results We validate the detected occurrences manually.Discussions For each system, we select randomly a subset of classesConclusion and responsible for managing a specific task to limit the sizeFuture WorkQuestions of the source code given to the subjects.References For example, in JFreeChart, we choose the source code of the classes responsible for editing and displaying the properties of a plot. We refactor each subset of each system to remove all other occurrences of (other) antipatterns to reduce possible bias. 14 / 37
  15. 15. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudy Independent and Dependent VariablesStudy ResultsDiscussions The independent variable is the presence of theConclusion and occurrence(s) of the specific antipattern(s).Future Work Our dependent variables measure the subjects’Questions performance, in terms of effort, time spent, andReferences percentage of correct answers. 15 / 37
  16. 16. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction Dependent Variable: EffortRelated WorkExperimentalStudyStudy ResultsDiscussionsConclusion andFuture WorkQuestionsReferences Figure: TLX 16 / 37
  17. 17. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudy Dependent Variable: TimeStudy ResultsDiscussionsConclusion andFuture WorkQuestionsReferences Figure: Timer. 17 / 37
  18. 18. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudyStudy Results Dependent Variable: Correct AnswersDiscussions We compute the percentage of correct answers for eachConclusion andFuture Work question by dividing the number of correct elementsQuestions found by the subject by the total number of correctReferences elements that they should have found. 18 / 37
  19. 19. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction SubjectsRelated Work Each experiment was performed by 24 anonymousExperimentalStudy subjects, S1 to S24.Study Results Some subjects were enrolled in the M.Sc. and Ph.D.Discussions programs in Computer and Software engineering atConclusion andFuture Work ´ Ecole Polytechnique de Montr´al or in Computer eQuestions Science in Universit´ de Montr´al. e eReferences Others were professionals working for software companies in the Montr´al area, recruited through the e authors’ industrial contacts. All subjects were volunteers and could withdraw at any time, for any reason. 19 / 37
  20. 20. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e Antoniol QuestionsIntroductionRelated Work Category 1: Finding a focus points:Experimental Question 1: Where is the code involved in theStudy implementation of this behavior?Study Results Question 2: Which type represents this domainDiscussions concept or this UI element or action?Conclusion andFuture Work Category 2: Expanding a focus points:Questions Question 1: Where is this method called or this typeReferences referenced? Question 2: What data can we access from this object? Category 3: Understanding a code subset: Question 1: How are these types or objects related? Question 2: What is the behavior that these types provide together and how is it distributed over these types? 20 / 37
  21. 21. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction DesignRelated Work Our design is a 2×3 factorial design:Experimental We have three different systems, each with twoStudy possibilities.Study Results For each combination, we prepare a set ofDiscussions comprehension questions, making up treatments.Conclusion andFuture Work We have six different groups of subjects, each oneQuestions affected to each one treatment.References With Antipattern(s) Without Antipattern(s) System 1 S3 , S7 , S9 , S11 , S12 , S18 , S21 , S24 S1 , S5 , S8 , S10 , S15 , S16 , S20 , S22 System 2 S1 , S2 , S6 , S14 , S15 , S17 , S20 , S22 S4 , S7 , S9 , S11 , S13 , S18 , S19 , S23 System 3 S4 , S5 , S8 , S10 , S13 , S16 , S19 , S23 S2 , S3 , S6 , S12 , S14 , S17 , S21 , S24 Table: Experimental Study 21 / 37
  22. 22. Antipatterns and Comprehension Experimental Study Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction Analysis MethodRelated Work We use the (non-parametric) Mann-Whitney test toExperimentalStudy compare sets of dependent variables and assess whetherStudy Results their difference is statistically significant.Discussions We also test the hypotheses with the (parametric)Conclusion andFuture Work Student’s t-test.Questions We use the Cohen d effect size, which indicates theReferences magnitude of the effect of a treatment on the dependent variables. We use ANOVA to assess the dependence between the six sets of dependent variables, as we have six different groups affected to the different treatments. 22 / 37
  23. 23. Antipatterns and Comprehension Study Results Abbes, Khomh, Gu´h´neuc, and e e Antoniol Summary of the Collected DataIntroduction Systems C. A.T. (s.) A.E. A.C.A.Related Work System 1: B 261 43.42 71% YAMM B 149 27.99 71%Experimental System 2: B 251 43.30 67%Study JVerFileSystem B 206 33.68 79%Study Results System 3: B 189 33.31 58%Discussions Aura B 271 42.55 66%Conclusion and System 1: S 190 47.31 67%Future Work GanttProject S 194 53.44 51%Questions System 2: S 215 39.27 52% Xerces S 182 34.48 52%References System 3: S 195 42.37 52% JFreeChart S 218 45.84 45% System 1: BS 187 45.36 44% GanttProject BS 107 27.32 83% System 2: BS 184 42.83 73% Xerces BS 140 29.33 86% System 3: BS 208 48.68 43% JFreeChart BS 138 31.27 78% 23 / 37
  24. 24. Antipatterns and Comprehension Study Results Abbes, Khomh, Gu´h´neuc, and e e Antoniol p-values and Cohen’s d effect size resultsIntroduction TimesRelated Work M.-W. t-Test ANOVA CohenExperimental p p p dStudy Experiment 1 0.46 0.30 0.02 0.18Study Results Experiment 2 0.89 0.97 0.87 0.01 Experiment 3 <0.01 <0.01 <0.01 1.54Discussions EffortsConclusion and M.-W. t-Test ANOVA CohenFuture Work p p p dQuestions Experiment 1 0.25 0.17 0.29 0.24References Experiment 2 0.64 0.57 0.41 0.09 Experiment 3 <0.01 <0.01 0.01 1.20 Answers M.-W. t-Test ANOVA Cohen p p p d Experiment 1 0.41 0.21 0.58 0.27 Experiment 2 0.26 0.24 0.60 0.35 Experiment 3 <0.01 <0.01 <0.01 1.61 24 / 37
  25. 25. Antipatterns and Comprehension Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudyStudy ResultsDiscussionsConclusion andFuture WorkQuestionsReferences 25 / 37
  26. 26. Antipatterns and Comprehension Discussions Abbes, Khomh, Gu´h´neuc, and e e Antoniol ResultsIntroduction The results of Experiment 1 show an increase inRelated Work subjects’ average time and effort on systems with BlobExperimental and a decrease in their average percentages of correctStudyStudy Results answers.Discussions Experiment 2 reveals no significant difference betweenConclusion and systems without/with a Spaghetti Code.Future WorkQuestions Experiment 3 shows strong statistically significantReferences differences between systems without/with a Blob and a Spaghetti Code. ANOVA tests confirm that these differences are significant across the six groups of subjects. Cohen’s d effect size values (for efforts, times, and percentages of correct answers) are large; suggesting a strong relation. 26 / 37
  27. 27. Antipatterns and Comprehension Discussions Abbes, Khomh, Gu´h´neuc, and e e Antoniol Mitigating VariablesIntroduction We investigate if three mitigating variables impactedRelated Work our results: knowledge of Java, Eclipse, and in softwareExperimentalStudy engineering.Study Results We use 5-level Likert scales (bad, neutral, good,Discussions excellent, expert).Conclusion andFuture Work As the groups are non-equal in terms of size, we usedQuestions the Mann-Whitney test.References We found no significant differences between the different levels; p-values are high, expressing a lack of correlation. We perform an ANOVA test to assess the impact of the mitigating variables on the three measured variables, which shows that they do not impact our results. 27 / 37
  28. 28. Antipatterns and Comprehension Discussions Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudy Threats to ValidityStudy Results Some threats limit the validity of our study results:Discussions Construct Validity: Measurement errors, subjectivity,Conclusion andFuture Work “seriousness” of the antipattern occurrences.Questions Internal Validity: Learning, selection, measures (again),References diffusion of the treatments. External Validity: Generalisability. 28 / 37
  29. 29. Antipatterns and Comprehension Conclusion and Future Work Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated Work ConclusionExperimentalStudy Antipatterns are conjectured in the literature toStudy Results negatively impact the quality of systems.Discussions We performed three experiments to study whetherConclusion and systems with antipattern(s) are more difficult toFuture WorkQuestions understand than systems without any antipattern.References Each experiment was performed with 24 subjects and on three different Java systems. We measured the subjects’ performance in terms of efforts, times, and percentages of correct answers. 29 / 37
  30. 30. Antipatterns and Comprehension Conclusion and Future Work Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction ConclusionRelated Work Collected data showed that the occurrence of oneExperimentalStudy antipattern in the source code of a system does notStudy Results significantly make its understandability.Discussions The combination of two occurrences of antipatternsConclusion andFuture Work impacted negatively and significantly understandability.Questions Developers can cope with antipatterns in isolationReferences but combinations thereof should be avoided, possibly through detection and refactorings. Consequently, developers and quality assurance personnel should be wary with growing numbers of antipatterns in their systems. 30 / 37
  31. 31. Antipatterns and Comprehension Conclusion and Future Work Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimental Future WorkStudyStudy Results We want to investigate whether theseDiscussions statistically-significant differences are due to the densityConclusion and ˝ of antipatterns in the system andU-or to theFuture Work occurrences of specific antipatterns.QuestionsReferences We also plan to replicate this study in other contexts, with other subjects, other questions, other antipatterns, and other systems. 31 / 37
  32. 32. Antipatterns and Comprehension Any Question? Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudyStudy ResultsDiscussionsConclusion andFuture WorkQuestionsReferences 32 / 37
  33. 33. Antipatterns and Comprehension References Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction B. F. Webster, Pitfalls of Object Oriented Development,Related Work 1st ed. M & T Books, February 1995. [Online].Experimental Available:Study www.amazon.com/exec/obidos/ASIN/1558513973Study ResultsDiscussions W. J. Brown, R. C. Malveau, W. H. Brown, H. W.Conclusion and McCormick III, and T. J. Mowbray, Anti Patterns:Future WorkQuestions Refactoring Software, Architectures, and Projects inReferences Crisis, 1st ed. John Wiley and Sons, March 1998. [Online]. Available: www.amazon.com/exec/obidos/tg/ detail/-/0471197130/ref=ase theantipatterngr/ 103-4749445-6141457 M. Fowler, Refactoring – Improving the Design of Existing Code, 1st ed. Addison-Wesley, June 1999. 33 / 37
  34. 34. Antipatterns and Comprehension References Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction Naouel Moha, Y.-G. Gu´h´neuc, L. Duchien, and e eRelated Work A.-F. L. Meur, “DECOR: A method for the specificationExperimentalStudy and detection of code and design smells,” TransactionsStudy Results on Software Engineering (TSE), 2009. [Online].Discussions Available: http://www-etud.iro.umontreal.ca/˜ptidej/Conclusion and Publications/Documents/TSE09.doc.pdfFuture WorkQuestions M. J. Munro, “Product metrics for automaticReferences identification of “bad smell” design problems in java source-code,” in Proceedings of the 11th International Software Metrics Symposium. IEEE Computer Society Press, September 2005. [Online]. Available: http://doi. ieeecomputersociety.org/10.1109/METRICS.2005.38 34 / 37
  35. 35. Antipatterns and Comprehension References Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroduction R. Marinescu, “Detection strategies: Metrics-based rulesRelated Work for detecting design flaws,” in Proceedings of the 20thExperimentalStudy International Conference on Software Maintenance.Study Results IEEE CS Press, 2004, pp. 350–359.Discussions St´phane Vaucher, Foutse Khomh, Naouel Moha, and eConclusion andFuture Work Y.-G. Gu´h´neuc, “Prevention and cure of software e eQuestions defects: Lessons from the study of god classes,” inReferences Proceedings of the 16th Working Conference on Reverse Engineering (WCRE), G. Antoniol and A. Zaidman, Eds. IEEE Computer Society Press, October 2009, 10 pages. [Online]. Available: http://www-etud.iro.umontreal.ca/ ˜ptidej/Publications/Documents/WCRE09b.doc.pdf 35 / 37
  36. 36. Antipatterns and Comprehension References Abbes, Khomh, Gu´h´neuc, and e e Antoniol S. Olbrich, D. S. Cruzes, V. Basili, and N. Zazworka,Introduction “The evolution and impact of code smells: A case studyRelated Work of two open source systems,” in Third InternationalExperimental Symposium on Empirical Software Engineering andStudyStudy Results Measurement, 2009.Discussions D. Ignatios, S. Ioannis, A. Lefteris, R. Manos, andConclusion andFuture Work S. Martin, “A controlled experiment investigation of anQuestions object oriented design heuristic for maintainability,”References Journal of Systems and Software, vol. 65, no. 2, February 2003. B. D. Bois, S. Demeyer, J. Verelst, T. Mens, and M. Temmerman, “Does god class decomposition affect comprehensibility?” in Proceedings of the IASTED International Conference on Software Engineering. IASTED/ACTA Press, 2006, pp. 346–355. 36 / 37
  37. 37. Antipatterns and Comprehension References Abbes, Khomh, Gu´h´neuc, and e e AntoniolIntroductionRelated WorkExperimentalStudyStudy ResultsDiscussionsConclusion andFuture WorkQuestionsReferences 37 / 37

×