SVMDetect  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeur                      Support Vector ...
SVMDetect                    Outline  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeur          ...
SVMDetect                    Introduction  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIntro...
SVMDetect                    Introduction  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeur   Li...
SVMDetect                    Introduction  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIntro...
SVMDetect                    Related Work  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e e            Smell/Anti...
SVMDetect                    Related Work  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeur   SV...
SVMDetect                    Our Approach: SVMDetect  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e e           ...
SVMDetect                    Our Approach: SVMDetect  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A...
SVMDetect                    Empirical Study  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIn...
SVMDetect                    Empirical Study  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIn...
SVMDetect                    Empirical Study  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIn...
SVMDetect                    Empirical Study  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIn...
SVMDetect                    Study Results  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIntr...
SVMDetect                    Study Results  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIntr...
SVMDetect                    Study Results  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e e            Complete ...
SVMDetect                    Discussions  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeur      ...
SVMDetect                    Conclusion and Future Work  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol...
SVMDetect                    Conclusion and Future Work  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol...
SVMDetect                    Any Question?  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIntr...
SVMDetect                    References  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIntrodu...
SVMDetect                    References  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeur       ...
SVMDetect                    References  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeurIntrodu...
SVMDetect                    References  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeur       ...
SVMDetect                    References  Maiga, Ali, Bhattacharya,    Saban´,e  Gu´h´neuc,     e eAntoniol, A¨ımeur       ...
Upcoming SlideShare
Loading in...5
×

ASE12.ppt

106

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
106
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ASE12.ppt

  1. 1. SVMDetect Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur Support Vector Machines for Anti-patternIntroduction DetectionRelated WorkOur Approach:SVMDetect Abdou Maiga, Nasir Ali, Neelesh Bhattacharya, AminataStudy Results Saban´, Yann-Ga¨l Gu´h´neuc, Giuliano Antoniol, Esma e e e eDiscussions A¨ ımeurConclusion andFuture Work ´ DGIGL et DIRO, Ecole Polytechnique et Universit´ de Montr´al, Qu´bec, e e eQuestions CanadaReferences E-mails: {abdou.maiga, nasir.ali, n.bhattacharya, a.sabane, giuliano.antoniol}@polymtl.ca, {guehene, aimeur}@iro.umontreal.ca ASE 2012 September 6, 2012 Pattern Trace Identification, Detection, and Enhancement in Java SOftware Cost-effective Change and Evolution Research Lab
  2. 2. SVMDetect Outline Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur IntroductionIntroductionRelated Work Related WorkOur Approach:SVMDetectStudy Results Our Approach: SVMDetectDiscussionsConclusion and Study ResultsFuture WorkQuestionsReferences Discussions Conclusion and Future Work Questions 2 / 25
  3. 3. SVMDetect Introduction Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroductionRelated WorkOur Approach: MotivationSVMDetect Anti-patterns: “poor” solutions to recurring design andStudy Results implementation problems.DiscussionsConclusion and Impact program comprehension, software evolution andFuture Work maintenance activities [8].Questions Important to detect them early in softwareReferences development process, to reduce the maintenance costs 3 / 25
  4. 4. SVMDetect Introduction Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur LimitationsIntroduction Current anti-pattern detection approaches have severalRelated Work limitations:Our Approach:SVMDetect they require extensive knowledge of anti-patternsStudy Results they have limited precision and recallDiscussions they cannot be applied on subsets of systems.Conclusion andFuture Work We proposeQuestions Apply SVM on subsets because it considers systemReferences classes one at a time, not collectively as previous rule-based approaches do. To the best of our knowledge, researchers have not yet studied the potential benefits of using SVM to detect anti-patterns. 4 / 25
  5. 5. SVMDetect Introduction Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroduction ContributionsRelated Work SVMDetect to detect anti-patterns using SVMOur Approach:SVMDetect Use of precision and recall to compare SVMDetect toStudy Results DETEX [13], the best state-of-the-art approach, on 3Discussions programs and 4 anti-patterns.Conclusion and The accuracy of SVMDetect is greater than ofFuture WorkQuestions DETEX on subsets.References For whole system, SVMDetect find more anti-patterns occurrences than DETEX. We thus conclude that: a SVM-based approach can overcome the limitations of previous approaches. 5 / 25
  6. 6. SVMDetect Related Work Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e e Smell/Anti-pattern DetectionAntoniol, A¨ımeur Many researchers studied anti-patterns detection.Introduction Alikacem et al. [1] used meta-model for representingRelated WorkOur Approach: the source code and fuzzy thresholds.SVMDetect Langelier et al. [10] used a visual approach.Study ResultsDiscussions Marinescu [11] used quantifiable expression of rules.Conclusion and Sahraoui et al. [7] used search-based techniques.Future WorkQuestions Moha et al. [13] proposed an approach based on a setReferences of rules that describes each anti-pattern. The works carried out so far suffered from some limitations: they have limited precision and recall (if reported at all) had not been adopted by practitioners yet cannot be applied on subsets of systems required sufficient knowledge of anti-patterns. 6 / 25
  7. 7. SVMDetect Related Work Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur SVM ApplicationsIntroduction SVM in several domains for various applications, e.g.,Related Work bioinformatics [2], object recognition [4].Our Approach:SVMDetect SVM is a recent alternative to the classificationStudy Results problems.Discussions Guihong et al. [3] used SVM, for terms classification.Conclusion andFuture Work SVM used in image retrieval systems by Sethia et al.Questions [12]References Kim et al. [9] proposed the change classification approach for predicting latent software bugs based on SVM. To the best of our knowledge, no previous approach used SVM for anti-pattern Detection. 7 / 25
  8. 8. SVMDetect Our Approach: SVMDetect Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e e SVMDetectAntoniol, A¨ımeur SVMDetect is based on Support Vector Machines (SVM)Introduction using a polynomial kernel to detect occurrences ofRelated Work anti-patterns.Our Approach:SVMDetect We use SVMDetect to detect the well-known anti-patterns:Study Results Blob, Functional Decomposition, Spaghetti code, and SwissDiscussions Army Knife. For each anti-pattern detection, the detectionConclusion and process is identical.Future Work We illustrate the detection process with the BlobQuestionsReferences anti-pattern for the sake of clarity. We define: TDS = {Ci , i = 1, . . . , p}, a set of classes Ci derived from an object-oriented system that constitutes the training dataset; ∀i, Ci is labelled as Blob (B) or not (N); DDS is the set of the classes of a system in which we want to detect the Blob classes. 8 / 25
  9. 9. SVMDetect Our Approach: SVMDetect Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur SVMDetect - StepsIntroduction To detect the Blob classes in the set DDS, we applyRelated Work SVMDetect through the following steps:Our Approach:SVMDetect Step 1 (Object Oriented Metric Specification)Study Results SVMDetect takes as input the training dataset TDSDiscussions with object-oriented metrics for classes.Conclusion and Step 2 (Train the SVM Classifier) Train SVMDetectFuture Work with TDS defined in Step 1.QuestionsReferences Step 3 (Construction of the dataset DDS and detection of the occurrences of an anti-pattern) Build detection dataset DDS and apply SVMDetect trained in step 2 to DDS. We use Weka to implement SVMDetect using its SVM classifier. 9 / 25
  10. 10. SVMDetect Empirical Study Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroductionRelated Work Empirical StudyOur Approach: goal: validate that SVMDetect can overcome previousSVMDetect approaches’ limitationsStudy ResultsDiscussions quality focus: accuracy of SVMDetect, in terms ofConclusion and precision and recall.Future WorkQuestions perspective: researchers and practitioners interested inReferences verifying if SVMDetect can be effective in detecting various kinds of anti-patterns, and in overcoming the previous limitations. 10 / 25
  11. 11. SVMDetect Empirical Study Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroductionRelated Work Research QuestionsOur Approach: RQ1: How does the accuracy of SVMDetect compareSVMDetect with that of DETEX, in terms of precision and recall?Study Results We decompose RQ1 as follows:DiscussionsConclusion and RQ11 : How does the accuracy of SVMDetect compareFuture Work with that of DETEX, in terms of precision and recall,Questions when applied on a same subset of a system?References RQ12 : How many occurrences of Blob SVMDetect can detect when comparing with that of DETEX on a same entire system? 11 / 25
  12. 12. SVMDetect Empirical Study Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroductionRelated Work ObjectsOur Approach: Names Versions # Lines of Code # Classes # InterfacesSVMDetect ArgoUML 0.19.8 113,017 1,230 67Study Results A design tool for UMLDiscussions Azureus 2.3.0.6 191,963 1,449 546Conclusion and A peer-to-peer client that implements the protocol BitTorrentFuture Work Xerces 2.7.0 71,217 513 162Questions A syntaxic analyserReferences Table : Description of the objects of the study 12 / 25
  13. 13. SVMDetect Empirical Study Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroduction SubjectsRelated WorkOur Approach: The subjects of our study are the following fourSVMDetect anti-patterns:Study ResultsDiscussions BlobConclusion and Functional Decomposition (FD)Future WorkQuestions Spaghetti Code (SC)References Swiss Army Knife (SAK) These four anti-patterns because known anti-patterns, commonly studied in previous work for comparison. 13 / 25
  14. 14. SVMDetect Study Results Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroduction Subsets of System: RQ11Related WorkOur Approach: Table : Precision of SVMDetect vs. DETEX in subsets (%)SVMDetectStudy Results ArgoUML Azureus Xerces DETEX 0.00 0.00 0.00Discussions Blob SVMDetect 97.09 97.32 95.51Conclusion and DETEX 0.00 0.00 0.00Future Work FD SVMDetect 70.68 72.01 66.93Questions DETEX 0.00 0.00 0.00 SCReferences SVMDetect 85.00 88.00 86.00 DETEX 10.00 10.00 0.00 SAK SVMDetect 75.46 84.54 80.76 14 / 25
  15. 15. SVMDetect Study Results Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroduction Subsets of System: RQ11Related WorkOur Approach: Table : Recall of SVMDetect vs. DETEX in subsets (%)SVMDetectStudy Results ArgoUML Azureus Xerces DETEX 0.00 0.00 0.00Discussions Blob SVMDetect 84.09 91.33 95.29Conclusion and DETEX 0.00 0.00 0.00Future Work FD SVMDetect 57.50 84.28 70.00Questions DETEX 0.00 0.00 0.00 SCReferences SVMDetect 71.00 89.00 86.00 DETEX 0.00 0.00 0.00 SAK SVMDetect 77.14 85.71 75.50 15 / 25
  16. 16. SVMDetect Study Results Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e e Complete System: RQ12Antoniol, A¨ımeurIntroduction Table : Total recovered occurrences of BLOB by DETEX and SVMDetectRelated WorkOur Approach: DETEX SVMDetectSVMDetect ArgoUML 25 40Study ResultsDiscussions Azureus 38 48Conclusion and Xerces 39 55Future Work Total 102 143QuestionsReferences We answer RQ1: “How does the accuracy of SVMDetect compare with that of DETEX, in terms of precision and recall?” as follows: on subsets of systems, SVMDetect dramatically outperforms DETEX. on entire systems, SVMDetect detects more 16 / 25 occurrences of Blob than DETEX.
  17. 17. SVMDetect Discussions Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur Threats to ValidityIntroduction Threats to the validity of our results.Related Work Construct validity (Measurement errors, subjectivity):Our Approach:SVMDetect occurrences of anti-patterns manually validated.Study Results Internal Validity (dependence of the obtained resultsDiscussions on chosen anti-patterns and systems.): used fourConclusion andFuture Work well-known and representative anti-patterns. used inQuestions previous works. used 3 open-source systems withReferences different sizes, used in previous works. Reliability Validity (possibility of replication): used 3 open-source systems available on-line. External Validity (Generalisability): 3 systems with different sizes and different domains. Representative subset of anti-patterns. 17 / 25
  18. 18. SVMDetect Conclusion and Future Work Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur ConclusionIntroduction introduced a novel approach to detect anti-patterns,Related Work SVMDetect, based on SVM.Our Approach: SVMDetect performs on 3 systems (ArgoUML v0.19.8,SVMDetect Azureus v2.3.0.6, and Xerces v2.7.0) and 4Study ResultsDiscussions anti-patterns (Blob, Functional Decomposition,Conclusion and Spaghetti Code, and Swiss Army Knife)Future Work the accuracy of SVMDetect is greater than that ofQuestions DETEX on a subset of classes.References on whole system, SVMDetect is able to find more anti-patterns occurrences than DETEX SVM-based approach can overcome the limitations of the previous approaches and could be more readily adopted by practitioners. 18 / 25
  19. 19. SVMDetect Conclusion and Future Work Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroductionRelated Work Future WorkOur Approach: Future work includes:SVMDetectStudy Results use SVMDetect in real-world environments.Discussions reproduce the study with other systems andConclusion and anti-patterns to increase our confidence in theFuture Work generalisability of our conclusions.QuestionsReferences take into account the user feedback. evaluate the impact of the quality of training dataset and feedback set on SVMDetect results. 19 / 25
  20. 20. SVMDetect Any Question? Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroductionRelated WorkOur Approach:SVMDetectStudy ResultsDiscussionsConclusion andFuture WorkQuestionsReferences 20 / 25
  21. 21. SVMDetect References Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroduction E. H. Alikacem and H. A. Sahraoui.Related Work D´tection d’anomalies utilisant un langage de r`gle de e eOur Approach: qualit´. eSVMDetect In LMO. Hermes Science Publications, 2006.Study ResultsDiscussions J. Bedo, C. Sanderson, and A. Kowalczyk.Conclusion and An efficient alternative to svm based recursive featureFuture Work elimination with applications in natural languageQuestions processing and bioinformatics.References In A. Sattar and B.-h. Kang, editors, AI 2006: Advances in Artificial Intelligence, volume 4304 of Lecture Notes in Computer Science, pages 170–180. Springer Berlin Heidelberg, 2006. 21 / 25
  22. 22. SVMDetect References Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur G. Cao, J.-Y. Nie, J. Gao, and S. Robertson. Selecting good expansion terms for pseudo-relevanceIntroduction feedback.Related Work In Proceedings of the 31st Annual International ACMOur Approach:SVMDetect SIGIR Conference on Research and Development inStudy Results Information Retrieval, SIGIR 2008, Singapore, JulyDiscussions 20-24, 2008, pages 243–250. ACM, 2008.Conclusion andFuture Work M. J. Choi, A. Torralba, and A. S. Willsky.Questions A tree-based context model for object recognition.References IEEE Trans. Pattern Anal. Mach. Intell., 34:240–252, Feb. 2012. C. Cortes and V. Vapnik. Support-vector networks. Machine Learning, 20:273–297, 1995. 10.1007/BF00994018. 22 / 25
  23. 23. SVMDetect References Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeurIntroduction Y.-G. Gu´h´neuc, H. Sahraoui, and Farouk Zaidi. e eRelated Work Fingerprinting design patterns.Our Approach: In E. Stroulia and A. de Lucia, editors, Proceedings ofSVMDetect the 11th Working Conference on Reverse EngineeringStudy Results (WCRE). IEEE Computer Society Press, NovemberDiscussions 2004.Conclusion andFuture Work M. Kessentini, S. Vaucher, and H. Sahraoui.Questions Deviance from perfection is a better criterion thanReferences closeness to evil when identifying risky code. In Proceedings of the IEEE/ACM international conference on Automated software engineering, ASE ’10, pages 113–122, New York, NY, USA, 2010. ACM. 23 / 25
  24. 24. SVMDetect References Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur F. Khomh, M. D. Penta, and Y.-G. Gu´h´neuc. e eIntroduction An exploratory study of the impact of antipatterns onRelated Work class change-and fault-proneness.Our Approach: Journal of Empirical Software Engineering (EMSE),SVMDetect 2011.Study ResultsDiscussions S. Kim, E. J. W. Jr., and Y. Zhang.Conclusion and Classifying software changes: Clean or buggy?Future Work IEEE Trans. Software Eng., 34(2):181–196, 2008.QuestionsReferences G. Langelier, H. Sahraoui, and P. Poulin. Visualization-based analysis of quality for large-scale software systems. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, ASE ’05, pages 214–223, New York, NY, USA, 2005. ACM. 24 / 25
  25. 25. SVMDetect References Maiga, Ali, Bhattacharya, Saban´,e Gu´h´neuc, e eAntoniol, A¨ımeur R. Marinescu.Introduction Detection strategies: Metrics-based rules for detectingRelated Work design flaws.Our Approach: In In Proceedings of the IEEE 20th InternationalSVMDetect Conference on Software Maintenance. IEEE ComputerStudy Results Society Press, 2004.DiscussionsConclusion and L. Setia, J. Ick, and H. Burkhardt.Future Work Svm-based relevance feedback in image retrieval usingQuestions invariant feature histograms.References Naouel Moha, Y.-G. Gu´h´neuc, L. Duchien, and e e A.-F. L. Meur. DECOR: A method for the specification and detection of code and design smells. Transactions on Software Engineering (TSE), 2009. 25 / 25
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×