Successfully reported this slideshow.
Your SlideShare is downloading. ×

Searching for Configurations in Clone Evaluation: A Replication Study [SSBSE'16]

Loading in …3
×

Check these out next

1 of 20
1 of 20

Searching for Configurations in Clone Evaluation: A Replication Study [SSBSE'16]

Download to read offline

A slide accompanying my talk at 8th International Symposium on Search-based Software Engineering (SSBSE): Challenge Track, 2016. North Carolina, USA.

A slide accompanying my talk at 8th International Symposium on Search-based Software Engineering (SSBSE): Challenge Track, 2016. North Carolina, USA.

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Searching for Configurations in Clone Evaluation: A Replication Study [SSBSE'16]

  1. 1. Searching for Configurations in Clone Evaluation: A Replication Study C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke J. H. Drake CENTRE FOR RESEARCH ON EVOLUTION, SEARCH AND TESTING DEPARTMENT OF COMPUTER SCIENCE UNIVERSITY COLLEGE LONDON
  2. 2. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Code Clone 2
  3. 3. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Clone Detectors 3 if (x==0) then y=y+1; if (check==0) then count=count+1; $p ($p==0) $p $p=$p+1; $p ($p==0) $p $p=$p+1; if_s if ( cond_e ) then assign_e if_s if ( cond_e ) then assign_e Deckard CCFinder Simian NiCad
  4. 4. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Oracle Problem in Code Clone Absence of the possibility to establish a ground truth, we do not know if code is actually cloned 4 ?
  5. 5. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Agreement 5 ?
  6. 6. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Parameters Tuning 6
  7. 7. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake EvaClone 7 T. Wang, M. Harman., Y. Jia, & J. Krinke. Searching for Better Configurations: A Rigorous Approach to Clone Evaluation. in FSE’13 6 Clone Detectors: PMD, iClones ConQAT, Simian, NiCad, CCFinder 8 Software Projects: weltab, cook, snns, psql, javadoc, ant, jdtcore, swing 15 years
  8. 8. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Maximising Agreement 8 C D N S Maximise Clone detectors
  9. 9. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake EvaClone (cont.) 9 EvaClone favors recall over precision 
 and more candidates will be reported.
  10. 10. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Replication Study 10
  11. 11. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Fitness Function 11 4x3x2x1x ++ + 4 x (All clone lines)
  12. 12. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Replication Study (cont.) 12 Deckard CCFinder Simian NiCad 25 parameters Population size 100 No. of Generation 100 Crossover 0.8 Mutation 0.1 Elitism 0.25 2 x 1012
  13. 13. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake 13 Ver. 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 2.0.0 2.0.44 SLOC (k) 5.5 6.7 6.78 6.82 7.2 7.6 8.4 8.9 10.1 12.4 17.9 22.8 23.6 25.3 %Inc N/A 21% 2% 1% 6% 5% 11% 7% 13% 23% 44% 28% 3% 8% Note: there are 2 complete libraries (cglib and asm) embedded in release 1.5 — 1.9 and have been removed before the analysis
  14. 14. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake RQ1: Optimised Agreement How do the default parameters perform in terms of clone agreement on each Mockito release compared to the optimised ones? 14 0.30 0.35 0.40 0.45 0.50 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 2.0.0 2.0.44 Mockito FitnessValue Default EvaClone Highest EvaClone Lowest Comparison of optimised tools agreement (the highest and the lowest in 20 runs) to the default agreement over 14 Mockito releases
  15. 15. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake RQ2: Stability of Optimised Parameters 15 Are there noticeable differences in the values of optimised parameters over releases? Tool Parameter DF Optimised 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 2.0.0 2.0.44 CCFinder MinToken TKS 50 12 10 10 70 16 70 18 70 19 80 18 80 18 80 19 80 20 10 14 10 17 10 10 10 10 10 10 10 10 Deckard MinToken Stride Similarity 30 5 0.9 30 inf 0.9 50 8 1.0 50 8 1.0 50 8 1.0 50 8 1.0 50 8 1.0 50 8 1.0 50 8 1.0 50 16 0.95 50 5 1.0 50 inf 0.9 50 inf 0.9 50 inf 0.9 50 inf 0.9 NiCad MinLine MaxLine UPI Blind Abstract 6 1K 0.3 0 0 5 200 0.3 1 4 7 100 0.0 0 6 7 100 0.1 0 6 7 400 0.0 0 6 6 400 0.0 0 6 6 200 0.1 0 5 6 200 0.1 0 5 7 200 0.0 1 6 6 200 0.3 1 6 5 100 0.1 1 2 5 100 0.3 1 4 5 100 0.3 1 4 5 200 0.3 1 4 5 200 0.3 1 4
  16. 16. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake RQ2: Stability of Optimised Parameters 16 Tool Parameter DF Optimised 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 2.0.0 2.0.44 Simian ignoreCurlyBraces 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 ignoreIdentifiers 0 1 0 0 0 0 0 0 0 1 1 1 1 1 1 ignoreIdentifierCase 0 ✱ ✱ ✱ ✱ ✱ ✱ ✱ ✱ ✱ ✱ ✱ ✱ ✱ ✱ ignoreStrings 0 1 0 0 0 0 0 0 0 1 0 ✱ ✱ ✱ ✱ ignoreStringCase 1 ✱ 1 1 0 0 0 0 0 ✱ 0 ✱ ✱ ✱ ✱ ignoreNumbers 0 1 0 1 0 1 1 0 1 1 0 ✱ ✱ ✱ ✱ ignoreCharacters 0 0 0 1 0 0 0 1 0 0 1 ✱ ✱ ✱ ✱ ignoreCharacterCase 1 0 0 ✱ 1 1 0 ✱ 1 1 ✱ ✱ ✱ ✱ ✱ ignoreLiterals 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 ignoreSubtypeNames 0 1 0 0 0 0 0 0 0 0 0 0 0 1 1 ignoreModifiers 1 1 1 0 1 0 0 0 0 0 0 1 1 1 1 ignoreVariableNames 0 1 0 0 0 0 0 0 0 1 1 0 0 0 1 balanceParentheses 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 balanceSquareBrackets 0 1 0 0 0 1 1 0 1 1 1 1 1 1 0 MinLine 6 5 6 6 6 6 6 6 6 7 7 5 5 5 5 Are there noticeable differences in the values of optimised parameters over releases?
  17. 17. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake RQ3: Clones over Releases 17 How many clones in Mockito are reported with the highest agreement over releases? DefaultEvaClone
  18. 18. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Maximising Agreement 18 C D N S Maximise Clone detectors
  19. 19. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake Open Challenge A better fitness function 
 for EvaClone is needed It must not only rely on the number of cloned lines, but also include other aspects: How often a line is found to be cloned to other places? Precision vs. Recall? Location of clones 19 ???
  20. 20. Searching for Configurations in Clone Evaluation: A Replication Study — C. Ragkhitwetsagul, M. Paixao, M. Adham, S. Busari, J. Krinke, J. H. Drake 20 0.30 0.35 0.40 0.45 0.50 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 2.0.0 2.0.44 Mockito FitnessValue Default EvaClone Highest EvaClone Lowest Opt. params vs Def. params Tool Parameter D F Optimised 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 2. 0. 0 2. 0. 44 CCFinder MinToken TKS 5 0 1 2 10 10 70 16 70 18 70 19 80 18 80 18 80 19 80 20 10 14 10 17 10 10 10 10 10 10 10 10 Deckard MinToken Stride Similarity 3 0 5 0. 9 30 inf 0. 9 50 8 1. 0 50 8 1. 0 50 8 1. 0 50 8 1. 0 50 8 1. 0 50 8 1. 0 50 8 1. 0 50 16 0. 95 50 5 1. 0 50 inf 0. 9 50 inf 0. 9 50 inf 0. 9 50 inf 0. 9 NiCad MinLine MaxLine UPI Blind Abstract 6 1 K 0. 3 0 0 5 20 0 0. 3 1 4 7 10 0 0. 0 0 6 7 10 0 0. 1 0 6 7 40 0 0. 0 0 6 6 40 0 0. 0 0 6 6 20 0 0. 1 0 5 6 20 0 0. 1 0 5 7 20 0 0. 0 1 6 6 20 0 0. 3 1 6 5 10 0 0. 1 1 2 5 10 0 0. 3 1 4 5 10 0 0. 3 1 4 5 20 0 0. 3 1 4 5 20 0 0. 3 1 4 Opt. params are not stable over releases DefaultEvaClone Fitness func. needs improvements

×