SlideShare a Scribd company logo

Wcre13a.ppt

•
•
Yann-Gaël Guéhéneuc
Yann-Gaël Guéhéneuc

When developers perform a maintenance task, they follow an exploration strategy (ES) that is characterised by how they navigate through the program entities. Studying ES can help to assess how developers understand a program and perform a change task. Various factors could influence how developers explore a program and the way in which they explore a program may affect their performance for a certain task. In this paper, we investigate the ES followed by developers during maintenance tasks and assess the impact of these ES on the duration and effort spent by developers on the tasks. We want to know if developers frequently revisit one (or a set) of program entities (referenced exploration), or if they visit program entities with almost the same frequency (unreferenced exploration) when performing a maintenance task. We mine 1,705 Mylyn interaction histories (IH) from four open-source projects (ECF, Mylyn, PDE, and Eclipse Platform) and perform a user study to verify if both referenced exploration (RE) and unreferenced exploration (UE) were followed by some developers. Using the Gini inequality index on the number of revisits of program entities, we automatically classify interaction histories as RE and UE and perform an empirical study to measure the effect of program exploration on the task duration and effort. We report that, although a UE may require more exploration effort than a RE, a UE is on average 12.30% less time consuming than a RE.

Software
1 of 61
Download to read offline
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work On the Effect of Program Exploration on Maintenance Tasks Z´ephyrin Soh, Foutse Khomh, Yann-Ga¨el Gu´eh´eneuc, Giuliano Antoniol, Bram Adams Department of Computer and Software Engineering ´Ecole Polytechnique de Montr´eal, Qu´ebec, Canada October 14, 2013 Pattern Trace Identification, Detection, and Enhancement in Java SOftware Cost-effective Change and Evolution Research Lab
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Outline Introduction User Study Empirical Study Conclusion and Future Work 2 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (1/3) Program Exploration Developers interact with program entities, e.g., open files, edit methods, etc. Developers explore the program, i.e., move from program entities to program entities. Types of program entities. 3 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) 4 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) 4 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) 4 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) 4 / 23 Containment principle: Aggregate the exploration graph at class level
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) Exploration graph aggregated at class level 4 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (3/3) 5 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (3/3) 5 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (3/3) 5 / 23 A graph represents a program exploration (a) Graph 1 (b) Graph 2 Is there any difference between exploration graphs? How to find the difference? How does the difference affect the maintenance tasks?
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Why do we care? (1/3) Research on Program Exploration Developers spend on average 35% of their time navigating within and between source files [1] Developers perform on average 19.31 navigation actions between two modifications [2] [1] Ko et al., An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. TSE 2006 [2] D. R¨othlisberger, SmartGroups: Focusing on Task-Relevant Source Artifacts in IDEs, ICPC 2011 6 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Why do we care? (2/3) Reasons of Program Exploration Developers may have several different ways to explore a program e.g., Looking for relevant entities ⇒ Some developers (may) not know where to look at. Validate previous changes ⇒ Developers (may) know the target entity to look at. ⇒ In some cases, developers may perform more back and forth navigations (i.e., revisitation) on program entities 7 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Why do we care? (3/3) Revisitation-based Exploration Some recommendation systems (e.g., Mylyn [3], NavClus [4]) are based on the revisitation of program entities We wonder if revisitation is “good“ or “bad“ ⇒ Knowing that is helpful to improve revisitation-based approach for recommendation systems. [3] Kersten and Murphy, Mylar: a degree-of-interest model for IDEs, AOSD 2005 [4] Lee and Kang, Clustering and recommending collections of code relevant to tasks, ICSM 2011 8 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Research Question Exploration Strategies In the revisitation-based exploration, we focus on two extreme cases: Referenced Exploration (RE): A developer revisits one (or a set of) entity(ies) already visited ⇒ referenced entities Unreferenced Exploration (UE): There is no set of referenced entities ⇒ Almost the same revisitation of program entity(ies) 9 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Research Question Exploration Strategies In the revisitation-based exploration, we focus on two extreme cases: Referenced Exploration (RE): A developer revisits one (or a set of) entity(ies) already visited ⇒ referenced entities Unreferenced Exploration (UE): There is no set of referenced entities ⇒ Almost the same revisitation of program entity(ies) Do developers follow a referenced exploration when performing maintenance tasks? 9 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (1/5) Data Collection Mylyn’s IHs appear as attachment (XML file with name “mylyn-context.zip”) to a bug 10 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (1/5) Data Collection Mylyn’s IHs appear as attachment (XML file with name “mylyn-context.zip”) to a bug Top four Eclipse projects with more IHs (from 2,601 bugs) ECF Mylyn PDE Platform Total # IHs 26 1,273 131 275 1,705 10 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? Random sample of IHs (proportional among projects) 11 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? Random sample of IHs (proportional among projects) 11 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs Random sample of IHs (proportional among projects) 11 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs Random sample of IHs (proportional among projects) 11 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs RE UE D Random sample of IHs (proportional among projects) 11 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs RE UE D Random sample of IHs (proportional among projects) How much the subjects agreed? Fliess' Kappa interater agreement coeficient - Overall: fair agreement (0.36) - RE: almost moderate agreement (0.38) - UE: almost moderate agreement (0.39) - Doubt: No agreement (-0.009) 11 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (3/5) How do we aggregate the results? RE UE DRE UE 12 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (3/5) How do we aggregate the results? RE UE DRE UE >= 2/3 subjects RE 12 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (3/5) How do we aggregate the results? RE UE DRE UE UE >= 2/3 subjects 12 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (3/5) How do we aggregate the results? RE UE DRE UE < 2/3 subjects D < 2/3 subjects 12 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (4/5) Identification Technique Subjects count the number of nodes and the num- ber of in/out arrows in the graphs ⇒ look at the distribution/inequality of revisits across graphs 13 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (4/5) Identification Technique Subjects count the number of nodes and the num- ber of in/out arrows in the graphs ⇒ look at the distribution/inequality of revisits across graphs Gini Inequality Index [5] We measure how much classes are (in)equally revisited Used in econometrics to measure the inequality of income among a population Population = classes Income of a class = NumRevisits Gini(IH) = 1 2n2µ n i=1 n j=1 | NumRevisit(eni ) − NumRevisit(enj ) | [5] K. Xu, How has the literature on Gini’s index evolved in the past 80 years?, Technical report, Department of Economics, Dalhouse University, 2004 13 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Define a threshold to know if classes are (in)equally revisited If Gini(IH) < Threshold ⇒ Classes are almost equally revisited ⇒ UE: Unreferenced Exploration If Gini(IH) ≥ Threshold ⇒ Some classes are more revisited than others ⇒ RE: Referenced Exploration 14 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality 14 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality Threshold ? 14 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold RE 14 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality RE 0.28 14 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality RE 0.28 - Threshold = 0.2 ==> RE - Threshold = 0.3 ==> UE 0.2 0.3 14 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality RE Use F-Measure to maximize both precision and recall 0.4 14 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4 14 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (1/2) Percentage RE vs. UE RE: 28.03% of Interaction Histories UE: 71.96% of Interaction Histories ⇒ Developers follow mostly the unreferenced exploration (UE) when performing a maintenance task. 15 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (1/2) Percentage RE vs. UE RE: 28.03% of Interaction Histories UE: 71.96% of Interaction Histories ⇒ Developers follow mostly the unreferenced exploration (UE) when performing a maintenance task. Methodical developers do not reinvestigate methods as frequently as opportunistic developers [6] RE = opportunistic developers? UE = methodical developers? ⇒ We need more investigations [6] Robillard et al., How effective developers investigate source code: An exploratory study, TSE 2004 15 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system 16 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system 16 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference. 16 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference. Task interruption: The interruption time seems to push developers to be concentrate on a set of (referenced) entities (RE) 16 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference. Task interruption: The interruption time seems to push developers to be concentrate on a set of (referenced) entities (RE) Type of the task: Bug severity because developers may have more back and forth navigation when fixing severe bugs wrt. less severe bugs Inequality Indexes are not significantly different for different types of bugs. 16 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study Research Questions Research Questions Does any difference exist in maintenance time be- tween RE and UE? Does any difference exist in edit/navigation ratio between RE and UE? 17 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study RQ1 Approach Overall Time spend = entity∈IH Duration(entity) #entity Unpaired Wilcoxon test 18 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study RQ1 Approach Overall Time spend = entity∈IH Duration(entity) #entity Unpaired Wilcoxon test Results The UE is on average 12.30% less time consuming than the RE. 18 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study RQ2 Approach Edit/navigation ratio = NumEdit(IH) NumEvent(IH) Unpaired Wilcoxon test 19 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study RQ2 Approach Edit/navigation ratio = NumEdit(IH) NumEvent(IH) Unpaired Wilcoxon test Results An UE requires less edit/navigation ratio than a RE. 19 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study Discussions 20 / 23 Is there a good strategy? RE (more) UE (less) ⇒ The back and forth actions on ref- erenced entities are time consuming RE (more) UE (less) ⇒ More edit actions compare to nav- igation actions. Wrong edits/modif. then come back to revert/cancel?
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study Discussions 20 / 23 Is there a good strategy? RE (more) UE (less) ⇒ The back and forth actions on ref- erenced entities are time consuming RE (more) UE (less) ⇒ More edit actions compare to nav- igation actions. Wrong edits/modif. then come back to revert/cancel? F1 F2 F1 F3 F1 F4 F2 F5 RE F1 F2 F3 F1 F4 F5 UE 4/8 3/6 4/8 > 3/6 (RE better)
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study Discussions 20 / 23 Is there a good strategy? RE (more) UE (less) ⇒ The back and forth actions on ref- erenced entities are time consuming RE (more) UE (less) ⇒ More edit actions compare to nav- igation actions. Wrong edits/modif. then come back to revert/cancel? F1 F2 F1 F3 F1 F4 F2 F5 RE F1 F2 F3 F1 F4 F5 UE 4/8 3/6 4/8 > 3/6 (RE better) F1 F2 F1 F3 F1 F4 F2 F5 RE F1 F2 F3 F1 F4 F5 UE 4/8 2/8 3/6 2/6 4/8 > 3/6 (RE better) 2/8 < 2/6 (UE better)
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Conclusion 21 / 23 Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Conclusion 21 / 23 Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Conclusion 21 / 23 Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work 22 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data 22 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data Internal validity: We use only Mylyn’s IH ⇒ Use the IH from other tools 22 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data Internal validity: We use only Mylyn’s IH ⇒ Use the IH from other tools External validity: Our subject projets are Eclipse-based projects ⇒ Use other systems 22 / 23
Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 23 / 23 Thanks for your attention! Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4

Recommended

Wcre13a.ppt
Wcre13a.pptWcre13a.ppt
Wcre13a.pptPtidej Team
 
130411 francis palma - detection of process antipatterns -- a bpel perspective
130411 francis palma - detection of process antipatterns -- a bpel perspective130411 francis palma - detection of process antipatterns -- a bpel perspective
130411 francis palma - detection of process antipatterns -- a bpel perspectivePtidej Team
 
130321 zephyrin soh - on the effect of exploration strategies on maintenanc...
130321 zephyrin soh - on the effect of exploration strategies on maintenanc...130321 zephyrin soh - on the effect of exploration strategies on maintenanc...
130321 zephyrin soh - on the effect of exploration strategies on maintenanc...Ptidej Team
 
Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...
Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...
Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...CSCJournals
 
Case Study Research in Software Engineering
Case Study Research in Software EngineeringCase Study Research in Software Engineering
Case Study Research in Software Engineeringalessio_ferrari
 
Exploratory testing STEW 2016
Exploratory testing STEW 2016Exploratory testing STEW 2016
Exploratory testing STEW 2016Per Runeson
 
Pareto-Optimal Search-Based Software Engineering (POSBSE): A Literature Survey
Pareto-Optimal Search-Based Software Engineering (POSBSE): A Literature SurveyPareto-Optimal Search-Based Software Engineering (POSBSE): A Literature Survey
Pareto-Optimal Search-Based Software Engineering (POSBSE): A Literature SurveyAbdel Salam Sayyad
 
Survey Research In Empirical Software Engineering
Survey Research In Empirical Software EngineeringSurvey Research In Empirical Software Engineering
Survey Research In Empirical Software Engineeringalessio_ferrari
 

Recommended

Wcre13a.ppt
Wcre13a.pptWcre13a.ppt
Wcre13a.pptPtidej Team
 
130411 francis palma - detection of process antipatterns -- a bpel perspective
130411 francis palma - detection of process antipatterns -- a bpel perspective130411 francis palma - detection of process antipatterns -- a bpel perspective
130411 francis palma - detection of process antipatterns -- a bpel perspectivePtidej Team
 
130321 zephyrin soh - on the effect of exploration strategies on maintenanc...
130321 zephyrin soh - on the effect of exploration strategies on maintenanc...130321 zephyrin soh - on the effect of exploration strategies on maintenanc...
130321 zephyrin soh - on the effect of exploration strategies on maintenanc...Ptidej Team
 
Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...
Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...
Comprehensive Testing Tool for Automatic Test Suite Generation, Prioritizatio...CSCJournals
 
Case Study Research in Software Engineering
Case Study Research in Software EngineeringCase Study Research in Software Engineering
Case Study Research in Software Engineeringalessio_ferrari
 
Exploratory testing STEW 2016
Exploratory testing STEW 2016Exploratory testing STEW 2016
Exploratory testing STEW 2016Per Runeson
 
Pareto-Optimal Search-Based Software Engineering (POSBSE): A Literature Survey
Pareto-Optimal Search-Based Software Engineering (POSBSE): A Literature SurveyPareto-Optimal Search-Based Software Engineering (POSBSE): A Literature Survey
Pareto-Optimal Search-Based Software Engineering (POSBSE): A Literature SurveyAbdel Salam Sayyad
 
Survey Research In Empirical Software Engineering
Survey Research In Empirical Software EngineeringSurvey Research In Empirical Software Engineering
Survey Research In Empirical Software Engineeringalessio_ferrari
 
Controlled experiments, Hypothesis Testing, Test Selection, Threats to Validity
Controlled experiments, Hypothesis Testing, Test Selection, Threats to ValidityControlled experiments, Hypothesis Testing, Test Selection, Threats to Validity
Controlled experiments, Hypothesis Testing, Test Selection, Threats to Validityalessio_ferrari
 
Empirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an OverviewEmpirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an Overviewalessio_ferrari
 
Programming with GUTs
Programming with GUTsProgramming with GUTs
Programming with GUTscatherinewall
 
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...Abdel Salam Sayyad
 
Industry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software EngineeringIndustry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software EngineeringPer Runeson
 
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...Abdel Salam Sayyad
 
On the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software TestingOn the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software Testingjfrchicanog
 
Qualitative data analysis in design research
Qualitative data analysis in design researchQualitative data analysis in design research
Qualitative data analysis in design researchEva Durall
 
Search-based testing of procedural programs:iterative single-target or multi-...
Search-based testing of procedural programs:iterative single-target or multi-...Search-based testing of procedural programs:iterative single-target or multi-...
Search-based testing of procedural programs:iterative single-target or multi-...Vrije Universiteit Brussel
 
PM2 ARTICALS
PM2 ARTICALSPM2 ARTICALS
PM2 ARTICALSra na
 
2011 EASE - Motivation in Software Engineering: A Systematic Review Update
2011 EASE - Motivation in Software Engineering: A Systematic Review Update2011 EASE - Motivation in Software Engineering: A Systematic Review Update
2011 EASE - Motivation in Software Engineering: A Systematic Review UpdateHASE – Human Aspects in Software Engineering
 
M018147883
M018147883M018147883
M018147883IOSR Journals
 
Software Engineering Ontology and Software Testing
Software Engineering Ontology and Software Testing�Software Engineering Ontology and Software Testing�
Software Engineering Ontology and Software TestingKamal Patel
 
Wcre13b.ppt
Wcre13b.pptWcre13b.ppt
Wcre13b.pptPtidej Team
 
Dc35579583
Dc35579583Dc35579583
Dc35579583IJERA Editor
 
[2017/2018] RESEARCH in software engineering
[2017/2018] RESEARCH in software engineering[2017/2018] RESEARCH in software engineering
[2017/2018] RESEARCH in software engineeringIvano Malavolta
 
Sound Empirical Evidence in Software Testing
Sound Empirical Evidence in Software TestingSound Empirical Evidence in Software Testing
Sound Empirical Evidence in Software TestingJaguaraci Silva
 
A cluster-based analysis to diagnose students’ learning achievements
A cluster-based analysis to diagnose students’ learning achievementsA cluster-based analysis to diagnose students’ learning achievements
A cluster-based analysis to diagnose students’ learning achievementsMiguel R. Artacho
 
Software engineering ontology and software testing
Software engineering ontology and software testingSoftware engineering ontology and software testing
Software engineering ontology and software testingKhushbu Patel
 
131014 wcre-exploration
131014 wcre-exploration131014 wcre-exploration
131014 wcre-explorationZephyrin Soh
 
Wcre13b.ppt
Wcre13b.pptWcre13b.ppt
Wcre13b.pptYann-Gaël Guéhéneuc
 
131016 wcre-matching
131016 wcre-matching131016 wcre-matching
131016 wcre-matchingZephyrin Soh
 

More Related Content

What's hot

Controlled experiments, Hypothesis Testing, Test Selection, Threats to Validity
Controlled experiments, Hypothesis Testing, Test Selection, Threats to ValidityControlled experiments, Hypothesis Testing, Test Selection, Threats to Validity
Controlled experiments, Hypothesis Testing, Test Selection, Threats to Validityalessio_ferrari
 
Empirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an OverviewEmpirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an Overviewalessio_ferrari
 
Programming with GUTs
Programming with GUTsProgramming with GUTs
Programming with GUTscatherinewall
 
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...Abdel Salam Sayyad
 
Industry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software EngineeringIndustry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software EngineeringPer Runeson
 
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...Abdel Salam Sayyad
 
On the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software TestingOn the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software Testingjfrchicanog
 
Qualitative data analysis in design research
Qualitative data analysis in design researchQualitative data analysis in design research
Qualitative data analysis in design researchEva Durall
 
Search-based testing of procedural programs:iterative single-target or multi-...
Search-based testing of procedural programs:iterative single-target or multi-...Search-based testing of procedural programs:iterative single-target or multi-...
Search-based testing of procedural programs:iterative single-target or multi-...Vrije Universiteit Brussel
 
PM2 ARTICALS
PM2 ARTICALSPM2 ARTICALS
PM2 ARTICALSra na
 
Software Engineering Ontology and Software Testing
Software Engineering Ontology and Software Testing�Software Engineering Ontology and Software Testing�
Software Engineering Ontology and Software TestingKamal Patel
 
[2017/2018] RESEARCH in software engineering
[2017/2018] RESEARCH in software engineering[2017/2018] RESEARCH in software engineering
[2017/2018] RESEARCH in software engineeringIvano Malavolta
 
Sound Empirical Evidence in Software Testing
Sound Empirical Evidence in Software TestingSound Empirical Evidence in Software Testing
Sound Empirical Evidence in Software TestingJaguaraci Silva
 
A cluster-based analysis to diagnose students’ learning achievements
A cluster-based analysis to diagnose students’ learning achievementsA cluster-based analysis to diagnose students’ learning achievements
A cluster-based analysis to diagnose students’ learning achievementsMiguel R. Artacho
 
Software engineering ontology and software testing
Software engineering ontology and software testingSoftware engineering ontology and software testing
Software engineering ontology and software testingKhushbu Patel
 

What's hot (19)

Controlled experiments, Hypothesis Testing, Test Selection, Threats to Validity
Controlled experiments, Hypothesis Testing, Test Selection, Threats to ValidityControlled experiments, Hypothesis Testing, Test Selection, Threats to Validity
Controlled experiments, Hypothesis Testing, Test Selection, Threats to Validity
 
Empirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an OverviewEmpirical Methods in Software Engineering - an Overview
Empirical Methods in Software Engineering - an Overview
 
Programming with GUTs
Programming with GUTsProgramming with GUTs
Programming with GUTs
 
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...
Evolutionary Search Techniques with Strong Heuristics for Multi-Objective Fea...
 
Industry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software EngineeringIndustry-Academia Communication In Empirical Software Engineering
Industry-Academia Communication In Empirical Software Engineering
 
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
On Parameter Tuning in Search-Based Software Engineering: A Replicated Empiri...
 
On the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software TestingOn the application of SAT solvers for Search Based Software Testing
On the application of SAT solvers for Search Based Software Testing
 
Qualitative data analysis in design research
Qualitative data analysis in design researchQualitative data analysis in design research
Qualitative data analysis in design research
 
Search-based testing of procedural programs:iterative single-target or multi-...
Search-based testing of procedural programs:iterative single-target or multi-...Search-based testing of procedural programs:iterative single-target or multi-...
Search-based testing of procedural programs:iterative single-target or multi-...
 
PM2 ARTICALS
PM2 ARTICALSPM2 ARTICALS
PM2 ARTICALS
 
2011 EASE - Motivation in Software Engineering: A Systematic Review Update
2011 EASE - Motivation in Software Engineering: A Systematic Review Update2011 EASE - Motivation in Software Engineering: A Systematic Review Update
2011 EASE - Motivation in Software Engineering: A Systematic Review Update
 
M018147883
M018147883M018147883
M018147883
 
Software Engineering Ontology and Software Testing
Software Engineering Ontology and Software Testing�Software Engineering Ontology and Software Testing�
Software Engineering Ontology and Software Testing
 
Wcre13b.ppt
Wcre13b.pptWcre13b.ppt
Wcre13b.ppt
 
Dc35579583
Dc35579583Dc35579583
Dc35579583
 
[2017/2018] RESEARCH in software engineering
[2017/2018] RESEARCH in software engineering[2017/2018] RESEARCH in software engineering
[2017/2018] RESEARCH in software engineering
 
Sound Empirical Evidence in Software Testing
Sound Empirical Evidence in Software TestingSound Empirical Evidence in Software Testing
Sound Empirical Evidence in Software Testing
 
A cluster-based analysis to diagnose students’ learning achievements
A cluster-based analysis to diagnose students’ learning achievementsA cluster-based analysis to diagnose students’ learning achievements
A cluster-based analysis to diagnose students’ learning achievements
 
Software engineering ontology and software testing
Software engineering ontology and software testingSoftware engineering ontology and software testing
Software engineering ontology and software testing
 

Similar to Wcre13a.ppt

131014 wcre-exploration
131014 wcre-exploration131014 wcre-exploration
131014 wcre-explorationZephyrin Soh
 
131016 wcre-matching
131016 wcre-matching131016 wcre-matching
131016 wcre-matchingZephyrin Soh
 
A Survey on Software Release Planning Models - Slides for the Presentation @ ...
A Survey on Software Release Planning Models - Slides for the Presentation @ ...A Survey on Software Release Planning Models - Slides for the Presentation @ ...
A Survey on Software Release Planning Models - Slides for the Presentation @ ...Supersede
 
A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...
A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...
A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...acijjournal
 
A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...
A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...
A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...acijjournal
 
Thesis+of+zohreh+sharafi.ppt
Thesis+of+zohreh+sharafi.pptThesis+of+zohreh+sharafi.ppt
Thesis+of+zohreh+sharafi.pptPtidej Team
 
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...CS, NcState
 
The Influence of Participant Personality in Usability Tests
The Influence of Participant Personality in Usability TestsThe Influence of Participant Personality in Usability Tests
The Influence of Participant Personality in Usability TestsCSCJournals
 
Spiral model explanation
Spiral model explanationSpiral model explanation
Spiral model explanationUmar Farooq
 
Recommendation System for Design Patterns in Software Development
Recommendation System for Design Patterns in Software DevelopmentRecommendation System for Design Patterns in Software Development
Recommendation System for Design Patterns in Software DevelopmentFrancis Palma
 
A REVIEW OF SECURITY INTEGRATION TECHNIQUE IN AGILE SOFTWARE DEVELOPMENT
A REVIEW OF SECURITY INTEGRATION TECHNIQUE IN AGILE SOFTWARE DEVELOPMENTA REVIEW OF SECURITY INTEGRATION TECHNIQUE IN AGILE SOFTWARE DEVELOPMENT
A REVIEW OF SECURITY INTEGRATION TECHNIQUE IN AGILE SOFTWARE DEVELOPMENTijseajournal
 
Empirical research methods for software engineering
Empirical research methods for software engineeringEmpirical research methods for software engineering
Empirical research methods for software engineeringsarfraznawaz
 
Research issues in object oriented software testing
Research issues in object oriented software testingResearch issues in object oriented software testing
Research issues in object oriented software testingAnshul Vinayak
 
Carol Harstad Research Proposal
Carol Harstad Research ProposalCarol Harstad Research Proposal
Carol Harstad Research ProposalCarol Harstad
 
A guide to deal with uncertainties in software project management
A guide to deal with uncertainties in software project managementA guide to deal with uncertainties in software project management
A guide to deal with uncertainties in software project managementijcsit
 
Ch 6 only 1. Distinguish between a purpose statement, research p
Ch 6 only 1. Distinguish between a purpose statement, research pCh 6 only 1. Distinguish between a purpose statement, research p
Ch 6 only 1. Distinguish between a purpose statement, research pMaximaSheffield592
 

Similar to Wcre13a.ppt (20)

131014 wcre-exploration
131014 wcre-exploration131014 wcre-exploration
131014 wcre-exploration
 
Wcre13b.ppt
Wcre13b.pptWcre13b.ppt
Wcre13b.ppt
 
131016 wcre-matching
131016 wcre-matching131016 wcre-matching
131016 wcre-matching
 
A Survey on Software Release Planning Models - Slides for the Presentation @ ...
A Survey on Software Release Planning Models - Slides for the Presentation @ ...A Survey on Software Release Planning Models - Slides for the Presentation @ ...
A Survey on Software Release Planning Models - Slides for the Presentation @ ...
 
A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...
A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...
A Model To Compare The Degree Of Refactoring Opportunities Of Three Projects ...
 
A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...
A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...
A MODEL TO COMPARE THE DEGREE OF REFACTORING OPPORTUNITIES OF THREE PROJECTS ...
 
Thesis+of+zohreh+sharafi.ppt
Thesis+of+zohreh+sharafi.pptThesis+of+zohreh+sharafi.ppt
Thesis+of+zohreh+sharafi.ppt
 
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
Promise 2011: "A Principled Evaluation of Ensembles of Learning Machines for ...
 
The Influence of Participant Personality in Usability Tests
The Influence of Participant Personality in Usability TestsThe Influence of Participant Personality in Usability Tests
The Influence of Participant Personality in Usability Tests
 
Spiral model explanation
Spiral model explanationSpiral model explanation
Spiral model explanation
 
Recommendation System for Design Patterns in Software Development
Recommendation System for Design Patterns in Software DevelopmentRecommendation System for Design Patterns in Software Development
Recommendation System for Design Patterns in Software Development
 
PowerPoint-based quizzes in wave motion: Performance and experiences of students
PowerPoint-based quizzes in wave motion: Performance and experiences of studentsPowerPoint-based quizzes in wave motion: Performance and experiences of students
PowerPoint-based quizzes in wave motion: Performance and experiences of students
 
A REVIEW OF SECURITY INTEGRATION TECHNIQUE IN AGILE SOFTWARE DEVELOPMENT
A REVIEW OF SECURITY INTEGRATION TECHNIQUE IN AGILE SOFTWARE DEVELOPMENTA REVIEW OF SECURITY INTEGRATION TECHNIQUE IN AGILE SOFTWARE DEVELOPMENT
A REVIEW OF SECURITY INTEGRATION TECHNIQUE IN AGILE SOFTWARE DEVELOPMENT
 
Empirical research methods for software engineering
Empirical research methods for software engineeringEmpirical research methods for software engineering
Empirical research methods for software engineering
 
Rsse12.ppt
Rsse12.pptRsse12.ppt
Rsse12.ppt
 
Research issues in object oriented software testing
Research issues in object oriented software testingResearch issues in object oriented software testing
Research issues in object oriented software testing
 
Carol Harstad Research Proposal
Carol Harstad Research ProposalCarol Harstad Research Proposal
Carol Harstad Research Proposal
 
A guide to deal with uncertainties in software project management
A guide to deal with uncertainties in software project managementA guide to deal with uncertainties in software project management
A guide to deal with uncertainties in software project management
 
De carlo rizk 2010 icelw
De carlo rizk 2010 icelwDe carlo rizk 2010 icelw
De carlo rizk 2010 icelw
 
Ch 6 only 1. Distinguish between a purpose statement, research p
Ch 6 only 1. Distinguish between a purpose statement, research pCh 6 only 1. Distinguish between a purpose statement, research p
Ch 6 only 1. Distinguish between a purpose statement, research p
 

More from Yann-Gaël Guéhéneuc

Advice for writing a NSERC Discovery grant application v0.5
Advice for writing a NSERC Discovery grant application v0.5Advice for writing a NSERC Discovery grant application v0.5
Advice for writing a NSERC Discovery grant application v0.5Yann-Gaël Guéhéneuc
 
Ptidej Architecture, Design, and Implementation in Action v2.1
Ptidej Architecture, Design, and Implementation in Action v2.1Ptidej Architecture, Design, and Implementation in Action v2.1
Ptidej Architecture, Design, and Implementation in Action v2.1Yann-Gaël Guéhéneuc
 
Evolution and Examples of Java Features, from Java 1.7 to Java 22
Evolution and Examples of Java Features, from Java 1.7 to Java 22Evolution and Examples of Java Features, from Java 1.7 to Java 22
Evolution and Examples of Java Features, from Java 1.7 to Java 22Yann-Gaël Guéhéneuc
 
Consequences and Principles of Software Quality v0.3
Consequences and Principles of Software Quality v0.3Consequences and Principles of Software Quality v0.3
Consequences and Principles of Software Quality v0.3Yann-Gaël Guéhéneuc
 
Some Pitfalls with Python and Their Possible Solutions v0.9
Some Pitfalls with Python and Their Possible Solutions v0.9Some Pitfalls with Python and Their Possible Solutions v0.9
Some Pitfalls with Python and Their Possible Solutions v0.9Yann-Gaël Guéhéneuc
 
An Explanation of the Unicode, the Text Encoding Standard, Its Usages and Imp...
An Explanation of the Unicode, the Text Encoding Standard, Its Usages and Imp...An Explanation of the Unicode, the Text Encoding Standard, Its Usages and Imp...
An Explanation of the Unicode, the Text Encoding Standard, Its Usages and Imp...Yann-Gaël Guéhéneuc
 
An Explanation of the Halting Problem and Its Consequences
An Explanation of the Halting Problem and Its ConsequencesAn Explanation of the Halting Problem and Its Consequences
An Explanation of the Halting Problem and Its ConsequencesYann-Gaël Guéhéneuc
 
Informaticien(ne)s célèbres (v1.0.2, 19/02/20)
Informaticien(ne)s célèbres (v1.0.2, 19/02/20)Informaticien(ne)s célèbres (v1.0.2, 19/02/20)
Informaticien(ne)s célèbres (v1.0.2, 19/02/20)Yann-Gaël Guéhéneuc
 
On Java Generics, History, Use, Caveats v1.1
On Java Generics, History, Use, Caveats v1.1On Java Generics, History, Use, Caveats v1.1
On Java Generics, History, Use, Caveats v1.1Yann-Gaël Guéhéneuc
 
On Reflection in OO Programming Languages v1.6
On Reflection in OO Programming Languages v1.6On Reflection in OO Programming Languages v1.6
On Reflection in OO Programming Languages v1.6Yann-Gaël Guéhéneuc
 

More from Yann-Gaël Guéhéneuc (20)

Advice for writing a NSERC Discovery grant application v0.5
Advice for writing a NSERC Discovery grant application v0.5Advice for writing a NSERC Discovery grant application v0.5
Advice for writing a NSERC Discovery grant application v0.5
 
Ptidej Architecture, Design, and Implementation in Action v2.1
Ptidej Architecture, Design, and Implementation in Action v2.1Ptidej Architecture, Design, and Implementation in Action v2.1
Ptidej Architecture, Design, and Implementation in Action v2.1
 
Evolution and Examples of Java Features, from Java 1.7 to Java 22
Evolution and Examples of Java Features, from Java 1.7 to Java 22Evolution and Examples of Java Features, from Java 1.7 to Java 22
Evolution and Examples of Java Features, from Java 1.7 to Java 22
 
Consequences and Principles of Software Quality v0.3
Consequences and Principles of Software Quality v0.3Consequences and Principles of Software Quality v0.3
Consequences and Principles of Software Quality v0.3
 
Some Pitfalls with Python and Their Possible Solutions v0.9
Some Pitfalls with Python and Their Possible Solutions v0.9Some Pitfalls with Python and Their Possible Solutions v0.9
Some Pitfalls with Python and Their Possible Solutions v0.9
 
An Explanation of the Unicode, the Text Encoding Standard, Its Usages and Imp...
An Explanation of the Unicode, the Text Encoding Standard, Its Usages and Imp...An Explanation of the Unicode, the Text Encoding Standard, Its Usages and Imp...
An Explanation of the Unicode, the Text Encoding Standard, Its Usages and Imp...
 
An Explanation of the Halting Problem and Its Consequences
An Explanation of the Halting Problem and Its ConsequencesAn Explanation of the Halting Problem and Its Consequences
An Explanation of the Halting Problem and Its Consequences
 
Are CPUs VMs Like Any Others? v1.0
Are CPUs VMs Like Any Others? v1.0Are CPUs VMs Like Any Others? v1.0
Are CPUs VMs Like Any Others? v1.0
 
Informaticien(ne)s célèbres (v1.0.2, 19/02/20)
Informaticien(ne)s célèbres (v1.0.2, 19/02/20)Informaticien(ne)s célèbres (v1.0.2, 19/02/20)
Informaticien(ne)s célèbres (v1.0.2, 19/02/20)
 
Well-known Computer Scientists v1.0.2
Well-known Computer Scientists v1.0.2Well-known Computer Scientists v1.0.2
Well-known Computer Scientists v1.0.2
 
On Java Generics, History, Use, Caveats v1.1
On Java Generics, History, Use, Caveats v1.1On Java Generics, History, Use, Caveats v1.1
On Java Generics, History, Use, Caveats v1.1
 
On Reflection in OO Programming Languages v1.6
On Reflection in OO Programming Languages v1.6On Reflection in OO Programming Languages v1.6
On Reflection in OO Programming Languages v1.6
 
ICSOC'21
ICSOC'21ICSOC'21
ICSOC'21
 
Vissoft21.ppt
Vissoft21.pptVissoft21.ppt
Vissoft21.ppt
 
Service computation20.ppt
Service computation20.pptService computation20.ppt
Service computation20.ppt
 
Serp4 iot20.ppt
Serp4 iot20.pptSerp4 iot20.ppt
Serp4 iot20.ppt
 
Msr20.ppt
Msr20.pptMsr20.ppt
Msr20.ppt
 
Iwesep19.ppt
Iwesep19.pptIwesep19.ppt
Iwesep19.ppt
 
Icsoc20.ppt
Icsoc20.pptIcsoc20.ppt
Icsoc20.ppt
 
Icsoc18.ppt
Icsoc18.pptIcsoc18.ppt
Icsoc18.ppt
 

Recently uploaded

Cloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEECloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEEVICTOR MAESTRE RAMIREZ
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)OPEN KNOWLEDGE GmbH
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...stazi3110
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...OnePlan Solutions
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureDinusha Kumarasiri
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024StefanoLambiase
 
Intelligent Home Wi-Fi Solutions | ThinkPalm
Intelligent Home Wi-Fi Solutions | ThinkPalmIntelligent Home Wi-Fi Solutions | ThinkPalm
Intelligent Home Wi-Fi Solutions | ThinkPalmSujith Sukumaran
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityNeo4j
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackVICTOR MAESTRE RAMIREZ
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...soniya singh
 
React Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaReact Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaHanief Utama
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWave PLM
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...gurkirankumar98700
 
Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Andreas Granig
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样umasea
 
software engineering Chapter 5 System modeling.pptx
software engineering Chapter 5 System modeling.pptxsoftware engineering Chapter 5 System modeling.pptx
software engineering Chapter 5 System modeling.pptxnada99848
 

Recently uploaded (20)

Cloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEECloud Data Center Network Construction - IEEE
Cloud Data Center Network Construction - IEEE
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with Azure
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
 
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort ServiceHot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
Hot Sexy call girls in Patel Nagar🔝 9953056974 🔝 escort Service
 
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
Dealing with Cultural Dispersion — Stefano Lambiase — ICSE-SEIS 2024
 
Intelligent Home Wi-Fi Solutions | ThinkPalm
Intelligent Home Wi-Fi Solutions | ThinkPalmIntelligent Home Wi-Fi Solutions | ThinkPalm
Intelligent Home Wi-Fi Solutions | ThinkPalm
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStack
 
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
Russian Call Girls in Karol Bagh Aasnvi ➡️ 8264348440 💋📞 Independent Escort S...
 
React Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief UtamaReact Server Component in Next.js by Hanief Utama
React Server Component in Next.js by Hanief Utama
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need It
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
 
Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024
 
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
办理学位证(UQ文凭证书)昆士兰大学毕业证成绩单原版一模一样
 
software engineering Chapter 5 System modeling.pptx
software engineering Chapter 5 System modeling.pptxsoftware engineering Chapter 5 System modeling.pptx
software engineering Chapter 5 System modeling.pptx
 

Wcre13a.ppt

  • 1. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work On the Effect of Program Exploration on Maintenance Tasks Z´ephyrin Soh, Foutse Khomh, Yann-Ga¨el Gu´eh´eneuc, Giuliano Antoniol, Bram Adams Department of Computer and Software Engineering ´Ecole Polytechnique de Montr´eal, Qu´ebec, Canada October 14, 2013 Pattern Trace Identification, Detection, and Enhancement in Java SOftware Cost-effective Change and Evolution Research Lab
  • 2. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Outline Introduction User Study Empirical Study Conclusion and Future Work 2 / 23
  • 3. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (1/3) Program Exploration Developers interact with program entities, e.g., open files, edit methods, etc. Developers explore the program, i.e., move from program entities to program entities. Types of program entities. 3 / 23
  • 4. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) 4 / 23
  • 5. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) 4 / 23
  • 6. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) 4 / 23
  • 7. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) 4 / 23 Containment principle: Aggregate the exploration graph at class level
  • 8. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (2/3) Exploration graph aggregated at class level 4 / 23
  • 9. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (3/3) 5 / 23
  • 10. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (3/3) 5 / 23
  • 11. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Context and Example (3/3) 5 / 23 A graph represents a program exploration (a) Graph 1 (b) Graph 2 Is there any difference between exploration graphs? How to find the difference? How does the difference affect the maintenance tasks?
  • 12. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Why do we care? (1/3) Research on Program Exploration Developers spend on average 35% of their time navigating within and between source files [1] Developers perform on average 19.31 navigation actions between two modifications [2] [1] Ko et al., An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. TSE 2006 [2] D. R¨othlisberger, SmartGroups: Focusing on Task-Relevant Source Artifacts in IDEs, ICPC 2011 6 / 23
  • 13. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Why do we care? (2/3) Reasons of Program Exploration Developers may have several different ways to explore a program e.g., Looking for relevant entities ⇒ Some developers (may) not know where to look at. Validate previous changes ⇒ Developers (may) know the target entity to look at. ⇒ In some cases, developers may perform more back and forth navigations (i.e., revisitation) on program entities 7 / 23
  • 14. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Introduction Why do we care? (3/3) Revisitation-based Exploration Some recommendation systems (e.g., Mylyn [3], NavClus [4]) are based on the revisitation of program entities We wonder if revisitation is “good“ or “bad“ ⇒ Knowing that is helpful to improve revisitation-based approach for recommendation systems. [3] Kersten and Murphy, Mylar: a degree-of-interest model for IDEs, AOSD 2005 [4] Lee and Kang, Clustering and recommending collections of code relevant to tasks, ICSM 2011 8 / 23
  • 15. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Research Question Exploration Strategies In the revisitation-based exploration, we focus on two extreme cases: Referenced Exploration (RE): A developer revisits one (or a set of) entity(ies) already visited ⇒ referenced entities Unreferenced Exploration (UE): There is no set of referenced entities ⇒ Almost the same revisitation of program entity(ies) 9 / 23
  • 16. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Research Question Exploration Strategies In the revisitation-based exploration, we focus on two extreme cases: Referenced Exploration (RE): A developer revisits one (or a set of) entity(ies) already visited ⇒ referenced entities Unreferenced Exploration (UE): There is no set of referenced entities ⇒ Almost the same revisitation of program entity(ies) Do developers follow a referenced exploration when performing maintenance tasks? 9 / 23
  • 17. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (1/5) Data Collection Mylyn’s IHs appear as attachment (XML file with name “mylyn-context.zip”) to a bug 10 / 23
  • 18. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (1/5) Data Collection Mylyn’s IHs appear as attachment (XML file with name “mylyn-context.zip”) to a bug Top four Eclipse projects with more IHs (from 2,601 bugs) ECF Mylyn PDE Platform Total # IHs 26 1,273 131 275 1,705 10 / 23
  • 19. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? Random sample of IHs (proportional among projects) 11 / 23
  • 20. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? Random sample of IHs (proportional among projects) 11 / 23
  • 21. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs Random sample of IHs (proportional among projects) 11 / 23
  • 22. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs Random sample of IHs (proportional among projects) 11 / 23
  • 23. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs RE UE D Random sample of IHs (proportional among projects) 11 / 23
  • 24. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (2/5) How do we build an oracle? 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs RE UE D Random sample of IHs (proportional among projects) How much the subjects agreed? Fliess' Kappa interater agreement coeficient - Overall: fair agreement (0.36) - RE: almost moderate agreement (0.38) - UE: almost moderate agreement (0.39) - Doubt: No agreement (-0.009) 11 / 23
  • 25. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (3/5) How do we aggregate the results? RE UE DRE UE 12 / 23
  • 26. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (3/5) How do we aggregate the results? RE UE DRE UE >= 2/3 subjects RE 12 / 23
  • 27. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (3/5) How do we aggregate the results? RE UE DRE UE UE >= 2/3 subjects 12 / 23
  • 28. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (3/5) How do we aggregate the results? RE UE DRE UE < 2/3 subjects D < 2/3 subjects 12 / 23
  • 29. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (4/5) Identification Technique Subjects count the number of nodes and the num- ber of in/out arrows in the graphs ⇒ look at the distribution/inequality of revisits across graphs 13 / 23
  • 30. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (4/5) Identification Technique Subjects count the number of nodes and the num- ber of in/out arrows in the graphs ⇒ look at the distribution/inequality of revisits across graphs Gini Inequality Index [5] We measure how much classes are (in)equally revisited Used in econometrics to measure the inequality of income among a population Population = classes Income of a class = NumRevisits Gini(IH) = 1 2n2µ n i=1 n j=1 | NumRevisit(eni ) − NumRevisit(enj ) | [5] K. Xu, How has the literature on Gini’s index evolved in the past 80 years?, Technical report, Department of Economics, Dalhouse University, 2004 13 / 23
  • 31. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Define a threshold to know if classes are (in)equally revisited If Gini(IH) < Threshold ⇒ Classes are almost equally revisited ⇒ UE: Unreferenced Exploration If Gini(IH) ≥ Threshold ⇒ Some classes are more revisited than others ⇒ RE: Referenced Exploration 14 / 23
  • 32. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality 14 / 23
  • 33. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality Threshold ? 14 / 23
  • 34. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold RE 14 / 23
  • 35. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality RE 0.28 14 / 23
  • 36. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality RE 0.28 - Threshold = 0.2 ==> RE - Threshold = 0.3 ==> UE 0.2 0.3 14 / 23
  • 37. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality RE Use F-Measure to maximize both precision and recall 0.4 14 / 23
  • 38. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Approach (5/5) Inequality Index Threshold Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4 14 / 23
  • 39. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (1/2) Percentage RE vs. UE RE: 28.03% of Interaction Histories UE: 71.96% of Interaction Histories ⇒ Developers follow mostly the unreferenced exploration (UE) when performing a maintenance task. 15 / 23
  • 40. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (1/2) Percentage RE vs. UE RE: 28.03% of Interaction Histories UE: 71.96% of Interaction Histories ⇒ Developers follow mostly the unreferenced exploration (UE) when performing a maintenance task. Methodical developers do not reinvestigate methods as frequently as opportunistic developers [6] RE = opportunistic developers? UE = methodical developers? ⇒ We need more investigations [6] Robillard et al., How effective developers investigate source code: An exploratory study, TSE 2004 15 / 23
  • 41. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system 16 / 23
  • 42. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system 16 / 23
  • 43. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference. 16 / 23
  • 44. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference. Task interruption: The interruption time seems to push developers to be concentrate on a set of (referenced) entities (RE) 16 / 23
  • 45. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference. Task interruption: The interruption time seems to push developers to be concentrate on a set of (referenced) entities (RE) Type of the task: Bug severity because developers may have more back and forth navigation when fixing severe bugs wrt. less severe bugs Inequality Indexes are not significantly different for different types of bugs. 16 / 23
  • 46. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study Research Questions Research Questions Does any difference exist in maintenance time be- tween RE and UE? Does any difference exist in edit/navigation ratio between RE and UE? 17 / 23
  • 47. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study RQ1 Approach Overall Time spend = entity∈IH Duration(entity) #entity Unpaired Wilcoxon test 18 / 23
  • 48. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study RQ1 Approach Overall Time spend = entity∈IH Duration(entity) #entity Unpaired Wilcoxon test Results The UE is on average 12.30% less time consuming than the RE. 18 / 23
  • 49. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study RQ2 Approach Edit/navigation ratio = NumEdit(IH) NumEvent(IH) Unpaired Wilcoxon test 19 / 23
  • 50. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study RQ2 Approach Edit/navigation ratio = NumEdit(IH) NumEvent(IH) Unpaired Wilcoxon test Results An UE requires less edit/navigation ratio than a RE. 19 / 23
  • 51. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study Discussions 20 / 23 Is there a good strategy? RE (more) UE (less) ⇒ The back and forth actions on ref- erenced entities are time consuming RE (more) UE (less) ⇒ More edit actions compare to nav- igation actions. Wrong edits/modif. then come back to revert/cancel?
  • 52. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study Discussions 20 / 23 Is there a good strategy? RE (more) UE (less) ⇒ The back and forth actions on ref- erenced entities are time consuming RE (more) UE (less) ⇒ More edit actions compare to nav- igation actions. Wrong edits/modif. then come back to revert/cancel? F1 F2 F1 F3 F1 F4 F2 F5 RE F1 F2 F3 F1 F4 F5 UE 4/8 3/6 4/8 > 3/6 (RE better)
  • 53. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Empirical Study Discussions 20 / 23 Is there a good strategy? RE (more) UE (less) ⇒ The back and forth actions on ref- erenced entities are time consuming RE (more) UE (less) ⇒ More edit actions compare to nav- igation actions. Wrong edits/modif. then come back to revert/cancel? F1 F2 F1 F3 F1 F4 F2 F5 RE F1 F2 F3 F1 F4 F5 UE 4/8 3/6 4/8 > 3/6 (RE better) F1 F2 F1 F3 F1 F4 F2 F5 RE F1 F2 F3 F1 F4 F5 UE 4/8 2/8 3/6 2/6 4/8 > 3/6 (RE better) 2/8 < 2/6 (UE better)
  • 54. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Conclusion 21 / 23 Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4
  • 55. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Conclusion 21 / 23 Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4
  • 56. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Conclusion 21 / 23 Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4
  • 57. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work 22 / 23
  • 58. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data 22 / 23
  • 59. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data Internal validity: We use only Mylyn’s IH ⇒ Use the IH from other tools 22 / 23
  • 60. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Conclusion and Future Work Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data Internal validity: We use only Mylyn’s IH ⇒ Use the IH from other tools External validity: Our subject projets are Eclipse-based projects ⇒ Use other systems 22 / 23
  • 61. Program Exploration Z´ephyrin Soh et al. Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 23 / 23 Thanks for your attention! Perfect equality 0 1 Maximum inequality RE UE RE Use F-Measure to maximize both precision and recall 0.4