Wcre13a.ppt
Upcoming SlideShare
Loading in...5
×
 

Wcre13a.ppt

on

  • 226 views

Developers' effort, exploration, strategies, Mylyn

Developers' effort, exploration, strategies, Mylyn

Statistics

Views

Total Views
226
Views on SlideShare
186
Embed Views
40

Actions

Likes
0
Downloads
0
Comments
0

2 Embeds 40

http://www.ptidej.net 39
http://squirrel.polymtl.ca 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Wcre13a.ppt Wcre13a.ppt Presentation Transcript

  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results On the Effect of Program Exploration on Maintenance Tasks Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Z´phyrin Soh, Foutse Khomh, Yann-Ga¨l Gu´h´neuc, e e e e Giuliano Antoniol, Bram Adams Department of Computer and Software Engineering ´ Ecole Polytechnique de Montr´al, Qu´bec, Canada e e October 14, 2013 Pattern Trace Identification, Detection, and Enhancement in Java SOftware Cost-effective Change and Evolution Research Lab
  • Program Exploration Outline Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Introduction Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Empirical Study Conclusion and Future Work 2 / 23
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example (1/3) Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 3 / 23 Program Exploration Developers interact with program entities, e.g., open files, edit methods, etc. Developers explore the program, i.e., move from program entities to program entities. Types of program entities.
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 4 / 23 Introduction Context and Example (2/3)
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 4 / 23 Introduction Context and Example (2/3)
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 4 / 23 Introduction Context and Example (2/3)
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 4 / 23 Containment Introduction principle: the exploration graph Context and Example (2/3) Aggregate at class level
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example (2/3) Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 4 / 23 Exploration graph aggregated at class level
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 5 / 23 Introduction Context and Example (3/3)
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 5 / 23 Introduction Context and Example (3/3)
  • Program Exploration Introduction Z´phyrin Soh et al. e Context and Example (3/3) A graph represents a program exploration Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work (a) Graph 1 (b) Graph 2 Is there any difference between exploration graphs? How to find the difference? How does the difference affect the maintenance tasks? 5 / 23
  • Program Exploration Z´phyrin Soh et al. e Introduction Why do we care? (1/3) Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Research on Program Exploration Developers spend on average 35% of their time navigating within and between source files [1] Developers perform on average 19.31 navigation actions between two modifications [2] Conclusion Threats to Validity and Future Work [1] Ko et al., An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. TSE 2006 [2] D. R¨thlisberger, SmartGroups: Focusing on Task-Relevant Source o Artifacts in IDEs, ICPC 2011 6 / 23
  • Program Exploration Z´phyrin Soh et al. e Introduction Why do we care? (2/3) Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Reasons of Program Exploration Developers may have several different ways to explore a program e.g., Looking for relevant entities ⇒ Some developers (may) not know where to look at. Validate previous changes ⇒ Developers (may) know the target entity to look at. ⇒ In some cases, developers may perform more back and forth navigations (i.e., revisitation) on program entities 7 / 23
  • Program Exploration Z´phyrin Soh et al. e Introduction Why do we care? (3/3) Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Revisitation-based Exploration Some recommendation systems (e.g., Mylyn [3], NavClus [4]) are based on the revisitation of program entities We wonder if revisitation is “good“ or “bad“ ⇒ Knowing that is helpful to improve revisitation-based approach for recommendation systems. [3] Kersten and Murphy, Mylar: a degree-of-interest model for IDEs, AOSD 2005 [4] Lee and Kang, Clustering and recommending collections of code relevant to tasks, ICSM 2011 8 / 23
  • Program Exploration Z´phyrin Soh et al. e User Study Research Question Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 9 / 23 Exploration Strategies In the revisitation-based exploration, we focus on two extreme cases: Referenced Exploration (RE): A developer revisits one (or a set of) entity(ies) already visited ⇒ referenced entities Unreferenced Exploration (UE): There is no set of referenced entities ⇒ Almost the same revisitation of program entity(ies)
  • Program Exploration Z´phyrin Soh et al. e User Study Research Question Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Exploration Strategies In the revisitation-based exploration, we focus on two extreme cases: Referenced Exploration (RE): A developer revisits one (or a set of) entity(ies) already visited ⇒ referenced entities Unreferenced Exploration (UE): There is no set of referenced entities ⇒ Almost the same revisitation of program entity(ies) Do developers follow a referenced exploration when performing maintenance tasks? 9 / 23
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 10 / 23 User Study Approach (1/5) Data Collection Mylyn’s IHs appear as attachment (XML file with name “mylyn-context.zip”) to a bug
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results User Study Approach (1/5) Data Collection Mylyn’s IHs appear as attachment (XML file with name “mylyn-context.zip”) to a bug Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Top four Eclipse projects with more IHs (from 2,601 bugs) ECF Mylyn PDE Platform Total # IHs 26 1,273 131 275 1,705 10 / 23
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Approach (2/5) How do we build an oracle? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 11 / 23 Random sample of IHs (proportional among projects)
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Approach (2/5) How do we build an oracle? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 11 / 23 Random sample of IHs (proportional among projects)
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (2/5) Introduction How do we build an oracle? Context and Example Why do we care? User Study Research Question Approach Results Random sample of IHs (proportional among projects) Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs 11 / 23
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (2/5) Introduction How do we build an oracle? Context and Example Why do we care? User Study Research Question Approach Results Random sample of IHs (proportional among projects) Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs 11 / 23
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (2/5) Introduction How do we build an oracle? Context and Example Why do we care? User Study Research Question Approach Results Random sample of IHs (proportional among projects) Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs RE 11 / 23 UE D
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (2/5) Introduction How do we build an oracle? Context and Example Why do we care? User Study Research Question Approach Results Random sample of IHs (proportional among projects) Empirical Study Research Questions RQ1 RQ2 Discussions How much the subjects agreed? Fliess' Kappa interater agreement coeficient Conclusion and Future Work - Overall: fair agreement (0.36) - RE: almost moderate agreement (0.38) - UE: almost moderate agreement (0.39) - Doubt: No agreement (-0.009) Conclusion Threats to Validity and Future Work 9 subjects Java experience - mean : 4.16 yrs -sd : 2.80 yrs RE 11 / 23 UE D
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (3/5) Introduction Context and Example Why do we care? How do we aggregate the results? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work RE 12 / 23 RE UE UE D
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (3/5) Introduction Context and Example Why do we care? How do we aggregate the results? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work RE RE UE >= 2/3 subjects RE 12 / 23 UE D
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (3/5) Introduction Context and Example Why do we care? How do we aggregate the results? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work RE RE UE UE >= 2/3 subjects UE 12 / 23 D
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (3/5) Introduction Context and Example Why do we care? How do we aggregate the results? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work RE RE UE < 2/3 subjects < 2/3 subjects D 12 / 23 UE D
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 13 / 23 User Study Approach (4/5) Identification Technique Subjects count the number of nodes and the number of in/out arrows in the graphs ⇒ look at the distribution/inequality of revisits across graphs
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results User Study Approach (4/5) Identification Technique Subjects count the number of nodes and the number of in/out arrows in the graphs ⇒ look at the distribution/inequality of revisits across graphs Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Gini Inequality Index [5] We measure how much classes are (in)equally revisited Used in econometrics to measure the inequality of income among a population Population = classes Income of a class = NumRevisits Gini(IH) = 1 2n2 µ n n i=1 j=1 | NumRevisit(eni ) − NumRevisit(enj ) | [5] K. Xu, How has the literature on Gini’s index evolved in the past 80 years?, Technical report, Department of Economics, Dalhouse University, 2004 13 / 23
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (5/5) Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 14 / 23 Inequality Index Threshold Define a threshold to know if classes are (in)equally revisited If Gini(IH) < Threshold ⇒ Classes are almost equally revisited ⇒ UE: Unreferenced Exploration If Gini(IH) ≥ Threshold ⇒ Some classes are more revisited than others ⇒ RE: Referenced Exploration
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (5/5) Introduction Inequality Index Threshold Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 0 Perfect equality 14 / 23 1 Maximum inequality
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (5/5) Introduction Inequality Index Threshold Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Threshold ? Conclusion and Future Work Conclusion Threats to Validity and Future Work 0 Perfect equality 14 / 23 1 Maximum inequality
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Approach (5/5) Inequality Index Threshold User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 14 / 23 RE
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (5/5) Introduction Inequality Index Threshold Context and Example Why do we care? User Study Research Question Approach Results RE Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work 0.28 Conclusion Threats to Validity and Future Work 0 Perfect equality 14 / 23 1 Maximum inequality
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (5/5) Introduction Inequality Index Threshold Context and Example Why do we care? User Study Research Question Approach Results RE Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work 0.28 Conclusion Threats to Validity and Future Work 0 Perfect equality 14 / 23 0.2 0.3 - Threshold = 0.2 ==> RE - Threshold = 0.3 ==> UE 1 Maximum inequality
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (5/5) Introduction Inequality Index Threshold Context and Example Why do we care? User Study Research Question Approach Results RE Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work 0.4 Conclusion Threats to Validity and Future Work 0 Perfect equality 1 Maximum inequality Use F-Measure to maximize both precision and recall 14 / 23
  • Program Exploration Z´phyrin Soh et al. e User Study Approach (5/5) Introduction Inequality Index Threshold Context and Example Why do we care? User Study Research Question Approach Results RE Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work 0.4 Conclusion Threats to Validity and Future Work 0 Perfect equality 1 UE RE Maximum inequality Use F-Measure to maximize both precision and recall 14 / 23
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 15 / 23 User Study Results (1/2) Percentage RE vs. UE RE: 28.03% of Interaction Histories UE: 71.96% of Interaction Histories ⇒ Developers follow mostly the unreferenced exploration (UE) when performing a maintenance task.
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (1/2) Percentage RE vs. UE RE: 28.03% of Interaction Histories UE: 71.96% of Interaction Histories ⇒ Developers follow mostly the unreferenced exploration (UE) when performing a maintenance task. Methodical developers do not reinvestigate methods as frequently as opportunistic developers [6] RE = opportunistic developers? UE = methodical developers? ⇒ We need more investigations [6] Robillard et al., How effective developers investigate source code: An exploratory study, TSE 2004 15 / 23
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 16 / 23 User Study Results (2/2) Confounding Factors Architecture of the system
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 16 / 23 User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 16 / 23 User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference.
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 16 / 23 User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference. Task interruption: The interruption time seems to push developers to be concentrate on a set of (referenced) entities (RE)
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work User Study Results (2/2) Confounding Factors Architecture of the system Number of common classes (NCC): The more A and B have CC, the more they used the same part of the system Compute NCC for each pair of IH. NCC pairs with same ES vs. NCC pairs with different ES ⇒ No difference. Task interruption: The interruption time seems to push developers to be concentrate on a set of (referenced) entities (RE) Type of the task: Bug severity because developers may have more back and forth navigation when fixing severe bugs wrt. less severe bugs Inequality Indexes are not significantly different for different types of bugs. 16 / 23
  • Program Exploration Z´phyrin Soh et al. e Empirical Study Research Questions Introduction Context and Example Why do we care? User Study Research Question Approach Results Research Questions Empirical Study Research Questions RQ1 RQ2 Discussions Does any difference exist in maintenance time between RE and UE? Conclusion and Future Work Conclusion Threats to Validity and Future Work 17 / 23 Does any difference exist in edit/navigation ratio between RE and UE?
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 18 / 23 Empirical Study RQ1 Approach Overall Time spend = Unpaired Wilcoxon test Duration(entity ) #entity entity ∈IH
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study RQ1 Approach Overall Time spend = Duration(entity ) #entity entity ∈IH Unpaired Wilcoxon test Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 18 / 23 Results The UE is on average 12.30% less time consuming than the RE.
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 19 / 23 Empirical Study RQ2 Approach Edit/navigation ratio = Unpaired Wilcoxon test NumEdit(IH) NumEvent(IH)
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 19 / 23 Empirical Study RQ2 Approach Edit/navigation ratio = NumEdit(IH) NumEvent(IH) Unpaired Wilcoxon test Results An UE requires less edit/navigation ratio than a RE.
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 20 / 23 Empirical Study Discussions Is there a good strategy? RE (more) ⇒ The back and forth actions on refUE (less) erenced entities are time consuming ⇒ More edit actions compare to navRE (more) igation actions. Wrong edits/modif. UE (less) then come back to revert/cancel?
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? Empirical Study Discussions Is there a good strategy? RE (more) ⇒ The back and forth actions on refUE (less) erenced entities are time consuming User Study Research Question Approach Results Empirical Study ⇒ More edit actions compare to navRE (more) igation actions. Wrong edits/modif. UE (less) then come back to revert/cancel? Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work RE F1 F2 F1 F3 F1 F4 F1 F2 F3 F1 F4 F5 UE 20 / 23 4/8 > 3/6 (RE better) F2 3/6 F5 4/8
  • Program Exploration Z´phyrin Soh et al. e Introduction Context and Example Why do we care? Empirical Study Discussions Is there a good strategy? RE (more) ⇒ The back and forth actions on refUE (less) erenced entities are time consuming User Study Research Question Approach Results Empirical Study ⇒ More edit actions compare to navRE (more) igation actions. Wrong edits/modif. UE (less) then come back to revert/cancel? Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work RE F1 F2 F1 F3 F1 F4 F1 F2 F3 F1 F4 F5 UE 20 / 23 4/8 > 3/6 (RE better) F2 3/6 F5 4/8 2/8 2/6 2/8 < 2/6 (UE better)
  • Program Exploration Z´phyrin Soh et al. e Conclusion and Future Work Conclusion Introduction Context and Example Why do we care? User Study Research Question Approach Results RE Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 21 / 23 0.4 0 Perfect equality 1 UE RE Maximum inequality Use F-Measure to maximize both precision and recall
  • Program Exploration Z´phyrin Soh et al. e Conclusion and Future Work Conclusion Introduction Context and Example Why do we care? User Study Research Question Approach Results RE Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 21 / 23 0.4 0 Perfect equality 1 UE RE Maximum inequality Use F-Measure to maximize both precision and recall
  • Program Exploration Z´phyrin Soh et al. e Conclusion and Future Work Conclusion Introduction Context and Example Why do we care? User Study Research Question Approach Results RE Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 21 / 23 0.4 0 Perfect equality 1 UE RE Maximum inequality Use F-Measure to maximize both precision and recall
  • Program Exploration Z´phyrin Soh et al. e Conclusion and Future Work Threats to Validity and Future Work Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 22 / 23 Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work
  • Program Exploration Z´phyrin Soh et al. e Conclusion and Future Work Threats to Validity and Future Work Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 22 / 23 Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data
  • Program Exploration Z´phyrin Soh et al. e Conclusion and Future Work Threats to Validity and Future Work Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 22 / 23 Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data Internal validity: We use only Mylyn’s IH ⇒ Use the IH from other tools
  • Program Exploration Z´phyrin Soh et al. e Conclusion and Future Work Threats to Validity and Future Work Introduction Context and Example Why do we care? User Study Research Question Approach Results Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work Threats to Validity and Future Work Construct validity: An IH can be a part of developers’ work Conclusion validity: The time recorded can be different to the “real” time spent ⇒ Perform an experiment to collect a data Internal validity: We use only Mylyn’s IH ⇒ Use the IH from other tools External validity: Our subject projets are Eclipse-based projects ⇒ Use other systems 22 / 23
  • Program Exploration Z´phyrin Soh et al. e Thanks for your attention! Introduction Context and Example Why do we care? User Study Research Question Approach Results RE Empirical Study Research Questions RQ1 RQ2 Discussions Conclusion and Future Work Conclusion Threats to Validity and Future Work 23 / 23 0.4 0 Perfect equality 1 UE RE Maximum inequality Use F-Measure to maximize both precision and recall