What is the Long-Term Impact of Changes?

827 views

Published on

Presentation by Irina Brudaru at the RSSE 2008 Workshop.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
827
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

What is the Long-Term Impact of Changes?

  1. 1. What is the Long-Term Impact of Changes? Irina Ioana Brudaru • Andreas Zeller Saarland University, Germany
  2. 2. medical school Italy CS Erasmus (Trento) Highschool Germany (Braunschweig) Master Saarbrücken UK (MPI) PhD Saarbrücken Industry (Computational Linguistics)
  3. 3. medical school Italy CS Erasmus (Trento) Highschool What if? Germany (Braunschweig) Master Saarbrücken UK (MPI) PhD Saarbrücken Industry (Computational Linguistics)
  4. 4. medical school Italy or y! CS Erasmus hi st (Trento) Highschool t oj ec pr ” in Whathat if? if ? Germany (Braunschweig) “W no Master ve e ha Saarbrücken UK W (MPI) PhD Saarbrücken Industry (Computational Linguistics)
  5. 5. Assessing Impact Impact of a change = Its effect throughout the lifetime of a project Good Changes: Bad Changes: • Fixes • Bugs • Refactorings • Bug-Inducing
  6. 6. Impact • How do we know a change A impacts another change B? • General solution: change impact analysis (a generally undecidable problem) • Pragmatic solution: B uses (or redefines) an entity (class, method, variable) defined by A
  7. 7. AnnotationTestMixed 1.5 1.2 1.4 1.1 1.7 1.3 1.6 Version archives + bug information = change genealogy graph
  8. 8. AnnotationTestMixed 1.5 1.2 1.4 1.1 1.7 1.3 1.6 Version archives + bug information = change genealogy graph
  9. 9. AnnotationTestMixed 1.5 1.2 1.4 1.1 1.7 1.3 1.6 Version archives + bug information = change genealogy graph
  10. 10. AnnotationTestMixed 1.5 1.2 1.4 1.1 1.7 1.3 1.6 Version archives + bug information = change genealogy graph
  11. 11. AnnotationTestMixed 1.5 1.2 1.4 1.1 1.7 1.3 1.6 Version archives + bug information = change genealogy graph
  12. 12. AnnotationTestMixed 1.5 1.2 1.4 1.1 1.7 1.3 1.6 Version archives + bug information = change genealogy graph
  13. 13. AnnotationTestMixed 1.5 ! 1.2 1.4 1.1 1.7 1.3 1.6 Version archives + bug information = change genealogy graph
  14. 14. 58 EclipseFile 25 71 28 17 55 24 33 60 43 69 49 61 56 27 38 23 39 72 46 67 34 8 30 44 54 2 40 41 59 13 22 52 50 5 16 51 18 4 11 20 53 12 36 42 47
  15. 15. 58 EclipseFile 25 71 28 Log message: Bug 33792 No progress dialog at start of replaceWith command[...]This 17 55 caused many 24 33 dependencies to be 60 affected. 43 69 49 61 56 27 38 23 39 72 46 67 34 8 30 44 54 2 40 41 59 13 22 52 50 5 16 51 18 4 11 20 53 12 36 42 47
  16. 16. 58 EclipseFile 25 71 28 Log message: Bug 33792 No progress dialog at start of replaceWith command[...]This 17 55 caused many 24 33 dependencies to be 60 affected. 43 69 49 61 56 27 38 23 39 72 46 67 34 8 30 44 54 2 40 41 59 13 22 52 50 5 16 51 18 4 11 20 53 Performing changes in a 12 36 42 properties file not 47 viewed as a change
  17. 17. 58 EclipseFile 25 71 28 Log message: Bug 33792 No progress dialog at start of replaceWith command[...]This 17 55 caused many 24 33 dependencies to be 60 affected. 43 69 49 61 56 27 38 23 39 72 46 67 34 8 30 44 54 15/24 2 40 41 59 13 22 52 50 5 16 51 18 4 11 20 53 Performing changes in a 12 36 42 properties file not 47 viewed as a change
  18. 18. 58 EclipseFile 25 71 28 Log message: Bug 33792 No progress dialog at start of replaceWith command[...]This 17 55 caused many 24 33 dependencies to be 60 affected. 43 69 49 61 56 27 38 23 39 72 46 67 34 8 30 44 54 15/24 2 40 41 59 13 22 52 50 5 16 51 18 4 11 20 53 Performing changes in a 12 36 properties file not 42 18/31 viewed as a change 47
  19. 19. 58 EclipseFile 25 71 28 Log message: Bug 33792 No progress dialog at start of replaceWith command[...]This 17 55 caused many 24 33 dependencies to be 60 affected. 43 69 49 61 56 27 38 23 39 72 46 67 34 8 30 44 54 0.62 2 40 41 59 13 22 52 50 5 16 51 18 4 11 20 53 Performing changes in a 12 36 properties file not 42 0.58 viewed as a change 47
  20. 20. Metrics for Changes • Quality – the average ratio fixes vs. features A change decreases quality if it spins off many fixes • Effort – the long-term maintenance effort A change increases effort if its descendants require lots of time • Maintainability – the average change spread A change decreases maintainability (and modularity) if its descendants change several locations at once
  21. 21. Future Work • More metrics to assess the long-term impact • Finer dependencies most important: distinguish between method targets • Similarity measures for recommendations and evaluate their suitability for making accurate predictions
  22. 22. Recommendations
  23. 23. Related Work
  24. 24. Related Work Bug 42233 was reported. 1.11 1.14 1.16 1.18 "#$%&'$ !quot;#$%&'$ (quot;#$%&'$ ./0,-$12+$ ()&*+,- ()&*+,- ()&*+,- 34455 6)&*+,-7 "#$!quot;#$(quot;# Figure 3: Relating changes to locations. Hatari Figure 2: Locating fix-inducing changes for bug 42233 Right now, we use the CVS annotate command, but it is straight- 4. HOW HATARI WORKS forward to implement a similar feature for other version control Let us now briefly discuss how HATARI obtains the risk informa- systems. tion. HATARI proceeds in four automatic steps: HATARI uses these annotations and the set of changed lines to find candidates for fix-inducing changes. For our example, we as- 1. HATARI identifies the fixes made to a software system. sume that lines 20, 40, and 60 have been changed; thus our candi- 2. For each of these fixes, HATARI identifies the fix-inducing dates are the changes leading to revisions 1.11, 1.14, and 1.16. change(s) that last touched the fixed locations. Finally, HATARI uses the bug database to rule out changes that cannot be fix-inducing because they have been made after the bug 3. For each of the fix-inducing changes, HATARI determines was reported, i.e., they cannot be real causes for the bug. In our their location. example, revisions 1.14 and 1.16 are not fix-inducing. Without the connection to the bug database, they would be marked fix-inducing 4. Finally, for every location in the source code, HATARI mea- and therefore be false positives. sures their risk of change. In our example, revision 1.11 is the only fix-inducing change. Frequently, but not always, such fix-inducing changes introduced The following sections provide details on these steps. the bug that has been fixed later on. However, such changes are always unstable and thus risky. 4.1 Identifying Fixes Formally, we define a induced-change relation J ⊆ C × C that In order to locate fix-inducing changes, HATARI first needs to know connects two changes ci and cj with each other if and only if cj if a change is a fix. While advanced version control systems al- changed a line that was introduced in ci ; in terms of CVS this means
  25. 25. Related Work some additional information relevant to th Original Program P Set of Unit Tests Changed Program P’ Bug 42233 was reported. (e.g., the choice of call graph constructio used and some policy settings for dealing Call Graph Builder Users can also point Chianti directly at a sentation of the call graphs that are to be CHIANTI enable the use of call graphs that have be Call Graphs of Tests in Call Graphs of Tests external tools. [5] presented the experime Atomic Change P in P’ call graphs. Decoder 1.11 1.14 1.16 1.18 When the analysis results are available anti results view will stack on top of the o "#$%&'$ !quot;#$%&'$ (quot;#$%&'$ ./0,-$12+$ Atomic Changes Java perspective. Since Chianti is expec ()&*+,- ()&*+,- ()&*+,- & Dependences 34455 programming and debugging, we integra Java perspective rather than define a new 6)&*+,-7 Change Impact Chianti results view provides users two w "#$!quot;#$(quot;# Affected Tests Analyzer Affecting Changes the analysis results. • The affecting changes view shows a Figure1: Chianti architecture. Figure 3: Relating changes to locations. Hatari Chianti Figure 2: Locating fix-inducing changes for bug 42233 view. Each affected test can be exp set of affecting changes. Each affec atomic change that can be expand Rightby isolating theuse the CVS annotate command, but it is straight- now, we changes responsible for the failure, Chianti 4. HOW HATARI WORKS show its prerequisite changes. This an idea of the different “threads” of forward been integrated closely a similar afeature for other version control has to implement with Eclipse, widely used ex- occurred. Let us now briefly discuss how HATARI obtains the risk informa- tensible open-source development environment for Java (see systems. www.eclipse.org). • The atomic-changes-by-category view tion. HATARI proceeds in four automatic steps: HATARI uses these annotations and the set of changed lines to ent atomic changes grouped by cate 2. CHIANTI PROTOTYPE changes and dependences shown in find candidates for as an Eclipse plugin,changes. For our example, we as- tests, it just s fix-inducing which contributes related to any specific 1. HATARI identifies the fixes made to a software system. Chianti is designed sults of comparison of two versions o sume athat lines 20, 40, aand 60 have been Java launch configuration and Chianti results view to the changed; thus our candi- perspective. Chianti conceptually can be divided into three Each of these user interface components 2. For each of these fixes, HATARI identifies the fix-inducing dates functional parts. One part is responsiblerevisions a1.11, 1.14, with the standard Java IDE in Ecl are the changes leading to for deriving set grated and 1.16. of atomic changes from two versions of a Java project, which change(s) that last touched the fixed locations. is achieved via a pairwise comparison of database to on an atomic change Finally, HATARI uses the bugthe abstract syntax rule outonchanges in the affecting chan editor that the associated program fragmen cannot betest call graphsthe two project versions. Another part made after the bug trees of fix-inducing because they have been the classes in reads for the original and edited projects, 3. theREFERENCES was reported, i.e.,tests andcannot be real causes for Shawn bug. In our S. Arnold. An computes affected they their affecting changes. The 3. For each of the fix-inducing changes, HATARI determines [1] A. Bohner and Robert their location. third revisions 1.14 that 1.16 user not fix-inducing. Without the In Shawn A change impact information. Chianti’s the example,part manages the viewsandallowGUIare to visualize also includes a software change impact analysis. Robert S. Arnold, editors, Software Change pages 1–26. IEEE Computer Society Press, 1 connection to bethe bug the set of tests associated with the launch configuration thatdatabase,to selectwould be marked Law and Gregg Rothermel. Whole pro versions to analyzed, allows users they the project [2] James fix-inducing dynamic impact analysis. In Proc. of the Int 4. Finally, for every location in the source code, HATARI mea- and therefore the call graphs to be used. Figure 1 depicts project and be false positives. on Software Engineering, pages 308–318, 20 [3] Alessandro Orso, Taweesup Apiwattanapong Chianti’s architecture. sures their risk of change. In our example, isrevision 1.11 is the we cur- fix-inducing change.Harrold. An Proc Although Chianti intended for interactive use, only Rothermel, and Mary Jean dynamic impact analysis algorithms. In emp rently require two separate Java not always,of such fix-inducing two versions program are Frequently, butthat projects, insteadaof comparing saved in changes Edinburgh, on Software Engineerin the cur- 491–500, introduced International Conf. Scotland, 2004. The following sections provide details on these steps. the bug rent that has its local fixed Thus, a on. However, such changes are for impact an version with been history. later typical scenario [4] Alessandro Orso, Taweewup Apiwattanapon Harrold. Leveraging field data of a Chianti session begins with the programmer editing the testing. In Proc. of European Software Eng always unstable extracting therisky.stable version of this current project, and thus lastest ACM SIGSOFT Symp. on the Foundations 4.1 Identifying Fixes project from grammer we define a into the workspace. The con- Formally,thenCVS repository induced-change relation J ⊆ C × C that starts the change impact analysis launch pro- Engineering (ESEC/FSE’03), Helsinki, Fin 2003. [5] Xiaoxia Ren, Fenil Shah, Frank Tip, Barbara In order to locate fix-inducing changes, HATARI first needs to know figuration, and changes citwo these j with Currently, connects twosuiteselects these with projects of interest as well as the test associated Ophelia only if c tool for and cprojects. each other if andChesley.InChianti:ofAthe Conf.change Java programs. Proc. j on Ob if a change is a fix. While advanced version control systems al- changed a line that have a separate main() routine i ; in terms [6] Xiaoxia Ren, Fenil Shah, Frank Tip, Barbara we allow tests that was introduced in cand JUnit of CVS this means Programming, Systems, Languages and App tests
  26. 26. Related Work some additional information relevant to th Original Program P Set of Unit Tests Changed Program P’ Bug 42233 was reported. (e.g., the choice of call graph constructio used and some policy settings for dealing Call Graph Builder Users can also point Chianti directly at a sentation of the call graphs that are to be CHIANTI enable the use of call graphs that have be Call Graphs of Tests in Call Graphs of Tests external tools. [5] presented the experime Atomic Change P in P’ call graphs. Decoder 1.11 1.14 1.16 1.18 When the analysis results are available anti results view will stack on top of the o "#$%&'$ !quot;#$%&'$ (quot;#$%&'$ ./0,-$12+$ Atomic Changes Java perspective. Since Chianti is expec ()&*+,- ()&*+,- ()&*+,- & Dependences 34455 programming and debugging, we integra Java perspective rather than define a new 6)&*+,-7 Change Impact Chianti results view provides users two w "#$!quot;#$(quot;# Affected Tests Analyzer Affecting Changes the analysis results. • The affecting changes view shows a Figure1: Chianti architecture. Figure 3: Relating changes to locations. Hatari Chianti Figure 2: Locating fix-inducing changes for bug 42233 view. Each affected test can be exp set of affecting changes. Each affec atomic change that can be expand Rightby isolating theuse the CVS annotate command, but it is straight- now, we changes responsible for the failure, Chianti 4. HOW HATARI WORKS show its prerequisite changes. This an idea of the different “threads” of forward been integrated closely a similar afeature for other version control has to implement with Eclipse, widely used ex- occurred. Let us now briefly discuss how HATARI obtains the risk informa- tensible open-source development environment for Java (see systems. www.eclipse.org). • The atomic-changes-by-category view tion. HATARI proceeds in four automatic steps: HATARI uses these annotations and the set of changed lines to ent atomic changes grouped by cate 2. CHIANTI PROTOTYPE changes and dependences shown in find candidates for as an Eclipse plugin,changes. For our example, we as- tests, it just s fix-inducing which contributes related to any specific 1. HATARI identifies the fixes made to a software system. Chianti is designed sults of comparison of two versions o sume athat lines 20, 40, aand 60 have been Java launch configuration and Chianti results view to the changed; thus our candi- perspective. Chianti conceptually can be divided into three Each of these user interface components 2. For each of these fixes, HATARI identifies the fix-inducing dates functional parts. One part is responsiblerevisions a1.11, 1.14, with the standard Java IDE in Ecl are the changes leading to for deriving set grated and 1.16. of atomic changes from two versions of a Java project, which change(s) that last touched the fixed locations. is achieved via a pairwise comparison of database to on an atomic change Finally, HATARI uses the bugthe abstract syntax rule outonchanges in the affecting chan editor that the associated program fragmen cannot betest call graphsthe two project versions. Another part made after the bug trees of fix-inducing because they have been the classes in reads for the original and edited projects, 3. theREFERENCES was reported, i.e.,tests andcannot be real causes for Shawn bug. In our S. Arnold. An computes affected they their affecting changes. The 3. For each of the fix-inducing changes, HATARI determines [1] A. Bohner and Robert their location. third revisions 1.14 that 1.16 user not fix-inducing. Without the In Shawn A change impact information. Chianti’s the example,part manages the viewsandallowGUIare to visualize also includes a software change impact analysis. Robert S. Arnold, editors, Software Change pages 1–26. IEEE Computer Society Press, 1 connection to bethe bug the set of tests associated with the launch configuration thatdatabase,to selectwould be marked Law and Gregg Rothermel. Whole pro versions to analyzed, allows users they the project [2] James fix-inducing dynamic impact analysis. In Proc. of the Int 4. Finally, for every location in the source code, HATARI mea- and therefore the call graphs to be used. Figure 1 depicts project and be false positives. on Software Engineering, pages 308–318, 20 [3] Alessandro Orso, Taweesup Apiwattanapong Chianti’s architecture. sures their risk of change. In our example, isrevision 1.11 is the we cur- fix-inducing change.Harrold. An Proc Although Chianti intended for interactive use, only Rothermel, and Mary Jean dynamic impact analysis algorithms. In emp rently require two separate Java not always,of such fix-inducing two versions program are Frequently, butthat projects, insteadaof comparing saved in changes Edinburgh, on Software Engineerin the cur- 491–500, introduced International Conf. Scotland, 2004. The following sections provide details on these steps. the bug rent that has its local fixed Thus, a on. However, such changes are for impact an version with been history. later typical scenario [4] Alessandro Orso, Taweewup Apiwattanapon Change Distiller Harrold. Leveraging field data of a Chianti session begins with the programmer editing the testing. In Proc. of European Software Eng always unstable extracting therisky.stable version of this current project, and thus lastest ACM SIGSOFT Symp. on the Foundations 4.1 Identifying Fixes project from grammer we define a into the workspace. The con- Formally,thenCVS repository induced-change relation J ⊆ C × C that starts the change impact analysis launch pro- Engineering (ESEC/FSE’03), Helsinki, Fin 2003. [5] Xiaoxia Ren, Fenil Shah, Frank Tip, Barbara In order to locate fix-inducing changes, HATARI first needs to know figuration, and changes citwo these j with Currently, connects twosuiteselects these with projects of interest as well as the test associated Ophelia only if c tool for and cprojects. each other if andChesley.InChianti:ofAthe Conf.change Java programs. Proc. j on Ob if a change is a fix. While advanced version control systems al- changed a line that have a separate main() routine i ; in terms [6] Xiaoxia Ren, Fenil Shah, Frank Tip, Barbara we allow tests that was introduced in cand JUnit of CVS this means Programming, Systems, Languages and App tests
  27. 27. Related Work some additional information relevant to th Original Program P Set of Unit Tests Changed Program P’ Bug 42233 was reported. (e.g., the choice of call graph constructio used and some policy settings for dealing Call Graph Builder Users can also point Chianti directly at a sentation of the call graphs that are to be CHIANTI enable the use of call graphs that have be Call Graphs of Tests in Call Graphs of Tests external tools. [5] presented the experime Atomic Change P in P’ call graphs. Decoder 1.11 1.14 1.16 1.18 When the analysis results are available anti results view will stack on top of the o "#$%&'$ !quot;#$%&'$ (quot;#$%&'$ ./0,-$12+$ Atomic Changes Java perspective. Since Chianti is expec ()&*+,- ()&*+,- ()&*+,- & Dependences 34455 programming and debugging, we integra Java perspective rather than define a new 6)&*+,-7 Change Impact Chianti results view provides users two w "#$!quot;#$(quot;# Affected Tests Analyzer Affecting Changes the analysis results. • The affecting changes view shows a Figure1: Chianti architecture. Figure 3: Relating changes to locations. Hatari Chianti Figure 2: Locating fix-inducing changes for bug 42233 view. Each affected test can be exp set of affecting changes. Each affec atomic change that can be expand Rightby isolating theuse the CVS annotate command, but it is straight- now, we changes responsible for the failure, Chianti 4. HOW HATARI WORKS show its prerequisite changes. This an idea of the different “threads” of forward been integrated closely a similar afeature for other version control has to implement with Eclipse, widely used ex- occurred. Let us now briefly discuss how HATARI obtains the risk informa- tensible open-source development environment for Java (see systems. www.eclipse.org). • The atomic-changes-by-category view tion. HATARI proceeds in four automatic steps: HATARI uses these annotations and the set of changed lines to ent atomic changes grouped by cate 2. CHIANTI PROTOTYPE changes and dependences shown in find candidates for as an Eclipse plugin,changes. For our example, we as- tests, it just s fix-inducing which contributes related to any specific 1. HATARI identifies the fixes made to a software system. Chianti is designed sults of comparison of two versions o sume athat lines 20, 40, aand 60 have been Java launch configuration and Chianti results view to the changed; thus our candi- perspective. Chianti conceptually can be divided into three Each of these user interface components 2. For each of these fixes, HATARI identifies the fix-inducing dates functional parts. One part is responsiblerevisions a1.11, 1.14, with the standard Java IDE in Ecl are the changes leading to for deriving set grated and 1.16. of atomic changes from two versions of a Java project, which change(s) that last touched the fixed locations. is achieved via a pairwise comparison of database to on an atomic change Finally, HATARI uses the bugthe abstract syntax rule outonchanges in the affecting chan editor that the associated program fragmen cannot betest call graphsthe two project versions. Another part made after the bug trees of fix-inducing because they have been the classes in reads for the original and edited projects, 3. theREFERENCES was reported, i.e.,tests andcannot be real causes for Shawn bug. In our S. Arnold. An computes affected they their affecting changes. The 3. For each of the fix-inducing changes, HATARI determines [1] A. Bohner and Robert their location. third revisions 1.14 that 1.16 user not fix-inducing. Without the In Shawn A change impact information. Chianti’s the example,part manages the viewsandallowGUIare to visualize also includes a software change impact analysis. Robert S. Arnold, editors, Software Change pages 1–26. IEEE Computer Society Press, 1 connection to bethe bug the set of tests associated with the launch configuration thatdatabase,to selectwould be marked Law and Gregg Rothermel. Whole pro versions to analyzed, allows users they the project [2] James fix-inducing dynamic impact analysis. In Proc. of the Int 4. Finally, for every location in the source code, HATARI mea- and therefore the call graphs to be used. Figure 1 depicts project and be false positives. on Software Engineering, pages 308–318, 20 [3] Alessandro Orso, Taweesup Apiwattanapong Chianti’s architecture. sures their risk of change. In our example, isrevision 1.11 is the we cur- fix-inducing change.Harrold. An Proc Although Chianti intended for interactive use, only Rothermel, and Mary Jean dynamic impact analysis algorithms. In emp rently require two separate Java not always,of such fix-inducing two versions program are Frequently, butthat projects, insteadaof comparing saved in changes Edinburgh, on Software Engineerin the cur- 491–500, introduced International Conf. Scotland, 2004. The following sections provide details on these steps. the bug rent that has its local fixed Thus, a on. However, such changes are for impact an version with been history. later typical scenario [4] Alessandro Orso, Taweewup Apiwattanapon Change Distiller CodeCrawler Harrold. Leveraging field data of a Chianti session begins with the programmer editing the testing. In Proc. of European Software Eng always unstable extracting therisky.stable version of this current project, and thus lastest ACM SIGSOFT Symp. on the Foundations 4.1 Identifying Fixes project from grammer we define a into the workspace. The con- Formally,thenCVS repository induced-change relation J ⊆ C × C that starts the change impact analysis launch pro- Engineering (ESEC/FSE’03), Helsinki, Fin 2003. [5] Xiaoxia Ren, Fenil Shah, Frank Tip, Barbara In order to locate fix-inducing changes, HATARI first needs to know figuration, and changes citwo these j with Currently, connects twosuiteselects these with projects of interest as well as the test associated Ophelia only if c tool for and cprojects. each other if andChesley.InChianti:ofAthe Conf.change Java programs. Proc. j on Ob if a change is a fix. While advanced version control systems al- changed a line that have a separate main() routine i ; in terms [6] Xiaoxia Ren, Fenil Shah, Frank Tip, Barbara we allow tests that was introduced in cand JUnit of CVS this means Programming, Systems, Languages and App tests
  28. 28. What is the long-term impact of change?
  29. 29. What is the long-term impact of change?
  30. 30. What is the long-term impact of change?
  31. 31. What is the long-term impact of change?
  32. 32. What is the long-term impact of change?

×