Mining Cause Effect Chains from Version Archives - ISSRE 2011

Mining Cause-Effect-Chains from Version HistoriesFunded by Kim Herzig & Andreas Zeller Faculty Grant from Saarland University, Germany

Cause Effect Chain Initial Change

Cause Effect Chain Initial ChangeWhat is the long-term impact of the initial change? and can we predict them?

What does this has to do with reliability?

What does this has to do with reliability? Funded by Faculty Grant

Web-Application Development• Changing live systems• One large repository• Direct impact on functionality, stability,

Web-Application Development• Changing live systems• One large repository Funded by• Direct impact on Faculty Grant functionality, stability,

Cause Effect chain Initial Change

Cause Effect chain Initial Change C1 Cn dependency

Change GenealogiesChange C2 depends on change 1. Analyzing source code changes C1 (not every revision can be compiled)Change C2 can only be applied 2. Extracting method definitions after applying C1. and method calls from changes (added, modified, deleted) T1 T2 3. Computing dependencies ✓ •• between method definition T1 T2 and call changes •• ✗ (e.g. call depends on previous definition) [1] Capturing the long-term impact of changes , Herzig, ICSE ’10 Ph.D. Symposium

Change Genealogies - int A.foo(int)File 1 + int A.foo(int) + ﬂoat A.foo(ﬂoat)File 2 + int B.bar(int) + d = A.foo(5d) - x = B.bar(5)File 3 + B.bar(5) + x = A.foo(5f)File 4 + d = A.foo(d) = A.foo(-1f) +e C1 C2 C3 C4 C5 time

Change Genealogies • Graph structure ‣ models structural dependencies • directed & acyclic✗ ‣ future cannot inﬂuence past •2 dimensional (time & space)2D ‣ vertex annotation: changed ﬁles, bug

Change Genealogies of formal methods! a llo ws the use • Graph structure ‣ models structural dependencies • directed & acyclic✗ ‣ future cannot inﬂuence past •2 dimensional (time & space)2D ‣ vertex annotation: changed ﬁles, bug

Long Term Couplings example on genealogy usage known from Amazon

Long Term Couplings example on genealogy usage known from Amazon developers who changed this artifact also changed ...[2] Mining Version Histories to Guide Software Changes, Zimmermann, Weißgerber, Diehl, Zeller, ICSE 04

Long Term Couplings example on genealogy usage known from Amazon developers who changed this artifact also changed ...[2] Mining Version Histories to Guide Software Changes, Zimmermann, Weißgerber, Diehl, Zeller, ICSE 04 this ﬁle got frequently changed by ... [3] Codebook Discovering and Exploiting Relationships in Software Repositories, Begel, Phang, Zimmermann, FSE 10

Long Term Couplings example on genealogy usage known from Amazon developers who changed this artifact also ime! in t changed ... lim ited Zeller, ICSE 04[2] Mining Version Histories to Guide Software Changes, Zimmermann, Weißgerber, Diehl, this ﬁle got frequently changed by ... spa ce! [3] Codebook Discovering and Exploiting Relationships in Software t e d in i Repositories, Begel, Phang, Zimmermann, FSE 10 lim

Long Term Couplings example on genealogy usage developers who changed this artifact also ime! in t changed ... ited Zeller, ICSE 04 lim[2] Mining Version Histories to Guide Software Changes, Zimmermann, Weißgerber, Diehl, spa ce! this ﬁle got frequently changed by ... e d in li mit [3] Codebook Discovering and Exploiting Relationships in Software Repositories, Begel, Phang, Zimmermann, FSE 10 changing this artifact always eventually causes ... [4] Using multivariate time series and association rules to detect logical change coupling: an empirical study, G. Canfora, M. Ceccarelli, L. Cerulo, and M. Di Penta, ICSM 2010.

Long Term Couplings example on genealogy usage developers who changed this artifact also ime! in t changed ... ited Zeller, ICSE 04 lim[2] Mining Version Histories to Guide Software Changes, Zimmermann, Weißgerber, Diehl, spa ce! this ﬁle got frequently changed by ... e d in li mit [3] Codebook Discovering and Exploiting Relationships in Software Repositories, Begel, Phang, Zimmermann, FSE 10 changing this artifact always eventually causes ... [4] Using multivariate time series and association rules to detect logical change coupling: an empirical study, G. Canfora, M. Ceccarelli, L. Cerulo, and M. Di Penta, ICSM 2010. does not consider structural dependencies!

Long Term Couplings example on genealogy usage developers who changed this artifact also ime! in t changed ... ited Zeller, ICSE 04 lim[2] Mining Version Histories to Guide Software Changes, Zimmermann, Weißgerber, Diehl, spa ce! this ﬁle within time window got frequently changed by ... e d in li mit [3] Codebook Discovering and Exploiting Relationships in Software Repositories, Begel, Phang, Zimmermann, FSE 10 changing this artifact always eventually causes ... [4] Using multivariate time series and association rules to detect logical change coupling: an empirical study, G. Canfora, M. Ceccarelli, L. Cerulo, and M. Di Penta, ICSM 2010. does not consider structural dependencies!

Long Term Couplings example on genealogy usagechanging this artifact always eventually causes ... ai ) EF aj Computational Tree Logic (CTL)

Long Term Couplings example on genealogy usage within time windowchanging this artifact always eventually causes ... ai ) EF aj Computational Tree Logic (CTL)

Model Checking Genealogies A E C B D extract changegenealogy fromversion archive

Model Checking Genealogies A A E E C C B B D D extract extract valid change CTL rules usinggenealogy from model checkingversion archive

Model Checking Genealogies recommendations A A 1 a1 ) EF a2 E E C C 2 a1 ) EF (a2 ^ a3 ) B B D D 3 a1 ) AG (a2 ) EF a3 ) ... transform extract frequent extract valid change occurring rules CTL rules usinggenealogy from into model checkingversion archive recommendatio ns

Recommendation GenerationC1 C2 C3 time window time

Recommendation GenerationC1 C2 C3 time window time Extract subgraph and add ﬁnal state S C2 C3

Recommendation GenerationC1 C2 C3 time window time Extract subgraph and add ﬁnal state S C2 C3 F

Recommendation GenerationC1 C2 C3 time window time Extract subgraph and add ﬁnal state S C2 C3 F Change labels with names of corresponding changed ﬁlesF1,F2 F2 F1,F3 F

Using CTL Templates F1,F2 F2 F1,F3 F

Using CTL Templates F1,F2 F2 F1,F3 FEF FxEF (Fx ^ Fy )(EF Fx ) ^ (EF Fy )AG (Fx ) EF Fy ) CTL Templates

Using CTL Templates F1,F2 F2 F1,F3 FEF Fx ✔EF (Fx ^ Fy )(EF Fx ) ^ (EF Fy )AG (Fx ) EF Fy ) CTL Templates

Using CTL Templates F1,F2 F2 F1,F3 FEF Fx ✔ ...EF (Fx ^ Fy ) F1 ) EF F3(EF Fx ) ^ (EF Fy )AG (Fx ) EF Fy ) F2 ) EF F3 ... CTL Templates Recommendations

Recommendation Ranking <premiss> ) <implication> conﬁdence(F) support(F ) # times premiss true support(F) # Kripke structures formula F evaluated true

Conditional Rules• Genealogyvertices annotated with change properties ‣ bug fix, big change, modified definition, authors, dependency types, ...

Conditional Rules• Genealogyvertices annotated with change properties ‣ bug fix, big change, modified definition, authors, dependency types, ... ai ^ ‘ ‘is fix” ) EF aj

Conditional Rules• Genealogyvertices annotated with change properties ‣ bug fix, big change, modified definition, authors, dependency types, ... ai ^ ‘ ‘is fix” ) EF aj• Certain rules become only important under certain conditions!

Recommendations Examples from JRuby VariableCompiler ) EF StandardASMCompiler• Never changed together, support=20, conﬁdence=0.8

Recommendations Examples from JRuby VariableCompiler ) EF StandardASMCompiler• Never changed together, support=20, conﬁdence=0.8 MainTestSuite ) (EF RubyObject)

Recommendations Examples from JRuby VariableCompiler ) EF StandardASMCompiler• Never changed together, support=20, confidence=0.8 MainTestSuite ) (EF RubyObject)RubyIO ) ((EF RubyStructure) ^ (EF Visibility))• support=20, confidence=0.5 / if bug fix: confidence=0.8

Experimental Setup10% training phase project history

Experimental Setup10% training phase project history use the top 3 ranked recommendations to predict files that will change in time window. (ranked by confidence, support) support > 2, confidence ≥ 0.5

Experimental Setup10% training phase project history use the top 3 ranked recommendations to predict files that will change in time window. (ranked by confidence, support)use formulas forfurther training support > 2, confidence ≥ 0.5

Experimental Setuptraining phase project history use the top 3 ranked recommendations to predict files that will change in time window. (ranked by confidence, support)use formulas forfurther training support > 2, confidence ≥ 0.5

Benchmark Model the three top mostConstant ly predicts iles to cha nge again! changed f

Precision of Recommendations file changes predicted and applied within time window #true positivesprecision = #true positives+#false positives file changes predicted but not applied within time window

Precision of Recommendations #true positivesrecall = #true positives+#false negatives file changes not predicted but applied within time window

Precision of Recommendations #true positives recall = #true positives+#false negatives file changes not predicted but applied within time windowHow much of a systems’s future evolution can be predicted from the past?

Precision of Recommendations y na ture! ery l w b positives e sense o #true recall =V#true positives+#falsek s not ma negatives asur e doe This me h ere! file changes not predicted but applied within time windowHow much of a systems’s future evolution can be predicted from the past?

Project Details ArgoUML Jaxen JRuby XStream history 12 years 9 years 9 years 7 years#transactio 16,481 1,353 11,060 1,683 ns #authors 50 20 66 11 #ﬁles 16,658 9,831 15,029 1,188#LTC >0.7 94 10 99 19 #vertices 8,716 1,330 11,055 1,680 ∅ out 8 7 10 5 degree time 16 days 9 days 8 days 4 days window time window = median gap between vertex and youngest child

How Precise are our Predictions? Precision CTL Benchmark0.8 0.7 0.7 0.6 0.6 0.60.4 0.3 0.3 0.3 0 ArgoUML Jaxen JRuby XStream

How Precise are our Predictions? Precision Avg. rank of highest hit CTL Benchmark CTL Benchmark0.8 2.4 2.4 0.7 0.7 2.1 2.1 2.0 0.6 1.9 0.6 0.6 1.8 1.8 1.80.4 1.2 0.3 0.3 0.3 0 0 ArgoUML Jaxen JRuby XStream ArgoUML Jaxen JRuby XStream

Including Inner-Commit Rules Precision Avg. rank of highest hit with inner rules w/o inner rules with inner rules w/o inner rules Benchmark Benchmark0.8 2.4 2.4 0.7 0.70.7 2.1 2.1 2.1 0.7 0.7 2.0 2.0 2.0 2.0 0.6 1.9 0.6 0.6 0.6 1.8 1.8 1.80.4 1.2 0.3 0.3 0.3 0 0 ArgoUML Jaxen JRuby XStream ArgoUML Jaxen JRuby XStream

% Commits all Predictions True w/o inner-commit rules with inner-commit rules 70 68.8 58.0 52.3 54.2 47 49.1 48.0 47.8 43.8 23 0 ArgoUML Jaxen JRuby XStream

Funded Faculty

http://www.st.cs.uni-saarland.de	32
http://www.kim-herzig.de	19

Mining Cause Effect Chains from Version Archives - ISSRE 2011

by Kim Herzig on Nov 30, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 51

Statistics

Mining Cause Effect Chains from Version Archives - ISSRE 2011 — Presentation Transcript