Promise 2011: "Are Change Metrics Good Predictors for an Evolving Software Product Line?"

2,958 views

Published on

Promise 2011:
"Are Change Metrics Good Predictors for an Evolving Software Product Line?"
Sandeep Krishnan, Chris Strasburg, Robyn Lutz and Katerina Goseva-Popstojanova.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,958
On SlideShare
0
From Embeds
0
Number of Embeds
2,229
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Promise 2011: "Are Change Metrics Good Predictors for an Evolving Software Product Line?"

  1. 1. Are Change Metrics Good Predictors for an Evolving Software Product Line?<br />Sandeep Krishnan, ISU<br />Chris Strasburg, ISU & Ames Laboratory <br />Robyn R. Lutz, ISU & JPL, California Institute of Technology<br />Katerina Goseva-Popstojanova, WVU<br />1<br />This research is supported by NSF grants 0916275 and 0916284<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  2. 2. Background<br />Product line – “A family of products designed to take advantage of their common aspects and predicted variabilities” [Weiss and Lai 1999]<br />e.g., Nokia cellphones, HP printers, etc.<br />Products - <br />Commonalities – Shared by all products. e.g., Platform<br />Variabilities – Differentiate the products<br /><ul><li>High-reuse variation
  3. 3. JDT, PDE, Mylyn, Webtools, etc.
  4. 4. Reused in more than three products and for more than six years.
  5. 5. Low-reuse variation
  6. 6. CDT, Datatools, Java EE tools.
  7. 7. Reused in three or fewer products and for more than four years.</li></ul>2<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  8. 8. Related Work<br />Eclipse as a product line. [Chastek, McGregor, and Northrop, 2007], [Linden, 2009], [Krishnan et al., 2011]. <br />Summary of previous work<br />3<br />Failure-prone file -A file with one or more non-trivial post-release<br />bugs recorded in the Eclipse Bugzilla database.<br />Important/Good predictor – Predictor providing high information gain for classification of failure-prone files<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  9. 9. Product Line Evolution<br />Product line evolution in two dimensions<br />4<br />New Releases<br />P1<br />R1<br />P1<br />R2<br />P1<br />R3<br />P1<br />Rn<br />P2<br />R1<br />P2<br />R2<br />P2<br />R3<br />P2<br />Rn<br />New Products<br />Pn<br />R1<br />Pn<br />R2<br />Pn<br />R3<br />Pn<br />Rn<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  10. 10. Motivation<br />Can we leverage the reducing amount of change in product lines to better predict failure-prone files?<br />5<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  11. 11. Eclipse case study<br />6<br />Blocker<br />Eclipse Classic<br />Critical<br />Eclipse Java<br />Major<br />Eclipse C/C++<br />Normal<br />Eclipse JavaEE<br />Minor<br />
  12. 12. Research Questions<br />As a product evolves, do any change metrics serve as good predictors of failure-prone files?<br />Is there a subset of change metrics which are good predictors across all product line members?<br />Does our ability to predict failure-prone files improve as product line evolves?<br />7<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  13. 13. Findings<br />The change metrics provide good classification of the failure-prone files in the Eclipse product line.<br />As each product evolves, there is a stable set of change metrics that are prominent predictors of failure-prone files across its releases.<br />There is a subset of change metrics that is among the prominent predictors of all the products across most of the releases.<br />As the product line matures, prediction performance improves for each of the four Eclipse products.<br />8<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  14. 14. Data Source<br />9<br />Data Timeline<br />Data Timeline<br />Source of failure reports-<br />Source of change reports – CVS repository of Eclipse.<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  15. 15. Approach<br />10<br />Get prediction results + best predictors<br />Weka J48 decision tree learner<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  16. 16. Replication Results<br />11<br />Learner performance compared to previous results<br />Classification performance comparison for Eclipse Classic 2.0, 2.1, and 3.0<br />PC- Percentage of correctly classified instances<br />TPR- True positive rate<br />FPR- False positive rate<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  17. 17. 12<br />Top five predictors for earlier releases of Eclipse Classic<br />Replication Results<br />Top predictors from this study<br /><ul><li>Revisions, Weighted_age</li></ul>Top predictors from previous study<br /><ul><li>Max_changeset, Bugfixes, Revisions</li></ul>Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  18. 18. 13<br />Learner performance improves as single product evolves<br />Extension Results<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  19. 19. 14<br />Top five predictors for later releases of Eclipse Classic<br />Extension Results<br />Revisions is good predictor for later releases also.<br />Max_changeset is a good predictor also.<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  20. 20. 15<br />Learner performance improves as product line evolves<br />Extension Results<br />Percentage of correctly classified instances increases across releases for each product<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  21. 21. 16<br />Learner performance improves as product line evolves<br />Extension Results<br />Percentage of true positives shows improvement across releases for each product<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  22. 22. 17<br />Learner performance improves as product line evolves<br />Extension Results<br />Percentage of false positives shows reduces across releases for each product<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  23. 23. 18<br />Top five predictors for four products of Eclipse Product Line<br />Extension Results<br />No common set of predictors across each product and each release.<br />Max_changeset, Revisions and Authors are prominent predictors for all products.<br />Some predictors are prominent for only one product.<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  24. 24. 19<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />
  25. 25. Thank You!<br />20<br />Our data is available at <br />http://www.cs.iastate.edu/~lss/PROMISE11Data.tar.gz<br />Dept. of Computer Science, Iowa State University, PROMISE, September 20, 2011<br />

×