Factors Relating Field Failures and Dependencies - ICST 2011

presented @ Factors Relating Field Failures and DependenciesThomas Zimmermann ● Nachiappan Nagappan ● Kim Herzig Rahul Premraj ● Laurie Williams

The Value Of A House

The Value Of A House Condition matters!

The Value Of A HouseAnd so does the neighborhood!

What has this to do with code quality?

What has this to do with code quality? Consider software projects as a city with classes as houses

What has this to do with code quality? Consider software projects as a city with classes as houses The value of a class determined by the number of bugs found!

What has this to do with code quality? Consider software projects as a city with classes as houses The value of a class determined by the number of bugs found! How to determine the condition of a class?

The Quality Of Source Code /** * Used by {@link #getThreadInformation()} for a traversal search of * {@link Thread}s/{@link ThreadGroup}s * * @param group * @param level * @return /** */ * Used by {@link #getThreadInformation()} for a traversal search of private String visit(final ThreadGroup group, * {@link Thread}s/{@link ThreadGroup}s final int level) { * // Get threads in `group /** * @param group * @param level StringBuilder builder = new StringBuilder(); #getThreadInformation()} for a traversal search of * Used by {@link * @return int numThreads = group.activeCount(); Thread}s/{@link ThreadGroup}s * {@link Thread[] threads = new Thread[numThreads * 2]; * */ numThreads = group.enumerate(threads, group * @param false); private String visit(final ThreadGroup group, * @param level final int level) { StringBuilder indent = new StringBuilder(); * @return // Get threads in `group for (int i = 0; i < level; ++i) { */ StringBuilder builder = new StringBuilder(); indent.append(" "); private String visit(final ThreadGroup group, group.activeCount(); int numThreads = } final int level) { Thread[] threads = new Thread[numThreads * 2]; // Get threads in `group numThreads = group.enumerate(threads, false); // Enumerate each thread in `group StringBuilder builder = new StringBuilder(); for (int i = 0; i < numThreads; i++) { StringBuilder indent = new StringBuilder(); int numThreads = group.activeCount(); // Get thread Thread[] threads = new Thread[numThreads * i2]; level; ++i) { for (int i = 0; < Thread thread = threads[i]; numThreads = group.enumerate(threads, false); "); indent.append(" builder.append(indent); } builder.append("|-"); StringBuilder indent = new StringBuilder(); builder.append(thread.getName()).append(" ["); level; // Enumerate each thread in `group for (int i = 0; i < ++i) { builder.append(thread.getClass().getSimpleName()).append("], "); = 0; i < numThreads; i++) { indent.append(" "); for (int i builder.append(thread.getPriority()).append(", "); } // Get thread builder.append(thread.getState().name()); Thread thread = threads[i]; builder.append(indent); builder.append(FileUtils.lineSeparator); each thread in `group // Enumerate builder.append("|-"); for (StackTraceElement element : thread.getStackTrace()) { i++) { for (int i = 0; i < numThreads; builder.append(indent); // Get thread builder.append(thread.getName()).append(" ["); builder.append("| "); Thread thread = threads[i]; builder.append(thread.getClass().getSimpleName()).append("], "); builder.append(element.toString()); builder.append(indent); builder.append(thread.getPriority()).append(", "); builder.append(FileUtils.lineSeparator); builder.append("|-"); builder.append(thread.getState().name()); } builder.append(FileUtils.lineSeparator); builder.append(thread.getName()).append(" ["); // builder.append(FileUtils.lineSeparator); for (StackTraceElement element : thread.getStackTrace()) { builder.append(thread.getClass().getSimpleName()).append("], "); } builder.append(indent); builder.append(thread.getPriority()).append(", "); builder.append("| "); builder.append(thread.getState().name()); // Get thread subgroups of `group builder.append(element.toString()); builder.append(FileUtils.lineSeparator); int numGroups = group.activeGroupCount(); builder.append(FileUtils.lineSeparator); for (StackTraceElement element : thread.getStackTrace()) { builder.append(indent); } ThreadGroup[] groups = new ThreadGroup[numGroups * 2]; numGroups = group.enumerate(groups, false); builder.append("| "); // builder.append(FileUtils.lineSeparator); } builder.append(element.toString()); // Recursively visit each subgroup builder.append(FileUtils.lineSeparator); for (int i = 0; i < numGroups; i++) { } // Get thread subgroups of `group builder.append(indent); int numGroups = group.activeGroupCount(); // builder.append(FileUtils.lineSeparator); builder.append(visit(groups[i], level + 1)); } ThreadGroup[] groups = new ThreadGroup[numGroups * 2]; } numGroups = group.enumerate(groups, false); // Get thread subgroups of `group return builder.toString(); int numGroups = group.activeGroupCount(); visit each subgroup // Recursively } ThreadGroup[] groups = new for (int i = 0; i < numGroups; i++) { ThreadGroup[numGroups * 2]; builder.append(indent); numGroups = group.enumerate(groups, false); builder.append(visit(groups[i], level + 1)); } // Recursively visit each subgroup for (int i = 0; i < numGroups; i++) { builder.append(indent); return builder.toString(); } builder.append(visit(groups[i], level + 1)); } return builder.toString(); } Code Metrics

The Quality Of Source Code /** * Used by {@link #getThreadInformation()} for a traversal search of * {@link Thread}s/{@link ThreadGroup}s * * @param group * @param level * @return /** */ * Used by {@link #getThreadInformation()} for a traversal search of private String visit(final ThreadGroup group, * {@link Thread}s/{@link ThreadGroup}s final int level) { * // Get threads in `group /** * @param group * @param level StringBuilder builder = new StringBuilder(); #getThreadInformation()} for a traversal search of * Used by {@link * @return int numThreads = group.activeCount(); Thread}s/{@link ThreadGroup}s * {@link Thread[] threads = new Thread[numThreads * 2]; * */ numThreads = group.enumerate(threads, group * @param false); private String visit(final ThreadGroup group, * @param level final int level) { StringBuilder indent = new StringBuilder(); * @return // Get threads in `group for (int i = 0; i < level; ++i) { */ StringBuilder builder = new StringBuilder(); indent.append(" "); private String visit(final ThreadGroup group, group.activeCount(); int numThreads = } final int level) { Thread[] threads = new Thread[numThreads * 2]; // Get threads in `group numThreads = group.enumerate(threads, false); // Enumerate each thread in `group StringBuilder builder = new StringBuilder(); for (int i = 0; i < numThreads; i++) { StringBuilder indent = new StringBuilder(); int numThreads = group.activeCount(); // Get thread Thread[] threads = new Thread[numThreads * i2]; level; ++i) { for (int i = 0; < Thread thread = threads[i]; numThreads = group.enumerate(threads, false); "); indent.append(" builder.append(indent); } builder.append("|-"); StringBuilder indent = new StringBuilder(); builder.append(thread.getName()).append(" ["); level; // Enumerate each thread in `group for (int i = 0; i < ++i) { builder.append(thread.getClass().getSimpleName()).append("], "); = 0; i < numThreads; i++) { indent.append(" "); for (int i builder.append(thread.getPriority()).append(", "); } // Get thread builder.append(thread.getState().name()); Thread thread = threads[i]; builder.append(indent); builder.append(FileUtils.lineSeparator); each thread in `group // Enumerate builder.append("|-"); for (StackTraceElement element : thread.getStackTrace()) { i++) { for (int i = 0; i < numThreads; builder.append(indent); // Get thread builder.append(thread.getName()).append(" ["); builder.append("| "); Thread thread = threads[i]; builder.append(thread.getClass().getSimpleName()).append("], "); builder.append(element.toString()); builder.append(indent); builder.append(thread.getPriority()).append(", "); builder.append(FileUtils.lineSeparator); builder.append("|-"); builder.append(thread.getState().name()); } builder.append(FileUtils.lineSeparator); builder.append(thread.getName()).append(" ["); // builder.append(FileUtils.lineSeparator); for (StackTraceElement element : thread.getStackTrace()) { builder.append(thread.getClass().getSimpleName()).append("], "); } builder.append(indent); builder.append(thread.getPriority()).append(", "); builder.append("| "); builder.append(thread.getState().name()); // Get thread subgroups of `group builder.append(element.toString()); builder.append(FileUtils.lineSeparator); int numGroups = group.activeGroupCount(); builder.append(FileUtils.lineSeparator); for (StackTraceElement element : thread.getStackTrace()) { builder.append(indent); } ThreadGroup[] groups = new ThreadGroup[numGroups * 2]; numGroups = group.enumerate(groups, false); builder.append("| "); // builder.append(FileUtils.lineSeparator); } builder.append(element.toString()); // Recursively visit each subgroup builder.append(FileUtils.lineSeparator); for (int i = 0; i < numGroups; i++) { } // Get thread subgroups of `group builder.append(indent); int numGroups = group.activeGroupCount(); // builder.append(FileUtils.lineSeparator); builder.append(visit(groups[i], level + 1)); } ThreadGroup[] groups = new ThreadGroup[numGroups * 2]; } numGroups = group.enumerate(groups, false); // Get thread subgroups of `group return builder.toString(); int numGroups = group.activeGroupCount(); visit each subgroup // Recursively } ThreadGroup[] groups = new for (int i = 0; i < numGroups; i++) { ThreadGroup[numGroups * 2]; builder.append(indent); numGroups = group.enumerate(groups, false); builder.append(visit(groups[i], level + 1)); } // Recursively visit each subgroup for (int i = 0; i < numGroups; i++) { builder.append(indent); return builder.toString(); } builder.append(visit(groups[i], level + 1)); } return builder.toString(); } Code Metrics Neighborhood

Does code neighborhood inﬂuence code quality?

What We Need for each software artifact• Determine the value (quality)• Determine the neighborhood• Determine the condition (metrics)

Quality As ValueWe consider failure-proness as a measure of quality.

Quality As Value We consider failure-proness as a measure of quality. 12 bugs• determine all post- release bug 1 bugs 0 bugs 2 bugs

Quality As Value We consider failure-proness as a measure of quality. 12 bugs• determine all post- FP release bug 1 bugs FP• failure-prone <=> at 0 bugs least one post- NFP release bug 2 bugs FP

The NeighborhoodArtifacts communicate with each other (functions/methods, variables) B

The NeighborhoodArtifacts communicate with each other (functions/methods, variables) B incoming neighborhood

The NeighborhoodArtifacts communicate with each other (functions/methods, variables) B incoming neighborhood outgoing neighborhood

The NeighborhoodArtifacts communicate with each other (functions/methods, variables) B incoming neighborhood outgoing neighborhood } neighborhood

Code-Metrics As Condition Code metrics are useful to predict bugs.Churn & Size Metrics LOC, Churn, Churn FrequencyCode Metrics FanIn, FanOut Cyclomatic Complexity #methods, inheritance, couplingCode Coverage Block Coverage, Arc CoveragePeople Metrics Org. level, #authors and more ...

Code-Metrics As Condition Code metrics are useful to predict bugs.Churn & Size Metrics LOC, Churn, Churn FrequencyCode Metrics FanIn, FanOut B Cyclomatic Complexity #methods, inheritance, couplingCode Coverage Block Coverage, Arc CoveragePeople Metrics Org. level, #authors and more ...

Code-Metrics As Condition Code metrics are useful to predict bugs.Churn & Size Metrics LOC, Churn, Churn Frequency Compute theCode Metrics metrics for FanIn, FanOut B neighbors Cyclomatic Complexity #methods, inheritance, couplingCode Coverage Block Coverage, Arc CoveragePeople Metrics Org. level, #authors and more ...

Code-Metrics As Condition Code metrics are useful to predict bugs.Churn & Size Metrics LOC, Churn, Churn Frequency Compute theCode Metrics metrics for FanIn, FanOut B neighbors Cyclomatic Complexity #methods, inheritance, couplingCode Coverage Aggregate Block Coverage, Arc Coverage metricsPeople Metrics (med,max,total) Org. level, #authors and more ...

Code-Metrics As Condition Code metrics are useful to predict bugs.Churn & Size Metrics LOC, Churn, Churn Frequency Compute theCode Metrics metrics for FanIn, FanOut B neighbors Cyclomatic Complexity #methods, inheritance, couplingCode Coverage Aggregate Block Coverage, Arc Coverage Assign to B metricsPeople Metrics (med,max,total) Org. level, #authors and more ...

Data Collection Quality: FP or NFP for eachcomponent Neighborhood metrics (aggregated)

Data Collection Quality: FP or NFP Is there a relation between quality and neighborhood condition? for eachcomponent Neighborhood metrics (aggregated)

Experimental Setup FP NFP FP NFP FP

Experimental Setup m = churn_median FP NFP FP NFP FP

Experimental Setup m = churn_median FP NFP FP NFP FP NFP FP

Experimental Setup m = churn_median FP NFP FP 400 NFP 330 FP 450 NFP 405 FP 500

Experimental Setup m = churn_median FP NFP FP 400 NFP 330 FP 450 NFP 405 FP 500 Are the values in FP signiﬁcantlygreater/lower that values in NFP?

Projects

Projectsfunctional & object oriented object oriented metrics on binary level metrics on class level slightly different set of metrics

Survey We asked 700 Microsoft developers to share their opinion on:How do dependency characteristics inﬂuence the risk of failures?

Survey 110 responded (15.7%) We asked 700 Microsoft developers to share their opinion on:How do dependency characteristics inﬂuence the risk of failures?

Survey Binary has a dependency with another that ... 11%6% 83% churned a lot No Effect Increases Decreases No Opinion risk of failure of binary

Survey Binary has a dependency with another that ... 1% 11% 18%6% 4% 83% 77% churned a lot has high test coverage No Effect Increases Decreases No Opinion risk of failure of binary

Survey Binary has a dependency with another that ... 1% 11% 2% 18%6% 6% 1% 4% 77% 91% 83% churned a lot has high test failed many times in coverage the past No Effect Increases Decreases No Opinion risk of failure of binary

Results Vista indicates medians for FP are signiﬁcantly lower than for NFP indicates medians for FP are signiﬁcantly higher than for NFP

Results Vista Survey

Results Vista Survey Developers were right! High degree of churn increases failure- proness.

Results Vista Survey Code coverage does not matter!

Results Vista Survey Many users requires better testing!

Results Vista Survey Depending on bad code does not harm!

Results Eclipse Survey

Results Eclipse Survey as expected!

Results Eclipse Survey depending on components with fewer interfaces more bugs

Results EclipseResults between Vista and EclipseSurvey a lot! differ depending on components with fewer interfaces more bugs

Contradictions! Vista Eclipse Survey churn complexity coverage n/a post-release failuresContradiction raises questions regarding the relationship between dependencies and code quality!

Factors Relating Field Failures and Dependencies - ICST 2011

by Kim Herzig on Mar 25, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 20

Statistics

Factors Relating Field Failures and Dependencies - ICST 2011 — Presentation Transcript

http://www.kim-herzig.de	10
http://www.st.cs.uni-saarland.de	10