Sègla Kpodjedo, Filippo Ricca, Philippe Galinier and Giuliano (Giulio) Antoniol RSSE 2008, Atlanta Toward a Recommendation System for focusing Testing

Outline The Challenge Related work Our Approach ClassRank ECGM and Evolution Cost The Case Study: Mozilla Results Discussion Conclusion RSSE 2008, Atlanta

The Challenge

Related work

Our Approach

ClassRank

ECGM and Evolution Cost

The Case Study: Mozilla

Results

Discussion

Conclusion

The Challenge Context: An OO software project Software Testing: Efforts aimed at providing information about the quality of a software Necessary compromise between information accuracy and available resources (computation power, time …) Recommend key classes to focus on RSSE 2008, Atlanta

Context: An OO software project

Software Testing:

Efforts aimed at providing information about the quality of a software

Necessary compromise between information accuracy and available resources (computation power, time …)

Recommend key classes to focus on

Related Work Ostrand et al. Method: negative binomial regression model Metrics: LOC (Lines Of Code) File size, number of faults in earlier releases, number of changes etc. Result: identify the top 20% most fault-prone files Nagappan Method: logistic regression technique Metrics dependency graphs between software components code churns (i.e., changes) of the components between subsequent versions (delta LOCs, changed files and number of changes) Limitations Strong prior information and heavy infrastructure Large subset of returned files (20% may be too much) RSSE 2008, Atlanta

Ostrand et al.

Method: negative binomial regression model

Metrics:

LOC (Lines Of Code)

File size, number of faults in earlier releases, number of changes etc.

Result: identify the top 20% most fault-prone files

Nagappan

Method: logistic regression technique

Metrics

dependency graphs between software components

code churns (i.e., changes) of the components between subsequent versions (delta LOCs, changed files and number of changes)

Limitations

Strong prior information and heavy infrastructure

Large subset of returned files (20% may be too much)

Our Approach Assumptions (i) “the most important classes” must be tested more deeply (ii) frequently changed classes are the most complex and then the most fault-prone Metrics PageRank score: relative importance of each class in a system accounting for overall structure of relations among classes « Evolution Cost »: quantifies the amount of change for a class and its relations in a time frame. Grid of criticality X axis: Evolution Cost Y axis: ClassRank RSSE 2008, Atlanta

Assumptions

(i) “the most important classes” must be tested more deeply

(ii) frequently changed classes are the most complex and then the most fault-prone

Metrics

PageRank score: relative importance of each class in a system accounting for overall structure of relations among classes

« Evolution Cost »: quantifies the amount of change for a class and its relations in a time frame.

Grid of criticality

X axis: Evolution Cost

Y axis: ClassRank

Class Diagrams are labeled graphs RSSE 2008, Atlanta Classes: Nodes labeled with properties (class name, attributes, methods …) Relations: Labeled Edges (i.e., association, aggregation or inheritance)

Random Walks and “ClassRank” Given a directed graph, what is the probability of being in a given node at a time t? A possibility of answer: Random Walks proceeding from node to node using the arcs. A very efficient RW algorithm: PageRank measures the relative importance of each element of a hyperlinked set and assigns it a numerical weighting ClassRank PageRank applied to class diagrams RSSE 2008, Atlanta

Given a directed graph, what is the probability of being in a given node at a time t?

A possibility of answer: Random Walks

proceeding from node to node using the arcs.

A very efficient RW algorithm: PageRank

measures the relative importance of each element of a hyperlinked set and assigns it a numerical weighting

ClassRank

PageRank applied to class diagrams

Evolution Cost RSSE 2008, Atlanta Snapshot N Snapshot N-1 Snapshot 1 1 2 3 4 5 … 6 2 8

Error Correcting Graph Matching G 1 G 2 M G1 M G2 D G1 I G2 An ECGM indicate edit operations transforming the first graph into the second Deletion of D G1 : the nodes and all their adjacent edges Insertion of I G2 : the nodes and all their adjacent edges Matching M G1 to M G2 => « errors » Visualisation of an ECGM RSSE 2008, Atlanta M G1 M G2 D G1 I G2

An ECGM indicate edit operations transforming

the first graph into the second

Deletion of D G1 : the nodes and all their adjacent edges

Insertion of I G2 : the nodes and all their adjacent edges

Matching M G1 to M G2 => « errors »

ECGM Costs « Errors » and related costs Node matching errors (M G1 to M G2 ) Dissimilarity of related information (labels) Edge Matching errors (M G1 to M G2 ) Structural error : a relation present in only one graph or Label error : different relations from one graph to the other xyz yxw a b C nm C es C el * C nd , C ni , C ed , C ed RSSE 2008, Atlanta

« Errors » and related costs

Node matching errors (M G1 to M G2 )

Dissimilarity of related information (labels)

Edge Matching errors (M G1 to M G2 )

Structural error : a relation present in only one graph

or

Label error : different relations from one graph to the other

* C nd , C ni , C ed , C ed

Node Cost RSSE 2008, Atlanta How we assign a cost to a matched node Internal Changes Cost of the dissimilarity between information from matched nodes + Structural changes Cost of changes for outgoing edges Deleted from the first graph Inserted in the second graph Matched with an other edge Incoming edges are not changes within the node

How we assign a cost to a matched node

Internal Changes

Cost of the dissimilarity between information from matched nodes

+

Structural changes

Cost of changes for outgoing edges

Deleted from the first graph

Inserted in the second graph

Matched with an other edge

Incoming edges are not changes within the node

Evolution Cost RSSE 2008, Atlanta Snapshot N Snapshot N-1 Snapshot 1 1 2 3 4 5 … 6 2 8

Case Study Software project: Mozilla suite open-source suite implementing aWeb browser and other tools such as mailers and newsreaders. Data Collection CVS repository each 15 days through 2007 24 Reverse engineered class diagrams RandomWalk Implementation PageRank ECGM Implementation Tabu Algorithm RSSE 2008, Atlanta

Software project: Mozilla suite

open-source suite implementing aWeb browser and other tools such as mailers and newsreaders.

Data Collection

CVS repository each 15 days through 2007

24 Reverse engineered class diagrams

RandomWalk Implementation

PageRank

ECGM Implementation

Tabu Algorithm

Mozilla Case Study: Results RSSE 2008, Atlanta

Case Study: Discussion Grid for assessing the criticality of any class splitted in 4 areas Upper Left: High Rank Low Cost Upper Right: High Rank High Cost Lower Left: Low Rank Low Cost Lower Right: Low Rank High Cost Pareto Front Additional information about restructuring Evolution of ClassRank through history (dramatic changes) RSSE 2008, Atlanta

Grid for assessing the criticality of any class splitted in 4 areas

Upper Left: High Rank Low Cost

Upper Right: High Rank High Cost

Lower Left: Low Rank Low Cost

Lower Right: Low Rank High Cost

Pareto Front

Additional information about restructuring

Evolution of ClassRank through history (dramatic changes)

Conclusion A RS aiming at identifying critical classes in an OO application. Critical classes: frequently subject to changes and with high connectivity with other classes. Preliminary study No empirical evidence of a correlation with error proneness, severity or priority of defects Assumptions highly worth exploring Future work Study correlation between our index and bug severity/priority Implement an Eclipse plug-in supporting the approach RSSE 2008, Atlanta

A RS aiming at identifying critical classes in an OO application.

Critical classes: frequently subject to changes and with high connectivity with other classes.

Preliminary study

No empirical evidence of a correlation with error proneness, severity or priority of defects

Assumptions highly worth exploring

Future work

Study correlation between our index and bug severity/priority

Implement an Eclipse plug-in supporting the approach