Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-Grained Repository- Based Co-Evolution Analysis

Have Java Production Methods
Co-Evolved With Test Methods
Properly?:
A Fine-Grained Repository-
Based Co-Evolution Analysis
Tenma Kita*1
Hirohisa Aman*1
Sousuke Amasaki*2
Tomoyuki Yokogawa*2
Minoru Kawahara*1
SEAA2022 (C) 2022 Hirohisa Aman
*1) Ehime University,
Matsuyama, Ehime, Japan
*2) Okayama Prefectural Univ.,
Soja, Okayama, Japan
0

Overview
 Aim
To detect Java methods that might not be properly
tested through code evolutions
 Method
We examine co-evolution relationships
(logical couplings) of production code with test code,
using a fine-grained code repository;
We propose a metric for evaluating their logical links
 Result
We empirically proved that most Java methods (98%
on average) have co-evolved with test methods, but
some have not; our metric detected some risky ones
SEAA2022 (C) 2022 Hirohisa Aman 1

Outline
 Background & Related Work
 Logical Coupling Between Production &
Test Methods
 Case Study
 Conclusion & Future Work

Outline
Test Methods
 Case Study

Co-Evolution
Between Production and test code
 A set of source code (repository) includes not
only the production code but the test code
src
main
test
ConnectionStateManager.java
TestConnectionStateManager.java
…
…
update
update
When developers update a production code,
they should update or add the test code too
Co‐Evolution

Logical Coupling
 A successful co-evolution between production
and test code form a logical coupling
 Our focus is on a poor logical coupling
 Such a production code may have little or no
corresponding test code
commits
P1 T1 P1 T1 P1 T2
new code
creation
functionality
change
bugfix
P1
change
T1 T2
change
logical
coupling

Related Work (1/2)
 Luben et al.[9] emphasized the successful
maintenance of test cases in test-driven
development
 They analyzed logical couplings between
Java production classes and JUnit test
classes for a successful maintenance
 Although it is a valuable previous study, the
analysis is at class-level and coarse-grained
 To analyze those relationships from a finer-
grained point of view, we utilize a method-
level code repository in this study

Related Work (2/2)
 Marsavina et al.[5] & Vidacs and Pinzger[10]
performed a fine-grained code change analysis
to extract co-evolution patterns
 Ex. “A production class addition/removal tends to
cause a test class addition/removal”
 Those studies yield profound insights into co-
evolution relationships
 However, their main focuses were on the co-
evolution patterns, but poorly tested code (poor
logical coupling) was out of their analysis;
We focus on such a poor logical coupling to
detect a problematic code

Outline
Test Methods
 Case Study

Analysis Granularity: Method
 As a file-level analysis seems to be coarse
and insufficient, we will trace logical
couplings at method level
 To this end, we utilize a fine-grained code
repository, FinerGit[8]
 FinerGit can convert a file-level normal Git
repository to a method-level Git repository
 In the converted repository, each Java method
corresponds to each file
 We can easily trace a method-level change history
through Git commands

Logical Coupling & Its Strength
 For a relation 𝒑 𝒕, the following
confidence value presents its strength of link:
When developers change a Java method
𝒑 at a commit, they tend to also change
another method 𝒕 at the same commit
𝒑 𝒕
𝒑 𝒕) =
𝒑 𝒕
𝒑
𝜎
𝜎(x) is # of
x’s changes

Two or more test methods may
correspond to a production method
 The confidence value corresponds to a
conditional probability P( 𝒕 | 𝒑)
 Similarly, we can consider probability that at
least one test method is changed as
0.4
0.3 The probability that
both 𝑚 and 𝑚
are not changed
P( 𝒕 𝒕 | 𝒑) P( 𝒕 𝒕 | 𝒑)

Proposed metric:
 Some production methods might be linked to
other production methods, and the latter
ones are tested, i.e., indirect test links
 We propose a novel metric, , to cover
such indirect logical couplings as well
0.4
0.3
0.1
0.7
0.2
0.3
𝑐𝑜𝑛𝑓 0.58
𝑐𝑜𝑛𝑓 0.73
𝑇𝑐𝑜𝑛𝑓
1 1 0.2 · 0.58 · 1 0.3 · 0.73
1 0.884 · 0.781
0.310
𝑇𝑐𝑜𝑛𝑓 0.310

What evaluates?
 The value of a production method
evaluates the likelihood that the production
method co-evolves with one or more test
methods in either direct or indirect manners
 A production method with a low
value might be problematic as it has not
been co-evolved with test methods

Outline
Test Methods
 Case Study

Studied Projects
 We conducted a case study using ten Apache
top-level projects to demonstrate how
helps detect the problematic Java methods

Results(1/2): Curator project
 Distribution of values in the Curator project
mean 0.9948
𝑇𝑐𝑜𝑛𝑓
0.5 0.5 0.999 ≃ 1
0.43% 0.64% 𝟗𝟖. 𝟗𝟑%
Most production methods
(over 98%) in this project
have co-evolved with at
least one test method

Results(2/2): Other Projects
 All other projects showed similar results
Fineract Flume Maven Parquet
PDFBox Ranger RocketMQ Shiro
Zookeeper
𝑇𝑐𝑜𝑛𝑓 𝐚𝐥𝐥 𝐩𝐫𝐨𝐣𝐞𝐜𝐭𝐬
0.5 0.5 0.999 ≃ 1
0.67% 1.33% 𝟗𝟖. 𝟎𝟎%

Discussions(1/4)
 As a result, we found that 98% of production
methods have , i.e., they have co-
evolved with one or more test methods
 Nonetheless, we also found some production
methods with low values
 They included simple accessor methods like:
Indeed, the above simple method does not need to be
carefully tested …
𝑇𝑐𝑜𝑛𝑓 0

Discussions(2/4)
 So, we filtered out the following methods:
 It has appeared in the commit history only once
 It has the lowest cyclomatic complexity (=1)
 Even if we filtered simple methods, there are
some production methods with

Discussions(3/4)
 An example of
production method
with
 LOC = 44
 Cyclomatic complexity = 7
 Number of commits = 2
Although the developers
might prepare test code for
this method independently,
it is worth it to detect such
a method and alert it

Discussions(4/4)
 Notice that a low value does not
directly indicate the poor quality
 Nonetheless, can detect the methods
that have not been successfully co-evolved
with test methods
 By alerting such potentially problematic
production methods to the developers,
can contribute somewhat to the
successful software evolution

Threats to Validity
 Construct Validity:
 There might be a delayed logical coupling
 Internal Validity:
 Some non-functional changes (refactoring or
comment changes) might become noise data
 External Validity:
 We studied only ten Java projects
 It is an impactful factor if the project adopt the
test-driven development or not

Outline
Test Methods
 Case Study

Conclusion
 We focused on the co-evolution (logical
coupling) between Java production and test
methods
 We proposed as a metric for evaluating
how a Java production method has properly
co-evolved with test methods
 A case study showed that
 Most methods (98% on average) had been co-
evolved successfully
 helps us to detect some potentially
problematic methods as well

Future Work
 A validation study of by getting
feedback from the developers
 An empirical study of relationship between
value and the overlooked faults
 Further analyses focusing on the code change
details and delayed logical couplings where
the test method’s commit occurs after the
production one’s commit

Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-Grained Repository- Based Co-Evolution Analysis

Recommended

Recommended

More Related Content

Similar to Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-Grained Repository- Based Co-Evolution Analysis

Similar to Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-Grained Repository- Based Co-Evolution Analysis (20)

More from SEAA 2022

More from SEAA 2022 (18)

Recently uploaded

Recently uploaded (20)

Have Java Production Methods Co-Evolved With Test Methods Properly?: A Fine-Grained Repository- Based Co-Evolution Analysis