SlideShare a Scribd company logo
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Java Code Coverage 
with JCov 
Implementation Details and Use Cases 
Alexandre (Shura) Iline 
JDK Test Architect 
JDK SQE team 
Oct 1, 2014
Dmitry Fazunenko 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 3 
JCov development 
Alexey Fedorchenko
Safe Harbor Statement 
The following is intended to outline our general product direction. It is intended for 
information purposes only, and may not be incorporated into any contract. It is not a 
commitment to deliver any material, code, or functionality, and should not be relied 
upon in making purchasing decisions. The development, release, and timing of any 
features or functionality described for Oracle’s products remains at the sole discretion of 
Oracle. 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
4
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Program Agenda 
History, facts 
Getting the data with JCov 
Applied to OpenJDK 
Using the data 
Links, more information 
1 
2 
3 
4 
5 
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 6 
Facts 
• JCov is a Java Code Coverage tool. 
• The code coverage tool for JCK 
• The code coverage tool for Oracle JDK. 
• Other products: 
–JavaFX 
–SceneBuilder 
–...
History 
• 1996: First prototype 
• 1997: Integration with JDK 1.1 
• 1998: Used “in production” 
• JDK 1.2 
• … 
• JDK 7 
• 2014: Open-source as part of OpenJDK codetools project. 
• 2014: JCov 3.0 – a public “release” supporting JDK 8 
• JDK 9 – in progress 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 7
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
More on JCov dev 
1.5 engineer at a time, on average :) 
Leonid Arbouzov, Alexander Petrov, Vladimir Generalov, Serguei Chukhontsev, Oleg 
Uliankin, Gregory Steuck, Pavel Ozhdikhin, Konstantin Bobrovsky, Robert Field, 
Alexander Kuzmin, Leonid Mesnik, Sergey Borodin, Andrey Titov, Dmitry Fazunenko, 
Alexey Fedorchenko
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Code coverage is 
● An information on what source code is exercised in execution 
9
most often ... 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Code coverage is 
● An information on what source code is exercised in testing 
10 
Testing is 
● Activities to prove that the code behaves as expected 
where ...
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Other possible usages. 
• Usage monitoring 
• Save data in a test environment 
• Check coverage from generated logic 
• Fuzzing 
• Load testing 
• Regression test generation
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Program Agenda 
History, facts 
Getting the data with JCov 
Applied to OpenJDK 
Using the data 
Links, more information 
2 
12 
1 
3 
4 
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 13 
Getting the data 
• A simplest use case 
• A pick or two under the hood 
• More possibilities 
– Dynamic vs. static instrumentation 
–Grabber 
– Individual test coverage 
– Abstract coverage 
– Drop points 
– Direct coverage 
– Running big test suites
Simplest use case 
• Compile java files as usual 
• “Instrument” the byte code 
java -jar jcov.jar Instr <application classes> 
• Run the code 
java -classpath ...:jcov_file_saver.jar ... 
• Create a report 
java -jar jcov.jar RepGen <jcov xml file> 
demo 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
A pick under the hood 
15
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
JCov bytecode insertion 
A pick under the hood 
public int getX(); 
Code: 
0: ldc #31 // int 2 
2: invokestatic #29  
// Method com/sun/tdk/jcov/runtime/Collect.hit:(I)V 
5: aload_0 
6: getfield #2 // Field x:I 
9: ireturn 
16 
demo 
Warning: the code is a subject to change!!!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
JCov bytecode insertion 
As if it was in Java 
public int getX() { 
com.sun.tdk.jcov.runtime.Collect.hit(2); 
return x; 
} 
17 
Warning: the code is a subject to change!!!
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Increasing the count 
One step deeper 
public class Collect { 
... 
private static long counts[]; 
... 
public static void hit(int slot) { 
counts[slot]++; 
} 
... 
} 
18 
demo
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Saving the data 
Finally ... 
Runtime.getRuntime().addShutdownHook(new Thread() { 
public void run() { 
... 
Collect.disable(); 
Collect.saveResults(); 
Collect.enable(); 
... 
} 
}); 
19
Performance impact 
• Instrumentation phase: 
–proportional to the application code size. 
• Execution phase: 
–Counters increments - very low. Depends on the size of blocks. 
–Saving data – from significant to blocking. 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 20 
● There is a cure! :) More on next slides. 
• Report generation phase.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
XML coverage data 
<class name="Point" supername="java/lang/Object" ...> 
... 
<meth name="getX" vmsig="()I" ...> 
<bl s="0" e="4"> 
<methenter s="0" e="4" id="5" count="1"/> 
<exit s="4" e="4" opcode="ireturn"/> 
</bl> 
</meth> 
</class> 
21 
demo 
Warning: the format is a subject to change!!!
Back to the simplest use case 
• Compile java files as usual 
• “Instrument” the byte code 
java -jar jcov.jar Instr <application classes> 
• Run the code 
java -classpath ...:jcov_file_saver.jar ... 
• Create a report 
java -jar jcov.jar RepGen <jcov xml file> 
demo 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Incomplete data!!!
Code coverage template 
• Describes all code there is to cover 
• Created during instrumentation: 
java -jar jcov.jar Instr -t template.xml <in> <out> 
• Created explicitly: 
java -jar jcov.jar TmplGen -t template.xml <in> 
• A regular JCov XML file. 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
● Merge 
● Filter 
● etc. etc.
Simplest use case corrected 
demo 
• Compile java files as usual 
• “Instrument” the byte code 
java -jar jcov.jar Instr -t template.xml <directory or jar> 
• Run the code 
java -classpath ...:jcov_file_saver.jar ... 
• Merge with the template 
java -jar jcov.jar Merger -o merge.xml template.xml result.xml 
• Create a report 
java -jar jcov.jar RepGen result.xml 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Simplest use case with filtering 
• Compile java code 
• “Instrument” the byte code 
java -jar jcov.jar Instr -t template.xml  
-i com.company.product  
<application classes> 
• Run the code, merge with the template 
• Create a report 
java -jar jcov.jar RepGen  
-e com.company.product.test result.xml 
demo
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
More on getting the data 
26
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Dynamic instrumentation 
• Compile java files as usual 
• Run with an extra VM option 
java -classpath ... -javaagent:jcov.jar ... 
• result.xml generated. 
• Merge and filter by the template 
java -jar jcov.jar Merger -o merge.xml  
-t template.xml result.xml 
demo 
demo
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Dynamic vs. static instrumentation 
Dynamic Static 
Modification of the tested 
application 
Not needed Needed 
Coverage collected for All code(*) Instrumented code only 
Test execution Modified to pass more options 
Modified to add jcov code to 
the classpath (**) 
Performance Slows the classloading Faster 
28 
* There are options to limit the instrumented code 
** Or inject the classes into the application itself
“Network” grabber 
demo 
• Start the grabber on the network 
java -jar jcov.jar Grabber … -hostname host123 -port 3333  
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
-t template.xml 
• Run the tests 
java -classpath ...  
-javaagent:jcov.jar=grabber,host=host123,port=3333  
... 
• Stop the grabber – a data file is generated 
java -jar jcov.jar GrabberManager -kill 
• Merge, generate report
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Individual code coverage 
• Track code coverage on a per-test level 
• “Usefulness” of the tests 
Test 2 
Code 
Code 
Code 
Test 1
Individual test coverage with JCov 
• Data is encoded into the XML file: 
<class name="Component" supername="java/lang/Object" ...> 
... 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
<meth name="&lt;init&gt;" vmsig="()V" ...> 
<bl s="0" e="151"> 
<methenter s="0" e="151" id="20" count="90359" 
scale="08fffffffffee7d8effffffffffffffffffffffffffffffffffffffffffffffff 
fffff3efffffefffffffffffffffffffffffffffffffffffffefffbfffffffffffffffbf 
fffffffffffffffffd83fffffffffffffffffffffff1"/> 
<exit s="151" e="151" opcode="return"/> 
</bl> 
</meth> 
... 
</class>
Test scales with files 
• Run tests separately, save result.xml's 
$ java -classpath ... -javaagent:jcov.jar ... my.tests.TestN 
$ cp result.xml result_TestN.xml 
• Merge data files 
$ java -jar jcov.jar Merger -o merge.xml  
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
-outTestList testlist.txt  
-t template.xml result_*.xml 
demo
Test scales with grabber 
• Start the grabber 
$ java -jar jcov.jar -scale -outTestList testlist.txt 
-t template.xml 
• Run tests separately, let the grabber know test names 
$ java -classpath ... -javaagent:jcov.jar:grabber  
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
-Djcov.testname=TestN ... my.tests.TestN 
● Stop the grabber, generate report. 
demo
Field coverage 
• Generate template 
java -jar jcov.jar TmplGen -field on -t template.xml <in> 
• Instrumentation 
ldc // int 1951 
invokestatic  
demo 
// Method com/sun/tdk/jcov/runtime/CollectDetect.invokeHit:(I)getfield // Field x:I 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Abstract API coverage 
demo 
• Generate template 
java -jar jcov.jar TmplGen -abstract on -t template.xml <in> 
• Run with abstract 
java -classpath ... -javaagent:jcov.jar=abstract=on ... 
• Instrumentation 
ldc // int 3012 
invokestatic  
// Method com/sun/tdk/jcov/runtime/CollectDetect.invokeHit:(I)invokeinterface // InterfaceMethod ... 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Save points 
• Instrument 
java -jar jcov.jar Instr -savebegin <a method name>  
demo 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
-t template.xml <classes> 
• Instrumentation for the method: 
0: ldc // int 14 
2: invokestatic  
// Method com/sun/tdk/jcov/runtime/Collect.hit:(I)V 
5: invokestatic  
// Method com/sun/tdk/jcov/runtime/Collect.saveResults:()V
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Direct coverage 
• Only report code called directly from tests 
• “Fair” coverage 
● Controlled environment 
● Meaningful parameters 
Test 
Code 
Code 
Code
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Direct coverage with no “inner invocation” 
• Instrument code only (not tests) 
java -jar jcov.jar Instr -type method  
-innerinvocation off  
-t template.xml <classes> 
• Instrumentation 
ldc // int -1 
invokestatic // Method  
com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V 
... // Some other code 
ldc // int 0 
invokestatic // Method  
com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V 
return 
demo
demo 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Direct coverage with “caller include” 
• Run tests, instrumenting code and the tests 
java -classpath ... 
-javaagent:jcov.jar=ci=<test packages>,type=method  
... 
• Instrumentation of test code 
invokestatic  
// Method com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V 
invokevirtual // some product code call 
• Instrumention of product code 
invokestatic  
// Method com/sun/tdk/jcov/runtime/CollectDetect.hit:(III)V
Collecting data for big suites 
• Decide on dynamic vs static 
• Save data periodically 
–Even in case of same vm 
–Otherwise risking loosing the data 
• Using file 
–One file for all tests – getting bigger in time – huge performance impact, if 
multiple VMs 
–Separate files for tests – takes a lot of time to merge 
• Use the grabber! 
–Run test consequently for individual test coverage. 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 41 
Getting the data 
• Instrument dynamically or statically 
• Save the data onto a file or grabber 
• Individual test coverage 
• Abstract coverage 
• Fields coverage 
• Data save points 
• “Direct” coverage
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
There is more. 
Exec Executes a command collecting coverage data in a grabber 
Agent print help on usage jcov in dynamic mode 
Instr instruments classfiles and creates template.xml 
JREInstr instrumenter designed for instumenting rt.jar 
ProductInstr 
Instr2 instrumenter designed for abstract, native methods and TmplGen generates the jcov template.xml 
Grabber gathers information from JCov runtime via sockets 
GrabberManager control commands to the Grabber server 
Merger merges several jcov data files 
RepMerge merges jcov data files at method level not caring of blocks Filter filters out result data 
DiffCoverage check whether changed lines were covered 
RepGen generates text or HTML (or custom) reports 
JCov gets product coverage with one command
There is more. TmplGen 
Verbosity: -verbose 
Template specification: -template(t) 'string value' 
Type of template: -type [all|branch|block|method] 
Filtering conditions: 
-include(i) 'string value', -exclude(e) 'string value', 
-include_list 'string value', -exclude_list 'string value' 
Specify which items should be additionally included in template: 
-abstract [on|off], -native [on|off], -field [on|off] 
-synthetic [on|off], -anonym [on|off] 
Flush instrumented classes: -flush 'string value' 
Basic options: -help(h, ?), -help-verbose(hv) 
-print-env(env), -propfile 'string value' 
-plugindir 'string value', -log.file 'string value' 
-log.level(log) 'string value' 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
There is more. Instr 
Output: -instr.output(output, o) 'string value' 
Verbose mode: -verbose 
Type of template: -type [all|branch|block|method] 
Filtering conditions: -include(i) 'string value’, -include_list 'string 
value’, -exclude(e) 'string value’, -caller_include(ci) 'string value’, 
-caller_exclude(ce) 'string value’, -exclude_list 'string value' 
Save points: -savebegin 'string value’, -saveatend 'string value' 
Template specification: -template(t) 'string value’, -subsequent 
Items to be included: -abstract [on|off], -native [on|off], -field [on|off], 
-synthetic [on|off], -anonym [on|off], -innerinvocation [on|off] 
Flush instrumented classes: -flush 'string value' 
Runtime management: -implantrt(rt) 'string value’, -recursive 
Basic options: -help(h, ?), -help-verbose(hv), -print-env(env), -propfile 
'string value’, -plugindir 'string value’, -log.file 'string value’, 
-log.level(log) 'string value' 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
There is more. Merger. 
Output file: -merger.output(output, o) 'string value' 
File to read jcov input files from: -filelist 'string value' 
Filtering conditions: -include(i) 'string value' 
-exclude(e) 'string value', -fm 'string value' 
-include_list 'string value', -exclude_list 'string value' 
-fm_list 'string value' 
Process/generate test scales: -scale, -outTestList 'string value' 
Verbose mode: -verbose(v) 
Looseness level: -loose [0|1|2|3|blocks] 
Compress test scales: -compress 
Break on error: -breakonerror [file|error|test|skip|none], -critwarn 
Template path: -template(tmpl, t) 'string value' 
Skipped files: -outSkipped 'string value' 
Basic options: -help(h, ?) -help-verbose(hv) -print-env(env) 
-propfile 'string value' -plugindir 'string value 
-log.file 'string value' -log.level(log) 'string value 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Program Agenda 
History, facts 
Getting the data 
Applied to OpenJDK 
Using the data 
Links, more information 
2 
46 
1 
3 
4 
5
OpenJDK coverage. Very simple 
demo 
• Template 
java -jar jcov.jar TmplGen -t template.xml rt.jar 
• Start grabber 
java -jar jcov.jar Grabber -v -t template.xml start 
• Execute tests 
jtreg ... -javaoptions:"-javaagent:$JCOV_JAR=grabber" <tests> 
• Stop grabber, merge, generate report 
java -jar jcov.jar GrabberManager -kill 
java -jar jcov.jar Merger -o merged.xml -t template.xml  
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
result.xml 
java -jar jcov.jar RepGen -src jdk/src/share/classes merged.xml
OpenJDK coverage. Static. 
• Instrument 
java -jar jcov.jar JREInstr ... -implant jcov.jar  
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
-t template.xml .../jre 
• Run tests, etc
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Program Agenda 
History, facts 
Getting the data 
Applied to OpenJDK 
Using the data 
Links, more information 
2 
49 
1 
3 
4 
5
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 50 
Using the data 
• 100% coverage. Needed? Possible? Enough? 
• Prioritizing 
• Public API 
• Monitoring coverage 
• Speed up test execution 
• More on test development prioritization 
• Comparing suites, runs
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 51 
Block coverage target value 
Cost of testing 
- 
Defects found * COD 
= 
Return
100% block coverage 1 
false • 1 test 
• 100% pass rate 
• 100% coverage. 
• The bug is not discovered 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 52
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 53 
100% block and branch coverage 1 
true 
-1 
false • 2 tests 
• 100% pass rate 
• 100% block coverage. 100% branch coverage. 
• The bug is not discovered
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 54 
Prioritizing test development 
• 100% coverage is not the goal 
• Too much code to choose from 
• Filter the coverage data to leave only code to cover 
–Public API 
–UI 
–Controller code (as in MVC) 
• Prioritize code 
–By age 
–By complexity 
–By bug density
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 55 
Public API
Public API 
• “Methods which supposed to be called directly from user code” 
• Every method in public API needs to be called at least once (*) 
• 100% public API does not prove anything 
• Necessary. Not sufficient. 
• Rather blunt solution: 
java -jar jcov.jar RepGen -publicapi ... 
–In JDK8: public and protected methods of public and protected class in 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 56 
java.* and javax.* (**) 
–In JDK9: … something different 
* More later 
** Not completely accurate
demo 
JCov API 
• com.sun.tdk.jcov.instrument.DataRoot 
–Load, save 
• com.oracle.java.sqe.inspection.JCovFinder, 
JCovInspector, JCovVisitor 
–walk over coverage data 
• com.sun.tdk.jcov.instrument.DataBlock, DataBranch, 
DataClass, DataMethod, DataPackage 
–Investigate, edit 
• com.sun.tdk.jcov.filter.MemberFilter, FilterSPI, 
Filter JCov vommand 
–filter 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 57
Abstract public API 
• Code coverage 
• Good API is abstract 
• An abstract public API is covered when at least one of its implementations 
is covered. 
• Done with JCov abstract coverage and filtering 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 58
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 59 
“Public API implementation”
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 60 
Public API implementation coverage 
• Create template with abstract API 
• Filter, only leaving 
–Implementations of public API 
–Extensions of public API 
• Filter coverage data by the template
Public API implementation closure 
• It may be required to extend the original set 
//original public API 
public interface Action { public abstract void perform();} 
//some implementation 
public class MySomethingAction implements Action{ 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 61 
@Override 
public void perform() { 
//prepare something 
doPerform(); 
} 
protected abstract void doPerform(); 
} 
add to 
filter
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 62 
Public API closure 
• A set of methods implementing library interface 
–Non-abstract public API 
–Implementations of abstract public API 
–Overrides of non-abstract public API 
–“Delegation” API 
● closure of the delegation API 
demo
demo 
Sane public API 
“Every method in public API needs to be called at least once” - Is this so? 
• Some code is trivial 
–One line getters 
–Overloads 
• Different code requires different techniques. 
–java.lang.Object.hashCode() 
• Some code has low impact 
–Exception constructors 
• More 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 63
UI code coverage 
• Similar to public API: 
–If there is a form in the product UI, it needs to be covered at least once 
• Identify UI code 
–constructing UI objects 
–displaying UI objects 
–actions with UI 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 64 
● javax.sing.Action.actionPerformed
Controller code coverage 
• Per MVC 
–Controller accepts inputs and and converts to commands to view or 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 65 
model 
• Very little boilerplate code 
• In some cases identifiable by classes and packages 
• Could be marked explicitly. @Important
Linking CC to other characteristics of the source code 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 66 
Characteristic Reasoning 
Age 
New code is better to be tested before getting to customer. 
Old code is either already tested or not needed. :) 
Number of changes 
More times the code was changed, 
more atomic improvements were implemented 
Bug density 
Many bugs already found means there are more 
hidden by existing bugs and more could be introduced with fixes 
Complexity 
Assuming similar engineering talent and the same technology … 
more bugs are likely to exist in complex code
Linking CC to other characteristics of the source code 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 67 
● A formula (1 – cc) * (a 
1 
*x1 + a 
*x2 + a 
2 
*x3 
3 
+ ...) 
–cc – code coverage (0, 1) 
–x- characteristic 
i 
–a 
i 
– importance coefficient 
• Coefficients are important
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 68 
Code coverage monitoring 
• Build by build 
• Platform to platform 
• JCov comparison report 
• Homegrown database solution (*) 
• Requires an apparatus to compare the coverage data 
* More at the end of the presentation
Compare two run on the same build 
• Comparison report 
java -jar jcov.jar RepGen file1.xml file2m.xml 
demo 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 69
demo 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 70 
Compare two runs of different builds 
• Bytecode is not comparable. 
• Loose the details 
java -jar jcov.jar Merger -loose blocks  
file1.xml file2m.xml 
• Only methods left 
<meth name="isInvalid" vmsig="()Z" ... id="18" count="150"/> 
<meth name="getPrefixLength" vmsig="()I" ... id="30" count="28"/> 
Comparison report 
java -jar jcov.jar RepGen file1.xml file2m.xml
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 71 
Subtraction 
• Compare two suites A and B 
• In (cc 
a 
– cc 
b 
) only that code which is covered in cc 
a 
and not in cc 
b 
• Using JCov API 
–Load cc 
, cc 
a 
b 
–Walk over cc 
a 
–Search in cc 
b 
–If found – null the coverage 
• Review (cc 
a 
– cc 
) and (cc 
b 
b 
– cc 
) 
a 
demo
Test base reduction 
• Collect coverage for tests individually and together. 
• Compare coverage from every test to the combined coverage. 
–if a test does not add coverage, execute it less 
• Conscious choice. 
–Remember that code coverage proves nothing. 
–Only use for acceptance test cycles. 
–Do not throw tests away – run every test although less often. 
–Analyze tests with give no additional coverage. 
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 72
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 73 
Using the data 
• 100% coverage. Needed? Possible? Enough? 
• Getting coverage subsets to cover 100% 
• Public API 
• UI 
• Controller 
• Monitoring coverage 
• Test base reduction 
• Prioritizing test dev by ranking 
• Comparing suites, runs
More info 
• Wiki: https://wiki.openjdk.java.net/display/CodeTools/jcov 
• Source: http://hg.openjdk.java.net/code-tools/jcov 
• Tutorial: http://hg.openjdk.java.net/code-tools/jcov/raw-file/tip/examples/tutorial/• How to build JCov: https://wiki.openjdk.java.net/display/CodeTools/How+To+Build+• Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 74 
“Pragmatic code coverage” on slideshare.net
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 75 
More to come 
• Plugins 
–IDE: Netbeans, Eclipse, Intellij Idea 
–maven 
• Backend 
–Coverage storage database 
–Trends, statistics 
–Ranking 
• Test base reduction 
Hey, it's open-source! Contribute.
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 
Java Code Coverage 
with JCov 
Implementation Details and Use Cases 
Alexandre (Shura) Iline 
JDK Test Architect 
JDK SQE team 
Sept 1, 2014

More Related Content

Viewers also liked

Ec2013 tutorial-mb variability-final
Ec2013 tutorial-mb variability-finalEc2013 tutorial-mb variability-final
Ec2013 tutorial-mb variability-final
University of Rennes, INSA Rennes, Inria/IRISA, CNRS
 
Couverture de code
Couverture de codeCouverture de code
Couverture de code
CocoaHeads France
 
Collection pipeline par Mathieu Godart
Collection pipeline par  Mathieu GodartCollection pipeline par  Mathieu Godart
Collection pipeline par Mathieu Godart
CocoaHeads France
 
Code Coverage Web Application
Code Coverage Web ApplicationCode Coverage Web Application
Code Coverage Web Application
Ajay Singh
 
Fight your technical debt with Jenkins, Jacoco and Sonar
Fight your technical debt with Jenkins, Jacoco and SonarFight your technical debt with Jenkins, Jacoco and Sonar
Fight your technical debt with Jenkins, Jacoco and Sonar
Mickael Istria
 
The Impact of Code Review Coverage and Participation on Software Quality
The Impact of Code Review Coverage and Participation on Software QualityThe Impact of Code Review Coverage and Participation on Software Quality
The Impact of Code Review Coverage and Participation on Software Quality
Shane McIntosh
 
The Nightmare Fuzzing Suite and Blind Code Coverage Fuzzer
The Nightmare Fuzzing Suite and Blind Code Coverage FuzzerThe Nightmare Fuzzing Suite and Blind Code Coverage Fuzzer
The Nightmare Fuzzing Suite and Blind Code Coverage Fuzzer
Joxean Koret
 
Code Coverage Revised : EclEmma on JaCoCo
Code Coverage Revised : EclEmma on JaCoCoCode Coverage Revised : EclEmma on JaCoCo
Code Coverage Revised : EclEmma on JaCoCo
Evgeny Mandrikov
 
Code Coverage in Theory and in practice form the DO178B perspective
Code Coverage in Theory and in practice form the DO178B perspective   Code Coverage in Theory and in practice form the DO178B perspective
Code Coverage in Theory and in practice form the DO178B perspective
Engineering Software Lab
 
Code coverage & tools
Code coverage & toolsCode coverage & tools
Code coverage & tools
Rajesh Kumar
 
Code coverage analysis in testing
Code coverage analysis in testingCode coverage analysis in testing
Code coverage analysis in testing
Ni
 
Code coverage
Code coverageCode coverage
Code coverage
Vijayan Reddy
 
Code Coverage
Code CoverageCode Coverage
Code Coverage
Ernani Omar Cruz
 
JMockit Framework Overview
JMockit Framework OverviewJMockit Framework Overview
JMockit Framework Overview
Mario Peshev
 
Code coverage
Code coverageCode coverage
Code coverage
Return on Intelligence
 
Top 5 Code Coverage Tools in DevOps
Top 5 Code Coverage Tools in DevOpsTop 5 Code Coverage Tools in DevOps
Top 5 Code Coverage Tools in DevOps
scmGalaxy Inc
 
Code coverage for automation scripts
Code coverage for automation scriptsCode coverage for automation scripts
Code coverage for automation scripts
vodQA
 
Ja coco ignite
Ja coco igniteJa coco ignite
Ja coco ignite
Aleksandr Zhuikov
 

Viewers also liked (18)

Ec2013 tutorial-mb variability-final
Ec2013 tutorial-mb variability-finalEc2013 tutorial-mb variability-final
Ec2013 tutorial-mb variability-final
 
Couverture de code
Couverture de codeCouverture de code
Couverture de code
 
Collection pipeline par Mathieu Godart
Collection pipeline par  Mathieu GodartCollection pipeline par  Mathieu Godart
Collection pipeline par Mathieu Godart
 
Code Coverage Web Application
Code Coverage Web ApplicationCode Coverage Web Application
Code Coverage Web Application
 
Fight your technical debt with Jenkins, Jacoco and Sonar
Fight your technical debt with Jenkins, Jacoco and SonarFight your technical debt with Jenkins, Jacoco and Sonar
Fight your technical debt with Jenkins, Jacoco and Sonar
 
The Impact of Code Review Coverage and Participation on Software Quality
The Impact of Code Review Coverage and Participation on Software QualityThe Impact of Code Review Coverage and Participation on Software Quality
The Impact of Code Review Coverage and Participation on Software Quality
 
The Nightmare Fuzzing Suite and Blind Code Coverage Fuzzer
The Nightmare Fuzzing Suite and Blind Code Coverage FuzzerThe Nightmare Fuzzing Suite and Blind Code Coverage Fuzzer
The Nightmare Fuzzing Suite and Blind Code Coverage Fuzzer
 
Code Coverage Revised : EclEmma on JaCoCo
Code Coverage Revised : EclEmma on JaCoCoCode Coverage Revised : EclEmma on JaCoCo
Code Coverage Revised : EclEmma on JaCoCo
 
Code Coverage in Theory and in practice form the DO178B perspective
Code Coverage in Theory and in practice form the DO178B perspective   Code Coverage in Theory and in practice form the DO178B perspective
Code Coverage in Theory and in practice form the DO178B perspective
 
Code coverage & tools
Code coverage & toolsCode coverage & tools
Code coverage & tools
 
Code coverage analysis in testing
Code coverage analysis in testingCode coverage analysis in testing
Code coverage analysis in testing
 
Code coverage
Code coverageCode coverage
Code coverage
 
Code Coverage
Code CoverageCode Coverage
Code Coverage
 
JMockit Framework Overview
JMockit Framework OverviewJMockit Framework Overview
JMockit Framework Overview
 
Code coverage
Code coverageCode coverage
Code coverage
 
Top 5 Code Coverage Tools in DevOps
Top 5 Code Coverage Tools in DevOpsTop 5 Code Coverage Tools in DevOps
Top 5 Code Coverage Tools in DevOps
 
Code coverage for automation scripts
Code coverage for automation scriptsCode coverage for automation scripts
Code coverage for automation scripts
 
Ja coco ignite
Ja coco igniteJa coco ignite
Ja coco ignite
 

Similar to Java code coverage with JCov. Implementation details and use cases.

DevDays: Profiling With Java Flight Recorder
DevDays: Profiling With Java Flight RecorderDevDays: Profiling With Java Flight Recorder
DevDays: Profiling With Java Flight Recorder
Miro Wengner
 
ASML_FlightRecorderMeetsJava.pdf
ASML_FlightRecorderMeetsJava.pdfASML_FlightRecorderMeetsJava.pdf
ASML_FlightRecorderMeetsJava.pdf
Miro Wengner
 
Troubleshooting Tools In JDK
Troubleshooting Tools In JDKTroubleshooting Tools In JDK
Troubleshooting Tools In JDK
Poonam Bajaj Parhar
 
JDK 8 and JDK 8 Updates in OpenJDK
JDK 8 and JDK 8 Updates in OpenJDKJDK 8 and JDK 8 Updates in OpenJDK
JDK 8 and JDK 8 Updates in OpenJDK
Wolfgang Weigend
 
Java Mission Control in Java SE 7U40
Java Mission Control in Java SE 7U40Java Mission Control in Java SE 7U40
Java Mission Control in Java SE 7U40
Roger Brinkley
 
Batch Applications for the Java Platform
Batch Applications for the Java PlatformBatch Applications for the Java Platform
Batch Applications for the Java Platform
Sivakumar Thyagarajan
 
JVMs in Containers - Best Practices
JVMs in Containers - Best PracticesJVMs in Containers - Best Practices
JVMs in Containers - Best Practices
David Delabassee
 
Java mission control and java flight recorder
Java mission control and java flight recorderJava mission control and java flight recorder
Java mission control and java flight recorder
Wolfgang Weigend
 
Java is Container Ready - Vaibhav - Container Conference 2018
Java is Container Ready - Vaibhav - Container Conference 2018Java is Container Ready - Vaibhav - Container Conference 2018
Java is Container Ready - Vaibhav - Container Conference 2018
CodeOps Technologies LLP
 
Java Cloud and Container Ready
Java Cloud and Container ReadyJava Cloud and Container Ready
Java Cloud and Container Ready
CodeOps Technologies LLP
 
Preparing your code for Java 9
Preparing your code for Java 9Preparing your code for Java 9
Preparing your code for Java 9
Deepu Xavier
 
Ebs performance tuning session feb 13 2013---Presented by Oracle
Ebs performance tuning session  feb 13 2013---Presented by OracleEbs performance tuning session  feb 13 2013---Presented by Oracle
Ebs performance tuning session feb 13 2013---Presented by Oracle
Akash Pramanik
 
JDK 10 Java Module System
JDK 10 Java Module SystemJDK 10 Java Module System
JDK 10 Java Module System
Wolfgang Weigend
 
Diagnosing Your Application on the JVM
Diagnosing Your Application on the JVMDiagnosing Your Application on the JVM
Diagnosing Your Application on the JVM
Staffan Larsen
 
Profiling documentforaltrec
Profiling documentforaltrecProfiling documentforaltrec
Profiling documentforaltrec
Ravikiran Allamsetty
 
Java Performance & Profiling
Java Performance & ProfilingJava Performance & Profiling
Java Performance & Profiling
Isuru Perera
 
JVMs in Containers
JVMs in ContainersJVMs in Containers
JVMs in Containers
David Delabassee
 
Audit your reactive applications
Audit your reactive applicationsAudit your reactive applications
Audit your reactive applications
OCTO Technology
 
Monitoring and Troubleshooting Tools in Java 9
Monitoring and Troubleshooting Tools in Java 9Monitoring and Troubleshooting Tools in Java 9
Monitoring and Troubleshooting Tools in Java 9
Poonam Bajaj Parhar
 
Troubleshooting Java HotSpot VM
Troubleshooting Java HotSpot VMTroubleshooting Java HotSpot VM
Troubleshooting Java HotSpot VM
Poonam Bajaj Parhar
 

Similar to Java code coverage with JCov. Implementation details and use cases. (20)

DevDays: Profiling With Java Flight Recorder
DevDays: Profiling With Java Flight RecorderDevDays: Profiling With Java Flight Recorder
DevDays: Profiling With Java Flight Recorder
 
ASML_FlightRecorderMeetsJava.pdf
ASML_FlightRecorderMeetsJava.pdfASML_FlightRecorderMeetsJava.pdf
ASML_FlightRecorderMeetsJava.pdf
 
Troubleshooting Tools In JDK
Troubleshooting Tools In JDKTroubleshooting Tools In JDK
Troubleshooting Tools In JDK
 
JDK 8 and JDK 8 Updates in OpenJDK
JDK 8 and JDK 8 Updates in OpenJDKJDK 8 and JDK 8 Updates in OpenJDK
JDK 8 and JDK 8 Updates in OpenJDK
 
Java Mission Control in Java SE 7U40
Java Mission Control in Java SE 7U40Java Mission Control in Java SE 7U40
Java Mission Control in Java SE 7U40
 
Batch Applications for the Java Platform
Batch Applications for the Java PlatformBatch Applications for the Java Platform
Batch Applications for the Java Platform
 
JVMs in Containers - Best Practices
JVMs in Containers - Best PracticesJVMs in Containers - Best Practices
JVMs in Containers - Best Practices
 
Java mission control and java flight recorder
Java mission control and java flight recorderJava mission control and java flight recorder
Java mission control and java flight recorder
 
Java is Container Ready - Vaibhav - Container Conference 2018
Java is Container Ready - Vaibhav - Container Conference 2018Java is Container Ready - Vaibhav - Container Conference 2018
Java is Container Ready - Vaibhav - Container Conference 2018
 
Java Cloud and Container Ready
Java Cloud and Container ReadyJava Cloud and Container Ready
Java Cloud and Container Ready
 
Preparing your code for Java 9
Preparing your code for Java 9Preparing your code for Java 9
Preparing your code for Java 9
 
Ebs performance tuning session feb 13 2013---Presented by Oracle
Ebs performance tuning session  feb 13 2013---Presented by OracleEbs performance tuning session  feb 13 2013---Presented by Oracle
Ebs performance tuning session feb 13 2013---Presented by Oracle
 
JDK 10 Java Module System
JDK 10 Java Module SystemJDK 10 Java Module System
JDK 10 Java Module System
 
Diagnosing Your Application on the JVM
Diagnosing Your Application on the JVMDiagnosing Your Application on the JVM
Diagnosing Your Application on the JVM
 
Profiling documentforaltrec
Profiling documentforaltrecProfiling documentforaltrec
Profiling documentforaltrec
 
Java Performance & Profiling
Java Performance & ProfilingJava Performance & Profiling
Java Performance & Profiling
 
JVMs in Containers
JVMs in ContainersJVMs in Containers
JVMs in Containers
 
Audit your reactive applications
Audit your reactive applicationsAudit your reactive applications
Audit your reactive applications
 
Monitoring and Troubleshooting Tools in Java 9
Monitoring and Troubleshooting Tools in Java 9Monitoring and Troubleshooting Tools in Java 9
Monitoring and Troubleshooting Tools in Java 9
 
Troubleshooting Java HotSpot VM
Troubleshooting Java HotSpot VMTroubleshooting Java HotSpot VM
Troubleshooting Java HotSpot VM
 

Recently uploaded

Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
TaghreedAltamimi
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
Ayan Halder
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
Remote DBA Services
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
ICS
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
Alberto Brandolini
 
Top 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptxTop 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptx
devvsandy
 
Requirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional SafetyRequirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional Safety
Ayan Halder
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
ToXSL Technologies
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
Drona Infotech
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
Quickdice ERP
 
Mobile app Development Services | Drona Infotech
Mobile app Development Services  | Drona InfotechMobile app Development Services  | Drona Infotech
Mobile app Development Services | Drona Infotech
Drona Infotech
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
Rakesh Kumar R
 
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
Marcin Chrost
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
Sven Peters
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
Yara Milbes
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemUI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
Peter Muessig
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative AnalysisOdoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Envertis Software Solutions
 

Recently uploaded (20)

Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
 
Oracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptxOracle 23c New Features For DBAs and Developers.pptx
Oracle 23c New Features For DBAs and Developers.pptx
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
 
Top 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptxTop 9 Trends in Cybersecurity for 2024.pptx
Top 9 Trends in Cybersecurity for 2024.pptx
 
Requirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional SafetyRequirement Traceability in Xen Functional Safety
Requirement Traceability in Xen Functional Safety
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
 
Mobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona InfotechMobile App Development Company In Noida | Drona Infotech
Mobile App Development Company In Noida | Drona Infotech
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
 
Mobile app Development Services | Drona Infotech
Mobile app Development Services  | Drona InfotechMobile app Development Services  | Drona Infotech
Mobile app Development Services | Drona Infotech
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
 
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
 
SMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API ServiceSMS API Integration in Saudi Arabia| Best SMS API Service
SMS API Integration in Saudi Arabia| Best SMS API Service
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemUI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
UI5con 2024 - Keynote: Latest News about UI5 and it’s Ecosystem
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative AnalysisOdoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
Odoo ERP Vs. Traditional ERP Systems – A Comparative Analysis
 

Java code coverage with JCov. Implementation details and use cases.

  • 1. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
  • 2. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Java Code Coverage with JCov Implementation Details and Use Cases Alexandre (Shura) Iline JDK Test Architect JDK SQE team Oct 1, 2014
  • 3. Dmitry Fazunenko Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 3 JCov development Alexey Fedorchenko
  • 4. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 4
  • 5. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Program Agenda History, facts Getting the data with JCov Applied to OpenJDK Using the data Links, more information 1 2 3 4 5 5
  • 6. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 6 Facts • JCov is a Java Code Coverage tool. • The code coverage tool for JCK • The code coverage tool for Oracle JDK. • Other products: –JavaFX –SceneBuilder –...
  • 7. History • 1996: First prototype • 1997: Integration with JDK 1.1 • 1998: Used “in production” • JDK 1.2 • … • JDK 7 • 2014: Open-source as part of OpenJDK codetools project. • 2014: JCov 3.0 – a public “release” supporting JDK 8 • JDK 9 – in progress Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 7
  • 8. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. More on JCov dev 1.5 engineer at a time, on average :) Leonid Arbouzov, Alexander Petrov, Vladimir Generalov, Serguei Chukhontsev, Oleg Uliankin, Gregory Steuck, Pavel Ozhdikhin, Konstantin Bobrovsky, Robert Field, Alexander Kuzmin, Leonid Mesnik, Sergey Borodin, Andrey Titov, Dmitry Fazunenko, Alexey Fedorchenko
  • 9. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Code coverage is ● An information on what source code is exercised in execution 9
  • 10. most often ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Code coverage is ● An information on what source code is exercised in testing 10 Testing is ● Activities to prove that the code behaves as expected where ...
  • 11. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Other possible usages. • Usage monitoring • Save data in a test environment • Check coverage from generated logic • Fuzzing • Load testing • Regression test generation
  • 12. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Program Agenda History, facts Getting the data with JCov Applied to OpenJDK Using the data Links, more information 2 12 1 3 4 5
  • 13. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 13 Getting the data • A simplest use case • A pick or two under the hood • More possibilities – Dynamic vs. static instrumentation –Grabber – Individual test coverage – Abstract coverage – Drop points – Direct coverage – Running big test suites
  • 14. Simplest use case • Compile java files as usual • “Instrument” the byte code java -jar jcov.jar Instr <application classes> • Run the code java -classpath ...:jcov_file_saver.jar ... • Create a report java -jar jcov.jar RepGen <jcov xml file> demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  • 15. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. A pick under the hood 15
  • 16. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. JCov bytecode insertion A pick under the hood public int getX(); Code: 0: ldc #31 // int 2 2: invokestatic #29 // Method com/sun/tdk/jcov/runtime/Collect.hit:(I)V 5: aload_0 6: getfield #2 // Field x:I 9: ireturn 16 demo Warning: the code is a subject to change!!!
  • 17. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. JCov bytecode insertion As if it was in Java public int getX() { com.sun.tdk.jcov.runtime.Collect.hit(2); return x; } 17 Warning: the code is a subject to change!!!
  • 18. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Increasing the count One step deeper public class Collect { ... private static long counts[]; ... public static void hit(int slot) { counts[slot]++; } ... } 18 demo
  • 19. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Saving the data Finally ... Runtime.getRuntime().addShutdownHook(new Thread() { public void run() { ... Collect.disable(); Collect.saveResults(); Collect.enable(); ... } }); 19
  • 20. Performance impact • Instrumentation phase: –proportional to the application code size. • Execution phase: –Counters increments - very low. Depends on the size of blocks. –Saving data – from significant to blocking. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 20 ● There is a cure! :) More on next slides. • Report generation phase.
  • 21. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. XML coverage data <class name="Point" supername="java/lang/Object" ...> ... <meth name="getX" vmsig="()I" ...> <bl s="0" e="4"> <methenter s="0" e="4" id="5" count="1"/> <exit s="4" e="4" opcode="ireturn"/> </bl> </meth> </class> 21 demo Warning: the format is a subject to change!!!
  • 22. Back to the simplest use case • Compile java files as usual • “Instrument” the byte code java -jar jcov.jar Instr <application classes> • Run the code java -classpath ...:jcov_file_saver.jar ... • Create a report java -jar jcov.jar RepGen <jcov xml file> demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Incomplete data!!!
  • 23. Code coverage template • Describes all code there is to cover • Created during instrumentation: java -jar jcov.jar Instr -t template.xml <in> <out> • Created explicitly: java -jar jcov.jar TmplGen -t template.xml <in> • A regular JCov XML file. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. ● Merge ● Filter ● etc. etc.
  • 24. Simplest use case corrected demo • Compile java files as usual • “Instrument” the byte code java -jar jcov.jar Instr -t template.xml <directory or jar> • Run the code java -classpath ...:jcov_file_saver.jar ... • Merge with the template java -jar jcov.jar Merger -o merge.xml template.xml result.xml • Create a report java -jar jcov.jar RepGen result.xml Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  • 25. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Simplest use case with filtering • Compile java code • “Instrument” the byte code java -jar jcov.jar Instr -t template.xml -i com.company.product <application classes> • Run the code, merge with the template • Create a report java -jar jcov.jar RepGen -e com.company.product.test result.xml demo
  • 26. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. More on getting the data 26
  • 27. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Dynamic instrumentation • Compile java files as usual • Run with an extra VM option java -classpath ... -javaagent:jcov.jar ... • result.xml generated. • Merge and filter by the template java -jar jcov.jar Merger -o merge.xml -t template.xml result.xml demo demo
  • 28. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Dynamic vs. static instrumentation Dynamic Static Modification of the tested application Not needed Needed Coverage collected for All code(*) Instrumented code only Test execution Modified to pass more options Modified to add jcov code to the classpath (**) Performance Slows the classloading Faster 28 * There are options to limit the instrumented code ** Or inject the classes into the application itself
  • 29. “Network” grabber demo • Start the grabber on the network java -jar jcov.jar Grabber … -hostname host123 -port 3333 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -t template.xml • Run the tests java -classpath ... -javaagent:jcov.jar=grabber,host=host123,port=3333 ... • Stop the grabber – a data file is generated java -jar jcov.jar GrabberManager -kill • Merge, generate report
  • 30. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Individual code coverage • Track code coverage on a per-test level • “Usefulness” of the tests Test 2 Code Code Code Test 1
  • 31. Individual test coverage with JCov • Data is encoded into the XML file: <class name="Component" supername="java/lang/Object" ...> ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved. <meth name="&lt;init&gt;" vmsig="()V" ...> <bl s="0" e="151"> <methenter s="0" e="151" id="20" count="90359" scale="08fffffffffee7d8effffffffffffffffffffffffffffffffffffffffffffffff fffff3efffffefffffffffffffffffffffffffffffffffffffefffbfffffffffffffffbf fffffffffffffffffd83fffffffffffffffffffffff1"/> <exit s="151" e="151" opcode="return"/> </bl> </meth> ... </class>
  • 32. Test scales with files • Run tests separately, save result.xml's $ java -classpath ... -javaagent:jcov.jar ... my.tests.TestN $ cp result.xml result_TestN.xml • Merge data files $ java -jar jcov.jar Merger -o merge.xml Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -outTestList testlist.txt -t template.xml result_*.xml demo
  • 33. Test scales with grabber • Start the grabber $ java -jar jcov.jar -scale -outTestList testlist.txt -t template.xml • Run tests separately, let the grabber know test names $ java -classpath ... -javaagent:jcov.jar:grabber Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -Djcov.testname=TestN ... my.tests.TestN ● Stop the grabber, generate report. demo
  • 34. Field coverage • Generate template java -jar jcov.jar TmplGen -field on -t template.xml <in> • Instrumentation ldc // int 1951 invokestatic demo // Method com/sun/tdk/jcov/runtime/CollectDetect.invokeHit:(I)getfield // Field x:I Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  • 35. Abstract API coverage demo • Generate template java -jar jcov.jar TmplGen -abstract on -t template.xml <in> • Run with abstract java -classpath ... -javaagent:jcov.jar=abstract=on ... • Instrumentation ldc // int 3012 invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.invokeHit:(I)invokeinterface // InterfaceMethod ... Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  • 36. Save points • Instrument java -jar jcov.jar Instr -savebegin <a method name> demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -t template.xml <classes> • Instrumentation for the method: 0: ldc // int 14 2: invokestatic // Method com/sun/tdk/jcov/runtime/Collect.hit:(I)V 5: invokestatic // Method com/sun/tdk/jcov/runtime/Collect.saveResults:()V
  • 37. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Direct coverage • Only report code called directly from tests • “Fair” coverage ● Controlled environment ● Meaningful parameters Test Code Code Code
  • 38. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Direct coverage with no “inner invocation” • Instrument code only (not tests) java -jar jcov.jar Instr -type method -innerinvocation off -t template.xml <classes> • Instrumentation ldc // int -1 invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V ... // Some other code ldc // int 0 invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V return demo
  • 39. demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Direct coverage with “caller include” • Run tests, instrumenting code and the tests java -classpath ... -javaagent:jcov.jar=ci=<test packages>,type=method ... • Instrumentation of test code invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.setExpected:(I)V invokevirtual // some product code call • Instrumention of product code invokestatic // Method com/sun/tdk/jcov/runtime/CollectDetect.hit:(III)V
  • 40. Collecting data for big suites • Decide on dynamic vs static • Save data periodically –Even in case of same vm –Otherwise risking loosing the data • Using file –One file for all tests – getting bigger in time – huge performance impact, if multiple VMs –Separate files for tests – takes a lot of time to merge • Use the grabber! –Run test consequently for individual test coverage. Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  • 41. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 41 Getting the data • Instrument dynamically or statically • Save the data onto a file or grabber • Individual test coverage • Abstract coverage • Fields coverage • Data save points • “Direct” coverage
  • 42. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. There is more. Exec Executes a command collecting coverage data in a grabber Agent print help on usage jcov in dynamic mode Instr instruments classfiles and creates template.xml JREInstr instrumenter designed for instumenting rt.jar ProductInstr Instr2 instrumenter designed for abstract, native methods and TmplGen generates the jcov template.xml Grabber gathers information from JCov runtime via sockets GrabberManager control commands to the Grabber server Merger merges several jcov data files RepMerge merges jcov data files at method level not caring of blocks Filter filters out result data DiffCoverage check whether changed lines were covered RepGen generates text or HTML (or custom) reports JCov gets product coverage with one command
  • 43. There is more. TmplGen Verbosity: -verbose Template specification: -template(t) 'string value' Type of template: -type [all|branch|block|method] Filtering conditions: -include(i) 'string value', -exclude(e) 'string value', -include_list 'string value', -exclude_list 'string value' Specify which items should be additionally included in template: -abstract [on|off], -native [on|off], -field [on|off] -synthetic [on|off], -anonym [on|off] Flush instrumented classes: -flush 'string value' Basic options: -help(h, ?), -help-verbose(hv) -print-env(env), -propfile 'string value' -plugindir 'string value', -log.file 'string value' -log.level(log) 'string value' Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  • 44. There is more. Instr Output: -instr.output(output, o) 'string value' Verbose mode: -verbose Type of template: -type [all|branch|block|method] Filtering conditions: -include(i) 'string value’, -include_list 'string value’, -exclude(e) 'string value’, -caller_include(ci) 'string value’, -caller_exclude(ce) 'string value’, -exclude_list 'string value' Save points: -savebegin 'string value’, -saveatend 'string value' Template specification: -template(t) 'string value’, -subsequent Items to be included: -abstract [on|off], -native [on|off], -field [on|off], -synthetic [on|off], -anonym [on|off], -innerinvocation [on|off] Flush instrumented classes: -flush 'string value' Runtime management: -implantrt(rt) 'string value’, -recursive Basic options: -help(h, ?), -help-verbose(hv), -print-env(env), -propfile 'string value’, -plugindir 'string value’, -log.file 'string value’, -log.level(log) 'string value' Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  • 45. There is more. Merger. Output file: -merger.output(output, o) 'string value' File to read jcov input files from: -filelist 'string value' Filtering conditions: -include(i) 'string value' -exclude(e) 'string value', -fm 'string value' -include_list 'string value', -exclude_list 'string value' -fm_list 'string value' Process/generate test scales: -scale, -outTestList 'string value' Verbose mode: -verbose(v) Looseness level: -loose [0|1|2|3|blocks] Compress test scales: -compress Break on error: -breakonerror [file|error|test|skip|none], -critwarn Template path: -template(tmpl, t) 'string value' Skipped files: -outSkipped 'string value' Basic options: -help(h, ?) -help-verbose(hv) -print-env(env) -propfile 'string value' -plugindir 'string value -log.file 'string value' -log.level(log) 'string value Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
  • 46. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Program Agenda History, facts Getting the data Applied to OpenJDK Using the data Links, more information 2 46 1 3 4 5
  • 47. OpenJDK coverage. Very simple demo • Template java -jar jcov.jar TmplGen -t template.xml rt.jar • Start grabber java -jar jcov.jar Grabber -v -t template.xml start • Execute tests jtreg ... -javaoptions:"-javaagent:$JCOV_JAR=grabber" <tests> • Stop grabber, merge, generate report java -jar jcov.jar GrabberManager -kill java -jar jcov.jar Merger -o merged.xml -t template.xml Copyright © 2014, Oracle and/or its affiliates. All rights reserved. result.xml java -jar jcov.jar RepGen -src jdk/src/share/classes merged.xml
  • 48. OpenJDK coverage. Static. • Instrument java -jar jcov.jar JREInstr ... -implant jcov.jar Copyright © 2014, Oracle and/or its affiliates. All rights reserved. -t template.xml .../jre • Run tests, etc
  • 49. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Program Agenda History, facts Getting the data Applied to OpenJDK Using the data Links, more information 2 49 1 3 4 5
  • 50. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 50 Using the data • 100% coverage. Needed? Possible? Enough? • Prioritizing • Public API • Monitoring coverage • Speed up test execution • More on test development prioritization • Comparing suites, runs
  • 51. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 51 Block coverage target value Cost of testing - Defects found * COD = Return
  • 52. 100% block coverage 1 false • 1 test • 100% pass rate • 100% coverage. • The bug is not discovered Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 52
  • 53. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 53 100% block and branch coverage 1 true -1 false • 2 tests • 100% pass rate • 100% block coverage. 100% branch coverage. • The bug is not discovered
  • 54. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 54 Prioritizing test development • 100% coverage is not the goal • Too much code to choose from • Filter the coverage data to leave only code to cover –Public API –UI –Controller code (as in MVC) • Prioritize code –By age –By complexity –By bug density
  • 55. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 55 Public API
  • 56. Public API • “Methods which supposed to be called directly from user code” • Every method in public API needs to be called at least once (*) • 100% public API does not prove anything • Necessary. Not sufficient. • Rather blunt solution: java -jar jcov.jar RepGen -publicapi ... –In JDK8: public and protected methods of public and protected class in Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 56 java.* and javax.* (**) –In JDK9: … something different * More later ** Not completely accurate
  • 57. demo JCov API • com.sun.tdk.jcov.instrument.DataRoot –Load, save • com.oracle.java.sqe.inspection.JCovFinder, JCovInspector, JCovVisitor –walk over coverage data • com.sun.tdk.jcov.instrument.DataBlock, DataBranch, DataClass, DataMethod, DataPackage –Investigate, edit • com.sun.tdk.jcov.filter.MemberFilter, FilterSPI, Filter JCov vommand –filter Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 57
  • 58. Abstract public API • Code coverage • Good API is abstract • An abstract public API is covered when at least one of its implementations is covered. • Done with JCov abstract coverage and filtering Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 58
  • 59. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 59 “Public API implementation”
  • 60. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 60 Public API implementation coverage • Create template with abstract API • Filter, only leaving –Implementations of public API –Extensions of public API • Filter coverage data by the template
  • 61. Public API implementation closure • It may be required to extend the original set //original public API public interface Action { public abstract void perform();} //some implementation public class MySomethingAction implements Action{ Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 61 @Override public void perform() { //prepare something doPerform(); } protected abstract void doPerform(); } add to filter
  • 62. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 62 Public API closure • A set of methods implementing library interface –Non-abstract public API –Implementations of abstract public API –Overrides of non-abstract public API –“Delegation” API ● closure of the delegation API demo
  • 63. demo Sane public API “Every method in public API needs to be called at least once” - Is this so? • Some code is trivial –One line getters –Overloads • Different code requires different techniques. –java.lang.Object.hashCode() • Some code has low impact –Exception constructors • More Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 63
  • 64. UI code coverage • Similar to public API: –If there is a form in the product UI, it needs to be covered at least once • Identify UI code –constructing UI objects –displaying UI objects –actions with UI Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 64 ● javax.sing.Action.actionPerformed
  • 65. Controller code coverage • Per MVC –Controller accepts inputs and and converts to commands to view or Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 65 model • Very little boilerplate code • In some cases identifiable by classes and packages • Could be marked explicitly. @Important
  • 66. Linking CC to other characteristics of the source code Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 66 Characteristic Reasoning Age New code is better to be tested before getting to customer. Old code is either already tested or not needed. :) Number of changes More times the code was changed, more atomic improvements were implemented Bug density Many bugs already found means there are more hidden by existing bugs and more could be introduced with fixes Complexity Assuming similar engineering talent and the same technology … more bugs are likely to exist in complex code
  • 67. Linking CC to other characteristics of the source code Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 67 ● A formula (1 – cc) * (a 1 *x1 + a *x2 + a 2 *x3 3 + ...) –cc – code coverage (0, 1) –x- characteristic i –a i – importance coefficient • Coefficients are important
  • 68. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 68 Code coverage monitoring • Build by build • Platform to platform • JCov comparison report • Homegrown database solution (*) • Requires an apparatus to compare the coverage data * More at the end of the presentation
  • 69. Compare two run on the same build • Comparison report java -jar jcov.jar RepGen file1.xml file2m.xml demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 69
  • 70. demo Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 70 Compare two runs of different builds • Bytecode is not comparable. • Loose the details java -jar jcov.jar Merger -loose blocks file1.xml file2m.xml • Only methods left <meth name="isInvalid" vmsig="()Z" ... id="18" count="150"/> <meth name="getPrefixLength" vmsig="()I" ... id="30" count="28"/> Comparison report java -jar jcov.jar RepGen file1.xml file2m.xml
  • 71. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 71 Subtraction • Compare two suites A and B • In (cc a – cc b ) only that code which is covered in cc a and not in cc b • Using JCov API –Load cc , cc a b –Walk over cc a –Search in cc b –If found – null the coverage • Review (cc a – cc ) and (cc b b – cc ) a demo
  • 72. Test base reduction • Collect coverage for tests individually and together. • Compare coverage from every test to the combined coverage. –if a test does not add coverage, execute it less • Conscious choice. –Remember that code coverage proves nothing. –Only use for acceptance test cycles. –Do not throw tests away – run every test although less often. –Analyze tests with give no additional coverage. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 72
  • 73. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 73 Using the data • 100% coverage. Needed? Possible? Enough? • Getting coverage subsets to cover 100% • Public API • UI • Controller • Monitoring coverage • Test base reduction • Prioritizing test dev by ranking • Comparing suites, runs
  • 74. More info • Wiki: https://wiki.openjdk.java.net/display/CodeTools/jcov • Source: http://hg.openjdk.java.net/code-tools/jcov • Tutorial: http://hg.openjdk.java.net/code-tools/jcov/raw-file/tip/examples/tutorial/• How to build JCov: https://wiki.openjdk.java.net/display/CodeTools/How+To+Build+• Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 74 “Pragmatic code coverage” on slideshare.net
  • 75. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 75 More to come • Plugins –IDE: Netbeans, Eclipse, Intellij Idea –maven • Backend –Coverage storage database –Trends, statistics –Ranking • Test base reduction Hey, it's open-source! Contribute.
  • 76. Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Java Code Coverage with JCov Implementation Details and Use Cases Alexandre (Shura) Iline JDK Test Architect JDK SQE team Sept 1, 2014

Editor's Notes

  1. This is a Remote Speaker Picture slide ideal for including a picture with the speaker’s name and title and company. To Replace the Picture on this sample slide (this applies to all slides in this template that contain replaceable pictures) Select the sample picture and press Delete. Click the icon inside the shape to open the Insert Picture dialog box. Navigate to the location where the picture is stored, select desired picture and click on the Insert button to fit the image proportionally within the shape. Note: Do not right-click the image to change the picture inside the picture placeholder. This will change the frame size of the picture placeholder. Instead, follow the steps outlined above.
  2. This is a Safe Harbor Front slide, one of two Safe Harbor Statement slides included in this template. One of the Safe Harbor slides must be used if your presentation covers material affected by Oracle’s Revenue Recognition Policy To learn more about this policy, e-mail: Revrec-americasiebc_us@oracle.com For internal communication, Safe Harbor Statements are not required. However, there is an applicable disclaimer (Exhibit E) that should be used, found in the Oracle Revenue Recognition Policy for Future Product Communications. Copy and paste this link into a web browser, to find out more information. http://my.oracle.com/site/fin/gfo/GlobalProcesses/cnt452504.pdf For all external communications such as press release, roadmaps, PowerPoint presentations, Safe Harbor Statements are required. You can refer to the link mentioned above to find out additional information/disclaimers required depending on your audience.
  3. This slide can also be used as a Q and A slide
  4. This slide can also be used as a Q and A slide
  5. This slide can also be used as a Q and A slide
  6. This slide can also be used as a Q and A slide
  7. This slide can also be used as a Q and A slide
  8. This slide can also be used as a Q and A slide
  9. This is a sample Table slide, ideal for comparing data by year, product, revenues, etc. To Edit this Table, follow the steps below (this applies to all slides in this template that contain tables): To Change Text in Table: Select text in table. Begin typing desired text. To move from one cell to another, press Tab. To Change Font Color/Size: Select text, right-click and adjust the font setting on the Mini toolbar. Select desired attributes to change: font, size, boldness, color, etc. Note: many of the same commands can also be accessed from the Font group of the Home tab. To Insert or Delete Rows in Table: To delete a row place cursor in a cell of the row, right-click for a pop up menu, select Delete Row. To Insert a row, place cursor in the row you want to place a row before, right-click for a pop up menu, select Insert and make a selection from the submenu. You can also use the Insert and Delete Rows and Columns tools on the Table Tools Layout tab to perform these actions. To Insert or Delete Columns in Table: To delete a column, place cursor above the column (you will see a solid downward pointing arrow) click to select the column, right-click on the selected column, chose Delete Column from the pop up menu. To insert a column, place cursor above the column you wish to insert the new column before (you will see a solid downward pointing arrow) click to select the column, right-click for a pop up menu, select Insert and make a selection from the submenu. You can also use the Insert and Delete Rows and Columns tools on the Table Tools Layout tab to perform these actions. To Highlight a Cell, Column or Row (for emphasis): Click to select an individual cell or click and drag through the desired column(s) or row(s). From the Table Tools Design tab, select the Shading button and select a desired fill color. (Note: It is best to use white text over dark backgrounds and black or dark gray text over light backgrounds.) While a table is selected, other formatting options are available on the Table Tools Design and Layout tabs.
  10. This slide can also be used as a Q and A slide
  11. This slide can also be used as a Q and A slide
  12. This slide can also be used as a Q and A slide
  13. This slide can also be used as a Q and A slide
  14. This slide can also be used as a Q and A slide
  15. This slide can also be used as a Q and A slide
  16. This slide can also be used as a Q and A slide
  17. This slide can also be used as a Q and A slide
  18. This slide can also be used as a Q and A slide
  19. This slide can also be used as a Q and A slide
  20. This slide can also be used as a Q and A slide
  21. This slide can also be used as a Q and A slide
  22. This slide can also be used as a Q and A slide
  23. This slide can also be used as a Q and A slide
  24. This slide can also be used as a Q and A slide
  25. This slide can also be used as a Q and A slide
  26. This slide can also be used as a Q and A slide
  27. This slide can also be used as a Q and A slide
  28. This slide can also be used as a Q and A slide
  29. This slide can also be used as a Q and A slide
  30. This slide can also be used as a Q and A slide
  31. This slide can also be used as a Q and A slide
  32. This slide can also be used as a Q and A slide
  33. This slide can also be used as a Q and A slide
  34. This slide can also be used as a Q and A slide
  35. This slide can also be used as a Q and A slide
  36. This slide can also be used as a Q and A slide