Code coverage is a measure of how much of the source code of a program is tested by a test suite. It helps ensure quality by enabling early detection of flaws. Common types of code coverage include statement, function, path, condition, and branch coverage. Tools like Cobertura, Clover, and Emma can help measure and analyze code coverage. Aim for 70-80% test coverage but 100% is not always cost effective or possible. Code coverage should be implemented from the start of development.

1. Code Coverage
a stab to the questions of What, When, Why, and How
Aug 2011 the nugget presentation
prepared by Ernani Omar Cruz
Thursday, August 25, 11

2. Usual Developer’s Coding Cycle
Code Test Diagnose Fix
usually a time
consuming process
= COST
Overall Goal is to minimize time spent on “Fixing”
Thursday, August 25, 11

3. What?
Code Coverage
is a measure used in software testing. It describes the degree
to which the source code of a program has been tested.
It is a form of testing that inspects the code directly
and is therefore a form of white box testing.
Thursday, August 25, 11

4. Some Types of Code
Coverages
• Statement
• Function
• Path
• Condition
• Branch
Statement coverage is a code coverage metric that tells you whether the flow of control reached every
executable statement of source code at least once.
Has each function in the program been called? This metric reports whether you invoked each
function or procedure.
This metric reports whether each of the possible paths in each function have been followed. A
path is a unique sequence of branches from the function entry to the exit.
Condition coverage reports the true or false outcome of each condition.
A branch is the outcome of a decision, so branch coverage simply measures which decision outcomes
have been tested.
and more!
Thursday, August 25, 11

5. Which Do We Use?
It Depends to What We are trying to Code!
Accomplish
But the Point to Note is To Have A Tested Code that it
does what is EXPECTED of It to Do.
Thursday, August 25, 11

6. When?
It Starts with Construction
Thursday, August 25, 11

7. Early detection of flaws
Higher Confidence
Everyone Happy
Thursday, August 25, 11

8. Why?
Early detection of flaws
Higher Confidence
Everyone Happy
Thursday, August 25, 11

9. Why?
Why do we need to do this?
Early detection of flaws
Higher Confidence
Everyone Happy
Thursday, August 25, 11

10. Why?
Why do we need to do this?
To Ensure Quality
Early detection of flaws
Higher Confidence
Everyone Happy
Thursday, August 25, 11

11. Why?
Why do we need to do this?
To Ensure Quality
Minimize Bugs/Defects
Early detection of flaws
Higher Confidence
Everyone Happy
Thursday, August 25, 11

12. Why?
Why do we need to do this?
To Ensure Quality
Minimize Bugs/Defects
Early detection of flaws
Avoid Cost due to Rework and Delay
Higher Confidence
Everyone Happy
Thursday, August 25, 11

13. How much?
How much are we going to need to do this properly?
It Depends.
Requires Discretion
70-80% Coverage
Code coverage of 70-80% is a reasonable goal for system test of most
projects with most coverage metrics. Use a higher goal for projects
specifically organized for high testability or that have high failure costs.
Minimum code coverage for unit testing can be 10-20% higher than for
system testing. --- Bullseye “http://www.bullseye.com/minimum.html”
Thursday, August 25, 11

14. Why Not 100%
Coverage?
Although 100% code coverage may appear like a best
possible effort, even 100% code coverage is estimated to
only expose about half the faults in a system. Low code
coverage indicates inadequate testing, but high code
coverage guarantees nothing.
In a large system, achieving 100% code coverage is generally not cost effective. Some reasons are listed below.
• Some test cases are expensive to reproduce but are highly improbable. The cost to benefit ratio does not justify repeating these tests
simply to record the code coverage.
• Checks may exist for unexpected error conditions. Layers of code might obscure whether errors in low-level code propagate up to
higher level code. An engineer might decide that handling all errors creates a more robust solution than tracing the possible errors.
• Unreachable code in the current version might become reachable in a future version. An engineer might address uncertainty about
future development by investing a little more effort to add some capability that is not currently needed.
• Code shared among several projects is only partially utilized by the project under test.
Generally, the tester should stop increasing code coverage when the tests become contrived. When you focus more and more on making the
coverage numbers better, your motivation shifts away from finding bugs.
Thursday, August 25, 11

15. “100% code coverage isn't an achievement, it's a minimum
requirement. If you write a line of code, you'd better test it.”
- Robert Cecil Martin aka “Uncle Bob”
In the end it all boils down to ROI and pragmatism. Some apps need more testing than others.
Some modules need more testing than others. Some bugs need more fixing than others. There
will always be a judgment call about whether additional time and money spent on automated
testing and coverage are justified or are just a premature optimization.
Thursday, August 25, 11

16. Roles Each Has to Fill
• Project Manager
• Team Lead
• Developer
• Peer
• Tester/QAs
The one who allocates resources, shapes priorities, coordinates interactions with
project stakeholders and keeps team focused on goals.
The leader of the Development team on the project.
The team member who writes the software and performs the unit tests.
Co-Team member; The one who can help out, review, and check the
work of a team member (usually developer)
The one responsible to drive out proper testing and validates
functionality
Everyone Has A Responsibility/Stake in Ensuring Quality
Thursday, August 25, 11

17. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Thursday, August 25, 11

18. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Thursday, August 25, 11

19. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
Thursday, August 25, 11

20. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
DevPartner Java Edition (DPJ) integrates a set of functionality enabling developers to analyze Java code for Code Quality and
Complexity, Memory Leak Detection, Memory Profiling and Optimization, Performance Profiling and Optimization, Thread
Analysis and dead-lock detection, Code Coverage Analysis.
Thursday, August 25, 11

21. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
DevPartner Java Edition (DPJ) integrates a set of functionality enabling developers to analyze Java code for Code Quality and
Complexity, Memory Leak Detection, Memory Profiling and Optimization, Performance Profiling and Optimization, Thread
Analysis and dead-lock detection, Code Coverage Analysis.
EMMA supports class, method, line, and basic block coverage, aggregated at "all", package, source file, class, and method
levels. Line and block metrics are offered in "normal" and "weighted" varieties. Other entries in this FAQ provide details on
these. EMMA does not do branch or path coverage. However, EMMA reports fractional line coverage to help you visualize
untested branches in code.
Thursday, August 25, 11

22. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
DevPartner Java Edition (DPJ) integrates a set of functionality enabling developers to analyze Java code for Code Quality and
Complexity, Memory Leak Detection, Memory Profiling and Optimization, Performance Profiling and Optimization, Thread
Analysis and dead-lock detection, Code Coverage Analysis.
EMMA supports class, method, line, and basic block coverage, aggregated at "all", package, source file, class, and method
levels. Line and block metrics are offered in "normal" and "weighted" varieties. Other entries in this FAQ provide details on
these. EMMA does not do branch or path coverage. However, EMMA reports fractional line coverage to help you visualize
untested branches in code.
JTest’s basic functionality includes Unit test-case generation, static analysis, regression testing, runtime error
detection, and code review.
Thursday, August 25, 11

23. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
DevPartner Java Edition (DPJ) integrates a set of functionality enabling developers to analyze Java code for Code Quality and
Complexity, Memory Leak Detection, Memory Profiling and Optimization, Performance Profiling and Optimization, Thread
Analysis and dead-lock detection, Code Coverage Analysis.
EMMA supports class, method, line, and basic block coverage, aggregated at "all", package, source file, class, and method
levels. Line and block metrics are offered in "normal" and "weighted" varieties. Other entries in this FAQ provide details on
these. EMMA does not do branch or path coverage. However, EMMA reports fractional line coverage to help you visualize
untested branches in code.
JTest’s basic functionality includes Unit test-case generation, static analysis, regression testing, runtime error
detection, and code review.
A third party application that analyzes the codes with different perspectives that aids users in strategizing release plans.
Thursday, August 25, 11

24. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
DevPartner Java Edition (DPJ) integrates a set of functionality enabling developers to analyze Java code for Code Quality and
Complexity, Memory Leak Detection, Memory Profiling and Optimization, Performance Profiling and Optimization, Thread
Analysis and dead-lock detection, Code Coverage Analysis.
EMMA supports class, method, line, and basic block coverage, aggregated at "all", package, source file, class, and method
levels. Line and block metrics are offered in "normal" and "weighted" varieties. Other entries in this FAQ provide details on
these. EMMA does not do branch or path coverage. However, EMMA reports fractional line coverage to help you visualize
untested branches in code.
JTest’s basic functionality includes Unit test-case generation, static analysis, regression testing, runtime error
detection, and code review.
A third party application that analyzes the codes with different perspectives that aids users in strategizing release plans.
Codign Software is committed to providing a valuable, affordable solution for metrics, automating JUnit
tests, mock object creation, path & branch coverage and code testability. We keep our costs low because
we only develop features that have benefit and make sense.
Thursday, August 25, 11

25. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
DevPartner Java Edition (DPJ) integrates a set of functionality enabling developers to analyze Java code for Code Quality and
Complexity, Memory Leak Detection, Memory Profiling and Optimization, Performance Profiling and Optimization, Thread
Analysis and dead-lock detection, Code Coverage Analysis.
EMMA supports class, method, line, and basic block coverage, aggregated at "all", package, source file, class, and method
levels. Line and block metrics are offered in "normal" and "weighted" varieties. Other entries in this FAQ provide details on
these. EMMA does not do branch or path coverage. However, EMMA reports fractional line coverage to help you visualize
untested branches in code.
JTest’s basic functionality includes Unit test-case generation, static analysis, regression testing, runtime error
detection, and code review.
A third party application that analyzes the codes with different perspectives that aids users in strategizing release plans.
Codign Software is committed to providing a valuable, affordable solution for metrics, automating JUnit
tests, mock object creation, path & branch coverage and code testability. We keep our costs low because
we only develop features that have benefit and make sense.
BullseyeCoverage is a code coverage analyzer for C++ and C that tells you how much of your source code
was tested. You can use this information to quickly focus your testing effort and pinpoint areas that need to
be reviewed. Code coverage analysis is useful during unit testing, integration testing, and final release.
Thursday, August 25, 11

26. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
DevPartner Java Edition (DPJ) integrates a set of functionality enabling developers to analyze Java code for Code Quality and
Complexity, Memory Leak Detection, Memory Profiling and Optimization, Performance Profiling and Optimization, Thread
Analysis and dead-lock detection, Code Coverage Analysis.
EMMA supports class, method, line, and basic block coverage, aggregated at "all", package, source file, class, and method
levels. Line and block metrics are offered in "normal" and "weighted" varieties. Other entries in this FAQ provide details on
these. EMMA does not do branch or path coverage. However, EMMA reports fractional line coverage to help you visualize
untested branches in code.
JTest’s basic functionality includes Unit test-case generation, static analysis, regression testing, runtime error
detection, and code review.
A third party application that analyzes the codes with different perspectives that aids users in strategizing release plans.
Codign Software is committed to providing a valuable, affordable solution for metrics, automating JUnit
tests, mock object creation, path & branch coverage and code testability. We keep our costs low because
we only develop features that have benefit and make sense.
BullseyeCoverage is a code coverage analyzer for C++ and C that tells you how much of your source code
was tested. You can use this information to quickly focus your testing effort and pinpoint areas that need to
be reviewed. Code coverage analysis is useful during unit testing, integration testing, and final release.
Sonar is an open platform to manage code quality. As such, it covers the 7 axes of code quality: Architecture and Design, Duplications,
Thursday, August 25, 11

27. (some) Tools Available
that can help us measure/analyze code
coverage
• Cobertura
• Clover
• DevPartner
• Emma
• JTest
• Kalistick
• CoView and CoAnt
• Bullseye for C++
• Sonar
Cobertura is an open source tool that measures test coverage by instrumenting a code base and watching which lines of
code are and are not executed as the test suite runs
Some features include historical reporting, complete control over the coverage gathering process, command line toolset and
API for legacy integration and more.Clover also allows testing time to be reduced by only running the tests that cover the
application code that was modified since the previous build.
DevPartner Java Edition (DPJ) integrates a set of functionality enabling developers to analyze Java code for Code Quality and
Complexity, Memory Leak Detection, Memory Profiling and Optimization, Performance Profiling and Optimization, Thread
Analysis and dead-lock detection, Code Coverage Analysis.
EMMA supports class, method, line, and basic block coverage, aggregated at "all", package, source file, class, and method
levels. Line and block metrics are offered in "normal" and "weighted" varieties. Other entries in this FAQ provide details on
these. EMMA does not do branch or path coverage. However, EMMA reports fractional line coverage to help you visualize
untested branches in code.
JTest’s basic functionality includes Unit test-case generation, static analysis, regression testing, runtime error
detection, and code review.
A third party application that analyzes the codes with different perspectives that aids users in strategizing release plans.
Codign Software is committed to providing a valuable, affordable solution for metrics, automating JUnit
tests, mock object creation, path & branch coverage and code testability. We keep our costs low because
we only develop features that have benefit and make sense.
BullseyeCoverage is a code coverage analyzer for C++ and C that tells you how much of your source code
was tested. You can use this information to quickly focus your testing effort and pinpoint areas that need to
be reviewed. Code coverage analysis is useful during unit testing, integration testing, and final release.
Sonar is an open platform to manage code quality. As such, it covers the 7 axes of code quality: Architecture and Design, Duplications,
comments, coding rules, unit tests, potential bugs, and complexity.
Thursday, August 25, 11

28. So What do we Use?
Any.
Thursday, August 25, 11

29. Thursday, August 25, 11

30. Quality is central
Sonar is a web-based application. Rules, alerts, thresholds, exclusions, settings… can be configured online. By leveraging its database, Sonar not only allows to combine metrics
altogether but also to mix them with historical measures.
Sonar has got a very efficient way of navigating, a balance between high-level view, dashboard, TimeMachine and defect hunting
tools. This enables to quickly uncover projects and / or components that are in Technical Debt to establish action plans.
Thursday, August 25, 11

31. Features
All projects at a glance
Drill down to source code
Coding rules
Unit Tests
Standard Metrics
Time Machine
Maven Ready
Easiness
Leverage existing components
Plugins
Security
Thursday, August 25, 11

32. Thursday, August 25, 11

33. Software Quality Assessment based on
Lifecycle Expectations
The SQALE method is particularly devoted to the
management of the Technical debt (or design debt) of
AGILE software developments. It allows:
1 To define clearly what creates the technical debt
2 To estimate correctly this debt
3 To describe this debt into various parts relating to
the testability, the reliability, the changeability, the
maintainability… This classification allows to
analyze the impact of the debt and to define the
priority of code refactoring/remediation activities.
Text
Thursday, August 25, 11

34. Having Automated Test Suites
Prepares Us Towards Automated Integration
Thursday, August 25, 11

35. It’s Free.
Thursday, August 25, 11

36. Thank You.
Thursday, August 25, 11

37. References
• http://en.wikipedia.org/wiki/Code_coverage
• http://www.bullseye.com/statementCoverage.html
• http://www.bullseye.com/paper.html
• http://www.bullseye.com/minimum.html
• http://www.sonarsource.org/
• http://www.atlassian.com/software/clover/
• http://www.bullseye.com/coverage.html
• http://www.codign.com/pathbranchcode.html
• http://software-development-thoughts.blogspot.com/2011/04/
how-much-code-coverage-do-you-really.html
Thursday, August 25, 11

38. References
• http://en.wikipedia.org/wiki/Code_coverage#Software_tools
• http://www.codecoveragetools.com/index.php/coverage-process/
code-coverage-tools-java.html
• http://emma.sourceforge.net/faq.html#faq-N10042
• http://en.wikipedia.org/wiki/EMMA_(code_coverage_tool)
• http://en.wikipedia.org/wiki/DevPartner
• http://en.wikipedia.org/wiki/Clover_(software)
• http://cobertura.sourceforge.net/
• http://en.wikipedia.org/wiki/Jtest
Thursday, August 25, 11

39. References
• http://www.kalistick.com/software-radiography-
platform.html
• http://www.codign.com/
• http://www.bullseye.com/productInfo.html
• http://www.sqale.org/details
Thursday, August 25, 11