SlideShare a Scribd company logo
100% Code Coverage
in Real World Software
W-JAX 2017
2017-11-09
Once upon a time
*** *** Shipment Tracking
 Only a few
deployments slots per year
 per deployment -> $$$
 Re-deployment?
 wait for next slot
 pay more $$$
 -> Quality or die!
Why should we want
Automated Testing?
Why automated testing?
It‘s 2017, so a part of me wants to say…
Srsly?
Pixabay, License: CC0 Creative Commons
Why automated testing? (1/4)
OK… we want automated testing in order to…
 Protect against (regression) bugs
 Keep confidence & control
 Manage knowledge
 Ensure long term adaptability
 Work faster
 Build upon working parts
Q = 1..n qi
Systems Engineering: Software Lifecycle (2/4)
”Be nice to future-you“ – unknown
People working on software after…
 … 1 year? -> new people
 … 3 years? -> new > original
 … 6 years? -> 0 originals anymore
Future “generations“ of team members must be prepared for…
 … changes, enhancements etc.
 … patches, updates, upgrades, replacements etc.
1 2 5 †*
The Power of Metrics
Reduce complexity to merely binary information (3/4)
”Complexity”by Mark Skipper, License: CC BY 2.0
9
FAIL
10
PASS
Testability & Code Design (4/4)
TDD promotes these ideas:
 If it‘s hard to test…
 …it likely has design issues
 So don‘t try to find more and
more clever ways to test…
 … instead change the code to
make it more testable
 -> Testable is the new clever
Test Driven Development & Design
Why should we want
Code Coverage?
Why Code Coverage?
We want to find out what we don‘t know.
 Use it as a checklist
 Find blind spots
 Find dead, unused code
static double divide(double a, double b) {
if (b == 0) {
}
throw new DivisionByZeroException();
}
return a / b;
Why should we want
100% Code Coverage?
Why 100% Code Coverage? (1/6)
 Tests help a lot…
 … for managing the code
the tests cover
 … for code they don‘t cover
test offer no help
 That‘s like feeling bigly about
having the engine running…
 … when you should have all 4 of them running
Pixabay, License: CC0 Creative Commons
Why 100% Code Coverage? (2/6)
 Maybe you had a reason not to
cover some code… at that time…
 Will the future team still
know that reason?
 And, after the usual and
inevitable drifts in the code base…
 …will the reason still
be valid in the future?
”U Turn“ by André-Pierre du Plessis,
License: CC BY 2.0
Why 100% Code Coverage? (3/6)
 Often, bug clump in some areas
 As the saying goes: “90% of problems
are caused by 10% of the code“
 90% Code Coverage means you don‘t
cover 10%
 What if that‘s THOSE 10%?
 No? How can you be sure w/out tests?
Why 100% Code Coverage? (3/6)
 OK, that was number magic…
 …let‘s put it a different way:
 Which code will likely
cause more problems:
 highly tested code?
 untested, potentially even
untestable code?
Edvard Munch, “The Scream”
100% Code Coverage to prevent decay (4/6)
”It wasn’t me. It was that way already.“ – my children
Prevent decay by
“Broken Windows“:
 Some code sub-standard…
-> more code will follow
 … and developers can blame it
on the darned legacy code
 Instead, have a
“No code left behind“ strategy
“Broken windows, Northampton State Hospital”
by Karan Jain, License: CC BY-SA 2.0
100% Code Coverage as the Team’s Rule (5/6)
Use the power of metrics
 Rules that everyone needs to stick to
 … objectively measured by a 3rd party
 … let the metric “reward“ the team
when at 100%
 … use the metric to
reduce complexity to…
“Complexity kills. It sucks the life out of developers […]“ – Ray Ozzie
Source: Geek & Poke
21
FAIL
22
PASS
23
For professional developers,
automated testing is an obligation.
Uncovered Code does not comply
to that obligation.
24
Bottom line…
!
25
There is no reason why you
SHOULDN‘T WANT 100% Code Coverage.
(There may be reasons why you
CAN‘T get to 100%, though…)
Code Coverage is a fallacy just a tool.lielie fallacy
It depends on what you use it for…
… and how you interpret its output.
How to trick Code Coverage
“Ohh, I‘m a liar. Yeah.“ – Henry Rollins
 When you run a Test Case
it will always cover some code
 E.g. assertNotNull(new MyClass())
 -> x% Code Coverage
 -> 0 value
@Test
void test_init() {
assertNotNull(
new MyClass());
}
Just because Code Coverage can be faked
does not mean it is irrelevant.
Code Coverage != Test Coverage
Code Coverage means…
 … code is covered by tests
 … but you still can have bugs
Code Coverage does NOT mean…
… all scenarios are covered
-> that would be Test Coverage
static double divide(double a,
double b) {
return a / b;
}
@Test
void test_divide() {
double actual = divide(6, 2);
assertEquals(3, actual, .1);
}
// no validations,
// weak precision check,
// division by zero is possible!
// ...
Understand the metric
Where Code Coverage cannot be measured
“I don't think anybody tests enough of anything.“ – James Gosling
Views
 JSP, JSTL, JSF and others that
allow for markup + conditional code
Configuration
 Did we test ALL configuration entries?
 How can we spot obsolete configs?
“Game over”by porpom, License CC BY 2.0,
<c:if test="${temp > 18 && temp < 27}">
<p>Let‘s take a walk.</p>
</c:if>
Where Code Coverage breaks:
Aliens, Trade Offs & Lack of Knowledge
Alien Code
Generally: full coverage can be impossible
if you cannot change the production code
 3rd party code
 pasted “alien“ code
 generated alien code
e.g. XSD to Java, UML to code exports,
RAD tools, scaffolding scripts etc
 -> consider moving it out (JAR, src, …)
Pixabay, License: CC0 Creative Commons
Legacy Code
(Your own, untested) legacy code…
 static calls
 global calls
 global state
 wrong dependencies
 …
 …is like alien code to new team members
https://www.amazon.de/dp/0131177052/
Conscious Trade Offs
Sometimes you find yourself saying:
“I don‘t really know…
 … what I want yet.“
 … how to do it yet.“
-> You need a Proof of Concept (PoC)
 But TDD and 100% Code Coverage
are hard to do when you don‘t know what to assert
 However, professionals should create PoCs only as throw-away code
Pixabay, License: CC0 Creative Commons
Lack of Knowledge
Things you don‘t know how to test (yet)
 Sometimes you need to
master the technology first
 It may take some time & effort
to find solutions for testing
 -> Don‘t write testing off for good,
keep looking for solutions
 + look beyond “The Common Practice™”
Image: “Digging the Drain”by
International Disaster Volunteers, License: CC BY 2.0
3 tips that work in
The Real World™
Advice #1:
Start @ 100%, Stay @ 100%
TDD – Test Driven/First Development
Start @ 100%, Stay @ 100%
Image by user “Jgu7“
Get familiar with
 Test Driven Development
 … and the Test First approach
TDD DON‘Ts
Start @ 100%, Stay @ 100%
“Hexagonal Architecture”a.k.a.
“Ports and Adapters”by Alistair Cockburn,
Don‘t start on the “outside“,
i.e. do NOT start with:
 Controllers / Commands
 Views
 Data Access Objects / Queries
 File System Accessors
 …
Also, it‘s a psych thing
If you start on the outside you‘ll
 have slow progress
 feel the need for non-Unit tests
(i.e. slow tests)
 feel the need to mock stuff
 have a Coverage below 100%
 have less fun (always in debt)
Sigmund Freud, Wikipedia
Also, it‘s a psych thing
You might make bad decisions
 You might put more into the
class, e.g. the Controller,
than you should…
 … to kind of make the effort
spent “pay off“
 -> poor design and wrong
dependencies are possible
Pixabay, License: CC0 Creative Commons
The Dependency Rule
Inspired by “The Clean Architecture”
by Uncle Bob Martin
Enterprise
Domain
specific
Controllers
“source code
dependencies
can only point
inwards”
Actros, MPC,
Enterp. Rules
Trucker Profile,
App Rules
DI Container, OSGi
SysDate
DAOs
FS
SQL DB
NoSQL DB
SOAP/REST
SQL, Queries
Ports & Adapters / Dependency Inversion
 “But the Use Cases and Domain objects
need data from the outer layers!”
 “How can I read from and write
to the outer layer without depending on it?”
 -> Dependency INVERSION
 -> “Ports”
(simplified: outer layers behind interfaces)
Advice #2: Identify the Domain
The “No Logic“ argument
”Tests are for logic. My code does not contain logic “
 You might not see any logic…
 …but, usually, it‘s there…
 …it‘s probably just
in the wrong place
BIG
Controller
BIG
DAO
tiny
Service
“Weight Training Crossfit Fitness Models”
by ThoroughlyReviewed.com, License: CC BY 2.0
Identify the Domain
“The Clean Architecture”by Uncle Bob Martin
 … yes, it exists
 Start at the core: the Domain
 … also, it’s the code that’s the
easiest to test!
 you’ll start at 100% coverage
 … then stay at 100%
 … while progressing outwards
“source code dependencies can only point inwards”
Advice #3: When in doubt,
trust the test
Hard to test? -> keep it out
Pixabay, License: CC0 Creative Commons
Keep your Domain, Rules and
APIs clean, i.e. keep
 hard to test stuff out
 e.g. 3rd party frameworks
& libraries
 … (especially) if they
interfere with testability
Trust the test:
If (your own) code is hard to test
then it likely has a design problem.
So….?
!
Reality Check
Our experience shows:
 100% Code Coverage is
feasible & relevant
 However, you might need to
code and design differently…
 … but that will eventually
lead you to Clean Code and
Clean Architecture
“Big_Data_Higgs”by KamiPhuc, License: CC BY 2.0
Useful Links
 Ports & Adapters
 Dependency Rule
 Clean Architecture (new Book)
 DAO / Repository Pattern
 3 Rules of TDD
 Bowling Game Kata
 “The more testing you do…”
53
Thanks!
!
54
Questions?
?
#HappyHacking!
Andreas Czakaj
Managing Director IT / CTO
mensemedia Gesellschaft für Neue Medien mbH
Neumannstr. 10
40235 Düsseldorf, Germany
Email: andreas.czakaj@mensemedia.net
Twitter: @AndreasCzakaj
Blog: https://blog.acnebs.com/

More Related Content

What's hot

Technical and Testing Challenges: Using the "Protect The Square" Game
Technical and Testing Challenges: Using the "Protect The Square" GameTechnical and Testing Challenges: Using the "Protect The Square" Game
Technical and Testing Challenges: Using the "Protect The Square" Game
Alan Richardson
 
Secure Programming With Static Analysis
Secure Programming With Static AnalysisSecure Programming With Static Analysis
Secure Programming With Static Analysis
ConSanFrancisco123
 
10 Ways To Improve Your Code
10 Ways To Improve Your Code10 Ways To Improve Your Code
10 Ways To Improve Your Code
ConSanFrancisco123
 
Secure Coding in C/C++
Secure Coding in C/C++Secure Coding in C/C++
Secure Coding in C/C++
Dan-Claudiu Dragoș
 
Secure coding practices
Secure coding practicesSecure coding practices
Secure coding practices
Scott Hurrey
 
Null meet Code Review
Null meet Code ReviewNull meet Code Review
Null meet Code Review
Naga Venkata Sunil Alamuri
 

What's hot (6)

Technical and Testing Challenges: Using the "Protect The Square" Game
Technical and Testing Challenges: Using the "Protect The Square" GameTechnical and Testing Challenges: Using the "Protect The Square" Game
Technical and Testing Challenges: Using the "Protect The Square" Game
 
Secure Programming With Static Analysis
Secure Programming With Static AnalysisSecure Programming With Static Analysis
Secure Programming With Static Analysis
 
10 Ways To Improve Your Code
10 Ways To Improve Your Code10 Ways To Improve Your Code
10 Ways To Improve Your Code
 
Secure Coding in C/C++
Secure Coding in C/C++Secure Coding in C/C++
Secure Coding in C/C++
 
Secure coding practices
Secure coding practicesSecure coding practices
Secure coding practices
 
Null meet Code Review
Null meet Code ReviewNull meet Code Review
Null meet Code Review
 

Similar to 100% Code Coverage in Real World Software

100% Code Coverage in Symfony applications
100% Code Coverage in Symfony applications100% Code Coverage in Symfony applications
100% Code Coverage in Symfony applications
Andreas Czakaj
 
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red TeamWhat is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
MITRE ATT&CK
 
Unit Testing
Unit TestingUnit Testing
Unit Testing
Scott Leberknight
 
Writing malware while the blue team is staring at you
Writing malware while the blue team is staring at youWriting malware while the blue team is staring at you
Writing malware while the blue team is staring at you
Rob Fuller
 
Showing How Security Has (And Hasn't) Improved, After Ten Years Of Trying
Showing How Security Has (And Hasn't) Improved, After Ten Years Of TryingShowing How Security Has (And Hasn't) Improved, After Ten Years Of Trying
Showing How Security Has (And Hasn't) Improved, After Ten Years Of Trying
Dan Kaminsky
 
Good++
Good++Good++
Software Security : From school to reality and back!
Software Security : From school to reality and back!Software Security : From school to reality and back!
Software Security : From school to reality and back!
Peter Hlavaty
 
Thinking Outside the Sand[box]
Thinking Outside the Sand[box]Thinking Outside the Sand[box]
Thinking Outside the Sand[box]
Juniper Networks
 
Sandbox detection: leak, abuse, test - Hacktivity 2015
Sandbox detection: leak, abuse, test - Hacktivity 2015Sandbox detection: leak, abuse, test - Hacktivity 2015
Sandbox detection: leak, abuse, test - Hacktivity 2015
Zoltan Balazs
 
App locker
App lockerApp locker
Security vulnerabilities for grown ups - GOTOcon 2012
Security vulnerabilities for grown ups - GOTOcon 2012Security vulnerabilities for grown ups - GOTOcon 2012
Security vulnerabilities for grown ups - GOTOcon 2012
Vitaly Osipov
 
Abusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryAbusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec glory
Priyanka Aash
 
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Yevgeniy Brikman
 
Joxean Koret - Database Security Paradise [Rooted CON 2011]
Joxean Koret - Database Security Paradise [Rooted CON 2011]Joxean Koret - Database Security Paradise [Rooted CON 2011]
Joxean Koret - Database Security Paradise [Rooted CON 2011]
RootedCON
 
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting ClassThe Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
Rob Fuller
 
How to write clean & testable code without losing your mind
How to write clean & testable code without losing your mindHow to write clean & testable code without losing your mind
How to write clean & testable code without losing your mind
Andreas Czakaj
 
SmartphoneHacking_Android_Exploitation
SmartphoneHacking_Android_ExploitationSmartphoneHacking_Android_Exploitation
SmartphoneHacking_Android_Exploitation
Malachi Jones
 
The limits of unit testing by Craig Stuntz
The limits of unit testing by Craig StuntzThe limits of unit testing by Craig Stuntz
The limits of unit testing by Craig Stuntz
QA or the Highway
 
The Limits of Unit Testing by Craig Stuntz
The Limits of Unit Testing by Craig StuntzThe Limits of Unit Testing by Craig Stuntz
The Limits of Unit Testing by Craig Stuntz
QA or the Highway
 
Us 17-krug-hacking-severless-runtimes
Us 17-krug-hacking-severless-runtimesUs 17-krug-hacking-severless-runtimes
Us 17-krug-hacking-severless-runtimes
Ravishankar Somasundaram
 

Similar to 100% Code Coverage in Real World Software (20)

100% Code Coverage in Symfony applications
100% Code Coverage in Symfony applications100% Code Coverage in Symfony applications
100% Code Coverage in Symfony applications
 
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red TeamWhat is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
What is ATT&CK coverage, anyway? Breadth and depth analysis with Atomic Red Team
 
Unit Testing
Unit TestingUnit Testing
Unit Testing
 
Writing malware while the blue team is staring at you
Writing malware while the blue team is staring at youWriting malware while the blue team is staring at you
Writing malware while the blue team is staring at you
 
Showing How Security Has (And Hasn't) Improved, After Ten Years Of Trying
Showing How Security Has (And Hasn't) Improved, After Ten Years Of TryingShowing How Security Has (And Hasn't) Improved, After Ten Years Of Trying
Showing How Security Has (And Hasn't) Improved, After Ten Years Of Trying
 
Good++
Good++Good++
Good++
 
Software Security : From school to reality and back!
Software Security : From school to reality and back!Software Security : From school to reality and back!
Software Security : From school to reality and back!
 
Thinking Outside the Sand[box]
Thinking Outside the Sand[box]Thinking Outside the Sand[box]
Thinking Outside the Sand[box]
 
Sandbox detection: leak, abuse, test - Hacktivity 2015
Sandbox detection: leak, abuse, test - Hacktivity 2015Sandbox detection: leak, abuse, test - Hacktivity 2015
Sandbox detection: leak, abuse, test - Hacktivity 2015
 
App locker
App lockerApp locker
App locker
 
Security vulnerabilities for grown ups - GOTOcon 2012
Security vulnerabilities for grown ups - GOTOcon 2012Security vulnerabilities for grown ups - GOTOcon 2012
Security vulnerabilities for grown ups - GOTOcon 2012
 
Abusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec gloryAbusing bleeding edge web standards for appsec glory
Abusing bleeding edge web standards for appsec glory
 
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutionsCloud adoption fails - 5 ways deployments go wrong and 5 solutions
Cloud adoption fails - 5 ways deployments go wrong and 5 solutions
 
Joxean Koret - Database Security Paradise [Rooted CON 2011]
Joxean Koret - Database Security Paradise [Rooted CON 2011]Joxean Koret - Database Security Paradise [Rooted CON 2011]
Joxean Koret - Database Security Paradise [Rooted CON 2011]
 
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting ClassThe Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
 
How to write clean & testable code without losing your mind
How to write clean & testable code without losing your mindHow to write clean & testable code without losing your mind
How to write clean & testable code without losing your mind
 
SmartphoneHacking_Android_Exploitation
SmartphoneHacking_Android_ExploitationSmartphoneHacking_Android_Exploitation
SmartphoneHacking_Android_Exploitation
 
The limits of unit testing by Craig Stuntz
The limits of unit testing by Craig StuntzThe limits of unit testing by Craig Stuntz
The limits of unit testing by Craig Stuntz
 
The Limits of Unit Testing by Craig Stuntz
The Limits of Unit Testing by Craig StuntzThe Limits of Unit Testing by Craig Stuntz
The Limits of Unit Testing by Craig Stuntz
 
Us 17-krug-hacking-severless-runtimes
Us 17-krug-hacking-severless-runtimesUs 17-krug-hacking-severless-runtimes
Us 17-krug-hacking-severless-runtimes
 

Recently uploaded

Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
Marcin Chrost
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
Reetu63
 
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
OnePlan Solutions
 
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and MoreManyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
narinav14
 
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
kalichargn70th171
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
Alberto Brandolini
 
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management AppsBoost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
Jhone kinadey
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Paul Brebner
 
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
Luigi Fugaro
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
brainerhub1
 
42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
vaishalijagtap12
 
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
The Third Creative Media
 
What’s New in Odoo 17 – A Complete Roadmap
What’s New in Odoo 17 – A Complete RoadmapWhat’s New in Odoo 17 – A Complete Roadmap
What’s New in Odoo 17 – A Complete Roadmap
Envertis Software Solutions
 
Orca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container OrchestrationOrca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container Orchestration
Pedro J. Molina
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data PlatformAlluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio, Inc.
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
ToXSL Technologies
 

Recently uploaded (20)

Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
 
ppt on the brain chip neuralink.pptx
ppt  on   the brain  chip neuralink.pptxppt  on   the brain  chip neuralink.pptx
ppt on the brain chip neuralink.pptx
 
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...Transforming Product Development using OnePlan To Boost Efficiency and Innova...
Transforming Product Development using OnePlan To Boost Efficiency and Innova...
 
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and MoreManyata Tech Park Bangalore_ Infrastructure, Facilities and More
Manyata Tech Park Bangalore_ Infrastructure, Facilities and More
 
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
The Power of Visual Regression Testing_ Why It Is Critical for Enterprise App...
 
Upturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in NashikUpturn India Technologies - Web development company in Nashik
Upturn India Technologies - Web development company in Nashik
 
Modelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - AmsterdamModelling Up - DDDEurope 2024 - Amsterdam
Modelling Up - DDDEurope 2024 - Amsterdam
 
Boost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management AppsBoost Your Savings with These Money Management Apps
Boost Your Savings with These Money Management Apps
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
 
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
WMF 2024 - Unlocking the Future of Data Powering Next-Gen AI with Vector Data...
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
 
42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
 
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
Unlock the Secrets to Effortless Video Creation with Invideo: Your Ultimate G...
 
What’s New in Odoo 17 – A Complete Roadmap
What’s New in Odoo 17 – A Complete RoadmapWhat’s New in Odoo 17 – A Complete Roadmap
What’s New in Odoo 17 – A Complete Roadmap
 
Orca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container OrchestrationOrca: Nocode Graphical Editor for Container Orchestration
Orca: Nocode Graphical Editor for Container Orchestration
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data PlatformAlluxio Webinar | 10x Faster Trino Queries on Your Data Platform
Alluxio Webinar | 10x Faster Trino Queries on Your Data Platform
 
How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?How Can Hiring A Mobile App Development Company Help Your Business Grow?
How Can Hiring A Mobile App Development Company Help Your Business Grow?
 

100% Code Coverage in Real World Software

  • 1. 100% Code Coverage in Real World Software W-JAX 2017 2017-11-09
  • 2. Once upon a time *** *** Shipment Tracking  Only a few deployments slots per year  per deployment -> $$$  Re-deployment?  wait for next slot  pay more $$$  -> Quality or die!
  • 3. Why should we want Automated Testing?
  • 4. Why automated testing? It‘s 2017, so a part of me wants to say… Srsly? Pixabay, License: CC0 Creative Commons
  • 5. Why automated testing? (1/4) OK… we want automated testing in order to…  Protect against (regression) bugs  Keep confidence & control  Manage knowledge  Ensure long term adaptability  Work faster  Build upon working parts Q = 1..n qi
  • 6. Systems Engineering: Software Lifecycle (2/4) ”Be nice to future-you“ – unknown People working on software after…  … 1 year? -> new people  … 3 years? -> new > original  … 6 years? -> 0 originals anymore Future “generations“ of team members must be prepared for…  … changes, enhancements etc.  … patches, updates, upgrades, replacements etc. 1 2 5 †*
  • 7. The Power of Metrics
  • 8. Reduce complexity to merely binary information (3/4) ”Complexity”by Mark Skipper, License: CC BY 2.0
  • 11. Testability & Code Design (4/4) TDD promotes these ideas:  If it‘s hard to test…  …it likely has design issues  So don‘t try to find more and more clever ways to test…  … instead change the code to make it more testable  -> Testable is the new clever Test Driven Development & Design
  • 12. Why should we want Code Coverage?
  • 13. Why Code Coverage? We want to find out what we don‘t know.  Use it as a checklist  Find blind spots  Find dead, unused code static double divide(double a, double b) { if (b == 0) { } throw new DivisionByZeroException(); } return a / b;
  • 14. Why should we want 100% Code Coverage?
  • 15. Why 100% Code Coverage? (1/6)  Tests help a lot…  … for managing the code the tests cover  … for code they don‘t cover test offer no help  That‘s like feeling bigly about having the engine running…  … when you should have all 4 of them running Pixabay, License: CC0 Creative Commons
  • 16. Why 100% Code Coverage? (2/6)  Maybe you had a reason not to cover some code… at that time…  Will the future team still know that reason?  And, after the usual and inevitable drifts in the code base…  …will the reason still be valid in the future? ”U Turn“ by André-Pierre du Plessis, License: CC BY 2.0
  • 17. Why 100% Code Coverage? (3/6)  Often, bug clump in some areas  As the saying goes: “90% of problems are caused by 10% of the code“  90% Code Coverage means you don‘t cover 10%  What if that‘s THOSE 10%?  No? How can you be sure w/out tests?
  • 18. Why 100% Code Coverage? (3/6)  OK, that was number magic…  …let‘s put it a different way:  Which code will likely cause more problems:  highly tested code?  untested, potentially even untestable code? Edvard Munch, “The Scream”
  • 19. 100% Code Coverage to prevent decay (4/6) ”It wasn’t me. It was that way already.“ – my children Prevent decay by “Broken Windows“:  Some code sub-standard… -> more code will follow  … and developers can blame it on the darned legacy code  Instead, have a “No code left behind“ strategy “Broken windows, Northampton State Hospital” by Karan Jain, License: CC BY-SA 2.0
  • 20. 100% Code Coverage as the Team’s Rule (5/6) Use the power of metrics  Rules that everyone needs to stick to  … objectively measured by a 3rd party  … let the metric “reward“ the team when at 100%  … use the metric to reduce complexity to… “Complexity kills. It sucks the life out of developers […]“ – Ray Ozzie Source: Geek & Poke
  • 23. 23 For professional developers, automated testing is an obligation. Uncovered Code does not comply to that obligation.
  • 25. 25 There is no reason why you SHOULDN‘T WANT 100% Code Coverage. (There may be reasons why you CAN‘T get to 100%, though…)
  • 26. Code Coverage is a fallacy just a tool.lielie fallacy It depends on what you use it for… … and how you interpret its output.
  • 27. How to trick Code Coverage “Ohh, I‘m a liar. Yeah.“ – Henry Rollins  When you run a Test Case it will always cover some code  E.g. assertNotNull(new MyClass())  -> x% Code Coverage  -> 0 value @Test void test_init() { assertNotNull( new MyClass()); }
  • 28. Just because Code Coverage can be faked does not mean it is irrelevant.
  • 29. Code Coverage != Test Coverage Code Coverage means…  … code is covered by tests  … but you still can have bugs Code Coverage does NOT mean… … all scenarios are covered -> that would be Test Coverage static double divide(double a, double b) { return a / b; } @Test void test_divide() { double actual = divide(6, 2); assertEquals(3, actual, .1); } // no validations, // weak precision check, // division by zero is possible! // ... Understand the metric
  • 30. Where Code Coverage cannot be measured “I don't think anybody tests enough of anything.“ – James Gosling Views  JSP, JSTL, JSF and others that allow for markup + conditional code Configuration  Did we test ALL configuration entries?  How can we spot obsolete configs? “Game over”by porpom, License CC BY 2.0, <c:if test="${temp > 18 && temp < 27}"> <p>Let‘s take a walk.</p> </c:if>
  • 31. Where Code Coverage breaks: Aliens, Trade Offs & Lack of Knowledge
  • 32. Alien Code Generally: full coverage can be impossible if you cannot change the production code  3rd party code  pasted “alien“ code  generated alien code e.g. XSD to Java, UML to code exports, RAD tools, scaffolding scripts etc  -> consider moving it out (JAR, src, …) Pixabay, License: CC0 Creative Commons
  • 33. Legacy Code (Your own, untested) legacy code…  static calls  global calls  global state  wrong dependencies  …  …is like alien code to new team members https://www.amazon.de/dp/0131177052/
  • 34. Conscious Trade Offs Sometimes you find yourself saying: “I don‘t really know…  … what I want yet.“  … how to do it yet.“ -> You need a Proof of Concept (PoC)  But TDD and 100% Code Coverage are hard to do when you don‘t know what to assert  However, professionals should create PoCs only as throw-away code Pixabay, License: CC0 Creative Commons
  • 35. Lack of Knowledge Things you don‘t know how to test (yet)  Sometimes you need to master the technology first  It may take some time & effort to find solutions for testing  -> Don‘t write testing off for good, keep looking for solutions  + look beyond “The Common Practice™” Image: “Digging the Drain”by International Disaster Volunteers, License: CC BY 2.0
  • 36. 3 tips that work in The Real World™
  • 37. Advice #1: Start @ 100%, Stay @ 100%
  • 38. TDD – Test Driven/First Development Start @ 100%, Stay @ 100% Image by user “Jgu7“ Get familiar with  Test Driven Development  … and the Test First approach
  • 39. TDD DON‘Ts Start @ 100%, Stay @ 100% “Hexagonal Architecture”a.k.a. “Ports and Adapters”by Alistair Cockburn, Don‘t start on the “outside“, i.e. do NOT start with:  Controllers / Commands  Views  Data Access Objects / Queries  File System Accessors  …
  • 40. Also, it‘s a psych thing If you start on the outside you‘ll  have slow progress  feel the need for non-Unit tests (i.e. slow tests)  feel the need to mock stuff  have a Coverage below 100%  have less fun (always in debt) Sigmund Freud, Wikipedia
  • 41. Also, it‘s a psych thing You might make bad decisions  You might put more into the class, e.g. the Controller, than you should…  … to kind of make the effort spent “pay off“  -> poor design and wrong dependencies are possible Pixabay, License: CC0 Creative Commons
  • 42. The Dependency Rule Inspired by “The Clean Architecture” by Uncle Bob Martin Enterprise Domain specific Controllers “source code dependencies can only point inwards” Actros, MPC, Enterp. Rules Trucker Profile, App Rules DI Container, OSGi SysDate DAOs FS SQL DB NoSQL DB SOAP/REST SQL, Queries
  • 43. Ports & Adapters / Dependency Inversion  “But the Use Cases and Domain objects need data from the outer layers!”  “How can I read from and write to the outer layer without depending on it?”  -> Dependency INVERSION  -> “Ports” (simplified: outer layers behind interfaces)
  • 44. Advice #2: Identify the Domain
  • 45. The “No Logic“ argument ”Tests are for logic. My code does not contain logic “  You might not see any logic…  …but, usually, it‘s there…  …it‘s probably just in the wrong place BIG Controller BIG DAO tiny Service “Weight Training Crossfit Fitness Models” by ThoroughlyReviewed.com, License: CC BY 2.0
  • 46. Identify the Domain “The Clean Architecture”by Uncle Bob Martin  … yes, it exists  Start at the core: the Domain  … also, it’s the code that’s the easiest to test!  you’ll start at 100% coverage  … then stay at 100%  … while progressing outwards “source code dependencies can only point inwards”
  • 47. Advice #3: When in doubt, trust the test
  • 48. Hard to test? -> keep it out Pixabay, License: CC0 Creative Commons Keep your Domain, Rules and APIs clean, i.e. keep  hard to test stuff out  e.g. 3rd party frameworks & libraries  … (especially) if they interfere with testability
  • 49. Trust the test: If (your own) code is hard to test then it likely has a design problem.
  • 51. Reality Check Our experience shows:  100% Code Coverage is feasible & relevant  However, you might need to code and design differently…  … but that will eventually lead you to Clean Code and Clean Architecture “Big_Data_Higgs”by KamiPhuc, License: CC BY 2.0
  • 52. Useful Links  Ports & Adapters  Dependency Rule  Clean Architecture (new Book)  DAO / Repository Pattern  3 Rules of TDD  Bowling Game Kata  “The more testing you do…”
  • 55. #HappyHacking! Andreas Czakaj Managing Director IT / CTO mensemedia Gesellschaft für Neue Medien mbH Neumannstr. 10 40235 Düsseldorf, Germany Email: andreas.czakaj@mensemedia.net Twitter: @AndreasCzakaj Blog: https://blog.acnebs.com/