SlideShare a Scribd company logo
THE ART OF UNIT
TESTING –
TRUSTWORTHY TEST
READING NOTES BY BENJAMIN FAN
AGENDA
• What makes unit tests good
• Untrustworthy failing tests
• Untrustworthy passing tests
• Dealing with flaky test
WHAT MAKES UNIT TESTS GOOD
• Trustworthiness
• The test fails and you’re genuinely worried that something broke.
• The test passes and you feel relaxed.
• Maintainability
• Readability
• Without readability, the other two pillars fall pretty quickly.
UNTRUSTWORTHY FAILINGTESTS
• A real bug not been covered in tests -> you can only trust this kind of fail
• A buggy test gives a false failure
• The test is out of date due to a change in functionality -> will dig in deeper in next
chapter
• The test conflicts with another test
• The test is flaky
HOW BUGGYTEST HAPPENED
• Logic in unit test
• switch, if, or else statements
• foreach, for, or while loops
• Concatenations (+ sign, etc.)
• try, catch
• The best way to avoid buggy test is to adoptTest-driven development (TDD).
HOW BUGGYTEST HAPPENED
• Code under test
• Test with logic in it
HOW BUGGYTEST HAPPENED
• A more trustworthy test
HOW BUGGYTEST HAPPENED
• A name-finding function • Loops and ifs in a test
UNTRUSTWORTHY PASSING TESTS
• The test contains no asserts -> may has hidden asserts
• Can’t (Hard to) understand the test -> Readability
• Unit tests are mixed with flaky integration tests
• keeping your integration and unit tests in separate places
• The test verifies multiple concerns or exit points
• The test keeps changing
• random numbers, dynamically generated values
UNTRUSTWORTHY PASSING TESTS
• Can’t (Hard to) understand the test
• Tests with bad names
• Tests that are too long or have convoluted code
• Tests containing confusing variable names
• Tests containing hidden logic or assumptions that cannot be understood easily
• Test results that are inconclusive (neither failed nor passed)
• Test messages that don’t provide enough information
THE TESTVERIFIES MULTIPLE CONCERNS OR EXIT
POINTS
• A sample function to test
• Checking two exit points in the same
test
DEALINGWITH FLAKY TEST
Unit tests Component
tests
Integration
tests
API tests E2E/UI
isolated tests
E2E/UI
system tests
Flakiness caused by
• Shared memory resources
• Threads
• Random values
• Dynamically generated
inputs/outputs
• Time
• Logic bugs
Flakiness also caused by
• Shared resources
• Network issues
• Configuration issues
• Permission issues
• Load issues
• Security issues
• Other systems are down
• And more...
TAKE AWAY
• Trustworthy matter especially in test.
• Avoid complexity in tests, such as creating dynamic expected values or duplicating logic
from the underlying production code.
• If a test doesn’t have any asserts, you can’t understand what's it’s doing.
• To better identify flaky tests, put them in a special category or folder that can be run
separately.
• To reduce test flakiness, either fix the tests, convert flaky higher-level tests into less flaky
lower-level ones, or delete them.
Q&A

More Related Content

Similar to Reading Notes of The Art Of Unit Test Ch7

2020-Feb: Testing: Cables and Chains
2020-Feb: Testing: Cables and Chains2020-Feb: Testing: Cables and Chains
2020-Feb: Testing: Cables and Chains
Mark Windholtz
 
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
TEST Huddle
 
Test Driven Development
Test Driven DevelopmentTest Driven Development
Test Driven Development
Md. Enamul Haque Chowdhury
 
Why Automated Testing Matters To DevOps
Why Automated Testing Matters To DevOpsWhy Automated Testing Matters To DevOps
Why Automated Testing Matters To DevOps
dpaulmerrill
 
Unit Testing Best Practices
Unit Testing Best PracticesUnit Testing Best Practices
Unit Testing Best Practices
Tomaš Maconko
 
Benefits from AATs
Benefits from AATsBenefits from AATs
Benefits from AATs
Wyn B. Van Devanter
 
Agile Software Testing the Agilogy Way
Agile Software Testing the Agilogy WayAgile Software Testing the Agilogy Way
Agile Software Testing the Agilogy Way
Jordi Pradel
 
#1 unit testing
#1 unit testing#1 unit testing
#1 unit testing
eleksdev
 
Unit testing
Unit testingUnit testing
Unit testing
Brian Hu
 
Breaking Dependencies to Allow Unit Testing - DevIntersection Spring 2016
Breaking Dependencies to Allow Unit Testing - DevIntersection Spring 2016Breaking Dependencies to Allow Unit Testing - DevIntersection Spring 2016
Breaking Dependencies to Allow Unit Testing - DevIntersection Spring 2016
Steven Smith
 
Database Unit Testing Made Easy with VSTS
Database Unit Testing Made Easy with VSTSDatabase Unit Testing Made Easy with VSTS
Database Unit Testing Made Easy with VSTS
Sanil Mhatre
 
Unit testing
Unit testingUnit testing
Unit testing
Adam Birr
 
Clean tests
Clean testsClean tests
Clean tests
Agileee
 
Agile Acceptance testing with Fitnesse
Agile Acceptance testing with FitnesseAgile Acceptance testing with Fitnesse
Agile Acceptance testing with Fitnesse
ClareMcLennan
 
Introduction to software testing
Introduction to software testingIntroduction to software testing
Introduction to software testing
ASIT Education
 
Black box testing
Black box testingBlack box testing
Black box testing
Syeda Khadizatul maria
 
Unit testing
Unit testingUnit testing
Unit testing
Vinod Wilson
 
Software Engineering TESTING AND MAINTENANCE
Software Engineering TESTING AND MAINTENANCESoftware Engineering TESTING AND MAINTENANCE
Software Engineering TESTING AND MAINTENANCE
Dr Anuranjan Misra
 
Test-Driven Development
Test-Driven DevelopmentTest-Driven Development
Test-Driven Development
Meilan Ou
 
Level Up Your Salesforce Unit Testing
Level Up Your Salesforce Unit TestingLevel Up Your Salesforce Unit Testing
Level Up Your Salesforce Unit Testing
Gordon Bockus
 

Similar to Reading Notes of The Art Of Unit Test Ch7 (20)

2020-Feb: Testing: Cables and Chains
2020-Feb: Testing: Cables and Chains2020-Feb: Testing: Cables and Chains
2020-Feb: Testing: Cables and Chains
 
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
Testing As A Bottleneck - How Testing Slows Down Modern Development Processes...
 
Test Driven Development
Test Driven DevelopmentTest Driven Development
Test Driven Development
 
Why Automated Testing Matters To DevOps
Why Automated Testing Matters To DevOpsWhy Automated Testing Matters To DevOps
Why Automated Testing Matters To DevOps
 
Unit Testing Best Practices
Unit Testing Best PracticesUnit Testing Best Practices
Unit Testing Best Practices
 
Benefits from AATs
Benefits from AATsBenefits from AATs
Benefits from AATs
 
Agile Software Testing the Agilogy Way
Agile Software Testing the Agilogy WayAgile Software Testing the Agilogy Way
Agile Software Testing the Agilogy Way
 
#1 unit testing
#1 unit testing#1 unit testing
#1 unit testing
 
Unit testing
Unit testingUnit testing
Unit testing
 
Breaking Dependencies to Allow Unit Testing - DevIntersection Spring 2016
Breaking Dependencies to Allow Unit Testing - DevIntersection Spring 2016Breaking Dependencies to Allow Unit Testing - DevIntersection Spring 2016
Breaking Dependencies to Allow Unit Testing - DevIntersection Spring 2016
 
Database Unit Testing Made Easy with VSTS
Database Unit Testing Made Easy with VSTSDatabase Unit Testing Made Easy with VSTS
Database Unit Testing Made Easy with VSTS
 
Unit testing
Unit testingUnit testing
Unit testing
 
Clean tests
Clean testsClean tests
Clean tests
 
Agile Acceptance testing with Fitnesse
Agile Acceptance testing with FitnesseAgile Acceptance testing with Fitnesse
Agile Acceptance testing with Fitnesse
 
Introduction to software testing
Introduction to software testingIntroduction to software testing
Introduction to software testing
 
Black box testing
Black box testingBlack box testing
Black box testing
 
Unit testing
Unit testingUnit testing
Unit testing
 
Software Engineering TESTING AND MAINTENANCE
Software Engineering TESTING AND MAINTENANCESoftware Engineering TESTING AND MAINTENANCE
Software Engineering TESTING AND MAINTENANCE
 
Test-Driven Development
Test-Driven DevelopmentTest-Driven Development
Test-Driven Development
 
Level Up Your Salesforce Unit Testing
Level Up Your Salesforce Unit TestingLevel Up Your Salesforce Unit Testing
Level Up Your Salesforce Unit Testing
 

Recently uploaded

UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
Sven Peters
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptxLORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
lorraineandreiamcidl
 
openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain SecurityopenEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
Shane Coughlan
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
Remote DBA Services
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
timtebeek1
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Crescat
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
Aftab Hussain
 
E-commerce Application Development Company.pdf
E-commerce Application Development Company.pdfE-commerce Application Development Company.pdf
E-commerce Application Development Company.pdf
Hornet Dynamics
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
SOCRadar
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
Deuglo Infosystem Pvt Ltd
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
Aftab Hussain
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
Octavian Nadolu
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
Green Software Development
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
rodomar2
 

Recently uploaded (20)

UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
Microservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we workMicroservice Teams - How the cloud changes the way we work
Microservice Teams - How the cloud changes the way we work
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptxLORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx
 
openEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain SecurityopenEuler Case Study - The Journey to Supply Chain Security
openEuler Case Study - The Journey to Supply Chain Security
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
 
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdfAutomated software refactoring with OpenRewrite and Generative AI.pptx.pdf
Automated software refactoring with OpenRewrite and Generative AI.pptx.pdf
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
 
E-commerce Application Development Company.pdf
E-commerce Application Development Company.pdfE-commerce Application Development Company.pdf
E-commerce Application Development Company.pdf
 
socradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdfsocradar-q1-2024-aviation-industry-report.pdf
socradar-q1-2024-aviation-industry-report.pdf
 
Empowering Growth with Best Software Development Company in Noida - Deuglo
Empowering Growth with Best Software  Development Company in Noida - DeugloEmpowering Growth with Best Software  Development Company in Noida - Deuglo
Empowering Growth with Best Software Development Company in Noida - Deuglo
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
 
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of CodeA Study of Variable-Role-based Feature Enrichment in Neural Models of Code
A Study of Variable-Role-based Feature Enrichment in Neural Models of Code
 
Artificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension FunctionsArtificia Intellicence and XPath Extension Functions
Artificia Intellicence and XPath Extension Functions
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
Energy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina JonuziEnergy consumption of Database Management - Florina Jonuzi
Energy consumption of Database Management - Florina Jonuzi
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
 

Reading Notes of The Art Of Unit Test Ch7

  • 1. THE ART OF UNIT TESTING – TRUSTWORTHY TEST READING NOTES BY BENJAMIN FAN
  • 2. AGENDA • What makes unit tests good • Untrustworthy failing tests • Untrustworthy passing tests • Dealing with flaky test
  • 3. WHAT MAKES UNIT TESTS GOOD • Trustworthiness • The test fails and you’re genuinely worried that something broke. • The test passes and you feel relaxed. • Maintainability • Readability • Without readability, the other two pillars fall pretty quickly.
  • 4. UNTRUSTWORTHY FAILINGTESTS • A real bug not been covered in tests -> you can only trust this kind of fail • A buggy test gives a false failure • The test is out of date due to a change in functionality -> will dig in deeper in next chapter • The test conflicts with another test • The test is flaky
  • 5. HOW BUGGYTEST HAPPENED • Logic in unit test • switch, if, or else statements • foreach, for, or while loops • Concatenations (+ sign, etc.) • try, catch • The best way to avoid buggy test is to adoptTest-driven development (TDD).
  • 6. HOW BUGGYTEST HAPPENED • Code under test • Test with logic in it
  • 7. HOW BUGGYTEST HAPPENED • A more trustworthy test
  • 8. HOW BUGGYTEST HAPPENED • A name-finding function • Loops and ifs in a test
  • 9. UNTRUSTWORTHY PASSING TESTS • The test contains no asserts -> may has hidden asserts • Can’t (Hard to) understand the test -> Readability • Unit tests are mixed with flaky integration tests • keeping your integration and unit tests in separate places • The test verifies multiple concerns or exit points • The test keeps changing • random numbers, dynamically generated values
  • 10. UNTRUSTWORTHY PASSING TESTS • Can’t (Hard to) understand the test • Tests with bad names • Tests that are too long or have convoluted code • Tests containing confusing variable names • Tests containing hidden logic or assumptions that cannot be understood easily • Test results that are inconclusive (neither failed nor passed) • Test messages that don’t provide enough information
  • 11. THE TESTVERIFIES MULTIPLE CONCERNS OR EXIT POINTS • A sample function to test • Checking two exit points in the same test
  • 12. DEALINGWITH FLAKY TEST Unit tests Component tests Integration tests API tests E2E/UI isolated tests E2E/UI system tests Flakiness caused by • Shared memory resources • Threads • Random values • Dynamically generated inputs/outputs • Time • Logic bugs Flakiness also caused by • Shared resources • Network issues • Configuration issues • Permission issues • Load issues • Security issues • Other systems are down • And more...
  • 13. TAKE AWAY • Trustworthy matter especially in test. • Avoid complexity in tests, such as creating dynamic expected values or duplicating logic from the underlying production code. • If a test doesn’t have any asserts, you can’t understand what's it’s doing. • To better identify flaky tests, put them in a special category or folder that can be run separately. • To reduce test flakiness, either fix the tests, convert flaky higher-level tests into less flaky lower-level ones, or delete them.
  • 14. Q&A