Unit testing
(Exploring
the other side as a
tester)
Abhijeet Vaikar
Abhijeet Vaikar
Senior Software Engineer Test @ Carousell Singapore
Co-organizer @ taqelah!
Testing software and writing test code since 8 years
LinkedIn: https://www.linkedin.com/in/abhijeetvaikar/
Twitter: @AbhijeetVaikar
Email: vaikar.abhijeet@gmail.com
Friendly disclaimer!
This session intends to just be an introduction to unit
testing based on my recent experiences with
learning, writing & understanding unit tests as a
tester.
What we’ll look at
● Jump straight into code - understand a Java application
● What is a unit test?
● Types of unit tests.
● How can unit tests help?
● Code coverage
● Effective unit tests
● How are unit tests adopted in the real world?
● Should testers involve themselves with unit testing? And how?
● FAQs
Store application
What is a unit test
There are multiple ways developers implement their unit tests. Regardless of the approach, there are
some common characteristics.
A unit test is an automated test that:
● Verifies output of a small piece of code for a given input.
● Is fast.
● Is Isolated.
Mockist style
● Isolation is on the system under test - usually a class or functions in the class.
● Any dependency of a unit (class / function) is replaced with a test double (For example mocks)
● With this approach we can verify interactions too. (For eg: Verify that my class under test called a
specific function of the dependency)
● Also useful when the dependency graph is too deep/complicated.
● These tests can get tied tightly to implementation details of the code under test. Thus they can
tend to become brittle.
Mockist style
Class 1
Test
Class 2
Test
Class 3
Test
Test
Double
Test
Double Test
Double
Mockist style demo
Classical style
● Isolation is on the test instead of the system under test. This means the tests need to be isolated
from each other.
● This also means tests can cover multiple classes without the need to replace every dependency
with a test double.
● We may still replace a shared dependency like a database with a test double.
● Some may call this a component test.
● Behaviour is tested rather than implementation details.
Classical style
Class 2
Class 1 Class 3
Test Database
Replace only the shared
database with a test double.
Classical style Isolation of tests
Class 2
Class 1 Class 3
Test 1 Fake
Database
Class 2
Class 1 Class 3
Test 2 Fake
Database
Classical style demo
Comparison*
Isolation of A unit is Uses test double for
Mockist style Units A class All but immutable
dependencies
Classic style Unit tests A class or a set of
classes
Shared
dependencies
*From Unit Testing Principles Practices & Patterns by Vladimir Khorikov
How can unit tests help?
● Isolated tests help in figuring out root cause faster.
● Unit tests are run the earliest in the development cycle so that highlight issues (regressions)
earliest.
● Helps with sustainable growth of the project code over time.
● Help in creating a good design of the code. For example they help with decoupling of the code.
● When written first before writing product code, it enables developers to think of all possible cases.
Writing unit tests later may cause developers to miss some cases.
● With unit tests in place, a developer can push their changes faster and with confidence.
Code coverage metrics
● Code coverage = (number of lines of code verified by tests / total number of lines of code) * 100
● A clearer formula = (number of lines of code verified by tests / total number of eligible lines of
code) * 100
● Coverage gives an idea of what parts of the code have been tested by tests (as an extension also
what parts are not covered).
● It’s meant to be used as an indicator / guiding signal of code quality and not an absolute metric.
● Low code coverage - could mean code is not well tested, thus poor quality.
● High code coverage - does not mean code is well tested and thus high quality.
Code coverage demo
Effective unit tests 💪
Effective unit tests
● are easy to write and maintain.
● have proper assertions. Assertion free tests = no tests at all.
● test the areas of the codebase that really matter (for example business logic). Don't just add unit
tests for the sake of adding unit tests everywhere. For example, there's no point adding unit tests
to POJOs.
● are part of the development cycle and build pipeline i.e., they should run every time you push your
code changes to the next stage.
There is no automated way to figure out whether existing unit tests are of good quality. This needs to be done with the help of
regular reviews (individual / peer).
How are unit tests adopted in the real world?
● Following a mix of both classic and mockist style approach.
● Many prefer writing tests first before writing the implementation code.
● Unit tests are integrated into the build pipeline. They run whenever their code is built on a pull
request.
● Some developers also add a pre-commit hook to run unit tests on their machine locally even before
pushing their code to remote version control.
● Many adopt a quality gate where if unit tests fail they are not allowed to merge their code for
deployment. Same applies for code coverage violation.
● Unit tests are not just written for product code but also for test automation frameworks!
● A lot of developers focus more on integration tests rather than writing unit tests.
● Some developers religiously write unit tests and maintain a high code coverage.
● Some developers do not write any unit tests at all.
● Parameterised / data driven unit tests are also written (Popular in the GoLang dev community)
Should testers be involved in unit testing? And how?
YES.
How?
● Write unit tests for your core components of test automation framework. Add them to your framework’s build
pipeline.
● Start conversations with your developers (frontend, backend both) to seek information:
○ Whether they write unit tests or not? If not, why not? If yes, how? What are the tools used?
○ What is the current code coverage of the product code? If low, why is it low? If high, is it a good test suite?
○ Are unit tests run as part of the product code’s build pipeline? If no, why not? If yes, do we block PRs when
unit tests fail?
○ Do we add unit tests to existing suite when a bug is found?
○ If a feature is deployed without unit tests, do they consider to add unit tests as a backlog/tech debt?
What does it mean to write unit tests for test automation
framework?🤔
Driver management Test Data handlers
Cloud service
handlers
Testcase
management
system handlers
Configuration
handlers
All classes that
support correct
functioning of the
test framework
Page Objects
Test classes /
Feature files / Test
specs
Reporting handlers
Write unit tests
Don’t write unit
tests
A test framework is also a software product
used by engineers.
FAQs❓
Q: Do unit tests find bugs?
A: Yes they do especially the obvious ones (provided they are written in the first place and the tests are
testing the right thing). If no unit tests are written, naturally no bugs will be found (at that level)! 😉
Q: Does that mean we should not focus on further levels of testing? (integration, end-end)?
A: Absolutely not. Regardless of whether unit tests are being written or not, it’s extremely important to
perform integration and end-end tests. We should not expect unit tests to find all bugs.
Q: Since they are unit tests and written by developers, should they cover just basic positive and
negative cases?
A: No. They should cover as many cases as possible.
Learning resources📚
Unit Testing - Principles, Practices and Patterns by Vladimir Khorikov (Book highly recommended)
Effective Unit Testing - A talk by Eliotte Rusty Harold (YouTube video)
TDD - Where Did it all go wrong - A talk by Ian Cooper (YouTube video)
Demo example created for this session (GitHub) - You can fork and write more unit tests against the code.

Unit testing (Exploring the other side as a tester)

  • 1.
    Unit testing (Exploring the otherside as a tester) Abhijeet Vaikar
  • 2.
    Abhijeet Vaikar Senior SoftwareEngineer Test @ Carousell Singapore Co-organizer @ taqelah! Testing software and writing test code since 8 years LinkedIn: https://www.linkedin.com/in/abhijeetvaikar/ Twitter: @AbhijeetVaikar Email: vaikar.abhijeet@gmail.com
  • 3.
    Friendly disclaimer! This sessionintends to just be an introduction to unit testing based on my recent experiences with learning, writing & understanding unit tests as a tester.
  • 5.
    What we’ll lookat ● Jump straight into code - understand a Java application ● What is a unit test? ● Types of unit tests. ● How can unit tests help? ● Code coverage ● Effective unit tests ● How are unit tests adopted in the real world? ● Should testers involve themselves with unit testing? And how? ● FAQs
  • 6.
  • 7.
    What is aunit test There are multiple ways developers implement their unit tests. Regardless of the approach, there are some common characteristics. A unit test is an automated test that: ● Verifies output of a small piece of code for a given input. ● Is fast. ● Is Isolated.
  • 8.
    Mockist style ● Isolationis on the system under test - usually a class or functions in the class. ● Any dependency of a unit (class / function) is replaced with a test double (For example mocks) ● With this approach we can verify interactions too. (For eg: Verify that my class under test called a specific function of the dependency) ● Also useful when the dependency graph is too deep/complicated. ● These tests can get tied tightly to implementation details of the code under test. Thus they can tend to become brittle.
  • 9.
    Mockist style Class 1 Test Class2 Test Class 3 Test Test Double Test Double Test Double
  • 10.
  • 11.
    Classical style ● Isolationis on the test instead of the system under test. This means the tests need to be isolated from each other. ● This also means tests can cover multiple classes without the need to replace every dependency with a test double. ● We may still replace a shared dependency like a database with a test double. ● Some may call this a component test. ● Behaviour is tested rather than implementation details.
  • 12.
    Classical style Class 2 Class1 Class 3 Test Database Replace only the shared database with a test double.
  • 13.
    Classical style Isolationof tests Class 2 Class 1 Class 3 Test 1 Fake Database Class 2 Class 1 Class 3 Test 2 Fake Database
  • 14.
  • 15.
    Comparison* Isolation of Aunit is Uses test double for Mockist style Units A class All but immutable dependencies Classic style Unit tests A class or a set of classes Shared dependencies *From Unit Testing Principles Practices & Patterns by Vladimir Khorikov
  • 16.
    How can unittests help? ● Isolated tests help in figuring out root cause faster. ● Unit tests are run the earliest in the development cycle so that highlight issues (regressions) earliest. ● Helps with sustainable growth of the project code over time. ● Help in creating a good design of the code. For example they help with decoupling of the code. ● When written first before writing product code, it enables developers to think of all possible cases. Writing unit tests later may cause developers to miss some cases. ● With unit tests in place, a developer can push their changes faster and with confidence.
  • 17.
    Code coverage metrics ●Code coverage = (number of lines of code verified by tests / total number of lines of code) * 100 ● A clearer formula = (number of lines of code verified by tests / total number of eligible lines of code) * 100 ● Coverage gives an idea of what parts of the code have been tested by tests (as an extension also what parts are not covered). ● It’s meant to be used as an indicator / guiding signal of code quality and not an absolute metric. ● Low code coverage - could mean code is not well tested, thus poor quality. ● High code coverage - does not mean code is well tested and thus high quality.
  • 18.
  • 19.
    Effective unit tests💪 Effective unit tests ● are easy to write and maintain. ● have proper assertions. Assertion free tests = no tests at all. ● test the areas of the codebase that really matter (for example business logic). Don't just add unit tests for the sake of adding unit tests everywhere. For example, there's no point adding unit tests to POJOs. ● are part of the development cycle and build pipeline i.e., they should run every time you push your code changes to the next stage. There is no automated way to figure out whether existing unit tests are of good quality. This needs to be done with the help of regular reviews (individual / peer).
  • 20.
    How are unittests adopted in the real world? ● Following a mix of both classic and mockist style approach. ● Many prefer writing tests first before writing the implementation code. ● Unit tests are integrated into the build pipeline. They run whenever their code is built on a pull request. ● Some developers also add a pre-commit hook to run unit tests on their machine locally even before pushing their code to remote version control. ● Many adopt a quality gate where if unit tests fail they are not allowed to merge their code for deployment. Same applies for code coverage violation. ● Unit tests are not just written for product code but also for test automation frameworks! ● A lot of developers focus more on integration tests rather than writing unit tests. ● Some developers religiously write unit tests and maintain a high code coverage. ● Some developers do not write any unit tests at all. ● Parameterised / data driven unit tests are also written (Popular in the GoLang dev community)
  • 21.
    Should testers beinvolved in unit testing? And how? YES. How? ● Write unit tests for your core components of test automation framework. Add them to your framework’s build pipeline. ● Start conversations with your developers (frontend, backend both) to seek information: ○ Whether they write unit tests or not? If not, why not? If yes, how? What are the tools used? ○ What is the current code coverage of the product code? If low, why is it low? If high, is it a good test suite? ○ Are unit tests run as part of the product code’s build pipeline? If no, why not? If yes, do we block PRs when unit tests fail? ○ Do we add unit tests to existing suite when a bug is found? ○ If a feature is deployed without unit tests, do they consider to add unit tests as a backlog/tech debt?
  • 22.
    What does itmean to write unit tests for test automation framework?🤔 Driver management Test Data handlers Cloud service handlers Testcase management system handlers Configuration handlers All classes that support correct functioning of the test framework Page Objects Test classes / Feature files / Test specs Reporting handlers Write unit tests Don’t write unit tests A test framework is also a software product used by engineers.
  • 23.
    FAQs❓ Q: Do unittests find bugs? A: Yes they do especially the obvious ones (provided they are written in the first place and the tests are testing the right thing). If no unit tests are written, naturally no bugs will be found (at that level)! 😉 Q: Does that mean we should not focus on further levels of testing? (integration, end-end)? A: Absolutely not. Regardless of whether unit tests are being written or not, it’s extremely important to perform integration and end-end tests. We should not expect unit tests to find all bugs. Q: Since they are unit tests and written by developers, should they cover just basic positive and negative cases? A: No. They should cover as many cases as possible.
  • 24.
    Learning resources📚 Unit Testing- Principles, Practices and Patterns by Vladimir Khorikov (Book highly recommended) Effective Unit Testing - A talk by Eliotte Rusty Harold (YouTube video) TDD - Where Did it all go wrong - A talk by Ian Cooper (YouTube video) Demo example created for this session (GitHub) - You can fork and write more unit tests against the code.