This document introduces automated testing and provides guidance on how to implement it. It discusses: 1) The benefits of unit tests such as finding bugs quickly, preventing regressions, and improving design. 2) The different types of automated tests including unit, system, and performance tests. Unit tests are fast and robust while system tests test entire systems. 3) How to set up an automated testing environment including tools for different languages, integrating tests into development workflows, and using build servers to run tests on commits.

1. Introduction to Automated Testing
by Lars Thorup, ZeaLake
May 26th 2011

2. Who is Lars Thorup?
• Software developer
• Coach: Teaching TDD and
automated testing
• Advisor: Assesses software
projects and companies
• Founder and CEO of
BestBrains and ZeaLake

3. Why are we here today?
• How do we do automated testing?
– Write test programs
– Run the tests automatically
• Why should we write tests?
– Enjoy more efficient and predictable course of development
– Find and fix bugs fast
– Prevent bugs from reappearing
– Improve the design of our software

4. Different kinds of automated tests
• Unit tests
– Tests individual pieces of code and the interaction between code blocks
• System tests
– Tests the entire system against the requirements
• Performance tests
– Tests non functional requirements

5. Unit tests or system tests?
• Unit tests are efficient
– Fast to run (a few seconds)
– Robust and predictable
– Easy to write
– Is written together with the code it is testing
• System tests are thorough
– Tests all layers together
– Most efficient way to create a set of tests for existing code

6. Can we automate performance tests?
• Performance tests are brittle
– Tip: create performance trend curves instead

7. So, how do we actually do this?
• IsNumeric
– C#
– Ruby
• TransferFunds
– C++

8. How do we run the tests automatically?
• From our programming environment (IDE)
– Command line: make test
– Right click | Run Tests
• On every commit
– Setup a build server
• Hudson / Jenkins
• TeamCity
– Let the build server run all tests
– Get build notifications
– Keep the build green
• Fixing a broken build has priority over any other development task

9. How can tests help improve our design?
• The software design needs to evolve over time
• A refactoring modifies the design without changing behavior
• Tests ensure that behavior is not accidentally changed
• Without tests, refactoring is scary
– and with no refactoring, the design decays over time
• With tests, we have the courage to refactor
– so we continually keep our design healthy

10. What is good design?
• One element of good design is loose dependencies
– Use interfaces (for static languages)
– Inject dependencies
• Avoid this
• Do this instead

11. Are we wasting valuable developer time writing tests?
• No
• The time spent writing tests is not taken from the time spent
coding
– It is taken from the time otherwise spent on manual testing and debugging
• The cost of a bug keeps increasing until we fix it
• Find bugs fast
– Avoid losing customer confidence
– Free QA to do exploratory testing
so they find the hard-to-find bugs
– Spend less time trying to figure out
what is causing the bug and how to fix it
• Avoid spending time testing again

12. How do we get started?
• When we have a lot of existing code without tests
– Create a set of system tests to get a safety net
• When we are writing new code
– Write unit tests in conjunction with the new code
• Set up a standard test environment for our specific application
– Test data
• Automate the creation of standard testdata in a local database
– External dependencies
• Write stubs to use in the tests

13. What does a real-world project look like?
• wizerize.com
– Web application: C# and JavaScript
– 3 years of production
– 2-4 developers
• 40% test code, 60% production code (in lines of code)
• 71% code coverage of unit tests
• 614 unit tests – run in 1 minute
• 54 system tests – run in 20 minutes
• No functional errors seen by end users in production (yet)

14. Where can I read more?
• http://googletesting.blogspot.com/
• http://testdrivendeveloper.com/
• http://codesheriff.blogspot.com/

15. Which tools do we use?
Environment Tool
C# NUnit
JavaScript qUnit
HTML-based UI WatiN, Selenium
C++ CppUnit, googletest
Python unittest
Ruby Test::Unit
C check, cunit
Java JUnit
... ...