An Introduction to Developer Testing

Testing: 1… 2… 3…
An Introduction to DeveloperTesting
Will Green
will@hotgazpacho.org
http://hotgazpacho.org/
@hotgazpacho

Caveat
I am Not an Expert
http://www.flickr.com/photos/eprater/2375127206/

I am not Kent Beck
Father of Extreme Programming
Pioneer of Software Design Patterns
Originator of CRC Cards
One of original signatories of the Agile Manifesto
Author, JUnit (unit testing framework for Java)
Author of several books, including Test Driven Development: By Example
Co-Author: Refactoring: Improving the Design of Existing Code
http://en.wikipedia.org/wiki/Kent_Beck

I am not Ward Cunningham
Creator of the Wiki
CTO of AboutUs.com
Pioneer of Extreme Programming
Inventor of FIT (Framework for Integrated Tests)
http://en.wikipedia.org/wiki/Ward_Cunningham

I am not Martin Fowler
Chief Scientist at Thoughtworks
Co-Author: Refactoring: Improving the Design of Existing Code
Co-Author: Patterns of Enterprise Application Architecture
Author of several other books
http://en.wikipedia.org/wiki/Martin_Fowler

I am not Robert C. Martin
Author, Clean Code: A Handbook of Agile Software Craftsmanship
Author, Agile Principles, Patterns, and Practices in C#
Author, Agile Software Development: Principles, Patterns and Practices
Creator of FITNesse(combining FIT with a Wiki)
Founder and President of Object Mentor, Inc.
http://en.wikipedia.org/wiki/Robert_Cecil_Martin

I amWill Green
I am an Aspiring Software Craftsman
Co-host of The Iron Languages Podcasthttp://ironlanguages.net/
I Read A LOT
I Practice A LOT
I Learn A LOT
http://hotgazpacho.org/

Ready to drink from the fire hose?
http://www.flickr.com/photos/joevans/4292116723/

Don’t Think “Test”
To-may-to, To-mah-to…
http://www.flickr.com/photos/rhinoneal/2706246863/sizes/m/

http://twitter.com/ampgt/status/21689847806
Testing as Design

Testing as Design
http://twitter.com/jbogard/statuses/19838298235

Testing as Design
http://twitter.com/venkat_s/status/24480762059

Tests aren’t Tests
“Test” doesn’t describe what is going on
Tests are specification
Tests are design
Test suites are an artifact of the design process

TDD: Test Driven Design
Write the code you wish you had
Hard to test == Hard to use
Better to find this out now than 2 months from now when you have to change it
Done right, helps you adhere to the Principles of Object Oriented Design

SOLID
Like a Rock
http://www.flickr.com/photos/gustty/2908229462/

Dependency Management
Dependency Management is
Hard
The single greatest influencer of software quality
The foundation for creating software that exhibits the qualities we desire

Poorly Managed Dependencies
Lead to code that is
Hard to change
Fragile
Not reusable

Well Managed Dependencies
Lead to code that is
Flexible
Robust
Reusable

Qualities of Software
Undesirable
Hard to change
Fragile
Not reusable
Desirable
Easy to change
Robust
Reusable

Dependency Management and SOLID
SOLID is
The foundational set of principles for managing dependencies
Therefor the foundation for creating software that exhibits the qualities we desire

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/SOLID_5F00_6EC97F9C.jpg

What is SOLID?
Single Responsibility Principle
Open Closed Principle
Liskov Substitution Principle
Interface Segregation Principle
Dependency Inversion Principle

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/SingleResponsibilityPrinciple2_5F00_71060858.jpg

Single Responsibility Principle
A class should have one, and only one, reason to change
http://www.objectmentor.com/resources/articles/srp.pdf

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/OpenClosedPrinciple2_5F00_2C596E17.jpg

Open Closed Principle
You should be able to extend the behavior of a class, without modifying it
Classes should be open for extension but closed for modification
http://www.objectmentor.com/resources/articles/ocp.pdf

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/LiskovSubtitutionPrinciple_5F00_52BB5162.jpg

Liskov Substitution Principle
Derived classes must be substitutable for their base classes
http://www.objectmentor.com/resources/articles/lsp.pdf

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/InterfaceSegregationPrinciple_5F00_60216468.jpg

Interface Segregation Principle
Make fine-grained interfaces that are client-specific
http://www.objectmentor.com/resources/articles/isp.pdf

http://www.lostechies.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/derickbailey/DependencyInversionPrinciple_5F00_0278F9E2.jpg

Dependency Inversion Principle
Depend on abstractions, not on specific implementations
http://www.objectmentor.com/resources/articles/dip.pdf

Test Driven Design
The Path to Success: 3 Simple Rules
http://www.flickr.com/photos/stuckincustoms/4070581709/

Rule #1
You are not allowed to write any production code unless it is to make a failing unit test pass.

Rule #2
You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.

Rule #3
You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

The Flow of Test Driven Design
Write a Test
Watch Test Fail
Write Code
Watch Test Pass
Refactor
See step 1
http://gamesfromwithin.com/when-is-it-ok-not-to-tdd

TDD Demo!
Fibonacci Number Generator
Sequence of numbers
First two numbers are 0 and 1
Each subsequent number is the sum of the two previous numbers
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, …

How To Test
http://www.flickr.com/photos/modestchanges/3214701196/

An Example

Test Naming
Method being tested
Conditions
Expected Outcome

Testing Pattern
Arrange, Act, Assert
Separates what is being tested from the setup and verification steps
Makes some Test Smells more obvious
Assertions mixed in with “Act” code.
Test methods that try to test too much at once

Arrange
Arrange all necessary preconditions and inputs

Act
Act on the object or method under test

Assert
Assert that the expected results have occurred

Things to Remember
Triple A
Take Small Steps
Run tests after every change
Tests are Code; maintain them!

Test Doubles
http://www.flickr.com/photos/herbstkind/3791698698/

Mock Repository?
Mock? Stub? What is this?

Test Doubles
Think “stunt double”
Replacement for a Production object for testing purposes
There are many types of Test Doubles, each with its own purpose

Dummy Objects
Passed around but never used
Simply to fill parameter lists

Fake Objects
Have working implementations
Usually just enough for them to work in the context of the specific test
However, not suitable for Production
E.g. In-Memory Database

Stubs
Provide canned answers to SUT
Usually don’t respond to anything more than what was programmed for the context of the test

Spies
Stubs that record information on how they were called
Allows inspection for Assertion
Example
Email Service that records the messages it was sent (without actually sending them)

Mocks
Pre-programmed with expectations
Specify how they should be called by the SUT
Throws an exception if they receive a call they were not expecting
Checked during the Assert to ensure they got all the calls they were expecting

Refactoring
You could do that better…
http://www.flickr.com/photos/29108968@N06/2852038148/

Why Refactoring?
Improve human readability of code
Reduce the complexity in order to improve maintainability
Create a more expressive internal structure
It’s about DESIGN!!

What is Refactoring?
Adisciplinedtechnique for restructuring an existing body of code
Altering code’s internal structure without changing its external behavior
A series of small steps that do very little on their own
Risk of change is mitigated by running automated tests after each step

Refactoring is NOT
An undisciplined technique
Making changes to code that change its external behavior
One giant step
Done without a safety net of automated tests

You Are Not Refactoring
If you are doing any of those things, your are not refactoring.
You’re just changing things willy-nilly.
http://www.flickr.com/photos/brymo/2807243701/

How NOT to Test
Yeah, don’t do that
http://www.flickr.com/photos/bobcatnorth/1199505016/

Test Smells
Things that make you go “that just ain’t right”
Code Smells
Test Smells
Project Smells
Most of what follows is summarized from
xunitpatterns.com/Test%20Smells.html
the book XUnit Test Patterns: Refactoring Test Code

Test Smells – Code Smells
Smells that are found while looking at test code
Obscure Test
Conditional Test Logic
Hard-to-Test Code
Test Code Duplication
Test Logic in Production

Obscure Test
There is trouble understanding what behavior a test is verifying
too much or too little information in the test method
Possible Solutions
Keep tests small and focused
Do not depend on external resources (files, databases, web services, etc.)

Conditional Test Logic
Hard to know exactly what a test is going to do when it really matters
conditional logic to handle when SUT fails to return valid data
Loops to verify contents of collections
Conditionals to verify complex objects
Possible Solutions
Custom Assertion Methods

Hard to Test Code
Code is difficult to test
Highly Coupled Code
Asynchronous Code
Untestable Test Code
Possible Solutions
Reduce coupling by using TDD and adhering to SOLID principles
Use Test Doubles to isolate the SUT

Test Code Duplication
The same code is repeated many times
Cut-and-Paste Code Reuse
Reinventing the Wheel
Possible Solutions
Refactor your test code to create Utility Methods

Test Logic in Production
Production code contains logic that should only be exercised during tests
Conditional logic “For Tests Only”
Test Dependency in Production
Test-Specific Equality
Possible Solutions
Test-specific sub-classes of SUT
Manage Test Dependencies
Custom Assertions

Test Smells – Behavior Smells
Smells that jump out at you while running tests
Assertion Roulette
Erratic Test
Fragile Test
Frequent Debugging
Manual Intervention
Slow Tests

Assertion Roulette
Hard to tell which of several assertions in the same test caused the test to fail
Test tries to verify too much
Assertions with unclear failure messages
Possible Solutions
One logical assertion per test
Include a failure message for the assertion
Use a GUI test runner

Erratic Test
One or more test pass or fail intermittently
Tests depend on each other
Resource Leakage / Scarcity
Shared state
Using different/random values for each test run
Possible Solutions
Use a fresh fixture for each test
Clean up after yourself
Use Database sandboxes or a Fake Database

Fragile Test
A test fails to compile or run when the SUT is changed in ways that do not affect the part the test is exercising
Data Sensitivity
Over-specifying behavior
Context Sensitivity
Possible Solutions
Do not depend on a database
Encapsulate setup behind Creation Methods
Abstract away dependencies (including Time)

Frequent Debugging
Manual debugging is required to determine the cause of most test failures
Caused by
Lack of Defect Localization (missing tests)
Infrequent Test Runs
Possible Solutions
Only create behavior after a failing test case
Runs tests as part of Continuous Integration builds

Manual Intervention
A test requires a person to do some manual action each time it is run, or verify results manually
Lack of attention to automated test fixture setup
Test is not self-checking
Test conditions that are hard to generate programmatically
Possible Solutions
Invest in automating test setup
Use Assertion Methods
Use Test stubs to simulate test conditions

Slow Tests
Tests take a long time to run
Interaction with external resources (Databases, File Systems, Web Services)
Heavy Test fixture rebuilds for every test
Asynchronous Test / Explicit delays in the test
Too many tests
Possible Solutions
Abstract away database, file system, web services
Use an immutable shared fixture
Separate event from the behavior it executes
Run a fast subset more, and the entire suite less

Test Smells – Project Smells
Smells that a project manager can watch out for
Buggy Tests
Developers Not Writing Tests
High Test Maintenance Cost
Production Bugs

Buggy Tests
Bugs are regularly found in automated tests
Fragile Tests
Obscure Tests
Hard to Test Code
Possible Solutions
Learn to write tests properly
Refactor legacy code to make testing easier & more robust
Use TDD

Developers Not Writing Tests
The Usual Excuses
Not Enough Time
Hard to Test Code
Wrong Test Automation Strategy
Possible Solutions
Learn & Practice
writing tests takes less time as you become more familiar with doing it

High Test Maintenance Cost
Too much time is spent maintaining tests
Fragile Tests
Obscure Tests
Hard to Test code
Possible Solutions
Learning good Test Automation
Practicing TDD

Production Bugs
Too many bugs found during formal test or in production
Infrequently Run Tests
Untested Code
Missing Unit Tests
Lost Tests
Possible Solutions
Speed up test suite
Use TDD
Write tests to cover untested code
Ensure all tests are part of some suite that is run regularly

.NET Testing Tools
http://www.flickr.com/photos/booleansplit/2376359338/

Testing Frameworks
NUnit, MBUnit, XUnit.net
Testing frameworks in the xUnit tradition
Assertion Engine with syntactical sugar on top

Support Libraries
Object Factories
NBuilder
Test Doubles / Isolation Frameworks
Hand-rolled mock objects
Rhino Mocks
Moq
NMock
EasyMock.NET
Typemock Isolator
Telerik Just Mock

Test Runners
NUnitcommand line
NUnit GUI
TestDriven.NET & Visual Nunit
Addins for Visual Studio
JetBrainsReSharper or DevExpressCodeRush
Integrates with Visual Studio
test status next to the test method in the editor
JetBrainsTeamCity
Continuous Integration server

Test Guidance
Pair Programming (Dev and QA)
Code Reviews
Automated Tools
Typemock Test Lint

Resources
http://www.flickr.com/photos/44442915@N00/3467438742/

Agile Principles, Patterns, and Practices in C#
Robert C. Martin
Micah Martin

Working Effectively with Legacy Code
Michael Feathers

Refactoring: Improving the Design of Existing Code
Martin Fowler
Kent Beck
John Brant
William Opdyke
Don Roberts

The Art of Unit Testing with Examples in C#
Roy Osherove

Growing Object-Oriented Software, Guided by Tests
Steve Freeman
Nat Pryce

Testing ASP.NET Web Applications
Jeff McWherter
Ben Hall

MSDN Magazine Articles
June 2008 – Patterns in Practice: The Open Closed Principlehttp://msdn.microsoft.com/en-us/magazine/cc546578.aspx
October 2008 – Patterns in Practice: Cohesion and Couplinghttp://msdn.microsoft.com/en-us/magazine/cc947917.aspx
December 2008 – Patterns in Practice: Design for Testabilityhttp://msdn.microsoft.com/en-us/magazine/dd263069.aspx
September 2007 – Unit Testing: Exploring the Continuum of Test Doubleshttp://msdn.microsoft.com/en-us/magazine/cc163358.aspx

Videos
TDD with Kent Beckhttp://www.pragprog.com/screencasts/v-kbtdd/test-driven-development
Test First and Test Driven conversations with Corey Haines and JB Rainsbergerhttp://programmingtour.blogspot.com/2009/07/test-first-and-test-driven-conversation.html
Moving Specificity Towards the Tests with Corey Haines and JB Rainsbergerhttp://programmingtour.blogspot.com/2009/08/moving-specificity-towards-tests-with.html
Codemanship’s Videos with Jason Gormanhttp://www.youtube.com/user/parlezuml
TekPub Concepts serieshttp://tekpub.com/production/concepts

Podcasts
Hanselminutes Podcast 145 - SOLID Principles with Uncle Bob - Robert C. Martinhttp://www.hanselminutes.com/default.aspx?showID=163
Hanselminutes Podcast 150 – Uncle Bob Martin: SOLID: This Time With Feelinghttp://www.hanselminutes.com/default.aspx?showID=168
Hanselminutes Podcast 146 – Test Driven Development is Design – The Last Word on TDD with Scott Bellwarehttp://www.hanselminutes.com/default.aspx?showID=164
Hanselminutes Podcast 31 – Test Drivenhttp://www.hanselminutes.com/default.aspx?showID=42
Coding QA Podcast 41 – iPhone TDDhttp://codingqa.com/index.php?post_id=622464
Software Engineering Radio Podcast 155 - Johannes Link & LasseKoskelaon TDDhttp://www.se-radio.net/2010/02/episode-155-johannes-link-lasse-koskela-on-tdd/

Web Sites
The Principles of Object Oriented Designhttp://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod
xUnitTest Patternshttp://xunitpatterns.com/
Object Mentor Bloghttp://blog.objectmentor.com/
Martin Fowler’s Bloghttp://martinfowler.com/bliki/
Roy Osherove’s Bloghttp://weblogs.asp.net/ROsherove/

An Introduction to Developer Testing

by Will Green on Sep 26, 2010

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 36

Statistics

An Introduction to Developer Testing Presentation Transcript

http://hotgazpacho.org	29
http://localhost	6
http://www.linkedin.com	1