Making Test Automation Work in Agile Projects 2012

1
Making Test
Automation Work in
Agile Projects
ØREDEV 2012
Lisa Crispin
With Material from Janet Gregory

2
Introductions: Experience, Goals
§  Form groups:
§  New to automation
§  One to two years experience w/ automation
§  More than two years experience
§  Talk with at least two other people
§  Tell each other your learning goals for today
§  Note the most interesting one you hear

Copyright 2012: Lisa Crispin
3
About me…
Programmer, tester, agile team member

4
Introduction - You

5
Vive la Rebellion!
Successful test automation is an innovation!
•  Let go of traditional roles, reach across them
•  Visualize how tests should look
•  Be smart about when to automate
•  Experiment to overcome obstacles
•  Find/build the right tools

6
Exercise: Your Learning Goals
§  Write one interesting learning goal you heard
from another participant at the start of the
class on a sticky note.
§  Write your number one learning goal for
today on a sticky note.
§  Put the sticky notes on your “learning goals”
sheet
§  Group similar ones together

7
Let’s start by defining “agile”
Agile teams:
q  Deliver business value
frequently
q  at a sustainable pace
q  while adapting to the changing
needs of the business
Source: Elisabeth Hendrickson

8
The key is “sustainable pace”
Technical debt slows us down

High technical debt +
insufﬁcient automation = even
less time

10
Barriers to Test Automation
What’s holding you back?

11
Exercise: Your barriers
Individually and silently – write one barrier
hindering your team per sticky note
Put these on the “impediments” wall chart
for your table group
Talk to your fellow group members – do you
see any patterns?

12
Questions?

13
The Whole-Team Approach
§  Automated tests are code
§  Respecting the tests
§  Collaborating
§  Commitment to quality
§  Return on investment

14
Automated tests are code
public class CalculatorFixture extends
ColumnFixture {
public String startDate;
public String endDate;
public double startBalance;
public double endBalance;
public String irrTarget;
private Calculator calculator = new
Calculator();
private Double irr;

Source: Gojko Adzic, StarEast 2011 keynote

17
Testers are especially good at…
§ Eliciting examples
§ Turning them into tests
§ Ensuring the right testing gets done
§ Exploratory testing

18
Give testers time to do what we do best
How?

20
Experiment: Iteration 1
§  Pair up: one will be tester, one programmer
§  Sit back to back so you face away from each
other
§  Tester gets a drawing which needs to be
replicated
§  Tester tells the programmer what to draw, all at
one time.
§  Programmer draws the shapes based on what
the tester explained.
§  No talking during ‘coding’!
§  Tester “tests” the drawing, reports “bugs” on index
cards
§  Programmer fixes the “bugs”
§  How long did it take? Will the customer be
happy?

21
Experiment: Iteration 2
§  Collaborate!
§  Tester tells programmer what to draw, watches
the programmer draw, answers questions, points
out ‘defects’ for programmer to fix immediately
§  (Don’t show the programmer the drawing, that
makes it too easy, we’re trying to simulate real
coding)
§  How long did it take? Will the customer be
happy?
Thanks to the members of the agile-games group and
Kane Mar for ideas & pictures for this game

22
Ways to collaborate
Pair:
Tester-Coder
Tester-Tester
Any role – any role

23
Team responsibility
Automate all regression tests

24

25
Learn
Experiment
Meaningful
Commitment

Under-commit
Plan less work than you think you can do
Including all test automation

27
Learn to write maintainable tests
§  Get over the “hump of pain”
From Gerard Meszaros’
XUnit Test Patterns

Whole-team software development
28
Create a
user
story
Write
Customer
(Q2)
Tests
Start
thinking
how to
code
Expand
tests –
Story
Tests
Automate
Q2 Tests
Exploratory
testing
TDD
Customer
User
Acceptance
Product owner
Product owner/ Tester
Tester
Tester/Programmer
Programmer
Pair,
“Show
Me”

Exercise
• Think of an experiment to get your
whole team engaged in automating
tests
• Share with your group

30
Getting Over the Hump
§  The test automation pyramid
§  The agile testing quadrants
§  What should be automated
§  What shouldn't
§  Difficult areas

31
Test Automation Pyramid

Test Automation Volcano
From Matt
Barcomb

33
Agile Testing Quadrants

34
Shared Understanding
§  Start with tests
§  Collaborate

35
What Should We Automate?
§  Quadrant 1 tests
§  Unit, component, TDD
§  API, service-level
§  Load, performance, stress
§  Quadrant 3 tests?
§  Leverage automation where useful

36
What Shouldn’t We Automate?
§  Wizard of Oz, prototyping
§  Usability, UAT, ET
§  Tests that will never fail?
§  Assess risk
§  ROI not enough
§  One-off tests

37
Where Should We Be Careful?
§  GUI tests
§  Watch ROI
§  End-to-End tests
§  Push testing down to lowest level
§  Remember the Pyramid

38
Hard to Automate?
§  Legacy code
§  Hard to automate, or just lack of skill?
§  “Working Effectively with Legacy Code” –
Feathers
§  “Strangling” – Fowler, Thomas

39
Exercise: Low-Hanging Fruit

40
Agile Automation Strategy
§  What hurts the most
§  Layered approach
§  Applying agile principles
§  Small chunks/thin slices
§  Smart test design
§  Choosing the right tools

41
What Hurts the Most
§  Use retrospectives
§  Keep an impediment backlog

42
Multi-Layered Approach
Example:
§  Learn TDD at unit level
§  Automate GUI smoke tests

43
Simplicity
§  Make the biggest problem 20% better
§  Understand the problem first
§  Try simplest approach first
§  Work in small chunks, thin slices
§  Incremental & iterative

44
Automate a Slice at a Time
Example: 4-step UI to validate, upload profit
sharing contribution data
•  Thread 1: All four pages with navigation
•  Thread 2: Select year, enter description on page
1, display on page 2, browse and upload file on
page 2
•  Thread 3: Validate data in file, display on page 3
•  Thread 4: Persist data, display ‘success’
message on page 4

45
Thin Slice Example

46
Exercise: Thin Slices
Here’s our user story:
As an internet shopper, I want to create an
account so that I do not have to enter my address
and billing information each time I make a
purchase
Draw a mind map for this feature on a big sheet of
paper
Identify a basic end-to-end slice of functionality
that can be coded, tested, and automated.
If you have time, identify additional slices.

47
Test Design Patterns/Principles
§  Patterns, eg, Page
Object
§  One clear purpose
§  Don’t Repeat Yourself

48
Page Object Pattern
A “page object” is a test object that holds
the details of all the elements on a web
page that might be involved in an
automated test.

49
Page Object Pattern
§  Test object holds all details of one page
§  Hide details and eliminate duplication
Go To ePS Admin Page
Click Link eps admin
Wait Until Page Contains plan administration
Location Should Be ${EPSADMIN URL}

50
Facilitating Exploratory Testing
§  Templates to set up tests
§  Consoles to facilitate visual inspection

51
Demo

52
Iterative Feedback
§  Spike two different approaches
§  Pick one to try for N # of iterations
§  Use retrospectives to evaluate

53
Learn by Doing
§  Courage – don’t be afraid to fail
§  Production code practices for test code
§  Incremental, thin slices
§  Experiment

54
Questions About Automation Strategy?

55
Choosing Tools
§  Team effort
§  Time
§  Requirements
§  Focus on goals, problems, not tools.
§  Experiment

56
Understand the Purpose
§  Who’s using the tests? What for?
§  What’s being automated?
§  Existing tools, environment
§  Who’s doing what for automating?

57
What Fits Your Situation
•  Existing skills
•  Language of application under test
•  Collaboration needs
•  What’s being automated
•  Life span, future use of tests

58
Test Drivers/Frameworks
§  Divide into sub-teams and prototype
solutions
§  Have a “bake-off” showing each prototype

59
Where To Find Tools
§  http://bit.ly/AgileTestTools
§  www.softwareqatest.com/qattls1.html
§  www.testingfaqs.org
§  www.opensourcetesting.org
§  groups.yahoo.com/group/agile-testing
§  Grow your own!

60
Example: One Team’s Tool Choices
•  IntelliJ Idea
•  Jenkins, ant, Maven
•  JUnit
•  FitNesse
•  Canoo WebTest
•  Watir
•  Robot Framework
•  JMeter
•  Selenium 2.0 / WebDriver with Geb framework

61
Exercise: Test Frameworks
•  What format might work for your team’s
tests?
•  Write examples on sticky notes
•  What patterns, frameworks and drivers do
you want to learn more about?
•  Place sticky notes next to impediment
they might help overcome
•  Discuss how this would help solve
problem

62
Making Test Automation Work
§  Time to do it right
§  Experiments, bake-offs
§  Testable architecture
§  Test data
§  Managing tests

63
Time To Do It Right
§  Limit scope, don’t over-commit
§  Write automation task cards
§  Quality must be team goal
§  Long-term, will let you go faster

64
Bake-offs
§  Sub-teams/pairs experiment
§  Time box
§  Pros & cons

65
Testable Architecture
•  Layered architecture
•  eg. UI, business logic, data access
•  Ports and Adapters pattern
•  App can work without UI or database
•  Ports accept outside events
•  Adapters convert for human or automated
users

66
Test Data
§  Avoid database access when possible
§  Setup/Teardown
§  Independent, rerunnable tests
§  Canonical data
§  Refresh before each test run
§  Customizable data for ET
§  Production-like data
§  Get customers to provide example data

67
Managing Automated Tests
§  Tests as Living Documentation
§  Continuous Integration
§  Reporting results
§  Metrics

68
Tests as Living Documentation
§  Understandable
§  Who will really use them?
§  Once passing, must always pass

69
Any Example Can Become a Test

70
Given/Then/When Example
Scenario: Valid name search returns results
GIVEN that Kant is a supervisor with employees
AND Kant has an employee named Smith
WHEN Kant navigates to the employee name
search page
AND enters the value “S”
THEN Kant will see a search result that includes
Smith

71
Continuous integration/testing…
…short feedback loop

72
Keep tests passing
Stop the line to fix problems

73
Test Management Tools
§  Manage tests, code together
§  Some tools have own management
§  What problem are you trying to solve?

74
Exercise: Tests as Documentation
•  Pair up
•  Walk around the room and talk about
what approach may work for your
team
•  BDD?
•  Tabular style?
•  Examples?
•  Something else?
•  How will people in different roles
use them?

75
Key Success Factors
Whole Team
Approach

Thin Slices

Short
Feedback
Loops

Experiments

Speciﬁcation
by Example

76
Exercise: Breaking Barriers

77
Remember
§  It’s a team problem!
§  Tackle automation problems with diversity
§  Try small experiments
§  Start the test automation rebellion with baby
steps

78
Questions? “Aha” Moments?

79
Agile Testing: A Practical Guide for Testers and Agile
Teams
By Lisa Crispin and Janet Gregory
www.agiletester.ca

80
Experiences of Test Automation
Dorothy Graham and Mark Fewster

81
Beautiful Testing: Leading Professionals Reveal How
They Improve Software
Edited by Tim Riley, Adam Goucher
Includes chapter by yours truly

82
Test Patterns
Xunit Test Patterns: Refactoring Test Code
By Gerard Meszaros

Copyright 2008 Janet Gregory, DragonFire
83
Specification by Example
How successful teams deliver the right
software
Gojko Adzic
Case studies from > 50 teams

Copyright 2008 Janet Gregory, DragonFire
84
ATDD by Example
A practical guide to Acceptance Test-Driven
development
Markus Gaertner

85
Agile Test Automation Resources
§ dhemery.com/pdf/
writing_maintainable_automated_acceptance_tests.pdf
§ lisacrispin.com
§ janetgregory.ca
§ gokjo.net
§ exampler.com
§ agile-testing@yahoogroups.com
§ testobsessed.com
§ testingreflections.com
§ www.beer30.org

Making Test Automation Work in Agile Projects 2012

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Making Test Automation Work in Agile Projects 2012

Similar to Making Test Automation Work in Agile Projects 2012 (20)

More from lisacrispin

More from lisacrispin (16)

Recently uploaded

Recently uploaded (20)

Making Test Automation Work in Agile Projects 2012