This document summarizes a presentation on test automation in agile projects. It discusses:
- The importance of a whole-team approach to test automation, with testers, programmers, and others collaborating.
- Strategies for successful test automation, like prioritizing the most painful areas, using a layered approach, and applying agile principles of working in small batches.
- Choosing the right tools and frameworks by understanding needs, experimenting with different options, and making it a team effort.
Belgium Testing Days - Making Test Automation Work in Agile Projectslisacrispin
Slides from tutorial. Note that the most important part of the tutorial is the exercises, and I can't capture that in the slide deck. Please do not use these for public paid courses, I'm tired of our stuff being ripped off for agile testing classes.
Pair Programming, TDD and other impractical thingsMarcello Duarte
"Why should we write our tests first? Isn't that going to slow my development?" "What? Assigning a single task to 2 developers? How is that efficient? What a waste of resources!" "Look, in the perfect world your advises are great, but I have a project to finish here." In this talk Marcello explores efficiency in contrast to effectiveness. He looks into how practices, traditionally accepted as efficient, sometimes turn out to be less effective than a few "impractical" things he has come across.
Belgium Testing Days - Making Test Automation Work in Agile Projectslisacrispin
Slides from tutorial. Note that the most important part of the tutorial is the exercises, and I can't capture that in the slide deck. Please do not use these for public paid courses, I'm tired of our stuff being ripped off for agile testing classes.
Pair Programming, TDD and other impractical thingsMarcello Duarte
"Why should we write our tests first? Isn't that going to slow my development?" "What? Assigning a single task to 2 developers? How is that efficient? What a waste of resources!" "Look, in the perfect world your advises are great, but I have a project to finish here." In this talk Marcello explores efficiency in contrast to effectiveness. He looks into how practices, traditionally accepted as efficient, sometimes turn out to be less effective than a few "impractical" things he has come across.
Discussion about Input and Output of every Scrum Events. Inside about what to inspect and adapt within these events. Entirely based on Scrum Guide and pretty much similar to PSM workshop.
Improving Tests With Object Mothers And Internal DslsChris Richardson
Test code needs to be as clean and as simple as production code. However, when writing tests there is the ever present temptation to not be as disciplined as you should be. As a result, test code quality gradually decays over time and becomes difficult to maintain and brittle. For example, a common problem is bloated and duplicated test fixture logic. Another problem is tests that are written at too low-level, which makes them difficult to understand and change. If you are not careful, you run the risk of your test code falling into disrepair and being ignored, which defeats the purpose of having tests.
In this talk you will learn how to make tests easier to develop and maintain by using a coding style that abstracts away the details and eliminates code duplication. We describe how to simplify test fixtures by designing domain objects with fluent interfaces, and centralizing test object creation in object mothers. You will also learn how to simplify verification logic with custom assertions. We describe how to improve web tests by writing them in terms of test utility methods, instead of calling Selenium RC directly. These utility methods form an internal domain-specific language that hides low-level details, such as mouse and button clicks.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/31W9IZv.
Stephen Klabnik takes a look at some tradeoffs in the design of Rust, and how that makes it more suitable for some kinds of projects than others. In particular, he talks about Rust's "bend the curve" philosophy towards tradeoffs. Filmed at qconlondon.com.
Stephen Klabnik is on the core team of Rust, leads the documentation team, and is an author of "The Rust Programming Language." He is a frequent speaker at conferences and is a prolific open source contributor, previously working on projects such as Ruby and Ruby on Rails.
UXDX London 2018 Nik Crabtree - Enhancing the Processes of Test Driven Develo...UXDXConf
As the Lead Principal Software Engineer in ASOS Technology, Nik Crabtree is responsible for Software and QA Engineering, UI Engineering and Data Engineering across the organisation. His role has a simple brief: come in every day with one aim - make engineering and engineers at ASOS better.
On May 10th, Nik spoke to the UXDX London audience about test, deploy and solving problems fast! He highlights why people should adopt a test-driven-development approach.
Shared understanding, shared ownership and a test-first approach leads to high quality software, trust and high performance.
Learn how Acceptance Test Driven Development (ATDD) provides the process for capturing detailed requirements as acceptance criteria and turn them into as test cases before development begins using Behavior Driven Development (BDD). The BDD approach and Gherkin format is the language used to create easy to understand and actionable scenarios that map from the functional level to the components and units. We will discuss the different approaches to TDD including a realistic approach leveraging BDD to a purest standpoint where TDD use the tests to drive the design of the application. Finally understand how the tools in Visual Studio and Team foundation Server to support BDD such as SpecFlow (Cucumber in .NET), Refactoring tools, and Test Cases in MTM.
Pragmatic Not Dogmatic TDD Agile2012 by Joseph Yoder and Rebecca Wirfs-BrockJoseph Yoder
This presentation challenges the "norm" for TDD. Testing should be an integral part of your daily programming practice. But you don’t always need to derive your code via many test-code-revise-retest cycles to be test-driven. Some find it more natural to outline a related set of tests first, and use those test scenarios to guide them as they write code. Once they’ve completed a “good enough” implementation that supports the test scenarios, they then write those tests and incrementally fix any bugs as they go. As long as you don’t write hundreds of lines of code without any testing, there isn’t a single best way to be Test Driven. There’s a lot to becoming proficient at TDD. Developing automated test suites, refactoring and reworking tests to eliminate duplication, and testing for exceptional conditions, are just a few. Additionally, acceptance tests, smoke tests, integration, performance and load tests support incremental development as well. If all this testing sounds like too much work, well…let’s be practical. Testing shouldn’t be done just for testing’s sake. Instead, the tests you write should give you leverage to confidently change and evolve your code base and validate the requirements of the system. That’s why it is important to know what to test, what not to test, and when to stop testing.
Quality Jam: BDD, TDD and ATDD for the EnterpriseQASymphony
During Quality Jam 2016 I had the privilege of presenting with one of QASymphony's earliest customers, Better Cloud, on how methodologies like BDD, TDD and ATDD scale for the enterprises. Adam Satterfield is the VP of Quality Assurance at Bettercloud and has been in QA for many years; he has taught me a lot about Behavior Driven Development, Test Driven Development, Acceptance Test Driven Development. In the session we share a new way of testing-- what Adam and I believe to be the next generation of testing development.
We know that there are several ways to do testing and we are just showing one new way to do it - If this session doesn't inspire action, hopefully it will at least give you and your team something to think about.
It was back in ‘97 when Brian Foote and I first opined that: while much attention had been focused on high-level software architectural patterns, what is, in effect, the de-facto standard software architecture had seldom been discussed: the Big Ball of Mud. A Big Ball of Mud is haphazardly structured, sprawling, sloppy, duct-tape and bailing wire, spaghetti code jungle. We’ve all seen them. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated. The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition. Programmers with a shred of architectural sensibility shun these quagmires. Only those who are unconcerned about architecture, and, perhaps, are comfortable with the inertia of the day-to-day chore of patching the holes in these failing dikes, are content to work on such systems. Somewhat to our astonishment, since our original statement, no one has ever undertaken to dispute this premise. Still, this approach endures and thrives. Why is this architecture so popular? Is it as bad as it seems, or might it serve as a way-station on the road to more enduring, elegant artifacts? What forces drive good programmers to build ugly systems? Can we avoid this? Should we? How can we make such systems better?
This keynote will examine the paradoxes that underlie Big Balls of Mud, what causes them, and why they are so prominent. What Agile Practices help us avoid or cope with mud? Does Agile practices such as TDD really help minimize mud? What are we doing RIGHT? What Agile Practices contribute to the problem? Encourage mud? Is Mud really the best that Agile can do? Is Agility’s utilitarian focus on process rather than design its secret weapon, or its Achilles heel?
Being agile, with its attention on extensive testing, frequent integration, and focusing on important product features, has proven invaluable to many software teams. When building complex systems it can be all too easy to primarily focus on features and overlook software qualities, specifically those related to software architecture. Time has shown that agile practices are not sufficient to prevent or eliminate technical debt, which can ultimately affect reliability. Many issues arise when there isn’t good validation through tests and constant attention to the architecture and code quality. It is important to recognize what is core to the architecture and the problem at hand while evolving it. If there is not enough attention on the architecture and the code, technical debt will creep in to the point where it can become muddy, making it hard to deliver new features quickly and reliably. Two principles that can help teams deliver more quickly and with confidence is to focus on code quality and delivery size. Small frequent delivery with constant attention to a good codebase is crucial to being able to sustain faster reliable delivery. Practices that can help keep the code clean or prevent it from getting muddier include: Testing, Divide & Conquer, Gentrification, Quarantine, Refactoring, and Craftsmanship. This talk examines various practices and techniques such as Continuous Integration, Continuous Delivery, Continuous Inspection, along with techniques to pay good attention to software quality, all of which enable teams to deliver fast and with confidence.
Discussion about Input and Output of every Scrum Events. Inside about what to inspect and adapt within these events. Entirely based on Scrum Guide and pretty much similar to PSM workshop.
Improving Tests With Object Mothers And Internal DslsChris Richardson
Test code needs to be as clean and as simple as production code. However, when writing tests there is the ever present temptation to not be as disciplined as you should be. As a result, test code quality gradually decays over time and becomes difficult to maintain and brittle. For example, a common problem is bloated and duplicated test fixture logic. Another problem is tests that are written at too low-level, which makes them difficult to understand and change. If you are not careful, you run the risk of your test code falling into disrepair and being ignored, which defeats the purpose of having tests.
In this talk you will learn how to make tests easier to develop and maintain by using a coding style that abstracts away the details and eliminates code duplication. We describe how to simplify test fixtures by designing domain objects with fluent interfaces, and centralizing test object creation in object mothers. You will also learn how to simplify verification logic with custom assertions. We describe how to improve web tests by writing them in terms of test utility methods, instead of calling Selenium RC directly. These utility methods form an internal domain-specific language that hides low-level details, such as mouse and button clicks.
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/31W9IZv.
Stephen Klabnik takes a look at some tradeoffs in the design of Rust, and how that makes it more suitable for some kinds of projects than others. In particular, he talks about Rust's "bend the curve" philosophy towards tradeoffs. Filmed at qconlondon.com.
Stephen Klabnik is on the core team of Rust, leads the documentation team, and is an author of "The Rust Programming Language." He is a frequent speaker at conferences and is a prolific open source contributor, previously working on projects such as Ruby and Ruby on Rails.
UXDX London 2018 Nik Crabtree - Enhancing the Processes of Test Driven Develo...UXDXConf
As the Lead Principal Software Engineer in ASOS Technology, Nik Crabtree is responsible for Software and QA Engineering, UI Engineering and Data Engineering across the organisation. His role has a simple brief: come in every day with one aim - make engineering and engineers at ASOS better.
On May 10th, Nik spoke to the UXDX London audience about test, deploy and solving problems fast! He highlights why people should adopt a test-driven-development approach.
Shared understanding, shared ownership and a test-first approach leads to high quality software, trust and high performance.
Learn how Acceptance Test Driven Development (ATDD) provides the process for capturing detailed requirements as acceptance criteria and turn them into as test cases before development begins using Behavior Driven Development (BDD). The BDD approach and Gherkin format is the language used to create easy to understand and actionable scenarios that map from the functional level to the components and units. We will discuss the different approaches to TDD including a realistic approach leveraging BDD to a purest standpoint where TDD use the tests to drive the design of the application. Finally understand how the tools in Visual Studio and Team foundation Server to support BDD such as SpecFlow (Cucumber in .NET), Refactoring tools, and Test Cases in MTM.
Pragmatic Not Dogmatic TDD Agile2012 by Joseph Yoder and Rebecca Wirfs-BrockJoseph Yoder
This presentation challenges the "norm" for TDD. Testing should be an integral part of your daily programming practice. But you don’t always need to derive your code via many test-code-revise-retest cycles to be test-driven. Some find it more natural to outline a related set of tests first, and use those test scenarios to guide them as they write code. Once they’ve completed a “good enough” implementation that supports the test scenarios, they then write those tests and incrementally fix any bugs as they go. As long as you don’t write hundreds of lines of code without any testing, there isn’t a single best way to be Test Driven. There’s a lot to becoming proficient at TDD. Developing automated test suites, refactoring and reworking tests to eliminate duplication, and testing for exceptional conditions, are just a few. Additionally, acceptance tests, smoke tests, integration, performance and load tests support incremental development as well. If all this testing sounds like too much work, well…let’s be practical. Testing shouldn’t be done just for testing’s sake. Instead, the tests you write should give you leverage to confidently change and evolve your code base and validate the requirements of the system. That’s why it is important to know what to test, what not to test, and when to stop testing.
Quality Jam: BDD, TDD and ATDD for the EnterpriseQASymphony
During Quality Jam 2016 I had the privilege of presenting with one of QASymphony's earliest customers, Better Cloud, on how methodologies like BDD, TDD and ATDD scale for the enterprises. Adam Satterfield is the VP of Quality Assurance at Bettercloud and has been in QA for many years; he has taught me a lot about Behavior Driven Development, Test Driven Development, Acceptance Test Driven Development. In the session we share a new way of testing-- what Adam and I believe to be the next generation of testing development.
We know that there are several ways to do testing and we are just showing one new way to do it - If this session doesn't inspire action, hopefully it will at least give you and your team something to think about.
It was back in ‘97 when Brian Foote and I first opined that: while much attention had been focused on high-level software architectural patterns, what is, in effect, the de-facto standard software architecture had seldom been discussed: the Big Ball of Mud. A Big Ball of Mud is haphazardly structured, sprawling, sloppy, duct-tape and bailing wire, spaghetti code jungle. We’ve all seen them. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated. The overall structure of the system may never have been well defined. If it was, it may have eroded beyond recognition. Programmers with a shred of architectural sensibility shun these quagmires. Only those who are unconcerned about architecture, and, perhaps, are comfortable with the inertia of the day-to-day chore of patching the holes in these failing dikes, are content to work on such systems. Somewhat to our astonishment, since our original statement, no one has ever undertaken to dispute this premise. Still, this approach endures and thrives. Why is this architecture so popular? Is it as bad as it seems, or might it serve as a way-station on the road to more enduring, elegant artifacts? What forces drive good programmers to build ugly systems? Can we avoid this? Should we? How can we make such systems better?
This keynote will examine the paradoxes that underlie Big Balls of Mud, what causes them, and why they are so prominent. What Agile Practices help us avoid or cope with mud? Does Agile practices such as TDD really help minimize mud? What are we doing RIGHT? What Agile Practices contribute to the problem? Encourage mud? Is Mud really the best that Agile can do? Is Agility’s utilitarian focus on process rather than design its secret weapon, or its Achilles heel?
Being agile, with its attention on extensive testing, frequent integration, and focusing on important product features, has proven invaluable to many software teams. When building complex systems it can be all too easy to primarily focus on features and overlook software qualities, specifically those related to software architecture. Time has shown that agile practices are not sufficient to prevent or eliminate technical debt, which can ultimately affect reliability. Many issues arise when there isn’t good validation through tests and constant attention to the architecture and code quality. It is important to recognize what is core to the architecture and the problem at hand while evolving it. If there is not enough attention on the architecture and the code, technical debt will creep in to the point where it can become muddy, making it hard to deliver new features quickly and reliably. Two principles that can help teams deliver more quickly and with confidence is to focus on code quality and delivery size. Small frequent delivery with constant attention to a good codebase is crucial to being able to sustain faster reliable delivery. Practices that can help keep the code clean or prevent it from getting muddier include: Testing, Divide & Conquer, Gentrification, Quarantine, Refactoring, and Craftsmanship. This talk examines various practices and techniques such as Continuous Integration, Continuous Delivery, Continuous Inspection, along with techniques to pay good attention to software quality, all of which enable teams to deliver fast and with confidence.
This episode takes what you have learned about testing and static code analysis to uncover the hidden costs of development that we all know about, but rarely confront. Error-prone tasks, lack of actionable data, waiting for resources, and unaccounted “bug fix crowdsourcing” are just a few of the sources for latency in software projects.
Agile Anti-Patterns. Yes your agile projects can and will fail too.Sander Hoogendoorn
This is the slide deck for a smooth presentation on agile and agile anti-patterns I did recently at several international conferences, including GIDS (Bangalore), ACCU (Oxford), Camp Digital (Manchester), Agile Open Holland (Dieren) and Jazoon (Zurich).
Scrum is incomplete. It's a framework. It's gaps become blatantly visible when trying to map the Agile Principles. Extreme Programming can fill these gaps.
The slides of the Global Day of Coderetreat Munich 2018 I facilitated and we organized in the context of our Software Craft Meetup Munich on 17.11.2018.
Do your unit tests feel like a chore to maintain? Are they sometimes useful but come with a high maintenance cost? Do you secretly believe you’d work faster without them there? A great test suite should help speed up your development process, not slow you down. This talk highlights some techniques you can use to improve the front-end unit tests you write.
Exploring Requirements for Shared Understandinglisacrispin
Janet Gregory and Lisa Crispin explain techniques for teams to build shared understanding across all roles of the features and stories they are building.
Get testing bottlenecks out of your pipelineslisacrispin
When teams move towards continuous delivery and deployment, how do they manage the manual stages in their deployment pipeline? This talk gives some techniques to visualize pipelines, identify bottlenecks, find ways to remove them.
Thinking Outside the Box: Cognitive bias and testinglisacrispin
Cognitive biases can get in the way of effective testing. How can we compensate for them and do more "outside the box" thinking? Presented at Motrix Ministry of Testing Cork. Meetup April 15 2020
The Whole Team Approach to Quality in Continuous Deliverylisacrispin
Lisa shares her teams' experiences with making a team commitment to quality and learning ways to build it in and fit all testing activities into continuous delivery.
Advanced Topics in Agile Tsting: Focus on Automationlisacrispin
Slide deck for workshop facilitated by Lisa Crispin and Janet Gregory at Quality in Agile Vancouver 2015. Outcomes from the workshop including all the mind maps will appear eventually on lisacrispin.com.
1. Making Test
Automation Work in
Agile Projects
Agile Testing Days 2012
Lisa Crispin
With Material from Janet Gregory
1
2. Introductions: Experience, Goals
§ Form groups:
§ New to automation
§ One to two years experience w/ automation
§ More than two years experience
§ Talk with at least two other people
§ Tell each other your learning goals for today
§ Note the most interesting one you hear
2
5. Takeaways
Successful test automation is an innovation!
• Let go of traditional roles, reach across them
• Visualize how tests should look
• Be smart about when to automate
• Experiment to overcome obstacles
• Find/build the right tools
We won’t do any hands-on automation, but will demo
some examples
5
Copyright 2012: Lisa Crispin
6. Exercise: Your Learning Goals
§ Write one interesting learning goal you heard
from another participant at the start of the
class on a sticky note.
§ Write your number one learning goal for
today on a sticky note.
§ Put the sticky notes on your table group
“learning goals” sheet
§ Group similar ones together
6
7. Let’s start by defining “agile”
Agile teams:
q Deliver business value
frequently
q at a sustainable pace
q while adapting to the changing
needs of the business
Source: Elisabeth Hendrickson
7
Copyright 2012: Lisa Crispin
8. The key is “sustainable pace”
Technical debt slows us down
8
Copyright 2012: Lisa Crispin
10. High technical debt +
insufficient automation =
even less time
Copyright 2012: Lisa Crispin
11. Barriers to Test Automation
What’s holding you back?
11
Copyright 2012: Lisa Crispin
12. Exercise: Your barriers
Individually and silently – write one barrier
hindering your team per sticky note
Put these on the “impediments” wall chart
for your table group
Work with your group to identify patterns,
group similar items
12
Copyright 2012: Lisa Crispin
14. The Whole-Team Approach
§ Automated tests are code
§ Respecting the tests
§ Collaborating
§ Commitment to quality
§ Return on investment
14
Copyright 2012: Lisa Crispin
15. Automated tests are code
public class CalculatorFixture extends
ColumnFixture {
public String startDate;
public String endDate;
public double startBalance;
public double endBalance;
public String irrTarget;
private Calculator calculator = new
Calculator();
private Double irr;
15
Copyright 2012: Lisa Crispin
18. Testers are especially good at…
§ Eliciting examples
§ Turning them into tests
§ Ensuring the right testing gets done
§ Exploratory testing
18
Copyright 2012: Lisa Crispin
21. Experiment: Iteration 1
§ Pair up: one will be tester, one programmer
§ Sit back to back so you face away from each other
§ Tester: You have a drawing which you need to
instruct your developer to draw
§ Programmer: Remain silent while working, only do
what you’re instructed.
§ No talking!
§ No peeking!
When “done”:
§ Programmer – pass your drawing to the tester
§ Tester – note problems on separate card/paperWill
the customer be happy?
21
Copyright 2012: Lisa Crispin
22. Experiment: Iteration 2
§ Now, turn to face each other.
§ Tester:
§ Still don’t show your picture!
§ Tell the programmer what to draw, but -
§ Watch what the programmer draws, and point
out any mistakes immediately
§ Programmer:
§ Still no peeking!
§ Ask questions, ask for feedback, make
corrections immediately
Thanks to the members of the agile-games group and
Kane Mar for ideas & pictures for this game
22
Copyright 2012: Lisa Crispin
23. Ways to collaborate
Pair:
Tester-Coder
Tester-Tester
Any role – any role
23
Copyright 2012: Lisa Crispin
28. Under-commit
Plan less work than you think you can do
Including all test automation
Copyright 2012: Lisa Crispin
29. Learn to write maintainable tests
§ Get over the “hump of pain”
From Gerard Meszaros’
XUnit Test Patterns
29
Copyright 2012: Lisa Crispin
30. Whole-team software development
Create a Expand
user tests –
story Story
Tests
Automate
Write Q2 Tests Pair,
Customer Start “Show
(Q2) thinking Me”
Tests how to
code
TDD
Exploratory
testing
Product owner
Product owner/ Tester
Tester
Tester/Programmer
Programmer Customer
User
Acceptance
30
31. Exercise
• Think of an experiment to get your
whole team engaged in automating
tests
• Share with your table group
• Pick two to share with the class
Copyright 2012: Lisa Crispin
32. Getting Over the Hump
§ The test automation pyramid
§ The agile testing quadrants
§ What should be automated
§ What shouldn't
§ Difficult areas
32
Copyright 2012: Lisa Crispin
37. What Should We Automate?
§ Quadrant 1 tests
§ Unit, component, TDD
§ Quadrant 2 tests
§ API, service-level
§ Quadrant 4 tests
§ Load, performance, stress
§ Quadrant 3 tests?
§ Leverage automation where useful
37
Copyright 2012: Lisa Crispin
38. What Shouldn’t We Automate?
§ Quadrant 2 tests
§ Wizard of Oz, prototyping
§ Quadrant 3 tests
§ Usability, UAT, ET
§ Tests that will never fail?
§ Assess risk
§ ROI not enough
§ One-off tests
38
Copyright 2012: Lisa Crispin
39. Where Should We Be Careful?
§ GUI tests
§ Watch ROI
§ End-to-End tests
§ Push testing down to lowest level
§ Remember the Pyramid
39
Copyright 2012: Lisa Crispin
40. Hard to Automate?
§ Legacy code
§ Hard to automate, or just lack of skill?
§ “Working Effectively with Legacy Code” –
Feathers
§ “Strangling” – Fowler, Thomas
40
Copyright 2012: Lisa Crispin
42. Agile Automation Strategy
§ What hurts the most
§ Layered approach
§ Applying agile principles
§ Small chunks/thin slices
§ Smart test design
§ Choosing the right tools
42
Copyright 2012: Lisa Crispin
43. What Hurts the Most
§ Use retrospectives
§ Keep an impediment backlog
43
Copyright 2012: Lisa Crispin
45. Simplicity
§ Address one or two needs at a time
§ Understand the problem first
§ Try simplest approach first
§ Work in small chunks, thin slices
§ Incremental & iterative
45
Copyright 2012: Lisa Crispin
46. Automate a Slice at a Time
Example: 4-step UI to validate, upload profit
sharing contribution data
• Thread 1: All four pages with navigation
• Thread 2: Select year, enter description on page
1, display on page 2, browse and upload file on
page 2
• Thread 3: Validate data in file, display on page 3
• Thread 4: Persist data, display ‘success’
message on page 4
46
Copyright 2012: Lisa Crispin
48. Exercise: Thin Slices
Here’s our user story (or theme):
As an internet shopper, I want to create an
account so that I do not have to enter my address
and billing information each time I make a
purchase
Draw a mind map for this feature on a big sheet of
paper
Identify a basic end-to-end slice of functionality
that can be coded, tested, and automated.
If you have time, identify additional slices.
48
Copyright 2012: Lisa Crispin
50. Page Object Pattern
A “page object” is a test object that holds
the details of all the elements on a web
page that might be involved in an
automated test.
50
Copyright 2012: Lisa Crispin
51. More on the Page Object Pattern
http://www.beer30.org/?p=54
51
Copyright 2012: Lisa Crispin
53. Iterative Feedback
§ Spike two different approaches
§ Pick one to try for N # of iterations
§ Use retrospectives to evaluate
53
Copyright 2012: Lisa Crispin
54. Learn by Doing
§ Courage – don’t be afraid to fail
§ Production code practices for test code
§ Incremental, thin slices
§ Experiment
54
Copyright 2012: Lisa Crispin
56. Choosing Tools, Frameworks, Drivers
§ Team effort
§ Time
§ Requirements
§ Focus on goals, problems, not tools.
§ Experiment
56
Copyright 2012: Lisa Crispin
57. Understand the Purpose
§ Who’s using the tests? What for?
§ What’s being automated?
§ Existing tools, environment
§ Who’s doing what for automating?
57
Copyright 2012: Lisa Crispin
58. What Fits Your Situation
• Existing skills
• Language of application under test
• Collaboration needs
• What’s being automated
• Life span, future use of tests
58
Copyright 2012: Lisa Crispin
59. Test Drivers/Frameworks
§ Divide into sub-teams and prototype
solutions
§ Have a “bake-off” showing each prototype
59
Copyright 2012: Lisa Crispin
60. Where To Find Tools
§ www.softwareqatest.com/qattls1.html
§ http://bit.ly/AgileTestTools
§ www.testingfaqs.org
§ www.opensourcetesting.org
§ groups.yahoo.com/group/agile-testing
§ Grow your own!
60
Copyright 2012: Lisa Crispin
61. Example: One Team’s Tool Choices
• IntelliJ Idea
• Jenkins, ant, Maven
• JUnit
• FitNesse
• Canoo WebTest
• Watir
• Robot Framework
• JMeter
• Selenium 2.0 / WebDriver with Geb framework
61
Copyright 2012: Lisa Crispin
62. Exercise: Test Frameworks
What format would work for your team? EG,
BDD style? Tables? Assertions? Scenarios?
Think of the test format that might work for
your team. Write examples on sticky notes.
Write down any particular types of test
patterns, frameworks and driverss you want
to learn more about.
and place these sticky notes next to an
impediment it might help overcome..
Discuss with group how that tool might help
solve the problem. Copyright 2012: Lisa Crispin
62
63. Making Test Automation Work
§ Time to do it right
§ Experiments, bake-offs
§ Testable architecture
§ Test data
§ Managing tests
63
Copyright 2012: Lisa Crispin
64. Time To Do It Right
§ Limit scope, don’t over-commit
§ Write automation task cards
§ Plan test code maintenance/ refactoring time
§ Quality must be team goal
§ Long-term, will let you go faster
64
Copyright 2012: Lisa Crispin
65. Bake-offs
§ Have sub-teams/pairs experiment with
prototypes
§ Budget lots of time for learning,
experimenting
§ Look at pros & cons
65
Copyright 2012: Lisa Crispin
66. Testable Architecture
• Layered architecture
• eg. UI, business logic, data access
• Ports and Adapters pattern
• App can work without UI or database
• Ports accept outside events
• Adapters convert for human or automated
users
66
Copyright 2012: Lisa Crispin
67. Test Data
§ Avoid database access when possible
§ Setup/Teardown
§ Independent, rerunnable tests
§ Canonical data
§ Refresh before each test run
§ Customizable data for ET
§ Production-like data
§ Get customers to provide example data
67
Copyright 2012: Lisa Crispin
68. Managing Automated Tests
§ Tests as Living Documentation
§ Continuous Integration
§ Reporting results
§ Metrics
68
Copyright 2012: Lisa Crispin
69. Tests as Living Documentation
§ Understandable
§ Who will really use them?
§ Once passing, must always pass
69
Copyright 2012: Lisa Crispin
70. Any Example Can Become a Test
70
Copyright 2012: Lisa Crispin
71. Given/Then/When Example
Scenario: Valid name search returns results
GIVEN that Kant is a supervisor with employees
AND Kant has an employee named Smith
WHEN Kant navigates to the employee name
search page
AND enters the value “S”
THEN Kant will see a search result that includes
Smith
71
Copyright 2012: Lisa Crispin
74. Test Management Tools
§ Manage tests, code together
§ Some tools have own management
§ What problem are you trying to solve?
74
Copyright 2012: Lisa Crispin
80. Agile Testing: A Practical Guide for Testers and Agile
Teams
By Lisa Crispin and Janet Gregory
www.agiletester.ca
Copyright 2012: Lisa Crispin
80
81. Experiences of Test Automation
Dorothy Graham and Mark Fewster
Copyright 2012: Lisa Crispin
81
82. Beautiful Testing: Leading Professionals Reveal How
They Improve Software
Edited by Tim Riley, Adam Goucher
Includes chapter by yours truly
Copyright 2012: Lisa Crispin
82
83. Test Patterns
Xunit Test Patterns: Refactoring Test Code
By Gerard Meszaros
Copyright 2012: Lisa Crispin
83
84. Specification by Example
How successful teams deliver the right
software
Gojko Adzic
Case studies from > 50 teams
84
Copyright 2012: Lisa Crispin
Copyright 2008 Janet Gregory, DragonFire