1. The document discusses lessons learned about agile testing and automation. It emphasizes that testing is more than just checking and that both automated and exploratory testing are important. 2. It recommends automating output checking where possible but also using exploratory testing. It also stresses the importance of unit, integration and end-to-end tests as well as code reviews. 3. The document advocates for test-driven development and notes how automated tests can reduce regression testing time. It emphasizes that successful testing requires collaboration between developers, testers and business stakeholders.

1. {
Agile Testing & Automation:
Lessons Learned & Challenges Issued
You have to start somewhere to get
somewhere

2. Things I’ve Learned About Agile Testing
& Automation so far…

3. { {Output Checking
 Automatable
 Assert requirements
 Verify expected behavior
 Happy paths & unhappy
paths
Exploratory Testing
 Not completely
automatable
 A live thought process
 Tests beyond the
requirements
 Will this actually provide
value?
 Will the customer want
to and be able to use it?
1. Testing is more than checking. Automate
as much of the checking as possible.
Automation doesn’t replace exploratory
testing.

4. {
2. Testers are really good at writing
automated acceptance/end-to-end tests.
• Protractor & Web Driver with Behavior Driven
Development (BDD) style
• Test the requirements and workflow through the UI
• Act as living documentation
• Simple, business-user language
• Are we making the right software?
• Touch as much of the real world as possible
• Everyone can learn to write them, and we should
write them, but…

5. { If you put all of your efforts into only automating
end-to-end tests, you will probably end up with a
lot of fragile, broken, unmaintainable tests. Tests
can be technical debt too.
• End-to-end tests validate through the UI so any
time the UI changes, they can break
• They rely on data points that need to be
managed

6. {
• Confirm the code is doing
what we say it will do
• Are we making the
application right?
• Small, light, and less
fragile
• Unit tests don’t touch the
external world and don’t
rely on data points
• Require more technical
skill to write – embrace it!
3. Quality products also need unit and
integration tests.

7. {
4. Code review improves
code quality, reduces bugs,
and leads to writing the
right tests.
• Keep it lightweight, small, frequent, and do it early
• Asynchronous reviews allow people to review when it is
convenient
• Actively share interesting findings or design decisions and
champion quality!
• Treat code reviews as an active state in your workflow, or
they probably won’t happen
• Change the mindset to view code review feedback as a
learning opportunity for both the reviewer and the
reviewee

8. {
• Leads to clearly thinking about what your code needs
to do before you write it
• Reduces waste (code and tests)
• Sets the goal
• It doesn’t just happen. You make it happen.
5. Test Driven Development (TDD) is
hard & it takes commitment.

9. 6. Automated tests reduce regression
testing time, enabling more rapid
delivery and a shortened feedback loop.

10. Regression testing estimate 2012/2013 = 1 week +++

11. Regression testing estimate 2014 = 1 day +/-.
Imagine what we could do in 2015!

12. { • User stories are the basis for tests
• Tests set the goal for code
• The relationship between tests and requirements needs
to be tight
• The communication between testers and programmers
needs to be even tighter
7. It takes a whole team.
Developers + Testers + Business People = Beautiful Tests

13. {
Opportunities
• A consistent approach to test automation to…
• Add value up front and reduce technical debt and maintenance
time down the road
• Build confidence in our releases
• Reducing the feedback loop even more through truly
continuous delivery
• Leveraging Sauce Labs and other tools to automate
cross-browser and mobile testing
• Support and empower teams toward consistency,
quality, and testing

15. {
What’s next?
• Outreach to teams. What do you need to make testing a
part of your development process?
• Automation coverage metrics. How are we doing?
Where do we need to focus our attention?
• Big Team’s Definition of Done (DOD)
• Training Roadmap to support DOD
• Team representation at monthly QA COE meetings
• Working with teams to run Watchfire AppScan and
report on security analysis quarterly
• Following up with teams to champion resolution of
security, eAccessibility, or other defects.

16. Challenge Issued!
 Start changing your mindset right now to
champion quality. Push back when you believe
quality is at risk!
 Don’t wait for the perfect idea, perfect steps, perfect
plan. Start doing something right now!
But what???

17.  Before you write code, write down what your code
should do.

18.  Then write some simple expectations. What would
the user expect at the end of the test?

19.  Fill in the test steps and run your tests.

20.  Practice continuous improvement.
 This is an attitude.
 I have a set of tests and they all pass.
 I bet I can find 2 more tests that will make the product even
better, or
 How could I refactor some of my tests to be more readable,
more efficient, more valuable?
 I don’t have all of my tests automated.
 I can automate 3 new tests each sprint, or
 I’ve been automating 3 per sprint, I bet I can do 5.

21.  Pair with pretty much anyone you can an hold one
another accountable for adding tests.
 Pick one story in each sprint and commit as a team to
pair or public pair programming for that story.
Commit to writing tests first for that story.
 Make code review a part of your process. Do a pull
request. Look at someone else’s pull request. I bet you
will find something to improve upon, and maybe even
a bug.

22.  Fix bugs instead of putting them on the backlog.
Better yet! When you find a bug, write a test for it, and then
fix it or ask someone else to. When the test passes, it’s as if
the bug never existed. You just fixed your bug and added
to your regression test suite.

23.  Review your Team Agreement and Definition of Done.
 Do you have items your team has agreed upon relating
to quality and testing?
 If not, then you’re probably missing some important
discussions.
Remember, it takes a team!