The document discusses strategies for improving regression testing efficiency through an 'intelligent execution' approach that prioritizes tests and encourages collaboration between businesses and developers. It emphasizes the importance of communication and involvement in unit testing, as well as the use of tagging and concurrent build functionality in testing processes. Lastly, it suggests leveraging Docker and spot instances for more efficient execution of expensive tests.

1. Achieve Intelligent
Test Execution
Adam Satterfield | Quality Leader, Anthem
Strategies for Streamlining Regression
Testing & Delivering Higher Business Value

2. Get Tickets accelerate.tricentis.com

3. 3
The “Intelligent Execution” Approach
How We Get There: Conversations + Technology =
Efficiency
1
2
3
4 Q&A
How Disconnects Cause Inefficiencies in
Testing
Agenda

4. 4
George Bernard Shaw
“The single biggest problem in
communication is the illusion that it
has taken place.”

5. 5
Often times a disconnect occurs
between the business, leadership and IT
that can cause inefficiencies in testing.

6. 6
Problems With The Traditional Approach
The test automation pyramid is like a mirage. It can lead you wrong if you
trust it too much.
Integration Tests
UI Tests
Unit Tests

7. 7
Achieving Intelligent Execution
By taking an “intelligent execution”
approach, we can be smarter about this
and use whatever automated tests we
want, and prioritize tests within each
category

8. 8
Leveraging the Priority Based Method

9. 9
Intelligent execution and priority tests
sound good, but what are they besides
buzzwords and how do we get there?

10. 10
Don’t Underestimate the Value of Good Conversation
• Business conversations
uncover high business
value tests
• Combined with
technology, they create
efficiencies in test
execution

11. 11
Have Business Conversations Up Front

12. 12
Old Waterfall Habits Die Hard

13. 13
Step 1: Define Business Impact
Ask Big Questions
- “What keeps you up at night?”
- Which functionality that’s part of this release would cause us to
stop doing business?”

14. 14
Step 2: Collaborate with Developers
(Yes, developers are important for
test automation too!!!)
The bottom line: Testers need to
get involved in unit test & build
output reviews

15. 15
How to Get More Involved: Borrow From BDD!
(But make it even more effective
with a priority-based method)
The bottom line: Use the communication
techniques from BDD to get involved in
unit test & build output reviews. Don’t be
afraid to look under the hood.

16. 16
Get Involved at the Unit Level
The true purpose of QA is to provide
information. Being involved at the unit
testing level can help achieve this goal.

17. 17
Gather Information About Probability

18. 18
Look at the Build Output
Why should QA should also be looking at the build locks / build output? They’ll spot things
that developers won’t.

19. 19
Now that we understand how to prioritize some of
our tests, how do we translate that into actionable
steps?

20. 20
Start With Test Tags
• Tag your tests (using Serenity or
Cucumber as examples) to allow you
to pass build parameters
• This allows you to only execute a sub-
set of your tests instead of the entire
test suite

21. 21
Get Threaded
• Jenkins multi-threaded tests: use
concurrent build functionality
from your build server
• Run quick feedback tests with
HBV as soon as code is ready
• Kick off testing using a headless
browser, unit and integration
tests + any necessary smoke tests

22. 22
For Example…
If I want to create a new billing UI:
• Compile code & run unit tests
• If any of those fail, we go back to
development
• Next, run unit & integration tests
• If you’re smart, you put tags in our
tests – i.e. for billing, tag it to run
smoke, billing & report tests (tests
relevant to that build)
• Then start your deploy process.
From there, spin off two
threads…

23. 23
What About
“Expensive” Tests?
• Using Docker & Spot instances for
performance & load tests
• A Dockerfile is a ‘text document
that contains all the commands a
user could call on the command
line to assemble an image.’¹ It
contains all the instructions
needed to build an image
automatically (used by Jenkins).

24. 24
Let’s Recap
We learned that as testers we need to re-educate leadership on the
testing pyramid – specifically, that UI tests are still valuable. We also
learned that in order to ensure we are running the correct test
automation, we are able to more strategically define our test suites.

25. 25
Let’s Recap
We learned some techniques to streamline test case execution by
using tags and build server functionality as well as some information
on docker and AWS spot instances to execute expensive tests more
efficiently.

26. 26
Thank You!

27. 27
?
Ask Your
Questions