Jesse Reed, QA Director at Questar, and Kyle McMeekin discuss how Questar made the switch to qTest and the key factors you should consider in test case management and exploratory testing.

1. Test Case Management
& Exploratory Testing
Kyle McMeekin & Jesse Reed

2. Agenda
- Introductions
- Challenges
- Why the move to qTest
- Results of implementation
- Current best practices of Questar
- Future of Questar testing
- Exploratory Testing
- Q & A

3. Introductions

4. INSERT PHOTO ON TOP
Jesse Reed
Title: Director of QA
Where: Questar Assessment Inc.
How long: IT industry for 15
How long: QA industry for 8
Where from: Minnesota Grown
Things I like to do: Travel
Other hobbies: Entrepreneur

5. INSERT PHOTO ON TOP
Kyle McMeekin
Title: Senior Sales Engineer
Where: QASymphony
How long: 2 years
How long: 5 years
Where from: Maryland
Things I like to do: Travel and Ski
Other hobbies: Video Games

6. Challenges

7. Questar Assessment is an
educational assessment
company focused on
developing and delivering
meaningful measures of
student progress. It is not just
one of the things we do. It’s all
we do.
- Scoring and reporting
- Items and forms
- Test delivery system and
Admin
- Paper and Pencil, Item and
Form Development
- Offshore partners –
MentorMate (Bulgarian)
- Atlassian Suite (Jira/Confluence)
- Microsoft shop mainly using C#
.Net, MS SQL 2012, 2016
- Jenkins, GIT
- MAQS Framework (Selenium
Webdriver,Grid, Appium, .Net
service, MS SQL)
- Jmeter
- Workfront
K-12
Assessment QA Team Technology

8. • Rapidly growing company
• Poor planning
• Lack of process/SDLC/requirements
• Limited automation on legacy systems
• Newly created platform

9. • Overworked and frustrated team
• Limited test cases/No traceability
• No formal QA processes
• Excel and gSheets holders of test cases
• Lack of knowledge of Jira and Confluence

10. Bugs being written on paper and sorted
into piles

11. “We don’t know what we have tested.
I don’t know what I did to create the bug.
Questar QA team
member

12. Why the move to
qTest?

13. INSERT PHOTO ON TOP
Selling the dream of TCM
Business Case
Tell the story
Why
What
How

14. • The main purpose of Questar’s
Quality Assurance and Testing team
is assisting in Questar creating a
Higher Quality Product for our
clients. The team is continuously
searching for ways to improve.
• As the team continues to change it’s
ideologies of how to shift left,
creating efficiencies, and promote
quality throughout, the need for
better tooling is necessary.
• Questar QAT is in an ideal position
to consolidate it’s test management
efforts into one centralized location,
a Test Management Tool
14
Why a Test Management Tool?
Higher
Quality/Lower
Cost
Traceability
Consistency
Robust
reporting
Transparency
Ease of
use/Efficiency

15. Quality Assurance Strategy
• Continuous improvement of Process
• Centralized test case repository
• Traceability of work
• Requirements
• Test cases
• Bugs
Test Planning
•Provide clarity of testing focus
•Assignment of testing duties
•Test coverage
•Transparency
•Reporting
Testing Techniques
•Functional and integration
•System and End-to-end
•Ad hoc/Exploratory
•Regression
•Automation
Technology focused testing
•Utilizing automated strategies
•Continuously increase efficiencies and coverage
Quality
15
Why a Test Management Tool?

16. What tools we compared?
Full featured test
management, integrated into
JIRA with the same look-n-
feel. Testing is integrated
into the project cycle now,
allowing you to track
software quality and make
empowered go/no-go
decisions.
Robust & Easy-To-Use
Test Case Management,
Zepgyr transforms the test
case management
process, helping teams get
faster, more efficient and
effective.
16
Google Sheets a versatile
tool that has be used in
several instances as our
default testing templates.
Flexible, inexpensive, and
easily accessible.

17. Criteria for success
17
Ease of use Functionality Implementation
Reporting
capabilities
Added Value Licensing
Customer
Support
Training Jira Integration
Automation
Integration Future needs
Future features

18. Criteria for success - Results
18

19. Results of
implementation

20. “I absolutely love using qTest, I am able to
manage projects more easily, and reporting is
so much easier.
Questar QA
manager

21. “I love the reports this produces
Questar
CEO

22. “This is a really big win for this
organization.
Questar CTO

23. “It is configurable, and you can manage
it in your own way.
Senior QA Lead - Offshore consultant

24. “Can I get a license for qTest?
Anyone who doesn’t have a license

25. Onboarding
Requirements
Traceability
Reporting and
test status
Repeatability
Effective bug
creation

26. Robust
reporting
Test planning
and Setup
Integration of
Automation
Love the
insight
Visibility to
the value of
the QA team

27. Best practices for
Questar

28. Questar best practices for qTest
Transparency
is Key
• Large software projects define our
projects
• Test plans are created with 2 week
sprints in mind
• Test case approval from peers
• Import of requirements or stories
from Jira
• Linking of test cases to
requirements or stories
• Hierarchical test case structure
• Modular test cases
• Intuitive test case naming

29. Future of testing

30. Questar’s plans for the future
Jenkins
integration of
qTest API
Continuous
Integration
Exploratory, Ad
hoc, Session
based
Custom
Reporting
Efficiency

31. Exploratory Testing

32. PROBLEMS IN TRADITIONAL TESTING
32
Prescribing detailed test steps,
and test planning, can cloud
interactions.
Focus on compressive documentation
blocks cognitive thinking abilities
Customer collaboration in testing
remains in silo or nonexistent.
Comprehensive test plans create a
tunnel vision effect
Individuals and interactions over
processes and tools
Working software over
comprehensive documentation
Customer collaboration over
contract negotiation
Responding to change over
following a plan

33. WHAT IS EXPLORATORY TESTING?
33
1. Parallel test planning, test design, and test
execution
1. Specific yet flexible
1. Aligned towards investigation of potential
opportunities
1. Values depth and attention to detail during testing
1. Fosters knowledge sharing and accountability

34. PARALLEL PLANNING, DESIGNING, EXECUTION
34
Unlike traditional testing techniques, planning, design, and execution happen concurrently,
allowing efficiencies of time as well as flexibility in approach
PLAN
DESIGNEXECUTE
REPORT
PLAN
DESIGN
EXECUTE
REPORT
Traditional Scripted Testing Exploratory Testing

35. Exploratory Testing provides a specific lens through which to perform testing –
whether that be a user person, functionality, criteria (i.e. Localization), etc.
However, it allows testers to use the tool as an end user would, not necessarily as the
product owner envisioned it.
SPECIFIC YET FLEXIBLE
35
I tested the application as the
end user would
I tested the application as
the script prescribed

36. INVESTIGATE OPPORTUNITIES
36
Exploratory testing rewards testers who identify unknown areas of “opportunity”
within the application, as they are essential in maintaining a backlog of future test
charters
Traditional Scripted Testing Exploratory Testing

37. HOW TO STRUCTURE ET: SESSION BASED
37
Session Based Test Management is a popular framework, because it tracks important
data on the testing, while also adding some much needed structure.
• Session charter (includes a mission statement, and
areas to be tested)
• Tester name(s)
• Date and time started
• Task breakdown (the TBS metrics)
• Data files
• Test notes
• Issues
• Bugs
Session Report
More info on SBTM: http://www.satisfice.com/articles/sbtm.pdf

38. UAT WITH EXPLORATORY TESTING
UAT Challenge ET Benefit
UATer’s are unfamiliar test case syntax
and need continual clarification
Allow UATer’s to perform the business
flows they know well without test
scripts
UATer’s are not trained on test case
management, automation tools, etc.
Focus UATer’s time on learning how to
document proper defects, reduce time
to ramp
UATer’s have a shorter attention span
– they are not used to testing 6-8 hrs.
per day
Allow UATer’s to veer off the rails from
time to time and investigate areas of
interest
UATer’s have a short period of time in
which to provide feedback
Ensure that as much of the UATer’s
time as possible is dedicated to ET

39. 39
Keep your Best Testing Talent
IMPROVE JOB SATISFACTION MOVE FROM TESTER TO SME
WHY DO EXPLORATORY TESTING?

40. QASymphony’s qTest Explorer tool
• Applicable on any application;
Including mobile, desktop, and web
• Automatically generate TCs from
your sessions
• Capture environment details
• All sessions stored in central
repository
qTest Explorer
40

41. Thank you