PNSQC Summer Series Webinar with Clyneice Chaney. In this webinar, Clyneice, an expert in the field of software testing and one of our invited speakers for 2017, discusses how to determine and find the most critical tests to execute. In a time when software releases become more and more frequent, we need to be judicious about where we spend our time. Listen in on how we can make our software testing more effective and efficient.

1. PNSQC Summer Webinar Series
Identifying and Executing the Most Critical Tests
Featuring Guest Speaker CLYNEICE CHANEY
@PNSQC
#PNSQC17

2. PNSQC – The organization
• The oldest software quality conference in North America
now in its 35th year.
• Annual conference held in Portland, Oregon.
• Non-profit organization run by volunteers.
• PNSQC is an all volunteer conference that focuses on the
quality practitioner.
• Conference speakers and participants are people
describing their own experiences, not by consultants.
• A range of topics and speakers - everything from
automation and distributed teams to agile, devops and
security. 2

3. House Rules
§ Participants other than the speakers are muted
§ Questions via the GoToWebinar control on the
right side of your screen or through Twitter
@PNSQC
§ Questions may be asked throughout the webinar -
we’ll try to answer them at the end
§ You will receive info on recording and slides after
the webinar 3
Webinar Hashtag: # FindCriticalTests

4. Moderating Today for PNSQC
Philip Lew
PNSQC Board Member
• CEO, XBOSoft
• Relevant specialties and passions
o Software quality process,
evaluation, measurement and
improvement
o Software quality in use / UX
design
o Mobile User Experience and
usability
o Cycling and travel
4

5. 5
• Over 30 years of testing, quality and project management, and process improvement
experience and is innovative professional with demonstrated success in quality.
• Extensive and broad based experience and knowledge in software and systems engineering
• Significant time in career in test organization improvement and organizational change
• Broad experience quality industry as State Baldrige Examiner, ASQ Team Quality Judge
• Frequent speaker at Quality Assurance Institute, International Institute for Software Testing
• Principal engineer for MTRE Corporation. Advises organizations in the creation of winning
mobile strategies to develop best of breed applications.
• Certified Quality Manager, ASQ, TMMi Professional, Professional Project Manager
Clyneice.chaney@linked
Clyneice.chaney@gmail.com
Today’s Webinar
Identifying and Executing the Most Critical Tests
CLYNEICE CHANEY
PNSQC 2017 Invited Speaker

6. Agenda
1. Software Testing : Why Most Important Test Focus?
2. Initiating Most Important Test Focus Strategy
3. Coverage and Most Important Tests Areas
4. Identifying MIT Paths and Data
5. Executing and Managing The MIT Strategy
6

7. Figuring Out What’s Important: Bug
Removals
Formal QA 32 %- 55%No Formal Methods
30 %- 50%
Capers Jones. Software Quality for 1995
7

8. 50%
50%
60%
35%
5%
60% of all code may be
defect free
50 % of all errors
may be found in 5%
of the code
Code
Defects
Figuring Out What’s Important:
Defect Clustering & Test Strategy
8

9. Figuring Out What’s Important:
Different Expectations
• 39 % of consumers – speed is more important that functionality
• 47% expect web page to load 2 seconds or less
• 40% will abandon web page that takes >3 seconds
• Low performance & unreliability has adverse business impacts
• Security and protection of private information are important to customers
9

10. Figuring Out What’s Important:
Customer-Critical Quality Risks
10

11. Answering the Questions?
Most Important Test Method
What do we know
about this
project?
How big is the effort?
If we can’t test everything
what must we test?
How long will effort take?
How to
identify
test to
run?
Are we on track and
schedule?
Software Testing Fundamentals Marnie Hutchinson
11

12. MITS Process
12

13. Planning a Successful Test
Effort
Riddle: Is it possible for test coverage to be
sufficient but the test effort not be?
13

14. Testing Decisions!!
Available Should
Can
14

15. Test Inventory
•Path tests
•Module tests
•Data tests
•End-end system tests
•Functional tests
•Installation tests
•Environment tests
•Configuration tests
•Requirement
•Regression etc.
Complete enumeration of all tests
15

16. Inventory: Risk Analysis
• Provides…
• Prioritization for initial inventory
• Focusing on important items
• Preparation for early sizing, cost, schedule
• Basis for test coverage
• Basis for initial contract on strategy
• Utilizes Risk Index
• Based on probability and severity of failure
• Used through out the test process
• Identifies what needs to be done to know it works
16

17. MITS Rank Average
Inventory Item
T #
Tests ID
AVG
Rank
% Cov
(100)/ Rank
Past
performanc
e
T1 Test
Id/Rank =
Num Run
Bug fix
Information Fix for Error 3 124 7 1.00 100
Arrange payment (Path) 4 3.33 30
Method Payment (path) 4 1.60 62.5
Method Payment (data) 5 1.25 80
Purchase Option (not
available (data) 11 1.60 62.5
Minimum order (data) 12 1.25 80
50 1.00 100
3 1.00 100
12 1.20 83.33333
5 1.00 100
1 1.00 100
Total New
Tests 114 1.38
Existing
application 65 1.00 100
Other
functions 61 1.00 100
All Test suite for ver 1.0 76 1.00 100
202 1.00 100
MITS total All
Tests 316 1.38 72.21007
Software Testing Fundamentals Marnie Hutchinson
17

18. MITS Work Sheet cont.
18
Total
Total new + old tests 315.00 1.31 64 231.74
MITS Totals- All tests 201.60 MITS= 232
Minimum % Test coverage min MITS/T x 100= 201/315 64.00
Proposed % Test coverage (T1/T) X 100= 231/315 74 232
T1= 232
T= 315

19. Designing and
Selecting Tests

20. Functional Paths Overview
• Need a good estimate of size of test effort
• Good estimate need good analysis
• Maps of menu options quick way understand
depth of project
• Logic flow map invaluable tool understand size
• Use Cases provide a means of quickly identifying
paths
• Path analysis--sound foundation to support
minimal testing 20

21. Selecting Testing….Path Analysis
• Use Cases
• MENUS
21

22. Designing Tests
BVA EPDT
State Transition
22

23. Techniques # 1:
Equivalence Partition
• Customer is eligible for a discount if they are at least 18 years
and no older than 56
Invalid partition
Less than or equal
to <=18
Valid partition
18<Range <56
Invalid partition
Greater than 56
18 56
23

24. Techniques # 2:
Boundary Value Analysis
Outside
Inside
Maximum
Minimum
Errors
Typical
24

25. Adding Boundary Value
Analysis• Customer is eligible for a discount if they are at least 18 years
and no older than 56
Invalid partition
Less than or equal
to <=18
Valid partition
18<Range <56
Invalid partition
Greater than 56
17, 18, 19 55, 56, 57

26. Techniques # 3 :
Using Decision Tables
• When different combination of inputs results in
different actions
• Cause- effect table
• Systematic way who complex business rules and
logic
• Testing combinations may be huge, impractical
• Decision tables aid in selection of effective test
cases
26

27. USING DECISION TABLES FOR TEST
DESIGN
Conditions ( are inputs) Rule
1
Rule
2
Rule
3
Rule
4
Rule
5
Rule
6
Rule
7
Rule
8
Q1: number of accidents >N Y Y Y N N N F
Q2: Type of car = {……} Y Y N N Y Y N
Q3: Age of car > M Y N N F Y N Y
Action
Check message refuse to
insure
Return to main menu
X X X
Accept to insure
Select standard rate
Check price
Display information
X
Accept to insure
Select special rate
Check price
Display information
X X X

28. Summary
28
• MITS- Most important Tests
• Risk based approach
• Test inventory
• Test selection using path and data analysis

29. References
• Software Testing Fundamentals Marnie Hutchinson. Wiley
Publishing Co. 2003
• Managing the Testing Process: Practical Tools and Techniques for
Managing Hardware and Software Testing, Third Edition by Rex
Black John Wiley & Sons © 2009 (672 pages) Citation
ISBN:9780470404157
• Software quality for 1995: what works and what doesn’t. For SW
quality Research. Capers Jones report.
• Agile Management- June 2004. the S-Curve Explained. David J.
Anderson
• Test Management Using the S-Curve. WWW.Gate2quality.com
• http”//testersparadise.com/test fundamentals docs/
29

30. Join Us This Fall #PNSQC17
Clyneice’s Invited Talk: Trimming Down your QA
Effort While Maintaining Quality: 2017
www.pnsqc.org
Today’s webinar has been
recorded and will be available
via the PNSQC Youtube
Channel and via slideshare.
@PNSQC
#PNSQC17