The document summarizes a presentation by Jay Philips and Jean Ann Harrison on reaching "The Purple Standard" in software testing. The Purple Standard aims for a higher level of quality than the Gold Standard through prevention of defects, efficient testing and process improvement. The presentation defines different types of defects, emphasizes efficient documentation and bug reporting, and recommends tools and standardized procedures to improve testing.

1. Reaching The Purple Standard
STPCon – Fall 2014
@2014 Jay Philips & Jean Ann Harrison

2. Your Presenters
Mobile & Software Test Consultant
Email: yagysyjah@gmail.com
Twitter: @JA_Harrison
LinkedIn Profile:
http://www.linkedin.com/pub/jeanann
-harrison/4/b55/865/
Company: Project Realms & TeamQualityPro
Website: www.projectrealms.com
www.teamqualitypro.com
Email: jay@projectrealms.com
Twitter: @JayPhilips
LinkedIn Profile:
www.linkedin.com/in/jayphilips
@2014 Jay Philips & Jean Ann Harrison

3. Agenda
• Definition of Standard
• Defect types, definitions and solutions
• Documentation
• Bug Reporting
• Tools & Characteristics
@2014 Jay Philips & Jean Ann Harrison

4. The Purple Standard Definition
• The Purple Standard is
Prevention, Efficiency, and
Learning
• The Purple Standard
reaches a level of quality
higher than The Gold
Standard.
• Purpose: to release a
software product with a
higher quality standard
than the last product
released
@2014 Jay Philips & Jean Ann Harrison

5. The Purple Standard
• Achieve true commitment from all stakeholders.
• Achieve Prevention of Defects throughout the
software project.
• Set up of procedures to limit defects and increase
efficient testing, fixing, reviewing.
• Achieve completion of test coverage to meet time
limits of the Software Project schedule.
• Review mistakes for resolutions to be
implemented into the software project process to
prevent future defects and maintain schedule
integrity
@2014 Jay Philips & Jean Ann Harrison

6. Defect Types
There are a variety of defect types
but do you know which ones are
which?
Can you name types of defects?
@2014 Jay Philips & Jean Ann Harrison

7. Requirement Defects
• Poorly written requirements can lead to design defects
and can delay the project.
• Lack of requirements can lead to lack of design creating
functional and performance defects.
• Understood but not documented requirements can lead
to misunderstood design, function and use.
@2014 Jay Philips & Jean Ann Harrison

8. Design Defects
• Blockages of functions
• Inter-dependencies are not completely
understood
• Variables are mislabeled within
components or applications within a
system
• Hidden or unknown resource limitations
are reached
@2014 Jay Philips & Jean Ann Harrison

9. Performance Defects
Timing Defects
• What is the response
from the application
to install, to load a
page, search for data,
perform a function?
• Does the speed or
lack of speed meet
users’ needs?
• Was the need for
speed defined in the
requirements, follow
a standard or is
unknown?
Load/Stress Defects
• Does the application
crash because too
many users are on
the system?
• Crashing occurs due
to limited resources
like memory or
storage?
• Is the CPU load
maxed out
performing other
functions?
@2014 Jay Philips & Jean Ann Harrison

10. Functional Defects
When an application or
object within the
application does not
behave as stated in the
business or design
requirements.
@2014 Jay Philips & Jean Ann Harrison

11. User Experience Defects
• Look - display
• Feel - accessibility
• Performance
• Installations/Downloads
• Trainability - Intuitive
Experience
• Recovery of Unexpected
Behavior
@2014 Jay Philips & Jean Ann Harrison

12. System Integration Defects
• More than one application are part of a
system to accomplish one task and that task
cannot be completed due to a breakdown in
connecting the applications.
@2014 Jay Philips & Jean Ann Harrison

13. Process Defects
• How software project activities
and tasks are carried out
throughout the project can have
defects based on imposing
“what we’ve always done”
without learning from mistakes.
• Lack of processes developed
can create a slow down or dead
stop in a project.
• Procedures which are
ambiguous with no detailed
instructions.
@2014 Jay Philips & Jean Ann Harrison

14. Documentation Defects
• Documentation defects can be
found when testing any written
or pictorial information that
describes, defines, specifies,
reports, or certifies activities,
requirements, procedures, or
results.
• If the documentation is poor,
non-existent, or wrong, it
reflects on the quality of the
product and the vendor.
@2014 Jay Philips & Jean Ann Harrison

15. Code Migration/Environment Defects
• Defects can also occur due to how code is
migrated from one environment to another.
• Example: Patches need to be installed in order
but if they are not could cause functionality
issues.
• While you might think your environments are
exactly the same that is not always true. Just
having a different version of Java can cause
environment issues.
@2014 Jay Philips & Jean Ann Harrison

16. Defect Priority
Severity Priority
Severity is associated with
standards/functionality.
Priority is associated with scheduling.
Severity refers to the seriousness of the
bug on the functionality of the product.
Higher effect on the functionality will lead
to assignment of higher severity to the
bug.
Priority refers to how soon the bug should
be fixed.
Generally, the Quality Assurance Engineer
decides the severity level.
Priority to fix a bug is decided in
consultation with the client/manager.
@2014 Jay Philips & Jean Ann Harrison

17. Does your desk look like this??
@2014 Jay Philips & Jean Ann Harrison

18. DOCUMENTATION ENVIRONMENTS
• Regulated: complying to standards in a
particular industry
– Purpose of Software, Hardware & Interaction
– Proof of Design, Testing and results
• Requirements
• Test Cases, Test Reports,
– Bugs/issues found, fixed, tested, results
• Non-Regulated
– Track design & testing coverage
– Record bugs
@2014 Jay Philips & Jean Ann Harrison

19. TESTING DOCUMENTATION
• Test Plan
• Test Cases
• Test Reports
• Test Summary Report
• Test Case Matrix
• Bug Reports
• Release Notes
@2014 Jay Philips & Jean Ann Harrison

20. Documentation Methods
• Written textural
word processing
document
• Checklist/Spread
sheet document
• Video format
document
• Automating
@2014 Jay Philips & Jean Ann Harrison

21. Elements of an Efficient Test Plan
• Test Plan
– Introduction to include Objectives & Team
– Scope of testing
– Assumptions & Risks
– Test Approach including Automation
– Test Environment including Tools, Hardware etc
– Milestones & Schedule / Deliverables
@2014 Jay Philips & Jean Ann Harrison

22. Efficient Tricks – Test Plan
• Keep test plan documentation
general and brief
• It’s not necessary to document
everything
• Test team – one meeting
dedicated to work out plan,
helps to train whole team
• Meeting should combine test
types developing Test Approach,
assign each category to team
member for design
@2014 Jay Philips & Jean Ann Harrison

23. Efficient Tricks – Test Case Design
• Discussion on pairing up testers in test design
– Testers writing requirements tests
– Highly experienced testers to develop
combination tests & train less experienced
– Schedule the pairs to combine efforts on
Exploratory Testing session
• Do NOT write out all the steps to perform the
test. Write stories regarding test description
@2014 Jay Philips & Jean Ann Harrison

24. Efficient Tricks – Test Execution
• Pair Testing, video tape during test execution
• Combine tests which are related to not only
test specific functionality but test inter-dependencies
• Automate overall expected functionality to
become more efficient between builds and
releases.
@2014 Jay Philips & Jean Ann Harrison

25. Efficient Environments
• Maintain your test
environment – test new
ones between releases
• Use the testing community
for research on tools to
help the testing effort
• Create new set ups of
hardware where
appropriate i.e. mobile labs
@2014 Jay Philips & Jean Ann Harrison

26. Schedules & Deliverables
• Create short term Milestones for Deliverables
• Testers
• Create Milestones for evaluation to maintain
documentation and testing
• Add small contingency milestones
• Make deliverables a single task
• Schedule Bug Triage meetings early in
schedule
@2014 Jay Philips & Jean Ann Harrison

27. Bug/Issues/Problems Reporting
This image is courtesy of CartoonTester: Andy Glover
http://cartoontester.blogspot.com/
@2014 Jay Philips & Jean Ann Harrison

28. Required Elements for a Bug Report?
• Brief one sentence desc
• Full desc to include equipment, software versions
• Steps to reproduce – do not assume any steps
• Expected Results
• Actual results including error messages, screen
shots &/or video capturing bug
• Reproducible – how often (finding intermittent,
how to reproduce…)
@2014 Jay Philips & Jean Ann Harrison

29. Your Toolbox
Utilize the right tools to increase speed to market, improve quality and cost
Requirements
Management
Tools
Test
Management
Test Design Automation &
Performance
Test
Execution
Defect
Management
@2014 Jay Philips & Jean Ann Harrison

30. Elements of efficiency
» Testing assets, manual
and automated, are
stored in a central
repository.
» Standardized testing
tools are utilized to
increase speed to market
and improve productivity
and quality.
» Tooling are integrated to
provide efficiency.
» Test assets are easily
reused to reduce cost.
» Real-time reporting and
metrics are provided for
visibility and decision
making.
» Automatic document
versioning exists to
track changes.
@2014 Jay Philips & Jean Ann Harrison

31. Standard procedures
• Requirements Reviews
– Test requirements for ambiguity, omission
– Get testers involved in requirement analysis
• Design/Engineering Reviews
– Testers should be involved in design discussions
– Testers can start test design if understanding the design
• Code Reviews
– Testers can start to test the system rather than unit or
component level
• Test Design Reviews
– Get development to help review if the test coverage is
efficiently covering the scope of the project
@2014 Jay Philips & Jean Ann Harrison

32. The Purple Standard
@2014 Jay Philips & Jean Ann Harrison

33. Thank You
• Thank you for attending our session. Please make
sure to fill out the feedback forms.
@2014 Jay Philips & Jean Ann Harrison

34. Image References
• Slide 7: Requirement defects
• http://bit.ly/ZMAxOn
• Slide 9 Shopping Cart:
– http://bit.ly/1tIxogl
• Slide 11 User Expérience
– http://bit.ly/10irXYx
• Slide 12 System Integration
– http://bit.ly/1yfqhKq
• Slide 13 Process Defects
– http://bit.ly/1tIppzK
• Slide 14 Documentation Defects
– http://bit.ly/1oqmbPq
• Slide 17 Desk top Papers
– http://bit.ly/1sL0Uyu
• Slide 18 Audit image
– http://bit.ly/1uzCPIR
• Slide 19 Testing Documentation
– http://bit.ly/1uzCHsZ
• Slide 19 Test Case Documentation
– Http://bit.ly/1nxWprG
– Automation Testing
– http://bit.ly/1t8qn7F
• Slide 21 Test Planning
– http://bit.ly/1CQkiwk
• Slide 22 Efficient Tips
– http://bit.ly/1DwyPQd
• Slide 24 Test Execution
– http://bit.ly/1tIpZxE
• Slide 25 Test Environments
– http://bit.ly/ZMClXO
• Slide 33 Feedback
– http://bit.ly/1t8AqJN
@2014 Jay Philips & Jean Ann Harrison