The document discusses best practices for quality assurance (QA) in the software industry. It covers when QA engineers should be involved, how they can contribute, test planning, estimation, case design and execution, defect reporting, data management, and soft skills. The presenter is a project manager with over 7 years experience who will discuss industry standards for QA including requirement analysis, test deliverables, risk assessment, scope, and strategy.
2. Who Am I ?
Nadun De Silva
•Project Manager in profession
•7+ years of IT Industry Experience
•Graduated from Faculty of Applied Science, USJP ‘2010
•Professional Qualifications – PMP, PMI-ACP, CSM, CSPO, SAFe-Agilist
•Worked with Government bodies, Local Private Companies, Fortune
500 companies (USA/ UK)
•Led Teams (co-located, remote, vendors)
Industry Best Practices with SQA Aspects - Nadun De Silva 2
3. Agenda
• Why we test software?
• What if software is not tested?
• When should QA Engineer involve?
• How QA Engineer should contribute?
• Requirement analyzing
• Test planning with QA deliverables
• Effective and realistic estimations
• Test case designing
• Test case execution
• Test data management
• Defect Reporting
• What are the soft skills needed for a proactive QA engineer?
Industry Best Practices with SQA Aspects - Nadun De Silva 3
4. Why we test software?
Industry Best Practices with SQA Aspects - Nadun De Silva 4
5. What if software is not tested?
Industry Best Practices with SQA Aspects - Nadun De Silva 5
6. When should QA Engineer involve?
Industry Best Practices with SQA Aspects - Nadun De Silva 6
7. How QA Engineer should contribute?
Industry Best Practices with SQA Aspects - Nadun De Silva 7
8. Imagination Clarification Testability Determination
Involvement With Requirement
Analysis
Industry Best Practices with SQA Aspects - Nadun De Silva 8
9. Test Planning With QA Deliverables
• Why we have the test plan and what are the test objectives?
Industry Best Practices with SQA Aspects - Nadun De Silva 9
10. Test Planning With … - Assumptions
• Human Resources are allocated before
the starting the testing phase.
• Project schedules are pre-define with
product release dates.
• Applicable Hardware are available
before proceeding to the Testing.
• Required test environments, Databases
and log servers are up and running.
Industry Best Practices with SQA Aspects - Nadun De Silva 10
11. Test Planning With … - QA Risk
• The probability that the negative
event will occur.
• The potential loss or impact
associated with that event.
Industry Best Practices with SQA Aspects - Nadun De Silva 11
12. Test Planning With … - Testing Scope
• Defining boundaries
and limitation
• Responsibilities of each project phase
Industry Best Practices with SQA Aspects - Nadun De Silva 12
13. Test Planning With … - Testing
Strategy
What to test? How to test? When to test? Data
Requirements
Industry Best Practices with SQA Aspects - Nadun De Silva 13
14. Effective And Realistic QA
Estimations
• Familiarizing with requirements.
• Preparing Test plan.
• Consider the number of human resources
• Test environment preparation
• Test data preparation
• KTs on domain, other technical and non
technical trainings.
• Test case preparation and reviews.
• Test case execution
Industry Best Practices with SQA Aspects - Nadun De Silva 14
15. What is a test case?
Test case
Document
Developed by QA
Engineers based on the
Requirement document
Helps to validate features
of the application
Helps to identify defects/bugs
of the application
It has set of activities
Industry Best Practices with SQA Aspects - Nadun De Silva 15
16. Why do we design test cases?
The basic objective of writing test cases is not only to validate the
requirements against the software.
Application Test Case SRS
Validate Validate
Industry Best Practices with SQA Aspects - Nadun De Silva 16
17. What are the elements of a test case?
• Test case ID – (Unique Identifier) and Priority
• Test case Title – (Short description of test case & should be effective to convey the test
case)
• Module and Requirement ID – (Include relevant module name and business rule
number.)
• Test case Summary – (Detailed description of test case & additional information
needed for the test case to execute)
• Pre-condition/Test data – (Any pre-requisite required to execute the test case)
• Test Steps – (Actual step to be followed or executed)
• Excepted Result – (Result which is expected as a normal behavior)
• Test Case Status - (Pass/Fail/Block/Obsolete)
• Actual result(If not passed) – (Result which we actually get after we execute the test
step)
Industry Best Practices with SQA Aspects - Nadun De Silva 17
18. What are the test case types?
• Unit Test cases
• Integration test cases
• System test cases
• Regression test cases
• Smoke test cases
• Alternatives to test cases
Component A
Component B
Component C
New
Enhancements
Build Version Smoke Test
Reject the
Build
Accept the
Build
Passed
Failed
Developer QA Engineer
Industry Best Practices with SQA Aspects - Nadun De Silva 18
19. Test Case Execution Overview
• Test case execution is the one of the major duties of QA engineer and
It can do by manually or automatically
• Pre- requisite for Test case execution,
Well organized and proper test cases
Proper test execution plan
Well understanding about domain
Test environment and valid test data
• Major responsibilities come along with test case execution,
Managing test cases
Reporting execution results and defects
Optimizing plans
Industry Best Practices with SQA Aspects - Nadun De Silva 19
20. What Is Test Case Execution?
Only finding BUGS?
Actually No!!
Industry Best Practices with SQA Aspects - Nadun De Silva 20
21. Why Do We Execute Test Cases
Industry Best Practices with SQA Aspects - Nadun De Silva 21
22. Smoke Test Vs Sanity Test
Industry Best Practices with SQA Aspects - Nadun De Silva 22
23. Best Practices Of Test Execution
• Should have proper test plan
• Should have up to date test cases
• Use analytical risk-based testing strategies.
• Be realistic and use common sense.
• Institute continuous test process
improvement based on lessons learned from
previous projects.
• Have trained test teams.
• Distribute testing work intelligently.
Industry Best Practices with SQA Aspects - Nadun De Silva 23
24. Test Results Reporting
• Why we are reporting….
Result reporting is a mechanism for
presenting to the customer, from
different angles, the state of the
product
Industry Best Practices with SQA Aspects - Nadun De Silva 24
25. Major Attributes Should Consist With
Report
• Report should have enough information.
• Should be well formatted and
grammatically correct.
• Test aspects both quantitatively and
qualitatively.
• Clarity about team efficiencies /
deficiencies.
• Graphical representation using charts,
tables and other mechanism
Industry Best Practices with SQA Aspects - Nadun De Silva 25
26. Importance of Test Result
Management
• Predictions
• Risk analysis
• Defect prevention
• Avoiding defect duplication
Industry Best Practices with SQA Aspects - Nadun De Silva 26
27. Best Practices With Test Data
Management
• Discover and understand the test data
• Extract a subset of production data from
multiple data sources
• Mask or de-identify sensitive test data
• Automate expected and actual result
comparisons
• Refresh test data
• Handle secure data carefully.
Industry Best Practices with SQA Aspects - Nadun De Silva 27
28. What is a Defect?
• A Defect is a deviation from requirements which causes incorrect
results in the software.
Example: Logging into a system using a blank password.
Industry Best Practices with SQA Aspects - Nadun De Silva 28
29. What is a Bug?
• A Bug is a result of a coding or logic error that causes unexpected
results in the software.
Example: When logging into the system, if the system crashes during
login
Industry Best Practices with SQA Aspects - Nadun De Silva 29
30. Why do we report defects/bugs?
• To inform project team about the defects/bugs in the system.
• To avoid customer dissatisfaction due to faulty releases.
• For tracking purposes: to generate various reports, make decisions on
which defects to fix etc
Industry Best Practices with SQA Aspects - Nadun De Silva 30
31. How to report defects/bugs
effectively?
Industry Best Practices with SQA Aspects - Nadun De Silva 31
32. Defect/Bug Life Cycle
OPEN
ASSIGN
IN PROGRESS
REJECTED
DEFERRED
READY TO TEST
VERIFIED
CLOSED
REOPENED
Industry Best Practices with SQA Aspects - Nadun De Silva 32
33. Best Practices when reporting
defects/bugs
• Reproduce defects/bugs
• Test in different test environments
• Think from the users’ perspective
• Predict defects & bugs
• Precise & concise defect reports
• Include helpful information
• Report intermittent defects carefully
Industry Best Practices with SQA Aspects - Nadun De Silva 33
34. Soft Skills Needed For a Proactive QA
Engineer?
Industry Best Practices with SQA Aspects - Nadun De Silva 34
35. Best Practices To Smoothen QA
Projects
• Paying attention to details
• Always prepare well have a plan for the activities
• Set realistic goals and achieve them
• Make quality a habit
• Be proactive rather than reactive
• Think from the end users’ perspective
• Test for quality over quantity
• Improve writing and speaking skills
• Learn from your (and others) mistakes
• Be objective and professional
Industry Best Practices with SQA Aspects - Nadun De Silva 35
36. How To Handle Critical Situations?
•Remain calm and do not panic
•Measure the options available and think about the
impact/consequences of each option
•Take action wisely
Multi-tasking
•Prepare well and plan out all activities
•Prioritize tasks and attend to them
•Always follow up tasks accordingly
•Time management
Industry Best Practices with SQA Aspects - Nadun De Silva 36
37. Decision Making
• Always be unbiased
• Consider all parameters and consequences
• Use data at hand to foresee risks and mitigate
them
• Develop self confidence
• Always evaluate alternatives with risk calculation
Industry Best Practices with SQA Aspects - Nadun De Silva 37
38. People Relationships
• Be a good listener
• Different strokes for different people
• Be a good team member
• Be a good leader
Industry Best Practices with SQA Aspects - Nadun De Silva 38
Most of the time Engineers are unable to build defect-free software. If the development processes were perfect, meaning no defects were produced, testing would not be necessary.
Ex: Developing a unique software is different than manufacturing a box of cereal where they manufactures thousands of identical boxes per day.
Verification must not be isolated to a single phase in the development process. but rather, incorporated into each phase of development.
Initial test scenarios will be generated here, through that can identify the test data and the expected results.
Through this methodology helps to clarify the system requirements and it ensures that requirements are testable.
Determination of the criticality of software quality attributes and the importance of validation should be made at this stage.
Basically to cover all the assure that the software development project objectives are met.
Example:
Late internal releases and human resource constraints
3rd Party application dependencies.
In sufficient data to test the application.
Unplanned Events: Unavailability of engineers due to sickness.
High number of releases due to high volume of defects.
What to test? Functional and Non-Functional requirements
How to test?
Blackbox: Unit, Integration, System (UAT if needed) and Regression
WhiteBox: Unit testing methods such as Junit, nunit and cunit.
Automation testing (if needed)
Tools use to test.
Non Functional Testing: Load, Performance, Stress and Security Testing (If needed)
When to test? What test method to be used according to the each phase (Release)
What are the Data Requirements? Post production (Maintenance) support strategy (if needed)
Test case execution
- Release Execution
- Defect Verification
- Production Deployment support (If needed)
Document which designed by QA engineers based on the SRS or User story
Effective testcases will help the software tester to find defects in the application or the system being tested.
It has set of activities with expected and actual results executed in sequence to validate the feature/ behavior of an application.
The basic objective of writing test cases is
to validate the requirements against the software to ensure all success and failure scenarios are well captured.
Validate application from user perspective
Before start writing a test case QA engineers must have a better understand about Business domain, Business needs and requirements.
Prior to design the test cases figure out all features of application.
Identify the scope and the purpose of the test case.
Ensure that test case covers functionality mention in the requirement.
Test cases should be simple, transparent and easy to understand.
Test cases should be Valid, brief and short.
Test cases should be Traceable.
Test cases should be maintainable.
Test cases should be reviewed.
Smoke testing is the initial testing process verify the critical functionalities of the program is working fine and software is ready/stable for further testing.
Sanity testing is carry out to check whether the bugs reported in previous build are fixed & there is regression introduced due to these fixes i.e. not breaking any previously working functionality. The main aim of Sanity testing to check the planned functionality is working as expected. Instead of doing whole regression testing the Sanity testing is perform.
Test Data Management (TDM) is fundamental to the success of your data strategy. After all, data drives the entire testing procedure. With bad data comes poor testing, results you can't trust and a whole lot of wasted time, money and effort.
Impact: the effect it will have on the module/other modules/system/integrated systems etc.
Severity: the extent to which the defect can affect the software. In other words it defines the impact that a given defect has on the system. Severity can be categorized as major, medium and minor.
Priority: defines the order in which a defect should be resolved. Priority can be categorized as immediate, urgent, high, normal and low.
Try to reproduce defects/bugs several times (if possible) before reporting the defect/bug.
Test in different test environments (QA environment/Pre-production environment/Previous release) and try to reproduce defects/bugs.
Think outside the box from the users’ perspective and test with a test-to-break attitude.
If time permits, do some ad-hoc testing on the functionalities and predict defects & bugs.
Defects/bugs reports should be precise and concise, enabling anyone going through the report reproduce them.
Include helpful information such as logs, queries, screenshots and test data to help reproducing the defects/bugs.
If a defect/bug is occurring intermittently, carefully record the exact steps followed and make sure the defect/bug is always reported.
Smoothen QA Projects:
Paying attention to details
Always prepare well have a plan for the activities
Set realistic goals and achieve them
Make quality a habit
Be proactive rather than reactive
Think from the end users’ perspective
Test for quality over quantity
Improve writing and speaking skills
Learn from your (and others) mistakes
Be objective and professional