The document discusses the business intelligence (BI) testing process and emphasizes the importance of evaluating and improving after each testing cycle. It outlines steps to evaluate a testing cycle, identify lessons learned, write a postmortem report, and prepare for the next cycle by updating tests and test plans. Key stakeholders like QA, business analysts, and developers should all provide input to improve the testing process.

1. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
1 a software division of
QuerySurge™
Module 8
BI Testing Process
Evaluate & Improve

2. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
2 a software division of
QuerySurge™
Module Outline
− Evaluate
− Improve
− Postmortem
− Next Steps

3. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
3 a software division of
QuerySurge™

4. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
4 a software division of
QuerySurge™
Evaluate
After each testing cycle, it is important to look back and analyze what went right and what went
wrong
• Did we cover all Functional and non-functional requirements?
• How well did the test case execution phase go?
• If automation was utilized, how did the automation scripts run?
• Were there any logistical problems?
• Was information regarding runs clearly disbursed to all stakeholders?
• Do we have a checklist of setup tasks?
• What metrics were compiled?
• Did we provide ROI?
• Determine solutions to be able to improve for the next cycle or iteration

5. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
5 a software division of
QuerySurge™
Improve – Lessons Learned
• It’s highly important to have lessons learned group sessions with the
QA, BA and development teams
• Determine solutions to be able to improve for the next cycle or
iteration
• Development team questions
o Was our code completion estimate accurate?
o What types of defects were discovered and how can we improve
to eliminate or reduce the number?
o Are we coding efficiently in order to reduce maintenance time
and performance constraints?

6. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
6 a software division of
QuerySurge™
Improve – Lessons Learned
Business analyst questions
• What types of issues were discovered in the mapping documents and
requirements?
• How can we minimize these issues in the future to prevent document
revisions during development?

7. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
7 a software division of
QuerySurge™
Improve – Lessons Learned
Quality assurance questions
• What tests worked and which ones had issues?
• How can we improve upon our suite of tests?
• Were there defects that got passed our testing cycle
and how can we reduce this number?
• Was our test coverage acceptable or do we need to
increase coverage?

8. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
8 a software division of
QuerySurge™
Postmortem
• A postmortem report should focus on which aspects of the testing cycle
worked and which did not work.
• It should include input from all team members (QA, BA, Developers, etc.)
• The report should be distributed to all stake holders.
• Key topics:
o Evaluation of the current test cycle
o Requirement and mapping coverage assessment
o Overall testing and defect metrics
o Environment stability assessment
o Areas for improvement

9. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
9 a software division of
QuerySurge™
Next Steps
Start preparing for the next test cycle
• Create additional tests for:
o New development or functionality
o Any gaps discovered during the previous cycle
o Defects that are not covered by an existing test
• Does the test plan need to be updated?
o New high level and detail level schedules will
need to be developed

10. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
10 a software division of
QuerySurge™
Summary
• After each testing cycle, it is important to look back and analyze what went right and what
went wrong.
• Focusing on lessons learned should be apply to all team members who played a role in the
testing process.
• A postmortem report should focus on aspects of the testing cycle that worked and which
did not work.