Fundamentals of BI Report Testing - Module 6

1. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
1 a software division of
QuerySurge™
Module 6
BI Testing Process
Design

2. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
2 a software division of
QuerySurge™
Module Outline
− Designing Test Cases
− Data Validation and Accuracy Test Cases
− Functional Test Cases
− Performance Test Cases

3. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
3 a software division of
QuerySurge™

4. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
4 a software division of
QuerySurge™
Designing Test Cases
In the Design phase, test cases are authored and reviewed. Test cases serve as the detailed instructions for executing tests,
evaluating report functionalities, and ensuring that the report meets its defined requirements. Components of a test case
should include:
• Test Case ID: A unique identifier for the test case, often in the form of a number or code.
• Test Case Name: As discussed during the planning phase, test case names should follow a clear and concise naming
convention.
• Test Case Description: A succinct description of the test scenario, outlining what will be tested.
• Preconditions: Any specific conditions or prerequisites that need to be in place before the test can be executed. This
ensures that the test environment is set up correctly.
• Test Steps: A detailed sequence of steps that the tester should follow to execute the test. Each step should be explicit and
unambiguous.
• Expected Results: A description of the expected outcome or behavior of the report after executing the test steps. This
includes expected data values, report visuals, or system responses.
• Pass/Fail Criteria: Criteria that determines whether the test case has passed or failed. These criteria should be specific and
based on the expected results.

5. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
5 a software division of
QuerySurge™
Data Validation and Accuracy Test Case Design
Designing a data validation and accuracy test case against a BI report involves several steps to ensure thorough and
effective testing. As we discussed in previous modules, mapping documentation will be utilized to account for any of
transformation in the data before it is displayed in the BI report.
• Identify the data set or data points you will use to validate the report. Be sure to fully understand the
relationship between the underlying data source and the data displayed in the BI Report visualization.
• Outline the steps to be taken during the test. This includes actions like running the report, actioning prompts or
interacting with the data. Be clear and specific in describing each step.
• Instructions on how to extract data from the underlying data source, including transformations, should also be
included in the test case steps.
• Prepare the test data, ensuring it aligns with the expected results. This might involve
creating or modifying data records to fit the scenario.
• Establish pass/fail criteria that indicate whether the test case was successful or not.
This could be a specific comparison of actual vs. expected results or a set of defined
rules.

6. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
6 a software division of
QuerySurge™
Data Validation and Accuracy Test Case Design
An example of a manual test case to validate a BI Report against the underlying source data.
Test Case ID 1.1
Test Case Title SalesDM_MonthlySalesReport_SalesChart_DataValidation
Description To verify that the Monthly Sales Report displays complete and accurate data
Preconditions
- The Monthly Sales Report is connected to the Sales DataMart.
- The Sales data source containing test data for the entire year is available
Pass/Fail
Criteria
The test case passes if all expected results are met. It fails if any data is missing/inaccurate in the report or if the total sales does not
match the source data.
Test Steps Expected Results
1. Launch the Monthly Sales Report. The Monthly Sales Report opens without errors.
2. Verify that all columns in the report table are populated with data for the expected
time range (All of 2023)
All columns in the report display complete and non-null data
3. Check for any missing or incomplete data points, such as blank cells or null values. There are no missing or incomplete data points in the report.
4. Validate that the total sales for the year matches the sum of individual sales for 2023 in
the source Datamart (Source SQL: Select Total FROM SalesDM.Sales)
The total sales in the report matches the sum of individual sales for 2023 from
the source datamart.
5. Cross-check a random sample of 100 sales records in the report with the corresponding
records in the sales DataMart to ensure data accuracy. (Source SQL: Select Total FROM
SalesDM.Sales where SaleID in (1,2,3…100)
The sample of 100 randomly chosen sales records in the report corresponds
exactly to the records in the source sales DataMart.

7. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
7 a software division of
QuerySurge™
Automating your Data Validation and Accuracy Test Cases
Manual testing can be effective in some scenarios, but it has limitations in terms of time, accuracy, scalability, and
coverage. Automation addresses these limitations, providing faster, more accurate, and comprehensive data validation.
• Automated tests can quickly validate large datasets and reports, saving time and effort compared to manual testing.
• Automated tests can be run repeatedly without human intervention, ensuring consistency and reproducibility in the
testing process.
• Automation reduces the risk of human error in data validation, leading to more accurate results.
• Automated tests can cover a wide range of test scenarios and data variations, improving test coverage.

8. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
8 a software division of
QuerySurge™
Automation Tools – Data Validation
QuerySurge is a fully automated end-to-end solution for testing from data sources (databases, flat files, XML, Excel, JSON,
etc.) through the Big Data lakes to the Data Warehouses/Data Marts to the BI and Analytics reports.
QuerySurge’s BI Tester add-on provides a successful approach to solving the problem of testing the data embedded in
these BI tools.
We will take a deep-dive into how you can
configure and execute automated tests
against your BI Reports using QuerySurge
during the final modules and lab exercises of
this course.

9. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
9 a software division of
QuerySurge™
Functional Test Case Design
Effective functional test cases involve in-depth understanding of requirements, diverse testing scenarios, and validation of
report functionality including filtering, sorting, and drill-downs.
• Clear Objectives: Define specific functional objectives, such as report functionality and data accuracy.
• Detailed Requirements: Thoroughly understand report requirements and user expectations.
• Representative Test Data: Use real or realistic data sets that reflect actual use cases.
• Test Scenarios: Create diverse test scenarios that cover various report functionalities.
• Filter and Parameter Testing: Test various combinations of filters and parameters.
• User Interactivity Testing: Evaluate user interactions like drill-downs and sorting.
• Edge Case Coverage: Test boundary conditions and outliers to assess report robustness.
• Documentation: Maintain comprehensive test documentation, including test cases, logs, and expected
results.
• Usability Assessment: Consider user experience aspects, such as report layout and ease of navigation.

10. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
10 a software division of
QuerySurge™
Filtering and Sorting Test Case
An example of a manual test case to verify filtering and sorting in a BI Report
Test Case ID 1.2
Test Case Title MonthlySalesReport_SalesChart_Filtering_Sorting
Description To verify that the Monthly Sales Report filtering and sorting functions properly
Preconditions
- The Monthly Sales Report is connected to the Sales DataMart.
- The Sales data source containing test data is available for the entire year
Pass/Fail
Criteria
The test case passes if all expected results are met. It fails if any data is missing/inaccurate in the report or if the filtering and sorting
does not display date in the correct order
Test Steps Expected Results
1. Launch the Monthly Sales Report. The Monthly Sales Report opens without errors.
2. Apply the "January 2023" filter to select data for the specified month The "January 2023" filter is successfully applied.
3. Check for any missing or incomplete data points, such as blank cells or null values. here are no missing or incomplete data points in the report.
4. Un-Apply the “January 2023” filter The “January 2023” filter is removed, and all data is once again displayed.
5. Sort the visual by month in ascending then descending order
Data is displayed in the proper order when toggling between ascending and
descending.

11. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
11 a software division of
QuerySurge™
Performance Test Case Design
Designing performance test cases for BI reports involves meticulous planning, realistic data, and a focus on scalability to
ensure reports perform optimally under real-world conditions.
• Clear Objectives: Define specific performance objectives, such as acceptable response times and system resource limits.
• Realistic Test Data: Use realistic, production-like data sets to ensure accurate performance assessment.
• Scalability Testing: Evaluate how the report scales with increased concurrent users or data volume.
• Use Load Testing Tools: Leverage load testing tools like JMeter or LoadRunner to simulate user activity.
• Gradual Load Increase: Start with a low user load and gradually increase to assess performance thresholds.
• User Scenarios: Create scenarios that mimic real user interactions, including filtering, sorting, and navigation.
• Resource Monitoring: Continuously monitor CPU, memory, and network usage during testing.
• Documentation: Maintain detailed logs and reports for analysis and reporting.
• Optimization Recommendations: If issues are identified, provide recommendations for performance optimization.
• Iterative Testing: Perform performance testing at multiple stages of report development.

12. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
12 a software division of
QuerySurge™
Performance Test Case
An example of a BI Report performance test case using a performance tool like JMeter
Test Case ID 1.2
Test Case Title MonthlySalesReport_SalesChart_Performance
Description To evaluate the performance of the Monthly Sales Dashboard under a high load of concurrent users.
Preconditions
- The Monthly Sales Dashboard is deployed and accessible.
- Test data representative of a large dataset is available.
- Load testing tools, such as JMeter, are set up.
Pass/Fail
Criteria
The test case passes if response times, and resource utilization remain within acceptable thresholds as the user load increases.
The test case fails if response times become unacceptably slow, errors occur, or resource utilization exceeds safe limits.
Test Steps Expected Results
1. Simulate concurrent access by configuring a load testing tool to create 50 concurrent
users.
The Monthly Sales Dashboard should respond promptly to user interactions,
even under high load.
2. Each user accesses the Monthly Sales Dashboard.
Response times should remain within acceptable thresholds as the user load
increases.
4. Users perform various actions, such as applying filters, sorting, and navigating through
the report.
The system should gracefully handle increased concurrent users without major
performance degradation.
4. Measure the time taken for the dashboard to respond to user actions.
Resource utilization should be within acceptable limits, and no critical errors or
system crashes should occur.
5. Gradually increase the load by adding more concurrent users (e.g., 100, 200, 300) and
repeat steps 2-4.

13. © 2019 Real-Time Technology Solutions, Inc.
22 West 38th Street FL 11, New York, NY 10018
www.rttsweb.com | (212) 240-9050
13 a software division of
QuerySurge™
Summary
• In the Design phase test cases are authored and reviewed.
• Components of a test case should include:
• Test Case ID
• Test Case Name
• Test Case Description
• Preconditions
• Test Steps
• Expected Results
• Pass/Fail Criteria
• Designing a data validation and accuracy test case against a BI report involves several steps to ensure
thorough and effective testing. Mapping documentation is utilized to account for any transformations in the
data before it is displayed in the BI report.
• Effective functional test cases in BI report testing involve in-depth understanding of requirements, diverse
testing scenarios, and validation of report functionality including filtering, sorting, and drill-downs.
• Designing performance test cases for BI reports involves meticulous planning, realistic data, and a focus on
scalability to ensure reports perform optimally under real-world conditions.