Learn how to strategically plan your manual and automated BI testing efforts so that it translates into a seamless and targeted tactical test execution phase leading to significant cost savings, increased reporting accuracy and greater business confidence.

1. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.1
Transforming BI Testing
Steps to Achieve Significant Cost Savings
Sunny Hiran
Director
Quality Assurance

2. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.2
Method360COMPANY OVERVIEWAbout Us
Leading Information Technology Consulting
Services Company
• Headquartered in San Francisco, CA
Local offices in San Diego, Chicago, Dallas,
Pittsburgh, and Calgary
• Celebrating Our 15th Year of Operations
• 100+ Employees Throughout the US
• Focus on Data, Analytics and Quality
Assurance
• Strong Emphasis on Consultant Quality and
Client Relationship
• Onsite, Offsite and Nearshore Capabilities
• Certified SAP Services Partner

3. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.3
Method360TRANSFORMING BI TESTINGIntroduction
Sunny Hiran
Director
Quality Assurance
Connect with me on LinkedIn: sunnyhiran
Sunny has worked both strategically and tactically with a broad selection of
fortune 500 companies touching upon multiple vertical industries including
retail, manufacturing, banking, finance, and high tech companies.
Bringing both project and program management experience along with PMP
certification, Sunny brings considerable testing expertise to clients.

4. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.4
Method360Session Overview
You will gain an understanding of:
• The need for rigorous BI testing – strategy, planning, and execution
• Why the prioritization of BI scope including data structures, report fields,
calculations, and filters is essential
• The importance of enabling your BI testing
• How a test strategy addresses multiple levels of BI-specific report testing that
includes report layout, field data, DSO, and query results validation and sign off
• How targeted test automation makes a significant impact
• The need to safeguard business/end user response times and ensure solution
scalability
TRANSFORMING BI TESTING

5. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.5
Method360TRANSFORMING BI TESTING

6. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.6
Method360The Cost of Quality
• Quality Assurance and testing
cannot be avoided – pay now or
pay later
• Ineffective testing will result in poor
quality deliverables, late delivery,
and overspend
• Failure is not an option (potential
loss of revenue and
competitiveness, lost customer
confidence, and bad publicity)
TRANSFORMING BI TESTING
Testing costs anywhere from 25-35% of a projects budget

7. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.7
Method360Common BI Testing Challenges
• There are inherently complex challenges associated with BI testing:
Extensive manual test labor required to perform complex validations at significant
cost and different levels of testing
• BI reconciliation. (Do things match – front end to BW, ECC)
• BO – multiple distribution channels such as email burst, online desktop vs. mobile report,
etc.
• ETL processes
• Data warehouses (OLAP cubes and queries)
• Report/Query filters, and data validation
• Test data and environment management
• Performance and response-time validation
• Dashboards and scorecards
TRANSFORMING BI TESTING

8. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.8
Method360Advantages of BI Risk-based Testing
Focus on functionality that:
Is most important to end users and key decision makers
Most commonly used
Has an impact to revenue generation
• Achieve cost savings by utilizing the 80/20 rule
• Provides for better impact analysis (of adjacent functionality)
• Early identification of issues by utilizing:
Smoke testing
Automated/accelerated validation based on priority
• Root cause analysis
TRANSFORMING BI TESTING
It’s impossible to test the entire solution, nor should you.
Cut costs by addressing appropriate risk areas and improve quality.

9. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.9
Method360Advantages of BI Risk-based Testing
(Cont.)
• Key reports
• Data model
• Indexes
• Transformations
TRANSFORMING BI TESTING
The areas of focus/risk tend to be around the following:
• Data capture points in ECC
• Extractors
• Queries (calculation change, formatting change)
• Distribution channels

10. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.10
Method360TRANSFORMING BI TESTING

11. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.11
Method360Identifying Solution Scope
• Gain a clear
understanding of scope,
priority, and impact
• Assess the impact of
failure points within the
infrastructure
• Document all items (by
priority) within a matrix
TRANSFORMING BI TESTING
Extractors
ECC data capture points
Key reports
Data model
Transformation

12. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.12
Method360Demystifying BI Testing TRANSFORMING BI TESTING

13. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.13
Method360Demystifying BI Testing (Cont.)
• Do we have the correct focus? Are we testing the right things to validate the
organization’s most important business priorities?
• Have we assessed potential business impacts and failure points in our strategy?
• Have we employed the right test approach to validate the different technologies
and address integrations and customizations?
• Do we understand our stakeholders reporting needs?
• Given the budget and scope, do we have the correct resourcing, test cycles,
strategies and tools?
• Are the acceptance criteria well defined?
• Do we have buy-in from the business?
• Does our testing address current and future business needs?
TRANSFORMING BI TESTING
A good test strategy should answer the following questions

14. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.14
Method360Where to Start
1. An assessment/roadmap will lay a foundation
2. Create a test plan that clarifies scope, risk, timelines, resourcing, goals and
objectives of testing, automation priorities, etc.
3. Identify stakeholders and ensure sign off
4. Identify a formal test strategy/framework/methodology that details
progressive levels of testing (data, queries, report layout, etc.)
5. Document priority (risk-based) items along with acceptance criteria
6. Create a test matrix that identifies all the in scope test objects as shown in
the diagram on the following slide
7. Create a requirements traceability matrix of all prioritized functionality
including queries, calculations, and reports
TRANSFORMING BI TESTING

15. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.15
Method360Where to Start
• Dedicated QA resources utilizing targeted test strategies
• Automation, performance (query and ETL), DR
• Clear communication on testing progress to all stakeholders
• Release Management/Change Control
• Quality gating criteria
• Standard best practices around environments
• I.e., Dev, QA, and Prod
• QA environment configuration
• Representative data sources/test data
TRANSFORMING BI TESTING
There are many BI test considerations that will enable the
success of your projects and further mitigate risks

16. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.16
Method360Supporting Successful Testing (Cont.)
• Data alignment and data models
• Engaging business teams early to validate the test scope, strategy, and
priorities will achieve several key objectives:
Business buy-in
Confidence in the solution
Quality outcomes and results
Knowledge transfer
• System-wide refresh strategy
• Formal risk management and risk-reassessment
• Return on Investment (ROI) based test automation to expand test coverage
and accelerate validation activities
• Validation of solution scalability and response times vs. SLA
TRANSFORMING BI TESTING

17. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.17
Method360Incorporating Test Automation
• Do your research when selecting a tool – any vendor should demonstrate the
claims they make on a tool
• Get educated about the value your project/organization will gain from Test
Automation
• Maximize ROI by performing validations across multiple environments, i.e.,
Dev and QA
• Have a clear roadmap that will guide your automation efforts
• Business buy-in/sign off will build confidence and accuracy
• The diagram on the following slide shows a recommended approach for
implementing Test Automation
TRANSFORMING BI TESTING
Test Automation when utilized successfully will significantly
expand test coverage and accelerate validation activities

18. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.18
Method360Incorporating Test Automation (Cont.) TRANSFORMING BI TESTING

19. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.19
Method360Realizing Value from Performance
Testing
• Having business buy-in on what to simulate will ensure an accurate and
meaningful activity
• Aim for a mirror of production
• Not all tools are equal
• Accurately simulating the entire infrastructure is a requirement
• Monitoring is essential as is an understanding of monitoring components and
sources of performance failures
TRANSFORMING BI TESTING
Many organizations struggle with performance and response
time issues of business critical reports; some in spite of having
internal performance testing. Obviously there is a problem!

20. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.20
Method360Why Metrics Matter
• Provide project/program visibility to stakeholders and key decision makers
• Communication tool
• Metrics provide a clear way of determining progress and readiness for go-
live/release
• Clear measure as to the level of conformance to requirements/ specifications
• Justification for:
QA efforts/resourcing
Inherent quality or the lack thereof
Pinpointing weaknesses across people, process, and tools
Understanding when the solution is complete
TRANSFORMING BI TESTING

21. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.21
Method360Why Metrics Matter (Cont.) TRANSFORMING BI TESTING

22. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.22
Method3607 Key Takeaways
• BI testing should not be underestimated and requires a detailed strategy and
planning in order to avoid unsatisfactory outcomes
• Subject BI projects to the same rigor/quality standards as any other project
• Risk-based testing with business buy-in is the key
• Dedicated QA resourcing will address ownership and accountability issues
around testing
• Use a formal test planning and management tool as your test system of record
• Use test automation – will make a significant difference
• Performance testing is becoming more critical to the success of organizations
with extensive BI capabilities
TRANSFORMING BI TESTING

23. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.23
Method360BI Testing White Paper
All attendees
will receive an
electronic copy
of Sunny’s
BI Testing white
paper via email
TRANSFORMING BI TESTING

24. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.24
Method360Questions TRANSFORMING BI TESTING

25. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.25
Method360More webinars…
Check out our YouTube channel to view more webinars on-demand
TRANSFORMING BI TESTING

26. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.26
Method360PRACTICE LEADERSHIPContact Information
P R E S E N T E R P R A C T I C E S
ANALYTICS
BLAKE DUGGAN
415.548.0762
bduggan@method360.com
ENTERPRISE INFORMATION
MANAGEMENT
JIM HANCK
415.694.8229
jhanck@method360.com
FINANCIAL MANAGEMENT
STEPHEN STONE
408.206.2830
sstone@method360.com
SALES & MARKETING
GREG CLARKE
714.380.2753
gclarke@method360.com
MANAGED SERVICES,
PMO AND DELIVERY
ERIK AXTON
925.389.6708
eaxton@method360.com
STRATEGIC ACCOUNTS
GREG STIMPSON
925.640.8785
gstimpson@method360.com
QUALITY ASSURANCE
SUNNY HIRAN
415.535.5453
shiran@method360.com

27. Method360SECTION NAME GOES HERE IN CAPSHeader goes here
© 2015 Method360, Inc. All rights reserved.27
Method360TRANSFORMING BI TESTINGThank You
E N D - T O - E N D S E R V I C E S . H I G H V A L U E S O L U T I O N S .
Follow / like us on