1.
built by
the automated Data Testing solution
QuerySurge™

2.
Where QuerySurge™ fits in your data strategy
built by
QuerySurge™

3.
Business Intelligence (BI) software
CxOs are using Business Intelligence & Analytics to make critical business decisions
– with the assumption that the underlying data is fine.
“The average organization loses
$8.2 million annually through
poor Data Quality.”
- Gartner
ETL
Data Architecture
The Executive Office and Critical Data
potential problem
areas

4.
Current Business Case for Data Testing
built by
QuerySurge™
“46% of companies cite data quality as a barrier for adopting
Business Intelligence products”
- InformationWeek
“On average, U.S. organizations believe 32% of their data is
inaccurate”
– Experian Data Quality research report
“Poor data quality is a primary reason for 40% of all business
initiatives failing to achieve their targeted benefits”
- analyst firm Gartner
“90% percent of U.S. companies have some sort of data
quality solution in place today”
– Experian Data Quality research report
Data quality solutions are not enough!

5.
o Profiling
o Parsing and standardization
o Generalized Cleansing
o Matching
o Monitoring
o Enrichment
o Subject-area-specific support
o Metadata management
o Configuration environment
Data Quality tools vs. Data Testing tool
built by
QuerySurge™
 Data Completeness
 Data Transformation
 Regression Testing
Primary Characteristics of Data Quality tools
courtesy of Gartner’s “Magic Quadrant for Data Quality Tools”
Data
Verification &
Validation?
Primary Characteristics of Data Testing tools
Courtesy of the book "Testing the Data Warehouse Practicum"
Data
Verification &
Validation?

6.
Definitive Data Testing Book
built by
QuerySurge™

7.
DataTesting Compare methods
built by
1) Sampling
(also known as “Stare & Compare” )
2) Minus Queries

8.
Method #1: Stare & Compare
built by
QuerySurge™
• Review Business Rules (i.e. Mapping Document: data flow mapping, data movement requirements)
• Write Tests in SQL editor
• Execute 2 Tests: 1 at Source & 1 at Target
• Dump results to 2 Excel files
• Compare results by eye (‘Stare & Compare’ or ‘sampling’)
Issue with Stare & Compare:
Impossible to visually compare billions of data sets.
Result: usually less than 1% of data is compared
Example:
Current QuerySurge customer has:
• a single test with 100 million rows & 200 columns
• = 20 billion data sets
• the client has > 7,000 total tests

9.
built by
QuerySurge™
MINUS QUERIES subtract one result set from another result set to show difference
Comment: MINUS QUERIES need to be executed 2x (Source MINUS Target; Target MINUS Source)
Result sets may not be accurate when dealing with duplicate rows of data
No historical data from past testing – audit and regulatory issues
Processing of minus queries puts pressure on the servers
Double execution means 2x testing time and resource utilization
Method #2: Minus Queries
Minus Query #1: Table_1 MINUS Table_2
Minus Query #2: Table_2 MINUS Table_1
Result Set #1
Result Set #2
ISSUES with MINUS QUERIES
Write 2 MINUS queries
in SQL editor
Execute
MINUS queries 2x

10.
DataTesting Compare Methods: 2 issues
built by
QuerySurge™
1) There is a fundamental issue with both current methods:
The assumption that all team members can write SQL/HQL code
2) Neither method fully satisfies any of the conditions below:
Data Completeness
Data Transformation
Regression Testing

11.
About QuerySurge™
built by

12.
What is QuerySurge™?
the collaborative
Data Testing solution that
finds bad data & provides
a holistic view of your
data’s health
built by

13.
the QuerySurge advantage
built by
QuerySurge™
Automate the entire testing cycle
 Automate the launch, tests, comparison, auto-emailed results
Create Tests easily with no SQL programming
 Query Wizards ensure minimal time & effort to create tests
Test across different platforms
 Data Warehouse, Hadoop, NoSQL, database, flat file, XML
Collaborate with team
 Data Health dashboard, shared tests & auto-emailed reports
Verify more data & do it quickly
 verifies up to 100% of all data up to 1,000 x faster
Integrate for Continuous Delivery (DevOps)
 Integrates with most Build, ETL & QA management software

14.
Flat Files
QuerySurge™ Architecture
Web-based…
Installs on...
Linux
Connects to…
…or any other JDBC compliant data source
built by
QuerySurge™
QuerySurge
Controller
QuerySurge
Server
QuerySurge
Agents

15.
SQL
HQL
SQL
HQL
SQL
SQL
 QS pulls data from data sources
 QS pulls data from target data store
 QS compares data quickly
 QS generates reports, audit trails
How QuerySurge Works
Reports, Data Health Dashboard, auto emails
built by
QuerySurge™
Source Data Target Data
Data Stores
• Databases
• Data Warehouses
• Data Marts
Flat Files
• Fixed Width
• Delimited
• Excel
Big Data stores
• Hadoop
• NoSQL
Data
Warehouses
XML
Web Services

16.
Data Process: Developer & Tester
built by
QuerySurge™
Developer: Codes data movement based on Business Requirements
Tester: Tests data movement based on Business Requirements
Business
Intelligence
ETL
Source Data
Big Data ETL Process Target DWH

17.
Collaboration
Testers
- functional testing
- regression testing
- result analysis
Developers / DBAs
- unit testing
- result analysis
Data Analysts
- review, analyze data
- verify mapping failures
Operations teams
- monitoring
- result analysis
Managers
- oversight
- result analysis
Share information on the
built by
QuerySurge™

18.
built by
QuerySurge™
QuerySurge™ Modules
Design
Library
SchedulingRun
Dashboard
Deep-Dive
Reporting
Data Health
Dashboard
Query
Wizards

19.
Design Library
• Create Query Pairs (source & target SQLs)
• Great for team members skilled with SQL
QuerySurge™ Modules
Scheduling
 Build groups of Query Pairs
 Schedule Test Runs
built by
QuerySurge™

20.
Deep-Dive Reporting
 Examine and automatically
email test results
Run Dashboard
 View real-time execution
 Analyze real-time results
QuerySurge™ Modules
built by
QuerySurge™

21.
built by
QuerySurge™
• view data reliability & pass rate
• add, move, filter, zoom-in on any
data widget & underlying data
• verify build success or failure
QuerySurge™ Modules

22.
Fast and Easy.
No programming needed.
built by
QuerySurge™
QuerySurge™ Modules
• Perform 80% of all data tests -
no SQL coding needed
• Opens up testing to novices &
non-technical team members
• Speeds up testing for skilled SQL coders
• provides a huge Return-On-Investment

23.
QuerySurge Test Management Connectors
built by
QuerySurge™
 Drive QuerySurge execution from your Test Management Solution
 See QuerySurge Pass/Fail results in your Test Management solution
 Click link to drill into detailed results in QuerySurge
• HP ALM (Quality Center)
• Microsoft Team Foundation Server
• IBM Rational Quality Manager
Integration with leading
Test Management Solutions

24.
QuerySurge & DevOps: Continuous Delivery & Integration
built by
QuerySurge™
Automated
Testing
Automated
Reporting
Automated
Launch
Data Integration/ETL
solutions
QuerySurge™
and many others…
email
report
Test Management
solutions
QuerySurge™
email
report
and many others…
QuerySurge™
Automated Build
solutions
email
report

25.
• Reduce your costs & risks
• Improve your data quality
• Accelerate your testing cycles
• Share information with your team
built by
QuerySurge™
• Realize a huge ROI (like 1,600%)
QuerySurge’s Impact

26.
CustomersQuerySurge™

27.
built by
QuerySurge™
About
FACTS
Founded:
1996
headquarters:
Manhattan, New York
Customer profile:
• Fortune 1000
• 600+ customers
Strategic Partners:
IBM, Microsoft, HP,
Oracle, Teradata,
HortonWorks, Cloudera,
MongoDB
Software Division:
QuerySurge
RTTS is the parent company of QuerySurge
and is the premier pure-play QA & Testing organization
that specializes in test automation

28.
QuerySurge
built by
built by
QuerySurge™
You
www.QuerySurge.com