SlideShare a Scribd company logo
1 of 30
built by
Introducing the new
We just made data testing
REALLY EASY!
No programming needed
Automate your
Data Warehouse & Big Data Testing
and Reap the Benefits
*available for download on August 3, 2015
built by
QuerySurge™
About
FACTS
Founded:
1996
Locations:
New York (HQ), Atlanta,
Philadelphia, Phoenix
Strategic Partners:
IBM, Microsoft, HP,
Oracle, Teradata,
HortonWorks, Cloudera,
Amazon
Software:
QuerySurge
RTTS is the leading provider of software & data quality
for critical business systems
“70% of enterprises have either deployed or are planning to
deploy big data projects and programs this year”
– analyst firm IDG
“46% of companies cite data quality as a barrier for adopting
Business Intelligence products.”
- InformationWeek
“Poor data quality is a primary reason for 40% of all business
initiatives failing to achieve their targeted benefits.”
- analyst firm Gartner
Data Quality Issues
built by
QuerySurge™
2 Prevalent DataTesting Strategies
built by
QuerySurge™
1) Stare & Compare
2) Minus Queries
DataTesting Strategy #1: Stare & Compare
built by
QuerySurge™
• Review Mapping Document (business rules, data flow mapping, data movement requirements)
• Write Tests in SQL editor
• Execute 2 Tests: 1 at Source & 1 at Target
• Dump results to 2 Excel files
• Compare results by eye (Stare & Compare)
Difficulty with Stare & Compare:
Impossible to visually compare millions/billions of data sets visually.
Example:
Current QuerySurge customer has:
• a single test with 100 million rows
• 200 columns
• = 20 billion data sets
• the client has > 7,000 total tests
Data Tester’s Current Process
built by
QuerySurge™
MINUS QUERIES subtract one result set from another result set to show difference
Comment: MINUS QUERIES need to be executed 2x (Source MINUS Target; Target MINUS Source)
Result sets may not be accurate when dealing with duplicate rows of data
No historical data from past testing – audit and regulatory issues
Processing of minus queries puts pressure on the servers
Double execution means 2x testing time and resource utilization
Potential for false positives (bad data could exist on both sides of an ETL leg)
DataTesting Strategy #2: Minus Queries
Minus Query #1: Table_1 MINUS Table_2
Minus Query #2: Table_2 MINUS Table_1
Result Set #1
Result Set #2
ISSUES with MINUS QUERIES
Write 2 MINUS queries
in SQL editor
Execute
MINUS queries 2x
DataTesting Strategies
built by
QuerySurge™
a fundamental issue with the 2 Strategies:
Assumption that all team members understand
and can write SQL code
What is QuerySurge™?
the collaborative
Data Testing solution that
finds bad data & provides
a holistic view of your
data’s health
built by
• Reduce your costs & risks
• Improve your data quality
• Accelerate your testing cycles
• Share information with your team
with QuerySurge™ you can:
built by
QuerySurge™
• Provides huge ROI (i.e. 1,300%)*
*based on client’s calculation of Return on Investment
the QuerySurge advantage
built by
QuerySurge™
Automate the entire testing cycle
 Automate kickoff, tests, comparison, auto-emailed results
Create Tests easily with no SQL programming
 ensures minimal time & effort to create tests / obtain results
Test across different platforms
 data warehouse, Hadoop, NoSQL, database, flat file, XML
Collaborate with team
 Data Health dashboard, shared tests & auto-emailed reports
Verify more data & do it quickly
 verifies up to 100% of all data up to 1,000 x faster
Integrate for Continuous Delivery
 Integrates with most Build, ETL & QA management software
Finding Bad Data
SQL
HQL
SQL
HQL
SQL
SQL
 QS pulls data from data sources
 QS pulls data from target data store
 QS compares data quickly
 QS generates reports, audit trails
How?
Reports, Data Health Dashboard
built by
QuerySurge™
Source Data
Target Data
QuerySurge™ Architecture
Web-based…
Installs on...
Linux
Connects to…
…or any other JDBC compliant data source
built by
QuerySurge™
QuerySurge
Controller
QuerySurge
Server
QuerySurge
Agents
Flat Files
Collaboration
Testers
- functional testing
- regression testing
- result analysis
Developers / DBAs
- unit testing
- result analysis
Data Analysts
- review, analyze data
- verify mapping failures
Operations teams
- monitoring
- result analysis
Managers
- oversight
- result analysis
Share information on the
built by
QuerySurge™
built by
QuerySurge™
built by
From a recent poll1 of:
• Big Data Experts
• Data Warehouse Architects
• Solution Architects
• ETL Architects
Recent Survey: Data Experts
Consensus Answer:
80% of data columns have no transformation at all
Our Question: What % of columns in your Data Warehouse
have no transformations at all?
1Poll conducted by RTTS on targeted LinkedIn groups
Why is this important?
Fast and Easy.
No programming needed.
built by
QuerySurge™
QuerySurge™ Modules
Compare by Table, Column & Row
• Perform 80% of all data tests
•Automatically generates SQL code
• Opens up testing to novice & non-
technical team members
• Speeds up testing for skilled SQL coders
• provides a huge Return-On-Investment
built by
QuerySurge™
QuerySurge™ Modules
3 Types of Data Comparison Wizards:
The also provide you with automated features for:
o filtering (‘Where’ clause) and
o sorting (‘Order By’ clause)
Column-Level Comparison:
This is great for Big Data stores and Data Warehouses where tables will have some columns
containing transformations and some columns with no transformations. Many tables and
columns can be compared simultaneously and quickly.
Table-Level Comparison:
This comparator is great for Data Migrations and Database Upgrades with no
transformations at all. Many tables can be compared simultaneously and quickly.
Row Count Comparison:
Great for all - Big Data stores, Data Warehouses, Data Migrations and Database Upgrades.
Many tables and rows can be compared simultaneously and quickly.
Uses:
Tests the columns that have no
transformations, which means it tests
approximately 80% of your data store without
you writing any SQL code
Tests:
Big data, data warehouses
Value added:
novice or non-technical: no coding needed,
productive immediately
experienced user: saves time
built by
QuerySurge™
built by
QuerySurge™
Uses:
Verifies data loads when no
transformation occurs
Tests:
data migrations, upgrades
Value added:
novice or non-technical:
no coding needed
experienced user:
saves time
built by
QuerySurge™
Use:
Verify that the amount of rows from the
source match the amount from the target
Tests:
Big data, data warehouse, data
migration, database upgrades, data
interfaces
Value added:
novice: no coding needed
experienced user: saves time
built by
QuerySurge™
_________
Total
built by
QuerySurge™
all QuerySurge™ Modules
Design Library
SchedulingDeep-Dive Reporting
Run Dashboard
Query Wizards
Data Health Dashboard
Design Library
• Create custom Query Pairs (source & target SQLs)
• Great for team members skilled with SQL
QuerySurge™ Modules
Scheduling
 Build groups of Query Pairs
 Schedule Test Runs for:
• immediately
• at a specific date/time
• automatically after build or
ETL process
built by
QuerySurge™
Deep-Dive Reporting
 Examine and automatically
email test results
Run Dashboard
 View real-time execution
 Analyze real-time results
QuerySurge™ Modules
built by
QuerySurge™
QuerySurge Test Management Connectors
built by
QuerySurge™
 Drive QuerySurge execution from your Test Management Solution
 Outcome results (Pass/Fail/etc.) are returned from QuerySurge to your Test Management Solution
 Results are linked in your Test Management Solution so that you can click directly into detailed QuerySurge
results
• HP ALM (Quality Center)
• Microsoft Team Foundation Server
• IBM Rational Quality Manager
Integration with leading
Test Management Solutions
Licensing
License Types:
 Full User
 Participant User (i.e. read-only)
built by
QuerySurge™
License Model for:
Perpetual
 own QuerySurgeTM, pay annual maintenance
Subscription
 use QuerySurgeTM for set period (12 months)
QuerySurge™
8/18/2015 27
built by
QuerySurge™
Training Courses
Data Warehouse Testing
• Data Warehouse & ETL Testing Fundamentals (1 day)
• Fundamentals of QuerySurge (1 day)
• Introduction to SQL for QuerySurge (1 day)
• Advanced SQL techniques for QuerySurge (1 day)
Big Data Testing
• Big Data And ETL Testing Fundamentals
• Introduction To Big Data Testing Using Hive And HQL
Consulting
RTTS, the software quality experts (and developer of QuerySurge), provides consulting
solutions to the challenges of Big Data & Data Warehouse / ETL Testing
• Jumpstart 2-week program – combines training courses, mentoring, consulting
• Staff Augmentation – add additional RTTS resources to your team
• Outsourcing - RTTS can perform all testing, including planning, design, execution
built by
QuerySurge™
Support
• Live Chat through QuerySurge or web site
• Email support through Zendesk
• Yammer Network
• Webex sessions
• Phone support
(1) Trial in the Cloud of QuerySurgeTM, including self-learning
tutorial that works with sample data for 3 days
(2) Downloaded Trial of QuerySurgeTM, including self-learning
tutorial with sample data or your data for 15 days
(3) Proof of Concept of QuerySurgeTM includes our team of experts
assisting you for 30 days
for more information on (1), (2) and (3),
Go to querysurge.com/compare-trial-options
TRIAL
IN THE CLOUD
built by
QuerySurge™
Free TrialsQuerySurge™
Proof
of
Concept
built by
QuerySurge™
QuerySurge
For more on the Query Wizards, go to querysurge.com/querysurge-query-wizards

More Related Content

What's hot

Creating a Data validation and Testing Strategy
Creating a Data validation and Testing StrategyCreating a Data validation and Testing Strategy
Creating a Data validation and Testing StrategyRTTS
 
QuerySurge - the automated Data Testing solution
QuerySurge - the automated Data Testing solutionQuerySurge - the automated Data Testing solution
QuerySurge - the automated Data Testing solutionRTTS
 
RTTS - the Software Quality Experts
RTTS - the Software Quality ExpertsRTTS - the Software Quality Experts
RTTS - the Software Quality ExpertsRTTS
 
Data Warehousing in Pharma: How to Find Bad Data while Meeting Regulatory Req...
Data Warehousing in Pharma: How to Find Bad Data while Meeting Regulatory Req...Data Warehousing in Pharma: How to Find Bad Data while Meeting Regulatory Req...
Data Warehousing in Pharma: How to Find Bad Data while Meeting Regulatory Req...RTTS
 
Data Warehouse Testing in the Pharmaceutical Industry
Data Warehouse Testing in the Pharmaceutical IndustryData Warehouse Testing in the Pharmaceutical Industry
Data Warehouse Testing in the Pharmaceutical IndustryRTTS
 
Completing the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = SuccessCompleting the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = SuccessRTTS
 
Whitepaper: Volume Testing Thick Clients and Databases
Whitepaper:  Volume Testing Thick Clients and DatabasesWhitepaper:  Volume Testing Thick Clients and Databases
Whitepaper: Volume Testing Thick Clients and DatabasesRTTS
 
QuerySurge Slide Deck for Big Data Testing Webinar
QuerySurge Slide Deck for Big Data Testing WebinarQuerySurge Slide Deck for Big Data Testing Webinar
QuerySurge Slide Deck for Big Data Testing WebinarRTTS
 
Iron Mountain: Fueling Big Testing with Big Data - SiriusDecisions 2013
Iron Mountain: Fueling Big Testing with Big Data - SiriusDecisions 2013Iron Mountain: Fueling Big Testing with Big Data - SiriusDecisions 2013
Iron Mountain: Fueling Big Testing with Big Data - SiriusDecisions 2013ion interactive
 
What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?RTTS
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World DistilledRTTS
 
Testing Big Data: Automated Testing of Hadoop with QuerySurge
Testing Big Data: Automated  Testing of Hadoop with QuerySurgeTesting Big Data: Automated  Testing of Hadoop with QuerySurge
Testing Big Data: Automated Testing of Hadoop with QuerySurgeRTTS
 
Hadoop testing workshop - july 2013
Hadoop testing workshop - july 2013Hadoop testing workshop - july 2013
Hadoop testing workshop - july 2013Ophir Cohen
 
How to Test Big Data Systems | QualiTest Group
How to Test Big Data Systems | QualiTest GroupHow to Test Big Data Systems | QualiTest Group
How to Test Big Data Systems | QualiTest GroupQualitest
 
A data driven etl test framework sqlsat madison
A data driven etl test framework sqlsat madisonA data driven etl test framework sqlsat madison
A data driven etl test framework sqlsat madisonTerry Bunio
 
How to Automate your Enterprise Application / ERP Testing
How to Automate your  Enterprise Application / ERP TestingHow to Automate your  Enterprise Application / ERP Testing
How to Automate your Enterprise Application / ERP TestingRTTS
 
Automate data warehouse etl testing and migration testing the agile way
Automate data warehouse etl testing and migration testing the agile wayAutomate data warehouse etl testing and migration testing the agile way
Automate data warehouse etl testing and migration testing the agile wayTorana, Inc.
 
Testing Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of HadoopTesting Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of HadoopRTTS
 
Oracle Real Application Testing: A Business Case
Oracle Real Application Testing: A Business CaseOracle Real Application Testing: A Business Case
Oracle Real Application Testing: A Business Caseoracleonthebrain
 

What's hot (20)

Creating a Data validation and Testing Strategy
Creating a Data validation and Testing StrategyCreating a Data validation and Testing Strategy
Creating a Data validation and Testing Strategy
 
QuerySurge - the automated Data Testing solution
QuerySurge - the automated Data Testing solutionQuerySurge - the automated Data Testing solution
QuerySurge - the automated Data Testing solution
 
RTTS - the Software Quality Experts
RTTS - the Software Quality ExpertsRTTS - the Software Quality Experts
RTTS - the Software Quality Experts
 
Data Warehousing in Pharma: How to Find Bad Data while Meeting Regulatory Req...
Data Warehousing in Pharma: How to Find Bad Data while Meeting Regulatory Req...Data Warehousing in Pharma: How to Find Bad Data while Meeting Regulatory Req...
Data Warehousing in Pharma: How to Find Bad Data while Meeting Regulatory Req...
 
Data Warehouse Testing in the Pharmaceutical Industry
Data Warehouse Testing in the Pharmaceutical IndustryData Warehouse Testing in the Pharmaceutical Industry
Data Warehouse Testing in the Pharmaceutical Industry
 
Completing the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = SuccessCompleting the Data Equation: Test Data + Data Validation = Success
Completing the Data Equation: Test Data + Data Validation = Success
 
Whitepaper: Volume Testing Thick Clients and Databases
Whitepaper:  Volume Testing Thick Clients and DatabasesWhitepaper:  Volume Testing Thick Clients and Databases
Whitepaper: Volume Testing Thick Clients and Databases
 
QuerySurge Slide Deck for Big Data Testing Webinar
QuerySurge Slide Deck for Big Data Testing WebinarQuerySurge Slide Deck for Big Data Testing Webinar
QuerySurge Slide Deck for Big Data Testing Webinar
 
Iron Mountain: Fueling Big Testing with Big Data - SiriusDecisions 2013
Iron Mountain: Fueling Big Testing with Big Data - SiriusDecisions 2013Iron Mountain: Fueling Big Testing with Big Data - SiriusDecisions 2013
Iron Mountain: Fueling Big Testing with Big Data - SiriusDecisions 2013
 
What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?
 
Test Automation for Data Warehouses
Test Automation for Data Warehouses Test Automation for Data Warehouses
Test Automation for Data Warehouses
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World Distilled
 
Testing Big Data: Automated Testing of Hadoop with QuerySurge
Testing Big Data: Automated  Testing of Hadoop with QuerySurgeTesting Big Data: Automated  Testing of Hadoop with QuerySurge
Testing Big Data: Automated Testing of Hadoop with QuerySurge
 
Hadoop testing workshop - july 2013
Hadoop testing workshop - july 2013Hadoop testing workshop - july 2013
Hadoop testing workshop - july 2013
 
How to Test Big Data Systems | QualiTest Group
How to Test Big Data Systems | QualiTest GroupHow to Test Big Data Systems | QualiTest Group
How to Test Big Data Systems | QualiTest Group
 
A data driven etl test framework sqlsat madison
A data driven etl test framework sqlsat madisonA data driven etl test framework sqlsat madison
A data driven etl test framework sqlsat madison
 
How to Automate your Enterprise Application / ERP Testing
How to Automate your  Enterprise Application / ERP TestingHow to Automate your  Enterprise Application / ERP Testing
How to Automate your Enterprise Application / ERP Testing
 
Automate data warehouse etl testing and migration testing the agile way
Automate data warehouse etl testing and migration testing the agile wayAutomate data warehouse etl testing and migration testing the agile way
Automate data warehouse etl testing and migration testing the agile way
 
Testing Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of HadoopTesting Big Data: Automated ETL Testing of Hadoop
Testing Big Data: Automated ETL Testing of Hadoop
 
Oracle Real Application Testing: A Business Case
Oracle Real Application Testing: A Business CaseOracle Real Application Testing: A Business Case
Oracle Real Application Testing: A Business Case
 

Similar to Query Wizards - data testing made easy - no programming

QuerySurge AI webinar
QuerySurge AI webinarQuerySurge AI webinar
QuerySurge AI webinarRTTS
 
Deliver Trusted Data by Leveraging ETL Testing
Deliver Trusted Data by Leveraging ETL TestingDeliver Trusted Data by Leveraging ETL Testing
Deliver Trusted Data by Leveraging ETL TestingCognizant
 
Copy of Alok_Singh_CV
Copy of Alok_Singh_CVCopy of Alok_Singh_CV
Copy of Alok_Singh_CVAlok Singh
 
TestGuild and QuerySurge Presentation -DevOps for Data Testing
TestGuild and QuerySurge Presentation -DevOps for Data TestingTestGuild and QuerySurge Presentation -DevOps for Data Testing
TestGuild and QuerySurge Presentation -DevOps for Data TestingRTTS
 
Creating a Project Plan for a Data Warehouse Testing Assignment
Creating a Project Plan for a Data Warehouse Testing AssignmentCreating a Project Plan for a Data Warehouse Testing Assignment
Creating a Project Plan for a Data Warehouse Testing AssignmentRTTS
 
DataOps , cbuswaw April '23
DataOps , cbuswaw April '23DataOps , cbuswaw April '23
DataOps , cbuswaw April '23Jason Packer
 
Data Warehouse Optimization
Data Warehouse OptimizationData Warehouse Optimization
Data Warehouse OptimizationCloudera, Inc.
 
Etl testing strategies
Etl testing strategiesEtl testing strategies
Etl testing strategiessivam_1
 
Automated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI ReportsRTTS
 
Rahul_Raj_Cse_Resume
Rahul_Raj_Cse_ResumeRahul_Raj_Cse_Resume
Rahul_Raj_Cse_ResumeRahul Raj
 
Presentation application change management and data masking strategies for ...
Presentation   application change management and data masking strategies for ...Presentation   application change management and data masking strategies for ...
Presentation application change management and data masking strategies for ...xKinAnx
 
Analyti x mapping manager product overview presentation
Analyti x mapping manager product overview presentationAnalyti x mapping manager product overview presentation
Analyti x mapping manager product overview presentationAnalytixDataServices
 
AnalytiX DS - Master Deck
AnalytiX DS - Master DeckAnalytiX DS - Master Deck
AnalytiX DS - Master DeckAnalytiX DS
 
State of the Market - Data Quality in 2023
State of the Market - Data Quality in 2023State of the Market - Data Quality in 2023
State of the Market - Data Quality in 2023RTTS
 
Varsha_CV_ETLTester5.8Years
Varsha_CV_ETLTester5.8YearsVarsha_CV_ETLTester5.8Years
Varsha_CV_ETLTester5.8YearsVarsha Hiremath
 
2010/10 - Database Architechs - Perf. & Tuning Tools
2010/10 - Database Architechs - Perf. & Tuning Tools2010/10 - Database Architechs - Perf. & Tuning Tools
2010/10 - Database Architechs - Perf. & Tuning ToolsDatabase Architechs
 

Similar to Query Wizards - data testing made easy - no programming (20)

QuerySurge AI webinar
QuerySurge AI webinarQuerySurge AI webinar
QuerySurge AI webinar
 
Deliver Trusted Data by Leveraging ETL Testing
Deliver Trusted Data by Leveraging ETL TestingDeliver Trusted Data by Leveraging ETL Testing
Deliver Trusted Data by Leveraging ETL Testing
 
Resume sailaja
Resume sailajaResume sailaja
Resume sailaja
 
Copy of Alok_Singh_CV
Copy of Alok_Singh_CVCopy of Alok_Singh_CV
Copy of Alok_Singh_CV
 
TestGuild and QuerySurge Presentation -DevOps for Data Testing
TestGuild and QuerySurge Presentation -DevOps for Data TestingTestGuild and QuerySurge Presentation -DevOps for Data Testing
TestGuild and QuerySurge Presentation -DevOps for Data Testing
 
Soumya sree Sridharala
Soumya sree SridharalaSoumya sree Sridharala
Soumya sree Sridharala
 
Creating a Project Plan for a Data Warehouse Testing Assignment
Creating a Project Plan for a Data Warehouse Testing AssignmentCreating a Project Plan for a Data Warehouse Testing Assignment
Creating a Project Plan for a Data Warehouse Testing Assignment
 
DataOps , cbuswaw April '23
DataOps , cbuswaw April '23DataOps , cbuswaw April '23
DataOps , cbuswaw April '23
 
Data Warehouse Optimization
Data Warehouse OptimizationData Warehouse Optimization
Data Warehouse Optimization
 
Etl testing strategies
Etl testing strategiesEtl testing strategies
Etl testing strategies
 
Taming the shrew Power BI
Taming the shrew Power BITaming the shrew Power BI
Taming the shrew Power BI
 
Automated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI ReportsAutomated Testing of Microsoft Power BI Reports
Automated Testing of Microsoft Power BI Reports
 
JKSQL
JKSQLJKSQL
JKSQL
 
Rahul_Raj_Cse_Resume
Rahul_Raj_Cse_ResumeRahul_Raj_Cse_Resume
Rahul_Raj_Cse_Resume
 
Presentation application change management and data masking strategies for ...
Presentation   application change management and data masking strategies for ...Presentation   application change management and data masking strategies for ...
Presentation application change management and data masking strategies for ...
 
Analyti x mapping manager product overview presentation
Analyti x mapping manager product overview presentationAnalyti x mapping manager product overview presentation
Analyti x mapping manager product overview presentation
 
AnalytiX DS - Master Deck
AnalytiX DS - Master DeckAnalytiX DS - Master Deck
AnalytiX DS - Master Deck
 
State of the Market - Data Quality in 2023
State of the Market - Data Quality in 2023State of the Market - Data Quality in 2023
State of the Market - Data Quality in 2023
 
Varsha_CV_ETLTester5.8Years
Varsha_CV_ETLTester5.8YearsVarsha_CV_ETLTester5.8Years
Varsha_CV_ETLTester5.8Years
 
2010/10 - Database Architechs - Perf. & Tuning Tools
2010/10 - Database Architechs - Perf. & Tuning Tools2010/10 - Database Architechs - Perf. & Tuning Tools
2010/10 - Database Architechs - Perf. & Tuning Tools
 

Recently uploaded

XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsXpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsMehedi Hasan Shohan
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...Christina Lin
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxbodapatigopi8531
 
What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number SystemsJheuzeDellosa
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptkotipi9215
 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsAlberto González Trastoy
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...MyIntelliSource, Inc.
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...kellynguyen01
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfkalichargn70th171
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software DevelopersVinodh Ram
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...gurkirankumar98700
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningVitsRangannavar
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWave PLM
 
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝soniya singh
 
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationProject Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationkaushalgiri8080
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...OnePlan Solutions
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideChristina Lin
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityNeo4j
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfPower Karaoke
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataBradBedford3
 

Recently uploaded (20)

XpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software SolutionsXpertSolvers: Your Partner in Building Innovative Software Solutions
XpertSolvers: Your Partner in Building Innovative Software Solutions
 
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
ODSC - Batch to Stream workshop - integration of Apache Spark, Cassandra, Pos...
 
Hand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptxHand gesture recognition PROJECT PPT.pptx
Hand gesture recognition PROJECT PPT.pptx
 
What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number Systems
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.ppt
 
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time ApplicationsUnveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
Unveiling the Tech Salsa of LAMs with Janus in Real-Time Applications
 
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
 
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
Short Story: Unveiling the Reasoning Abilities of Large Language Models by Ke...
 
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdfLearn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software Developers
 
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
(Genuine) Escort Service Lucknow | Starting ₹,5K To @25k with A/C 🧑🏽‍❤️‍🧑🏻 89...
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learning
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need It
 
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
 
Project Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanationProject Based Learning (A.I).pptx detail explanation
Project Based Learning (A.I).pptx detail explanation
 
Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...Advancing Engineering with AI through the Next Generation of Strategic Projec...
Advancing Engineering with AI through the Next Generation of Strategic Projec...
 
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop SlideBuilding Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Slide
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdf
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
 

Query Wizards - data testing made easy - no programming

  • 1. built by Introducing the new We just made data testing REALLY EASY! No programming needed Automate your Data Warehouse & Big Data Testing and Reap the Benefits *available for download on August 3, 2015
  • 2. built by QuerySurge™ About FACTS Founded: 1996 Locations: New York (HQ), Atlanta, Philadelphia, Phoenix Strategic Partners: IBM, Microsoft, HP, Oracle, Teradata, HortonWorks, Cloudera, Amazon Software: QuerySurge RTTS is the leading provider of software & data quality for critical business systems
  • 3. “70% of enterprises have either deployed or are planning to deploy big data projects and programs this year” – analyst firm IDG “46% of companies cite data quality as a barrier for adopting Business Intelligence products.” - InformationWeek “Poor data quality is a primary reason for 40% of all business initiatives failing to achieve their targeted benefits.” - analyst firm Gartner Data Quality Issues built by QuerySurge™
  • 4. 2 Prevalent DataTesting Strategies built by QuerySurge™ 1) Stare & Compare 2) Minus Queries
  • 5. DataTesting Strategy #1: Stare & Compare built by QuerySurge™ • Review Mapping Document (business rules, data flow mapping, data movement requirements) • Write Tests in SQL editor • Execute 2 Tests: 1 at Source & 1 at Target • Dump results to 2 Excel files • Compare results by eye (Stare & Compare) Difficulty with Stare & Compare: Impossible to visually compare millions/billions of data sets visually. Example: Current QuerySurge customer has: • a single test with 100 million rows • 200 columns • = 20 billion data sets • the client has > 7,000 total tests Data Tester’s Current Process
  • 6. built by QuerySurge™ MINUS QUERIES subtract one result set from another result set to show difference Comment: MINUS QUERIES need to be executed 2x (Source MINUS Target; Target MINUS Source) Result sets may not be accurate when dealing with duplicate rows of data No historical data from past testing – audit and regulatory issues Processing of minus queries puts pressure on the servers Double execution means 2x testing time and resource utilization Potential for false positives (bad data could exist on both sides of an ETL leg) DataTesting Strategy #2: Minus Queries Minus Query #1: Table_1 MINUS Table_2 Minus Query #2: Table_2 MINUS Table_1 Result Set #1 Result Set #2 ISSUES with MINUS QUERIES Write 2 MINUS queries in SQL editor Execute MINUS queries 2x
  • 7. DataTesting Strategies built by QuerySurge™ a fundamental issue with the 2 Strategies: Assumption that all team members understand and can write SQL code
  • 8. What is QuerySurge™? the collaborative Data Testing solution that finds bad data & provides a holistic view of your data’s health built by
  • 9. • Reduce your costs & risks • Improve your data quality • Accelerate your testing cycles • Share information with your team with QuerySurge™ you can: built by QuerySurge™ • Provides huge ROI (i.e. 1,300%)* *based on client’s calculation of Return on Investment
  • 10. the QuerySurge advantage built by QuerySurge™ Automate the entire testing cycle  Automate kickoff, tests, comparison, auto-emailed results Create Tests easily with no SQL programming  ensures minimal time & effort to create tests / obtain results Test across different platforms  data warehouse, Hadoop, NoSQL, database, flat file, XML Collaborate with team  Data Health dashboard, shared tests & auto-emailed reports Verify more data & do it quickly  verifies up to 100% of all data up to 1,000 x faster Integrate for Continuous Delivery  Integrates with most Build, ETL & QA management software
  • 11. Finding Bad Data SQL HQL SQL HQL SQL SQL  QS pulls data from data sources  QS pulls data from target data store  QS compares data quickly  QS generates reports, audit trails How? Reports, Data Health Dashboard built by QuerySurge™ Source Data Target Data
  • 12. QuerySurge™ Architecture Web-based… Installs on... Linux Connects to… …or any other JDBC compliant data source built by QuerySurge™ QuerySurge Controller QuerySurge Server QuerySurge Agents Flat Files
  • 13. Collaboration Testers - functional testing - regression testing - result analysis Developers / DBAs - unit testing - result analysis Data Analysts - review, analyze data - verify mapping failures Operations teams - monitoring - result analysis Managers - oversight - result analysis Share information on the built by QuerySurge™
  • 15. built by From a recent poll1 of: • Big Data Experts • Data Warehouse Architects • Solution Architects • ETL Architects Recent Survey: Data Experts Consensus Answer: 80% of data columns have no transformation at all Our Question: What % of columns in your Data Warehouse have no transformations at all? 1Poll conducted by RTTS on targeted LinkedIn groups Why is this important?
  • 16. Fast and Easy. No programming needed. built by QuerySurge™ QuerySurge™ Modules Compare by Table, Column & Row • Perform 80% of all data tests •Automatically generates SQL code • Opens up testing to novice & non- technical team members • Speeds up testing for skilled SQL coders • provides a huge Return-On-Investment
  • 17. built by QuerySurge™ QuerySurge™ Modules 3 Types of Data Comparison Wizards: The also provide you with automated features for: o filtering (‘Where’ clause) and o sorting (‘Order By’ clause) Column-Level Comparison: This is great for Big Data stores and Data Warehouses where tables will have some columns containing transformations and some columns with no transformations. Many tables and columns can be compared simultaneously and quickly. Table-Level Comparison: This comparator is great for Data Migrations and Database Upgrades with no transformations at all. Many tables can be compared simultaneously and quickly. Row Count Comparison: Great for all - Big Data stores, Data Warehouses, Data Migrations and Database Upgrades. Many tables and rows can be compared simultaneously and quickly.
  • 18. Uses: Tests the columns that have no transformations, which means it tests approximately 80% of your data store without you writing any SQL code Tests: Big data, data warehouses Value added: novice or non-technical: no coding needed, productive immediately experienced user: saves time built by QuerySurge™
  • 20. Uses: Verifies data loads when no transformation occurs Tests: data migrations, upgrades Value added: novice or non-technical: no coding needed experienced user: saves time built by QuerySurge™
  • 21. Use: Verify that the amount of rows from the source match the amount from the target Tests: Big data, data warehouse, data migration, database upgrades, data interfaces Value added: novice: no coding needed experienced user: saves time built by QuerySurge™ _________ Total
  • 22. built by QuerySurge™ all QuerySurge™ Modules Design Library SchedulingDeep-Dive Reporting Run Dashboard Query Wizards Data Health Dashboard
  • 23. Design Library • Create custom Query Pairs (source & target SQLs) • Great for team members skilled with SQL QuerySurge™ Modules Scheduling  Build groups of Query Pairs  Schedule Test Runs for: • immediately • at a specific date/time • automatically after build or ETL process built by QuerySurge™
  • 24. Deep-Dive Reporting  Examine and automatically email test results Run Dashboard  View real-time execution  Analyze real-time results QuerySurge™ Modules built by QuerySurge™
  • 25. QuerySurge Test Management Connectors built by QuerySurge™  Drive QuerySurge execution from your Test Management Solution  Outcome results (Pass/Fail/etc.) are returned from QuerySurge to your Test Management Solution  Results are linked in your Test Management Solution so that you can click directly into detailed QuerySurge results • HP ALM (Quality Center) • Microsoft Team Foundation Server • IBM Rational Quality Manager Integration with leading Test Management Solutions
  • 26. Licensing License Types:  Full User  Participant User (i.e. read-only) built by QuerySurge™ License Model for: Perpetual  own QuerySurgeTM, pay annual maintenance Subscription  use QuerySurgeTM for set period (12 months) QuerySurge™
  • 27. 8/18/2015 27 built by QuerySurge™ Training Courses Data Warehouse Testing • Data Warehouse & ETL Testing Fundamentals (1 day) • Fundamentals of QuerySurge (1 day) • Introduction to SQL for QuerySurge (1 day) • Advanced SQL techniques for QuerySurge (1 day) Big Data Testing • Big Data And ETL Testing Fundamentals • Introduction To Big Data Testing Using Hive And HQL Consulting RTTS, the software quality experts (and developer of QuerySurge), provides consulting solutions to the challenges of Big Data & Data Warehouse / ETL Testing • Jumpstart 2-week program – combines training courses, mentoring, consulting • Staff Augmentation – add additional RTTS resources to your team • Outsourcing - RTTS can perform all testing, including planning, design, execution
  • 28. built by QuerySurge™ Support • Live Chat through QuerySurge or web site • Email support through Zendesk • Yammer Network • Webex sessions • Phone support
  • 29. (1) Trial in the Cloud of QuerySurgeTM, including self-learning tutorial that works with sample data for 3 days (2) Downloaded Trial of QuerySurgeTM, including self-learning tutorial with sample data or your data for 15 days (3) Proof of Concept of QuerySurgeTM includes our team of experts assisting you for 30 days for more information on (1), (2) and (3), Go to querysurge.com/compare-trial-options TRIAL IN THE CLOUD built by QuerySurge™ Free TrialsQuerySurge™ Proof of Concept
  • 30. built by QuerySurge™ QuerySurge For more on the Query Wizards, go to querysurge.com/querysurge-query-wizards

Editor's Notes

  1. I typically read the quotes
  2. Talk through the process
  3. QuerySurge provides insight into the health of your data throughout your organization through BI dashboards and reporting at your fingertips. It is a collaborative tool that allows for distributed use of the tool throughout your organization and provides for a sharable, holistic view of your data’s health and your organization’s level of maturity of your data management.
  4. QuerySurge helps your team coordinate your data quality initiatives while speeding up your development and testing cycles and finding your bad data. Why risk having your team identify trends and develop strategic initiatives when the underlying data is incorrect? QuerySurge reduces this risk.
  5. QuerySurge finds bad data by natively connecting to: any data source, whether it is any type of database, flat file or xml and can connect to any data target, whether it is a db, file, xml, data warehouse or hadoop implementation. QuerySurge pulls data from the source and the target and compares them very quickly (typically in a few minutes) and then produces reports that show every data difference, even if there are millions of rows and hundreds of columns in the test. These reports can be automatically emailed to your team. You can pick from a multitude of reports or export the results so that you can build your own reports.
  6. Your distributed team from around the world can use any of these web browsers: Internet Explorer, Chrome, Firefox and Safari. Installs on operating systems: Windows & Linux. QS connects to any JDBC-compliant data source. Even if it is not listed here.
  7. QuerySurge can utilized by active practitioners such as testers & developers to create and launch tests, or by managers, analysts and operations to view data test results and the overall health of the data. QuerySurge facilitates this by providing 2 types of licenses: (1) full user & (2) participant user. (1) Full User – This type of user has unlimited access to create QueryPairs, Suites, and Scenarios. This user can also schedule and run tests, see results, run and export reports, and export data. Perfect for anyone creating and/or running data tests while performing analysis of results. (2) Participant User – This user cannot create or run tests, but has access to all other information - including viewing all query pairs, results, and reports, receiving email notifications, and exporting test results and reports. Perfect for managers, analysts, architects, DBAs, developers, and operations users who need to know the health of their data.