Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

QuerySurge - the automated Data Testing solution


Published on

QuerySurge is the leading Data Testing solution built specifically to automate the testing of Data Warehouses & Big Data. QuerySurge ensures that the data extracted from data sources remains intact in the target data store by analyzing and pinpointing any differences quickly.

And QuerySurge makes it easy for both novice and experienced team members to validate their organization's data quickly through Query Wizards while still allowing power users the flexibility they need.

All with deep dive reporting and data health dashboards that quickly provides you with a holistic view of your project’s data.

Types of Automated Data Testing
QuerySurge provides data testing solutions for all of your automated data testing needs

- Data Warehouse testing & ETL testing
- Big Data (Hadoop, NoSQL) testing
- Data Interface testing
- Data Migration testing
- Database Upgrade testing


Published in: Software
  • Be the first to comment

QuerySurge - the automated Data Testing solution

  1. 1. built by the automated Data Testing solution QuerySurge™
  2. 2. Where QuerySurge™ fits in your data strategy built by QuerySurge™
  3. 3. Business Intelligence (BI) software CxOs are using Business Intelligence & Analytics to make critical business decisions – with the assumption that the underlying data is fine. “The average organization loses $8.2 million annually through poor Data Quality.” - Gartner ETL Data Architecture The Executive Office and Critical Data potential problem areas
  4. 4. Current Business Case for Data Testing built by QuerySurge™ “46% of companies cite data quality as a barrier for adopting Business Intelligence products” - InformationWeek “On average, U.S. organizations believe 32% of their data is inaccurate” – Experian Data Quality research report “Poor data quality is a primary reason for 40% of all business initiatives failing to achieve their targeted benefits” - analyst firm Gartner “90% percent of U.S. companies have some sort of data quality solution in place today” – Experian Data Quality research report Data quality solutions are not enough!
  5. 5. o Profiling o Parsing and standardization o Generalized Cleansing o Matching o Monitoring o Enrichment o Subject-area-specific support o Metadata management o Configuration environment Data Quality tools vs. Data Testing tool built by QuerySurge™  Data Completeness  Data Transformation  Regression Testing Primary Characteristics of Data Quality tools courtesy of Gartner’s “Magic Quadrant for Data Quality Tools” Data Verification & Validation? Primary Characteristics of Data Testing tools Courtesy of the book "Testing the Data Warehouse Practicum" Data Verification & Validation?
  6. 6. Definitive Data Testing Book built by QuerySurge™
  7. 7. DataTesting Compare methods built by 1) Sampling (also known as “Stare & Compare” ) 2) Minus Queries
  8. 8. Method #1: Stare & Compare built by QuerySurge™ • Review Business Rules (i.e. Mapping Document: data flow mapping, data movement requirements) • Write Tests in SQL editor • Execute 2 Tests: 1 at Source & 1 at Target • Dump results to 2 Excel files • Compare results by eye (‘Stare & Compare’ or ‘sampling’) Issue with Stare & Compare: Impossible to visually compare billions of data sets. Result: usually less than 1% of data is compared Example: Current QuerySurge customer has: • a single test with 100 million rows & 200 columns • = 20 billion data sets • the client has > 7,000 total tests
  9. 9. built by QuerySurge™ MINUS QUERIES subtract one result set from another result set to show difference Comment: MINUS QUERIES need to be executed 2x (Source MINUS Target; Target MINUS Source) Result sets may not be accurate when dealing with duplicate rows of data No historical data from past testing – audit and regulatory issues Processing of minus queries puts pressure on the servers Double execution means 2x testing time and resource utilization Method #2: Minus Queries Minus Query #1: Table_1 MINUS Table_2 Minus Query #2: Table_2 MINUS Table_1 Result Set #1 Result Set #2 ISSUES with MINUS QUERIES Write 2 MINUS queries in SQL editor Execute MINUS queries 2x
  10. 10. DataTesting Compare Methods: 2 issues built by QuerySurge™ 1) There is a fundamental issue with both current methods: The assumption that all team members can write SQL/HQL code 2) Neither method fully satisfies any of the conditions below: Data Completeness Data Transformation Regression Testing
  11. 11. About QuerySurge™ built by
  12. 12. What is QuerySurge™? the collaborative Data Testing solution that finds bad data & provides a holistic view of your data’s health built by
  13. 13. the QuerySurge advantage built by QuerySurge™ Automate the entire testing cycle  Automate the launch, tests, comparison, auto-emailed results Create Tests easily with no SQL programming  Query Wizards ensure minimal time & effort to create tests Test across different platforms  Data Warehouse, Hadoop, NoSQL, database, flat file, XML Collaborate with team  Data Health dashboard, shared tests & auto-emailed reports Verify more data & do it quickly  verifies up to 100% of all data up to 1,000 x faster Integrate for Continuous Delivery (DevOps)  Integrates with most Build, ETL & QA management software
  14. 14. Flat Files QuerySurge™ Architecture Web-based… Installs on... Linux Connects to… …or any other JDBC compliant data source built by QuerySurge™ QuerySurge Controller QuerySurge Server QuerySurge Agents
  15. 15. SQL HQL SQL HQL SQL SQL  QS pulls data from data sources  QS pulls data from target data store  QS compares data quickly  QS generates reports, audit trails How QuerySurge Works Reports, Data Health Dashboard, auto emails built by QuerySurge™ Source Data Target Data Data Stores • Databases • Data Warehouses • Data Marts Flat Files • Fixed Width • Delimited • Excel Big Data stores • Hadoop • NoSQL Data Warehouses XML Web Services
  16. 16. Data Process: Developer & Tester built by QuerySurge™ Developer: Codes data movement based on Business Requirements Tester: Tests data movement based on Business Requirements Business Intelligence ETL Source Data Big Data ETL Process Target DWH
  17. 17. Collaboration Testers - functional testing - regression testing - result analysis Developers / DBAs - unit testing - result analysis Data Analysts - review, analyze data - verify mapping failures Operations teams - monitoring - result analysis Managers - oversight - result analysis Share information on the built by QuerySurge™
  18. 18. built by QuerySurge™ QuerySurge™ Modules Design Library SchedulingRun Dashboard Deep-Dive Reporting Data Health Dashboard Query Wizards
  19. 19. Design Library • Create Query Pairs (source & target SQLs) • Great for team members skilled with SQL QuerySurge™ Modules Scheduling  Build groups of Query Pairs  Schedule Test Runs built by QuerySurge™
  20. 20. Deep-Dive Reporting  Examine and automatically email test results Run Dashboard  View real-time execution  Analyze real-time results QuerySurge™ Modules built by QuerySurge™
  21. 21. built by QuerySurge™ • view data reliability & pass rate • add, move, filter, zoom-in on any data widget & underlying data • verify build success or failure QuerySurge™ Modules
  22. 22. Fast and Easy. No programming needed. built by QuerySurge™ QuerySurge™ Modules • Perform 80% of all data tests - no SQL coding needed • Opens up testing to novices & non-technical team members • Speeds up testing for skilled SQL coders • provides a huge Return-On-Investment
  23. 23. QuerySurge Test Management Connectors built by QuerySurge™  Drive QuerySurge execution from your Test Management Solution  See QuerySurge Pass/Fail results in your Test Management solution  Click link to drill into detailed results in QuerySurge • HP ALM (Quality Center) • Microsoft Team Foundation Server • IBM Rational Quality Manager Integration with leading Test Management Solutions
  24. 24. QuerySurge & DevOps: Continuous Delivery & Integration built by QuerySurge™ Automated Testing Automated Reporting Automated Launch Data Integration/ETL solutions QuerySurge™ and many others… email report Test Management solutions QuerySurge™ email report and many others… QuerySurge™ Automated Build solutions email report
  25. 25. • Reduce your costs & risks • Improve your data quality • Accelerate your testing cycles • Share information with your team built by QuerySurge™ • Realize a huge ROI (like 1,600%) QuerySurge’s Impact
  26. 26. CustomersQuerySurge™
  27. 27. built by QuerySurge™ About FACTS Founded: 1996 headquarters: Manhattan, New York Customer profile: • Fortune 1000 • 600+ customers Strategic Partners: IBM, Microsoft, HP, Oracle, Teradata, HortonWorks, Cloudera, MongoDB Software Division: QuerySurge RTTS is the parent company of QuerySurge and is the premier pure-play QA & Testing organization that specializes in test automation
  28. 28. QuerySurge built by built by QuerySurge™ You