Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Query Wizards - data testing made easy - no programming

11,436 views

Published on

Fast and easy. No Programming needed. The latest QuerySurge release introduces the new Query Wizards. The Wizards allow both novice and experienced team members to validate their organization's data quickly with no SQL programming required.

The Wizards provide an immediate ROI through their ease-of-use and ensure that minimal time and effort are required for developing tests and obtaining results. Even novice testers are productive as soon as they start using the Wizards!

According to a recent survey of Data Architects and other data experts on LinkedIn, approximately 80% of columns in a data warehouse have no transformations, meaning the Wizards can test all of these columns quickly & easily, (The columns with transformations can be tested using the QuerySurge Design library using custom SQL coding.)

There are 3 Types of automated Data Comparisons:
- Column-Level Comparison
- Table-Level Comparison
- Row Count Comparison

There are also automated features for filtering (‘Where’ clause) and sorting (‘Order By’ clause).

The Wizards provide both novices and non-technical team members with a fast & easy way to be productive immediately and speed up testing for team members skilled in SQL.

Trial our software either as a download or in the cloud at www.QuerySurge.com. The trial comes with a built-in tutorial and sample data.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Query Wizards - data testing made easy - no programming

  1. 1. built by Introducing the new We just made data testing REALLY EASY! No programming needed Automate your Data Warehouse & Big Data Testing and Reap the Benefits *available for download on August 3, 2015
  2. 2. built by QuerySurge™ About FACTS Founded: 1996 Locations: New York (HQ), Atlanta, Philadelphia, Phoenix Strategic Partners: IBM, Microsoft, HP, Oracle, Teradata, HortonWorks, Cloudera, Amazon Software: QuerySurge RTTS is the leading provider of software & data quality for critical business systems
  3. 3. “70% of enterprises have either deployed or are planning to deploy big data projects and programs this year” – analyst firm IDG “46% of companies cite data quality as a barrier for adopting Business Intelligence products.” - InformationWeek “Poor data quality is a primary reason for 40% of all business initiatives failing to achieve their targeted benefits.” - analyst firm Gartner Data Quality Issues built by QuerySurge™
  4. 4. 2 Prevalent DataTesting Strategies built by QuerySurge™ 1) Stare & Compare 2) Minus Queries
  5. 5. DataTesting Strategy #1: Stare & Compare built by QuerySurge™ • Review Mapping Document (business rules, data flow mapping, data movement requirements) • Write Tests in SQL editor • Execute 2 Tests: 1 at Source & 1 at Target • Dump results to 2 Excel files • Compare results by eye (Stare & Compare) Difficulty with Stare & Compare: Impossible to visually compare millions/billions of data sets visually. Example: Current QuerySurge customer has: • a single test with 100 million rows • 200 columns • = 20 billion data sets • the client has > 7,000 total tests Data Tester’s Current Process
  6. 6. built by QuerySurge™ MINUS QUERIES subtract one result set from another result set to show difference Comment: MINUS QUERIES need to be executed 2x (Source MINUS Target; Target MINUS Source) Result sets may not be accurate when dealing with duplicate rows of data No historical data from past testing – audit and regulatory issues Processing of minus queries puts pressure on the servers Double execution means 2x testing time and resource utilization Potential for false positives (bad data could exist on both sides of an ETL leg) DataTesting Strategy #2: Minus Queries Minus Query #1: Table_1 MINUS Table_2 Minus Query #2: Table_2 MINUS Table_1 Result Set #1 Result Set #2 ISSUES with MINUS QUERIES Write 2 MINUS queries in SQL editor Execute MINUS queries 2x
  7. 7. DataTesting Strategies built by QuerySurge™ a fundamental issue with the 2 Strategies: Assumption that all team members understand and can write SQL code
  8. 8. What is QuerySurge™? the collaborative Data Testing solution that finds bad data & provides a holistic view of your data’s health built by
  9. 9. • Reduce your costs & risks • Improve your data quality • Accelerate your testing cycles • Share information with your team with QuerySurge™ you can: built by QuerySurge™ • Provides huge ROI (i.e. 1,300%)* *based on client’s calculation of Return on Investment
  10. 10. the QuerySurge advantage built by QuerySurge™ Automate the entire testing cycle  Automate kickoff, tests, comparison, auto-emailed results Create Tests easily with no SQL programming  ensures minimal time & effort to create tests / obtain results Test across different platforms  data warehouse, Hadoop, NoSQL, database, flat file, XML Collaborate with team  Data Health dashboard, shared tests & auto-emailed reports Verify more data & do it quickly  verifies up to 100% of all data up to 1,000 x faster Integrate for Continuous Delivery  Integrates with most Build, ETL & QA management software
  11. 11. Finding Bad Data SQL HQL SQL HQL SQL SQL  QS pulls data from data sources  QS pulls data from target data store  QS compares data quickly  QS generates reports, audit trails How? Reports, Data Health Dashboard built by QuerySurge™ Source Data Target Data
  12. 12. QuerySurge™ Architecture Web-based… Installs on... Linux Connects to… …or any other JDBC compliant data source built by QuerySurge™ QuerySurge Controller QuerySurge Server QuerySurge Agents Flat Files
  13. 13. Collaboration Testers - functional testing - regression testing - result analysis Developers / DBAs - unit testing - result analysis Data Analysts - review, analyze data - verify mapping failures Operations teams - monitoring - result analysis Managers - oversight - result analysis Share information on the built by QuerySurge™
  14. 14. built by QuerySurge™
  15. 15. built by From a recent poll1 of: • Big Data Experts • Data Warehouse Architects • Solution Architects • ETL Architects Recent Survey: Data Experts Consensus Answer: 80% of data columns have no transformation at all Our Question: What % of columns in your Data Warehouse have no transformations at all? 1Poll conducted by RTTS on targeted LinkedIn groups Why is this important?
  16. 16. Fast and Easy. No programming needed. built by QuerySurge™ QuerySurge™ Modules Compare by Table, Column & Row • Perform 80% of all data tests •Automatically generates SQL code • Opens up testing to novice & non- technical team members • Speeds up testing for skilled SQL coders • provides a huge Return-On-Investment
  17. 17. built by QuerySurge™ QuerySurge™ Modules 3 Types of Data Comparison Wizards: The also provide you with automated features for: o filtering (‘Where’ clause) and o sorting (‘Order By’ clause) Column-Level Comparison: This is great for Big Data stores and Data Warehouses where tables will have some columns containing transformations and some columns with no transformations. Many tables and columns can be compared simultaneously and quickly. Table-Level Comparison: This comparator is great for Data Migrations and Database Upgrades with no transformations at all. Many tables can be compared simultaneously and quickly. Row Count Comparison: Great for all - Big Data stores, Data Warehouses, Data Migrations and Database Upgrades. Many tables and rows can be compared simultaneously and quickly.
  18. 18. Uses: Tests the columns that have no transformations, which means it tests approximately 80% of your data store without you writing any SQL code Tests: Big data, data warehouses Value added: novice or non-technical: no coding needed, productive immediately experienced user: saves time built by QuerySurge™
  19. 19. built by QuerySurge™
  20. 20. Uses: Verifies data loads when no transformation occurs Tests: data migrations, upgrades Value added: novice or non-technical: no coding needed experienced user: saves time built by QuerySurge™
  21. 21. Use: Verify that the amount of rows from the source match the amount from the target Tests: Big data, data warehouse, data migration, database upgrades, data interfaces Value added: novice: no coding needed experienced user: saves time built by QuerySurge™ _________ Total
  22. 22. built by QuerySurge™ all QuerySurge™ Modules Design Library SchedulingDeep-Dive Reporting Run Dashboard Query Wizards Data Health Dashboard
  23. 23. Design Library • Create custom Query Pairs (source & target SQLs) • Great for team members skilled with SQL QuerySurge™ Modules Scheduling  Build groups of Query Pairs  Schedule Test Runs for: • immediately • at a specific date/time • automatically after build or ETL process built by QuerySurge™
  24. 24. Deep-Dive Reporting  Examine and automatically email test results Run Dashboard  View real-time execution  Analyze real-time results QuerySurge™ Modules built by QuerySurge™
  25. 25. QuerySurge Test Management Connectors built by QuerySurge™  Drive QuerySurge execution from your Test Management Solution  Outcome results (Pass/Fail/etc.) are returned from QuerySurge to your Test Management Solution  Results are linked in your Test Management Solution so that you can click directly into detailed QuerySurge results • HP ALM (Quality Center) • Microsoft Team Foundation Server • IBM Rational Quality Manager Integration with leading Test Management Solutions
  26. 26. Licensing License Types:  Full User  Participant User (i.e. read-only) built by QuerySurge™ License Model for: Perpetual  own QuerySurgeTM, pay annual maintenance Subscription  use QuerySurgeTM for set period (12 months) QuerySurge™
  27. 27. 8/18/2015 27 built by QuerySurge™ Training Courses Data Warehouse Testing • Data Warehouse & ETL Testing Fundamentals (1 day) • Fundamentals of QuerySurge (1 day) • Introduction to SQL for QuerySurge (1 day) • Advanced SQL techniques for QuerySurge (1 day) Big Data Testing • Big Data And ETL Testing Fundamentals • Introduction To Big Data Testing Using Hive And HQL Consulting RTTS, the software quality experts (and developer of QuerySurge), provides consulting solutions to the challenges of Big Data & Data Warehouse / ETL Testing • Jumpstart 2-week program – combines training courses, mentoring, consulting • Staff Augmentation – add additional RTTS resources to your team • Outsourcing - RTTS can perform all testing, including planning, design, execution
  28. 28. built by QuerySurge™ Support • Live Chat through QuerySurge or web site • Email support through Zendesk • Yammer Network • Webex sessions • Phone support
  29. 29. (1) Trial in the Cloud of QuerySurgeTM, including self-learning tutorial that works with sample data for 3 days (2) Downloaded Trial of QuerySurgeTM, including self-learning tutorial with sample data or your data for 15 days (3) Proof of Concept of QuerySurgeTM includes our team of experts assisting you for 30 days for more information on (1), (2) and (3), Go to querysurge.com/compare-trial-options TRIAL IN THE CLOUD built by QuerySurge™ Free TrialsQuerySurge™ Proof of Concept
  30. 30. built by QuerySurge™ QuerySurge For more on the Query Wizards, go to querysurge.com/querysurge-query-wizards

×