Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Webinar:
automating the data testing
for
Bill Hayduk
CEO, President
RTTS
Jeff Bocarsly, PhD
VP & Chief Architect
RTTS
Pres...
built by
QuerySurge™
About
FACTS
Founded:
1996
Locations:
New York (HQ), Atlanta,
Philadelphia, Phoenix
Strategic Partners...
about
built by
QuerySurge™
What is MongoDB?
Name: MongoDB (from "humongous")
1NoSQL means now “not only SQL”
210gen changed its name to MongoDB, Inc....
built by
QuerySurge™
• Online real-time processing
• Data set is smaller
• Measured in milliseconds
• Offline big data pro...
MongoDB Use Cases
built by
QuerySurge™
Source: MongoDB, Inc.
Data Warehouse Batch Aggregation
ETL from MongoDB
ETL to Mong...
Use Cases: Data Warehouse
Relational DB & Data
Warehousing
Source Data
@
BI, Analytics &
Reporting
built by
QuerySurge™
In...
Data Quality Issues
built by
QuerySurge™
Data Quality Best Practices boost revenue by 66%.
The average organization loses ...
News Headlines
built by
QuerySurge™
Validating Data: 3 Big Issues
- need to verify more data and to do it faster
- need to automate the testing effort
- need ...
What is QuerySurge™?
a collaborative
data testing tool that
finds bad data & provides
a holistic view of your
data’s healt...
• Reduce your costs & risks
• Improve your data quality
• Accelerate your testing cycles
• Share information with your tea...
Finding Bad Data
SQL
SQL
SQL
SQL
SQL
SQL
 QS pulls data from data source(s)
 QS pulls data from data target(s)
 QS comp...
the QuerySurge advantage
built by
QuerySurge™
Automate the entire testing cycle
 Automate kickoff, tests, comparison, aut...
Collaboration
Testers
- functional testing
- regression testing
- result analysis
Developers / DBAs
- unit testing
- resul...
QuerySurge™ Architecture
Web-based…
Installs on...
Linux
Connects to…
…or any other JDBC compliant data source
built by
Qu...
built by
QuerySurge™
QuerySurge™ Modules
Design Library
SchedulingDeep-Dive Reporting
Run Dashboard
Query Wizards
Data Hea...
built by
From a recent poll1 of:
• Big Data Experts
• Data Warehouse Architects
• Solution Architects
• ETL Architects
Rec...
Fast and Easy.
No programming needed.
built by
QuerySurge™
QuerySurge™ Modules
Compare by Table, Column & Row
• Perform 80...
built by
QuerySurge™
QuerySurge™ Modules:
(we picked Column-Level Comparison)
Design Library
• Create Query Pairs (source & target SQLs)
• Great for team members skilled with SQL
QuerySurge™ Modules
S...
Deep-Dive Reporting
 Examine and automatically
email test results
Run Dashboard
 View real-time execution
 Analyze real...
built by
QuerySurge™
• view data reliability & pass rate
• add, move, filter, zoom-in on any data
widget & underlying data...
QuerySurge Test Management Connectors
built by
QuerySurge™
 Drive QuerySurge execution from your Test Management Solution...
Use Case: Big Data and
Relational DB & Data
Warehousing
Source Data
@
BI, Analytics &
Reporting
Ingestion
built by
QuerySu...
Value-Add
QuerySurge provides value by either:
in testing data coverage from < 1% to
upwards of 100%
in testing time by as...
Return on Investment
QuerySurge provides an increase in better data due to shorter / more
thorough testing cycle - saving ...
Contact us if your team would like:
(1) a Trial in the Cloud of QuerySurge, including self-learning
tutorial that works wi...
Upcoming SlideShare
Loading in …5
×

Big Data Testing: Ensuring MongoDB Data Quality

54,386 views

Published on

You've made the move to MongoDB for its flexible schema and querying capabilities in order to enhance agility and reduce costs for your business. Shouldn't your data quality process be just as organized and efficient?

Using QuerySurge for testing your MongoDB data as part of your quality effort will increase your testing speed, boost your testing coverage (up to 100%), and improve the level of quality within your Big Data store. QuerySurge will help you keep your team organized and on track too!

To learn more about QuerySurge, visit www.QuerySurge.com

Published in: Software
  • Be the first to comment

Big Data Testing: Ensuring MongoDB Data Quality

  1. 1. Webinar: automating the data testing for Bill Hayduk CEO, President RTTS Jeff Bocarsly, PhD VP & Chief Architect RTTS Presenters Ensuring MongoDB Data Quality built by QuerySurge™
  2. 2. built by QuerySurge™ About FACTS Founded: 1996 Locations: New York (HQ), Atlanta, Philadelphia, Phoenix Strategic Partners: IBM, Microsoft, HP, Oracle, Teradata, HortonWorks, Cloudera, Amazon Software: QuerySurge RTTS is the leading provider of software & data quality for critical business systems
  3. 3. about built by QuerySurge™
  4. 4. What is MongoDB? Name: MongoDB (from "humongous") 1NoSQL means now “not only SQL” 210gen changed its name to MongoDB, Inc. Source: Wikipedia built by QuerySurge™ • classified as a NoSQL1 database • does not implement the table-based relational db structure • cross-platform document-oriented database • makes the integration of data in certain types of apps easier & faster • free and open source • originally built by 10gen2 and released in 2009 “MongoDB is in 5th place as the most popular type of database management system, and 1st place for NoSQL database management systems.” April 2014
  5. 5. built by QuerySurge™ • Online real-time processing • Data set is smaller • Measured in milliseconds • Offline big data processing • Offline analytics • Measured in minutes & hours MongoDB versus Hadoop Source: classpattern.com When use MongoDB? / When use Hadoop?
  6. 6. MongoDB Use Cases built by QuerySurge™ Source: MongoDB, Inc. Data Warehouse Batch Aggregation ETL from MongoDB ETL to MongoDB
  7. 7. Use Cases: Data Warehouse Relational DB & Data Warehousing Source Data @ BI, Analytics & Reporting built by QuerySurge™ Ingestion
  8. 8. Data Quality Issues built by QuerySurge™ Data Quality Best Practices boost revenue by 66%. The average organization loses $8.2 million annually through poor Data Quality. 46% of companies cite Data Quality as a barrier for adopting Business Intelligence products. 80% of organizations… will underestimate the costs related to the data acquisition tasks by an average of 50 percent.
  9. 9. News Headlines built by QuerySurge™
  10. 10. Validating Data: 3 Big Issues - need to verify more data and to do it faster - need to automate the testing effort - need to be able to test across different platforms Need a testing tool! built by QuerySurge™
  11. 11. What is QuerySurge™? a collaborative data testing tool that finds bad data & provides a holistic view of your data’s health built by QuerySurge™
  12. 12. • Reduce your costs & risks • Improve your data quality • Accelerate your testing cycles • Share information with your team with QuerySurge™ you can: built by QuerySurge™ • Provides huge ROI (i.e. 1,300%)* *based on client’s calculation of Return on Investment
  13. 13. Finding Bad Data SQL SQL SQL SQL SQL SQL  QS pulls data from data source(s)  QS pulls data from data target(s)  QS compares data in seconds  QS generates reports, audit trails How? reports built by QuerySurge™
  14. 14. the QuerySurge advantage built by QuerySurge™ Automate the entire testing cycle  Automate kickoff, tests, comparison, auto-emailed results Create Tests easily with no SQL programming  ensures minimal time & effort to create tests / obtain results Test across different platforms  data warehouse, Hadoop, NoSQL, database, flat file, XML Collaborate with team  Data Health dashboard, shared tests & auto-emailed reports Verify more data & do it quickly  verifies up to 100% of all data up to 1,000 x faster Integrate for Continuous Delivery  Integrates with most Build, ETL & QA management software
  15. 15. Collaboration Testers - functional testing - regression testing - result analysis Developers / DBAs - unit testing - result analysis Data Analysts - review, analyze data - verify mapping failures Operations teams - monitoring - result analysis Managers - oversight - result analysis Share information on the built by QuerySurge™
  16. 16. QuerySurge™ Architecture Web-based… Installs on... Linux Connects to… …or any other JDBC compliant data source built by QuerySurge™ QuerySurge Controller QuerySurge Server QuerySurge Agents Flat Files
  17. 17. built by QuerySurge™ QuerySurge™ Modules Design Library SchedulingDeep-Dive Reporting Run Dashboard Query Wizards Data Health Dashboard
  18. 18. built by From a recent poll1 of: • Big Data Experts • Data Warehouse Architects • Solution Architects • ETL Architects Recent Survey: Data Experts Consensus Answer: 80% of data columns have no transformation at all Our Question: What % of columns in your Data Warehouse have no transformations at all? 1Poll conducted by RTTS on targeted LinkedIn groups Why is this important?
  19. 19. Fast and Easy. No programming needed. built by QuerySurge™ QuerySurge™ Modules Compare by Table, Column & Row • Perform 80% of all data tests - no SQL coding needed • Opens up testing to novices & non-technical team members • Speeds up testing for skilled SQL coders • provides a huge Return-On-Investment
  20. 20. built by QuerySurge™ QuerySurge™ Modules: (we picked Column-Level Comparison)
  21. 21. Design Library • Create Query Pairs (source & target SQLs) • Great for team members skilled with SQL QuerySurge™ Modules Scheduling  Build groups of Query Pairs  Schedule Test Runs built by QuerySurge™
  22. 22. Deep-Dive Reporting  Examine and automatically email test results Run Dashboard  View real-time execution  Analyze real-time results QuerySurge™ Modules built by QuerySurge™
  23. 23. built by QuerySurge™ • view data reliability & pass rate • add, move, filter, zoom-in on any data widget & underlying data • verify build success or failure
  24. 24. QuerySurge Test Management Connectors built by QuerySurge™  Drive QuerySurge execution from your Test Management Solution  Outcome results (Pass/Fail/etc.) are returned from QuerySurge to your Test Management Solution  Results are linked in your Test Management Solution so that you can click directly into detailed QuerySurge results • HP ALM (Quality Center) • Microsoft Team Foundation Server • IBM Rational Quality Manager Integration with leading Test Management Solutions
  25. 25. Use Case: Big Data and Relational DB & Data Warehousing Source Data @ BI, Analytics & Reporting Ingestion built by QuerySurge™ ™
  26. 26. Value-Add QuerySurge provides value by either: in testing data coverage from < 1% to upwards of 100% in testing time by as much as 1,000 x combination of in test coverage while in testing time 26 built by QuerySurge™
  27. 27. Return on Investment QuerySurge provides an increase in better data due to shorter / more thorough testing cycle - saving $$$. 27 built by QuerySurge™ Pharmaceutical Organization Saves $288,000 in Clinical Trials Data Migration Testing Project 1Since 2010, the pharmaceutical industry has been assessed over $13 billion in fines. Source: wikipedia http://en.wikipedia.org/wiki/List_of_largest_pharmaceutical_settlements This savings does not include savings from avoiding fines from regulatory bodies or lawsuits.1 Total Savings
  28. 28. Contact us if your team would like: (1) a Trial in the Cloud of QuerySurge, including self-learning tutorial that works with sample data for 3 days or (2) a downloaded Trial of QuerySurge, including self-learning tutorial with sample data or your data for 15 days or (3) a Proof of Concept of QuerySurge, including a kickoff & setup meeting and weekly meetings with our team of experts for 30 days http://www.querysurge.com/compare-trial-optionsfor more information, Go here QuerySurge built by QuerySurge™ TRIAL IN THE CLOUD

×