Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Test Automation for Data Warehouses


Published on

BI went Agile, but Testing didn’t. Presented by Stefaan De Vos (Automade)

Published in: Data & Analytics
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ ◀ ◀ ◀ ◀
    Are you sure you want to  Yes  No
    Your message goes here

Test Automation for Data Warehouses

  1. 1. Automade Test Automation Data Vault and Data Warehouse Automation 9th of December Stefaan De Vos
  2. 2. Test Automation
  3. 3. Challenges • Lead times of BI / Analytical projects are ever decreasing • Tools • Automation (Wherescape...) • Virtualization (Denodo, CIS...) • Appliances (Netezza, DATAllegro...) • Methodologies • Data Vault • Agile DWH BI went Agile, but Testing didn’t
  4. 4. Challenges • Data • Larger data volumes • IoT • Unstructured data / poor data quality • Social networks • No relevant test data sets available • The number of possible test cases are near infinite Data Volume & Quality
  5. 5. Challenges Review requirements Review requirements Define testing Strategy Define testing Strategy Prepare Test DataPrepare Test DataEntry / Exit criteria Design Test Cases & Scripts Design Test Cases & Scripts Configure the test environment Configure the test environment Prepare & Design Agree on Entry / Exit test criteria Agree on Entry / Exit test criteria Integration testing Integration testing Performance testingPerformance testing Acceptance testingAcceptance testing - Basic testing - DI jobs accessible - Reports accessible - Cleansed - Complete - Correct - Integrated - Valid reports - Relevant data - Available data - Consistent data - Accessible - DI & BI integration - Full test cycles - Check NFRs - Scalable - Check SLAs - Peak user - Peak loads - Functional - User - Production - SME - End user Construct Defect metrics review Defect metrics review Performance statistics Performance statistics Lessons learntLessons learnt Process & Quality Improvement Accept Data Completeness testing Data Completeness testing BI & Analytical Testing BI & Analytical Testing Smoke test Unit test Smoke test Unit test BI Testing Lifecycle
  6. 6. Challenges ** Regulatory compliance might require to use the v-model, e.g. validated environments. Functional analysis Requirements functional Non-functional Technical Design Construction Unit Smoke Test Data Completeness BI & Analytical Integration testing Performance User Production AcceptanceValidation Verification BI Testing V-Model
  7. 7. Challenge • Lots of data • Lots of testing to be done • Little time to do it Summary
  8. 8. Test Automation
  9. 9. Complete DWH Testing • Complete testing is a must • It requires: • Testing methodology • Right project culture/mindset and organization • Tools Our View
  10. 10. Complete DWH Testing Database integrity checking. Risk based testing Effective defect management and collaboration End to End Performance testing Adherence to compliance and regulatory standards and…AUTOMATE Critical Success Factors
  11. 11. Test Automation Leverage social development principles to deepen functionalities Testers - functional testing - regression testing - result analysis Developers / DBAs - unit testing - result analysis Data Analysts - review, analyze data - verify mapping failures Operations teams - monitoring - result analysis Collaboration/WorkflowTestManagement Rational Quality Manager JIRA Team Foundation ServerQuality Center
  12. 12. Test Automation
  13. 13. What to Automate? Complex Functional SQL Validation Reconciliation
  14. 14. Test Automation Basic functionality • Auto detection of anomalies (or at least prior to being detected by a user) • Targeted regression testing for planned changes • Data error identification errors by comparing (huge) data result sets • Data error detection via a rules engine • In between data layer reconciliation and auditing • Test data generation Additional functionality • No programming or coding • Heterogeneous connectivity • Collaboration and workflow capabilities • Visually attractive development and monitoring environment • Intuitive reports & dashboards Functionality
  15. 15. Test Automation • Shorten regression cycles • Save report developers time • Test the same data set in less time • Test more • Faster deployment of defect resolution cycle • Faster deployment of enhancements • Less cumbersome upgrades/migrations • Enabler for • Implementing continuous testing • operationalization of testing Benefits
  16. 16. Test Automation Vendors - tools • ICEDQ • RTTS - QuerySurge Unconnected ETL Source Data Layer Target Data Layer ETL - ELT 1 3 2 Load test data Execute Job externally Extract result 4 Compare & report • Validation of the business rules implemented in ETL processes are assessed by comparing the results against a ground truth. • The ETL processes are executed separated from the test automation tool • Less adequate for multi-step ETL processes. Approach
  17. 17. Test Automation Vendors - tools • Zuzena Connected ETL • The business rules present in the ETL tool are analyzed and the test results are assessed against the anticipated results. • The test automation tool executes the ETL processes. Approach Source Data Layer Target Data Layer 2 4Load test data Extract result 5 Compare & report 1 Analyze logic 3 Run job
  18. 18. Test Automation Vendors - tools • Report Valid8tor (BO) • 360Bind (BO) • Integrity Manager (MSTR) • Report Validator - BSP Software (Cognos) Report Integrity validator • Parallel testing of 2 live systems • Comparing against a (historical) ground truth • Comparing against known good baselines Approach Reports Reporting data Layer Load test data Scrape data 4 Compare & report 1 3 Run report 2
  19. 19. Test automation
  20. 20. QuerySurge • Integrates with HP QC / IBM RQM / MSFT TFS • Provides collaboration features pulls data from data sources pulls data from target data store compares data quickly generates reports, audit trails reports SQL Design Tests Scheduling Reporting Run Dashboard Wizards Data Health Dashboard • a SQL execution and data comparison tool working against heterogeneous datasources. • No programming needed
  21. 21. Automade introduction • Automade • Provides an agile answer to the ever-increasing information appetite • By automating the mind-numbing aspects of constructing, maintaining and testing of data warehouses. • Automade is a spinoff of MindThegap and has established partnerships with
  22. 22. Test automation Thank you for listening, Any questions? Feel free to send questions & feedback to