Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Data Analytics-testing spectrum

375 views

Published on

vodQA 2016

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Data Analytics-testing spectrum

  1. 1. Kokila Rudresh Shalini Saini DataAnalytics–TestingSpectrum V o d Q A 2 0 1 6
  2. 2. Data Analytics: An Introduction Collection Processing Modelling Inference Visualization
  3. 3. Data Analytics: Use Cases Business Intelligence Social Networks Astronomy and Astrophysics Finance and Stock Market Medical Imaging Computer Graphics Computer Vision Energy ExplorationMaps Retail
  4. 4. Data Analytics: Why Testing is Important Volume Domain Complexity Variety Computations Testing
  5. 5. Data Analytics: Testing Challenges Data Validation Model Implementation Business Perspective
  6. 6. Data Analytics: Typical System Implementation Extract Transform Load Source Data Modelling AggregationETL VisualizationRaw Data
  7. 7. Source Data Extract Transform Load Source Data
  8. 8. ETL Process Extract Transform Load Source Data
  9. 9. Modelling Extract Transform Load Source Data
  10. 10. Aggregation Extract Transform Load Source Data
  11. 11. Visualization Extract Transform Load Source Data
  12. 12. Data Analytics Testing - Approach Extract Transform Load Source Data Pre-ETL Validations Post-ETL Tests Model Validations Aggregation Validations Visualization Validations
  13. 13. Format Consistency Completeness Data Analytics - Testing Extract Transform Load Source Data Pre-ETL Validations
  14. 14. Pre ETL Testing
  15. 15. Data Analytics - Testing Extract Transform Load Source Data Post-ETL Tests Meta-data Data transformation Data quality checks Business-specific validations
  16. 16. Post ETL Testing
  17. 17. Data Analytics - Testing Extract Transform Load Source Data Model Validations Implementation Computation
  18. 18. Model Implementation Testing Sales = a(Seasonality) + b(Trend) + c(Promotions) + d(Sales Channel) + other factors
  19. 19. Data Analytics - Testing Extract Transform Load Source Data Aggregation Validations Data Hierarchy Data Scope Summarized Values
  20. 20. Data Analytics - Testing Extract Transform Load Source Data Visualization Validations Information Representation Data Format Result Intuitiveness
  21. 21. Visualization Testing
  22. 22. Learnings ANALYSE CODETEST Initial Data Flow • Pre defined data template • Pre-ETL data validations Domain Knowledge • KT Sessions involving SME’s • Core computations Business Involvement • Test data closer to real time data • User flows prioritization
  23. 23. Learnings Implementation • Alternate implementation • SME validation` Computation • Addressing the right problem • Computational Factors ANALYSE CODETEST
  24. 24. Learnings Testing Process • Step wise data validation • Defect investigation Test Automation • Data combinations • Xml test data Test Execution • CI test execution • Execution frequency Testing Tools • Spreadsheet gear • Excel macros ANALYSE CODETEST
  25. 25. Domain Context Integrating Business Use-cases Design and Testing Challenges Testing Approach Learnings Summary
  26. 26. kokila@thoughtworks.com sshalini@thoughtworks.com

×