Regression Testing OBIEE without the fuss

1,504 views
1,356 views

Published on

Just because OBIEE is presented to users through a web browser that doesn't mean the regression testing has to take place there too. This presentation will demonstrate how you can efficiently and effectively test the bulk of changes made to your system without going anywhere near the front end. Where front end testing is needing, it demonstrates the concept of visual regression testing using the latest tools. With hands-on examples of the tools involved this will be a practical presentation giving anyone interested in the topic a headstart in practical implementation.

Video available here: http://ritt.md/regression-testing-video

Published in: Data & Analytics, Technology
1 Comment
1 Like
Statistics
Notes
No Downloads
Views
Total views
1,504
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
52
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Regression Testing OBIEE without the fuss

  1. 1. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing OBIEE Without the Fuss
 Robin Moffatt Principal Consultant, Rittman Mead
  2. 2. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Smart Regression Testing Do better OBIEE development and deliver it more rapidly
  3. 3. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Why Regression Test? Confidence
  4. 4. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Why? Confidence Agility in development Happier users
  5. 5. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead What? •Regression Testing is not just a prolonged exercise on a Gantt chart, just before go live •Smart Regression Testing can be run on demand, targeting the change made and assuring quality during development
  6. 6. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Detect Problems Early ‣Fresh in the developer’s mind ‣Easier to fix ! ‣Fewer surprises during big test cycles ‣More honest & realistic project status reports ! ‣Better overall QA for the system ‣Happier users
  7. 7. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Save Time •Manually checking for regressions •slow & error-prone •limited test coverage •boring! ! •Automated regression testing •fast & reliable •much greater test coverage
  8. 8. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead OBIEE Conceptual Overview Analysis Analysis Data Sources Dashboard RPD BI Server -------------------- Sending query to database named orcl (id: <<19108>>), 
 connection pool named Connection Pool 01 ! select sum(T117.FCAST_COST_AMT) as c1, T107.PROD_TYPE_DESC as c2, T127.MONTH_YYYYMM as c4 from GCBC_SALES.TIMES T127 /* Dim_TIMES */ , […]
  9. 9. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead What Could Possibly Go Wrong? •Data ‣Are the numbers still correct?
 •Functionality ‣Do dashboards still run without error? ‣Do dashboards still render in the same way?
 •Security ‣Data ‣Visible Dashboards
  10. 10. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Whoops… I broke it (1) - the RPD •RPD may still pass consistency check - but do you know if you broke existing reports and dashboards? •Cleanup / Refactoring •Developments •Bugfixes
  11. 11. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Whoops… I broke it (2) - OBIEE front end •Cleanup / Refactored calculations •Presentation Catalog changes •Application Role changes
  12. 12. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Whoops… I broke it (3) - Database •Refactored designs •View -> Table •Table -> MView •Indexes •Partitioning
  13. 13. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead What Makes a Good Regression Test? Easy Better tests Fast Flexible Targeted Scalable
  14. 14. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead The OBIEE stack Request Response BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] User WebLogic Server OBIEE system components Data Source(s) Network Network Not all components listed, eg cluster controller -------------------- SQL Request, logical request hash: 810510fe SET VARIABLE QUERY_SRC_CD='Report',SAW_SRC_PATH='/shared/Test/test1 jan/feb';SELECT 0 s_0, "Sales - Fact Sales"."Dim Products"."Product Type" s_1, "Sales - Fact Sales"."Dim Times"."Month" s_2, SORTKEY("Sales - Fact Sales"."Dim Times"."Month") s_3, "Sales - Fact Sales"."Fact Sales"."Cost" s_4 FROM "Sales - Fact Sales" C1 C2 C3 C4 ---------- -------------------- -------------------- ---------- 4636.93 Bread Assortments FEB-2000 200002 4583.53 Bread Assortments JAN-2000 200001 3445.16 Bread Clubs FEB-2000 200002 3542.49 Bread Clubs JAN-2000 200001 4735.63 Cold Drinks FEB-2000 200002 4565.21 Cold Drinks JAN-2000 200001 4604.19 Gifts & Baskets FEB-2000 200002 […] ------------------------------------------------------------------------------------------- s_0 s_1 s_2 s_3 s_4 ------------------------------------------------------------------------------------------- 0 Bread Assortments JAN-2000 200001 4583.53 0 Bread Clubs JAN-2000 200001 3542.49 0 Cold Drinks JAN-2000 200001 4565.21 0 Gifts & Baskets JAN-2000 200001 4608.01 0 Hot Drinks JAN-2000 200001 5768.62 […] Sending query to database named orcl (id:
 connection pool named Connection Pool 01 ! select sum(T117.FCAST_COST_AMT) as c1, T107.PROD_TYPE_DESC as c2, T127.MONTH_DESC as c3, T127.MONTH_YYYYMM as c4 from GCBC_SALES.TIMES T127 /* Dim_TIMES */ , GCBC_SALES.PRODUCTS T107 /* Dim_PRODUCTS */ , GCBC_SALES.SALES T117 /* Fact_SALES */
  15. 15. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Where Can We Detect Problems? User WebLogic Server OBIEE system components Data Source(s) BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in Analysis request Logical SQL Physical Query x n Query results x n DatasetRendered Analysis RPD RPD DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] 2 3 45 1 •Logical
 
 
 •Physical
 
 
 •Front End 4 1 3 5 2
  16. 16. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing Opportunities •OBIEE Logical model ‣Does the RPD still work and return the same results for a given logical query?
 
 •Physical Database ‣Does the Physical SQL that OBIEE generates still run? 
 
 •OBIEE front end ‣From the point of view of an end user, does it look right?
  17. 17. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead User WebLogic Server OBIEE system components Data Source(s) BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in RPD RPD DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] PhysicalSQL Dataset Regression Testing Database Changes
  18. 18. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing Database Changes •If only the DB has changed, then just test the DB. Don’t complicate things by involving the stack further up. nqquery.log / Usage Tracking SQL*Plus •So long as the query still runs, and returns the same data, you’re guaranteed that nothing will change in OBIEE
  19. 19. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Logical OBIEE Regression Testing User WebLogic Server OBIEE system components Data Source(s) BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in RPD RPD DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] Logical SQL Dataset
  20. 20. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Logical SQL in OBIEE Analysis Logical SQL Resultset via nqcmd Analysis results BI Server RPD
  21. 21. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead RPD Regression Testing in Action 1. Tool automagically extracts Logical SQL for all analyses in targeted dashboards or analyses
  22. 22. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead RPD Regression Testing in Action 2. Tools runs captured Logical SQL to generate the baseline data
  23. 23. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead RPD Regression Testing in Action 3. Developers make their RPD changes and deploy
  24. 24. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead RPD Regression Testing in Action 4. Tool runs regression test to check both data and Logical SQL match the baseline
  25. 25. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing the OBIEE Front End User WebLogic Server OBIEE system components Data Source(s) BI Server BI Server Presentation Services Managed Server Managed Server JavaHost Presentation Services Web Browser Web Browser BI Plug-in BI Plug-in RPD RPD DWH [ ... ] [ ... ] DWH [ ... ] [ ... ] HTML, JS, CSS, PNG
  26. 26. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Regression Testing the OBIEE Front End •Front End testing is complex! ! •Two options to make sense of HTML, JavaScript, CSS, images: ‣Programatically - Parse server response ‣Visually - Render server response
  27. 27. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Front End test methods: Parsing vs Rendering •Parsing ‣Makes pass/fail tests easier -A string is present, or it is not ! ‣Higher rate of false negatives -It’s easier to miss problems ! ‣High development & maintenance cost -Need to reverse-engineer DOM -Often relies on non-deterministic internal IDs to accurately pinpoint assertions
  28. 28. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Front End test methods: Parsing vs Rendering •Rendering ‣More accurate -Can detect differences not expected or explicitly coded for
 ‣More prone to false positives -Lots of things change on a page for valid reasons (dates, alerts, etc)
 ‣Still requires DOM knowledge but less reliance on internal IDs
  29. 29. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Visual Regression Testing Before After Test Result: Fail •Suitable for testing front-end changes, such as: ‣Refactoring Analysis formulae into the RPD ‣Presentation Catalog restructuring
  30. 30. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead DEMO
  31. 31. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Visual Regression Testing In Action 1. Tool takes automatic snapshot of dashboard pages. This is the baseline.
  32. 32. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Visual Regression Testing In Action 2. Make Analysis/RPD changes
  33. 33. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Visual Regression Testing In Action Before After Test Output 3. Tool captures new screenshots and compares against baseline. Any differences are reported.
  34. 34. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Working smarter - summary Test Physical Logical Front End What Changed? Report / Dashboard No No Yes RPD No Yes Yes Database Yes Yes Yes
  35. 35. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead Smart Regression Testing Do better OBIEE development and deliver it more rapidly
  36. 36. www.rittmanmead.com inquiries@rittmanmead.com @rittmanmead www.facebook.com/rittmanmead #EOF ✴ email: robin.moffatt@rittmanmead.com ✴ web: http://ritt.md/rmoff ✴ twitter: @rmoff

×