Performance Tuning Oracle's BI Applications

4,611 views

Published on

http://www.kpipartners.com/webinar-Performance-Tuning-Oracle-BI-Applications/ ... From a virtual event that discusses techniques that can be used to optimize performance of the Oracle BI Apps.

The BI Apps from Oracle present customers with a nice head start to getting their BI environment up and running. But for many customers, their user community demands lighting-fast speeds while running dashboards, reports and ad-hoc queries. Learn about some of the key techniques you can use to take the BI Apps to performance levels you didn’t think were possible.

The discussion begins with a conceptual understanding of why performance problems can exist and the counteracting design considerations. Special attention will be paid to the concept of a Performance Layer, describing what it is, what it is comprised of and how to build it. The presentation includes several real world examples of the significant performance gains that can be had from a Performance Layer.

Objective 1: Learn about the concept of a performance layer and what is involved with building one.
Objective 2: Understand the most important steps to improve the performance of your system.

Published in: Technology, News & Politics
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
4,611
On SlideShare
0
From Embeds
0
Number of Embeds
2,188
Actions
Shares
0
Downloads
160
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Wider tables slow you down:
    Large and wide tables carry more fields than you need

    You usually only need a few of them for dashboards

    Smaller is faster
  • Peak performance starts with design

    Eliminate conflicting priorities
    ETL development effort
    ETL load times
    Overly simplistic model
    Overly complex model

    Singular focus on performance
  • Achieving great “performance” requires a design that works perfectly with how it is used
    Same concept as any transactional system
    Sports car or a travel coffee mug

    This mandates a pure top-down approach to tuning your BI application

    Top-Down design yields:
    Reduced data weight
    Tables which match usage by OBI
    Clean Star Model
    Prebuilt logic

     Specialized design for specialized usage yields optimal performance
  • Achieving great “performance” requires a design that works perfectly with how it is used
    Same concept as any transactional system
    Sports car or a travel coffee mug

    This mandates a pure top-down approach to tuning your BI application

    Top-Down design yields:
    Reduced data weight
    Tables which match usage by OBI
    Clean Star Model
    Prebuilt logic

     Specialized design for specialized usage yields optimal performance
  • Oracle published their latest DW Reference Architecture in 2010

    It has a clear understanding of the performance limitations of a DW for BI Reporting

    It identifies the need for a Performance Layer
  • Top Down design will force too many changes to the OOB Model
    Keep the BI Apps model mostly as it is

    Add a Performance Layer:
    Source data from the BI Apps tables
    Bring only what you need
    Improve the model with denormalizations and pre-calculations
    Use all of the database performance tools available to you

    Optimize to usage!
  • The Performance Layer is an industry standard architecture

    The Performance Layer design is driven only by reporting performance improvements

    Travel light by only bringing needed data

    You don’t need to alter your BI Apps or DW system
  • The Leader In Oracle BI & EPM

    KPI Partners is the Most Experienced Oracle BI & EPM Systems Implementation Partner

    KPI Partners is an Oracle Platinum Partner who specializes in Oracle Business Intelligence (BI) and Oracle Enterprise Performance Management solutions. The award-winning staff at KPI Partners comes directly from the product engineering departments at Oracle, Siebel, and Hyperion. In addition to consulting services, KPI Partners offers training, support, and exclusive pre-packaged analytic solution extensions for Oracle Business Intelligence.

    KPI Partners works with both corporate technology departments and corporate business units to develop value-added business intelligence solutions, not just new technology deployments.

    Industry Expertise

    We deliver enterprise technology solutions through the skill, experience, and power of our team. Our company experience is rooted within the origin of business intelligence and enterprise decision support technology.

    Results Driven

    We are driven to make a difference. From fundamental concepts to application, we deliver methods, tools, and technologies to help drive enterprise reporting and insight.

    Client Focused

    We provide the vision, technology, and leadership our clients need for success. We deliver results on time and on budget. Our clients rely on our expertise to help them define their future.

    Innovation

    We continue to drive the creation of new technology, best practices, and thought-leadership within the space.

    World Class

    We are recognized as leaders in enterprise-level business intelligence technology. With reputation comes responsibility, and KPI Partners strives to provide white glove, 5-star-level, service and support.
  • Performance Tuning Oracle's BI Applications

    1. 1. Contact Us 510.818.9480 | www.kpipartners.com © KPI Partners Inc. Start Here Jeff McQuigg | Senior BI Architect April 24, 2013 Performance Tuning Oracle’s BI Applications
    2. 2. Agenda 2
    3. 3. ①Introducing The Performance Layer ②Building The Performance Layer ③Mapping Into Oracle BI ④Implementation Considerations ⑤Q&A Agenda Performance Tuning Oracle’s BI Applications 3
    4. 4. Introducing The Performance Layer 4
    5. 5. Targeted At Organizations Who Have: Large Data Volumes Custom Tables & Data Sets Aggressive Performance Targets Questionable Design Extensions Slow Hardware 5 Introducing The Performance Layer Performance Tuning Oracle’s BI Applications
    6. 6. These performance concepts are applicable to any BI system Custom Built Systems Custom Stars in OBIA SQL Server 6 Introducing The Performance Layer Performance Tuning Oracle’s BI Applications
    7. 7. Wider tables slow you down Dashboards only need a few tables Smaller is faster 7 Introducing The Performance Layer Performance Tuning Oracle’s BI Applications
    8. 8. Eliminate conflicting priorities Singular focus on performance Peak performance starts with design 8 Introducing The Performance Layer Performance Tuning Oracle’s BI Applications
    9. 9. Great performance requires perfect design for how it is used Mandate a top-down approach to tuning your BI application Specialized design for specialized usage 9 Introducing The Performance Layer Performance Tuning Oracle’s BI Applications
    10. 10. Pre-built logic Clean star models Reduced data weight Tables which match usage by Oracle BI 10 Introducing The Performance Layer Performance Tuning Oracle’s BI Applications Top-Down design yields…
    11. 11. 11Performance Tuning Oracle’s BI Applications Introducing The Performance Layer >> Oracle’s View On Data Warehouse Architecture
    12. 12. Keep the BI Apps model mostly as-is & add a performance layer… ① Source data from BI Apps tables ② Bring only what you need ③ Denormalizations & pre-calculations ④ Use all database performance tools *Optimize To Usage* 12 Performance Mini-Fridge BIAppsDataFridge ETL Introducing The Performance Layer Performance Tuning Oracle’s BI Applications
    13. 13. 13Performance Tuning Oracle’s BI Applications ① The Performance Layer is industry standard architecture ② Design is driven only by report performance improvement ③ Travel light ④ No need to alter BI Apps or DW Introducing The Performance Layer - Takeaways
    14. 14. Building The Performance Layer 14
    15. 15. ① Start with priority areas (select a Fact table) ② Identify the use cases (reports w/ prompts & data security) ③ Analyze resulting physical SQL ④ Try to tune the BI Apps model first! 15 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    16. 16. ⑤ Prototype a new data model to match those needs ⑥ Adjust SQL & benchmark (SQL handcrafting needed) ⑦ Map into Oracle BI & test (Unit & Regression) ⑧ Benchmark the Oracle BI report using prototyped tables 16 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    17. 17. 9. Build the tables using INFA & DAC - Complete Oracle BI RPD mapping 10. Formal Regression Test 11. Deploy 12. Enjoy praise from users 17 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    18. 18. Reduce I/O with extreme prejudice • Tune the BI Apps model first! It may work for you with low effort • Employ techniques to eliminate I/O wherever possible • Partition Elimination, Compression, Indexes, Aggregates, Star Transformations • Let the Performance Layer do the work, not the report query • Follow the KISS principle: Use a simple and clean Star. No Snowflakes! • Ensure OBI is mapped properly and uses correct tables with perfect SQL • Favor a general approach as opposed to a case-by-case approach • A rising tide lifts all boats 18 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    19. 19.  There are 4 kinds of tables in the Performance Layer: 1. Skinny Dimension and Fact tables 2. New Dimension tables 3. Mini-Dimension tables 4. Fact Aggregate tables  Built directly from the base BI Apps or DW tables  Goal: use these tables in as many reports as possible (80/20 rule)  Guiding principles and performance influences: 1. Application use cases drive the layer’s design 2. Use minimal data for the job at hand 3. Aggregate Fact data when needed 4. Denormalize dimensions to eliminate extra joins 5. Pre-Build calculations to eliminate extra joins 6. Pre-Split data sets based on logical usage 19 BI Apps or DW Perf. Layer Building The Performance Layer Performance Tuning Oracle’s BI Applications
    20. 20. 20 Building The Performance Layer Performance Tuning Oracle’s BI Applications ① Single column, local bitmap indexes on all Fact table FKs (_WIDs) and filter fields (DELETE_FLG) ② Single column bitmap indexes on all dimensional fields used in any sort of prompt or report filter ③ Special composite B-Tree indexes to assist Snowflaked areas ④ Composite B-Tree indexes on large dimensions for join backs (for list reports) Query Indexing (4 Types)  For all Fact tables of a reasonable size (e.g., > 5M rows)  Usually partition on Month (Range or Interval)  The Database can easily eliminate the majority of the table  Allows for smaller, local indexes Table Partitioning Before Beginning: Tune the OOTB Model
    21. 21.  Skinny Tables are highly selective versions of the BI Apps or DW tables • “Horizontal Aggregation” – use only 10 columns vs. 100 from the base table  Both Dimensions and Facts  Very easy to build and use  Goal: Reduce Avg. Row Length to 1/5th - 1/20th original size  Include only the columns you will need for top-down reporting analysis • If you don’t need Customer Address, don’t include it • Ignore Meta Data columns (e.g., INTEGRATION_ID, etc.)  Row sets are identical (1:1) with the base tables • For Dimensions use the same ROW_WIDs - can be used with existing fact tables easily 21 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    22. 22.  Build using: 1. Create Table as Select (CTAS) 2. Insert /*+ APPEND */ 3. Materialized Views  Compress the table  Use Parallel hints & options  Don’t forget partitions  Enhance the tables with calculation logic  Database is very fast at these operations – expect only a few minutes for 100M rows 22 CREATE TABLE WC_ACCT_BUDGET_SF COMPRESS NOLOGGING PARALLEL (DEGREE 8) PARTITION BY RANGE(PERIOD_END_DT_WID) INTERVAL(NUMTOYMINTERVAL(1, 'MONTH')) (PARTITION Part_01 VALUES LESS THAN (20100101)) AS SELECT /*+ PARALLEL(F,8) */ F.PERIOD_END_DT_WID, F.X_PERIOD_END_DT_WID, F.COMPANY_ORG_WID, F.GL_ACCOUNT_WID, F.X_POSTED_TOTAL_AMT, case when GL_D."GL_ACCOUNT_NUM" = 'S250' then F."X_POSTED_TOTAL_AMT" end as PLAN_CASES, FROM W_ACCT_BUDGET_F F, W_GL_ACCOUNT_D GL_D WHERE F.GL_ACCOUNT_WID = GL_D.ROW_WID; Building The Performance Layer Performance Tuning Oracle’s BI Applications
    23. 23.  Enhance _SF tables with logic  Identify CASE WHEN statements which require other dimensions • Potential great benefit if the join can be eliminated • Don’t over do it – table will get less skinny with each column  Identify any data set splitting from the RPD • HR Workforce Events table has both Events and Snapshots records but they are always used separately in the RPD • Usage drives design: Split them out! • Huge benefit for Event counting metrics (~10% of table) 23 case when GL_D."GL_ACCOUNT_NUM" = 'S250' then F."X_POSTED_TOTAL_AMT" end as PLAN_CASES, FROM W_ACCT_BUDGET_F F, W_GL_ACCOUNT_D GL_D WHERE F.GL_ACCOUNT_WID = GL_D.ROW_WID; Create table WC_WRKFC_EVT_EVENTS_SF … … WHERE SNAPSHOT_IND = 0 Create table WC_WRKFC_EVT_MONTH_SNP_SF … … WHERE SNAPSHOT_IND = 1 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    24. 24. 24 Real Examples I/O Benefit Sub Ledger (custom) 24X Workforce Snap 11X Workforce Events 56X GL Balance 21X Acct Budget 32X Building The Performance Layer Performance Tuning Oracle’s BI Applications Typical to get a 10X to 20X and even 50X I/O benefit in the _SF vs. the base _F in size  Reduced AVG_ROW_LEN  COMPRESSION  Record Set Splitting  All without any aggregation Skinny Dimensions also have benefits: 1. All I/O is a killer and slows down the entire system 2. De-normalize into a Star (eliminate snowflakes & outer joins) 3. Real World Ex, #1: Swap out 2 wide dims for 2 skinny dims improved query time by 6X 4. Real World Ex. #2: One query going from 11s to 4s with one skinny dim 5. Real World Ex. #3: GL Account Dimension: 37X I/O benefit
    25. 25.  Pre-build major pieces of commonly used but complex logic into the Data Model • Over-relying on the RPD or Reports for logic can harm performance • Let the ETL for the Performance Layer do the work not the query  Example #1: Large binning and bucketing CASE WHEN FACT.ORDER_AMT BETWEEN 0 and 100 THEN ‘0-100’ ELSE CASE WHEN FACT.ORDER_AMT BETWEEN 101 and 200 THEN ‘101-200’ … END • Build a new dimension table to hold these values – WC_CUST_ORDER_QTY_BAND_D  Example #2: Date format conversions – dynamically building a new column with a string concatenation statement: substring(T66755."PER_NAME_MONTH" , 1, 4) , '') + '-' + isnull(right(T66755."PER_NAME_MONTH" , 2) , '') • Build a new column in the W_DAY_D table & index it 25 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    26. 26.  Simply a higher level or levels of a larger dimension • A combination of several Kimball concepts • Granularities will be mixed  Make a new table from the large, base dimension • Contains distinct combinations • Use only commonly used fields • Get cues from dashboard prompts, column selectors, report filters  Create a new ROW_WID  Compress and index as normal, Parallel if needed for creation  Easy to build and map 26 Create table WC_EMPLOYEE_MD COMPRESS as select ROWNUM AS ROW_WID, W_ETHNIC_GRP_DESC, WC_RACE_ETHNIC_DIVRSE_GRP_DESC, W_SEX_MF_CODE, W_SEX_MF_DESC, WC_NON_EMPLOYEE_VENDOR_NAME from ( select distinct W_ETHNIC_GRP_DESC, WC_RACE_ETHNIC_DIVRSE_GRP_DESC, W_SEX_MF_CODE, W_SEX_MF_DESC, WC_NON_EMPLOYEE_VENDOR_NAME from W_EMPLOYEE_D); This real world example created 5,400 records from a W_EMPLOYEE_D of 9+ Million rows. Building The Performance Layer Performance Tuning Oracle’s BI Applications
    27. 27. _MD tables are used in the Performance Layer in two places: 1. Link into Skinny Facts • Use a separate FK in addition to the base _WID • Fact table has both EMPLOYEE_WID and EMPLOYEE_MD_WID  Thus the _SF can join to all of the following: • New Mini Dimension (_MD) (~1% rows, some columns) • New Skinny Dimension (_SD) (100% rows, some columns) • Base BI Apps/DW Dimension (_D) (100% rows, 100% columns)  The OBI RPD can select which one is best for each query  Benefits of linking into the _SF 1. The same set of fact rows are selected – no benefit 2. Reduced dimension I/O, CPU and buffer space 3. Faster join-back on list reports 4. Very fast prompts, especially when constrained 27 _D _SD _MD Conceptual Size Difference s Building The Performance Layer Performance Tuning Oracle’s BI Applications
    28. 28. 2. Use them for very high level Fact Aggregates • Build a Fact aggregate at the Mini-Dimension level  Allows greater field inclusion at no expense to aggregation compression ratios • Multiple fields are available - not just one  A good Mini Dimension and Skinny Fact/Aggregate can serve a large % of dashboard queries  MD’s & Fact Aggregates offer extreme performance: 1. Real World Ex #1: From time-out after 10 minutes to 4 seconds 2. Real World Ex #2: GL Account MD: 131X I/O benefit 28 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    29. 29.  Aggregates are used when summary reports exist  Pre-aggregate the dataset to make it smaller & faster  Sometimes they are the only solution  Typically a minimum of a 10:1 ratio is used  Use all database tools as with any fact table • Partitioning, Indexing, Star Transformations, Compression  For extreme needs, consider merging facts together • Ex: Monthly Actuals and Budgets • Be mindful of gaps in datasets and non-conformed dimensions  Advanced implementations use partition management to build only changed data for faster load times 29 Building The Performance Layer Performance Tuning Oracle’s BI Applications
    30. 30. 30Performance Tuning Oracle’s BI Applications ① Building the underlying tables is relatively simple ② Take cues from dashboard, report & RPD configuration ③ Any savings in I/O helps the overall system ④ Use all of the available database performance tools ⑤ Tremendous benefits are possible with a few tables Building The Performance Layer- Takeaways
    31. 31. Mapping Into Oracle BI 31
    32. 32. Link as much as possible to allow for the best performance across all scenarios 32 BI Apps Performance Layer Best Better Base Performance Tuning Oracle’s BI Applications Mapping Into Oracle BI
    33. 33.  The 3 Fact tables are mapped like any aggregate  The Skinny Fact (_SF) will have fewer dimensions and fewer metrics mapped to it • Along the Employee dimension however it is the same as the base _F  OBI will prefer to use the _SF over the _F • Uses regular aggregate navigation concepts • Uses the _F when needed as a “backup plan” 33 _A _SF _F Performance Tuning Oracle’s BI Applications Mapping Into Oracle BI
    34. 34.  Raise the priority group on the base _D to have OBI prefer the _SD As both LTS grains are identical, OBI needs more info to make a choice 34 _D _SD Performance Tuning Oracle’s BI Applications Mapping Into Oracle BI
    35. 35. Create a dummy hierarchy level and map the LTSs for the Mini Dimension and Fact Aggregate to it  The grain of the Mini Dimension is arbitrary  As long as OBI knows it is higher than the other LTSs it will be preferred (Priority groups not needed) 35 _MD _A Performance Tuning Oracle’s BI Applications Mapping Into Oracle BI
    36. 36. 36Performance Tuning Oracle’s BI Applications Table Mapping The mapping of tables is straightforward Link Tables As Much As Possible Let Oracle BI make the best choice Mapping Into Oracle BI - Takeaways
    37. 37. Implementation Considerations 37
    38. 38. The whole prototyping process can be done on a simple star in roughly two weeks Allow for more time if you have: • Large data volumes • Difficult performance targets • More complex models and logic • Many disparate report patterns or lots of reports to consider • More stars are needed (e.g., Actuals and Budgets together) Development effort depends on # new objects • Typically only another two weeks needed (ETL & OBI RPD) • A few more for regression test and deployment 38Performance Tuning Oracle’s BI Applications Implementation Considerations
    39. 39.  Use Production data volumes for accurate analysis  Use Production DDL, ETL code, OBI RPD and OBI Webcat  Quiet, unused machine for accurate benchmarking  Use hardware that is as similar to Prod as possible • KPI uses a database benchmarking tool to compare environments 39Performance Tuning Oracle’s BI Applications Implementation Considerations
    40. 40.  Additional ETL & RPD Development • Use SQL scripts instead of Informatica mappings (less effort, faster execution)  Additional Testing – Regression test is easy  Additional ETL Run Time – may be critical  Additional Database size - minor  Customization Propagation / Impact Analysis • True of any aggregate  Complex logic will be more difficult • Financial Analytics - snowflake with multiple segment hierarchies • HR Workforce Event & Snapshot logic uses effective dates for future dated events 40Performance Tuning Oracle’s BI Applications Implementation Considerations
    41. 41. Q & A 41
    42. 42. www.kpipartners.com The Leader In Oracle BI & EPM 42 Strategic Consulting | Systems Implementation | Training Depot Repair Analytics Fixed Asset Analytics Manufacturing Analytics Salesforce.com Analytics Student Info Analytics Subledger (SLA) Analytics and more… Transform Data Into Insight  Staff built from Oracle/Siebel/Hyperion engineering teams  On-site, off-shore and blended shore delivery models  Exclusive pre-built solutions for Oracle BI & E-Business Suite Oracle BI Hyperion Endeca Exalytics
    43. 43. Email: info@kpipartners.com Web: kpipartners.com/contact KPI World Headquarters 39899 Balentine Drive Suite #375 Newark, CA 94560 Phone: (510) 818-9480 Contact Us The Leader In Oracle BI & EPM 43 New York, NY Chicago, IL Boston, MA Minneapolis, MN San Diego, CA Greensboro, NC North America Offices Bangalore, India Hyderabad, India Global Offices
    44. 44. www.kpipartners.com 44

    ×