Loading…

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Performance Tuning Oracle's BI Applications

on

  • 2,317 views

http://www.kpipartners.com/webinar-Performance-Tuning-Oracle-BI-Applications/ ... From a virtual event that discusses techniques that can be used to optimize performance of the Oracle BI Apps.

http://www.kpipartners.com/webinar-Performance-Tuning-Oracle-BI-Applications/ ... From a virtual event that discusses techniques that can be used to optimize performance of the Oracle BI Apps.

The BI Apps from Oracle present customers with a nice head start to getting their BI environment up and running. But for many customers, their user community demands lighting-fast speeds while running dashboards, reports and ad-hoc queries. Learn about some of the key techniques you can use to take the BI Apps to performance levels you didn’t think were possible.

The discussion begins with a conceptual understanding of why performance problems can exist and the counteracting design considerations. Special attention will be paid to the concept of a Performance Layer, describing what it is, what it is comprised of and how to build it. The presentation includes several real world examples of the significant performance gains that can be had from a Performance Layer.

Objective 1: Learn about the concept of a performance layer and what is involved with building one.
Objective 2: Understand the most important steps to improve the performance of your system.

Statistics

Views

Total Views
2,317
Views on SlideShare
1,308
Embed Views
1,009

Actions

Likes
1
Downloads
90
Comments
0

2 Embeds 1,009

http://www.kpipartners.com 1008
http://webcache.googleusercontent.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Wider tables slow you down:Large and wide tables carry more fields than you needYou usually only need a few of them for dashboardsSmaller is faster
  • Peak performance starts with designEliminate conflicting prioritiesETL development effortETL load timesOverly simplistic modelOverly complex modelSingular focus on performance
  • Achieving great “performance” requires a design that works perfectly with how it is usedSame concept as any transactional systemSports car or a travel coffee mugThis mandates a pure top-down approach to tuning your BI applicationTop-Down design yields:Reduced data weightTables which match usage by OBIClean Star ModelPrebuilt logic Specialized design for specialized usage yields optimal performance
  • Achieving great “performance” requires a design that works perfectly with how it is usedSame concept as any transactional systemSports car or a travel coffee mugThis mandates a pure top-down approach to tuning your BI applicationTop-Down design yields:Reduced data weightTables which match usage by OBIClean Star ModelPrebuilt logic Specialized design for specialized usage yields optimal performance
  • Oracle published their latest DW Reference Architecture in 2010It has a clear understanding of the performance limitations of a DWfor BI ReportingIt identifies the need for a Performance Layer
  • Top Down design will force too many changes to the OOB ModelKeep the BI Apps model mostly as it isAdd a Performance Layer:Source data from the BI Apps tablesBring only what you needImprove the model with denormalizations and pre-calculationsUse all of the database performance tools available to youOptimize to usage!
  • The Performance Layer is an industry standard architectureThe Performance Layer design is driven only by reporting performance improvementsTravel light by only bringing needed data You don’t need to alter your BI Apps or DW system
  • The Leader In Oracle BI & EPMKPI Partners is the Most Experienced Oracle BI & EPM Systems Implementation PartnerKPI Partners is an Oracle Platinum Partner who specializes in Oracle Business Intelligence (BI) and Oracle Enterprise Performance Management solutions. The award-winning staff at KPI Partners comes directly from the product engineering departments at Oracle, Siebel, and Hyperion. In addition to consulting services, KPI Partners offers training, support, and exclusive pre-packaged analytic solution extensions for Oracle Business Intelligence.KPI Partners works with both corporate technology departments and corporate business units to develop value-added business intelligence solutions, not just new technology deployments.Industry ExpertiseWe deliver enterprise technology solutions through the skill, experience, and power of our team. Our company experience is rooted within the origin of business intelligence and enterprise decision support technology.Results DrivenWe are driven to make a difference. From fundamental concepts to application, we deliver methods, tools, and technologies to help drive enterprise reporting and insight.Client FocusedWe provide the vision, technology, and leadership our clients need for success. We deliver results on time and on budget. Our clients rely on our expertise to help them define their future.InnovationWe continue to drive the creation of new technology, best practices, and thought-leadership within the space.World ClassWe are recognized as leaders in enterprise-level business intelligence technology. With reputation comes responsibility, and KPI Partners strives to provide white glove, 5-star-level, service and support.

Performance Tuning Oracle's BI Applications Performance Tuning Oracle's BI Applications Presentation Transcript

  • Contact Us510.818.9480 | www.kpipartners.com© KPI Partners Inc.Start HereJeff McQuigg | Senior BI ArchitectApril 24, 2013PerformanceTuning Oracle’sBI Applications
  • Agenda2
  • ① Introducing The Performance Layer② Building The Performance Layer③ Mapping Into Oracle BI④ Implementation Considerations⑤ Q&AAgendaPerformance Tuning Oracle’s BI Applications 3 View slide
  • Introducing ThePerformance Layer4 View slide
  • Targeted At OrganizationsWho Have:Large Data VolumesCustom Tables & Data SetsAggressive Performance TargetsQuestionable Design ExtensionsSlow Hardware5Introducing The Performance LayerPerformance Tuning Oracle’s BI Applications
  • These performance conceptsare applicable to any BI systemCustom Built SystemsCustom Stars in OBIASQL Server6Introducing The Performance LayerPerformance Tuning Oracle’s BI Applications
  • Wider tables slow you downDashboards only need a few tablesSmaller is faster7Introducing The Performance LayerPerformance Tuning Oracle’s BI Applications
  • Eliminate conflicting prioritiesSingular focus on performancePeak performance starts with design8Introducing The Performance LayerPerformance Tuning Oracle’s BI Applications
  • Great performance requires perfect design for how it is usedMandate a top-down approach to tuning your BI applicationSpecialized design for specialized usage9Introducing The Performance LayerPerformance Tuning Oracle’s BI Applications
  • Pre-built logicClean star modelsReduced data weightTables which match usage by Oracle BI10Introducing The Performance LayerPerformance Tuning Oracle’s BI ApplicationsTop-Down design yields…
  • 11Performance Tuning Oracle’s BI ApplicationsIntroducing The Performance Layer >> Oracle’s View On Data Warehouse Architecture
  • Keep the BI Apps modelmostly as-is & add aperformance layer…① Source data from BI Apps tables② Bring only what you need③ Denormalizations & pre-calculations④ Use all database performance tools*Optimize To Usage*12PerformanceMini-FridgeBIAppsDataFridgeETLIntroducing The Performance LayerPerformance Tuning Oracle’s BI Applications
  • Performance Tuning Oracle’s BI Applications① The Performance Layer is industrystandard architecture② Design is driven only by reportperformance improvement③ Travel light④ No need to alter BI Apps or DWIntroducing The Performance Layer - Takeaways
  • Building ThePerformance Layer14
  • ① Start with priority areas (select a Fact table)② Identify the use cases (reports w/ prompts & data security)③ Analyze resulting physical SQL④ Try to tune the BI Apps model first!15Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  • ⑤ Prototype a new data model to match thoseneeds⑥ Adjust SQL & benchmark (SQL handcrafting needed)⑦ Map into Oracle BI & test (Unit & Regression)⑧ Benchmark the Oracle BI report usingprototyped tables16Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  • 9. Build the tables using INFA & DAC - CompleteOracle BI RPD mapping10. Formal Regression Test11. Deploy12. Enjoy praise from users17Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  • Reduce I/O with extreme prejudice• Tune the BI Apps model first! It may work for you with low effort• Employ techniques to eliminate I/O wherever possible• Partition Elimination, Compression, Indexes, Aggregates, Star Transformations• Let the Performance Layer do the work, not the report query• Follow the KISS principle: Use a simple and clean Star. NoSnowflakes!• Ensure OBI is mapped properly and uses correct tables with perfectSQL• Favor a general approach as opposed to a case-by-case approach• A rising tide lifts all boats18Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  •  There are 4 kinds of tables in the Performance Layer:1. Skinny Dimension and Fact tables2. New Dimension tables3. Mini-Dimension tables4. Fact Aggregate tables Built directly from the base BI Apps or DW tables Goal: use these tables in as many reports as possible (80/20 rule) Guiding principles and performance influences:1. Application use cases drive the layer’s design2. Use minimal data for the job at hand3. Aggregate Fact data when needed4. Denormalize dimensions to eliminate extra joins5. Pre-Build calculations to eliminate extra joins6. Pre-Split data sets based on logical usage19BI Apps orDWBuilding The Performance LayerPerformance Tuning Oracle’s BI Applications
  • 20Building The Performance LayerPerformance Tuning Oracle’s BI Applications① Single column, local bitmapindexes on all Fact table FKs(_WIDs) and filter fields(DELETE_FLG)② Single column bitmap indexeson all dimensional fields used inany sort of prompt or report filter③ Special composite B-Treeindexes to assist Snowflakedareas④ Composite B-Tree indexes onlarge dimensions for join backs(for list reports)Query Indexing (4 Types) For all Fact tables of a reasonablesize (e.g., > 5M rows) Usually partition on Month (Rangeor Interval) The Database can easilyeliminate the majority of the table Allows for smaller, local indexesTable PartitioningBefore Beginning:Tune the OOTB Model
  •  Skinny Tables are highly selective versions of the BI Apps or DWtables• “Horizontal Aggregation” – use only 10 columns vs. 100 from the base table Both Dimensions and Facts Very easy to build and use Goal: Reduce Avg. Row Length to 1/5th - 1/20th original size Include only the columns you will need for top-down reportinganalysis• If you don’t need Customer Address, don’t include it• Ignore Meta Data columns (e.g., INTEGRATION_ID, etc.) Row sets are identical (1:1) with the base tables• For Dimensions use the same ROW_WIDs - can be used with existing fact tables easily21Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  •  Build using:1. Create Table as Select (CTAS)2. Insert /*+ APPEND */3. Materialized Views Compress the table Use Parallel hints & options Don’t forget partitions Enhance the tables withcalculation logic Database is very fast at theseoperations – expect only a fewminutes for 100M rows22CREATE TABLE WC_ACCT_BUDGET_SFCOMPRESS NOLOGGING PARALLEL (DEGREE 8)PARTITION BY RANGE(PERIOD_END_DT_WID)INTERVAL(NUMTOYMINTERVAL(1, MONTH))(PARTITION Part_01 VALUES LESS THAN(20100101))AS SELECT /*+ PARALLEL(F,8) */F.PERIOD_END_DT_WID,F.X_PERIOD_END_DT_WID,F.COMPANY_ORG_WID,F.GL_ACCOUNT_WID,F.X_POSTED_TOTAL_AMT,case when GL_D."GL_ACCOUNT_NUM" =S250 then F."X_POSTED_TOTAL_AMT" endas PLAN_CASES,FROM W_ACCT_BUDGET_F F,W_GL_ACCOUNT_D GL_DWHERE F.GL_ACCOUNT_WID = GL_D.ROW_WID;Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  •  Enhance _SF tables with logic Identify CASE WHEN statementswhich require other dimensions• Potential great benefit if the join can beeliminated• Don’t over do it – table will get less skinnywith each column Identify any data set splitting fromthe RPD• HR Workforce Events table has bothEvents and Snapshots records but theyare always used separately in the RPD• Usage drives design: Split them out!• Huge benefit for Event counting metrics(~10% of table)23case whenGL_D."GL_ACCOUNT_NUM" = S250then F."X_POSTED_TOTAL_AMT" endas PLAN_CASES,FROM W_ACCT_BUDGET_F F,W_GL_ACCOUNT_D GL_DWHERE F.GL_ACCOUNT_WID =GL_D.ROW_WID;Create tableWC_WRKFC_EVT_EVENTS_SF …… WHERE SNAPSHOT_IND = 0Create tableWC_WRKFC_EVT_MONTH_SNP_SF …… WHERE SNAPSHOT_IND = 1Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  • 24Real Examples I/O BenefitSub Ledger(custom)24XWorkforce Snap 11XWorkforce Events 56XGL Balance 21XAcct Budget 32XBuilding The Performance LayerPerformance Tuning Oracle’s BI ApplicationsTypical to get a 10X to 20X and even50X I/O benefit in the _SF vs. the base_F in size Reduced AVG_ROW_LEN COMPRESSION Record Set Splitting All without any aggregationSkinny Dimensions also have benefits:1. All I/O is a killer and slows down the entire system2. De-normalize into a Star (eliminate snowflakes & outer joins)3. Real World Ex, #1: Swap out 2 wide dims for 2 skinny dims improved querytime by 6X4. Real World Ex. #2: One query going from 11s to 4s with one skinny dim5. Real World Ex. #3: GL Account Dimension: 37X I/O benefit
  •  Pre-build major pieces of commonly used but complex logic into theData Model• Over-relying on the RPD or Reports for logic can harm performance• Let the ETL for the Performance Layer do the work not the query Example #1: Large binning and bucketingCASE WHEN FACT.ORDER_AMT BETWEEN 0 and 100 THEN ‘0-100’ ELSE CASE WHEN FACT.ORDER_AMTBETWEEN 101 and 200 THEN ‘101-200’ … END• Build a new dimension table to hold these values –WC_CUST_ORDER_QTY_BAND_D Example #2: Date format conversions – dynamically building a newcolumn with a string concatenation statement:substring(T66755."PER_NAME_MONTH" , 1, 4) , ) + - +isnull(right(T66755."PER_NAME_MONTH" , 2) , )• Build a new column in the W_DAY_D table & index it25Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  •  Simply a higher level or levels ofa larger dimension• A combination of several Kimballconcepts• Granularities will be mixed Make a new table from thelarge, base dimension• Contains distinct combinations• Use only commonly used fields• Get cues from dashboardprompts, column selectors, reportfilters Create a new ROW_WID Compress and index as normal,Parallel if needed for creation Easy to build and map26Create table WC_EMPLOYEE_MDCOMPRESS asselectROWNUM AS ROW_WID,W_ETHNIC_GRP_DESC,WC_RACE_ETHNIC_DIVRSE_GRP_DESC,W_SEX_MF_CODE, W_SEX_MF_DESC,WC_NON_EMPLOYEE_VENDOR_NAMEfrom (select distinctW_ETHNIC_GRP_DESC,WC_RACE_ETHNIC_DIVRSE_GRP_DESC,W_SEX_MF_CODE,W_SEX_MF_DESC,WC_NON_EMPLOYEE_VENDOR_NAMEfrom W_EMPLOYEE_D);This real world example created 5,400records from a W_EMPLOYEE_D of 9+Million rows.Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  • _MD tables are used in the Performance Layer in two places:1. Link into Skinny Facts• Use a separate FK in addition to the base _WID• Fact table has both EMPLOYEE_WID and EMPLOYEE_MD_WID Thus the _SF can join to all of the following:••• The OBI RPD can select which one is best for each query Benefits of linking into the _SF1. The same set of fact rows are selected – no benefit2. Reduced dimension I/O, CPU and buffer space3. Faster join-back on list reports4. Very fast prompts, especially when constrained27_D_SD_MDConceptualSizeDifferencesBuilding The Performance LayerPerformance Tuning Oracle’s BI Applications
  • 2. Use them for very high level Fact Aggregates• Build a Fact aggregate at the Mini-Dimension level Allows greater field inclusion at no expense toaggregation compression ratios• Multiple fields are available - not just one A good Mini Dimension and Skinny Fact/Aggregatecan serve a large % of dashboard queries MD’s & Fact Aggregates offer extreme performance:1. Real World Ex #1: From time-out after 10 minutes to 4 seconds2. Real World Ex #2: GL Account MD: 131X I/O benefit28Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  •  Aggregates are used when summary reports exist Pre-aggregate the dataset to make it smaller & faster Sometimes they are the only solution Typically a minimum of a 10:1 ratio is used Use all database tools as with any fact table• Partitioning, Indexing, Star Transformations, Compression For extreme needs, consider merging facts together• Ex: Monthly Actuals and Budgets• Be mindful of gaps in datasets and non-conformed dimensions Advanced implementations use partition management tobuild only changed data for faster load times29Building The Performance LayerPerformance Tuning Oracle’s BI Applications
  • Performance Tuning Oracle’s BI Applications① Building the underlying tables is relatively simple② Take cues from dashboard, report & RPD configuration③ Any savings in I/O helps the overall system④ Use all of the available database performance tools⑤ Tremendous benefits are possible with a few tablesBuilding The Performance Layer- Takeaways
  • Mapping Into Oracle BI31
  • Link as much as possible to allow for the best performance across allscenarios32BI AppsPerformanceLayerBestBetterBasePerformance Tuning Oracle’s BI ApplicationsMapping Into Oracle BI
  •  The 3 Fact tables are mappedlike any aggregate The Skinny Fact (_SF) willhave fewer dimensions andfewer metrics mapped to it• Along the Employee dimension however itis the same as the base _F OBI will prefer to use the _SFover the _F• Uses regular aggregate navigationconcepts• Uses the _F when needed as a “backupplan”33Performance Tuning Oracle’s BI ApplicationsMapping Into Oracle BI
  •  Raise the priority group on the base _D to haveOBI prefer the _SDAs both LTS grains are identical, OBI needs more info to make achoice34Performance Tuning Oracle’s BI ApplicationsMapping Into Oracle BI
  • Create a dummy hierarchy level and map the LTSs for the MiniDimension and Fact Aggregate to it The grain of the Mini Dimension is arbitrary As long as OBI knows it is higher than the other LTSs it willbe preferred (Priority groups not needed)35Performance Tuning Oracle’s BI ApplicationsMapping Into Oracle BI
  • Performance Tuning Oracle’s BI ApplicationsTable MappingThe mapping of tables is straightforwardLink Tables As Much As PossibleLet Oracle BI make the best choiceMapping Into Oracle BI - Takeaways
  • ImplementationConsiderations37
  • The whole prototyping process can be done on asimple star in roughly two weeksAllow for more time if you have:• Large data volumes• Difficult performance targets• More complex models and logic• Many disparate report patterns or lots of reports to consider• More stars are needed (e.g., Actuals and Budgets together)Development effort depends on # new objects• Typically only another two weeks needed (ETL & OBI RPD)• A few more for regression test and deployment38Performance Tuning Oracle’s BI ApplicationsImplementation Considerations
  •  Use Production data volumes foraccurate analysis Use Production DDL, ETL code,OBI RPD and OBI Webcat Quiet, unused machine foraccurate benchmarking Use hardware that is as similar toProd as possible• KPI uses a database benchmarking tool tocompare environments39Performance Tuning Oracle’s BI ApplicationsImplementation Considerations
  •  Additional ETL & RPD Development• Use SQL scripts instead of Informatica mappings (less effort, fasterexecution) Additional Testing – Regression test is easy Additional ETL Run Time – may be critical Additional Database size - minor Customization Propagation / Impact Analysis• True of any aggregate Complex logic will be more difficult• Financial Analytics - snowflake with multiple segment hierarchies• HR Workforce Event & Snapshot logic uses effective dates for futuredated events40Performance Tuning Oracle’s BI ApplicationsImplementation Considerations
  • Q & A41
  • www.kpipartners.comThe Leader In Oracle BI & EPMStrategic Consulting | Systems Implementation | TrainingDepot Repair AnalyticsFixed Asset AnalyticsManufacturing AnalyticsSalesforce.com AnalyticsStudent Info AnalyticsSubledger (SLA) Analyticsand more…Transform Data Into Insight Staff built fromOracle/Siebel/Hyperionengineering teams On-site, off-shore and blendedshore delivery models Exclusive pre-built solutionsfor Oracle BI & E-BusinessSuiteOracle BIHyperionEndecaExalytics
  • Email: info@kpipartners.comWeb: kpipartners.com/contactKPI World Headquarters39899 Balentine DriveSuite #375Newark, CA 94560Phone: (510) 818-9480Contact UsThe Leader In Oracle BI & EPM 43New York, NYChicago, ILBoston, MAMinneapolis, MNSan Diego, CAGreensboro, NCNorth America OfficesBangalore, India Hyderabad, IndiaGlobal Offices
  • www.kpipartners.com44