Unlocking the secrets to how essbase thinks e roske in sync10 oracle epm track


Published on

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Unlocking the secrets to how essbase thinks e roske in sync10 oracle epm track

  1. 1. Unlocking the Secrets to How Essbase Thinks Edward Roske eroske@interrel.com BLOG: LookSmarter.blogspot.com WEBSITE: www.interrel.com TWITTER: ERoske
  2. 2. About interRel • 2008 & 2009 Oracle Titan Award winner - EPM Solution of the year • 2008 Oracle EPM Excellence Award • 2009 Oracle EPM/BI Innovation Award • One of the fastest growing companies in the world (Inc. Magazine, ‘08 & ‗09) • Two of the four Hyperion Oracle ACE Directors in the world • Oracle Gold Specialized Partner in Oracle EPM • Focused exclusively on Oracle Hyperion EPM software – Consulting – Training – Infrastructure and Installation – Support – Software sales
  3. 3. • 5 Hyperion Books Available: – Essbase (7): Complete Guide – Essbase System 9: Complete Guide – Essbase System 9: End User Guide – Smart View 11: End User Guide – Essbase 11: Admin Guide – Hyperion Planning for End Users • Coming in March – Hyperion Planning for Administrators • To order, check out www.LuLu.com 3 Copyright © 2007, Hyperion. All rights reserved.
  4. 4. Agenda • Introduction • Internal Workings of Essbase: BSO • Internal Workings of Essbase: ASO • Question and Answer
  5. 5. Internal Workings of Essbase: Block Storage
  6. 6. Bob Earle Invented ―Sparse and Dense‖ • How is data distributed? SPARSE DENSE Products Time Periods X X X X Accounts X X X X X X Locations X X X X X X X X X X X X X X X
  7. 7. Block Structure Var% Var Bud Act UnitsSold Headcnt Profit Account (dense) Expense OtherExp Marketing Salary Rev COGs Sales Jan Feb Mar Q1 Q2 Q3 Yr Period (dense)
  8. 8. Data Cells Within the Block Actual -> Profit -> Jan Cross-dimensional operator (means Actual BY Profit BY Jan)
  9. 9. Blocks • An individual block is created for each combination of sparse stored members Cola->East Cola->New York Cola->Florida Cola->Massachusetts
  10. 10. Storage and Compression
  11. 11. Storage • PAG File – Contains the data (the blocks with a header for each) – Contains up to 2 Gb of data in each PAG file (32-bit) – Can be 1,024 different files – Can be compressed and fragmented – Can be stored on multiple drive locations • IND File – Contains the list or pointers to the blocks (intersection of sparse dimensions) – Contains up to 2 Gb of index in each IND file (32-bit) – Can be 1,024 different files – Can be fragmented – Can be stored on multiple drive locations
  12. 12. Compression • ―Simple‖ compression settings – None – zLib – Index Value Pair • Can‘t assign directly • Good for really large blocks with really sparse data • Following types use multiple compressions (one per block) – Bitmap • Good for non-repeating data • Will use Bitmap or IVP – RLE = Run Length Encoding • Good for data with zeros and data that repeats (such as budgeting) • Will use RLE, Bitmap, or IVP
  13. 13. Dimension Order & Compression • Dimension order affects compression • First dense dimension determines your ―columns‖ in PAG file • Compression is from left to right, top to bottom • Move Time to first dense dimension and we get: BUDGET Jan Feb Mar Apr May Jun Sales 100 100 100 120 120 120 COGS 50 50 50 50 50 50 Margin 50 50 50 70 70 70 Exp. 30 30 30 30 30 30 Profit 20 20 20 40 40 40 • Notice repeating values • Time should be dense then Measures for RLE compression
  14. 14. Calculations
  15. 15. Calculation Process Accounts Jan Feb Mar Qtr1 Sales 124.71 119.43 161.93 406.07 COGS 42.37 38.77 47.28 128.42 Margin 82.34 80.66 114.65 277.65
  16. 16. Dense Calculation After calc of Time Year dimension Qtr1 Data load from table XXXX XXXXXXX Jan After calc of XX ### Accounts XX ### Feb dimension XX ### Mar Sales COGS Margin Profit
  17. 17. Sparse Calculation East -> Cola  Calculated blocks  Upper-level blocks  Input blocks  Level zero blocks Vermont -> Cola New York -> Cola
  18. 18. Calculation Order: All Dimensions • First, Accounts • Second, Time • Third, remaining dense dimensions • Fourth, remaining sparse dimensions • Two Pass Calculation
  19. 19. Calculations • Default Calc - simplest method • Calc scripts • Dynamic calculations • Intelligent calculation
  20. 20. Commit Blocks • Using Uncommitted Access – When Commit Level is reached, blocks write to hard drive • Default is 3000 blocks • Setting Commit Blocks to Zero – Writes at completion of the entire transaction – Will dramatically improve calculation time – Will fragment your PAG file during a calculation
  21. 21. Retrievals
  22. 22. Index Cache Index pages New requests Index pages on in physical disk index cache Old requests Memory Disk
  23. 23. Data Cache New requests Disk blocks on Disk blocks in physical disk data cache Old requests Memory Disk
  24. 24. Internal Workings of Essbase: Aggregate Storage
  25. 25. BSO Limitations • Financial applications are more densely populated so BSO works great in those instances • BSO engine can handle sparse data but on a ―limited‖ scale • Outline size limited • Batch times required for loads and calcs
  26. 26. Aggregate Storage Option • Remember all the concepts we just learned: – Dense / Sparse – Index / page files X – Cache settings – Block storage
  27. 27. Aggregate Storage Databases • Similar to a BSO database – outline, dimensions, hierarchies… BUT • Different method for storing/calculating databases • New storage kernel built to handle ASO databases • Calculates 10-100x faster • Stores up to 252 dimension combinations
  28. 28. Aggregate Storage Databases • ASO addresses the following types of databases: – Read-only* • Write back to level zero available in 9.3.1 – ―Rack and stack‖ – Large dimensions • New Types of databases are possible – Customer analysis—data is analyzed from any dimension and there are potentially millions of customers – Procurement analysis—many products are tracked across many vendors – Logistics analysis—near real-time updates of product shipments – Market Basket analysis—products purchased along with other products
  29. 29. When to use ASO • Database is sparse and has many dimensions, and/or the dimensions have many levels of members • Database is used primarily for read-only purposes, with few or no data updates • Calculation of the database is frequent, is based mainly on summarizing of the data, and does not rely on calculation scripts • Starting in Essbase 11x, ASO should be your default/starting idea
  30. 30. BSO vs. ASO BSO Outline ASO Outline
  31. 31. End User Perspective • End users won‘t care whether their database is ASO or BSO • The way users access ASO is the same as BSO – Excel Add-in – Financial Reporting, Web Analysis, etc. – Smart View Add-in • Repeat… no differences (just more data/dimensions)
  32. 32. ASO Benefits from IT Perspective • Faster load and calc times provide – Lower hardware costs – Lower maintenance costs – Higher availability • Small disk footprints • Efficient tuning for storage and query response
  33. 33. How does ASO work? • Simple question with not so simple answer • Asked greatest minds in the business how ASO works and got the same resounding answer: • ―It‘s a black box‖ or ―it's top secret and hard to understand‖ • There must be a better answer!
  34. 34. ASO is Designed For… • More dimensions and members • Less time required for batches – Fast aggregation of sparse data sets – Faster loads – Incremental loads • Reduction in database footprint
  35. 35. Key Concepts • Storage • Sparse data • Indexing • Aggregation • Nodes
  36. 36. ASO Storage (ROLAP in disguise) • ASO can also be said to be ―ROLAP with a super fancy index scheme that rules.‖ • Big difference between ASO and BSO and ROLAP is the ASO storage mechanism • ROLAP stores data in a table and indexes a combination key between the rows • Essbase stores the concept of a cube of data in multiple dimensions or rather multiple keys
  37. 37. ASO Storage (ROLAP in disguise) • It‘s a multidimensional index • ASO takes it a step further and indexes the indexes in a way for more rapid aggregation of data • Storage is no longer in "blocks" but in highly optimized aggregation nodes • Visualize it as an asymmetric fractal Christmas tree flattened out and then indexed again
  38. 38. How does ASO work • Load Data only at level 0 • Create aggregate views • Algorithm selects and stores ―most taxing‖ queries • Dynamic queries at runtime and increased speed by leveraging nearest stored view
  39. 39. ASO Concepts • Concept of stored and dynamic hierarchies – Stored hierarchies can only aggregate – Dynamic hierarchies can utilize formulas and advanced unary operators • Formulas on dimension members use MDX syntax • Pre-aggregated views can be defined to help query performance • Aggregated design wizard helps you create aggregation scripts • Outline paging helps you page portions of the outline in and out of memory to assist in performance • You can convert BSO outlines to ASO outlines using a wizard
  40. 40. Storage and Compression
  41. 41. Directory Structure • Directory structure differs in both content and purpose from BSO • Tablespaces are utilized to store data and metadata – Default – stores numeric data (.dat file) – Log – records database activity – Metadata – stores metadata information about the objects in the database – Temp – temporary working space for the Essbase kernel
  42. 42. Tablespace Overview • Defines the database storage in the form of file locations • Each ASO application has 4 tablespaces – Default – database values – Temp – temporary work space – Log – transaction log files – Metadata – database data structure • File location specifies a physical disk space for storing database files • Each tablespace may contain one or more file locations – Can span multiple physical drives and/or logical volumes
  43. 43. Manage Tablespaces
  44. 44. Sizing Tablespaces • During the data load and aggregation process, data is stored in both the Temp and Default directories • ASO will always build the full .dat file in the temp tablespace while the default tablespace still has the production .dat • Hence, for your maximum drive size you have to plan on AT LEAST 3x your max bloated .dat size if you want to be "safe" (buffer to temp to default)
  45. 45. Compression Dimension ASO • In old releases, the Accounts dimension enabled database compression • Beginning in 9.3, the Accounts dimension is the default compression dimension BUT you can choose a different compression dimension • Essbase helps you choose a compression dimension by estimating what the database size would be depending on which dimension is tagged as compression • Time is an excellent candidate for compression dimension, especially if you have fiscal year as a separate dimension
  46. 46. Calculations
  47. 47. Aggregating ASO Data • For ASO databases, after data values are loaded into the level 0 cells of an outline, the database requires no separate calculation step • From any point in the database, users can retrieve and view values that are aggregated for only the current retrieval • ASO databases are smaller than block storage databases, enabling quick retrieval of data values • For even faster retrieval, you can precalculate data values and store the precalculated results in aggregations
  48. 48. Aggregations – The Down Side • Lengthy process • Requires extra disk space – Sometimes yes, but it is rare that default aggs are more than 40 percent the size of the input data. • You want to balance query time and storage space
  49. 49. Intelligent Aggregations for ASO • You can define hard restrictions for a dimension – Default (no restriction for primary hierarchy, no aggregation for alternate hierarchies) – Consider all levels – Do not aggregate – Consider top level only • (you only query top level) – Never aggregate to intermediate levels • (you only query level zero or top dimension) • Level based weighting – provide levels to consider • Process – ASO considers hints when creating aggregations – Attempts to create the most useful aggregations based on hints
  50. 50. Query Hints • You can define ―soft restrictions‖ as a query hint • Just select a representative member (any member) • Essbase will take this into consideration when creating aggregation views
  51. 51. Query Hints
  52. 52. Design Considerations • BSO – Complex calculations and allocations – Write back at upper levels from end users are required • ASO – Large analysis applications with many dimensions and members – Rolling up and analyzing large volumes of data • Both ASO and BSO – Take advantage of the strengths of both database types
  53. 53. Consider Using Both ASO and BSO BSO Budget Partition Product ASO SKU Analysis
  54. 54. Consider Using Both ASO and BSO – version 11 Product ASO SKU Analysis Partition BSO Budget
  55. 55. ASO vs. BSO Recap • What are the similarities between ASO and BSO? – Building dimensions – Loading data (level zero only for ASO) – Retrieving data • What are the differences? – Many dimensions, many members – No calc scripts – Use MDX member formulas – Aggregations for improved query performance • Lots of improvements in System 9 and version 11 – Understand the limitations in early versions of Essbase ASO – Don‘t miss the new features in 9.3.1 and 11x
  56. 56. Thank You!! Edward Roske eroske@interrel.com BLOG: LookSmarter.blogspot.com WEBSITE: www.interrel.com TWITTER: ERoske