Inside Exalytics and Oracle TimesTen for ExalyticsMark Rittman, Technical Director, Rittman MeadHotsos Symposium 2012, Dal...
Mark Rittman• Mark Rittman, Co-Founder of Rittman Mead• Oracle ACE Director, specialising in Oracle BI&DW• 14 Years Experi...
About Rittman Mead• Oracle BI and DW platinum partner• World leading specialist partner for technical excellence, solution...
Agenda• Introduction, and the context for Exalytics• Oracle Exalytics BI Machine• Exalytics Under the Covers• TimesTen for...
What is Oracle Business Intelligence?• Oracle BI provides a set of web-based tools for reporting and analyzing data• Based...
Oracle BI within the Oracle BI/DW Technology Stack• BI provides the query and presentation element for BI/DW• Catalog of r...
How Oracle Business Intelligence 11g Works• Data sources are mapped into an internal  dimensional model called the Oracle ...
In a BI/DW System, Where Does the Time and Heavy Lifting Go?• Ideal is to return data to users with consistent sub-second ...
BI System Performance Objectives• Every query returns data immediately• No performance penalty for drilling-into and inves...
DemonstrationOBIEE Dashboard without Exalytics OptimizationsT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries...
Potential Solutions to Query Performance Issues• Buy an Exadata box (or several Exadata boxes...)• Add materialized views,...
Another Solution - In-Memory Databases• Move some, or all, of the data being queried into RAM• Sub-second response times• ...
Oracle Exalytics In-Memory Machine X2-4• BI Appliance that combines hardware + BI software• Sun hardware, Four Intel Xeon ...
Exalytics as the Exa-Machine for BI• Runs the BI layer on a high-performance, multi-core, 1TB server• In-memory cache used...
Enables High-Density Analysis of Big Data• BI tier is rarely the bottleneck, but it can be if very dense visualizations ar...
What is Exalytics?• Hardware Element   ‣ Sun Fire X4470 M2 server   ‣ 1TB RAM, 40 Cores, 3.6TB HDD• Software Element   ‣ O...
Exalytics Architecture• Developer tools used for defining                                                          Develop...
Exalytics Under the Covers - How Does it Work?• Exalytics brings together different technologies, which are still standalo...
In-Memory Adaptive Data Mart• Commonly-used aggregates are copied into Oracle TimesTen for Exalytics• Past query patterns ...
In-Memory Intelligent Results Cache• OBIEE has an advanced result cache, which can support:   ‣ Queries matching previous ...
In-Memory Essbase Cubes• Essbase is being enhanced to provide better in-memory capabilities• Some features will be common ...
So What is Oracle TimesTen (for Exalytics?)• TimesTen dates back to 1998, acquired by Oracle in 2005   ‣ It is an In-Memor...
Oracle TimesTen for Exalytics• New version of TimesTen specifically for Exalytics (and only available with Exalytics)• Sup...
TimesTen and OBIEE Architecture• Oracle BI Server communicates with TimesTen through TimesTen Client• Summary Advisor, and...
TimesTen Extensions for Exalytics• Notable new functionality in this version includes:  ‣ Columnar Compression  ‣ Analytic...
TimesTen for Exalytics Columnar Compression• The version of TimesTen for Exalytics features “In-memory columnar compressio...
Enabling Columnar Compression in the Administration Tool• When initially created, the TimesTen database used by Exalytics ...
Summary Advisor within Oracle BI Administration Tool• Utility within Oracle BI Administrator tool that recommends aggregat...
Underlying Enhanced Usage Tracking Data• Historically, usage tracking has been tracked using S_NQ_ACCT  ‣ Holds basic usag...
Required Configuration Steps for Exalytics/Summary Advisor• Configure TimesTen DSN for in-memory aggregates• Configure opm...
DemonstrationThe Oracle Exalytics Summary AdvisorT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmea...
Creating In-Memory Aggs Step 1 : Select Facts and Time Scope• Select Tools > Utilities > Summary Advisor• Select facts fro...
Creating In-Memory Aggregates Step 2 : Set Min Query Time, Target• Select minimum elapsed time for queries, to consider fo...
Creating In-Memory Aggregates Step 3 : Set Script Target Location• Summary Advisor will generate a script that has to be r...
Creating In-Memory Aggregates Step 4 : Define Stopping Criteria• Select maximum run-time for summary advisor (0=unlimited ...
Creating In-Memory Aggregates Step 5 : Run + Filter Recommends• Aggregate recommendations can then be generated• Filter pa...
Creating In-Memory Aggregates Step 6 : Generate Aggregates• Wizard then creates the script, which can be run through the n...
What Does the Summary Advisor Create?• Aggregate tables in the TimesTen database   ‣ One per aggregate fact table   ‣ One ...
Oracle BI Server then Redirects Queries to Aggregates• When users then query the dashboard, queries will get redirected to...
Summary Advisor Usage Recommendations & Limitations• Start by generating and testing single aggregates each time  ‣ If one...
Alternative Use Case : Manually Load “Hot” Data into TimesTen• Summary Advisor only loads recommended aggregates into Time...
Troubleshooting and Diagnostics• Enable detailed logging for summary advisor scripts   ‣ Session variable in RPD: LOGLEVEL...
How Does it Perform?• Have performed some software-only testing, but Exalytics box not yet available• Performed some stand...
DemonstrationOBIEE Dashboard following Exalytics OptimizationsT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiri...
In-Memory Aggregate Caching Benefits & Limitations• For individual dashboards, analyses etc, caching works well• For gener...
Summary• Well it certainly looks interesting...!• Exalytics’ 40 core server with 1TB of RAM provides a very fast hardware ...
More Information• Thank you for attending this presentation• More information can be found at http://www.rittmanmead.com• ...
Inside Exalytics and Oracle TimesTen for ExalyticsMark Rittman, Technical Director, Rittman MeadHotsos Symposium 2012, Dal...
Upcoming SlideShare
Loading in …5
×

Inside Oracle Exalytics and Oracle TimesTen for Exalytics - Hotsos 2012

6,489 views

Published on

Published in: Technology, News & Politics
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,489
On SlideShare
0
From Embeds
0
Number of Embeds
1,639
Actions
Shares
0
Downloads
294
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Inside Oracle Exalytics and Oracle TimesTen for Exalytics - Hotsos 2012

  1. 1. Inside Exalytics and Oracle TimesTen for ExalyticsMark Rittman, Technical Director, Rittman MeadHotsos Symposium 2012, DallasT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  2. 2. Mark Rittman• Mark Rittman, Co-Founder of Rittman Mead• Oracle ACE Director, specialising in Oracle BI&DW• 14 Years Experience with Oracle Technology• Regular columnist for Oracle Magazine• Author of forthcoming Oracle Press book on OBIEE 11g• Writer for Rittman Mead Blog : http://www.rittmanmead.com/blog• Email : mark.rittman@rittmanmead.com• Twitter : @markrittman• First time at the Hotsos Symposium ;-) T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  3. 3. About Rittman Mead• Oracle BI and DW platinum partner• World leading specialist partner for technical excellence, solutions delivery and innovation in Oracle BI• Approximately 50 consultants worldwide• All expert in Oracle BI and DW• Offices in US (Atlanta), Europe, Australia and India• Skills in broad range of supporting Oracle tools: ‣ OBIEE ‣ OBIA ‣ ODIEE ‣ Essbase, Oracle OLAP ‣ GoldenGate ‣ Exadata ‣ Endeca T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  4. 4. Agenda• Introduction, and the context for Exalytics• Oracle Exalytics BI Machine• Exalytics Under the Covers• TimesTen for Exalytics• Summary T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  5. 5. What is Oracle Business Intelligence?• Oracle BI provides a set of web-based tools for reporting and analyzing data• Based around dashboards, built around graphs, tables, pivots etc• Multiple server architecture that splits data access / presentation elements• Based on Siebel Analytics technology• Connects to multiple data sources ‣ Oracle DB & Exadata ‣ SQL Server, DB/2, Teradata ‣ Essbase, Oracle OLAP, MSAS ‣ ADF View Objects etc• Now re-engineered as part of FMW11g T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  6. 6. Oracle BI within the Oracle BI/DW Technology Stack• BI provides the query and presentation element for BI/DW• Catalog of reports, analyses, alerts, KPIs• Additional calculation and data federation capabilities Web Server• Aggregate navigation• Security, scoping of data Oracle BI ERP/Apps DW Essbase SAN / NAS T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  7. 7. How Oracle Business Intelligence 11g Works• Data sources are mapped into an internal dimensional model called the Oracle BI Repository• Users request data through analyses, which translate into logical SQL requests• The Oracle BI Server converts these logical SQL requests into one or more physical SQL, MDX or other queries• These are then processed by the underlying data sources, using their optimizations• Data is then combined and returned to users in the form of dashboards T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  8. 8. In a BI/DW System, Where Does the Time and Heavy Lifting Go?• Ideal is to return data to users with consistent sub-second response times• But large data volumes and unpredictable queries can make this tough• When analyzing query response times, most of the times goes in ‣ The database returning the query results ‣ The storage tier returning data to the database ‣ The network connecting storage and the database• Tuning, efficient queries and Exadata can help with this Oracle BI ‣ But queries can still sometimes take >2 mins or so ‣ And query times can be erratic - Some are instant ERP/Apps DW - Some take 2-3 minutes - Depends on load on the database SAN / NAS T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  9. 9. BI System Performance Objectives• Every query returns data immediately• No performance penalty for drilling-into and investigating the data• Response time is consistent• Response-time stays constant even when you scale up to 100’s of usersT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  10. 10. DemonstrationOBIEE Dashboard without Exalytics OptimizationsT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  11. 11. Potential Solutions to Query Performance Issues• Buy an Exadata box (or several Exadata boxes...)• Add materialized views, more indexes, in-database OLAP to the DW• Move data into an OLAP server such as Essbase• But... ‣ Can be expensive ‣ Adds to DBA complexity ‣ Requires new skills (Essbase etc) ‣ Scaling issues for OLAP serversT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  12. 12. Another Solution - In-Memory Databases• Move some, or all, of the data being queried into RAM• Sub-second response times• Consistent response times• A solution being applied elsewhere for similar problems Oracle BI In-Memory DB/Cache ERP/Apps DW SAN / NAS T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  13. 13. Oracle Exalytics In-Memory Machine X2-4• BI Appliance that combines hardware + BI software• Sun hardware, Four Intel Xeon E7-4800 series processors• 1TB of RAM, made up of 64 16GB DDR3 ECC registered DIMMs• QDR (40Gb/second) InfiniBand connectivity to Exadata & Exalogic• 10Gb and 1Gb Ethernet interfaces• Optimized version of Oracle BI EE• In-memory version of Oracle Essbase• 3.6TB of disk, for internal use• Integrated Lights-Out Mgmt (ILOM)• Sold as an appliance/engineered system ‣ Software only available in combination with hardwareT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  14. 14. Exalytics as the Exa-Machine for BI• Runs the BI layer on a high-performance, multi-core, 1TB server• In-memory cache used to accelerate the BI part of the stack• If Exadata addresses 80% of the query performance, Exalytics addresses the remaining 20% Oracle BI ‣ Consistent response times for queries ‣ In-memory caching of aggregates ‣ 40 cores for high concurrency In-Memory DB/Cache ‣ Re-engineered BI and OLAP software that assumes 40 cores and 1TB RAM ERP/Apps DW T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  15. 15. Enables High-Density Analysis of Big Data• BI tier is rarely the bottleneck, but it can be if very dense visualizations are used ‣ Sparklines, grid of charts etc• Exalytics’ 40 cores and 1TB RAM make higher density presentation viable ‣ Single query sent to the database ‣ Exalytics breaks data up to create microcharts• Also helps support high numbers of concurrent users (100+) T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  16. 16. What is Exalytics?• Hardware Element ‣ Sun Fire X4470 M2 server ‣ 1TB RAM, 40 Cores, 3.6TB HDD• Software Element ‣ OBIEE 11.1.1.6 with Exalytics Enhancements ‣ Oracle Essbase 11.1.2 with Exalytics Enhancements ‣ Oracle TimesTen 11.2.2.2 for Exalytics ‣ Runs on 64-bit Oracle Linux (Exalogic distribution)• OBIEE and Essbase are licensed as Oracle BI Foundation• Exalytics features can only be used in conjunction with Exalytics hardwareT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  17. 17. Exalytics Architecture• Developer tools used for defining Developer Workstation and maintaining aggregate definitions Oracle BI Administration Tool• Client tools for OBIEE repository, Summary Advisor Wizard nqcmd TimesTen and Oracle Database TimesTen Client SQL Developer Oracle Database 11g Client Oracle Business Intelligence• Exalytics layer contains OBIEE, Oracle TimesTen for Exalytics WLS Admin Oracle BI Server TimesTen and Essbase System Comp. Aggregate WLS Managed BI Server Schema (not pictured) Server• Database tier contains FMW Database for RCU Schemas database repository, including Enhanced Usage Tracking Summary Statistics usage tracking and summary statistics T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  18. 18. Exalytics Under the Covers - How Does it Work?• Exalytics brings together different technologies, which are still standalone products in their own right• To harmonise and optimise their use within Exalytics, it utilises the following techniques: ‣ In-Memory Adaptive Data Mart ‣ In-Memory Intelligent Result Cache ‣ In-Memory Cubes• Some of these are genuine "secret sauce" ‣ New functionality and algorithms ‣ You can only get them through licensing Exalytics• Others are descriptions of DW/BI strategies, or existing product functionality, extended to take advantage of the capacity for processing in memory that Exalytics has T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  19. 19. In-Memory Adaptive Data Mart• Commonly-used aggregates are copied into Oracle TimesTen for Exalytics• Past query patterns are analyzed and suitable aggregates recommended• Oracle BI Server then uses these aggregates to make queries run faster• Aggregates change over time in response to changes in query patterns TimesTen BI Server• Tools are provided for managing Exalytics and populating these aggregates Aggregates Data Warehouse Detail-level Data T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  20. 20. In-Memory Intelligent Results Cache• OBIEE has an advanced result cache, which can support: ‣ Queries matching previous ones ‣ Queries matching a subset of the results of a cached query resultset ‣ Queries aggregating the results of a cached query resultset• Stored in cache files on the OBIEE server filesystem• With Exalytics, the 1TB of RAM typically means that these files are held in memory• RAMdisk can also be created for BI Server & Presentation Server TimesTen BI Server working area files Exalytics In-Memory Result Cache T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  21. 21. In-Memory Essbase Cubes• Essbase is being enhanced to provide better in-memory capabilities• Some features will be common to all Essbase releases, some will require Exalytics ‣ Smart Storage Manager stores frequently access blocks in-memory ‣ In-Memory Cache optimizes block access ‣ Lock Manager improves parallelism T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  22. 22. So What is Oracle TimesTen (for Exalytics?)• TimesTen dates back to 1998, acquired by Oracle in 2005 ‣ It is an In-Memory Database ‣ The whole of the database is loaded into memory ‣ It is persisted to disk, for full recovery capabilities• Originally designed for exceptionally fast and highly concurrent read/write of single rows of data (airline booking, trading desks, etc)• Can be deployed in two ways: ‣ Standalone database ‣ In conjunction with Oracle, as part of In Memory Database Cache• Previous versions of TimesTen were usable with OBIEE, but... ‣ Because of its focus on fast OLTP, was slow for reporting ‣ Limited analytical capabilities T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  23. 23. Oracle TimesTen for Exalytics• New version of TimesTen specifically for Exalytics (and only available with Exalytics)• Support for analytic functions ‣ Perform all the processing at source ‣ Combine with being in-memory = should be very fast• Column compression ‣ Whitepaper cites 5x ‣ Given the hardware capacity, we could seriously contemplate loading the whole Data Warehouse into memory ‣ Opens up lots of interesting design potential• We can load aggregates into TimesTen, leave base data at source, and use OBIEE’s Vertical Federation capability to seamlessly report across both ‣ All hidden from the end-user, all they will know is that their reports run fast! T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  24. 24. TimesTen and OBIEE Architecture• Oracle BI Server communicates with TimesTen through TimesTen Client• Summary Advisor, and nqcmd use Oracle BI Server to access TimesTen• Typical single TimesTen database per Exalyics machine ‣ Max TimesTen database size around 300MB - Due to need to set aside equal Summary Oracle BI nqcmd Advisor Server Temp size for the Perm size selected• Clustered Exalytics boxes can be daisy-chained ODBC together using InfinBand connections Checkpoint Files ‣ For HA scenarios, does not increase available RAM TimesTen Memory-Resident ‣ Summary advisor scripts write to both TimesTen Database databases, replicating aggregates ‣ TimesTen databases can be “wired together” for failover/HA purposes Log Files T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  25. 25. TimesTen Extensions for Exalytics• Notable new functionality in this version includes: ‣ Columnar Compression ‣ Analytic functions: AVG, SUM, COUNT, MAX, MIN, DENSE_RANK, RANK, ROW_NUMBER, FIRST_VALUE and LAST_VALUE ‣ Analytic clauses: OVER PARTITION BY and OVER ORDER BY ‣ Grouping clauses: GROUP BY CUBE, GROUP BY ROLLUP, GROUP BY GROUPING SETS ‣ Grouping functions: GROUP, GROUPING_ID, GROUP_ID ‣ WITH clause ‣ Aggregate expressions over DISTINCT expressionsT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  26. 26. TimesTen for Exalytics Columnar Compression• The version of TimesTen for Exalytics features “In-memory columnar compression”• Compression is specified against individual columns, or groups of columns• Some will compress better than others ‣ A compression group is where the same compression dictionary is applied to values in the columns within it ‣ Examples of use would be surname & maiden name, home_city & city_of_birth• Compressed columns take slightly longer to load, but should: ‣ be faster to query than uncompressed columns ‣ take up less disk space ‣ Compression typically reduces the 3x overhead of usual TT storage (compared to source data in a file) back to the original dataset size T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  27. 27. Enabling Columnar Compression in the Administration Tool• When initially created, the TimesTen database used by Exalytics is set up without columnar compression enabled• To enable, check the Compress Columns checkbox in the Database Features dialog ‣ Also, disable WITH_CLAUSE_SUPPORTED ‣ TimesTen can use them in SELECTS, but not during data load (disable during loads only?) T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  28. 28. Summary Advisor within Oracle BI Administration Tool• Utility within Oracle BI Administrator tool that recommends aggregates• Bases recommendations on usage tracking and summary statistics data• Captured based on past activity• Runs an iterative algorithm that searches, each iteration, for the best aggregate Proposed Aggregate Fact Table 20 rows Row Count X = Predicted Previous Query Time 30 secs New Query 0.06 secs Time Exisitng Base Fact Table 100000 rows Row Count T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  29. 29. Underlying Enhanced Usage Tracking Data• Historically, usage tracking has been tracked using S_NQ_ACCT ‣ Holds basic usage tracking statistics + logical SQL query• Now supplemented by S_NQ_DB_ACCT ‣ Extra usage tracking information, includes physical SQL• Exalytics Summary Advisor uses S_NQ_SUMMARY_ADVISOR ‣ Contains summary statistics, execution time etc ‣ Gathered at same time as usage tracking when Exalytics is enabed ‣ Contents can be derived from usage tracking if neededT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  30. 30. Required Configuration Steps for Exalytics/Summary Advisor• Configure TimesTen DSN for in-memory aggregates• Configure opmn.xml in middleware home for TIMESTEN_DLL• Create TimesTen database, user account and permissions4. Create sample table in TT database using SQL*Developer• Import or create the TT table in RPD• Add usage tracking and summary statistics tables to RPD• Enable usage tracking via EM/MBean• Restart system Assumes system has been enabled for Exalytics by Oracle ACS or PartnerT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  31. 31. DemonstrationThe Oracle Exalytics Summary AdvisorT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  32. 32. Creating In-Memory Aggs Step 1 : Select Facts and Time Scope• Select Tools > Utilities > Summary Advisor• Select facts from the BMM layer to consider for the exercise ‣ Can also generate stats from usage tracking data if none exists (upgrade etc) ‣ And can load parameters from a file• Select start and end date for wizard to consider (optional) T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  33. 33. Creating In-Memory Aggregates Step 2 : Set Min Query Time, Target• Select minimum elapsed time for queries, to consider for exercise• Select physical database details for TT database T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  34. 34. Creating In-Memory Aggregates Step 3 : Set Script Target Location• Summary Advisor will generate a script that has to be run through nqcmd.exe ‣ Same type of script as generated by Aggregate Persistence Wizard• Script can be re-run to refresh aggregates periodically ‣ Drops and recreates aggregates, no incremental refresh T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  35. 35. Creating In-Memory Aggregates Step 4 : Define Stopping Criteria• Select maximum run-time for summary advisor (0=unlimited time) ‣ Useful when lots of summary statistics data to process• Set limit for how little each additional aggregate improves performance ‣ Stops lots of little aggregates that barely improve performance being recommended ‣ Ideally, set to around 10% for good balance between good recommendations and good dashboard performance T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  36. 36. Creating In-Memory Aggregates Step 5 : Run + Filter Recommends• Aggregate recommendations can then be generated• Filter page lets you select which ones to implement ‣ Suggested approach is to add one aggregate at a time, test and implement ‣ For smaller, more tested RPDs you can select all if required T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  37. 37. Creating In-Memory Aggregates Step 6 : Generate Aggregates• Wizard then creates the script, which can be run through the nqcmd utility ‣ Command-line access to the BI Server• Run command either at the developer workstation, or on the Exalytics server• Re-run at any time to refresh aggregate tables with new datacd [obiee_client_home]oraclebiorahomebifoundationserverbinnqcmd.exe-d exalytics -u biadmin -p welcome1-s c:summary_advisor_scriptsagg_wiz.sql T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  38. 38. What Does the Summary Advisor Create?• Aggregate tables in the TimesTen database ‣ One per aggregate fact table ‣ One per shared dimension• Physical tables in the Oracle BI Repository physical layer• Logical Table Sources in the BMM layer ‣ All mapped in and active T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  39. 39. Oracle BI Server then Redirects Queries to Aggregates• When users then query the dashboard, queries will get redirected to the TimesTen tables• Aggregate navigation is additive, so single aggregates should satisfy other queries TimesTen BI Server ExalyticsSending query to database namedTT_AGGR_STORE, Aggregatesconnection pool named TT_AGGR_STORE,logical request hash 9b1a12f5, physicalrequest hash 2f1461f5: [[selectsum(T44087.Billed_Qua0000A5C2) as c1,sum(T44087.Revenue0000A5BC) as c2fromag_2123023723 T44087 Data Warehouse Detail-level Data T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  40. 40. Summary Advisor Usage Recommendations & Limitations• Start by generating and testing single aggregates each time ‣ If one aggregate creation fails, whole process fails; tricky to “un-pick” process ‣ Focus on a few facts, recover any errors, then move to next facts• Constraint time window to a representative range• Be aware of how best to use the repository and source data ‣ Run “estimate levels” prior to Summary Advisor, helps to ensure unique level keys ‣ Do not run against value-based hierarchies, or skip-level/ragged (not supported) ‣ Cleanse data before running Summary Advisor - ensure unique level keys, no NULLs in dimension key references ‣ Implement indices on the source systems T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  41. 41. Alternative Use Case : Manually Load “Hot” Data into TimesTen• Summary Advisor only loads recommended aggregates into TimesTen ‣ Usually a good approach, as based on historic usage and expected gain• It is possible to manually load source data into TimesTen ‣ Aggregates using Aggregate Persistence Wizard ‣ ETL using ODI ‣ Replication using Golden Gate• Aggregates would get automatically mapped in RPD• ETL’d data would need manual mapping ‣ Replacement LTSs ‣ Fragmentation ‣ Additional LTSs• Remember TT RAM restriction: maximum of around 300GB T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  42. 42. Troubleshooting and Diagnostics• Enable detailed logging for summary advisor scripts ‣ Session variable in RPD: LOGLEVEL >=4 ‣ Prefix aggregate statements - set variable LOGLEVEL=4 : delete aggregates - set variable LOGLEVEL=4 : create aggregates• Logs in nqquery.log, Errors in nqserver.log• Enable detailed logging for the Admin tool ‣ [instances]instance1configOracleBIServerComponentcoreapplication_obis1l ogconfig.xml ‣ <Level> <IncidentError>1</IncidentError> <Error>1</Error> <Warning>1</Warning> <Trace>32</Trace> </Level>T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  43. 43. How Does it Perform?• Have performed some software-only testing, but Exalytics box not yet available• Performed some standalone testing of TimesTen, and also compared TimesTen queries to standard queries• Difficult to judge without ‣ Running on an Exalytics box - BI Server has optimizations for this hardware ‣ Representative data volumes• Keep an eye on our blog for findings once the box is delivered ‣ http://www.rittmanmead.com/blog T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  44. 44. DemonstrationOBIEE Dashboard following Exalytics OptimizationsT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  45. 45. In-Memory Aggregate Caching Benefits & Limitations• For individual dashboards, analyses etc, caching works well• For general aggregate re-usability, depends on rest of model, query types etc ‣ Use in conjunction with results caching; similar benefits and limitations• Caching (of aggregates) makes TimesTen not suitable for real-time (OLTP) sources• Needs to be used in conjunction with database and storage tuning ‣ Ideal link is with Exadata (uses InfinBand) T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  46. 46. Summary• Well it certainly looks interesting...!• Exalytics’ 40 core server with 1TB of RAM provides a very fast hardware platform• Addresses the “mid-tier” performance issue for BI systems• Infiniband connnection makes integration with Exadata particularly effectivee• TimesTen for Exalytics provides the in-memory database cache• Tools provided as part of Exalytics make it possible to intelligently populate the cache• Benefits for Essbase users in-terms of specific optimizations for Essbase• Looking forward now to getting our hands on the hardware ‣ Contact us for a Proof of Concept exercise, using our “Exa-lab” in Brighton, UK T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  47. 47. More Information• Thank you for attending this presentation• More information can be found at http://www.rittmanmead.com• Contact us at info@rittmanmead.com or mark.rittman@rittmanmead.com• Look out for our book, “Oracle Business Intelligence Developers Guide” due Q3 2012• Follow-us on Twitter (@rittmanmead) or Facebook (facebook.com/rittmanmead) T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
  48. 48. Inside Exalytics and Oracle TimesTen for ExalyticsMark Rittman, Technical Director, Rittman MeadHotsos Symposium 2012, DallasT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

×