Oracle Exalytics - Tips and Tricks from the FieldMark Rittman, Technical Director, Rittman MeadUKOUG Engineered Systems Su...
Mark Rittman• Mark Rittman, Co-Founder of Rittman Mead• Oracle ACE Director, specialising in Oracle BI&DW• 14 Years Experi...
About Rittman Mead• Oracle BI and DW gold partner• World leading specialist partner for technical excellence, solutions de...
Rittman Mead in the Developer/User CommunityT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com...
Agenda• What is Oracle Exalytics (stop me if you’ve heard this before...)• How Exalytics works (again, stop me...)• Exalyt...
What is Oracle Exalytics?• Hardware Element   ‣ Sun Fire X4470 M2 server   ‣ 1TB RAM, 40 Cores, 3.6TB HDD• Software Elemen...
Rittman Mead and Oracle Exalytics• Rittman Mead are the first Oracle Partner in the UK to buy an Exalytics box• Ordered Fe...
Exalytics as the Exa-Machine for BI• Runs the BI layer on a high-performance, multi-core, 1TB server• In-memory cache used...
Some Example Exalytics Use-Cases• Customers who do not have the time/ability/means to put in-place an aggregation  strateg...
Exalytics Sweet-Spots, in our Experience...• Customers with BI systems that use an underlying data warehouse, with  batch-...
Exalytics Under the Covers - How Does it Work?• Exalytics brings together different technologies, which are still standalo...
In-Memory Adaptive Data Mart• Commonly-used aggregates are copied into Oracle TimesTen for Exalytics• Past query patterns ...
Summary Advisor for Aggregate Recommendation & Creation• Utility within Oracle BI Administrator tool that recommends aggre...
Oracle Exalytics Best Practices, and Experiences from the Field• Installation, Configuration for Multiple Environments, Ma...
Configuring Multiple Environments (Consolidation)• Exalytics main value proposition is around performance   ‣ In-memory ag...
Best Practice for Consolidation : Oracle Virtual Machine• Exalytics is now certified for use with                         ...
High Availability using Clustered Exalytics Servers• Multiple Exalytics servers can be clustered for HA and performance (l...
What Should the TimesTen Adaptive Data Mart be Used For?• Standard approach is to store aggregates in the TimesTen datamar...
Conclusions and Best Practices Around Detail/Aggregate Storage• Best use of OBIEE + TT is how the product was intended to ...
Refreshing Aggregates Within the TimesTen Data Mart• Aggregates are built in TimesTen to support sub-second response times...
Out-of-the-Box Summary Advisor Limitations• To refresh an aggregate, Summary Advisor deletes and rebuilds from scratch• Th...
Alternatives to using Aggregate Persistence• However aggregates are refreshed, they must be included  in the RPD   ‣ Can b...
Incremental refresh of aggregates using GoldenGate and ODI• Changes to base data are replicated by GoldenGate onto Staging...
Custom BI Server ETL Option : Incremental BI Server Refresh• Uses BI Server to do the refresh, but invokes just part of it...
Custom BI Server ETL Option : Incremental BI Server Refresh• Uses a number of “undocumented” but support BI Server ETL fea...
More Information• Thank you for attending, & more info can be found at http://www.rittmanmead.com• Contact us at info@ritt...
Oracle Exalytics - Tips and Tricks from the FieldMark Rittman, Technical Director, Rittman MeadUKOUG Engineered Systems Su...
Upcoming SlideShare
Loading in …5
×

UKOUG Engineered Systems Summit 2013 - Oracle Exalytics, Tips and Tricks from the Field

690
-1

Published on

A presentation at the UKOUG's Engineered Systems event, that goes through Rittman Mead's experiences with Exalytics in the 12 months since purchasing an Exalytics server, and some tips and techniques around multi-tenancy, HA, aggregate refresh and use cases.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
690
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
34
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Aggregate Persistence scripts can also be generated using the Aggregate Persistence Wizard, or by hand. delete aggregates"TT_AGGR_STORE".."DW_AGG"."ag_3110279699";create aggregates "ag_3110279699" for "Sales"."Fact Sales"("Sale Amount","Quantity") at levels ("Sales"."Times (Level-Based, Time)"."Month") using connection pool "TT_AGGR_STORE"."Connection Pool" in "TT_AGGR_STORE".."DW_AGG";
  • No incremental refresh Model Checker and Consistency Checker are useful, but do not eliminate build failures Online RPD changes - Risk of corruption, and also challenges to change management / source control Ploughing through nqquery.log is time consuming and root cause is not always obviousIf the RPD/TimesTen are left in inconsistent state by an aggregate rebuild, it can require manual work to unpick and rectify
  • Mentions ODI, but actually any ETL method would be valid
  • UKOUG Engineered Systems Summit 2013 - Oracle Exalytics, Tips and Tricks from the Field

    1. 1. Oracle Exalytics - Tips and Tricks from the FieldMark Rittman, Technical Director, Rittman MeadUKOUG Engineered Systems Summit, London April 2013T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    2. 2. Mark Rittman• Mark Rittman, Co-Founder of Rittman Mead• Oracle ACE Director, specialising in Oracle BI&DW• 14 Years Experience with Oracle Technology• Regular columnist for Oracle Magazine• Author of two Oracle Press Oracle BI books ‣ Oracle Business Intelligence Developers Guide ‣ Oracle Exalytics Revealed• Writer for Rittman Mead Blog : http://www.rittmanmead.com/blog• Email : mark.rittman@rittmanmead.com• Twitter : @markrittman T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    3. 3. About Rittman Mead• Oracle BI and DW gold partner• World leading specialist partner for technical excellence, solutions delivery and innovation in Oracle BI• Approximately 30 consultants worldwide• All expert in Oracle BI and DW• UK based• Offices in US, Europe (Belgium) and India• Skills in broad range of supporting Oracle tools: ‣ OBIEE ‣ OBIA ‣ ODIEE ‣ Essbase, Oracle OLAP ‣ GoldenGate ‣ Exadata T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    4. 4. Rittman Mead in the Developer/User CommunityT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    5. 5. Agenda• What is Oracle Exalytics (stop me if you’ve heard this before...)• How Exalytics works (again, stop me...)• Exalytics Best Practices, and Experiences from the Field, including... ‣ Installation, Configuration for Multiple Environments, Management ‣ Choosing What Goes in the Adaptive Data Mart ‣ Where Does Exalytics Add Value? ‣ Measuring Exalytics Performance and Metrics ‣ Maintaining and Refreshing the Adaptive Data Mart T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    6. 6. What is Oracle Exalytics?• Hardware Element ‣ Sun Fire X4470 M2 server ‣ 1TB RAM, 40 Cores, 3.6TB HDD• Software Element ‣ OBIEE 11.1.1.6 with Exalytics Enhancements ‣ Oracle Essbase 11.1.2 with Exalytics Enhancements ‣ Oracle TimesTen 11.2.2.2 for Exalytics ‣ New in v1.1 : Support for Oracle Endeca Information Discovery, Golden Gate, ODI ‣ Runs on 64-bit Oracle Linux (Exalogic distribution)• OBIEE and Essbase are licensed as Oracle BI Foundation• Exalytics features can only be used in conjunction with Exalytics hardwareT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    7. 7. Rittman Mead and Oracle Exalytics• Rittman Mead are the first Oracle Partner in the UK to buy an Exalytics box• Ordered February 2012 through Arrow (reseller), delivered end-April 2012• Hosted in a secure datacenter in London Docklands, access via VPN• To be used for customer PoCs, training, testing and possible cloud hosting T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    8. 8. Exalytics as the Exa-Machine for BI• Runs the BI layer on a high-performance, multi-core, 1TB server• In-memory cache used to accelerate the BI part of the stack• If Exadata addresses 80% of the query performance, Exalytics addresses the remaining 20% Oracle BI ‣ Consistent response times for queries ‣ In-memory caching of aggregates ‣ 40 cores for high concurrency In-Memory DB/Cache ‣ Re-engineered BI and OLAP software that assumes 40 cores and 1TB RAM ERP/Apps DW T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    9. 9. Some Example Exalytics Use-Cases• Customers who do not have the time/ability/means to put in-place an aggregation strategy for their dashboards, or who understand how labour-intensive this is• Customers who have done what they can with summaries, indexes etc, but need that extra 10% or so of performance, particularly around response-time consistency• Customers who want to create rich, visual dashboards that would otherwise stretch the BI Presentation Server• Customers who need to support high numbers of concurrent users• Customers running Hyperion Planning and Budgeting who want to reduce the planning time cycle, re-calculate data faster and generally do more in less time• Customers who are moving to a single supplier, engineered-systems hardware strategy, using Exadata for their databases, Exalogic for their Java application hosting, and now Exalytics for their BI• Customers looking to consolidate multiple BI systems into a single large instance T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    10. 10. Exalytics Sweet-Spots, in our Experience...• Customers with BI systems that use an underlying data warehouse, with batch-load refresh cycles that permit pre-aggregation of data• Customers who have a large number of users running the same reports and dashboards as each other• Customer with BI systems that aggregate data via predefined hierarchies• Customers with reasonably simple and well-tested / validated RPDs• Customers using Essbase for planning/budgeting applications, and who want to move the cube into memory• Customers moving from v10 to v11 OBIEE, with little in the way of visual, interactions etc, and use Exalytics as the platform for delivering these T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    11. 11. Exalytics Under the Covers - How Does it Work?• Exalytics brings together different technologies, which are still standalone products in their own right• To harmonise and optimise their use within Exalytics, it utilises the following techniques: ‣ In-Memory Adaptive Data Mart ‣ In-Memory Intelligent Result Cache ‣ In-Memory Cubes• Some of these are genuine "secret sauce" ‣ New functionality and algorithms ‣ You can only get them through licensing Exalytics• Others are descriptions of DW/BI strategies, or existing product functionality, extended to take advantage of the capacity for processing in memory that Exalytics has T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    12. 12. In-Memory Adaptive Data Mart• Commonly-used aggregates are copied into Oracle TimesTen for Exalytics• Past query patterns are analyzed and suitable aggregates recommended• Oracle BI Server then uses these aggregates to make queries run faster• Aggregates change over time in response to changes in query patterns TimesTen BI Server• Tools are provided for managing Exalytics and populating these aggregates Aggregates Data Warehouse Detail-level Data T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    13. 13. Summary Advisor for Aggregate Recommendation & Creation• Utility within Oracle BI Administrator tool that recommends aggregates• Bases recommendations on usage tracking and summary statistics data• Captured based on past activity• Runs an iterative algorithm that searches, each iteration, for the best aggregate T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    14. 14. Oracle Exalytics Best Practices, and Experiences from the Field• Installation, Configuration for Multiple Environments, Management• Choosing What Goes in the Adaptive Data Mart• Where Does Exalytics Add Value?• Measuring Exalytics Performance and Metrics• Maintaining and Refreshing the Adaptive Data Mart T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    15. 15. Configuring Multiple Environments (Consolidation)• Exalytics main value proposition is around performance ‣ In-memory aggregates speeding up query performance• However many customers wish to use Exalytics for consolidation ‣ Run multiple environments (test, dev, prod) on a single server ‣ Consolidate multiple applications on a single server• Desire is usually down to cost (multiple physical servers, per-processor licensing) Single Exalytics Server Exalytics Svr 1 Exalytics Svr 2 Exalytics Svr 3 MWHOME1 MWHOME2 MWHOME3 MWHOME1 MWHOME1 MWHOME1 DEV DEV DEV DEV TEST PROD vs. T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    16. 16. Best Practice for Consolidation : Oracle Virtual Machine• Exalytics is now certified for use with Single Exalytics Server Oracle Virtual Machine, Oracle’s OVM Guest 1 OVM Guest 2 OVM Guest 3 hard-partitioning virtualization platform MWHOME1 MWHOME1 MWHOME1• OVM is installed first, on Exalytics “bare metal”, with OVM Repository on external DEV DEV DEV DB server (as with BIPLATFORM, MDS)• Exalytics VM templates then used to create required environments (dev, test etc)• Also allows for licensing in smaller OVM Hypervisor (Oracle VM Server) per-processor license blocks (5, 10, 15 etc)• OVM provides for resource caging, Oracle VM Repository isolating environments from each other Oracle VM Manager T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    17. 17. High Availability using Clustered Exalytics Servers• Multiple Exalytics servers can be clustered for HA and performance (load balancing) ‣ Requires hardware load balancer above clustered servers• Linked together via InfiniBand (one for clustering, one for connection to Exadata)• Note: does not double available RAM - aggs are replicated in each TT database ‣ Note that Summary Advisor / nqcmd does not replicate agg tables - manual replication required• HA is between TT instances, and OBIEE managed server + system components T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    18. 18. What Should the TimesTen Adaptive Data Mart be Used For?• Standard approach is to store aggregates in the TimesTen datamart ‣ Aggregated by the source DB, aggregates then cached in TT database• Other approaches could be used, however ‣ Store whole detail-level dataset in the TT database ‣ Store just recent detail-level data in TT, and use OBIEE’s fragmentation feature ‣ Use Essbase to hold the aggregates instead ‣ Use Essbase to hold detail-level + aggregates• Where does Exalytics add value?• What should go in the TimesTen database? T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    19. 19. Conclusions and Best Practices Around Detail/Aggregate Storage• Best use of OBIEE + TT is how the product was intended to be used ‣ Use TimesTen for storing (caching) pre-created aggregates ‣ Aggregates created by source DB (Oracle, SQL Server etc) ‣ Aggregation not performed by TimesTen (as it would be if it held detail-level data)• IMDB interesting approach, but relies on TT performing aggregation (slow)• Same goes for holding whole detail-level source in TimesTen• Also, bear in mind that majority of performance benefit may come from re-hosting OBIEE application on the Exalytics hardware ‣ Particularly with Exadata sources, source DB may even be fastest host for aggs ‣ Test for your data, do not pre-suppose outcome T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    20. 20. Refreshing Aggregates Within the TimesTen Data Mart• Aggregates are built in TimesTen to support sub-second response times• Summary Advisor tool suggests aggregate dimensionality and grain, generates script• BI Server’s Aggregate Persistence executes script: 1. Create TimesTen aggregate table 2. Populate TimesTen aggregate 3. Update RPD online with new aggregate metadata• Handles supporting dimensions too Base data Aggregates OBIEE Aggregate Persistence Aggregate tables created Aggregate tables TimesTen loaded from base data RPD updated with new aggregate mappings RPD T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    21. 21. Out-of-the-Box Summary Advisor Limitations• To refresh an aggregate, Summary Advisor deletes and rebuilds from scratch• The RPD is edited directly on the BI Server each time an aggregate is created or rebuilt• Build failures can be difficult to debug• If a rebuild fails it can leave the RPD in an inconsistent state with TimesTen This is that the error the user sees After a failed build, the aggregates are nqquery.log suggests possible errors still in the RPD, but no longer exist in but no clear root cause TimesTen T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    22. 22. Alternatives to using Aggregate Persistence• However aggregates are refreshed, they must be included in the RPD ‣ Can be done manually ‣ Aggregate Persistence is useful for this• Complete refresh of aggregate data ‣ Extract the SQL that OBIEE generates in Aggregate Persistence, run this through ODI ‣ Write bespoke aggregate refresh code in ODI• Incremental refresh using GoldenGate and ODI ‣ Instead of rebuilding aggregates in their entirety each time, only update the part of the aggregate that has changed on the base data T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    23. 23. Incremental refresh of aggregates using GoldenGate and ODI• Changes to base data are replicated by GoldenGate onto Staging database• ODI Interfaces apply deltas to the TimesTen aggregates Aggregates Base data GoldenGate CDC staging data ODI TimesTen (any supported DB) Oracle TimesTen ODI GoldenGate GoldenGate JDBC client agentT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    24. 24. Custom BI Server ETL Option : Incremental BI Server Refresh• Uses BI Server to do the refresh, but invokes just part of it - the data refresh part• Avoids unnecessary aggregate table drop/rebuild, and online RPD edit• Uses BI Server logical ETL SQL features to just refresh the TT aggregate table, using only the latest set of detail-level data ‣ Avoids use of additional ETL tools ‣ No additional license cost ‣ Works against any BI Server source T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    25. 25. Custom BI Server ETL Option : Incremental BI Server Refresh• Uses a number of “undocumented” but support BI Server ETL features ‣ POPULATE command for loading data into an RPD table ‣ WHERE clause for limiting the refresh to just the incremental load ‣ INACTIVE_SCHEMAS to stop BI Server refreshing using existing RPD agg table SET VARIABLE DISABLE_CACHE_HIT=1, DISABLE_CACHE_SEED=1, DISABLE_SUMMARY_STATS_LOGGING=1, INACTIVE_SCHEMAS="TimesTen Aggregates".."EXALYTICS"; populate "ag_sales_month" mode ( append table connection pool "TimesTen aggregates"."TT_CP") as select_business_model "Sales"."Fact Sales"."Sale Amount" as "Sale_Amoun000000AD", "Sales"."Dim Times"."Month YYYYMM" as "Month_YYYY000000D0" from "Sales" where "Dim Times"."Month YYYYMM" = VALUEOF("THIS_MONTH");• See http://www.rittmanmead.com/2013/04/incremental-refresh-of-exalytics- for full details including scripts etcT : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    26. 26. More Information• Thank you for attending, & more info can be found at http://www.rittmanmead.com• Contact us at info@rittmanmead.com or mark.rittman@rittmanmead.com• Look out for our books, “Oracle Business Intelligence Developers Guide” and “Oracle Exalytics Revealed”• Follow-us on Twitter (@rittmanmead) or Facebook (facebook.com/rittmanmead) T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    27. 27. Oracle Exalytics - Tips and Tricks from the FieldMark Rittman, Technical Director, Rittman MeadUKOUG Engineered Systems Summit, London April 2013T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×