UKOUG Engineered Systems Summit 2013 - Oracle Exalytics, Tips and Tricks from the Field

Oracle Exalytics - Tips and Tricks from the Field
Mark Rittman, Technical Director, Rittman Mead
UKOUG Engineered Systems Summit, London April 2013

T : +44 (0) 8446 697 995 or (888) 631 1410 (USA) E : enquiries@rittmanmead.com W: www.rittmanmead.com

Mark Rittman

• Mark Rittman, Co-Founder of Rittman Mead
• Oracle ACE Director, specialising in Oracle BI&DW
• 14 Years Experience with Oracle Technology
• Regular columnist for Oracle Magazine
• Author of two Oracle Press Oracle BI books
‣ Oracle Business Intelligence Developers Guide
‣ Oracle Exalytics Revealed
• Writer for Rittman Mead Blog :
http://www.rittmanmead.com/blog
• Email : mark.rittman@rittmanmead.com
• Twitter : @markrittman


About Rittman Mead

• Oracle BI and DW gold partner
• World leading specialist partner for technical excellence, solutions delivery and
innovation in Oracle BI
• Approximately 30 consultants worldwide
• All expert in Oracle BI and DW
• UK based
• Offices in US, Europe (Belgium) and India
• Skills in broad range of supporting Oracle tools:
‣ OBIEE
‣ OBIA
‣ ODIEE
‣ Essbase, Oracle OLAP
‣ GoldenGate
‣ Exadata


Rittman Mead in the Developer/User Community


Agenda

• What is Oracle Exalytics (stop me if you’ve heard this before...)
• How Exalytics works (again, stop me...)
• Exalytics Best Practices, and Experiences from the Field, including...
‣ Installation, Configuration for Multiple Environments, Management
‣ Choosing What Goes in the Adaptive Data Mart
‣ Where Does Exalytics Add Value?
‣ Measuring Exalytics Performance and Metrics
‣ Maintaining and Refreshing the Adaptive Data Mart


What is Oracle Exalytics?

• Hardware Element
‣ Sun Fire X4470 M2 server
‣ 1TB RAM, 40 Cores, 3.6TB HDD
• Software Element
‣ OBIEE 11.1.1.6 with Exalytics Enhancements
‣ Oracle Essbase 11.1.2 with Exalytics Enhancements
‣ Oracle TimesTen 11.2.2.2 for Exalytics
‣ New in v1.1 : Support for Oracle Endeca
Information Discovery, Golden Gate, ODI
‣ Runs on 64-bit Oracle Linux
(Exalogic distribution)
• OBIEE and Essbase are licensed as
Oracle BI Foundation
• Exalytics features can only be used in
conjunction with Exalytics hardware

Rittman Mead and Oracle Exalytics

• Rittman Mead are the first Oracle Partner in the UK to buy an Exalytics box
• Ordered February 2012 through Arrow (reseller), delivered end-April 2012
• Hosted in a secure datacenter in London Docklands, access via VPN
• To be used for customer PoCs, training, testing and possible cloud hosting


Exalytics as the Exa-Machine for BI

• Runs the BI layer on a high-performance, multi-core, 1TB server
• In-memory cache used to accelerate the BI part of the stack
• If Exadata addresses 80% of the query performance,
Exalytics addresses the remaining 20% Oracle BI

‣ Consistent response times for queries
‣ In-memory caching of aggregates
‣ 40 cores for high concurrency In-Memory DB/Cache
‣ Re-engineered BI and OLAP software
that assumes 40 cores and 1TB RAM

ERP/Apps DW


Some Example Exalytics Use-Cases

• Customers who do not have the time/ability/means to put in-place an aggregation
strategy for their dashboards, or who understand how labour-intensive this is
• Customers who have done what they can with summaries, indexes etc, but need
that extra 10% or so of performance, particularly around response-time consistency
• Customers who want to create rich, visual dashboards that would otherwise
stretch the BI Presentation Server
• Customers who need to support high numbers of concurrent users
• Customers running Hyperion Planning and Budgeting who want to reduce the planning
time cycle, re-calculate data faster and generally do more in less time
• Customers who are moving to a single supplier, engineered-systems hardware strategy,
using Exadata for their databases, Exalogic for their Java application hosting, and now
Exalytics for their BI
• Customers looking to consolidate multiple BI systems into a single large instance


Exalytics Sweet-Spots, in our Experience...

• Customers with BI systems that use an underlying data warehouse, with
batch-load refresh cycles that permit pre-aggregation of data
• Customers who have a large number of users running the same reports and
dashboards as each other
• Customer with BI systems that aggregate data via predefined hierarchies
• Customers with reasonably simple and well-tested / validated RPDs
• Customers using Essbase for planning/budgeting applications, and who want to
move the cube into memory
• Customers moving from v10 to v11 OBIEE, with little in the way of visual, interactions
etc, and use Exalytics as the platform for delivering these


Exalytics Under the Covers - How Does it Work?

• Exalytics brings together different technologies, which are still standalone products in
their own right
• To harmonise and optimise their use within Exalytics, it utilises the following techniques:
‣ In-Memory Adaptive Data Mart
‣ In-Memory Intelligent Result Cache
‣ In-Memory Cubes
• Some of these are genuine "secret sauce"
‣ New functionality and algorithms
‣ You can only get them through licensing Exalytics
• Others are descriptions of DW/BI strategies, or existing product functionality, extended
to take advantage of the capacity for processing in memory that Exalytics has


In-Memory Adaptive Data Mart

• Commonly-used aggregates are copied into Oracle TimesTen for Exalytics
• Past query patterns are analyzed and suitable aggregates recommended
• Oracle BI Server then uses these aggregates to make queries run faster
• Aggregates change over time in
response to changes in query patterns TimesTen BI Server
• Tools are provided for managing

Exalytics
and populating these aggregates Aggregates

Data Warehouse
Detail-level
Data


Summary Advisor for Aggregate Recommendation & Creation

• Utility within Oracle BI Administrator tool that recommends aggregates
• Bases recommendations on usage tracking and summary statistics data
• Captured based on past activity
• Runs an iterative algorithm that searches,
each iteration, for the best aggregate


Oracle Exalytics Best Practices, and Experiences from the Field

• Installation, Configuration for Multiple Environments, Management
• Choosing What Goes in the Adaptive Data Mart
• Where Does Exalytics Add Value?
• Measuring Exalytics Performance and Metrics
• Maintaining and Refreshing the Adaptive Data Mart


Configuring Multiple Environments (Consolidation)

• Exalytics main value proposition is around performance
‣ In-memory aggregates speeding up query performance
• However many customers wish to use Exalytics for consolidation
‣ Run multiple environments (test, dev, prod) on a single server
‣ Consolidate multiple applications on a single server
• Desire is usually down to cost (multiple physical servers, per-processor licensing)

Single Exalytics Server Exalytics Svr 1 Exalytics Svr 2 Exalytics Svr 3

MWHOME1 MWHOME2 MWHOME3 MWHOME1 MWHOME1 MWHOME1

DEV DEV DEV DEV TEST PROD

vs.


Best Practice for Consolidation : Oracle Virtual Machine

• Exalytics is now certified for use with Single Exalytics Server
Oracle Virtual Machine, Oracle’s
OVM Guest 1 OVM Guest 2 OVM Guest 3
hard-partitioning virtualization platform
MWHOME1 MWHOME1 MWHOME1
• OVM is installed first, on Exalytics “bare
metal”, with OVM Repository on external DEV DEV DEV

DB server (as with BIPLATFORM, MDS)
• Exalytics VM templates then used to create
required environments (dev, test etc)
• Also allows for licensing in smaller
OVM Hypervisor (Oracle VM Server)
per-processor license blocks (5, 10, 15 etc)
• OVM provides for resource caging, Oracle VM
Repository
isolating environments from each other
Oracle VM
Manager


High Availability using Clustered Exalytics Servers

• Multiple Exalytics servers can be clustered for HA and performance (load balancing)
‣ Requires hardware load balancer above clustered servers
• Linked together via InfiniBand (one for clustering, one for connection to Exadata)
• Note: does not double available RAM
- aggs are replicated in each TT database
‣ Note that Summary Advisor / nqcmd
does not replicate agg tables -
manual replication required
• HA is between TT instances, and OBIEE
managed server + system components


What Should the TimesTen Adaptive Data Mart be Used For?

• Standard approach is to store aggregates in the TimesTen datamart
‣ Aggregated by the source DB, aggregates then cached in TT database
• Other approaches could be used, however
‣ Store whole detail-level dataset in the TT database
‣ Store just recent detail-level data in TT, and use OBIEE’s fragmentation feature
‣ Use Essbase to hold the aggregates instead
‣ Use Essbase to hold detail-level + aggregates
• Where does Exalytics add value?
• What should go in the TimesTen database?


Conclusions and Best Practices Around Detail/Aggregate Storage

• Best use of OBIEE + TT is how the product was intended to be used
‣ Use TimesTen for storing (caching) pre-created aggregates
‣ Aggregates created by source DB (Oracle, SQL Server etc)
‣ Aggregation not performed by TimesTen (as it would be if it held detail-level data)
• IMDB interesting approach, but relies on TT performing aggregation (slow)
• Same goes for holding whole detail-level source in TimesTen
• Also, bear in mind that majority of performance benefit may come from re-hosting
OBIEE application on the Exalytics hardware
‣ Particularly with Exadata sources, source DB may even be fastest host for aggs
‣ Test for your data, do not pre-suppose outcome


Refreshing Aggregates Within the TimesTen Data Mart

• Aggregates are built in TimesTen to support sub-second response times
• Summary Advisor tool suggests aggregate dimensionality and grain, generates script
• BI Server’s Aggregate Persistence executes script:
1. Create TimesTen aggregate table
2. Populate TimesTen aggregate
3. Update RPD online with new aggregate metadata
• Handles supporting dimensions too
Base data Aggregates
OBIEE Aggregate
Persistence
Aggregate tables
created

Aggregate tables TimesTen
loaded from base
data RPD updated with
new aggregate
mappings
RPD


Out-of-the-Box Summary Advisor Limitations

• To refresh an aggregate, Summary Advisor deletes and rebuilds from scratch
• The RPD is edited directly on the BI Server each time an aggregate is created or rebuilt
• Build failures can be difficult to debug
• If a rebuild fails it can leave the RPD in an inconsistent state with TimesTen

This is that the
error the user
sees

After a failed build, the aggregates are nqquery.log suggests possible errors
still in the RPD, but no longer exist in but no clear root cause
TimesTen


Alternatives to using Aggregate Persistence

• However aggregates are refreshed, they must be included
in the RPD
‣ Can be done manually
‣ Aggregate Persistence is useful for this
• Complete refresh of aggregate data
‣ Extract the SQL that OBIEE generates in Aggregate
Persistence, run this through ODI
‣ Write bespoke aggregate refresh code in ODI
• Incremental refresh using GoldenGate and ODI
‣ Instead of rebuilding aggregates in their entirety each
time, only update the part of the aggregate that has
changed on the base data


Incremental refresh of aggregates using GoldenGate and ODI

• Changes to base data are replicated by GoldenGate onto Staging database
• ODI Interfaces apply deltas to the TimesTen aggregates

Aggregates
Base data GoldenGate CDC staging data ODI

TimesTen
(any supported DB) Oracle

TimesTen ODI
GoldenGate GoldenGate JDBC client agent


Custom BI Server ETL Option : Incremental BI Server Refresh

• Uses BI Server to do the refresh, but invokes just part of it - the data refresh part
• Avoids unnecessary aggregate table drop/rebuild, and online RPD edit
• Uses BI Server logical ETL SQL features
to just refresh the TT aggregate table, using
only the latest set of detail-level data
‣ Avoids use of additional ETL tools
‣ No additional license cost
‣ Works against any BI Server source


Custom BI Server ETL Option : Incremental BI Server Refresh

• Uses a number of “undocumented” but support BI Server ETL features
‣ POPULATE command for loading data into an RPD table
‣ WHERE clause for limiting the refresh to just the incremental load
‣ INACTIVE_SCHEMAS to stop BI Server refreshing using existing RPD agg table

SET VARIABLE DISABLE_CACHE_HIT=1, DISABLE_CACHE_SEED=1, DISABLE_SUMMARY_STATS_LOGGING=1,
INACTIVE_SCHEMAS='"TimesTen Aggregates".."EXALYTICS"';
populate "ag_sales_month" mode ( append table connection pool
"TimesTen aggregates"."TT_CP") as
select_business_model "Sales"."Fact Sales"."Sale Amount" as "Sale_Amoun000000AD",
"Sales"."Dim Times"."Month YYYYMM" as "Month_YYYY000000D0"
from "Sales"
where "Dim Times"."Month YYYYMM" = VALUEOF("THIS_MONTH");

• See
http://www.rittmanmead.com/2013/04/incremental-refresh-of-exalytics-
for full details including scripts etc


More Information

• Thank you for attending, & more info can be found at http://www.rittmanmead.com
• Contact us at info@rittmanmead.com or mark.rittman@rittmanmead.com
• Look out for our books, “Oracle Business Intelligence Developers Guide” and
“Oracle Exalytics Revealed”
• Follow-us on Twitter (@rittmanmead) or Facebook (facebook.com/rittmanmead)


UKOUG Engineered Systems Summit 2013 - Oracle Exalytics, Tips and Tricks from the Field

Recommended

Recommended

More Related Content

More from Mark Rittman

More from Mark Rittman (20)

Recently uploaded

Recently uploaded (20)

UKOUG Engineered Systems Summit 2013 - Oracle Exalytics, Tips and Tricks from the Field

Editor's Notes