Extreme BI with Real-time Analytics with Oracle Exadata and Oracle Business Intelligence Apps, OBIEE, OBI

3,308 views

Published on

Implementing a data warehouse is often a daunting and time-consuming task. Many companies spend months or years, frequently engaging expensive consultants and a myriad of products, to implement a solution that is often hard to maintain or scale. This presentation depicts how Southwestern Energy avoided that endless cycle by utilizing powerful integrated solutions from Oracle. In deploying Oracle E-Business Suite and Oracle Business Intelligence Applications all on the Oracle Exadata platform, Southwestern Energy was able to fully leverage the powerful features of Oracle Database 11g to rapidly deploy a near-real-time data warehouse that provided hundreds of rich analytic dashboards/reports on an easy-to-maintain and rapidly scalable platform.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,308
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
111
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Extreme BI with Real-time Analytics with Oracle Exadata and Oracle Business Intelligence Apps, OBIEE, OBI

  1. 1. Extreme BI with Oracle Exadata and Business Intelligence Apps Presented by: Martin Paynter - Enkitec Brad Salva – Southwestern Energy October 4, 2011
  2. 2. Who am I•  Martin Paynter – BI Practice Manager•  EBS Developer since 2001 •  16 EBS implementations•  Business Intelligence Developer since 1999 •  OWB •  ODI •  Informatica •  Hyperion IR •  OBIEE – BI apps •  Discoverer 3
  3. 3. Who is Enkitec•  Oracle-centric Consulting Partner with a broad set of DBA & application development experience•  Database, application development, Oracle training•  Database consultants averaging 15+ years Oracle experience•  Exadata Specialized Partner status (one of a handful globally)•  Dedicated, In-house Exadata Lab (POV, Patch Validation)•  60+ Successful Exadata Implementations•  Enkitec Europe – Partnering with Tanel Poder 4
  4. 4. What is Exadata?—  Revolutionary Approach to Oracle Database Processing—  Database Servers, Storage and Network in a Single Enclosure—  High Bandwidth Network Connects Database and Storage Servers—  Intelligent Storage Filters Data and Minimizes Network Traffic—  Optimized for Both Data Warehouse and OLTP Activity 5
  5. 5. What is Exadata? Exadata Storage ServersExadata Database Servers 11gR2 / ASM iDB / RDS cellsrv 6
  6. 6. How is Exadata Different? The Bottleneck on Many (Most) Large Databases is between the Disk and the DB Server(s)! How to Speed it Up? Make the Pipe Bigger/Faster Reduce Traffic on the Pipe* The fast way to do anything is not to do it! 7
  7. 7. Offloading – The “Secret Sauce” Offloading vs. Smart Scan (what’s the difference) Offloading – generic term meaning doing work at the storage layer instead of at the database layer Smart Scan – query optimizations covered by “cell smart table/index scan” wait events 8
  8. 8. Traditional Architecture vs Exadata Traditional Exadata 2-Node RAC Exadata 2-Node RACSun e20K Quarter 48 cores each Rack 8 cores each Execution Time: Execution Time: 24 Hours + 45 MinutesEnterprise Batch Exadata BatchClass SAN Process Storage Process E x e c u t i o n Ti m e I m p r o v e d b y 3 2 x 9
  9. 9. Business Benefits of Exadata—  Pre-configured - Ready for Production in Weeks, not Months—  Single Source for Support – Hardware / Storage / Network—  Extreme Performance*—  No Need to Modify Oracle-based Applications—  Highly Scalable—  Optimized for Mixed Workload Environments 10
  10. 10. Who is Using Exadata?—  Healthcare Providers—  Market Research Companies—  Retail (Grocer)—  Logistics—  Government – State and Federal—  Mortgage Processing—  Oil and Gas—  Document Processing Outsourcing 11
  11. 11. Who am I•  Brad Salva - Lead DBA (Southwestern Energy)•  Team of 6 DBA’s (managing SQL Server and Oracle instances)•  Database Administration since 1997 •  Oracle and SQL Server •  12 years in the Energy Industry •  Lead DBA at Southwestern Energy since 2006 12
  12. 12. Who is SWN•  Southwestern Energy Company (SWN) is a growing independent energy company primarily engaged in natural gas and crude oil exploration, development and production within North America.•  We are also focused on creating and capturing additional value through our natural gas gathering and marketing businesses, which we refer to as Midstream Services.•  www.swn.com 13
  13. 13. SWN EBS Project•  Started E-business implementation in 2010 to replace legacy systems. Project includes Oracle E-business R12, P2ES Tobin Enterprise Land, P2ES Energy Upstream, P2ES Wellcore, OBIEE, Hyperion, Oracle UCM, Single sign-on, and more.•  16 EBS instances•  7 OBIEE instances•  12 Hyperion instances•  2 Standby instances 14
  14. 14. Why SWN Chose Exadata—  Lower TCO —  Exadata v. “Traditional Hardware” v. Hosted—  Performance—  Easy Installation/Configuration—  High Availability/Redundancy—  Integrated easy to maintain tech stack 15
  15. 15. SWN EBS Production —  Apps Tier (4 Servers – 2 internal, 2 DMZ) —  OS: OEL5 —  Memory: 17GB —  CPU: Intel Xeon X5670  @ 2.93GHz X 4 —  Concurrent Processing Tier ( 2 Servers) —  OS: OEL5 —  Memory: 17GB —  CPU: Intel Xeon X5670  @ 2.93GHz X 4 —  DB Tier (RAC – 4 node) —  OS: OEL5 —  Memory: 96GB per node —  CPU: Intel Xeon X5670 @ 2.93GHz X 24 —  8 GB SGA 16
  16. 16. Client X EBS Production —  Apps Tier/ (1 Server) —  OS: RHEL 5.5 —  Memory: 64GB —  CPU: Intel Xeon X5670  @ 2.93GHz X 12 —  Concurrent Processing Tier (1 server) —  OS: RHEL 5.5 —  Memory: 64GB —  CPU: Intel Xeon X5670  @ 2.93GHz X 12 —  DB Tier (RAC – 2 node) —  OS: RHEL 5.5 —  Memory: 64GB per node —  CPU: Intel Xeon X5670 @ 2.93GHz X 12 —  Version: 11.2.0.2 —  8 GB SGA 17
  17. 17. SWN BI Production —  BI Apps Tier (VM) —  OS: Windows Server 2003 R2 – x64 —  Memory: 16GB —  CPU: Intel Xeon X5670  @ 2.93GHz X 4 —  BI Database Tier (Exadata 4 Node RAC) —  OS: OEL5 —  Memory: 96GB per node —  CPU: Intel Xeon X5670 @ 2.93GHz X 24 —  Version: 11.2.0.2 —  8 GB SGA 18
  18. 18. Client X BI Production —  BI Apps Tier (VM) —  OS: Windows Server 2008 – x64 —  Memory: 16GB —  CPU: Intel Xeon X5670  @ 2.93GHz X 4 —  BI Database Tier (2 Node RAC) —  OS: RHEL 5.5 —  Memory: 64GB per node —  CPU: Intel Xeon X5670 @ 2.93GHz X 12 —  Version: 11.2.0.2 —  8 GB SGA 19
  19. 19. Typical BI Considerations—  Source System —  Availability —  Performance—  ETL —  Architecture installation/configuration —  Development —  Performance—  Visualization —  Architecture installation/configuration —  Dashboard/Report development 20
  20. 20. SWN/Client X BI Platform—  Source System —  Oracle EBS R12.1.2 —  11.2.0.2 4 node RAC Database (Client X on 2 Node)—  ETL —  Informatica 9.0.1 Hotfix 2 —  Oracle BI Applications 7.9.6.3 —  DAC 10.1.3.4 —  11.2.0.2 4 node RAC Database (Client X on 2 Node)—  Visualization —  OBIEE 11.1.1.5 —  Weblogic 10.3.5.0 21
  21. 21. Areas of Focus—  Implementation —  Exadata configuration —  Installation of Oracle RAC databases —  Create EBS clones and backups —  Install/configure BI Technology stack—  Performance —  EBS on Exadata —  Informatica/DAC on Exadata —  OBIEE/OBI Apps on Exadata 22
  22. 22. SWN EBS Implementation Metrics —  Configure Exadata and 4 node RAC DB (40 hours) —  Number of DB instances on Exadata ½ rack – 16 —  Production EBS 4 node RAC —  All other installs single node —  Time to Backup – 1 minute 30 seconds —  RMAN backup, high compression, 16 channels to ASM +RECO disk group —  Netapp backup takes 30 minutes —  Restore from backup takes 10 minutes —  Time to Clone – 10 minutes 23
  23. 23. SWN BI Implementation Metrics —  OBIEE installed/configured in 8 hours —  Weblogic —  Enterprise Manager —  BI Server —  Presentation Server —  Admin Tools —  Informatica and DAC installed/configured in 8 hours —  Powercenter Services and Client —  DAC Server and Client —  OBI Apps: —  Configured General Ledger subject area and ran initial full load in 24 hours 24
  24. 24. Extreme Performance Data WarehousingIntegrated Technology Stack BI Applications •  Single source of truth BI Tools •  Easy to deploy and manage ELT Tools •  Extreme performance Data Models •  Meets all end user requirements Database •  Lower cost of ownership Smart Storage
  25. 25. Oracle BI Apps Product Components Example: Financial Analytics1 3 Pre-built warehouse with 16 star-schemas Pre-mapped metadata, including designed for analysis and reporting on embedded best practice calculations and financial analytics metrics for financial, executives and other business users •  Presentation layer •  Logical business model •  Physical sources2 4 Pre-built ETL to extract data from over A “best practice” library of over 360 3,000 operational tables and load it into pre-built metrics, 30 intelligent the DW, sourced from PSFT, Oracle EBS dashboards, 200+ reports plus alerts for and other sources CFO, Finance Controller, Financial Analyst, AR/AP Managers and Executives
  26. 26. Performance —  EBS on Exadata —  GL Journal Load —  Informatica/DAC on Exadata —  Extract from EBS —  General Ledger subject area load into OBI DW —  OBIEE/OBI Apps on Exadata —  General Ledger Dashboard/report processing against OBI DW 27
  27. 27. Tuning Options —  OBIEE —  Aggregate navigation —  Presentation cache —  BI Server cache —  Informatica/DAC on Exadata —  Commit frequency —  DTM Buffer size —  Indexes —  Integration Services —  OBI DW —  Partitioning —  Parallelism —  Compression —  OLAP —  Materialized views 28
  28. 28. EBS Journal Load SWN and Client X use the gl_interface table to load in ~3,000,000 journal lines per dayTask # Rows SWN Time Client X GainLoad to gl_interface from external ~3,000,000 3 min 40 min 13XtableImport Journals ~3,000,000 15 min 7 hrs 28XPost Journals 10 min 2 hrs 12XTotal Time Spent 28 min ~10 hrs 21X 29
  29. 29. OBI General Ledger Subject Area—  268 Tasks—  Over 600 bitmap indexes—  Two main fact tables: —  W_GL_OTHER_F – GL Transactions —  W_GL_BALANCE_F – GL Balances 30
  30. 30. OBI DW Load—  Full load for General Ledger subject area with creation of indexes ~ 40,000,000 in 4 hours—  Incremental loads with indexes ~ 3,000,000 in 2 hours—  Low effort tuning: —  Identify long running tasks and change DTM Buffer and commit interval in PowerCenter Workflow Manager —  Increase number of Integration Services —  Confirm Bulk loading is occurring (/*+ SYS_DL_CURSOR*/) —  Drop bitmap indexes in DAC 31
  31. 31. Informatica Performance—  DTM Buffer Size —  Changed default of 32MB to Auto—  Target Commit Frequency —  For high volume loads changed commit frequency from 10,000 to 1,000,000—  Make sure direct loading is occurring: —  /*+ SYS_DL_CURSOR*/ in v$sql 32
  32. 32. ETL – Extraction from EBS EBS on Exadata provides an edge during the extraction process. —  SDE_ORA_GLJournals (2,995,320) —  SWN – 1 minute (All time spent on writing) —  Client X – 10 minutes (50/50 read and write) SQL_ID EXECS AVG_ETIME OFFLOADABLE OFFLOADED OFFLOAD_ELIGIBLE ------------------- ---------- ----------------- ----------------- -------------------- ----------------------------- fjh00b5mzq64c 1 11.61 Yes 5,917,486,584 10,090,102,784 33
  33. 33. DAC Performance—  Increase number of Integration Services —  Evaluated the DAC execution plan and determined an increase from 10 to 30 was optimal 34
  34. 34. DAC Performance—  Drop bitmap index creation —  Saves 46 minutes for SWN, but allows for partitioning without additional configuration as well as facilitates smart scan query processing Before Dropping indexes: After Dropping indexes: 35
  35. 35. DAC Performance —  Client X spends 82 minutes creating indexes during incremental load. It is not an option for them to drop indexes because queries would never return. 36
  36. 36. OBI DW Load Times - Total Time - Hours SWN Tuned SWN OOB Time Client X 0 2 4 6 8 10 12 37
  37. 37. OBI DW Load Times – Top Tasks TIME (minutes)Task Client X SWN OOB SWN Tuned GainSDE_ORA_GLJournals 18 3.5 1 18XSDE_ORA_GLBalanceFact 15 2 2 8XSDE_ORA_Stage_GLJournals_Derive 20 2 1 20XSDE_ORA_STAGE_GLOtherFact_Derive 28 8 1.5 19XSIL_GLOtherFact 35 8 6.5 5XSIL_GLBalanceFact 38 8 6.5 6XPLP_GLBalanceAggrByAcctSegCodes 62 21 11 6X 38
  38. 38. OBI DW Load – Incremental Tuned —  Top Task Metrics – Incremental load Tuned (32 minutes)Task # Rows Time (min) Throughput/minSDE_ORA_GLJournals 2,995,320 1 2,995,320SDE_ORA_GLBalanceFact 3,174,070 2 1,587,035SDE_ORA_Stage_GLJournals_Derive 2,995,320 1 2,995,320SDE_ORA_STAGE_GLOtherFact_Derive 2,995,320 1.5 1,330,213SIL_GLOtherFact 990,404 6.5 152,369SIL_GLBalanceFact 3,174,070 6.5 488,319PLP_GLBalanceAggrByAcctSegCodes 18,666,814 11 1,116,676 39
  39. 39. OBI DW Query Performance How will my queries perform with… •  No Bitmaps •  Parallelism •  Partitioning •  Compression 40
  40. 40. Look ma, no bit maps—  Every query is different, however there is no negative impact to performance overall (Client X ~ 10 min)SQL_ID EXECS AVG_ETIME OFFLOADABLE OFFLOADED OFFLOAD_ELIGIBLE IO Saved------------------- ---------- ----------------- ----------------- -------------------- ----------------------------- ------------5ycwp44p7crmg 1 13.61 Yes 1,173,569,000 27,537,399,808 95.74% 41
  41. 41. But is it faster… —  Query execution with bit maps in place:SQL_ID EXECS AVG_ETIME OFFLOADABLE OFFLOADED OFFLOAD_ELIGIBLE IO Saved------------------- ---------- ----------------- ----------------- -------------------- ----------------------------- ------------5m8q3bkuyrguk 1 439.25 NO 0 0 0% 42
  42. 42. But is it faster…—  Query execution with no bitmaps:SQL_ID EXECS AVG_ETIME OFFLOADABLE OFFLOADED OFFLOAD_ELIGIBLE IO Saved------------------- ---------- ----------------- ----------------- -------------------- ----------------------------- ------------Drdwhzcsm4cyf 1 65.98 Yes 1,264,079,768 28,597,270,528 95.58% 43
  43. 43. Parallelism Query dropped from 3 min to 8 seconds (Client X – 20 min) 44
  44. 44. Parallelism—  Where can we add parallelism: —  OBIEE Analysis report —  OBIEE RPD physical table (previous slide) —  Informatica source definition —  OBI DW tables 45
  45. 45. Partitioning—  Our desire is to get direct path reads in order to take advantage of Smartscan functionality —  Risk size of partition being too small—  We now have the option of partitioning without having to spend time adding in DAC Actions—  Partitioned W_GL_OTHER_F and W_GL_BALANCE_F by period —  Load time stayed at 32 minutes—  Mixed query performance results depending on size of partitions 46
  46. 46. Compression—  Now that we have the tables partitioned, we can compress historical data that is no longer being actively updated: —  W_GL_OTHER_FACT SEP11 partition HCC Query High: —  Before HCC ~ 23 GB, query execution 11.61 seconds —  After HCC ~ 699 MB, query execution 8.24 seconds Compression of 33x 47
  47. 47. Exadata Capabilities – Wrap-up•  Speed to Market•  Easy Transition from Legacy Systems to Exadata•  Simple and Accurate Application Migration•  Consolidation Platform•  Extreme Performance 48
  48. 48. The Kübler-Ross grief cycle Exposure to Exadata 49
  49. 49. Expert Oracle Exadata, co-authored by KerryOsborne, Randy Johnson & Tanel Poder tobe published July 18, 2011 Visit Enkitec at Oracle OpenWorld at Moscone Center in San Francisco from October 2 – 6 at Booth 1721 50
  50. 50. Questions?Contact Information : Martin Paynter martin.paynter@enkitec.com kerryosborne.oracle-guy.com www.enkitec.com Fastest Growing Companies in Dallas 51

×