Your SlideShare is downloading. ×
Real-Time Data Warehousing using Oracle Database and OBIEE - Collaborate'11
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Real-Time Data Warehousing using Oracle Database and OBIEE - Collaborate'11

2,773

Published on

Presentation by Stewart Bryson, Rittman Mead, on real-time data warehousing techniques using the Oracle Database and Oracle Business Intelligence Enterprise Edition. As delivered at Collaborate'11, …

Presentation by Stewart Bryson, Rittman Mead, on real-time data warehousing techniques using the Oracle Database and Oracle Business Intelligence Enterprise Edition. As delivered at Collaborate'11, Orlando, April 2011.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,773
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
269
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Real-Time DW with OBIEE and Oracle Database Stewart Bryson, Managing Director, Rittman Mead America Collaborate, Orlando, April 2011 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 1
  • 2. Who am I? T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 2
  • 3. • Stewart Bryson, Managing Director, Rittman Mead America • Oracle BI/DW Architect and Delivery Specialist • Oracle Database and BI Stack since 1996 • Regular Community Speaker • Writer for Rittman Mead Blog: http://www.rittmanmead.com/blog • Developer of Transcend Framework • Email : stewart.bryson@rittmanmead.com • Twitter : @stewartbryson T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 3
  • 4. Who is Rittman Mead? T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 4
  • 5. • Oracle BI&DW Experts • Consulting, Training and Support • Global Company, Offices in the US, UK and India • Clients in the UK, USA, Europe, Middle-East • Voted UKOUG BI Partner of the Year 2008 • 16 Sessions at Oracle Open World 2010 • Twitter : @rittmanmead T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 5
  • 6. Rittman Mead BI Forum, May 2011 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 6
  • 7. Registrations open for the Rittman Mead BI Forum, May 2011 • Brighton, UK, May 18-20th 2011 • Atlanta, GA, May 25-27th 2011 • Covers OBIEE, DW, EPM • Guest speakers including: ‣ Jeff McQuigg ‣ Kevin McGinley ‣ Jean-Pierre Dijcks ‣ Mike Durran ‣ Shyam Varan Nath ‣ Philippe Lions ‣ Emiel van Bockel • OBIEE 11g Masterclass with Mark Rittman and Tony Heljula • Full details at http://www.rittmanmead.com/biforum2011 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 7
  • 8. What will this session answer? T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 8
  • 9. • What do I mean by “real-time data warehousing?” ‣ Zero latency, or just reduced latency? ‣ Do I really mean “right-time data warehousing?” • What can I use to reduce latency? ‣ What does OBIEE offer? ‣ What does the Oracle Database (and OWB) offer? • Can I blend these different offerings? ‣ Of course you can! ‣ It’s the name of the presentation T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 9
  • 10. What do I mean by “real-time”? T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 10
  • 11. • Traditional data warehouses ‣ Daily to weekly batch-load cycles ‣ Key-metrics are always delayed ‣ Information may not be “actionable” • Real-time data warehouses ‣ Up to the minute or hour reporting ‣ Key-metrics are moving ‣ Information is actionable T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 11
  • 12. An interval will always exist between the occurrence of a measurable event and our ability to process that event as a “reportable fact” T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 12
  • 13. L1&-6)&/(&)7#.@@)&)2-#-"0)*#(@#.2$(/.23#%-%#$%2#6%?)#.@@)&.23#+%-)2$.)*=#L(&#)I%/0+)7#%#2)B#-&%2*% +%-)2$"#-6%2#%#$6%23)#-(#%#&)3.(2%+#3&(10.23#(@#*-(&)*#-6%-#/%"#2))#-(#>)#0&(0%3%-)#-6&(136#/%2 %33&)3%-.(2#-%>+)*#4*))#L.31&)#M<= K6)#.?.*.(2#>)-B))2#%-%#%$E1.*.-.(2#%2#%-%#0&)*)2-%-.(2#0&)*)2-*#%#D)"#E1)*-.(2#-6%-#%2"#$(/0% $(2-)/0+%-.23#/(?.23#-(#%#&)%+,-./)#95#*"*-)/N#%&)#"(1#O1*-#.2-)&)*-)#.2#0&(?..237#%*#&%0.+"#%*#0(* An interval will always exist between the occurrence of a measurable (2#%+&)%"#%$E1.&)#%-%7#(&#(#"(1#%+*(#2))#-(#&)0(&-#(2#2)B+"#%)#@%$-1%+#.2@(&/%-.(2C event and our ability to process that event as a “reportable fact” L(&#)I%/0+)7#%#B(&D)&#.2#%#/(>.+) FIGURE 1: BI latency $)2-&)#/%"#+.D)#-(#*))#.2@(&/%-.(2# Event Report $1*-(/)&#.*P#%2#-6)#+.D)+.6((#-6% $1*-(/)&P*#)/(3&%06.$#B(1+#*B *100+.)&= K6.*#.2@(&/%-.(2#2))*#-(#>)#0&)*)2 %&)#*0)%D.23#-(#-6)/<#%2#0&)@)&%> +.2),(@,>1*.2)**#%00+.$%-.(2#-6%-#/% .2-)&%$-.(2=#R(B)?)&#.-#.*#12+.D)+"#-6 Recording Propagation Aggregation Query >"#-6)#$%++#$)2-&)#%3)2-#@(&#$1*-(/) 0&).$-)#$61&2#B(1+#>)#.2@+1)2$) 6(1&*= S(2?)&*)+"7#%#*"*-)/#-6%-#.*#1*.23# Real-time query latency -(#%+)&-#-(#0(-)2-.%+#@&%1#4*-($D#/%& $(/0%2.)*#%&)#0(**.>+)#)I%/0+)#1* D2(B#%>(1-#10,-(,-6),/(/)2-#%$-.? ***+,-./0.12345,416,+537 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 12
  • 14. Business Case T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 13
  • 15. • We want to be able to segment our reporting between “real-time” needs and static reports ‣ Most users will want real-time numbers delivered with dashboards ‣ Some users (financial-minded) don’t want numbers to change each time they run a report • Use the standard Oracle BI Stack ‣ Oracle Database Enterprise Edition ‣ Oracle Business Intelligence Enterprise Edition T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 14
  • 16. Latency versus Query Performance T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 15
  • 17. Latency versus Query Performance T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 15
  • 18. Latency versus Query Performance Latency Query Performance T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 15
  • 19. Latency versus Query Performance Latency Query Performance T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 15
  • 20. Oracle Next-Generation Reference Architecture T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 16
  • 21. Requirement: Identifying and Capturing Change T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 17
  • 22. Requirement: Change Data Capture T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 18
  • 23. Requirement: Change Data Capture • Oracle CDC • OWB Trickle-feed mappings • Trigger-based capture • Recording modification dates • MINUS Command T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 18
  • 24. Requirement: Change Data Capture • Oracle CDC • Oracle CDC • OWB Trickle-feed • OWB Trickle-feed mappings mappings • Trigger-based • Message Queues capture • Views over • Recording database links modification dates • File movements • MINUS Command T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 18
  • 25. Requirement: Change Data Capture • Oracle CDC • OWB Mappings • Oracle CDC and Process Flows • OWB Trickle-feed • OWB Trickle-feed mappings • Other ETL Tools mappings • Message Queues • Custom ETL code • Trigger-based capture • Views over • Custom transaction database links code • Recording modification dates • File movements • MINUS Command T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 18
  • 26. Requirement: Change Data Capture • Oracle CDC • OWB Mappings • Oracle CDC and Process Flows • OWB Trickle-feed • OWB Trickle-feed mappings • Other ETL Tools mappings • Message Queues • Custom ETL code • Trigger-based capture • Views over • Custom transaction database links code • Recording modification dates • File movements • MINUS Command T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 18
  • 27. Oracle Asynchronous Change Data Capture T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 19
  • 28. Oracle Asynchronous Change Data Capture Hot Log Online redo logs mined locally Change tables are updated and stored locally Hot Log Distributed Online redo logs are mined locally Change tables are updated and stored remotely Autolog Online Online redo logs are mined remotely Change tables are updated and stored remotely Autolog Archive Archived redo logs are mined remotely Change tables are updated and stored remotely T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 19
  • 29. Oracle Asynchronous Change Data Capture Hot Log Online redo logs mined locally Change tables are updated and stored locally Hot Log Distributed Online redo logs are mined locally Latency Change tables are updated and stored remotely Autolog Online Online redo logs are mined remotely Change tables are updated and stored remotely Autolog Archive Archived redo logs are mined remotely Change tables are updated and stored remotely T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 19
  • 30. Requirement: Change Data Capture • Oracle CDC • OWB Mappings • Oracle CDC and Process Flows • OWB Trickle-feed • OWB Trickle-feed mappings • Other ETL Tools mappings • Message Queues • Custom ETL code • Trigger-based capture • Views over • Custom transaction database links code • Recording modification dates • File movements • MINUS Command T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 20
  • 31. Requirement: Change Data Capture • Oracle CDC • OWB Mappings • Oracle CDC and Process Flows • OWB Trickle-feed • OWB Trickle-feed mappings • Other ETL Tools mappings • Message Queues • Custom ETL code • Trigger-based capture • Views over • Custom transaction database links code • Recording modification dates • File movements • MINUS Command T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 20
  • 32. Requirement: Change Data Capture T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 21
  • 33. Requirement: Change Data Capture • Trickle-feed mappings allow the development of Streams Capture and Apply processes as continuously running mappings. T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 21
  • 34. Federated OLTP/EDW Reporting T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 22
  • 35. • Layering in OLTP data to provide the real-time component • General approach is to use a BI tool such as OBIEE • No real database or modeling involvement T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 23
  • 36. Classic Batch Load Star Schema: Process Flow T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 24
  • 37. Classic Batch Load Star Schema: Fact Table Load T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 25
  • 38. Federation with OBIEE T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 26
  • 39. OBIEE Metadata and Data Layers T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 27
  • 40. OBIEE Metadata and Data Layers • Physical Data Sources, such as relational databases, Relational, OLAP, File and Application Data Sources OLAP Cubes and application source adapters T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 27
  • 41. OBIEE Metadata and Data Layers • A Semantic Model made up of Metrics, dimensions, Enterprise Semantic Model hierarchies and calculations, derived from... • Physical Data Sources, such as relational databases, Relational, OLAP, File and Application Data Sources OLAP Cubes and application source adapters T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 27
  • 42. OBIEE Metadata and Data Layers • The Web Catalog contains definitions of reports, alerts Web Catalog dashboards, KPIs, based on... • A Semantic Model made up of Metrics, dimensions, Enterprise Semantic Model hierarchies and calculations, derived from... • Physical Data Sources, such as relational databases, Relational, OLAP, File and Application Data Sources OLAP Cubes and application source adapters T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 27
  • 43. Flow of Data Through the Three-Layer Semantic Model T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 28
  • 44. Flow of Data Through the Three-Layer Semantic Model Simplification of the Data Model Integration of Disparate DataSources Addition of Business Logic and Calculations T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 28
  • 45. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 29
  • 46. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 29
  • 47. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 29
  • 48. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 29
  • 49. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 30
  • 50. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 30
  • 51. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 30
  • 52. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 31
  • 53. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 31
  • 54. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 31
  • 55. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 31
  • 56. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 32
  • 57. Logical Table Sources Map to Physical Models T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 32
  • 58. Logical Table Sources Respect the Physical Model T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 33
  • 59. Logical Table Sources Respect the Physical Model T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 33
  • 60. Logical Table Sources Respect the Physical Model T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 33
  • 61. WITH SAWITH0 AS ((select T43901.CUST_FIRST_NAME as c2, T43901.CUST_LAST_NAME as c3, T43971.SAL_AMT as c4 from GCBC_CRM.CUSTOMERS T43901, GCBC_POS.POS_TRANS T43971, GCBC_POS.POS_TRANS_HEADER T43978 where ( T43901.CUST_ID = T43978.CUST_ID and T43971.TRANS_ID = T43978.TRANS_ID and TO_DATE (2010-09-18 00:00:00 , YYYY-MM-DD HH24:MI:SS) < T43978.TRANS_DATE ) union all select T44042.CUSTOMER_FIRST_NAME as c2, T44042.CUSTOMER_LAST_NAME as c3, T44105.AMOUNT as c4 from GCBC_EDW.CUSTOMER_DIM T44042, GCBC_EDW.SALES_FACT T44105 where ( T44042.CUSTOMER_KEY = T44105.CUSTOMER_KEY ) )), SAWITH1 AS (select sum(D3.c4) as c1, D3.c2 as c2, D3.c3 as c3 from SAWITH0 D3 group by D3.c2, D3.c3) select distinct 0 as c1, D2.c2 as c2, D2.c3 as c3, D2.c1 as c4 from SAWITH1 D2 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 34
  • 62. WITH SAWITH0 AS ((select T43901.CUST_FIRST_NAME as c2, T43901.CUST_LAST_NAME as c3, T43971.SAL_AMT as c4 from GCBC_CRM.CUSTOMERS T43901, GCBC_POS.POS_TRANS T43971, GCBC_POS.POS_TRANS_HEADER T43978 where ( T43901.CUST_ID = T43978.CUST_ID and T43971.TRANS_ID = T43978.TRANS_ID and TO_DATE (2010-09-18 00:00:00 , YYYY-MM-DD HH24:MI:SS) < T43978.TRANS_DATE ) union all select T44042.CUSTOMER_FIRST_NAME as c2, T44042.CUSTOMER_LAST_NAME as c3, T44105.AMOUNT as c4 from GCBC_EDW.CUSTOMER_DIM T44042, GCBC_EDW.SALES_FACT T44105 where ( T44042.CUSTOMER_KEY = T44105.CUSTOMER_KEY ) )), SAWITH1 AS (select sum(D3.c4) as c1, D3.c2 as c2, D3.c3 as c3 from SAWITH0 D3 group by D3.c2, D3.c3) select distinct 0 as c1, D2.c2 as c2, D2.c3 as c3, D2.c1 as c4 from SAWITH1 D2 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 34
  • 63. WITH SAWITH0 AS ((select T43901.CUST_FIRST_NAME as c2, T43901.CUST_LAST_NAME as c3, T43971.SAL_AMT as c4 from GCBC_CRM.CUSTOMERS T43901, GCBC_POS.POS_TRANS T43971, GCBC_POS.POS_TRANS_HEADER T43978 where ( T43901.CUST_ID = T43978.CUST_ID and T43971.TRANS_ID = T43978.TRANS_ID and TO_DATE (2010-09-18 00:00:00 , YYYY-MM-DD HH24:MI:SS) < T43978.TRANS_DATE ) union all select T44042.CUSTOMER_FIRST_NAME as c2, T44042.CUSTOMER_LAST_NAME as c3, T44105.AMOUNT as c4 from GCBC_EDW.CUSTOMER_DIM T44042, GCBC_EDW.SALES_FACT T44105 where ( T44042.CUSTOMER_KEY = T44105.CUSTOMER_KEY ) )), SAWITH1 AS (select sum(D3.c4) as c1, D3.c2 as c2, D3.c3 as c3 from SAWITH0 D3 group by D3.c2, D3.c3) select distinct 0 as c1, D2.c2 as c2, D2.c3 as c3, D2.c1 as c4 from SAWITH1 D2 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 34
  • 64. WITH SAWITH0 AS (select sum(T44105.AMOUNT) as c1, T44042.CUSTOMER_FIRST_NAME as c2, T44042.CUSTOMER_LAST_NAME as c3, T45416.CALENDAR_YEAR as c4 from GCBC_EDW.DATE_DIM T45416, GCBC_EDW.CUSTOMER_DIM T44042, GCBC_EDW.SALES_FACT T44105 where ( T44042.CUSTOMER_KEY = T44105.CUSTOMER_KEY and T44105.SALES_DATE_KEY = T45416.DATE_KEY and T45416.SQL_DATE = TO_DATE(2003-01-09 00:00:00 , YYYY-MM-DD HH24:MI:SS) ) group by T44042.CUSTOMER_FIRST_NAME, T44042.CUSTOMER_LAST_NAME, T45416.CALENDAR_YEAR) select distinct 0 as c1, D1.c2 as c2, D1.c3 as c3, D1.c4 as c4, D1.c1 as c5 from SAWITH0 D1 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 35
  • 65. WITH SAWITH0 AS (select sum(T44105.AMOUNT) as c1, T44042.CUSTOMER_FIRST_NAME as c2, T44042.CUSTOMER_LAST_NAME as c3, T45416.CALENDAR_YEAR as c4 from GCBC_EDW.DATE_DIM T45416, GCBC_EDW.CUSTOMER_DIM T44042, GCBC_EDW.SALES_FACT T44105 where ( T44042.CUSTOMER_KEY = T44105.CUSTOMER_KEY and T44105.SALES_DATE_KEY = T45416.DATE_KEY and T45416.SQL_DATE = TO_DATE(2003-01-09 00:00:00 , YYYY-MM-DD HH24:MI:SS) ) group by T44042.CUSTOMER_FIRST_NAME, T44042.CUSTOMER_LAST_NAME, T45416.CALENDAR_YEAR) select distinct 0 as c1, D1.c2 as c2, D1.c3 as c3, D1.c4 as c4, D1.c1 as c5 from SAWITH0 D1 • The BI Server understands from the Fragmentation content how to “optimize” the query • Only one LTS is needed to provide the answer T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 35
  • 66. Demonstration Fragmentation with OLTP and EDW T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 36
  • 67. • Strengths ‣ Low latency ‣ No complex modeling or ETL ‣ Incorporated into any standard dimensional EDW • Weaknesses ‣ Performance ‣ Difficult to conform multiple systems T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 37
  • 68. EDW with a Real-Time Component T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 38
  • 69. • Modeling complimentary real-time structures in an EDW • Use functionality of the Oracle Database and OWB • Complex ETL and Database interaction T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 39
  • 70. The Real-Time Partition Fact Table T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 40
  • 71. The Real-Time Partition Fact Real-Time Fact Table Partition T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 40
  • 72. The Real-Time Partition Fact Real-Time Fact Table Partition T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 40
  • 73. The Real-Time Partition • All ETL processing of fact records are directed at the Real-Time Partition using Micro-Batch • No indexes, small data set • Partition Exchange at the end of the day T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 41
  • 74. The Real-Time Partition • All ETL processing of fact records are directed at the Real-Time Partition using Micro-Batch • No indexes, small data set • Partition Exchange at the end of the day T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 41
  • 75. The Real-Time Partition • All ETL processing of fact records are directed at the Real-Time Partition using Micro-Batch • No indexes, small data set • Partition Exchange at the end of the day T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 41
  • 76. The Real-Time Partition • All ETL processing of fact records are directed at the Real-Time Partition using Micro-Batch • No indexes, small data set • Partition Exchange at the end of the day T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 41
  • 77. Early-Arriving Facts T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 42
  • 78. What if Type 2 dimension changes haven’t arrived yet? • Grab the most recent dimension record when that entity already exists • Use a default “Not Known Yet” record when they don’t • Prior to the partition exchange, we run our late- arriving dimension loads • We update the Real-Time Partition with the correct surrogate keys ‣ OWB generates an efficient MERGE ‣ Small data sets in the Real-Time Partition T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 43
  • 79. Real-Time Integration: Fact Table RT Load T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 44
  • 80. Real-Time Integration: Fact Table RT Load T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 44
  • 81. Real-Time Integration: Correct Dimensions T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 45
  • 82. Real-Time Integration: “Closing the Books” on the Day T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 46
  • 83. Plugging it together with OBIEE (Non-Fragmentation) • Using standard database functionality (UNION ALL) • Could be done with a view • Using Surrogate Keys in the RT Partition • OBIEE doesn’t have to manage different LTS’s T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 47
  • 84. Plugging it together with OBIEE (Non-Fragmentation) • Using standard database functionality (UNION ALL) • Could be done with a view • Using Surrogate Keys in the RT Partition • OBIEE doesn’t have to manage different LTS’s T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 47
  • 85. Plugging it together with OBIEE (Non-Fragmentation) • Using standard database functionality (UNION ALL) • Could be done with a view • Using Surrogate Keys in the RT Partition • OBIEE doesn’t have to manage different LTS’s T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 47
  • 86. Demonstration EDW with a Real-Time Component T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 48
  • 87. Summary T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 49
  • 88. • Latency versus performance and scalability • Latency and performance versus degree of difficulty • Different variations on the two test cases T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 50
  • 89. Real-Time DW with OBIEE and Oracle Database Stewart Bryson, Managing Director, Rittman Mead America Collaborate, Orlando, April 2011 T : (888) 631 1410 or +44 (0) 8446 697 995 E : info@rittmanmead.com W: www.rittmanmead.comThursday, 21 April 2011 51

×