Cox Business Intelligence & Oracle BI/DW Stack


Published on

Cox Communications Gartner BI Summit 2012 presentation.

Published in: Technology, Business
1 Comment
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Cox Business Intelligence & Oracle BI/DW Stack

  1. 1. Cox Business Intelligence& Oracle BI/DW StackJohn Landis, Sr. Director BI & Data ArchitectureApril, 2012
  2. 2. Cox Communications Company Overview• Is the third-largest cable entertainment and broadband services provider in the country• Has over 6 million customers• Has over 22,000 employees 2
  3. 3. Cox Communications Services• Residential TV, Internet, Phone, Tech Solutions, Home Security• Business TV, Internet, Phone, Security, Backups, Industry Services for Real Estate, Residential Communities, Education, Government, Healthcare, and Hospitality• Media On Air, Online and On- the-Go 3
  4. 4. CEBI 2008 Problem• Business customers not • Proliferation of tools has satisfied with multiple become expensive and hard platforms. Not sure where to maintain to get data the right data. • Data needs of the company• Business intelligence are growing, offline platform has multiple databases at all sites versions of the truth • Development taking place in• Data Integration is fractured multiple organizations• Data Warehouse has not • No standards exist in the had investment in 3 years enterprise 4
  5. 5. Cox Enterprise Business Intelligence (CEBI) 2008 As well as…… 5
  6. 6. CEBI 2008 Transition• Adopt a enterprise reporting application to encourage Strategic Dynamic collaborative enterprise development of reporting Modeling across the organization and lower the cost of reporting throughout Cox. Past Oriented Advanced Future Oriented Analytics• Reuse and optimization of Ad-Hoc Query & Reporting resources: Standardized • People and Processes Reporting • Application, Data, and Operational Static Services Death by 1000 paper cuts • Time and Cost 6
  7. 7. CEBI 2008 Solution• Oracle Business Intelligence Enterprise Edition 10G chosen as the enterprise BI platform• Oracle Database chosen as the Enterprise data platform and Infomatica chosen as integration platform• Business Intelligence Competency Center Deployed• Data Warehouse Clean-Up Begins 7
  8. 8. CEBI 2008 OBIEE 10G• Total Cost of Ownership• Common Semantic Layer• Prebuilt Analytical Options• Oracle’s Strategic BI Roadmap• Single Sign On• Embedded Metadata• Self Service Reporting 8
  9. 9. CEBI 2008 OBIEE 10G• Interactive Charts and Graphs• Personal Dashboards• One Suite of Tools• Open Source not ready for enterprise deployment• Hyperion Integration• Personalization 9
  10. 10. CEBI 2008 Solution Cont.Cox lived happily ever after and I got to retire to my dream location…..Not Exactly 10
  11. 11. CEBI 2008 Lesson’s Learned• Data governance is required • Garbage in, garbage out• IT can only facilitate data • Most people don’t understand governance, business needs data, therefore carefully create to lead your RPD• Training is critical • OBIEE resources are hard to find• Self Service Reporting requires supervision • The business wants you to challenge them on requirements• Start small • Reports are only as fast as the• Not everyone likes change database 11
  12. 12. CEBI 2010 Exadata• With the growing data and reporting needs within the organization, the platform needed to expand to handle the projected growth.• Business data needs went from daily updates to near real time updates.• Existing hardware reached it’s capacity and new technology was needed in order to meet the current and upcoming demands.• Without a platform and technology upgrade, data and reporting would not be made available to the organization. 12
  13. 13. CEBI 2010 Exadata• In April 2010, the EBI team partnered with Oracle EXADATA POC -OBIEE Queries Total Run Time (Total of 130 queries executed) BASELINE to perform a Proof of Concept (POC). 25,000• Based on the results of the POC, an executive 20,000 EXADATA AS IS decision was made to implement the full solution. 15,000 EXADATA NO INDEXES• In July 2010 the EBI team began the planning and 10,000 rollout of Exadata. EXADATA 5,000 COMPRESSIO N/ AGG• With the help of the Operations Support group, all - REMOVAL EXADATA EXADATA EBI databases were implemented on Exadata in BASELINE EXADATA AS EXADATA NO COMPRESSI EXADATA MIXED LOAD MIXED LOAD IS INDEXES ON/ AGG TESTING Production. REMOVAL TESTING Total Time in Seconds 22,077 9,774 3,998 2,316 2,399 13
  14. 14. CEBI 2010 Exadata Pre-Launch Concerns• People • Technology - Support structure is different - Oracle was new to the hardware market - Adoption - Technology had limited instances in - Learning curve for support and production. development - Switching from commodity based storage to appliance; risk of stranding assets.• Process - Backup strategy changes and recovery - Compression can mask the lack of a model changes data lifecycle management - Vendor “lock in” moving away from Oracle - It is not the way we have always becomes more expensive. done it - Performance increases can mask architectural issues Note: Degradations in performance caused by development code that should have been avoided. Nearly 600K IOPS. 14
  15. 15. CEBI 2010 Exadata• Reporting • Nearly 6X improvement out of the box • Up to 200X query performance improvement. 9X on average • Nearly 6X performance increase on the work orders load (non Exadata source). 2X on average for non Exadata sources and 10X on average for Exadata to Exadata loads. • Some reports showed worse performance 15
  16. 16. CEBI 2010 Exadata Results • 5-10x Compression saves Cox money over traditional storage Compression • Lowers backup time and tapes needed • Estimated savings in space through 2012 range approx. $2.4M – $4.8M • Less tuning reduces project timelines • Enables near real time processing Performance • Able to process data previously not possible • Estimated savings in time in 2012 approx. 5% or $400K • Highly available, has uncovered issues in other Cox Oracle systems helping to improve reliability Enterprise Availability • Reduces complexity of environment because Oracle has tested the integration points, all hardware is tested to work together unlike commodity solution • Oracle is our standard database today, no conversion costs were incurred. Leverage Existing Technologies • Cox employees already had a skill set in this technology • Development best practices were enhanced 16
  17. 17. CEBI 2012 17
  18. 18. CEBI 2012 OBIEE 11G• Score carding• Mobility• Improved Visualizations• Spatial Intelligence via Map- based Visualizations• Business Process Invocation• Packaged Apps• Exalytics 18
  19. 19. CEBI 2012 Architecture Enterprise Business Intelligence Platform Reporting and Analytics Interactive Reporting and Adhoc Office Detect and Collaborate & Mobile and Scorecards Dashboards Publishing Analysis Integration Alerts Seach Embedded Enterprise Metadata Layer Standards Process Financial Data Master Data OLTP Data Data Warehouse Financial ERP - Human ERP - HR Finance Consolidation ORGANIZATION FInancials Resources CUSTOMER Sales & CRM Logistics Field Marketing EMPLOYEE Planning, Budgeting Time & Business PROJECT Billing & Forecasting Attendance Operations Data Sources Governance and Monitoring 19
  20. 20. CEBI 2012 Exadata• Primary Database Areas: - Reporting - Applications - Web Services• Standby Database Areas: - Analysis - What If - Predictions - Data Mining - Ad-Hoc 20
  21. 21. CEBI 2012 HistoryProcess Time Data Volume Users2008 – 10.5 Hours/Night 2008 – < 2 Billion/Night 2008 – < 10002009 – 12.5 Hours/Night 2009 – 36+ Billion/Night 2009 – 2500+2010 – 4 Hours/Night 2010 – 43+ Billion/Night 2010 – 5000+2011 – 3.5 Hours/Night 2011 – 50+ Billion/Night 2011 – 9000+BICC Migrations/Reviews Errors Per 1M Sessions User Generated Reports2008 – 100 2008 – 500 2008 – <700 Usr Rpts2009 – 1300 2009 – 150 2009 – >6000 Usr Rpts2010 – 2218 2010– 125 2010 – 10000+ Usr Rpts2011 – 3000+ 2011– 100 2011 – 15000+ Usr RptsComplexity2008 – Single Billing, Weekly/NightlyNumbers2011 – Multi Billing, Near Real Time 21
  22. 22. CEBI Customer Goals Personal Product Personalization Customized Interfaces Personalized Services Recommendations Customers want… Cross Team Efforts Cross Product Usage Growing Analytical Needs Internal Cox Users want… Product Planning and Data Analysis and Real-time Operations Data-driven Sales and Optimization Research Monitoring Marketing 22
  23. 23. CEBI FutureAccording to Gartner, Enterprise Data will grow 650% by 2014. 80% of this data willbe Unstructured Data, with a CAGR of 62% per year, far larger than transactionaldata. Unstructured Data in Web Pages Growth is taking place in areas not well served by traditional databases This chart shows the growth over data over the next couple of years. It is projected that a large portion of this growth will be unstructured data (web logs, emails, social interactions, etc.). Unstructured data is driving an explosive growth in data Unstructured data does not work well with traditional databases. To achieve the low response times, traditional databases rely on strict data structures. These strict data structures work well for certain types of Structured data. However, the growth of unstructured data data in the enterprise and the proposed uses of it create the need for a new type of data processing to be introduced to the technology stack. The 2011 IDC Digital Universe Study Sponsored by EMC 23
  24. 24. CEBI GoalsCreate additional value from customer data • Increase the perceived value of products by enabling a high degree of individual personalization. • Give a highly tailored customer experience every time a customer interacts with Cox.Make Cox a more data-driven company • Improve the efficiency and security of Cox operational processes. • Allow the company to make decisions, spot trends, and react to competitive challenges more quickly.Democratize access to data • Allow the company can make quick, innovative use of the data that is already being generated every day. • Improve cross-team and company wide insight into how customers are using Cox’s services. 24
  25. 25. CEBI Goals Cox Data Data Decision Framework Traditional Data Architecture Big Data ArchitectureStructured Data Unstructured/Semi-structured DataCharacterized by well-known use cases, requiring only Characterized by lack of established use cases and on-repeatable, static data cubes and ETL’s. Highly the-fly analysis in a “Sandbox” manner. Useful inproductized results. developing new insights, products.Example Use Cases Example Use Cases• Ad-hoc Queries • Ad-hoc Queries• Financial and Operational Dashboards • Data Mining/Discovery • Large Datasets, Fast Response Times• Ad Impression Analyzer • Predictive Analysis• Marketing AnalyzerChallenges Challenges• Sizing to support new reporting dimensions is not •Latency is greater than with traditional databases.always economically feasible. • Large unstructured datasets will need to be monitored• Analysis against new datasets can slow Time to Market and managed at scale.for new products. 25
  26. 26. CEBI Sample Decision Framework Low Data Volumes • Used to evaluate analysis useLatency Complex Simple casesHigh Exadata Any • Can determine which system toLow Exadata Exadata use: • Traditional database High Data Volumes • Non-traditional data storeLatency Complex Simple • Can standardize reporting andHigh Big Data Big Data analysis use cases across theLow Big Data Exadata enterprise 26
  27. 27. Future Solution Design Present Presentation Analytics Services Applications Store EDW Master Store NOSQL Virtual Process Data Cleansing Anonymization Billing Mediation Transform MDM ETL/ELT Map Reduce Virtualization ODS NOSQL Hadoop Federated Stagie Replication Mediation Ingestion Virtualization Data Sources (Raw Data) 27
  28. 28. POC in Progress• Exalytics• Oracle Big Data Appliance• Endeca Information Discovery• Packaged Apps 28