6. real time integration with odi 11g & golden gate 11g & dq 11g 20101103 - share


Published on

Alessandro Cagnetti

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

6. real time integration with odi 11g & golden gate 11g & dq 11g 20101103 - share

  1. 1. <Insert Picture Here> High Performance, High Productivity Data Integration Alessandro Cagnetti – EMEA Business development - Data Integration Solutions Middleware Solution Specialist Team
  2. 2. 2 Data Integration Buzz What the Analysts are saying about Data Integration “Data services will grow in adoption as organizations recognize the importance of data and its integration for success of SOA initiatives.” - Gartner “Most business functions can benefit from automation, but data integration problems must first be solved .” - CIO Insight “By 2012, 75% of organizations will have implemented some form of modern data integration architecture.” - Gartner “Increasingly, the value-added advantage of a company isn't what it makes but what it knows.” - Chief Executive Magazine 2
  3. 3. 3 Oracle Data Integration The solution for enterprise-wide real-time data Dramatically improve the accessibility, reliability, and quality of critical data across enterprise systems Web Services Databases Distributed systems Legacy systems OLAP systems OLTP systems Mission critical systems and data Business Intelligence, Performance Management Data Warehouses, MDM Data Access Real-time Data Data Quality SOA
  4. 4. 4 Oracle Data Integration Best-in-class Real-time Data Integration Oracle Data Quality and Data Profiling Oracle Data Integrator EE MDM Applications SOA Platforms Oracle Applications Business Intelligence Activity Monitoring Fusion Applications Data Cleansing Data Profiling File System Real-Time Data OLTP System Data Warehouse/ Data Mart OLAP Cube Data Migration ServicesBulk Data Services Data Quality Services Data Access Services ELT/ETL Oracle GoldenGate Zero Downtime Migration Data Replication Real-time CDC Data Services Layer Enterprise Service BusBPEL Process Manager Data Services Data Transformation Bulk Data Movement Match and Merge Oracle Data Integration Solution
  5. 5. 5 Move and transform data. Mixed sources and targets. BENEFITS KEY DIFFERENTIATED FEATURES 1. Performance >>> Heterogeneous “E-LT” 2. Flexibility >>> Event-Driven Platform 3. Productivity >>> Declarative Design 4. Open >>> 100% Java and SOA Native 5. Hot-Pluggable >>> Knowledge Modules ODI-EE Value Proposition Agile Data Integration 5
  6. 6. 6 SOA Suite BPEL & OSB ODI Integrated Oracle Fusion ERP Applications Other Apps … (+more)Business Intelligence Enterprise Performance Management and Financials Reduce TCO with Pre-Built Solutions Lower-Cost Adoption of Data Integration Within Oracle Products Fusion MDM ODI Embedded Pre-built Content BAM & CEP ODI Embedded BIEE Suite ODI Integrated BI Apps ODI Embedded Pre-built Content Agile PLM ODI Embedded Pre-built Content Retek ODI Embedded Pre-built Content HFM ODI Embedded Planning ODI Embedded Spend Analyzer ODI Embedded MDM/DRM ODI Integrated Essbase ODI Integrated Governance & Risk ODI Embedded Siebel CRM GoldenGate Integrated Fusion Pillar Replication OGG Embedded Pre-built Content Fusion HCM ODI Embedded Pre-built Content Fusion AIA ODI Embedded Pre-built Content Fusion CRM ODI Embedded Pre-built Content Fusion Financials ODI Embedded Pre-built Content Fusion Migration ODI Embedded Pre-built Content Pre-built Content Pre-built Content Pre-built Content ID Mgmt Identity Analytics ODI Embedded Pre-built Content Pre-built Content ODI Integrated New New New
  7. 7. Differentiator: E-LT Architecture High Performance with lower TCO Transform Benefits in Existing RDBMS • Eliminates the high cost of conventional ETL • No New Servers Required • Leverage Compute Resources & Partition Workload efficiently • Exploits Database Optimizer • Optimized Performance and Scalability Transform in Separate ETL Server  Poor Performance  High Costs POC: 40% performance increase vs traditional ETL Proprietary Engine Proprietary Language Leverage RDBMS Native SQL
  8. 8. 8 8 Differentiator: Declarative Design Increased Productivity & lower Maintenance costs Conventional: Specify ETL Data Flow • Developer must define every step of Complex ETL Flow Logic • Traditional approach requires specialized ETL skills • Significant development & maintenance cost Oracle: Declarative Set-based Design • Abstracts Modeling to High Level Design • Simplifies the Number of Steps • Automatically generates the Data Flow whatever the sources and target DB • Easy to Handle Customizations: eg. Extension Columns in ERP Schemas Benefits • Significantly reduce the learning curve • Shorter implementation times • Streamline access to non-IT pros Conventional ETL DesignConventional ETL Design ODI Declarative Design Define How: Built-in Templates Define What You Want Automatically Generate Dataflow 11 22 ODI Declarative Design Define How: Built-in Templates Define What You Want Automatically Generate Dataflow 11 22 Define How: Built-in Templates Define What You Want Automatically Generate Dataflow 11 22 Define What You Want Automatically Generate Dataflow 11 2211 22
  9. 9. 9 9 Bloor Research: Lowest TCO over competitors While this survey was commissioned by Pervasive, Oracle fared well in a number of scenarios, especially against IBM and Informatica especially at TCO and Speed of Implementation Note: this survey is not available for re- distribution on our Oracle website
  10. 10. 10  Tailor to existing best practices  Reduce cost of ownership  Library of hot-pluggable templates  Open and extensible  Declarative set-based mapping • Spent 2-weeks at project initiation to tailor KMs to best practice • Now can do development to add a table to their DW in 20 seconds, assured of Best Practice Implemetation Customer Benefits: Differentiator: Knowledge Modules Engineer once, Reuse everywhere SAP/R3 Siebel Log Miner DB2 Journals SQL Server Triggers Oracle DBLink DB2 Exp/Imp JMS Queues Check MS Excel Check Sybase Oracle SQL*Loader TPump/ Multiload Type II SCD Oracle Merge Siebel EIM Schema Oracle Web Services DB2 Web Services
  11. 11. 1-11 ODI Design-Time Environment ODI Runtime Environment User Interfaces Architecture: from Development to Production 1. Retrieve/Enrich metadata 2. Design transformations 3. Orchestrate data flows 4. Generate/Deploy data flows 5. Monitor executions 6. Analyze impact / data lineage Development Development Servers and Applications Agent Data Flow Conductor CRM Legacy ERP Data Warehouse Files / XML User Interfaces Administrators Designers ESB Design-time RepositoriesDesign-time Repositories Production Production Servers and Applications Agent Data Flow Conductor CRM Legacy ERP Data Warehouse Files / XML Operator Metadata Navigator ESB Runtime Repository
  12. 12. 12
  13. 13. 13
  14. 14. 14
  15. 15. 15 15 Comprehensive Data Quality For Data Integration with Oracle Data Integrator StandardQuality Data Movement & Transformation Advanced Data Profiling Standard Profiling Ongoing Audit & Monitoring Advanced DataQuality Comprehensive Data Quality Sources Targets Oracle Data Quality and Oracle Data Profiling (OEM Agreement with Trillium)
  16. 16. Example of Data Quality Issues A Simple Customer Table Sample Name Address City State Zip Phone Email Bob Williams 36 Jones Avenue Newton MA 02106 617 555 000 bob.williams@yahoo.com Robert Williams 36 Jones Av. MA 02106 617555000 Burkes, Mike and Ilda 38 Jones av. Nweton MA 02106 617-532-9550 mburkes@gmail.com Jason Bourne, Bourne & Cie. 76 East 51st Newton MA 617-536-5480 6175541329 … … … … … … … Mis-fielded data Matching Records TyposMixed business and contact names Multiple Names Non Standard formats Missing Data
  17. 17. 17 How can this… = = this? Name1 Flugtaggen GMBH Name2 rhamer strasse 20 Address dus City/Town 40489 Post Code Country Business Name Flugtaggen GMBH Address Rhamer Str. 20 City/Town Düsseldorf Post Code 40489 Country DE The Importance of Data Quality
  18. 18. 18 Original Record 1 Original Record 2 Standardized Record 1 Standardized Record 2 Name Flugtaggen GMBH Contact Address Rhamer Str. 20 City Düsseldorf Post Code 40489 Country DE Name Flugtaggen GMBH Contact Klaus Webersinke Address Rhamer Str. 20 City Düsseldorf Post Code 40489 Country DE Name1 Flugtaggen GMBH Name2 rhamer strasse 20 Address dus City/Town 40489 Post Code Country Name1 Klaus Webersinke Name2 c/o Flugtaggen GMBH Address 20 ramerstr. City/Town Dusseldorf Post Code Country DE How Standardization Helps Matching
  19. 19. • Time Series controls data quality over time • Set right timeframe, right business thresholds • Alert if DQ metrics and business rules are not met Monitor Data Quality over time
  20. 20. 20 Oracle GoldenGate provides low-impact capture, routing, transformation, and delivery of transactional data across heterogeneous environments in real time Key Differentiators: Non-intrusive, low-impact, sub-second latency Open, modular architecture - Supports heterogeneous sources and targets Maintains transactional integrity - Resilient against interruptions and failures What is Oracle GoldenGate? Performance Flexible and Extensible Reliable
  21. 21. 21 Traditional ETL + CDC • Invasive Capture on OLTP systems using complex Adapters • Transformations in ETL engine on expensive middle tier servers • Bulk load to the data warehouse with large nightly/daily batch • Continuous feeds from operational systems • Non-invasive data capture • Thin middle tier with transformations on the database platform (target) • Mini-batches throughout the day or bulk processing nightly ODI + Oracle GoldenGate Staging Trickle Lookup Data Load Extract Lookup Data Xform Xform Bulk GG+ODI GG+ODI Heterogeneous Real-time CDC Integration Best-in-class solution for real-time with Oracle GoldenGate
  22. 22. 22 Oracle for Real-Time Data Warehousing Fastest Real-Time Data Integration, Fastest Bulk Data Transformation BI Application Production OLTP Databases DW tx6 tx5 tx4 tx2 tx1tx3 Real-time streaming of transactions LAN / WAN / InternetCapture Source Trail Target Trail Deliver • Sub-second data latency • Minimal overhead and no batch windows • High-performance, in-database transformations • Read-consistent changed data with referential integrity • Complete data recoverability via Trail files ODI
  23. 23. 23 Extreme Performance for Data Warehousing Exadata V2 is the Platform for Real Time Data Warehousing Oracle Data Integrator EMP DEPT DIM FACT DIM DIMDIM ODS Schema DW Schema Load Exadata with GoldenGate; Generate Transforms from ODI-EE • New Exadata platform optimized for OLTP and Data Warehousing suits Real Time DW • Extreme Exadata performance can handle huge data volumes that come with Real Time DW Oracle GoldenGate Oracle Database
  24. 24. 24 Oracle GoldenGate Overview Enterprise-wide Solution for Real Time Data Needs Log Based, Real- Time Change Data Capture Heterogeneous Source Systems EDWODS EDW Disaster Recovery, Data Protection Zero Downtime Migration and Upgrades Operational Reporting Real-time BI Standby (Open & Active) Reporting DatabaseOGG ETL ETL Query Offloading Data Distribution • Standardize on Single Technology for Multiple Needs • Deploy for Continuous Availability and Real-time Data Access for Reporting / BI •Highly Flexible • Fast Deployments •Lower TCO & Improved ROI
  25. 25. 25 Oracle GoldenGate Topologies Unidirectional Query Offloading Zero-Downtime Migration Bi-Directional Hot Standby or Active-Active for HA Peer-to-Peer Load Balancing, Multi-Master Broadcast Data Distribution Integration/Consolidation Data Warehouse Data Distribution via Messaging BP M
  26. 26. 26 Energy/Industrial Insurance / Health Retail Other Communications Finance / Banking Media Public Sector ODI: an horizontal product Ireland's Department of Social & Family Affairs
  27. 27. 27 Benefits of Real-time DI with Oracle Cut Costs, Reduce Risk, and Revolutionize Business Insight • Move only the changed data from redo logs and reduce source and network overhead • Shorten implementation times from months to weeks using pre- packaged integrations to well-known applications, sources and targets. Cut Costs and Improve Efficiencies. • Eliminate performance impact on source systems • Reduce the risk of missed orders, poor customer interactions, missed opportunities through improved recoverability, data quality Reduce Risk, Ensure Continuity • Enable near real-time decision making with real-time data flows • Combine real-time data with historical context for better insights Improve Business Insight
  28. 28. 28 Quote Attribution Title, Company • Visit the Oracle Fusion Middleware 11g web site at oracle.com/goto/fmw11g/index.html • Oracle Data Integration on oracle.com oracle.com/goto/odi • Oracle GoldenGate on oracle.com oracle.com/goto/goldengate • Oracle BI/EPM on oracle.com oracle.com/solutions/business_intelligenc e/index.html Get Started • Oracle GoldenGate Resource Kit: oracle.com/goto/goldengate • Blog: http://blogs.oracle.com/dataintegration • Technical information available at: oracle.com/technology/products/oracle- data-integrator/goldengate/index.html • Data Integration Events http://www.oracle.com/events Resources
  29. 29. 29
  30. 30. 31
  31. 31. 32 For More Information search.oracle.com or oracle.com