Downloaded 97 times
![Oracle Data Integration: Improving Information Agility Mark Rabne Senior Regional Sales Consultant [email_address]](https://image.slidesharecdn.com/fmwforumodifinal-12538384263417-phpapp01/85/Fusion-Middleware-Oracle-Data-Integrator-2-320.jpg)
![Hands-On ODI Workshop Columbus, OH – October 6 th Contact: [email_address]](https://image.slidesharecdn.com/fmwforumodifinal-12538384263417-phpapp01/85/Fusion-Middleware-Oracle-Data-Integrator-45-320.jpg)
Uploaded byMark Rabne
Fusion Middleware Oracle Data Integrator
Oracle Data Integration can help improve information agility by enabling unified information management across organizations. It provides optimized data loading and real-time integration capabilities. This allows businesses to gain improved insights, reduce IT costs, and enhance agility through features like declarative design, knowledge modules, and data quality tools. Customers have seen benefits such as cost savings, increased revenues, and better decision making through using Oracle Data Integration.
More Related Content
Similar to Fusion Middleware Oracle Data Integrator
Fusion Middleware Oracle Data Integrator
- 1.
- 2. Oracle Data Integration:Improving Information Agility Mark Rabne Senior Regional Sales Consultant [email_address]
- 3.
- 4. The following isintended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
- 5. Agenda Why UnifiedInformation Management is key to Information Agility Oracle’s Data Integration Offering & Strategy How Unified Information Management impacts IT and the Business Customer Case Studies Summary © 2009 Oracle Corporation
- 6. Business Challenges ManagingInformation Requires Accessible, Up-to-date, and Trusted Information © 2009 Oracle Corporation Q: Why can’t I see the reporting results that I need for all my inventory? Accessible Information Ensure information is connected, available, reliable, and secure. Connect information across mixed sources and targets. Q: Why do I have so many duplicate copies of my customer data? Why isn’t it accurate? Trusted Information Ensure information consistency, quality, and authoritative data for improved, actionable business insight. Q: Why are my applications referring to last weeks retail sales numbers? Up-to-date Information Integrate bulk data in real-time across data warehouses, data marts, business processes, MDM applications, BI applications.
- 7. IT Challenges toUnifying Information What is it costing you to unify your data? OLTP & ODS Systems Data Warehouse, Data Mart Oracle PeopleSoft, Siebel, SAP Custom Apps Files Excel XML OLAP © 2009 Oracle Corporation Enterprise Performance Custom Reporting Packaged Applications Business Intelligence Custom Analytics Data Federation Data Warehousing Custom Data Marts Data Access Data Silos SQL Java Batch Scripts Data Hubs Data Migration Data Replication Fragmented Data Silos Slow Performance Poor Data Quality Out of sync Missed SLAs
- 8. Agenda Why UnifiedInformation Management is key to Information Agility Oracle’s Data Integration Offering & Strategy How Unified Information Management impacts IT and the Business Customer Case Studies Summary © 2009 Oracle Corporation
- 9. Where Does DataIntegration Fit? Essential Ingredient for Information Agility SOA Business Intelligence Enterprise Data Warehousing Master Data Management
- 10. Oracle Data IntegratorEnterprise Edition (ODI-EE) Optimized E-LT for improved Performance, Productivity and Lower TCO Declarative Set-based design Real Time Change Data Capture E-LT Transformation vs. E-T-L High Speed Batch Data Movement Data Warehouse Planning System OLTP DB Sources Application Sources Legacy Sources Pluggable Knowledge Modules © 2009 Oracle Corporation Application ODI-EE is part of ODI Suite, which is part of Fusion Middleware
- 11. Oracle Data QualityEnsure Data Quality as Part of the Integration Process ODI EE Oracle Data Profiling Oracle Data Quality Better Data Visibility Tighter Data Control Greater Data Accuracy Metadata Profiling of Source & Target Duplicate Detection, Matching & Merging Visual Data Quality Tools © 2009 Oracle Corporation
- 12. Oracle Data IntegrationSuite Comprehensive solution for Data Integration Comprehensive Optimized Performance with E-LT Improved productivity Lower TCO Open, Java-based, SOA native © 2009 Oracle Corporation Oracle Data Integration Suite Heterogeneous Data Sources MDM, Business Intelligence, BAM, SOA Platforms, Applications Master Data Data Integrator Data Services Data Grid Process Manager Service Bus Data Quality Data Profiling Human Workflow Data ELT/ETL Data Migration Data Federation Data Replication Data Governance
- 13. Unified Information Strategyfrom Oracle Complete, Integrated Platform from Storage to Scorecard Application and Data Services Business Process Management Information Management © 2009 Oracle Corporation Content Management Services Business Intelligence Services Collaboration Services Transaction Processing Services File System Real-Time Data Feed OLTP Content Mgmt Data Warehouse OLAP Cube Oracle Data Integration Suite Oracle Data Warehousing Oracle BI Foundation OLAP OLTP Appliance Mining Partitioning Modeling ETL/ELT CDC Data Federation Data Services Data Quality Master Data SQL Tooling RAC
- 14. Agenda Why UnifiedInformation Management is key to Information Agility Oracle’s Data Integration Offering & Strategy How Unified Information Management impacts IT and the Business Customer Case Studies Summary © 2009 Oracle Corporation
- 15. Improving Information AgilityOracle Data Integration as part of the Fusion Middleware family © 2009 Oracle Corporation
- 16. Reduced IT Costs Lower TCO for integration and migration projects © 2009 Oracle Corporation
- 17. Reduced IT Costs Lower TCO for integration and migration projects © 2009 Oracle Corporation Reduced IT costs through consolidation and data migration
- 18. Heterogeneous sources Heterogeneous targets Any Data Warehouse Flexible data loading Incremental load Changed data capture Slowly changing dimensions Data integrity and consistency Data lineage Data Warehousing Using ODI Reduces costs by consolidating Data silos © 2009 Oracle Corporation Data Warehouse Cube Cube Cube ---- ---- ---- ---- Operational Analytics Metadata Load Transform Capture Changes Incremental Update Data Integrity Aggregate Export
- 19. Bulk-load historical datato new application Transform source format to target Synchronize new and old applications during overlap time Capture changes in a bi-directional way (CDC) Data Migration Using ODI Bulk Load Data To Ease Application Migration © 2009 Oracle Corporation Old Applications New Application Metadata Initial bulk load CDC for synchronization Transformation to new application format CDC for loop-back synchronization CDC CDC ---- ---- ---- ----
- 20. Verizon Reduces coststhrough consolidation Projected cost savings of $400M in 3 Years Significantly reduced the hardware footprint through combined SOA+ ODI ODI + SOA helped to build out a shared services layer which helped provide greater flexibility Consolidate database (DB2, Sybase, Informix to Oracle) Consolidate ERP (PeopleSoft HR/Payroll, Decommissioning SAP), In addition the program helped to decommission Cystal and Cognos and adopt Oracle BIEE Results Using Oracle Data Integration Streamline and automate business processes around new hire on boarding Provide better business insight and manage FiOS supply chain Lower the cost of rewriting SAP, PeopleSoft & Legacy interfaces (about 209) through custom code Lower the cost of IT infrastructure from multiple systems Challenges © 2009 Oracle Corporation
- 21. Eliminate Custom CodeImproved flexibility and declarative design lowers implementation costs © 2009 Oracle Corporation Reduced development costs through declarative design Pre-integrations (Knowledge Modules) for faster time to value
- 22. Declarative Set-based Design– “The WHAT” Provides significant reduction in custom coding ODI provides Declarative Set-based Design, simplifies number of steps Benefits: Significantly reduce the learning curve Streamline access to non-IT pros Shorten implementation times High performance every time © 2009 Oracle Corporation
- 23. Leverage Database Optimizations: Native SQL; Native Functions; Native Loads; Native Journaling / CDC Tailor to an organization’s existing best practices Ease administration work Reduce cost of ownership Journalize Read from CDC Source Load From Sources to Staging Check Constraints before Load Integrate Transform and Move to Targets Service Expose Data and Transformation Services Reverse Engineer Metadata Reverse Journalize Load Check Integrate Services Knowledge Modules Architecture CDC Sources Staging Tables Error Tables Target Tables WS WS WS SAP/R3 Siebel Log Miner DB2 Journals SQL Server Triggers Oracle DBLink DB2 Exp/Imp JMS Queues Check MS Excel Check Sybase Oracle SQL*Loader TPump/ Multiload Type II SCD Oracle Merge Siebel EIM Schema Oracle Web Services DB2 Web Services Sample out-of-the-box Knowledge Modules Benefits 100+ Pluggable Knowledge Modules – “The HOW” For Faster Time to Value, Key to Flexibility, Modularity, and Extensibility © 2009 Oracle Corporation
- 24. Financial Case Study:Interbank Reduced Development Costs for ETL Increases development productivity by at least 50% Increases market share by 50% by being able to correctly profile each client based on the risk Produces data interfaces that are better documented and easier to understand Executes loading process that used to take eight hours in less than two hours despite dramatic increases in data complexity of the transformations Results Using Oracle Data Integration Improve loading of data warehouse Being able to correctly profile each client based on the risk Improve the integration of banking systems, collection systems, and front office systems Modernize batch exchanges and EDI interfaces toward real time Challenges © 2009 Oracle Corporation
- 25. Improve Performance Real-timedata warehousing for real results © 2009 Oracle Corporation High volume performance for bulk data movement through ELT Real-time feeds, improved overall responsiveness
- 26. Optimized Data Loadingthrough E-LT The key to improved performance and reduced costs E-LT provides flexible architecture for optimized performance Benefits: Leverage Set-based transformations Improved performance for loading, no network hop Takes advantage of existing hardware © 2009 Oracle Corporation Lookup Conventional ETL Architecture Extract Load Transform Next Generation Architecture “ E-LT ” Load Extract Transform Transform
- 27. Changed Data CaptureEnables Real-time Data Warehousing Change Data Capture provides real-time updates of mixed data sources without impacting performance Benefits: Improved performance for extraction – delta based Operates in near-real time Timeliness of data © 2009 Oracle Corporation
- 28. Overstock.com Real timeEnterprise Data Warehouse “ Having access to key business metrics in real-time is no longer a fantasy. In short, Oracle Data Integrator gave us the ability to make better decisions and better manage our bottom line.” Found a way to ensure that data warehouse was constantly updated. Even highly complex transformations are automated Supporting several terabytes of data stored in the enterprise warehouse, and millions of daily transactions Results Using Oracle Data Integration Wanted to enable sales, finance, marketing and merchandising teams to have access to near real-time data so that they could make timely, more intelligent business decisions. Wanted to know at any point in time if company performance is meeting the target metrics. Needed a data integration product that could handle our high-volume loading and transformation requirements in near real time. Challenges © 2009 Oracle Corporation
- 29. Business Benefits ofUnified Information Management Agile and Intelligent for Business © 2009 Oracle Corporation
- 30. Improve Agility foryour Data Re-use information for improved business performance © 2009 Oracle Corporation Connect SOA + Data Integration for improved flexibility and agility of business processes
- 31. Bulk Data ServicesProvides business processes integration and improved re-use Decouple business apps from underlying data-centric applications Recombine existing services and app components instead of new development Benefit Enables efficiencies in transformation, data loading, data synchronization, all at improved performance © 2009 Oracle Corporation Oracle SOA Suite Business Activity Monitoring Web Services Manager Business Rules Engine Enterprise Service Bus Oracle BPEL Process Manager Bulk Data Processing Oracle Data Integrator ODI Knowledge Modules Transform. Services Changed Data Capture Data Services ODI Connectivity Framework
- 32. Ross Stores DataWarehouse Agility ‘for Less’ 75% improvement of Business Process Execution Together with Oracle SOA Suite (Oracle BPEL PM) provided business optimization, process visibility, exception handling Seamless data integration from heterogeneous sources leveraging SOA helped to create a more flexible, standards based integration approach Closed loop processing using BAM and Data Integration combined eliminated ordering/replenishing inventory errors by reducing inaccurate data Results Using Oracle Data Integration Hard to consolidate consistent, accurate inventory information across over 800 stores Lack of visibility to errors to the business Higher cost and time to delivery of new value Current state was complex, didn’t scale, and difficult to manage Non-standard approach required coding paradigm, inconsistent error handling Challenges © 2009 Oracle Corporation
- 33. Impact of CleanData Reduced risk and ensure compliance © 2009 Oracle Corporation Improve data quality, invoke data cleansing rules, data profiling
- 34. Data Quality LifecycleEnsure Data Quality as Part of the Integration Process Integration Process Target Sources Errors Corrections Error Recycling Data Quality and Profiling provide cleansing of data as part of integration process Benefits: Risk reduction from bad data Compliance – where information flows Improves visibility, business insight Trusted data © 2009 Oracle Corporation BPEL Process Manager Human Workflow Errors Flow Check Data Quality Process Static Check
- 35. Leading Online JobPlacement Firm in North America What’s the Return on investment for Clean Data? Increased overall revenue by $2M in one month by improving data quality Saved $1M in acquisition: company had bad data Increased direct/inside sales productivity Increased number of sales (more calls per day), avg order (better targeting, messaging) Increased cross-selling: Cross-divisional lead sharing 280% revenue growth in 3 years Results Using Oracle Data Integration Integrate data across multiple systems, including legacy acquisition infrastructure. Better profile and detect issues with data among multiple systems Remove duplicates and eliminate errors associated with better identifying customer data Challenges © 2009 Oracle Corporation
- 36. Improving Business Insightwith Data Integration Improved Data Lineage – better trusted data © 2009 Oracle Corporation End-to-end data lineage - ensure pervasive reach of BI/EPM applications to multiple source systems
- 37. Improve Visibility ofUnified Data Oracle Data Integrator EE & Oracle BI Data Warehouse/Data Mart Planning & Budgeting Metadata-driven Data Lineage Planning & Budgeting Analytic Dashboards © 2009 Oracle Corporation Oracle BI EE, Essbase & ODI EE
- 38. Oracle OnDemand, OracleHyperion Planning, Oracle Data Integrator “ By using Oracle Data Integrator, we were able to extract, transform, and aggregate financial information from two distinct systems and swiftly load data into our Hyperion Planning application. By automating the data integration process and eliminating manual steps, we have saved time and resources, greatly reduced the risk of error associated with manually entry, and have more confidence about the data critical to our key planning and decision making processes.” . © 2009 Oracle Corporation Lower TCO for Performance Management Using Oracle Data Integration
- 39. Summary: Improving InformationAgility Using Oracle Data Integration © 2009 Oracle Corporation Unique Design Advantage Efficiency and Innovation for IT Agile and Intelligent for Business
- 40. Next Steps Unifyenterprise information for improving information agility Leverage E-LT and Change Data Capture for real-time BI/DW Align business demands and IT requirements for data initiatives – Ask “What If” questions Assess the existing and future needs of data across your enterprise Leverage Oracle Partners and Oracle Professional Services
- 41. Additional Customer ExamplesGlobal Case Studies
- 42. Order Replenishment, OnlineCatalog J. Crew Simplified the end-to-end data integration with all their core IT systems and database platforms to reduce the total cost of ownership of their order replenishment and cataloging Benefited from the Oracle advantage in batch processing, bulk data transformation for increased performance Successfully leveraged SAP and Teradata integration to phase out legacy systems and ultimately reduce costs Order replenishment for online catalog system Define new corporate standard for data integration Better integrate data across 175 retail stores and 52 outlet chains Business Challenges Oracle Data Integration Solution Benefits IT Challenges Replace antiquated home-grown solution of scripts Teradata bulk data support, and help integrate data across SAP, Red Prairie Inventory systems
- 43. Sales and MarketingData Warehouse Nestle Provided 100GB of actionable data to the business users Replaced weekly insertion with daily recording of change in status with time imprint for all clients—allowing for more effective client follow-up, for example Improved response times by transferring reporting processes to the data warehouse Improved reactivity for restocking requirements thanks to daily updates on stock quantity and value for each store Provided overall view of the status of business processes, resulting in improved user communication Hard to consolidate consistent, accurate inventory information across over 800 multiple store chains Lack of visibility to errors to the business Higher cost and time to delivery of new value Business Challenges Oracle Data Integration Solution Benefits IT Challenges Manage a data warehouse for operational transactions and marketing information Lack of high-value-added actionable data on clients, orders, stock, etc. Enable the marketing and CRM team to better target its campaigns and activities across 50 countries Anticipate future reporting challenges, in light of the strong increase in the volume of production data
- 44. For More InformationQuote Attribution Title, Company Visit the Oracle Fusion Middleware 11g web site at http://www.oracle.com/fusionmiddleware11g Oracle Data Integration on oracle.com www.oracle.com/goto/odi Oracle Fusion Middleware on OTN http://otn.oracle.com/middleware Get Started Datasheet: http://www.oracle.com/products/middleware/odi/docs/odiee-datasheet.pdf Blog: http://blogs.oracle.com/dataintegration Technical information available at: http://www.oracle.com/technology/products/oracle-data-integrator/index.html Data Integration Events http://www.oracle.com/events Resources © 2009 Oracle Corporation
- 45. Hands-On ODI WorkshopColumbus, OH – October 6 th Contact: [email_address]
- 46. Information Management Pleaserate this session now. Thank You!! Session Evaluation
- 47. Questions © 2009Oracle Corporation
- 48.
Editor's Notes
- #6 1 – what does it mean to the business as well as to IT 2 - we will walk through Oracle’s product set and our strategy and directions and integration with other Oracle products 3 – how our strategy impacts IT and the business and will use real world examples and benefits.
- #7 Data Integration can be distilled into some basic everyday challenges, let’s look at these examples: “ Why can’t I see the reporting results for all my inventory?” Accessible Information requires bridging heterogeneous sources and connections. Managing a disparate set of data can be a daunting task. Connnected, Available, Reliable, and Secure. “ Why are my application still referring to last weeks retail sales numbers?” Up-to-date, real-time data would solve this problem-ensuring the timeliness of all data, bulk, batch, it shouldn’t matter. There are better ways to keep data integrated than simply waiting a week for the next batch to run. “ Why do I have so many duplicate copies of the same customer data?” Is your data in your enterprise clean? Ensuring that the data is of the right quality, consistency. That’s a huge problem for integrating data. Sometimes data is integrated but it isn’t matched correctly. Redundancies in product, customer names? Takeaway: As companies embrace Data Integration, we’ll start to see some of these types of business challenges being solved
- #8 The challenge many companies face today is that they expect their SOA or their BI or their Data Warehousing to solve many of their integration issues automatically. Worse yet, they simply sweep a lot of these issues under the rug and leverage custom code, scripts or even SQL. They overlook their architecture requirements for how data spans heterogeneous sources, and they need to not only be integrated, but considerations about keeping data in sync, timely, and with the right quality are of key importance. Instead they are left with an architecture like this one where the approach to solve the problem was admirable, but ultimately the fragmented approach leads to little tangible benefit. Here’s why: Fragmented approaches - lead to a lack of control, inconsistent data, no common visibility of how information is being used Doesn’t perform – inefficient in how it operates and doesn’t help you achieve the benefits of your business. Out of sync - Integration is complex, costly, requires custom code, development or silo’d, impacts performance and speed of implementation Lack of Trusted Data - Ambiguous sources of data and multiple sources of truth which aren’t matching, lack of integrations, inability to consolidate Finally what’s the cost of this approach? It’s possible to solve some of these challenges by hand using custom coding and custom approaches – typically this is what many organizations are forced to do.
- #10 Today’s organizations are looking at multiple paradigms for agility either separately or together. In each case, there is a common element among them. Data Integration. Data Integration provides the important aspects for moving data and turning it into re-usable information – it acts as a key enabler for each architecture principle (SOA, BI, or Data Warehousing) and can also act as a key bridge between them. SOA and DI Focus example: For example, lets say I want to implement a process in my Service Oriented Architecture that accesses large amounts of data. Unfortunately that data is part of a data warehouse, to ensure that I only get the updated information that is constantly being synchronized, I’ll need to leverage capabilities in Data Integration and make a call to it from the service in my SOA. If it isn’t connected easily the architects don’t know how to get to it. It also needs to be optimized for performance, the same way that it is from it being managed by my data warehouse.
- #11 Main Points: ODI-EE is a the strategic ETL tool from Oracle; it’s ELT based which is unique in the industry, it combines the offering from both ODI and OWB. Oracle Data Integrator Enterprise Edition delivers unique next-generation Extract Load and Transform (E-LT) technology that improves performance, reduces data integration costs, even across heterogeneous systems. Oracle Data Integrator offers the productivity of a declarative design approach, as well as the benefits of an active integration platform for seamless batch and real-time integration. In addition, Knowledge Modules provide modularity, flexibility, and extensibility. Oracle Data Integrator Enterprise Edition is optimized for Oracle Database to provide real-time data warehousing with advanced ETL/ELT and data modeling. Oracle Data Integration Enterprise Edition, a foundational component to the Oracle Data Integration Suite is Oracle Fusion Middleware's strategic data integration offering; which has emerged from combining the best of both Oracle Data Integrator (ODI) and Oracle Warehouse Builder (OWB) into a single unified product offering
- #12 Main Points: Oracle Data Quality and Data Profiling is the best of breed option for managing your data quality implementations as part of the integration process. It’s optimized to work directly with ODI-EE to ensure information is cleaned as part of a lifecycle – since cleaning information is never a one-time activity. Oracle Data Quality is a proven solution to help cleanse, validate your data as it is being moved and integrated, this gives you the flexibility to ensure that your data always consistent before it is moved across into the target. Oracle Data Quality also includes these key advantages: • Best of breed Data Quality for cleansing, validation (Trillium) • Pre-built Knowledge Modules and Quality Rules • Optimized for Oracle Data Integrator Oracle Data Profiling provides extensive data profiling, this is an important tool used by both Data Stewards as well as even Business Analysts to make better decisions about how Data is being re-used. It gives fine-grained control for managing data auditing and trace-ability without impacting the best of breed performance for Data Integration. It delivers auditing and traceability. Oracle Data Profiling also includes these 3 key advantages: (Also from Trillium) • Declarative design • Integrated metadata • High speed runtime
- #13 Main Points: Oracle Data Integration Suite is the leading data integration and management solution on the market today, providing a fully unified solution that enables comprehensive data integration and data management for building, deploying and managing complex data warehouses. Oracle Data Integration helps automate ETL, data migration, data federation, data replication, data governance, while ensuring that information is timely, accurate, and consistent across heterogeneous systems. Oracle Data Integration is the foundational data integration mechanism for Oracle SOA, Oracle Business Intelligence and Oracle Master Data Management. Oracle Data Integration is also hot-pluggable to integrate data between 3rd party applications for increased business agility. The additional benefit of the Oracle Data Integration Suite - Complete comprehensive offering with flexibility built in - Optimized performance leveraging EL-T for data movement across heterogeneous systems in batch or real-time Lowered TCO for integrating and managing Data-centric architectures. Improves productivity with declarative design tools, pre-integrated to BI and many well-known applications. - Hot Pluggable. Open, java-based, SOA Native
- #14 Main Point: Oracle offers a comprehensive approach to information management for the enterprise which is unique in the industry Oracle offers a complete information management offering which spans the gamut of Business Intelligence, Data Integration, and Data Warehousing to meet the growing demands of your business. Together this platform solution can be enabled what we call “extreme intelligence”. The extreme performance of Oracle Database and Oracle Exadata together with Oracle’s Business Intelligence foundation; in the middle is ODI as the glue holding it all together. The Services abstraction layer above offers re-use and flexibility to hook directly into your business processes. That’s where Oracle SOA Suite comes in to help with pre-integrations to BPEL and ODI that we’ll discuss later on in this presentation.
- #16 Main Point: These key differentiators give ODI a unique design advantage. We’ll later see how this translates into both IT and Business benefits. We saw earlier how Oracle Data Integration solutions offer a complete set of tools for data integration from data movement, data transformation, data services, data quality, profiling. The solution also is Integrated into Fusion Middleware as well as Oracle’s Information Management solutions including Oracle BI/EPM solutions, Oracle Data Warehousing, Oracle Exadata and Oracle Master Data Management. The solution is best-of-breed in that we’re the leading solution offering for E-LT. No other solution is like Oracle in that respect. Finally the solution is hot-pluggable to work in multiple java platforms as well as load data sources and targets in not just Oracle but 3 rd party applications / RDBMS systems as well.
- #17 Main Point: For IT these unique design advantages help companies improve the cost structure and improve efficiencies to gain new ground. Specifically in this section we’ll see how ODI helps companies consolidate and integrate, eliminate custom code and improve performance
- #18 Main Point: Consolidation and Integration is a common way to reduce IT costs. We’ll look at specifically some ways that Data Integration can help in this capacity. Can reduce hardware and software and labor.
- #19 Main Points: ODI is optimal for loading data into Data Warehouses because it supports the gamut of requirements for heterogeneity, incremental loads, data integrity and data lineage Often times organizations are interested in reporting on data for a specific purpose, say getting reports of all users that purchase items on the web. So to build that type of reporting infrastructure, you can build a data-mart, expose a reporting engine and you’re done. But say you want to report on all purchases on the web, and relate that to customers who have registered, and gain visibility to your supply chain. Now you’re talking about taking data and using it more flexibly – you need a consolidated data warehouse as opposed to multiple fragmented data marts. The whole premise of “data warehousing” has data consolidation at the heart. A consolidation architecture is one where you use an ETL tool to consolidate data sources into a target data warehouse. This target that can then be leveraged by BI applications, reporting applications, or any application that needs the data. Keeping this data consistently updated is also a key requirement. ODI-EE support the full needs of data warehousing loads by supporting: Heterogeneous sources and targets Incremental load Slowly changing dimensions Data integrity and consistency Changed data capture Data lineage
- #20 Main Point: ODI is optimal for bulk loading data as part of a migration effort because it supports bulk loading, transformation and importantly key synchronization capabilities that are needed for dealing with new vs. old applications. A common challenge many organizations face is with the need to upgrade one application and replace it with another as part of a migration effort; companies through acquisition may find themselves owning several applications and trying to rationalize them. But often overlooked is the need to deal with the underlying data as part of the migration effort. Data Integration is a key technique for managing data migration through a combination of Change Data Capture which tells the systems what changed recently along with the ability to deal with transformation large files and finally the requirement for being able to move data from one system to the next. ODI-EE support the full needs of data warehousing loads by supporting: Bulk-load historical data to new application Transform source format to target Synchronize new and old applications during overlap time Capture changes in a bi-directional way (CDC)
- #21 Main Point: Verizon (North America) managed to achieve cost savings projected in the order of $400M in 3 years simply by using Data Integration to consolidate and integrate their systems and applications. Verizon is a good example of a company that implemented consolidation through building out a data abstraction layer. They started with a project for new hire on-boarding. As you know it’s a global company with multiple stores, retail sites, various aspects of the company, wireless, traditional phone service, and an enormous online presence. They wanted to streamline these operations in addition to provide better business insight into their Fiber Optic (FiOS) supply chain To solve this, they needed to eliminate the redundancies of multiple interfaces… over 209 of them. They chose Oracle Data Integration, together with Oracle SOA Suite, to build out a consolidated shared services architecture, one which centered around a data warehouse. When they used ODI and Oracle SOA Suite together they were able to dramatically reduce the footprint of their IT infrastructure. Verizon is Projecting a cost savings of $400M in 3 Years through IT consolidation while eliminating complexity (from > 800 apps, 1000s servers to <200 apps, 100s servers) They’re consolidating their 209 interfaces, and consolidating the number of applications they needed to support and even consolidating their database infrastructure. Because ODI is heterogeneous in nature, this was easy to configure and maintain for Verizon.
- #22 Main Point: Reduce custom code and custom SQL through reduced development costs and pre-integrations for faster time to value.
- #23 Main Point: Declarative Set-based design can radically alter the way you design and deploy ETL and can eliminate complexity and dramatically reduce implementation times. In a conventional ETL design, the developer must define every step of Complex ETL Flow Logic, this traditional approach requires specialized ETL skills, which in turn impacts your development & maintenance costs accordingly, a better solution – and one that ODI embraces - is to abstracts the modeling to a High Level Design, in other words, you want to automatically generates the Data Flow whatever the sources and target DB, we call this set-based declarative design where you customize the flow to handle extensions as needed for ERP schemas, extension columns, etc. This benefits our ODI customers: reduce the learning curve, provide shorter implementation times, and streamline access to non-IT pros that are using the product. We’ve seen examples where we’ve dramatically cut implementation times in half, and at the same time provided lower cost for managing incremental change.
- #24 Main Point: Knowledge Modules are a core architectural element of ODI that help to reduce TCO and improve flexibility and extensibility. Knowledge Modules (KMs) are the core architectural element of ODI. They drive its modularity, flexibility and extensibility. KMs define the technical implementation of integration processes defined using declarative design. They implement the actual data flows and define the templates for code generation across the multiple systems involved in each process. KMs are at the same time generic (they allow the generation of data flows regardless of the transformation rules) and highly specific (the code they generate and the integration strategy they implement are finely tuned for a given technology). A comprehensive library of KMs (100+) is provided with ODI out of the box, but they can be tailored to implement existing best practices (e.g. for highest performance, for adhering to corporate standards, for specific vertical know-how, etc.). ODI’s KM framework helps companies capture and reuse technical expertise and best practices, thus reducing cost of ownership.
- #25 Main Point: Interbank (EMEA) used ODI to reduce develop costs, improving their developer productivity by 50%. One of the challenges that Interbank had – which was a large European financial services company. They wanted to: Improve loading of data warehouse Being able to correctly profile each client based on the risk Improve the integration of banking systems, collection systems, and front office systems Modernize batch exchanges and EDI interfaces toward real time By using ODI they were able to solve these challenges easily – with the optimized ELT and declarative set-based design environment they were able to improve productivity by 50% - additionally increased their market share and improved the performance of loading and transforming their data.
- #26 Main Point – Optimizing performance is of utmost concern especially for meeting the growing demands of your business. This translates into both performance of high volume loading and real-time feeds. We’ll discuss both individually through ODI.
- #27 Main Point: E-LT is the right type of architecture that translates to some compelling operational efficiency benefits that you can capitalize on This diagram illustrates how E-LT actually works, and is centerpiece of the ODI-Enterprise Edition architecture. It optimizes how transformation is done on the target source which exploits database optimizers as opposed to transformation that is performed in-flight or requiring a separate intermediary. Some of the benefits of ELT: • Eliminates Extra Network Hop • No dependency on an intermediary server - ODI Agent runs directly on the source or target. • Set-based Transformations, utilizing the optimizations of the relational database. Not a row-based approach • NET RESULTS to your business? Data-centric Apps Run Faster and Simpler than Before Ultimately, the ELT architecture approach pays dividends as well to users of your data warehouse for improved responsiveness. Smaller batch windows means more uptime immediately.
- #28 Main Point: Oracle Data Integrator provides built in Change Data Capture to keep data in sync – which helps trickle data in at faster performance to operate your data warehouse in near real-time. A simple example of CDC in action follows. Two separate datasources for a web storefront (one for customer data, one for order data) are consolidated into a single data warehouse. To simply update the order details in real-time, only the delta (or set of orders and new customer info) needs to be propagated across to the data warehouse. This does not require moving all the data for both systems. Without CDC, business managers would not be able to see daily trends. In addition, business managers would be forced to wait for the next batch of data to load into the data warehouse before they could look at the results. By then it might be too late to make important informed decisions. In summary, benefits of this type of approach, improved performance for extraction. Timeliness of data and your data warehouse how operates in near-real-time without paying performance Oracle Data Integrator includes built-in CDC to provide this type of near-real time operation combined with E-LT architecture, can dramatically improve the performance and speed of your data movement.
- #29 Main Point: According to Overstock: “Having access to key business metrics in real-time is no longer a fantasy. In short, Oracle Data Integrator give us the ability to make better decisions and better manage our bottom line.” Overstock (North America) wanted to: enable sales, finance, marketing and merchandising teams to have access to near real-time data so that they could make timely, more intelligent business decisions. This was important for them getting a leg up on many of their competitors in the online retail shopping business as well as being more competitive on price to meet the needs of Overstock’s cost conscious customers Wanted to know at any point in time if company performance is meeting the target metrics. Needed a data integration product that could handle our high-volume loading and transformation requirements in near real time. The demands of Overstock’s millions of customers. By using ODI they were able to meet these challenges head on and revolutionize their data warehouse. Even with terabytes of data and millions of daily transactions they were able constantly depend on ODI for being at the foundation of their real-time data warehouse.
- #30 Main Point: Data integration technologies do in fact lead to powerful business benefits when they are applied to best approaches for unifying information. The Benefits of Unified Information Management to the Business? Let’s go through these one by one…
- #31 Main Point: Connect SOA + Data Integration for improved flexibility of business processes helps to improve flexibility and ultimately the agility of the business. It’s especially in these pressing times to be more dynamic What does it mean to be flexible and efficient for adapting to business conditions - meaning provide rules for loading, cleansing or acting on your data rather than hardwiring it in piecemeal. Some of the things we’ll be looking at are new ways to consider going beyond ETL to achieve this.
- #32 Main Point: ODI is optimized for supporting Bulk Data Services which ultimately provide ways to loosely couple data from underlying business apps. These ‘bulk data services’ can include data access, bulk data loads, trickle feed data services. Companies initiating projects that require data must make choices. Should they continue to connect data in the same customized, rigid, point-to-point way that they use for applications? Or should they apply reusable principles to data integration—turning data into a service that is available as logical modules, each with a standards-based interface. This allows users to access and use the service more easily, improves data visibility, and promotes greater reuse. Data reuse and flexibility is often one of the key architectural requirements for many large enterprisewide data-centric architectures. We’re seeing more and more of these forms of data services being utilized from bulk data services, data access services, data federation and data quality services. And because Oracle is on the forefront of both SOA and Data Integration, we’re uniquely poised to provide our customers integrated data services solutions using Oracle Data Integration Suite which come bundled with components for the industry leading Oracle Service Bus and Oracle BPEL Process Manager. The main best practice for implementation: Decouple services, re-combine and then orchestrate them as part of a process orchestration. These are typically the best practices for SOA and can be used to provide benefits in efficiencies of transformation, data loading and thereby providing the heavy lifting off-loading process engines to do more of the orchestration/coordination and less data handling.
- #33 Main Point: We can learn from companies like Ross that use techniques of SOA together with Oracle Data Integration to capitalize on these new improvements in performance, and benefits of re-use. Ross (North America) had the business challenge of trying to reconcile inventory information across their multiple retail chains which included more than simply the Ross brand and included other acquisitions they had made. Their challenge (both Business and IT) Hard to consolidate consistent, accurate inventory information across over 800 multiple store chains. Bad inventory data was causing missed orders, bad orders, and these were costing the stores huge amounts of money in wasted time correcting orders, or when the orders/inventory weren’t being fulfilled fast enough, this was also costing the company money. Lack of visibility to errors to the business Higher cost and time to delivery of new value Current state was complex, didn’t scale, and difficult to manage Non-standard approach required coding paradigm, inconsistent error handling With Oracle SOA Suite and ODI together: 75% Bulk data transfer performance improvement Closed loop processing using BAM and Data Integration combined eliminated ordering/replenishing inventory errors by reducing inaccurate data Seamless data integration from heterogeneous sources leveraging SOA helped to create a more flexible, standards Together with Oracle SOA Suite (Oracle BPEL PM) provided business optimization, process visibility, exception handling
- #34 Main Point: Improve data quality, invoke data cleansing rules, data profiling helps to provide improved risk reduction and compliance. How much of the data in your organization to you trust so completely that you would be willing to bet your customer relationships on? Some common challenges that companies may consider when evaluating Data Integration? Reduce the risk of missed orders, poor customer interactions, missed opportunities through data cleansing and profiling Eliminate data duplication through a single view of data Ensure the governance for all processes, data, and information assets for security policies, regulation mandates, and audit requirements
- #35 Main Point: ODI’s innovative data profiling, data quality options provide a way to better assess the quality of your data, monitor historic information, and ensure that it remains accurate, consistent and reliable within the integration lifecycle. Ultimately, this clean data is essential to provide business users trusted information for improved decision making and opportunity discovery. Specifically, using Oracle Data Quality and Oracle Data Profiling together with ODI, companies can cleanse, standardize, enrich, and de-duplicate name and addresses as well as other business data. Oracle Data Quality includes comprehensive built-in data quality rules that can be customized for creating automated and re-usable data quality services. In addition, these Data Quality services can integrate with Oracle BPEL Process Manager for managing exceptions through a human workflow –affectionately known to us as an error hospital. Data errors go in – and clean data comes out. By managing these steps as an automated business process, companies are able to eliminate risk and at the same time cut down on costly manual processes associated with handling these errors. Ultimately this provides compliance, improved business visibility and better trust of your data.
- #36 Main Point: By implementing Data Quality – this leading online service provider saved over 2M and received the unexpected benefit of improving sales, and cutting additional IT costs from cleansing of bad data. This company (Monster.com) is an online service provider for helping find employment – but even before their business picked up recently, they had the problems many large companies from dealing with so many acquisitions: Integrate data across multiple systems, including legacy acquisition infrastructure. Better profile and detect issues with data among multiple systems Remove duplicates and eliminate errors associated with better identifying customer data When they implemented ODI, the result was a cost savings of over 2M in one month simply by implementing DQ solution with ODI. Increasing sales and revenue and lower IT costs from the eliminating bad customer data.
- #37 Main point: End-to-end data lineage - ensure pervasive reach of BI/EPM applications to multiple source systems helps improve business insight. Data Integration is a key pre-requisite for BI/EPM solutions to be effective both in terms of performance, TCO and accessibility. Common requirements for BI/EPM where Data Integration is a factor: Speed the timeliness of data for real-time decision making Improve the quality of data critical to make trusted business decisions Enable pervasive access of information through data delivery services.
- #38 Main Point: ODI can help to improve business visibility letting the BI solutions do their job faster, better and at a lower cost to the business. One of the important considerations when looking for improved visibility about information is ensuring that you know where the data originates. Using ODI together with Oracle BI EE and Essbase, companies can implement metadata driven data lineage. This means that you can essentially get information about where the sources are that impact your sources. We call that “report-to-source”. It’s one of the important features that ODI brings to the table. By implementing an end-to-end Oracle solution, companies can enjoy the benefits of this type of confidence and trust in their data.
- #39 Main Point: Cabot Microelectronics (In North America) uses Oracle Data Integrator together with Hyperion planning to improve the speed of loading their multiple source systems. By automating the data integration process and eliminating manual steps they save countless man hours and resources as well as lowered the risk and improved their confidence for decision making. These types of stories are common where using ODI together with a BI solution such as Hyperion Planning, Oracle BI EE, Essbase, really provides our customers improved business insight because they have improved confidence in how their data is integrated.
- #40 Oracle’s Unique Design Advantage: Complete Integrated Best of Breed Hot Pluggable Impact to IT: Double the speed of bulk data loads and data migrations while reducing the hardware footprint Shorten implementation times from months to weeks using pre-packaged integrations to well-known applications, sources and targets. Reduce development costs by 30% using declarative tooling Impact to the Business Reduce the risk of missed orders, poor customer interactions, missed opportunities through data cleansing and profiling Speed the timeliness of data for real-time decision making Improve the quality of data critical to make trusted business decisions
- #41 Script: Get Started with Data Integration: Unify enterprise information for improving information agility Leverage E-LT and Change Data Capture for real-time BI/DW Align business demands and IT requirements for data initiatives Assess the existing and future needs of data across your enterprise Leverage Oracle Partners and Oracle Professional Services
- #43 Script: J. Crew embarked on a Data Integration initiative that was designed to better integrate cataloging information for their online systems as well as their over 200 retail outlets. The importance of consistent data in both sides of the business was of prime importance: To do this they needed to 1) define a corporate standard for how they managed data integration, and 2) figure out how to replace all of their legacy, antiquated systems as well as support the existing more modern ERP, CRM and other packaged applications. The solution: Oracle Data Integrator How they benefited? Simplified the end-to-end data integration with all their core IT systems and database platforms to reduce the total cost of ownership of their order replenishment and cataloging Benefited from the Oracle advantage in batch processing, bulk data transformation for increased performance Was able to successfully leverage SAP and Teradata integration to phase out legacy systems and ultimately reduce costs Takeaway: This example is especially common where the need to integrate data across complex IT systems, multiple divisions. Simplifying how they integrated data wasn’t as simple as simply moving all the data to a single format. It is more complex than that and they needed a tool to manage how data was integrated.
- #44 Script: Ross is a great example to close on the earlier retail example we saw previously. Ross had the business challenge of trying to reconcile inventory information across their multiple retail chains which included more than simply the Ross brand and included other acquisitions they had made. Their challenge (both Business and IT) Hard to consolidate consistent, accurate inventory information across over 800 multiple store chains Lack of visibility to errors to the business Higher cost and time to delivery of new value Current state was complex, didn’t scale, and difficult to manage Non-standard approach required coding paradigm, inconsistent error handling With Oracle SOA Suite and ODI together: Seamless data integration from heterogeneous sources leveraging SOA helped to create a more flexible, standards based integration approach 73% Bulk data transfer performance improvement Together with Oracle SOA Suite (Oracle BPEL PM) provided business optimization, process visibility, exception handling Closed loop processing using BAM and Data Integration combined eliminated ordering/replenishing inventory errors by reducing inaccurate data Takeaway: We can learn from companies like Ross that use techniques of SOA together with Data Integration to capitalize on this new-found agility.
- #45 We’ve got you covered with Online materials, blogs, podcasts, papers, and much much more. Click “Data Integration” from the main Oracle.com page to get much much more.
- #46 The following demo focuses how to improve productivity for designing and building integrations to a BI/DW using Oracle Data Integrator Enterprise Edition (ODI-EE). Specifically this demo focuses on a couple of key areas: ODI’s declarative design environment, how to setup change data capture for real-time synchronization, how to load information into Oracle BI EE, and finally how to implement data lineage to improve confidence in where your information originates.