NEOOUG 2010 Oracle Data Integrator Presentation


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Oracle Data Integrator is the Oracle's new ETL tool, originally built by Sunopsis and now part of Oracle Fusion Middleware.
  • ODI supports ELT architecture Poor Performance due to the fact that it performs a row by row operation
  • Servers, schemas, filesystems, and agents. Information about the Infrastructure is stored in the Master Repository using this module.
  • NEOOUG 2010 Oracle Data Integrator Presentation

    1. 1. Oracle Data Integrator NEOOUG May 21, 2010 Ankit Jain
    2. 2. Agenda ( Or how long do I have to stay here ) <ul><li>What is ODI ? </li></ul><ul><li>Why do I need ODI ? </li></ul><ul><li>Architecture (ELT and ODI) </li></ul><ul><li>ODI Modules </li></ul><ul><li>Peek into Knowledge Modules </li></ul><ul><li>Wrap Up </li></ul>
    3. 3. What is ODI ? <ul><li>Java based Integration tool </li></ul><ul><li>Metadata driven </li></ul><ul><li>Support for heterogeneous technologies </li></ul><ul><li>Support for standard tools </li></ul><ul><li>Standard Data Integration functions </li></ul>
    4. 4. Why do I need ODI ? <ul><li>I want a Unified view of Information </li></ul><ul><li>I want to focus on the Business Logic </li></ul><ul><li>I want to implement processes in a standard way </li></ul>Data Sources CRM Legacy ERP Custom Database Files / XML ESB Targets Custom DB Apps Data Warehouse Files / XML ESB Staging Area Any DB
    5. 5. E-LT Architecture <ul><li>Conventional: Separate ETL Server </li></ul><ul><ul><li>Proprietary ETL Engine </li></ul></ul><ul><ul><li>Poor Performance </li></ul></ul><ul><ul><li>High Costs for Separate Standalone Server </li></ul></ul><ul><li>Oracle: No New Servers </li></ul><ul><ul><li>Lower Cost: Leverage Compute Resources & Partition Workload efficiently </li></ul></ul><ul><ul><li>Efficient: Exploits Database Optimizer </li></ul></ul><ul><ul><li>Fast: Exploits Native Bulk Load & Other Database Interfaces </li></ul></ul><ul><ul><li>Scalable: Scales as you add Processors to Source or Target </li></ul></ul><ul><li>Benefits </li></ul><ul><ul><li>Optimal Performance & Scalability </li></ul></ul><ul><ul><li>Better Hardware Leverage </li></ul></ul><ul><ul><li>Easier to Manage & Lower Cost </li></ul></ul>Conventional ETL Architecture E xtract L oad T ransform ODI ELT Architecture “ E-LT ” L oad E xtract Transform T ransform
    6. 6. ODI Architecture Repository
    7. 7. ODI – 4 Modules
    8. 8. Topology/Security Manager <ul><li>Define the physical and logical architecture of the infrastructure </li></ul><ul><li>Specify user credentials and their access rights </li></ul><ul><li>All the above information is stored in the master repository </li></ul>
    9. 9. Topology (Continued)
    10. 10. Topology (Physical/Logical Architecture) <ul><li>The logical view describes logical schemas that represent the physical schemas of the existing applications independently of their physical implementation. These logical schemas are then linked to the physical resources through contexts. </li></ul><ul><li>Designers always refer to the logical view defined in the Topology. All development done therefore becomes independent of the physical location of the resources. </li></ul>
    11. 11. Physical vs. Logical Architecture Contexts Boston Japan Physical Architecture Logical Architecture SALES DB_BOSTON SALES DB_TOKYO LG_SALES Visible Invisible Plumbing
    12. 12. Designer (Models) <ul><li>Define the blueprint of the source and target datastores </li></ul><ul><li>Apply constraints on the columns (PK, FK or any condition) </li></ul><ul><li>Define SCD behavior of the columns </li></ul>
    13. 13. Designer (Projects)
    14. 14. Operator
    15. 15. Extensible Knowledge Modules <ul><li>Provided out of the box and are user extensible </li></ul><ul><li>Written for specific source/target platforms </li></ul>
    16. 16. KMs -What’s below the surface ? <ul><li>update <%=snpRef.getTable(&quot;L&quot;, &quot;INT_NAME&quot;, &quot;A&quot;)%> </li></ul><ul><li>set IND_UPDATE = 'N' </li></ul><ul><li>where exists ( </li></ul><ul><li>select 'X' </li></ul><ul><li>from <%=snpRef.getTable(&quot;L&quot;, &quot;TARG_NAME&quot;, &quot;A&quot;)%> <%=snpRef.getInfo(&quot;DEST_TAB_ALIAS_WORD&quot;)%> T </li></ul><ul><li>where <%=snpRef.getColList(&quot;&quot;, snpRef.getTable(&quot;L&quot;, &quot;INT_NAME&quot;, &quot;A&quot;)+&quot;.[COL_NAME] = T.[COL_NAME]&quot;, &quot; and &quot;, &quot;&quot;, &quot;UK&quot;)%> </li></ul><ul><li><%=snpRef.getColList(&quot;and &quot;, &quot;((&quot;+snpRef.getTable(&quot;L&quot;, &quot;INT_NAME&quot;, &quot;A&quot;)+&quot;.[COL_NAME] = T.[COL_NAME]) or (&quot;+snpRef.getTable(&quot;L&quot;, &quot;INT_NAME&quot;, &quot;A&quot;)+&quot;.[COL_NAME] IS NULL and T.[COL_NAME] IS NULL))&quot;, &quot; and &quot;, &quot;&quot;, &quot;((UPD and !TRG) and !UK)&quot;)%> </li></ul><ul><li>) </li></ul>update I$_TRG_CITY set IND_UPDATE = 'N' where exists ( select 'X' from TRG_CITY T where I$_TRG_CITY.CITY_ID = T.CITY_ID and ((I$_TRG_CITY.REGION_ID = T.REGION_ID) or (I$_TRG_CITY.REGION_ID IS NULL and T.REGION_ID IS NULL)) and ((I$_TRG_CITY.CITY = T.CITY) or (I$_TRG_CITY.CITY IS NULL and T.CITY IS NULL)) and ((I$_TRG_CITY.POPULATION = T.POPULATION) or (I$_TRG_CITY.POPULATION IS NULL and T.POPULATION IS NULL)) ) Built at runtime
    17. 17. More Features <ul><li>Data Quality </li></ul><ul><li>Change Data Capture </li></ul><ul><li>Slowly Changing Dimensions </li></ul><ul><li>Bulk Load </li></ul><ul><li>Support for Functions, procedures, etc </li></ul><ul><li>Data Lineage </li></ul>
    18. 18. Best Practices <ul><li>Define conventions early in the game </li></ul><ul><li>Define Security policy </li></ul><ul><li>Repository IDs should be unique </li></ul><ul><li>Create different contexts for different OS </li></ul><ul><li>Do NOT add business logic to the KM </li></ul><ul><li>Documentation </li></ul><ul><li>Documentation </li></ul><ul><li>Documentation </li></ul><ul><li>Documentation </li></ul>
    19. 19. ODI – Getting Started <ul><li>Where to download </li></ul><ul><ul><li> </li></ul></ul><ul><li>What to install </li></ul><ul><ul><li>Current version </li></ul></ul><ul><ul><li>You can get a patch to upgrade to </li></ul></ul>
    20. 20. OTN – Data Integration Forum
    21. 21. Q & A <ul><li>Questions => Follow up via: </li></ul><ul><ul><li> </li></ul></ul><ul><li>Contact Info : </li></ul><ul><ul><li>[email_address] </li></ul></ul>