Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Migration Steps from OWB 2 ODI

3,459 views

Published on

It is the fact that Oracle Warehouse Builder (OWB) released the latest major version and final state. But business requirements are rapidly increasing. New applications are implemented in source systems and as a result new reports and new subject areas are needed urgently. It is needed to implement new features for growing business needs into our data warehouses. Resources are limited and conversion should be done as soon as possible.
In this presentation, see the most convenient methods to migrate from Oracle Warehouse Builder to Oracle Data Integrator with agile methodology without interrupting on going daily jobs as well as understanding of Oracle's OWB2ODI migration utility.

Published in: Data & Analytics
  • Be the first to comment

Migration Steps from OWB 2 ODI

  1. 1. http://gurcanorhan.wordpress.com @gurcan_orhan http://tr.linkedin.com/in/gurcanorhan Gürcan ORHAN Enterprise Data Warehouse Architect Ekol Logistics Corp. 16th June 2017 Migration Steps from Oracle Warehouse Builder (OWB) to Oracle Data Integrator (ODI)
  2. 2. Who Am I ? +20 years of IT experience. +14 years of DWH experience. +10 years of Oracle Data Integrator experience. +8 years of Oracle Warehouse Builder experience. Sybase Power Designer, ERwin Data Modeler, SDDM OBIEE, Cognos, Microstrategy, Business Objects, Qlikview, Tableau IBM Data Stage, SAP Data Services, Informatica, etc… Oracle Excellence Awards - Technologist of the Year 2011 : Enterprise Architect DWH & BI Chair : TROUG (Turkish Oracle User Group) Published Customer Snapshot for NODI @Oracle.com Published videos about ODI @Oracle.com (Oracle Media Network) Published OTN Podcasts about “Data Warehousing and ODI” “ODI and the Evolution of Data Integration” 3 different “2MTT”s Articles in OTech Magazine, SearchSoftwareQuality.com Annual panelist for ODTUG “Ask the Experts Panel : ODI” Presenter in OOW since 2010 (7 times in a row ⭐ ) Presenter in many OUG conferences in globe Presenter in various universities in Turkey 16 JUNE 2017 / #OGHTECH17 2
  3. 3. Ekol Germany Warehousing Solutions begin with the Kardelen Facility 1996 2003 2010 2012 2014 2016 201520132011200820021990 Acquire STS Int. Transport Ekol Bosnia Ekol France Ekol Greece Ekol Ukraine Ekol Spain Ekol Bulgaria Ekol Czech Rep. Ekol Iran Ekol PolandEkol Italy Ekol Romania Ekol HungaryAcquire Unok/Unatsan Rainbow Replaced by Quadro (software) Intermodal operations Ro-Ro operations Established Ekol Milestones
  4. 4. USA Ekol Locations Currentcountries 15 80 locations 2017 23 countries 92 locations 2018 31 countries 104 locations 2019 42 countries 122 locations CN
  5. 5. Overview
  6. 6. 16 JUNE 2017 / #OGHTECH17 My Definition of ODI 6
  7. 7. AGENDA
  8. 8. 16 JUNE 2017 / #OGHTECH17 Quote of the Day
  9. 9. 16 JUNE 2017 / #OGHTECH17 Some Facts  ODI is the strategic product for heterogeneous data integration as declared in “statement of direction”  No major releases in Oracle Warehouse Builder (latest release is 11GR2 - 11.2.0.4)  OWB will not be shipped with database 12c  No OWB documentation included in database 12c  OWB Support continues 😊  OWB is still supported by database 12c 😊 OWB 11.2.0.3 + CP2 is certified with database 12c 😊  OWB is not supported by Cloud environment
  10. 10. 16 JUNE 2017 / #OGHTECH17 Some Facts
  11. 11. 16 JUNE 2017 / #OGHTECH17 REQUIREMENTS (safe harbour) Oracle Warehouse Builder - version 11.2.0.4 (plus patch # 18537208) (plus patch # 21687102) Oracle Data Integrator - version 12.1.3.x.x or 12.2.1.x.x 2 Oracle Database instances recommendation : 11G R2 (11.2.0.4) A Linux based server - version 11.2.0.3 (plus CP3 #16568042)
  12. 12. 16 JUNE 2017 / #OGHTECH17 Build Up Laboratory Environment (safe harbour) SOURCE TARGET OWB ODI Run your OWB jobs initially before starting migration  Migration Utility is a command-line tool which runs on OWB installation directory, migrates design-time metadata.  Linux 64-Bit or Windows 64-Bit StandAlone ODI Agent (with patch 17224695 for Migration Utility) Upgrade your OWB, ODI installations to required versions
  13. 13. 16 JUNE 2017 / #OGHTECH17 When your manager asks you…
  14. 14. Data Integration on the Cloud
  15. 15. Oracle Data Integrator Cloud Services is Ready…
  16. 16. AGENDA
  17. 17. 16 JUNE 2017 / #OGHTECH17 Supported OWB Objects sourceandtargetmodules locations tables (columns, keys, indexes) views (columns,keys) sequences materialized views (columns, keys, indexes) classic mappings Code Template mappings Aggregator Constant Expression Flat File Mapping Input Parameter Sequence Set Splitter Transformatio n
  18. 18. AGENDA
  19. 19. 16 JUNE 2017 / #OGHTECH17 NOT Supported OWB Objects (Limitations) table (partitions attribute sets, data rules) dimensional modeling metadata custom PL/SQL (procedure, package, and so on) user-defined types streams CDCconfigurations process flow mappings using dimension and cube nameandaddress match-merge data rules data auditors expand configuration details (security, user extensions, transportable modules, schedules/collections, user folders) OMB*Plusscripts data profiles materialized view (partitions, attribute sets, data rules)
  20. 20. AGENDA
  21. 21. 16 JUNE 2017 / #OGHTECH17 Some Facts
  22. 22. 16 JUNE 2017 / #OGHTECH17 Prepare Migration Utility Configuration File migration.config (mandatory parameters) ODI_MASTER_USER=<user_name> ODI_MASTER_URL=<JDBC_URL> ODI_MASTER_DRIVER=<JDBC_driver_name> ODI_USERNAME=<user_name> ODI_WORK_REPOSITORY_NAME=<user_name> OWB_WORKSPACE_OWNER=<workspace_owner> OWB_URL=<URL> OWB_WORKSPACE_NAME=<workspace_name> {jdbc:oracle:thin:@localhost:1521:mydb} {oracle.jdbc.OracleDriver} {SUPERVISOR} {WORKREP1} {localhost:1521:mydb} {REP_1.WS1} {WS1}
  23. 23. 16 JUNE 2017 / #OGHTECH17 Prepare Migration Utility Configuration File migration.config (optional parameters) MIGRATION_LOG_FILE=<path_to_log_file> MIGRATION_REPORT_INCLUDE=<PASSED|FAILED|ALL˙> MIGRATION_MODE=<FAST_CHECK|DRY_RUN|RUN˙> MIGRATE_DEPENDENCIES=<TRUE|FALSE˙> STOP_ON_ERROR=<TRUE|FALSE˙> SPLIT_JOIN_FOR_ANSI_SYNTAX=<TRUE˙|FALSE> MIGRATE_UNBOUND_OPERATOR=<TRUE|FALSE˙> MIGRATION_OBJECTS=<objects {*˙}> FLUSH_BATCH_SIZE=<number_of_mappings {50˙}>
  24. 24. 16 JUNE 2017 / #OGHTECH17 Run & Review the Migration Utility ./migration.sh <odi_master_password> <odi_user_password> <owb_workspace_owner_password> ./migration.log ➙ {detailed} ./migration.report ➙ {exclusion}
  25. 25. 16 JUNE 2017 / #OGHTECH17 Exclusive Report******************************************************************************* * Oracle Warehouse Builder - Migration Utility - Summary Report * Created: 9/30/13 10:17 AM * Migration Report Style - RUN * OWB Release:11.2.0.4.0 - OWB Repository: OWB_REPO_MIG/machine.oracle.com:1521:orcl11204 - OWB Workspace: OWB_REPO_MIG.OWB_REPO_WKSP1 * ODI Release:12.1.2 - ODI Master Repository: mig12c/jdbc:oracle:thin:@machine:1521:orcl11203 - ODI User/Work Repository: SUPERVISOR/WORKREP1 * Report File: /scratch/jsmith/Migration/migration.report ****************************************************************************** Migration started at 9/30/13 10:17 AM Pacific Standard Time Statistics ----------- Total Projects Migrated: 2 ****************************************************************************** PROJECT: PUBLIC_PROJECT Object Types Migrated Not-Migrated ------------- --------- ------------ LOCATION: 1 0 ****************************************************************************** PROJECT: MY_PROJECT Object Types Migrated Not-Migrated ------------- --------- ------------ TABLE: 2 0 MAPPING_MODULE: 1 0 MODULE: 1 0 MAPPING: 1 0 Details ----------- ****************************************************************************** PROJECT: PUBLIC_PROJECT Object Types Status ------------ ------- LOCATION ORACLE_WH_LOCATION SUCCESS ****************************************************************************** PROJECT: MY_PROJECT Object Types Status ------------ ------- TABLE TAB1 SUCCESS TAB2 SUCCESS MAPPING_MODULE AA_UNBOUND_TEST SUCCESS MODULE AA_UNBOUND_TEST SUCCESS MAPPING MAP_UO_CASE2 SUCCESS MAPPING UNBOUND_LOOKUP_MAP [ERROR][Migration][MU-5011]Unable to find bound table for output attribute group OUTGRP1 in Lookup DEPT_LOOKUP. Migration ended at 9/30/13 10:17 AM Pacific Standard Time Total migration time (hh:mm:ss): 00:00:07
  26. 26. 16 JUNE 2017 / #OGHTECH17 Detailed Report******************************************************************************* * Oracle Warehouse Builder - Migration Utility - Log * Created: 9/30/13 10:17 AM • Migration Report Style - RUN * OWB Release:11.2.0.4.0 - OWB Repository: OWB_REPO_MIG/machine.oracle.com:1521:orcl11204 - OWB Workspace: OWB_REPO_MIG.OWB_REPO_WKSP1 * ODI Release:12.1.2 - ODI Master Repository: mig12c/jdbc:oracle:thin:@machine:1521:orcl11203 - ODI User/Work Repository: SUPERVISOR/WORKREP1 * Log File: /scratch/jsmith/Migration/migration.log ******************************************************************************* Migration started at 9/30/13 10:17 AM Pacific Standard Time ******************************************************************************* ----START MIGRATE LOCATION ORACLE_WH_LOCATION. ----SUCCESSFULLY MIGRATED ORACLE_WH_LOCATION. START MIGRATE PROJECT MY_PROJECT. FLUSH OdiDataServer[1] COST(MS):80 ----START MIGRATE MODULE AA_UNBOUND_TEST. FLUSH OdiLogicalSchema[1] COST(MS):16 ----SUCCESSFULLY MIGRATED AA_UNBOUND_TEST. ----START MIGRATE MAPPING_MODULE AA_UNBOUND_TEST. ------------START MIGRATE TABLE TAB2. FLUSH OdiFolder[1] COST(MS):343 ------------SUCCESSFULLY MIGRATED TAB2. ------------START MIGRATE TABLE TAB1. ------------SUCCESSFULLY MIGRATED TAB1. --------START MIGRATE MAPPING MAP_UO_CASE2. FLUSH MAPPING, MIGRATED 0 COST(MS):31 --------SUCCESSFULLY MIGRATED MAP_UO_CASE2. ----SUCCESSFULLY MIGRATED AA_UNBOUND_TEST. SUCCESSFULLY MIGRATED MY_PROJECT. ****************************************************************************** TABLE[TOTAL:2 MIGRATED:2 SKIPPED:0]. ----PASSED: PROJECT[MY_PROJECT].MODULE[AA_UNBOUND_TEST].MAPPING[MAP_UO_CASE2].OPERATOR[TAB1]. ----PASSED: PROJECT[MY_PROJECT].MODULE[AA_UNBOUND_TEST].MAPPING[MAP_UO_CASE2].OPERATOR[TAB2]. LOCATION[TOTAL:1 MIGRATED:1 SKIPPED:0]. ----PASSED: PROJECT[PUBLIC_PROJECT].LOCATION[ORACLE_WH_LOCATION]. MAPPING_MODULE[TOTAL:1 MIGRATED:1 SKIPPED:0]. ----PASSED: PROJECT[MY_PROJECT].MODULE[AA_UNBOUND_TEST]. MODULE[TOTAL:1 MIGRATED:1 SKIPPED:0]. ----PASSED: PROJECT[MY_PROJECT].MODULE[AA_UNBOUND_TEST]. PROJECT[TOTAL:1 MIGRATED:1 SKIPPED:0]. ----PASSED: PROJECT[MY_PROJECT]. MAPPING[TOTAL:1 MIGRATED:1 SKIPPED:0]. ----PASSED: PROJECT[MY_PROJECT].MODULE[AA_UNBOUND_TEST].MAPPING[MAP_UO_CASE2]. ******************************************************************************* Migration ended at 9/30/13 10:17 AM Pacific Standard Time Total migration time (hh:mm:ss): 00:00:07
  27. 27. 16 JUNE 2017 / #OGHTECH17 Special Cases During Migration 2 Operators connected to same operator
  28. 28. 16 JUNE 2017 / #OGHTECH17 Special Cases During Migration Tables with multiple Primary Keys if target table has multiple primary keys, since only one primary key is allowed in ODI, the redundant primary keys will be migrated as alternate keys.
  29. 29. 16 JUNE 2017 / #OGHTECH17 Special Cases During Migration Multiple operators connected from and to same operator
  30. 30. 16 JUNE 2017 / #OGHTECH17 Special Cases During Migration Lookup operator has a constant as input
  31. 31. 16 JUNE 2017 / #OGHTECH17 Special Cases During Migration Lookup Operators Have No Driver Table (Mapping Is Invalid)
  32. 32. 16 JUNE 2017 / #OGHTECH17 Special Cases During Migration Multiple operators connected to same operator, some with no upstream source
  33. 33. 16 JUNE 2017 / #OGHTECH17 Special Cases During Migration Multiple operators connected to same operator, all with different upstream operator
  34. 34. AGENDA
  35. 35. 16 JUNE 2017 / #OGHTECH17 Planning to Decide HOW Manual jobs after “successful” migration User Folders Packages Scheduling User Defined Data Types Model Folders
  36. 36. 16 JUNE 2017 / #OGHTECH17 Recommendations  Migrate everything and see what is migrating, what is not  Run everything without source data  Rollback everything (a fresh new ODI 12c)  Create folders, model folders in ODI 12c  Start migration per folder  Check migration logs for errors  Correct issues for mappings with error  Re-run ODI 12c with source data
  37. 37. AGENDA
  38. 38. 16 JUNE 2017 / #OGHTECH17 What If Scenarios (Migration on Production)
  39. 39. 16 JUNE 2017 / #OGHTECH17 What If All Mappings not Migrated?
  40. 40. 16 JUNE 2017 / #OGHTECH17 A CURE is Needed. The Cure is OdiStartOwbJob tool of ODI 12c Define OWB workspace in ODI 12c Topology Manager
  41. 41. 16 JUNE 2017 / #OGHTECH17 ODIStartOWBJob Parameters ˙WORKSPACE=<logical OWB repository> ˙LOCATION=<OWB location> ˙OBJECT_NAME=<OWB object name> ˙OBJECT_TYPE=<OWB object type> EXEC_PARAMS=<execution parameters> CONTEXT=<context code> LOG_LEVEL=<log level - 5˙> SYNC_MODE=<1˙|2> POLLINT=<#n - 0˙> SESSIONNAME=<session name> KEYWORDS=<keywords> <OWB Parameters>
  42. 42. 16 JUNE 2017 / #OGHTECH17 1. Call OWB Mappings from ODI Packages Create an ODI package with many ‘OdiStartOwbJob’ tool. Run everything in ODI, but call OWB mappings in right order and sequence (synchronous / asynchronous). 42
  43. 43. 16 JUNE 2017 / #OGHTECH17 Divide into Work Packages (Agile Methodology) Divide OWB mappings into Work Packages Perform Migration for WP #n Correct errors, rewrite if not migratedChange OdiStartOwbJob to ODI object.
  44. 44. 16 JUNE 2017 / #OGHTECH17 Step by Step Migration Facts
  45. 45. 16 JUNE 2017 / #OGHTECH17 http://gurcanorhan.wordpress.com @gurcan_orhan http://tr.linkedin.com/in/gurcanorhan …Final Words… 45
  46. 46. Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee (“DTTL”), its network of member firms, and their related entities. DTTL and each of its member firms are legally separate and independent entities. DTTL (also referred to as “Deloitte Global”) does not provide services to clients. Please see www.deloitte.com/about for a more detailed description of DTTL and its member firms. Deloitte provides audit, consulting, financial advisory, risk management, tax and related services to public and private clients spanning multiple industries. With a globally connected network of member firms in more than 150 countries and territories, Deloitte brings world-class capabilities and high-quality service to clients, delivering the insights they need to address their most complex business challenges. Deloitte’s more than 210,000 professionals are committed to becoming the standard of excellence. This communication contains general information only, and none of Deloitte Touche Tohmatsu Limited, its member firms, or their related entities (collectively, the “Deloitte network”) is, by means of this communication, rendering professional advice or services. No entity in the Deloitte network shall be responsible for any loss whatsoever sustained by any person who relies on this communication © 2015. For more information, contact Deloitte Touche Tohmatsu Limited.

×