Oracle Golden Gate<br />Real-time Data Integration<br />by Tomasz Mikołajczyk<br />GridwiseTech<br />
Real Time Data Integration<br />Oracle GoldenGate architecutre<br />Usage examples<br />Oracle GoldenGate vs others<br />R...
Real-Time Data Integration (RTDI)What is it?<br />
Oracle GoldenGate<br />Introduction<br />
OracleGoldenGateHowitworks?<br />MAPPING<br />map sch1.t1, target rep.r1;<br />map sch2.*, target test.*;<br />SELECTIVE/F...
Oracle GoldenGate - How it works?Phase 1: Extract and data pump<br />Extract<br />Example parameters<br />TABLE SCH1.*;<br...
DDL/DML replication
Full/Selective
Initial data load
Running queries and procedures</li></li></ul><li>Oracle GoldenGate - How it works? Phase 2: Transform, map and deliver<br ...
Delayed apply
Object mapping and manipulation
Userexits
Error handling</li></li></ul><li>Oracle GoldenGateTopologies<br />
Real-time reportingOffload reporting to a low cost reporting database<br />Operational Reporting<br />OLTP<br />Applicatio...
 ETL
 ODI</li></ul>Production<br />Database<br />Reporting<br />Database<br />
Consolidation/AggregationIntegration of heterogeneous environments<br />Organization<br />Units<br />Central<br />system<b...
 ETL
 ODI
 ERP</li></ul>Data <br />Warehouse<br />CDC<br />Delivery<br />SQL<br />Server<br />
Load balancing and HADistribute load into two or more live standby databases.<br />Active-Active <br />configuration<br />...
Zero downtimemigrationUpgrade/migratedatabase and applicationwithminimaldowntime<br />Zero downtime DB migration<br />Mini...
Real-time testingDistribute fresh data to testing environments<br />Real-time testing environment<br />Test <br />DB 1<br ...
 Many destinations
 Selection</li></ul>Delivery<br />Test <br />DB 2<br />Capture<br />LAN/WAN<br />TCP/IP<br />Delivery<br />Production<br /...
Oracle GoldenGateData replication<br />Source/Target<br />Trail files<br /><ul><li>DB2 (for i, Linux, Unix, z/OS)
Oracle DB, Oracle TimesTen
SQL Server
MySQL
SQL/MX
Upcoming SlideShare
Loading in …5
×

Tomasz Mikołajczyk - Oracle Golden Gate Real Time Data integration

1,808 views

Published on

18.05.2011 r.

Published in: Technology

Tomasz Mikołajczyk - Oracle Golden Gate Real Time Data integration

  1. 1. Oracle Golden Gate<br />Real-time Data Integration<br />by Tomasz Mikołajczyk<br />GridwiseTech<br />
  2. 2. Real Time Data Integration<br />Oracle GoldenGate architecutre<br />Usage examples<br />Oracle GoldenGate vs others<br />Retail case study<br />Agenda<br />Presentation plan<br />Presentation duration<br /> 35minutes<br />
  3. 3. Real-Time Data Integration (RTDI)What is it?<br />
  4. 4. Oracle GoldenGate<br />Introduction<br />
  5. 5. OracleGoldenGateHowitworks?<br />MAPPING<br />map sch1.t1, target rep.r1;<br />map sch2.*, target test.*;<br />SELECTIVE/FULL<br />table sch1.t1;<br />table sch2.*;<br />PROCESSING<br />SQLEXEC – procedures/queries<br />USER EXITS – C functions<br />AMOUNT_TOTAL = @IF (AMT >= 0, AMT * TOT, 0)<br />Oracle Environment<br />
  6. 6. Oracle GoldenGate - How it works?Phase 1: Extract and data pump<br />Extract<br />Example parameters<br />TABLE SCH1.*;<br />TABLE SCH2.T*1;<br />SPECIALRUN<br />DDL INCLUDEOBJTYPE ‘TRIGGER’;<br />DDL EXCLUDEOPTYPE DROP, OBJNAME ‘SCH1.*’;<br />SQLEXEC ”exec prepare_report ()” EVERY 4 HOURS<br /><ul><li>Many source types
  7. 7. DDL/DML replication
  8. 8. Full/Selective
  9. 9. Initial data load
  10. 10. Running queries and procedures</li></li></ul><li>Oracle GoldenGate - How it works? Phase 2: Transform, map and deliver<br />Example parameters<br />MAP SCH1.*, TARGET SCH2.*;<br />MAP SCH1.T1, TARGET SCH2.REP1, DEF T, TARGETDEF REPS;<br />DEFERAPPLYINTERVAL 10HOURS<br />REPERROR 942 DISCARD INCLUDE OPTYPE DROP<br />DDLSUBST ‘/opt/ora1/’ with ‘/opt/ora2/’<br />CUSEREXIT chupdate.dllAddVals, & INCLUDEUPDATEBEFORES, & PARAMS ”init.properties" <br />Replicat<br /><ul><li>Many destination types
  11. 11. Delayed apply
  12. 12. Object mapping and manipulation
  13. 13. Userexits
  14. 14. Error handling</li></li></ul><li>Oracle GoldenGateTopologies<br />
  15. 15. Real-time reportingOffload reporting to a low cost reporting database<br />Operational Reporting<br />OLTP<br />Applications<br />Report<br />generation<br />Real-time reporting<br />Manipulation<br />Mapping<br />CDC<br />LAN/WAN<br />TCP/IP<br /><ul><li> OLAP
  16. 16. ETL
  17. 17. ODI</li></ul>Production<br />Database<br />Reporting<br />Database<br />
  18. 18. Consolidation/AggregationIntegration of heterogeneous environments<br />Organization<br />Units<br />Central<br />system<br />Filtering<br />Transformation<br />Mapping<br />Transformation<br />Business<br />Consolidation<br />DB2<br />Aggregation<br />Universal format<br />CDC<br />Report<br />generation<br />Oracle<br />LAN/WAN<br />TCP/IP<br /><ul><li> OLAP
  19. 19. ETL
  20. 20. ODI
  21. 21. ERP</li></ul>Data <br />Warehouse<br />CDC<br />Delivery<br />SQL<br />Server<br />
  22. 22. Load balancing and HADistribute load into two or more live standby databases.<br />Active-Active <br />configuration<br />Application<br />Application<br />LAN/WAN<br />TCP/IP<br />Capture<br />Production<br />Database<br />Delivery<br />Production<br />Database<br />Delivery<br />Capture<br />
  23. 23. Zero downtimemigrationUpgrade/migratedatabase and applicationwithminimaldowntime<br />Zero downtime DB migration<br />Minimum downtimeapplicationmigration<br />SWITCHOVER<br />ERP<br />System<br />SynchronizeDBs<br />CDC<br />LAN/WAN<br />TCP/IP<br />Oracle<br />8i/9i/10g<br />Oracle<br />11g<br />Compare and verify<br />
  24. 24. Real-time testingDistribute fresh data to testing environments<br />Real-time testing environment<br />Test <br />DB 1<br />Business<br />Applications<br />Delivery<br />Data distribution<br /><ul><li> One source
  25. 25. Many destinations
  26. 26. Selection</li></ul>Delivery<br />Test <br />DB 2<br />Capture<br />LAN/WAN<br />TCP/IP<br />Delivery<br />Production<br />Database<br />…<br />
  27. 27. Oracle GoldenGateData replication<br />Source/Target<br />Trail files<br /><ul><li>DB2 (for i, Linux, Unix, z/OS)
  28. 28. Oracle DB, Oracle TimesTen
  29. 29. SQL Server
  30. 30. MySQL
  31. 31. SQL/MX
  32. 32. Sybase, Teradata and generic ODBC
  33. 33. Native GoldenGate format
  34. 34. Heterogeneousity
  35. 35. Separation of data extraction and delivery
  36. 36. Accuracy and fault tolerance
  37. 37. Flexibility</li></ul>DDL<br />DML<br /><ul><li>Log based replication
  38. 38. Archivelog mode
  39. 39. Operations and objects filtering
  40. 40. Transforming operations
  41. 41. Supplemental logging
  42. 42. User exits, SQLEXEC
  43. 43. Trigger based capture
  44. 44. Requires special objects in database
  45. 45. Logging of all operations in database
  46. 46. Supplemental logging
  47. 47. Recycle bin must be off</li></li></ul><li>Active Data Guardvs GoldenGate<br />
  48. 48. Retail Case StudyHA/DR use case<br />Facts<br /><ul><li>200 users
  49. 49. 6 companies
  50. 50. ERP system working 24/7
  51. 51. 40.53 transactions/s
  52. 52. One hour of data loss
  53. 53. 45 minutes unavailability
  54. 54. 5 hours of downtime</li></ul>Challenge<br />At maximum 15 minutes of downtime<br />No data loss in case of failure<br />Higher data security<br />
  55. 55. RetailCaseStudyBeforeimage<br />Users<br />Office and mobile.<br />Components<br />Databases<br />Web services<br />Intermediary systems<br />OLAP cubes<br />FREQUENCY<br />Backup – eachday<br />Archivelogs – eachhour<br />
  56. 56. RetailCaseStudyOracle GoldenGate<br />TWO INSTANCES<br />4 hoursdelayed<br />Real-time<br />REAL-TIME REPLICATION<br />Change Data Capture<br />Objectfiltering<br />TCP/IP Encryption<br />
  57. 57. Retail Case StudyPlanned downtimeScenario (Switchover)<br />Benefits<br /><ul><li>Switchoverinminutes
  58. 58. Coninuoususeractivity
  59. 59. Scenarios and tools for quickswitchover</li></ul>CAPTURE AND STORE<br />Capturealltransactions<br />Recordinlocaltrail<br />
  60. 60. RetailCase StudyPlanned downtimeScenario (Switchback)<br />Benefits<br /><ul><li> System canoperateduringplanned downtime
  61. 61. No operationislost</li></ul>SWITCH USERS<br />STOP USER ACTIVITY<br />DELIVER AND SYNC<br />Holduseractivity<br />Deliveraccumulated operations<br />Synchronizeproductiondatabase<br />Connectusers back to production<br />
  62. 62. RetailCase Study Unplanned downtimeScenario (Failover)<br />Benefits<br /><ul><li> No data lossincase of hardware failure
  63. 63. Minimum downtimeduringfailure</li></ul>Switchusers<br />CAPTURE AND STORE<br />Capturealltransactions<br />Recordinlocaltrail<br />
  64. 64. Questions?<br />
  65. 65. sebastian.czechowski@gridwisetech.com<br />
  66. 66. Application development and testingTelecommunication Provider Case Study<br />Facts<br />4 critical systems on 15 production databases<br />Each developer uses one copy of database<br />7 hour downtime due software upgrades<br />Challenge<br />New software version release in few minutes <br />Testing on living organism<br />Latest data supplied to developers<br />upgrades<br />Application<br />with near zero<br />are performed<br />downtime.<br />Make changes, test,<br />switch users <br />and<br />sync.<br />
  67. 67. Application development and testingTelecommunication Provider Case Study<br />Facts<br />Application development process<br />Preparing for release<br />
  68. 68. Application development and testingTelecommunication Provider Case Study<br />Synchronization<br /><ul><li>Transform old structures to newones
  69. 69. Transformqueries
  70. 70. Map old objects to newones
  71. 71. Convert data typeswherenecessary
  72. 72. Filterspecific data</li></ul>Switchover<br />PREPARE FOR SWITCH<br />Forwarduseractivity to testing<br />Captureallrecords and switch<br />SWITCH USERS AND START UPGRADE OF PRODUCTION<br />Upgradeproduction<br /><ul><li>Switchusers to testing environment
  73. 73. Capture data intrails
  74. 74. Upgradeproduction systems
  75. 75. Afterupgradepropagatechanges</li></li></ul><li>Real-Time ReportingElectronic Manufacturer Case Study<br />Facts<br /><ul><li>3 databasesineach of 3 factories
  76. 76. 300 thousanddisksmadeeachday
  77. 77. Production, staging and reporting
  78. 78. Monitoring of production 24/7
  79. 79. 30 minutesdelay of data feed</li></ul>Challenge<br />Reports on demandwithlatest data<br />Latest data feedinreal time<br />No intermediary system<br />
  80. 80. Real-Time Reporting Electronic Manufacturer Case Study<br />
  81. 81. Real-Time Reporting Electronic Manufacturer Case Study<br />Real-Time data feed<br />Reportsbased on latest data<br />
  82. 82. RTDItechnologies by Oracle <br />(Active) Data Guard – Whatisit?<br />
  83. 83. RTDI technologies by Oracle (Active) DataGuard – Howitworks?<br />SYNC/ASYNC<br />TWO APPLY MODES<br />REDO/SQL APPLY<br />DATA GUARD BROKER<br />3 MAXIMUM MODES<br />ONE CLICK SWITCH/FAILOVER<br />EM and CLI<br />

×