• Save
Tomasz Mikołajczyk - Oracle Golden Gate Real Time Data integration
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Tomasz Mikołajczyk - Oracle Golden Gate Real Time Data integration

on

  • 1,771 views

18.05.2011 r.

18.05.2011 r.

Statistics

Views

Total Views
1,771
Views on SlideShare
1,771
Embed Views
0

Actions

Likes
2
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Tomasz Mikołajczyk - Oracle Golden Gate Real Time Data integration Presentation Transcript

  • 1. Oracle Golden Gate
    Real-time Data Integration
    by Tomasz Mikołajczyk
    GridwiseTech
  • 2. Real Time Data Integration
    Oracle GoldenGate architecutre
    Usage examples
    Oracle GoldenGate vs others
    Retail case study
    Agenda
    Presentation plan
    Presentation duration
    35minutes
  • 3. Real-Time Data Integration (RTDI)What is it?
  • 4. Oracle GoldenGate
    Introduction
  • 5. OracleGoldenGateHowitworks?
    MAPPING
    map sch1.t1, target rep.r1;
    map sch2.*, target test.*;
    SELECTIVE/FULL
    table sch1.t1;
    table sch2.*;
    PROCESSING
    SQLEXEC – procedures/queries
    USER EXITS – C functions
    AMOUNT_TOTAL = @IF (AMT >= 0, AMT * TOT, 0)
    Oracle Environment
  • 6. Oracle GoldenGate - How it works?Phase 1: Extract and data pump
    Extract
    Example parameters
    TABLE SCH1.*;
    TABLE SCH2.T*1;
    SPECIALRUN
    DDL INCLUDEOBJTYPE ‘TRIGGER’;
    DDL EXCLUDEOPTYPE DROP, OBJNAME ‘SCH1.*’;
    SQLEXEC ”exec prepare_report ()” EVERY 4 HOURS
    • Many source types
    • 7. DDL/DML replication
    • 8. Full/Selective
    • 9. Initial data load
    • 10. Running queries and procedures
  • Oracle GoldenGate - How it works? Phase 2: Transform, map and deliver
    Example parameters
    MAP SCH1.*, TARGET SCH2.*;
    MAP SCH1.T1, TARGET SCH2.REP1, DEF T, TARGETDEF REPS;
    DEFERAPPLYINTERVAL 10HOURS
    REPERROR 942 DISCARD INCLUDE OPTYPE DROP
    DDLSUBST ‘/opt/ora1/’ with ‘/opt/ora2/’
    CUSEREXIT chupdate.dllAddVals, & INCLUDEUPDATEBEFORES, & PARAMS ”init.properties"
    Replicat
    • Many destination types
    • 11. Delayed apply
    • 12. Object mapping and manipulation
    • 13. Userexits
    • 14. Error handling
  • Oracle GoldenGateTopologies
  • 15. Real-time reportingOffload reporting to a low cost reporting database
    Operational Reporting
    OLTP
    Applications
    Report
    generation
    Real-time reporting
    Manipulation
    Mapping
    CDC
    LAN/WAN
    TCP/IP
    • OLAP
    • 16. ETL
    • 17. ODI
    Production
    Database
    Reporting
    Database
  • 18. Consolidation/AggregationIntegration of heterogeneous environments
    Organization
    Units
    Central
    system
    Filtering
    Transformation
    Mapping
    Transformation
    Business
    Consolidation
    DB2
    Aggregation
    Universal format
    CDC
    Report
    generation
    Oracle
    LAN/WAN
    TCP/IP
    • OLAP
    • 19. ETL
    • 20. ODI
    • 21. ERP
    Data
    Warehouse
    CDC
    Delivery
    SQL
    Server
  • 22. Load balancing and HADistribute load into two or more live standby databases.
    Active-Active
    configuration
    Application
    Application
    LAN/WAN
    TCP/IP
    Capture
    Production
    Database
    Delivery
    Production
    Database
    Delivery
    Capture
  • 23. Zero downtimemigrationUpgrade/migratedatabase and applicationwithminimaldowntime
    Zero downtime DB migration
    Minimum downtimeapplicationmigration
    SWITCHOVER
    ERP
    System
    SynchronizeDBs
    CDC
    LAN/WAN
    TCP/IP
    Oracle
    8i/9i/10g
    Oracle
    11g
    Compare and verify
  • 24. Real-time testingDistribute fresh data to testing environments
    Real-time testing environment
    Test
    DB 1
    Business
    Applications
    Delivery
    Data distribution
    • One source
    • 25. Many destinations
    • 26. Selection
    Delivery
    Test
    DB 2
    Capture
    LAN/WAN
    TCP/IP
    Delivery
    Production
    Database

  • 27. Oracle GoldenGateData replication
    Source/Target
    Trail files
    • DB2 (for i, Linux, Unix, z/OS)
    • 28. Oracle DB, Oracle TimesTen
    • 29. SQL Server
    • 30. MySQL
    • 31. SQL/MX
    • 32. Sybase, Teradata and generic ODBC
    • 33. Native GoldenGate format
    • 34. Heterogeneousity
    • 35. Separation of data extraction and delivery
    • 36. Accuracy and fault tolerance
    • 37. Flexibility
    DDL
    DML
    • Log based replication
    • 38. Archivelog mode
    • 39. Operations and objects filtering
    • 40. Transforming operations
    • 41. Supplemental logging
    • 42. User exits, SQLEXEC
    • 43. Trigger based capture
    • 44. Requires special objects in database
    • 45. Logging of all operations in database
    • 46. Supplemental logging
    • 47. Recycle bin must be off
  • Active Data Guardvs GoldenGate
  • 48. Retail Case StudyHA/DR use case
    Facts
    • 200 users
    • 49. 6 companies
    • 50. ERP system working 24/7
    • 51. 40.53 transactions/s
    • 52. One hour of data loss
    • 53. 45 minutes unavailability
    • 54. 5 hours of downtime
    Challenge
    At maximum 15 minutes of downtime
    No data loss in case of failure
    Higher data security
  • 55. RetailCaseStudyBeforeimage
    Users
    Office and mobile.
    Components
    Databases
    Web services
    Intermediary systems
    OLAP cubes
    FREQUENCY
    Backup – eachday
    Archivelogs – eachhour
  • 56. RetailCaseStudyOracle GoldenGate
    TWO INSTANCES
    4 hoursdelayed
    Real-time
    REAL-TIME REPLICATION
    Change Data Capture
    Objectfiltering
    TCP/IP Encryption
  • 57. Retail Case StudyPlanned downtimeScenario (Switchover)
    Benefits
    • Switchoverinminutes
    • 58. Coninuoususeractivity
    • 59. Scenarios and tools for quickswitchover
    CAPTURE AND STORE
    Capturealltransactions
    Recordinlocaltrail
  • 60. RetailCase StudyPlanned downtimeScenario (Switchback)
    Benefits
    • System canoperateduringplanned downtime
    • 61. No operationislost
    SWITCH USERS
    STOP USER ACTIVITY
    DELIVER AND SYNC
    Holduseractivity
    Deliveraccumulated operations
    Synchronizeproductiondatabase
    Connectusers back to production
  • 62. RetailCase Study Unplanned downtimeScenario (Failover)
    Benefits
    • No data lossincase of hardware failure
    • 63. Minimum downtimeduringfailure
    Switchusers
    CAPTURE AND STORE
    Capturealltransactions
    Recordinlocaltrail
  • 64. Questions?
  • 65. sebastian.czechowski@gridwisetech.com
  • 66. Application development and testingTelecommunication Provider Case Study
    Facts
    4 critical systems on 15 production databases
    Each developer uses one copy of database
    7 hour downtime due software upgrades
    Challenge
    New software version release in few minutes
    Testing on living organism
    Latest data supplied to developers
    upgrades
    Application
    with near zero
    are performed
    downtime.
    Make changes, test,
    switch users
    and
    sync.
  • 67. Application development and testingTelecommunication Provider Case Study
    Facts
    Application development process
    Preparing for release
  • 68. Application development and testingTelecommunication Provider Case Study
    Synchronization
    • Transform old structures to newones
    • 69. Transformqueries
    • 70. Map old objects to newones
    • 71. Convert data typeswherenecessary
    • 72. Filterspecific data
    Switchover
    PREPARE FOR SWITCH
    Forwarduseractivity to testing
    Captureallrecords and switch
    SWITCH USERS AND START UPGRADE OF PRODUCTION
    Upgradeproduction
    • Switchusers to testing environment
    • 73. Capture data intrails
    • 74. Upgradeproduction systems
    • 75. Afterupgradepropagatechanges
  • Real-Time ReportingElectronic Manufacturer Case Study
    Facts
    • 3 databasesineach of 3 factories
    • 76. 300 thousanddisksmadeeachday
    • 77. Production, staging and reporting
    • 78. Monitoring of production 24/7
    • 79. 30 minutesdelay of data feed
    Challenge
    Reports on demandwithlatest data
    Latest data feedinreal time
    No intermediary system
  • 80. Real-Time Reporting Electronic Manufacturer Case Study
  • 81. Real-Time Reporting Electronic Manufacturer Case Study
    Real-Time data feed
    Reportsbased on latest data
  • 82. RTDItechnologies by Oracle
    (Active) Data Guard – Whatisit?
  • 83. RTDI technologies by Oracle (Active) DataGuard – Howitworks?
    SYNC/ASYNC
    TWO APPLY MODES
    REDO/SQL APPLY
    DATA GUARD BROKER
    3 MAXIMUM MODES
    ONE CLICK SWITCH/FAILOVER
    EM and CLI