<Insert Picture Here>GoldenGate Introduction谢伟 william.xie@oracle.com高级售前顾问甲骨文公司大中华区渠道及联盟部
议程/Agenda            历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景  Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特...
议程/Agenda            历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景  Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特...
前GoldenGate公司简介• 创建于1995年,总部在美国旧金山。• 数据复制领域的专业公司。• 全球超过4,000个安装许可,分布在政府、银行、制造、传  媒、电信、证券、医疗和零售等各大行业,其中有多个全  球财富500强企业。• 与O...
GoldenGate的定位• GoldenGate 可以满足用户针对实时数据的需求 • 可持续的数据高可用性和实时商务智能 • 跨越多种异构数据库和操作系统实现实时数据同步 • 对源系统是非侵入式的   对源系统是非侵 式的• GoldenGa...
技术发展路线图Rapidly increasing heterogeneous platform and                                                                      ...
议程/Agenda            历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景  Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特...
整合后的产品和解决方案GoldenGate Products              Real-time Data Integration•   Oracle GoldenGate               • Real-Time Data...
GoldenGate的实时架构Capture: 实时读取交易日志捕捉数据变化并可实现过滤.                    Trail: 暂存数据变化.                                           ...
架构中的组件(进程)Capture                                          Deliver•   Log based                                    •   Nat...
交易数据管理    Transactional Data Management    T      ti   lD t M          t           TDM提供异构环境下交易数据的实时捕捉 变GoldenGate TDM提供异构...
GoldenGate TDM软件支持的环境Databases                          O/S and PlatformsCapture:                           Windows 2000, ...
GoldenGate For Oracle        支持的Oracle Database版本                 在当前的Oracle GoldenGate 10.4版本中,支持如下Oracle Database版本     ...
GoldenGate: 关键的架构优势•   Distributed Loosely Coupled Process Based Architecture•   Heterogeneous Initial Load•   Asynchronou...
GoldenGate: 关键的架构优势•   Conflict Detection and Resolution: for dual-active environments•   Parallel Coordination (Source, T...
Oracle GoldenGate 典型应用场景 企业级的实时数据集成解决方案                                                                   • 一套产品满足多种业务需求  ...
灵活的拓扑结构 单向复制     双向复制         点对点备份、查询   灾备、双业务中心      多业务中心广播复制      集中复制        多级复制数据分发    数据仓库、N+1灾备   层次化企业数据
GG在数据传输方面保证交易的一致性• GG有很好的机制来保证传输的交易一致性和断点续传。
GoldenGate TDM解决方案高可用性与容灾            实时数据集成•   容灾与应急备份        •   数据仓库实时供给•   消除计划内停机        •   实时报表•   双业务中心          • ...
高可用性: 容灾与应急备份实现:           用途: 快速恢复和切换       在非计划性停机时保持业务连续 最小化数据损失 重新同步主备两端数据
高可用性: 减少计划内停机实现:             用途: 保障业务零或者近似零停机    保障系统/应用/数据库在升级 滚动升级            、移植和维护期间业务的可用 降低业务中断带来的损失     性
高可用性: 双业务中心(Active-Active)实现:• 负载均衡,提高系统整体性能• 连续可用,快速的容灾接管  连续可用 快速的容灾接管• 冲突检测和处理
数据仓库实时供给(和ODI结合使用)实现:             用途: 数据仓库数据实时得到更新    提升商务智能和信息分析能力 对源系统低影响         实时可用的数据仓库
实时数据整合: 查询分离实现:                  用途: 将实时报表从主系统剥离,可根据客户    实时报表 需求定制                 提高系统性能和可用性 显著提高主系统性能
议程/Agenda            历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景  Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特...
全球范围内成功案例       • Over 400 customers with 4,000+ implementations         across Fortune 500 companies: Financial Services,...
案例:中华人民共和国海关总署                     全国各地41个口岸H2000应用                             网络                                        ...
案例:国家体育总局体彩中心                             •   N+1模式容灾                             •   各省市集中备份     Parallel Coordination   ...
案例:国家税务总局             集中容灾             远距离             低带宽
议程/Agenda            历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景  Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特...
Oracle GoldenGate 组件    Communications and Startup            GGSCI                                                       ...
Oracle GoldenGate 组件    Communications and Startup            GGSCI                                                       ...
Oracle GoldenGate 组件    Communications and Startup            GGSCI                                                       ...
Oracle GoldenGate 组件    Communications and Startup            GGSCI                                                       ...
Oracle GoldenGate 组件    Communications and Startup            GGSCI                                                       ...
Oracle GoldenGate Capture                                                                                                 ...
Oracle GoldenGate Delivery                                                                                                ...
GoldenGate Capture捕捉• Log-based change data capture  – High volume  – Low overhead• Decoupled architecture  – Multiple cap...
GoldenGate Delivery传送• Decoupled architecture  – Multiple delivery processes may be used to scale  – Possible to split “ho...
GoldenGate初始化• Start change data capture on Source Database  – Current source Commit Sequence Number (CSN) is 222Current C...
GoldenGate初始化• Wait for any open transactions to close• Use a backup (or export/import) to copy source to target as of a  ...
GoldenGate初始化• Start delivery at or after given backup CSN (245 in this case)  – Current source Commit Sequence Number (CS...
GoldenGate 检查点• Capture, Pump, and Delivery save positions to a checkpoint file  so they can recover in case of failure   ...
GoldenGate 检查点• Capture, Pump, and Delivery save positions to a checkpoint file so  they can recover in case of failure   ...
GoldenGate 检查点• Capture, Pump, and Delivery save positions to a checkpoint file  so they can recover in case of failure   ...
开箱即用的转换能力• Data selection and filtering数据选择和过滤• Column mapping列映射• Functions函数功能
数据的选择和过滤•   GoldenGate provides the ability to select or filter out data based on a    variety of levels and conditions   ...
列映射•   GoldenGate provides the capability to map columns from one table to another•   Data can be transformed between diss...
Functions函数功能•   GoldenGate provides the capability to transform columns by using a set of built-    in functions (current...
议程/Agenda            历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景  Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特...
各种数据容灾方案比较常见数据                方法描述及优点/缺点容灾方法基于存储                 其核心是通过SAN进行存储之间的数据拷贝 常见方案如IBM的Fl hC                     ...
Oracle GoldenGate 数据容灾特点        Oracle GoldenGate 用于数据容灾                  O l G ld G t 架构非常灵活                   Oracle Go...
GoldenGate Vs. Quest SharePlex  • GoldenGate国内主要竞争对手,两者架构类似              GoldenGate                     SharePlex异构数据库复制  ...
GoldenGate Vs. DSG RealSync•    GoldenGate国内次要竞争对手,两者架构类似•    DSG(迪思杰)是本土公司               GoldenGate         RealSync    异...
根据解决方案来选择数据移动和转换的Oracle产品                                                                Golden       Active         Data ...
针对数据集成的Oracle产品 Comparing How They Work                               Retrieve Data                     Load Data         ...
GoldenGate and Streams    Oracle GoldenGate                Oracle Streams•   GoldenGate is Oracle’s strategic replication ...
GoldenGate & StreamsGoldenGate is Oracle’s strategic Streams continues to be areplication solution   li ti      l ti      ...
Comparing GoldenGate & Streams•   GoldenGate    – Solution for all heterogeneous real-time data access and delivery    – N...
GoldenGate and Data Guard            Heterogeneous            Distributed                                        Oracle   ...
Product Comparison      Active Data Guard and GoldenGate                                 Oracle Active Data Guard         ...
Complementary and Used Together     A ti D t Guard and G ld G t     Active Data G d d GoldenGate                    Hetero...
Oracle GoldenGate产品包     New Products Based on GoldenGate Technology      Oracle Products                       Descriptio...
议程/Agenda            历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景  Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特...
Oracle GoldenGate 实施经验分享 (检查操作系统及数据库)      Oracle GoldenGate 检查操作系统及数据库                如果操作系统是AIX 5.3,须确保 xlC/C++ 的版本是9.0...
Oracle GoldenGate 实施经验分享 (数据初始化)      方法                            优点/缺点      EXP/IMP                        这种方法的好处是,可以...
Oracle GoldenGate 实施经验分享 (性能调优)        Oracle GoldenGate 性能调优                  Extract 仅做抓取                  过滤和转换, 尽量由D...
Q&A
Upcoming SlideShare
Loading in...5
×

Oracle+golden+gate+introduction

1,949

Published on

GoldenGate introduction

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,949
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
126
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Oracle+golden+gate+introduction

  1. 1. <Insert Picture Here>GoldenGate Introduction谢伟 william.xie@oracle.com高级售前顾问甲骨文公司大中华区渠道及联盟部
  2. 2. 议程/Agenda 历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景 Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特点• GoldenGate外争和内比• GoldenGate实施经验分享
  3. 3. 议程/Agenda 历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景 Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特点• GoldenGate外争和内比• GoldenGate实施经验分享
  4. 4. 前GoldenGate公司简介• 创建于1995年,总部在美国旧金山。• 数据复制领域的专业公司。• 全球超过4,000个安装许可,分布在政府、银行、制造、传 媒、电信、证券、医疗和零售等各大行业,其中有多个全 球财富500强企业。• 与Oracle、IBM、HP、Microsoft、Sybase、Teradata等著 名企业建立了紧密合作关系。• 2009年7月被O l 收购 2009年7月被Oracle收购。
  5. 5. GoldenGate的定位• GoldenGate 可以满足用户针对实时数据的需求 • 可持续的数据高可用性和实时商务智能 • 跨越多种异构数据库和操作系统实现实时数据同步 • 对源系统是非侵入式的 对源系统是非侵 式的• GoldenGate 在实时数据市场是全球一流的产品 在实时数据市场是全球 流的产品 • 在提供关键业务高可用性方面是公认的领导者 • 财富500强企业很多用户采用Golden Gate
  6. 6. 技术发展路线图Rapidly increasing heterogeneous platform and IMS EMCapplication support and developing additional high McKessonavailability solutions Convergys Fidelity Comverse Netezza IBM Info Srv. Infosphere Ingres Ingres Sybase Sybase Cerner Cerner Microsoft Microsoft Microsoft Initial Support t a Suppo t SQLServer SQLServer SQLServer Application Eclipsys Eclipsys Eclipsys IBM DB2 IBM DB2 IBM DB2 Platform Oracle-Siebel Oracle-Siebel Oracle-Siebel In progress Amdocs Amdocs Amdocs DST DST DST DST Teradata Teradata Teradata Teradata MySQL MySQL MySQL MySQL MySQL Oracle Oracle Oracle Oracle Oracle GE Medical GE Medical GE Medical GE Medical GE Medical GE Medical IDX IDX IDX IDX IDX IDX ACI ACI ACI ACI ACI ACI ACI ACIHP Non-Stop HP Non-Stop HP Non-Stop HP Non-Stop HP Non-Stop HP Non-Stop HP Non-Stop HP Non-Stop HP Non-Stop HP Non-Stop 1995 1996 - 1998 1999 2000 - 2002 2003 2004 2005 2006 2007 2008- 2009
  7. 7. 议程/Agenda 历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景 Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特点• GoldenGate外争和内比• GoldenGate实施经验分享
  8. 8. 整合后的产品和解决方案GoldenGate Products Real-time Data Integration• Oracle GoldenGate • Real-Time Data Warehousing• Oracle GoldenGate for Mainframe p g • Live Reporting• Oracle GoldenGate Veridata• Management Pack for Oracle • Transactional Data Integration GoldenGate• Oracle GoldenGate Adapters High Availability & Disaster Tolerance • Live Standby • Zero-Downtime Operations e o o t e Ope at o s • Active-Active
  9. 9. GoldenGate的实时架构Capture: 实时读取交易日志捕捉数据变化并可实现过滤. Trail: 暂存数据变化. Route: 数据经过压缩和加密传送到目的地. Deliver:执行所需的数据变化,然后将数据变 化提交到目的库. Pump Capture Source Trail 网络 Target Trail Delivery Target Trail源数据库 Delivery Pump Source Trail Capture 目的数据库 双向复制
  10. 10. 架构中的组件(进程)Capture Deliver• Log based • Native database API• VAM based • ODBC• Local/Remote Queuing • Batched Operations• Filtering • Filtering• Parallel Coordination • Transaction/Operations Grouping• Local/Remote* • Parallel Coordination • Local/Remote*PUMP MANAGER• Local/Remote oca / e ote  Queue (Trail) Management ( a ) a age e t• Filtering  Process Management (Dynamic)• TCP/IP  Monitoring and Administration (Lag reports)• Compression  Reporting (Events, Errors, Thresholds)• Encryption E ti  Receive/Route requests f R i /R t t from UI On-disk Components  Trail Fil T il Files  Configuration Files C fi ti Fil  Checkpoint Files  Output Files - Capture, Pump, Delivery  Report/Log Files  Data Definition Files (SOURCEDEFS)  Binaries - Utility to generate definitions
  11. 11. 交易数据管理 Transactional Data Management T ti lD t M t TDM提供异构环境下交易数据的实时捕捉 变GoldenGate TDM提供异构环境下交易数据的实时捕捉、变换、投递,主要是支持异构数据库环境下的变化数据同步。TDM 具有 : 特性: 实时性  高性能 秒一级延迟 能够以低资源消耗完成每秒数千交易的复制 异构环境支持  可扩展 在不同平台和数据库环境下复制数据 开放的结构使客户适应各种异构数据平台 以事务为单位复制  可靠性 维护交易一致性 保证数据的连续可用
  12. 12. GoldenGate TDM软件支持的环境Databases O/S and PlatformsCapture: Windows 2000, 2003, XP Oracle Linux DB2 UDB Sun Solaris Microsoft SQL Server Sybase ASE HP NonStop Teradata HP-UX Ingres HP TRU64 Enscribe SQL/MP IBM AIX SQL/MX IBM z/OSDelivery: All listed above MySQL, HP Neoview, Netezza And ODBC compatible databases ETL products JMS message queues MySQL TimesTen Ti T
  13. 13. GoldenGate For Oracle 支持的Oracle Database版本 在当前的Oracle GoldenGate 10.4版本中,支持如下Oracle Database版本 在当前的O l G ld G t 10 4版本中 支持如下O l D t b 版本  √Oracle 8i (DML support only)  √Oracle 9.1 and 9.2 (DML and DDL support) 91 92  √Oracle 10.1 and 10.2 (DML and DDL support)  √Oracle 11g (DML and DDL support)Copyright © 2008, Oracle and/or its affiliates. All rights reserved. 13
  14. 14. GoldenGate: 关键的架构优势• Distributed Loosely Coupled Process Based Architecture• Heterogeneous Initial Load• Asynchronous Change Propagation – Committed Transactions Only – DML, Metadata, DDL• Universal Trail Format – Interoperable across Heterogeneous Systems• Data Transformation Rules and Filtering (table row column) (table, row,• Archival and Audit Capabilities – Trail backups – Audit – History of all changes (Insert all records)
  15. 15. GoldenGate: 关键的架构优势• Conflict Detection and Resolution: for dual-active environments• Parallel Coordination (Source, Target)
  16. 16. Oracle GoldenGate 典型应用场景 企业级的实时数据集成解决方案 • 一套产品满足多种业务需求 套产品满足多种业务需求 Disaster Recovery, Data Protection Standby • 支持高可用性和实时数据集成 (Open & Active) Zero Downtime Migration and Upgrades Log Based, Real- Time Change Data Operational Capture Reporting Reporting Database OGG ETL ODS EDW ETL • 高度灵活Heterogeneous Real-time BI EDW • 部署迅速Source Systems • 运维简单 Query Offloading Data Distribution
  17. 17. 灵活的拓扑结构 单向复制 双向复制 点对点备份、查询 灾备、双业务中心 多业务中心广播复制 集中复制 多级复制数据分发 数据仓库、N+1灾备 层次化企业数据
  18. 18. GG在数据传输方面保证交易的一致性• GG有很好的机制来保证传输的交易一致性和断点续传。
  19. 19. GoldenGate TDM解决方案高可用性与容灾 实时数据集成• 容灾与应急备份 • 数据仓库实时供给• 消除计划内停机 • 实时报表• 双业务中心 • 政府、企业垂直部门级数据同步• OLTP和OLAP分开 • 主数据• 数据库移植、升级
  20. 20. 高可用性: 容灾与应急备份实现: 用途: 快速恢复和切换 在非计划性停机时保持业务连续 最小化数据损失 重新同步主备两端数据
  21. 21. 高可用性: 减少计划内停机实现: 用途: 保障业务零或者近似零停机 保障系统/应用/数据库在升级 滚动升级 、移植和维护期间业务的可用 降低业务中断带来的损失 性
  22. 22. 高可用性: 双业务中心(Active-Active)实现:• 负载均衡,提高系统整体性能• 连续可用,快速的容灾接管 连续可用 快速的容灾接管• 冲突检测和处理
  23. 23. 数据仓库实时供给(和ODI结合使用)实现: 用途: 数据仓库数据实时得到更新 提升商务智能和信息分析能力 对源系统低影响 实时可用的数据仓库
  24. 24. 实时数据整合: 查询分离实现: 用途: 将实时报表从主系统剥离,可根据客户 实时报表 需求定制 提高系统性能和可用性 显著提高主系统性能
  25. 25. 议程/Agenda 历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景 Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特点• GoldenGate外争和内比• GoldenGate实施经验分享
  26. 26. 全球范围内成功案例 • Over 400 customers with 4,000+ implementations across Fortune 500 companies: Financial Services, Communications, Healthcare Communications Healthcare, Public Sector Retail & Sector, Utilities industries • Top 3 of 5 largest commercial banks • Top 3 of 3 busiest ATM networks • Top 7 of 10 financial data services companies • Top 4 of 5 telecommunications providers • Top 3 of 5 largest food & drug stores
  27. 27. 案例:中华人民共和国海关总署 全国各地41个口岸H2000应用 网络 • 远距离容灾 GoldenGate • 低带宽 双向复制 Oracle 9i Oracle 9i • 低延时 可随时接管的双业务中心 RAC RAC • H2000生产库 H2000生产库GoldenGate 单向复制 GoldenGate 单向复制 • 快速业务切换 决策支持库 决策支持库 Oracle 9i RAC Oracle 9i 北京 广州
  28. 28. 案例:国家体育总局体彩中心 • N+1模式容灾 • 各省市集中备份 Parallel Coordination • 远距离 • 低带宽
  29. 29. 案例:国家税务总局  集中容灾  远距离  低带宽
  30. 30. 议程/Agenda 历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景 Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特点• GoldenGate外争和内比• GoldenGate实施经验分享
  31. 31. Oracle GoldenGate 组件 Communications and Startup GGSCI GGSCI Interactive Manager Manager Interactive Interface Interface Network Source Trail Target Trail Delivery Source Capture Pump Collector TargetDatabase Database Source Target TCP/IP Process Start Shared Memory (for Monitoring)
  32. 32. Oracle GoldenGate 组件 Communications and Startup GGSCI GGSCI Interactive Manager Manager Interactive Interface Interface Network Source Trail Target Trail Delivery Source Capture Pump Collector TargetDatabase Database Source Target TCP/IP Process Start Shared Memory (for Monitoring)
  33. 33. Oracle GoldenGate 组件 Communications and Startup GGSCI GGSCI Interactive Manager Manager Interactive Interface Interface Network Source Trail Target Trail Delivery Source Capture Pump Collector TargetDatabase Database Source Target TCP/IP Process Start Shared Memory (for Monitoring)
  34. 34. Oracle GoldenGate 组件 Communications and Startup GGSCI GGSCI Interactive Manager Manager Interactive Interface Interface Network Source Trail Target Trail Delivery Source Capture Pump Collector TargetDatabase Database Source Target TCP/IP Process Start Shared Memory (for Monitoring)
  35. 35. Oracle GoldenGate 组件 Communications and Startup GGSCI GGSCI Interactive Manager Manager Interactive Interface Interface Network Source Trail Target Trail Delivery Source Capture Pump Collector TargetDatabase Database Source Target TCP/IP Process Start Shared Memory (for Monitoring)
  36. 36. Oracle GoldenGate Capture GoldenGate GoldenGate Event Log Parameter Messaging Subsystem File Checkpoint Manager Parser Memory Manager GoldenGateTransaction Checkpoint Logs File VAM API/Redo Transaction API Transformation Serialization Formatting g Filtering g Log Reader and Mapping and Routing (log reader interface) GoldenGate Trail Files M t d t Metadata Metadata M t d t SQL T f ti Transformation Manager Manager Interface Engine Command Database Access and Control Capture GoldenGate Source Command Database Interpreter
  37. 37. Oracle GoldenGate Delivery GoldenGateGoldenGate Event LogParameter Messaging Subsystem File Checkpoint Manager Parser Memory Manager GoldenGate Checkpoint File GoldenGate Transformation SQL Trail Read API Formatting g Filtering g and Mapping Generation ReaderGoldenGate Trail Files Metadata M t d t SQL T f ti Transformation Manager Interface Engine Command and Control Database AccessGoldenGate Delivery Command TargetInterpreter Database
  38. 38. GoldenGate Capture捕捉• Log-based change data capture – High volume – Low overhead• Decoupled architecture – Multiple capture processes may be used to scale, but generally not required – Possible to split “hot” tables into a separate capture process• Filtering and Compression – Unnecessary change data if discarded immediately – U d t and d l t are “ Updates d deletes “compressed” b d f lt d” by default – Before images are discarded by default
  39. 39. GoldenGate Delivery传送• Decoupled architecture – Multiple delivery processes may be used to scale – Possible to split “hot” tables into a separate delivery process – Possible for multiple delivery p p y processes to split the work for a single p g table• T Transaction grouping ti i – Small transactions are grouped by default to reduce commit overhead
  40. 40. GoldenGate初始化• Start change data capture on Source Database – Current source Commit Sequence Number (CSN) is 222Current CSN is 222 CSNs 222 through … Commit Ordered Commit Ordered Capture Source Trail Pump Target Trail Source Target Database Database
  41. 41. GoldenGate初始化• Wait for any open transactions to close• Use a backup (or export/import) to copy source to target as of a consistency point – Current source Commit Sequence Number (CSN) is 245 ConsistentCurrent CSN is as of CSN 245 CSNs 222 245 through … Commit Ordered Commit Ordered Capture Source Trail Pump Target Trail Source Target Database Database Take Backup Apply as of CSN Backup as of 245 CSN 245
  42. 42. GoldenGate初始化• Start delivery at or after given backup CSN (245 in this case) – Current source Commit Sequence Number (CSN) is 356• Once Delivery is current the source and target are in sync ConsistentCurrent CSN is Deliver as of CSN 356 CSNs 222 transactions 356 through 356 after CSN In Sync! 245 Commit Ordered Commit Ordered Capture Source Trail Pump Target Trail Delivery Source Target Database Database Take Backup Apply as of CSN Backup as of 245 CSN 245
  43. 43. GoldenGate 检查点• Capture, Pump, and Delivery save positions to a checkpoint file so they can recover in case of failure Start of Oldest Open (Uncommitted) Begin, Begin TX 1 Transaction T ti Insert, TX 1 Begin, TX 2 Begin, TX 2 Update, TX 1 Insert, TX 2 Insert, TX 2 Commit, TX 2 Commit, TX 2 Capture Checkpoint Begin, TX 3 Begin, TX 3 Insert, TX 3 Insert, TX 3 Commit, TX 3 Current Begin, TX 4 Write Commit, TX 3 Position Delete, Delete TX 4 Current Read Position Commit Ordered Capture Source Trail Source Database
  44. 44. GoldenGate 检查点• Capture, Pump, and Delivery save positions to a checkpoint file so they can recover in case of failure Start of Oldest Open (Uncommitted) Begin, Begin TX 1 Transaction T ti Insert, TX 1 Begin, TX 2 Begin, TX 2 Pump Begin, TX 2 Checkpoint Update, TX 1 Insert, TX 2 Insert, TX 2 Insert, TX 2 Commit, TX 2 Commit, TX 2 Commit, TX 2 Capture Current Current Checkpoint Begin, TX 3 Read Write Begin, TX 3 Insert, TX 3 Position Position Insert, TX 3 Commit, TX 3 Current Begin, TX 4 Write Commit, TX 3 Position Delete, Delete TX 4 Current Read Position Commit Ordered Commit Ordered Capture Pump Source Trail Target Trail Source Database
  45. 45. GoldenGate 检查点• Capture, Pump, and Delivery save positions to a checkpoint file so they can recover in case of failure Start of Oldest Open (Uncommitted) Begin, Begin TX 1 Transaction T ti Insert, TX 1 Begin, TX 2 Pump Delivery Begin, TX 2 Begin, TX 2 Checkpoint Checkpoint Update, TX 1 Insert, TX 2 Insert, TX 2 Insert, TX 2 Commit, TX 2 Commit, TX 2 Commit, TX 2 Capture Current Current Current Checkpoint Begin, TX 3 Read Write Read Begin, TX 3 Insert, TX 3 Position Position Position Insert, TX 3 Commit, TX 3 Current Begin, TX 4 Write Commit, TX 3 Position Delete, Delete TX 4 Current Read Position Commit Ordered Commit Ordered Delivery Capture Pump Source Trail Target Trail Source Target Database Database
  46. 46. 开箱即用的转换能力• Data selection and filtering数据选择和过滤• Column mapping列映射• Functions函数功能
  47. 47. 数据的选择和过滤• GoldenGate provides the ability to select or filter out data based on a variety of levels and conditions Parameter Clause Selects TABLE or MAP –— Table WHERE Row FILTER Row, Operation Row Operation, Range TABLE COLS | COLSEXCEPT Columns
  48. 48. 列映射• GoldenGate provides the capability to map columns from one table to another• Data can be transformed between dissimilar database tables• Mapping can be applied either when extracting or replicating dataThis example uses special values to build history of operations dataINSERTALLRECORDSMAP SALES.ACCOUNT, TARGET REPORT.ACCTHISTORY, COLMAP (USEDEFAULTS, TRAN_TIME = @GETENV(“GGHEADER”,”COMMITTIMESTAMP”), OP_TYPE = @GETENV(“GGHEADER”, “OPTYPE”), _ BEFORE_AFTER_IND = @GETENV(“GGHEADER”, “BEFOREAFTERINDICATOR”), );
  49. 49. Functions函数功能• GoldenGate provides the capability to transform columns by using a set of built- in functions (currently 35)• Transformation functions can be applied either for Extract or Replicat• If you require more, you also have the ability to call your own logic through user exitsMAP SALES.ACCOUNT, TARGET REPORT.ACCOUNT,COLMAP ( USEDEFAULTS, TRANSACTION_DATE = @DATE (“YYYY-MM-DD”, “YY”, YEAR, YY , “MM”, MONTH, “DD”, DAY), AREA_CODE = @STREXT (PHONE-NO, 1, 3), PHONE_PREFIX = @STREXT (PHONE-NO, 4, 6), PHONE_NUMBER = @STREXT (PHONE-NO, 7, 10) ); _
  50. 50. 议程/Agenda 历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景 Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特点• GoldenGate外争和内比• GoldenGate实施经验分享
  51. 51. 各种数据容灾方案比较常见数据 方法描述及优点/缺点容灾方法基于存储  其核心是通过SAN进行存储之间的数据拷贝 常见方案如IBM的Fl hC 其核心是通过SAN进行存储之间的数据拷贝,常见方案如IBM的FlashCopy, EMC的SRDF等 EMC的SRDF等。  优点:速度快, 对操作系统和应用完全透明。 缺点: 成本高(尤其是远距离),对存储的型号有一定要求,无法避免数据库的逻辑坏块,对 数据的恢复状态不可知,为了保证数据的 致性,在某个时间点需要将数据库置为热备份模式或 数据的恢复状态不可知,为了保证数据的一致性,在某个时间点需要将数据库置为热备份模式或 者IO挂起,不能支持异构平台。基于卷管理  通过卷管理器的快照功能来实现数据同步, 常见方案如Veritas Volumn Manager Snapshot器 (类似于ASM Snapshot)  优点:可以通过SAN来完成数据同步,速度较快, 对存储无特定要求。  缺点: 大多与基于存储的方案相同, 另外这种方案需要额外的磁盘空间。基于数据库  其核心是把数据库的日志传送到远端数据库, 通过数据库的恢复机制将数据恢复出来。如的恢复 O l 的 D t Guard 和DB2的HADR Oracle Data G d 和DB2的HADR。 优点:方法简单, 对存储没有特别要求,过程对应用可见,可随时观测到已经恢复的数据,可 避免逻辑坏块。 缺点:对数据库之间的版本要求比较高,不能跨平台,对网络带宽有一定的要求。逻辑方式  通过外围应用来实现,实现方法如 GoldenGate.  优点:方法简单, 实时性强,灵活性强, 跨平台,可以忍受苛刻的网络条件。  缺点: 有一定的维护成本。Copyright © 2008, Oracle and/or its affiliates. All rights reserved. 51
  52. 52. Oracle GoldenGate 数据容灾特点 Oracle GoldenGate 用于数据容灾  O l G ld G t 架构非常灵活 Oracle GoldenGate  Oracle GoldenGate 容灾方案属于基于逻辑的数据容灾  Oracle GoldenGate 对操作系统没有特定的要求  Oracle GoldenGate 对数据库版本没有特定的要求  Oracle GoldenGate 能容忍苛刻的网络环境  Oracle GoldenGate 不存在逻辑坏块  Oracle GoldenGate 当生产系统的DDL操作比较多时,维护量比较大Copyright © 2008, Oracle and/or its affiliates. All rights reserved. 52
  53. 53. GoldenGate Vs. Quest SharePlex • GoldenGate国内主要竞争对手,两者架构类似 GoldenGate SharePlex异构数据库复制 支持 只支持Oracle复制粒度 事务级,大事务需要等待commit才复制,稍慢,但若系统 语句级,大事务复制快些,但若系统故障,源和目标就失去 故障,源和目标能自动同步。 同步,并且难于恢复同步。数据复制能力 > 1TB/天 300~500GB/天系统资源占用 无需与数据库交互,复制效率高,对源系统的影响非常小, 需要在ORACLE数据库中安装少量的对象用来存储有关复制 的一些基本信息;需要与数据库交互,复制效率相对低,对源 所有复制进程CPU占用率在5%以内 系统的影响比较小,所有复制进程CPU占用率在10%以内读取Log的接口 Oracle支持 Oracle不支持表映射1:N, N:1 支持 不支持容错能力 软件使用检查点机制记录当前完成复制的位置。在日常 Shareplex的技术原理,在不保证交易完整性的情况下,当 运行过程中,如果由于网络中断、数据库实例失败、存 复制过程中发生,网络主机数据库方面的故障,在异常情 复制过程中发生,网络主机 数据库方面的故障,在异常情 储空间不够等原因造成复制停止,GoldenGate能够以 况排除后,软件需要确认两端数据库中的大量信息,然 自定义间隔自动检测并在异常排除后立即自动恢复复制, 后才能重新开始复制。这个确认过程最少需要几十分钟 保障数据无丢失,使得管理和维护工作中人工介入降低 的时间,扩大了复制中断的时间。 到最小。
  54. 54. GoldenGate Vs. DSG RealSync• GoldenGate国内次要竞争对手,两者架构类似• DSG(迪思杰)是本土公司 GoldenGate RealSync 异构数据库复制 支持 只支持Oracle 只支持O l 复制粒度 事务级 事务级 性能 高 远低于GoldenGate 复制方向 单向、多向 单向 用途 容灾、HA、复制、分担查询、整合 侧重数据迁移 读取Log的接口 Oracle支持 Oracle不支持
  55. 55. 根据解决方案来选择数据移动和转换的Oracle产品 Golden Active Data Gate Data Integrator Guard EE Disaster Recovery / Data Protection Non-Cont Oracle Oracle tinuous Ava Data Distribution / Synchronization Zero Downtime Migrations / Upgrades ailability Query Offloading Hetero- Oracle geneous to Oracle Grid & Scalability (Oracle RAC) Operational ReportingData Inte Real Time BI Typical Data Warehouse yp egration Data Integration for SOA Migrating Between Different Apps
  56. 56. 针对数据集成的Oracle产品 Comparing How They Work Retrieve Data Load Data Disaster Recovery & Data Protection Redo in Active Data Memory Guard Buffer Direct Direct Memory Write to Logs Access 增加转换能力减少延迟 Real Time Data Integration & High Availability g g y GoldenGate Read Fast SQL 异构 On-Disk Logs g Data Integration for Data Warehouse & SOA Data D t Integrator Set-based, SQL Query Complex SQL 56
  57. 57. GoldenGate and Streams Oracle GoldenGate Oracle Streams• GoldenGate is Oracle’s strategic replication solution Oracle s• Streams continues to be a supported Oracle Database feature• Best Streams technology will be integrated into GoldenGate gy g
  58. 58. GoldenGate & StreamsGoldenGate is Oracle’s strategic Streams continues to be areplication solution li ti l ti supported Oracle Database feature t dO l D t b f t Oracle GoldenGate Oracle Streams• Out-of-the-box solutions • Highly customizable and flexible• Broad heterogeneous support • Optimized for Oracle Database• Excellent reference base • Few large referencesQuestion: I already have Streams for free? Why buy GoldenGate?• Better ROI overall: GoldenGate is lower cost to implement and maintain• Enterprise-wide solution: GoldenGate easily expands to new use cases• Support for Streams continues: Continue to use Streams where it’s deployed Lead with GoldenGate
  59. 59. Comparing GoldenGate & Streams• GoldenGate – Solution for all heterogeneous real-time data access and delivery – N i Non-invasive, t i transactionally consistent delivery of data across ti ll i t t d li fd t WAN – Very low latency - extreme data volumes – Solve wide range of HA challenges (active-active, multi-master, etc.) – Strategic Data Replication solution for Oracle DB and heterogeneous DBs g• Oracle Streams – Built in feature of Oracle DB allowing info sharing between Built-in databases – Can create data replication in homogeneous Oracle DB environment en ironment – Existing customers will continue to be supported – Continue and Converge: over time, best of Streams and GG products will converge
  60. 60. GoldenGate and Data Guard Heterogeneous Distributed Oracle Oracle Active Data Fully-active y GoldenGate G ld G t Guard Subset Replicas Oracle DB Primary Oracle DB Standby• Information Distribution & Consolidation, Application Upgrades & Changes – Use GoldenGate - heterogeneous, active-active, transformations, subsetting g , , , g• Disaster Recovery / Data Protection / HA – Simple Full Oracle Database Protection • Use Active Data Guard • High-performance, simple, drop-in solution for HA and DR, readable at standby • Works for all apps and data types, including packaged apps that can’t be changed – Application desiring flexible HA, active-active, schema changes, platform changes • Use GoldenGate • Avoid or manage active-active conflicts at application level active active
  61. 61. Product Comparison Active Data Guard and GoldenGate Oracle Active Data Guard Oracle GoldenGate Target  Any distributed, heterogeneous  Oracle Data Guard users Environment computing environment  Physical replica maintained by  Logical replica maintained by Technology shipping redo blocks replaying transactions  Bi-directional or uni-directional  Readable Data Guard standby  Capture once, apply many (N-way) Benefits  Highest transaction rates  Supports different schemas  Automatic repair of corrupt blocks  Supports heterogeneous platforms*  Oracle-only  Some data types unsupported Restrictions  Identical schemas  Manual switch-over / fail-over  “Similar” platforms Similar * Supported DBMS today: Oracle, MySQL, DB2, SQL Server, Sybase, Nonstop SQL, Enscribe, Teradata, Ingres, Greenplum, Netezza  * Supported DBMS today: Oracle MySQL DB2 SQL Server Sybase Nonstop SQL Enscribe Teradata Ingres Greenplum Netezza Supported Platforms today: IBM AIX, IBM Z/OS, HP‐UX, HP NonStop , Solaris, Linux Windows61
  62. 62. Complementary and Used Together A ti D t Guard and G ld G t Active Data G d d GoldenGate Heterogeneous Distributed Oracle Active Subset Oracle Data Guard Replicas GoldenGate Oracle DB Primary O l DB St db Oracle Standby Disaster Recovery & Data Distribution / Data Protection D t P t ti Synchronization S h i ti • Fast failover to physical standby database • Enterprise-wide heterogeneous replication • Fastest solution for Oracle • Zero downtime operations • Oracle Database only • Non-invasive by reading database logs • Allows standby to be open for reads to • Supports active-active scenarios leverage customer investment • Ensures transactional consistency • Transparent data protection to repair • High volume; low latency corrupted blocks on primary or standby • Guaranteed delivery across wide area networks Oracle Active Data Guard Oracle GoldenGate62
  63. 63. Oracle GoldenGate产品包 New Products Based on GoldenGate Technology Oracle Products Description 收购之前的产品Oracle GoldenGate v10.4 Includes GoldenGate Capture, Transactional Data Deliver and Active Data Guard and Management (TDM) XStream on Oracle DB and support Capture & Delivery, for other open systems databases. databases RollbackOracle GoldenGate for Includes GoldenGate Capture and TDM Capture&Delivery,Mainframe v10 Deliver for HP Non-Stop and IBM Syncfile, DB2 on Mainframe, plus Syncfile. RollbackOracle GoldenGate Veridata v3 Add-on capability to validate data in GoldenGate Veridata replicated systems.Management Pack fM t P k for Add-on management pack ( Add t k (was GoldenGate Di G ld G t Director tGoldenGate v2 “Director”). Restricted use WebLogic SE.Oracle GoldenGate Adapters (for Pre-packaged application content. Base 24 Add-on Modules,Siebel, Cerner, Amdocs, Base24, Integration Edition forJMS/Flat File, Logger-Enscribe) Java & Flat Files, Logger
  64. 64. 议程/Agenda 历史 定位和发展• GoldenGate历史,定位和发展• GoldenGate基本架构,应用场景 Go de Gate基本架构,应用场景• GoldenGate成功案例• GoldenGate技术特点• GoldenGate外争和内比• GoldenGate实施经验分享
  65. 65. Oracle GoldenGate 实施经验分享 (检查操作系统及数据库) Oracle GoldenGate 检查操作系统及数据库  如果操作系统是AIX 5.3,须确保 xlC/C++ 的版本是9.0或者以上.  如果源数据库的版本是9iR1或者9iR2,确保参数_log_parallelism或者 log_parallelism 的值为1.  确认源数据库处于归档模式 可用命令 archive log list查看 确认源数据库处于归档模式, list查看.  确认源数据库undo_retention的值足够大,undo tablespace的空间足够多.  确认源及目标数据库的open_cursors值足够大.  检查源库redo log的 Thread 数目.  支持DDL:如果数据库的版本是10gR1,将_recyclebin设置为false;如果数据库的 版本是10gR2或者以上,将recyclebin设置为off. 版本是10gR2或者以上 将recyclebin设置为offCopyright © 2008, Oracleand/or its affiliates. All rights 65reserved.
  66. 66. Oracle GoldenGate 实施经验分享 (数据初始化) 方法 优点/缺点 EXP/IMP  这种方法的好处是,可以通过管道直接将数据导入到目标 数据库, 这样做不需要额外的磁盘空间来存放导出文件。  缺点:比较慢。 DATA PUMP  优点:导出与导入速度都非常快。 缺点: 在源端及目标端都要有足够的空间来容纳导出文件, 且需要将导出文件传送到目标端。 Transportable Tablespace  优点:导出速度快, 使用RMAN TRANSPORT (TTS) TABLESPACE并不需要将表空间置为read only. 缺点:目标端的数据文件大小需要与与源端一致。 缺点:目标端的数据文件大小需要与与源端 致。 RMAN  优点:导出速度快, 适合整库导出. 缺点 如果数据库的版本是10 以下 不支持跨平台 源端 缺点:如果数据库的版本是10g以下, 不支持跨平台;源端 与目标端数据库版本一致,目标端只能通过数据库升级来升到 高版本。Copyright © 2008, Oracleand/or its affiliates. All rights 66reserved.
  67. 67. Oracle GoldenGate 实施经验分享 (性能调优) Oracle GoldenGate 性能调优  Extract 仅做抓取  过滤和转换, 尽量由Data Pump完成 在目标端查看表的统计信息, 如果统计信息太旧,别忘了对表做分析Copyright © 2008, Oracle and/or its affiliates. All rights reserved. 6
  68. 68. Q&A
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×