Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
What a DBA needs to know about
the differenc...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Introduction
Mike Donovan @dbvisitmike
Globa...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Dbvisit Software
• Database replication is o...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Dbvisit product lineup
World-leading Disaste...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Objective
1. Understanding the differences b...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Replication Use cases to consider
Disaster R...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Historic Replication options available
1. Tr...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
What is Redo?
Contains history of all databa...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
What is redo
• Online redo: instance/crash r...
• REDO RECORD - Thread:1 RBA: 0x000068.00000015.008c LEN: 0x02e0 VLD: 0x09 CON_UID:
3345156736
• SCN: 0x0000.002d22c4 SUBS...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
What is Redo?
Redo will include:
1. Change r...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Redo - Most important
– Does not contain SQL...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Basic structure of redo
• Since 9i, redo is ...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Two Oracle Replication Types
Based on Oracle...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Oracle replication based on redo
Physical Re...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Oracle replication based on redo
Logical Rep...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Two Oracle Replication Types compared
Block ...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Two Oracle Replication Types
1 - Physical Re...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Physical Replication
Advantages
• 100% binar...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Physical Replication
Disadvantages
• Replica...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Two Oracle Replication Types
2 - Logical Rep...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Logical Replication
Advantages
• Subset of d...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Logical Replication
Disadvantages
• Not as e...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Database requirements
Physical replication
•...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Deeper dive
Logical Replication
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
How does redo log mining logical replication...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Typical logical replication flow
Mine
Mines ...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Conversion process
Set based SQL operations ...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Logical replication principle
Observations:
...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Considerations with logical based replicatio...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Considerations with logical based replicatio...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Deeper dive
Physical Replication
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Standby protection modes
There are three pro...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Keeping the standby database up to date
Stan...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Considerations with physical based replicati...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Replication Use cases
Disaster Recovery
DRaa...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Covered Objective
1. Understand the differen...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Trusted in 80+ countries…. . .
…. . . By 800...
© 2 0 1 5 D b v i s i t
S o f t w a r e L i m i t e d |
d b v i s i t . c o m
Thank you - Q&A
Upcoming SlideShare
Loading in …5
×

[db tech showcase Tokyo 2015] D25:The difference between logical and physical replication on Oracle (ロジカルレプリケーションとフィジカルレプリケーションを使いこなす) by Dbvisit Software Limited. Mike Donovan

11,432 views

Published on

Published in: Technology
  • Be the first to comment

[db tech showcase Tokyo 2015] D25:The difference between logical and physical replication on Oracle (ロジカルレプリケーションとフィジカルレプリケーションを使いこなす) by Dbvisit Software Limited. Mike Donovan

  1. 1. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m What a DBA needs to know about the differences between Oracle physical and logical Database replication Presenter: Mike Donovan Global Digital Business Development Dbvisit Software Ltd
  2. 2. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Introduction Mike Donovan @dbvisitmike Global Digital Business Development Been with Dbvisit for 7+ years Based in Auckland, New Zealand Past Experience: • DBA • Developer/programmer • Technical presenter • Speaker at OOW, Collaborate
  3. 3. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Dbvisit Software • Database replication is our world • Dedicated software company • High level of Oracle database expertise • Deep bench of professional and technical services providers • HQ in New Zealand, US subsidiary, partners throughout the world • Worldwide leader in disaster recovery solutions for Oracle Standard Edition • 7+ years of exceptional growth (since inception)
  4. 4. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Dbvisit product lineup World-leading Disaster Recovery Solution for the Oracle SE database Affordable Alternative to GoldenGate: – Data Replication for Oracle-sourced databases
  5. 5. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Objective 1. Understanding the differences between logical and physical replication 2. Understanding the advantages and disadvantages of each 3. Understanding how they work and the underlying concepts 4. Understanding the use cases of each
  6. 6. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Replication Use cases to consider Disaster Recovery DRaaS Data Distribution HA Hybrid cloud Zero downtime Migration (cloud) Reporting Databases BI Real time dashboards Digital business
  7. 7. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Historic Replication options available 1. Trigger based • Changes to source database are required • Performance impact • A lot more maintenance 2. Redo log based • No changes to source database • Lower impact to source environment • Less maintenance • DDL changes can be replicated
  8. 8. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m What is Redo? Contains history of all database changes 1. Basic goal: recover database 2. Written from redo log buffer first and then to redo log (after commit, 3 seconds or 1/3 full) Redo log buffersDatabase buffers Redo Database Files 1 2
  9. 9. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m What is redo • Online redo: instance/crash recovery • Archive redo: media recovery • Every change is written to redo before data files • Allows to replay changes • Optimized for recovery and for performance • Binary format • Even when dumped to text, still very cryptic ALTER SYSTEM DUMP LOGFILE 'filename';
  10. 10. • REDO RECORD - Thread:1 RBA: 0x000068.00000015.008c LEN: 0x02e0 VLD: 0x09 CON_UID: 3345156736 • SCN: 0x0000.002d22c4 SUBSCN: 1 07/08/2014 15:58:58 • CHANGE #1 CON_ID:3 TYP:0 CLS:1 AFN:13 DBA:0x034000bb OBJ:95424 SCN:0x0000.00285536 SEQ:2 OP:11.2 ENC:0 RBL:0 • fb: --H-FL-- lb: 0x1 cc: 4 • col 0: [ 7] 54 45 5f 54 45 53 54 • col 1: [ 8] 54 65 73 74 20 6a 6f 62 • CHANGE #2 CON_ID:1 TYP:0 CLS:19 AFN:4 DBA:0x01000090 OBJ:4294967295 SCN:0x0000.002d228d SEQ:1 OP:5.2 ENC:0 RBL:0 • ktudh redo: slt: 0x0013 sqn: 0x000008d3 flg: 0x0052 siz: 112 fbi: 0 • uba: 0x01003327.0187.04 pxid: 0x0000.000.00000000 pdbid:3345156736 • CHANGE #4 CON_ID:1 TYP:0 CLS:19 AFN:4 DBA:0x01000090 OBJ:4294967295 SCN:0x0000.002d22c4 SEQ:1 OP:5.4 ENC:0 RBL:0 • ktucm redo: slt: 0x0013 sqn: 0x000008d3 srt: 0 sta: 9 flg: 0x2 • CHANGE #5 CON_ID:1 TYP:0 CLS:20 AFN:4 DBA:0x01003327 OBJ:4294967295 SCN:0x0000.002d228c SEQ:1 OP:5.1 ENC:0 RBL:0 • ktudb redo: siz: 112 spc: 7776 flg: 0x0012 seq: 0x0187 rec: 0x04 • xid: 0x0002.013.000008d3 • ktubl redo: slt: 19 rci: 0 opc: 11.1 [objn: 95424 objd: 95424 tsn: 3] • Undo type: Regular undo Begin trans Last buffer split: No • Temp Object: No
  11. 11. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m What is Redo? Redo will include: 1. Change record. Changes made by Insert, Update, Delete, Merge (DML) • Change vectors. These describe the transition of a database block from one state to another. 2. DDL Redo does not include: 1. Actual text from DML 2. Select statements
  12. 12. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Redo - Most important – Does not contain SQL ! – Contains operations to database blocks – There are 150 different redo operations – They are in binary format specific to OS and endianness – They are specific to the database SID and DBID – They are specific to the Oracle version – Transactions are not necessarily in order
  13. 13. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Basic structure of redo • Since 9i, redo is used by LogMiner and Streams • More information needed: How do you assemble row pieces together • How do you identify row if you cannot use rowid Answer: supplemental logging • Adds row assembly information • Adds primary key information (or whatever you specify) • Oracle also added actual DDL text to redo This is visible in the text dump
  14. 14. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Two Oracle Replication Types Based on Oracle Redo: Physical Replication • Binary copy • Example - Data Guard, Dbvisit Standby Logical Replication • Logical copy • Example - GoldenGate, Dbvisit Replicate
  15. 15. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Oracle replication based on redo Physical Replication • Use complete redo blocks for replication • Information is unaltered • Target applies binary changes (redo blocks) • Target is in constant “recovery” mode TargetSource Redo log buffersDatabase buffers (Archived) Redo Recovery process Database files
  16. 16. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Oracle replication based on redo Logical Replication • Mining of Redo - Extract information from redo blocks for replication • Information is altered/transformed into SQL • SQL is applied at target • Target is normal read/write database Source Target Redo log buffers Database buffers (Archived) Redo Mine Process Apply Process SQL
  17. 17. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Two Oracle Replication Types compared Block changes Binary parsed redo (Archived) Redo Mine Process Apply Process Read/Write mode SQL Whole redo Extract Process Recover Process Recovery mode Physical Replication Logical Replication
  18. 18. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Two Oracle Replication Types 1 - Physical Replication • Use database block level (redo) to keep up to date • Physically the replicated database is same as source database • 100% binary copy, structure and data • Referred to as primary (source) and standby database (target) • Standby database is in constant recovery mode • Examples – Oracle Data Guard (Oracle Enterprise Edition only) – Dbvisit Standby (Oracle XE, SE1, SE, EE)
  19. 19. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Physical Replication Advantages • 100% binary copy, structure and data – rowid, indexes, FK, security, DB links • Offload backup to standby database • Recover data files from standby database • Applying changes to standby is fast and efficient • Standby server can have fewer resources • Easy to create standby database using rman or third party tool • Easy to manage • Low admin overhead • Standby database can be used for read only queries • Roll reversal or graceful switchover is possible
  20. 20. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Physical Replication Disadvantages • Replication is all or nothing • Same OS must be used • Same Oracle version must be used • When Standby database is in recovery mode it is unusable • NOLOGGING changes break the standby database • All Archives are needed, no single sequence can be skipped • Activation (failover) of standby database breaks the link with primary • Failover can cause data loss • When standby database is in read only mode, it cannot apply latest changes (unless you have Active Data Guard) • No indexes can be added to standby database • No security settings can be added to standby database
  21. 21. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Two Oracle Replication Types 2 - Logical Replication • Uses SQL statements to keep database up to date • Replicated database has the same data • Separate physical database structure • Examples: – Oracle GoldenGate – Streams – Dbvisit Replicate
  22. 22. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Logical Replication Advantages • Subset of data can be replicated • Across different OS and Oracle versions • Target database is normal read-write database • Different indexes and security policies can be employed • Target database can be non-Oracle (Hadoop, MySQL MS SQL etc) • Target database can be used for other applications
  23. 23. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Logical Replication Disadvantages • Not as efficient as physical replication • Need the same resources as source database (if everything is replicated) • SQL is applied which can cause potential conflicts and locks • SQL applied is not the same as on source • Database not the same structure (tablespaces, indexes, partitions, MV) • Not everything is replicated (Data types, passwords, sequences, rowid) • Requires data-instantiation (data is in sync before replication starts)
  24. 24. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Database requirements Physical replication • Force logging is recommended Logical replication • Force logging is recommended • Supplemental logging must be turned on Any changes not recorded in redo will not be replicated ! • Direct load • APPEND hint in SQL statement INSERT /*+ APPEND */ INTO emp SELECT * FROM t_emp; COMMIT;
  25. 25. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Deeper dive Logical Replication
  26. 26. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m How does redo log mining logical replication work 1. Redo logs are mined for LCR* as Oracle is writing to the logs (real time) 2. LCR are pieced together in correct order (ie row chaining, row migration, RAC) 3. Filtered LCR are written to a file or a queue 4. Transferred to target server 5. Translated to SQL statements 6. Run against target database * Logical Change Record - describes a change to a row
  27. 27. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Typical logical replication flow Mine Mines the redo logs and converts into a parsed log PLOG Parsed logs – binary files specific to solution (trail file in OGG) Platform independent Apply Converts parsed log data into target DB native SQL
  28. 28. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Conversion process Set based SQL operations on the source database are converted to row-by-row SQL changes on the target database. Why? – This is the way that Oracle writes to the Redo – Redo does not contains SQL, it contains db block changes Observations: – SQL is not the same on source as on target – True for all logical replication based solutions – Each SQL on target only affects 1 row
  29. 29. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Logical replication principle Observations: 1. Source SQL updates 2 rows, then 2 individual update statements are produced 2. PK has been added to the WHERE to ensure row-by-row 3. Price formula has been replaced by hardcoded value Source Target update PRICES set PRICE = PRICE - (PRICE * .10) where PRODUCT_CAT = 'OLD_STOCK'; update PRICES set PRICE = 10 where PROD_ID = 101; update PRICES set PRICE = 23 where PROD_ID = 102;
  30. 30. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Considerations with logical based replication Commit strategies 1. Optimistic commit (do not wait for commit before mining and applying) 2. Wait for commit before applying (do not wait for commit before mining) 3. Wait for commit before mining and applying Examples of products with above strategies: 1. Dbvisit Replicate 2. Streams 3. GoldenGate
  31. 31. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Considerations with logical based replication Generic 1. Replicated data is same order as source (based on SCN) 2. Transaction integrity is maintained (replication is serial) 3. Be-aware of triggers on target database 4. Atomic transactions are maintained (commit & rollback) 5. Conflict detection warn of data divergence 6. Conflicts pauses the whole replication to ensure integrity
  32. 32. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Deeper dive Physical Replication
  33. 33. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Standby protection modes There are three protection modes for the primary database with physical replication: Maximum Performance: Transfer of redo information to the standby server is asynchronous, so it does not impact on performance of the primary. Available in SE & SE1 Maximum Availability: Transactions on the primary do not commit until redo information has been written to the online redo log and the standby redo logs of at least one standby location. Not available in SE & SE1 Maximum Protection: Transactions on the primary do not commit until redo information has been written to the online redo log and the standby redo logs of at least one standby location. If not suitable standby location is available, the primary database shuts down. Not available in SE & SE1
  34. 34. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Keeping the standby database up to date Standby databases are held up to date by applying archive (redo) log files  Standby database is in constant recovery mode  For Oracle EE, online redo (using standby redo) or archive logs can be used  For Oracle SE and SE1, only archive logs are used (no standby redo logs)  Uses standby controlfile  Oracle EE sqlplus> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT FROM SESSION;  Oracle SE and SE1 sqlplus> recover standby database  Archive log files from ALL threads are needed with RAC
  35. 35. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Considerations with physical based replication 1. Failover should not be treated lightly (data can be lost) 2. Archive missing or NOLOGGING invalidates standby database 3. Database patching requires standby database consideration 4. Requires regular DR testing 5. Graceful switchover (or role reversing) is useful for minimizing downtime during planned maintenance
  36. 36. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Replication Use cases Disaster Recovery DRaaS Data Distribution HA Hybrid cloud Zero downtime Migration (cloud) Reporting Databases BI Real time dashboards Digital business Physical Replication Logical Replication Logical Replication Logical Replication Logical Replication Physical Replication
  37. 37. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Covered Objective 1. Understand the differences between logical and physical replication 2. Understand the advantages and disadvantages of each 3. Understand how they work and the underlying concepts 4. Understand the use cases of each
  38. 38. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Trusted in 80+ countries…. . . …. . . By 800+ companies.
  39. 39. © 2 0 1 5 D b v i s i t S o f t w a r e L i m i t e d | d b v i s i t . c o m Thank you - Q&A

×