Consolidating Microsoft SQL Server Databases into an Oracle 11g Cluster Dylan Kucera Director – Data Architecture Ontario ...
Agenda <ul><ul><li>Gaining buy-in for a DW migration </li></ul></ul><ul><ul><li>Adoption of a new DW </li></ul></ul><ul><u...
Gaining buy-in for a DW migration <ul><ul><li>Patience isn’t just a virtue, it is a requirement </li></ul></ul><ul><ul><li...
Gaining buy-in for a DW migration <ul><ul><li>Scalability </li></ul></ul><ul><ul><ul><li>Don’t just focus on RAC </li></ul...
Gaining buy-in for a DW migration <ul><ul><li>Environment Capability </li></ul></ul><ul><ul><ul><li>PL/SQL capabilities ma...
A new Data Warehouse – If we build it, will they come? <ul><ul><li>Promotion </li></ul></ul><ul><ul><li>Skills Development...
Oracle Migration (Workbench) <ul><ul><li>OMW can help fast-track code migration </li></ul></ul><ul><ul><ul><li>Result may ...
OMW – Capturing a SQL Server model
OMW – Captured T-SQL Procedure
OMW – Converted PL/SQL Procedure
OMW – But what about the legacy? <ul><li>Big bang – not possible or risky </li></ul><ul><li>Migrating consumers will take ...
Views employing Transparent Gateway CREATE   OR   REPLACE   VIEW  PLAY.VALUE_TABLE_SAMPLE  AS SELECT &quot;IDENTIFIER&quot...
Pitfalls of Transparent Gateway - Views DECLARE tDate  DATE  := '2008-12-31'; BEGIN INSERT   INTO  PLAY.TEMP_SAMPLE_7445 (...
Procs employing Transparent Gateway CREATE   PROCEDURE  dbo.GetNHLTeamStats @inDate DATETIME, @inTeam VARCHAR(8) AS SELECT...
Procs employing Transparent Gateway CREATE   OR   REPLACE   PROCEDURE  PLAY.RPT_NHL_TEAM_STAT ( inDate  DATE , inTeam  VAR...
Pitfalls of Transparent Gateway – Procs and Result Sets {CALL PLAY.RPT_NHL_TEAM_STATS ('2010-08-31', 'SJS')} Results in OR...
Oracle Streams as an enabler of migration Oracle Streams ETL
Heterogeneous Streams – General Architecture Diagram Adapted from “Oracle Database 11g: Oracle Streams Replication, An Ora...
Heterogeneous Streams – Example Legacy Microsoft SQL Server Data Warehouse: TABLE NorthWind.dbo.NHL_PLAYER_STATS ( DATE DA...
Heterogeneous Streams – Heterogeneous Apply BEGIN  DBMS_APPLY_ADM.CREATE_APPLY( queue_name  => 'SAMPLE_STREAM_Q', apply_na...
Heterogeneous Streams – Heterogeneous Apply Transforms BEGIN DBMS_STREAMS_ADM.RENAME_TABLE( rule_name  => 'NHL_PLAYER_STAT...
Heterogeneous Streams – Before inserts to Captured Oracle table
Heterogeneous Streams – Insert some rows into Captured Oracle table SQL> INSERT INTO PLAY.NHL_PLAYER_STAT (DATE_, TEAM, SW...
Heterogeneous Streams – After inserts to Captured Oracle table
Streams – NULL vs. Empty String <ul><li>Oracle treats empty string = NULL </li></ul><ul><li>Convert empty strings on SQL S...
Heterogeneous Streams – Syncing tables containing Floats
Heterogeneous Streams – Syncing tables containing Floats
Heterogeneous Streams – Syncing tables containing Floats
Wrap-up <ul><ul><li>Gaining buy-in for a DW migration </li></ul></ul><ul><ul><li>Oracle Migration Workbench </li></ul></ul...
Questions? Dylan Kucera Director – Data Architecture Ontario Teachers’ Pension Plan Oracle OpenWorld 2010– S313546 Sunday ...
 
Upcoming SlideShare
Loading in …5
×

Oracle OpenWorld 2010 - Consolidating Microsoft SQL Server Databases into an Oracle 11g Cluster

547 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
547
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Oracle OpenWorld 2010 - Consolidating Microsoft SQL Server Databases into an Oracle 11g Cluster

  1. 2. Consolidating Microsoft SQL Server Databases into an Oracle 11g Cluster Dylan Kucera Director – Data Architecture Ontario Teachers’ Pension Plan Oracle OpenWorld 2010– S313546 Sunday September 19, 2010 3:00pm-4:00pm
  2. 3. Agenda <ul><ul><li>Gaining buy-in for a DW migration </li></ul></ul><ul><ul><li>Adoption of a new DW </li></ul></ul><ul><ul><li>Oracle Migration Workbench </li></ul></ul><ul><ul><li>Views over Transparent Gateway </li></ul></ul><ul><ul><li>Stored Procedure stubs over TG </li></ul></ul><ul><ul><li>Oracle Streams </li></ul></ul>
  3. 4. Gaining buy-in for a DW migration <ul><ul><li>Patience isn’t just a virtue, it is a requirement </li></ul></ul><ul><ul><li>Start thinking about metrics before anything else </li></ul></ul><ul><ul><li>Tie every message about the DW to business requirements </li></ul></ul>
  4. 5. Gaining buy-in for a DW migration <ul><ul><li>Scalability </li></ul></ul><ul><ul><ul><li>Don’t just focus on RAC </li></ul></ul></ul><ul><ul><ul><li>Locking Model </li></ul></ul></ul><ul><ul><ul><ul><li>Writers do not block Readers </li></ul></ul></ul></ul><ul><ul><ul><ul><li>No Lock Escalation </li></ul></ul></ul></ul><ul><ul><ul><li>Workload Balancing </li></ul></ul></ul><ul><ul><li>Availability </li></ul></ul><ul><ul><ul><li>RAC </li></ul></ul></ul><ul><ul><ul><li>Extprocs and links to foreign DB’s run out of process </li></ul></ul></ul><ul><ul><ul><li>Flashback </li></ul></ul></ul>
  5. 6. Gaining buy-in for a DW migration <ul><ul><li>Environment Capability </li></ul></ul><ul><ul><ul><li>PL/SQL capabilities may simplify your deployments (Language constructs, packages) </li></ul></ul></ul><ul><ul><ul><li>Advanced DW features eg. Materialized Views </li></ul></ul></ul><ul><ul><li>Maintainability </li></ul></ul><ul><ul><ul><li>Oracle Enterprise Manager </li></ul></ul></ul><ul><ul><ul><li>Less replication = lower support costs </li></ul></ul></ul><ul><ul><li>Fit with strategic deployment </li></ul></ul><ul><ul><ul><li>Best, if you are lucky enough! </li></ul></ul></ul>
  6. 7. A new Data Warehouse – If we build it, will they come? <ul><ul><li>Promotion </li></ul></ul><ul><ul><li>Skills Development </li></ul></ul><ul><ul><li>Mentoring </li></ul></ul><ul><ul><li>Standards </li></ul></ul><ul><ul><li>Architectural Reviews </li></ul></ul><ul><ul><li>Controls </li></ul></ul>
  7. 8. Oracle Migration (Workbench) <ul><ul><li>OMW can help fast-track code migration </li></ul></ul><ul><ul><ul><li>Result may not conform to your standards </li></ul></ul></ul><ul><ul><li>Can be used for tables as well, but </li></ul></ul><ul><ul><ul><li>Do you want to adjust model? </li></ul></ul></ul><ul><ul><ul><li>What about the legacy DW? </li></ul></ul></ul>
  8. 9. OMW – Capturing a SQL Server model
  9. 10. OMW – Captured T-SQL Procedure
  10. 11. OMW – Converted PL/SQL Procedure
  11. 12. OMW – But what about the legacy? <ul><li>Big bang – not possible or risky </li></ul><ul><li>Migrating consumers will take time </li></ul><ul><li>Take a staged approach </li></ul><ul><ul><li>Transparent Gateway </li></ul></ul><ul><ul><li>Oracle Streams </li></ul></ul>
  12. 13. Views employing Transparent Gateway CREATE OR REPLACE VIEW PLAY.VALUE_TABLE_SAMPLE AS SELECT &quot;IDENTIFIER&quot; AS ID_, &quot;VALUE&quot; AS VALUE_, FILE_DATE AS FILE_DATE FROM [email_address]
  13. 14. Pitfalls of Transparent Gateway - Views DECLARE tDate DATE := '2008-12-31'; BEGIN INSERT INTO PLAY.TEMP_SAMPLE_7445 (ID_, NAME_, PREV_VALUE, CURR_VALUE, VALUE_SUPPLIER, DATE_VALUE_CHANGED) SELECT ID_, '', '', '', 'SAMPLE', MAX (FILE_DATE) FROM PLAY.VALUE_TABLE_SAMPLE WHERE FILE_DATE <= tDate GROUP BY ID_; END ; Results in ORA-03113: end-of-file on communication channel Alert log says ORA-07445: exception encountered: core dump [intel_fast_memcpy.A()+18] [ACCESS_VIOLATION] [ADDR:0x115354414B] [PC:0x52A9DFE] [UNABLE_TO_READ] [] Fixed in 11.1.0.6 Patch 10 and 11.1.0.7 Patch 7
  14. 15. Procs employing Transparent Gateway CREATE PROCEDURE dbo.GetNHLTeamStats @inDate DATETIME, @inTeam VARCHAR(8) AS SELECT SWEATER_NO, NAME, POINTS, FACE_OFF_PCT FROM NorthWind..NHL_PLAYER_STATS WHERE DATE = @inDate AND TEAM = @inTeam
  15. 16. Procs employing Transparent Gateway CREATE OR REPLACE PROCEDURE PLAY.RPT_NHL_TEAM_STAT ( inDate DATE , inTeam VARCHAR2, RC1 IN OUT SYS_REFCURSOR ) IS tRC1_MS SYS_REFCURSOR ; tSweater_No NUMBER ; tName VARCHAR2(8); tPoints NUMBER ; tFace_Off_Pct NUMBER ; BEGIN DELETE FROM PLAY.TEMP_NHL_TEAM_STAT; dbo.GetNHLTeamStats@MSSQL(inDate, inTeam, tRC1_MS); LOOP FETCH tRC1_MS INTO tSweater_No, tName, tPoints, tFace_Off_Pct; EXIT WHEN tRC1_MS% NOTFOUND ; BEGIN INSERT INTO PLAY.TEMP_NHL_TEAM_STAT (SWEATER_NO, NAME_, POINTS, FACE_OFF_PCT) VALUES (tSweater_No, tName, tPoints, tFace_Off_Pct); END ; END LOOP ; CLOSE tRC1_MS; OPEN RC1 FOR SELECT SWEATER_NO, NAME_, POINTS, FACE_OFF_PCT FROM PLAY.TEMP_NHL_TEAM_STAT ORDER BY SWEATER_NO; END RPT_NHL_TEAM_STAT ; <ul><li>RECIPE: </li></ul><ul><li>Declare Variables for all MSSQL Result set columns </li></ul><ul><li>Call MSSQL Procedure </li></ul><ul><li>Fetch Result one row at a time </li></ul><ul><li>Insert row to Oracle Temporary Table </li></ul><ul><li>Open Ref Cursor result set </li></ul>
  16. 17. Pitfalls of Transparent Gateway – Procs and Result Sets {CALL PLAY.RPT_NHL_TEAM_STATS ('2010-08-31', 'SJS')} Results in ORA-06504: PL/SQL: Return types of Result Set variables or query do not match Fixed in 11.1.0.7 Patch 7
  17. 18. Oracle Streams as an enabler of migration Oracle Streams ETL
  18. 19. Heterogeneous Streams – General Architecture Diagram Adapted from “Oracle Database 11g: Oracle Streams Replication, An Oracle White Paper, July 2007”
  19. 20. Heterogeneous Streams – Example Legacy Microsoft SQL Server Data Warehouse: TABLE NorthWind.dbo.NHL_PLAYER_STATS ( DATE DATETIME, TEAM VARCHAR(8), SWEATER_NO INT, NAME VARCHAR(128), BIRTH_DATE DATETIME, POINTS INT, FACE_OFF_PCT FLOAT ) New Oracle Data Warehouse: TABLE PLAY. NHL_PLAYER_STAT ( DATE_ DATE, TEAM VARCHAR2(8), SWEATER_NO NUMBER, NAME_ VARCHAR2(128), BIRTH_DATE DATE, POINTS NUMBER, FACE_OFF_PCT NUMBER );
  20. 21. Heterogeneous Streams – Heterogeneous Apply BEGIN DBMS_APPLY_ADM.CREATE_APPLY( queue_name => 'SAMPLE_STREAM_Q', apply_name => 'SAMPLE_APPLY_NORTHWIND', apply_captured => TRUE, apply_database_link => 'MSSQL_STREAMS_NORTHWIND' ); END; / BEGIN DBMS_STREAMS_ADM.ADD_TABLE_RULES( table_name => 'PLAY.NHL_PLAYER_STAT', streams_type => 'APPLY', streams_name => 'SAMPLE_APPLY_NORTHWIND', queue_name => 'SAMPLE_STREAM_Q', include_dml => true, include_ddl => false ); END; /
  21. 22. Heterogeneous Streams – Heterogeneous Apply Transforms BEGIN DBMS_STREAMS_ADM.RENAME_TABLE( rule_name => 'NHL_PLAYER_STAT2283', from_table_name => 'PLAY.NHL_PLAYER_STAT', to_table_name => '&quot;dbo&quot;.NHL_PLAYER_STATS', step_number => 0, operation =>'ADD'); END; / BEGIN DBMS_STREAMS_ADM.RENAME_COLUMN( rule_name => 'NHL_PLAYER_STAT2283', table_name => 'PLAY.NHL_PLAYER_STAT', from_column_name => '&quot;DATE_&quot;', to_column_name => '&quot;DATE&quot;', value_type => '*', step_number => 0, operation => 'ADD'); END; /
  22. 23. Heterogeneous Streams – Before inserts to Captured Oracle table
  23. 24. Heterogeneous Streams – Insert some rows into Captured Oracle table SQL> INSERT INTO PLAY.NHL_PLAYER_STAT (DATE_, TEAM, SWEATER_NO, NAME_, BIRTH_DATE, POINTS, FACE_OFF_PCT) 2 VALUES ('2010-08-31', 'VAN', 33, 'Henrik Sedin', '1980-09-26', 112, 49.5); 1 row inserted SQL> INSERT INTO PLAY.NHL_PLAYER_STAT (DATE_, TEAM, SWEATER_NO, NAME_, BIRTH_DATE, POINTS, FACE_OFF_PCT) 2 VALUES ('2010-08-31', 'PIT', 87, 'Sidney Crosby', '1987-08-07', 109, 55.9); 1 row inserted SQL> INSERT INTO PLAY.NHL_PLAYER_STAT (DATE_, TEAM, SWEATER_NO, NAME_, BIRTH_DATE, POINTS, FACE_OFF_PCT) 2 VALUES ('2010-08-31', 'WSH', 8, 'Alex Ovechkin', '1985-09-17', 109, 45.4); 1 row inserted SQL> INSERT INTO PLAY.NHL_PLAYER_STAT (DATE_, TEAM, SWEATER_NO, NAME_, BIRTH_DATE, POINTS, FACE_OFF_PCT) 2 VALUES ('2010-08-31', 'SJS', 19, 'Joe Thornton', '1979-07-02', 89, 53.9); 1 row inserted SQL> INSERT INTO PLAY.NHL_PLAYER_STAT (DATE_, TEAM, SWEATER_NO, NAME_, BIRTH_DATE, POINTS, FACE_OFF_PCT) 2 VALUES ('2010-08-31', 'OTT', 11, 'Daniel Alfredsson', '1972-12-11', 71, 35.0); 1 row inserted SQL> INSERT INTO PLAY.NHL_PLAYER_STAT (DATE_, TEAM, SWEATER_NO, NAME_, BIRTH_DATE, POINTS, FACE_OFF_PCT) 2 VALUES ('2010-08-31', 'CGY', 12, 'Jarome Iginla', '1977-07-01', 69, 47.0 ); 1 row inserted SQL> INSERT INTO PLAY.NHL_PLAYER_STAT (DATE_, TEAM, SWEATER_NO, NAME_, BIRTH_DATE, POINTS, FACE_OFF_PCT) 2 VALUES ('2010-08-31', 'TOR', 15, 'Tomas Kaberle', '1978-03-02', 49, NULL); 1 row inserted SQL> COMMIT; Commit complete
  24. 25. Heterogeneous Streams – After inserts to Captured Oracle table
  25. 26. Streams – NULL vs. Empty String <ul><li>Oracle treats empty string = NULL </li></ul><ul><li>Convert empty strings on SQL Server target to NULL before streaming </li></ul><ul><li>Could require business analysis and application change </li></ul>
  26. 27. Heterogeneous Streams – Syncing tables containing Floats
  27. 28. Heterogeneous Streams – Syncing tables containing Floats
  28. 29. Heterogeneous Streams – Syncing tables containing Floats
  29. 30. Wrap-up <ul><ul><li>Gaining buy-in for a DW migration </li></ul></ul><ul><ul><li>Oracle Migration Workbench </li></ul></ul><ul><ul><li>Views over Transparent Gateway </li></ul></ul><ul><ul><li>Stored Procedure stubs over TG </li></ul></ul><ul><ul><li>Oracle Streams </li></ul></ul><ul><ul><li>Standards, metadata, education, mentoring </li></ul></ul>
  30. 31. Questions? Dylan Kucera Director – Data Architecture Ontario Teachers’ Pension Plan Oracle OpenWorld 2010– S313546 Sunday September 19, 2010 3:00pm-4:00pm Thank You! - I value your feedback!

×