1. Benchmarking the SSIS connectors for Oracle, DB2 and SAP Franck Sidi Technology Architect SQL & BI Microsoft Israel
2. Franck Sidi Microsoft – 2004 to Now Microsoft France – SQL Server PreSales Technical Leader Migrations Specialist Oracle, Sybase, DB2 SAP Integration, BI Architecture ATG – 2000 to 2004 ATG France – PreSales Technical Manager Keywords : Java, J2EE, Portal, eCommerce, Search, ECM, Mainframe Integration Sybase – 1994 – 2000 SYBASE INC USA 1999 – 2000 – Senior Developer in charge of PeoplesoftInc and Kenan (Telco Billing ISV) Sybase France 1994 – 1999 – Tech Support Engineer & Presales Benchmark Specialist (Sybase, Sybase IQ) 1993 - Master Degree Database Systems (6 Years at University) Learn on Teradata & Oracle
3. Session Objectives Session Objective(s): Give overview of the connectivity capabilities with SSIS for Enterprise databases and solutions Oracle, DB2 and SAP Give some quick feedback about Performance capabilities with Oracle Drivers Tips and Tricks
4. Data Flow Connectivity Options Data Flow Task OLEDBConnector ADO.NETConnector CustomConnector OLEDBProvider NativeADO.NETAdapter ADO.NETODBC Data SourceSpecificInterface/Protocol ODBC Data Source Specific Interface/ Protocol Data Storage
7. Microsoft OLEDB Provider for DB2 Interactive and scriptable Setup program SNA Trace Utility and Trace Viewer TCP/IP network connection Execution of dynamic SQL commands (DDL and DML), including CALL statement for stored procedures Customized Data Link property dialog boxes for creating and modifying file-persisted OLE DB data link files. Supports Integration Services, Replication, Analysis Services, Reporting Services, DQP Packaging and licensing Include with Microsoft SQL Server 2008 Enterprise and Developer editions Feature Pack for SQL Server 2008 Microsoft download site
34. Oracle Connector by Attunity Supported platforms & versions SQL Server 2008 9.2.0.4 and higher Oracle SSIS DB Oracle Client Any platform Windows XP x86, x64 Windows Vista x86, x64 Windows Server 2003 x86, x64, IA64 Windows Server 2008 x86, x64, IA64
35. Requirements Oracle client software version 10.x or 11.x, on the computer with SSIS. Use 32-bit client on 32 bit platforms, 64-bit on 64-bit platform If using a 64-bit platforms as a designer client (i.e.; using BIDS) both 64 and 32-bit clients should be installed. SQL Server 2008 - Enterprise or Developer Edition Windows 2003,2008
36. Oracle Connector by Attunity High Performance Architecture Oracle integration (using OCI): Bulk extract source using OCI Array Binding Bulk load destination using OCI Direct Path (fastest, constraints must turned off) Incremental Bulk load using OCI Array Binding Oracle Connection Manager SSIS integration: Direct integration into internal buffering APIs, cutting through .NET and other layers Oracle SSIS DB OCI Oracle Client
37. Oracle connector test setup A generous two-machine configuration Simple SSIS tasks! 24 cores @ 2.4 GHz 32 GB RAM HP disk array, partitioned as 17 logical drives 64-bit software 24 cores @ 2.4 GHz 32 GB RAM HP disk array, partitioned as 17 logical drives 64-bit software Attunity Oracle connectors SSIS (SQL Server 2008) Windows Server 2008 Oracle 11g Windows Server 2008 Flat files Oracle DB 1 GB Ethernet
38. Oracle connector test setup Oracle DDL “Mixed data types” create table LINEITEM ( L_SHIPDATE DATE not null, L_ORDERKEY NUMBER(19,0) not null, L_DISCOUNT NUMBER(10,4) not null, L_EXTENDEDPRICE NUMBER(19,4) not null, L_SUPPKEY NUMBER(10,0) not null, L_QUANTITY NUMBER(5,0) not null, L_RETURNFLAG char(1) not null, L_PARTKEY NUMBER(10,0) not null, L_LINESTATUS char(1) not null, L_TAX NUMBER(10,4) not null, L_COMMITDATE DATE not null, L_RECEIPTDATE DATE not null, L_SHIPMODE varchar2(10) not null, L_LINENUMBER NUMBER(10,0) not null, L_SHIPINSTRUCT varchar2(25) not null, L_COMMENT varchar2(44) not null ) “String data types” CREATE TABLE LINEITEM ( L_SHIPDATE varchar2(10), L_ORDERKEY varchar2(10), L_DISCOUNT varchar2(10), L_EXTENDEDPRICE varchar2(10), L_SUPPKEY varchar2(10), L_QUANTITY varchar2(10), L_RETURNFLAG varchar2(1), L_PARTKEY varchar2(10), L_LINESTATUS varchar2(1), L_TAX varchar2(10), L_COMMITDATE varchar2(10), L_RECEIPTDATE varchar2(10), L_SHIPMODE VARCHAR2(10), L_LINENUMBER varchar2(10), L_SHIPINSTRUCT VARCHAR2(25), L_COMMENT VARCHAR2(44) )
39. Test definitions EmptyTable SSIS reads from flat files, writes into Oracle Table is initially empty Attunity connector Fast Load calls Oracle DirectPath API AddToTable SSIS reads from flat files, writes into Oracle Table already has data in it Extract SSIS reads from Oracle, writes flat files Test measurements Measured throughput (rows/sec) Multiply by 133 bytes/row to get bytes/sec
40. Mixed data types vs. string-only Can we exploit this difference? mixed
41. CPU usage Fast Load Not Fast Load SSIS Process Oracle Process
42. Local vs. remote execution Local Remote mixed - local mixed - remote string - local string - remote Difference is generally not significant
43. 32-bit vs. 64-bit performance 64-bit32-bit Probably not enough to worry about
44. Moving data from Oracle to SQL Server Face it, we want to move data out of Oracle String data extracts are faster than mixed types Real-world data has various data types How to extract as fast as possible? Keep mixed types in Oracle Extract strings by converting in query Convert to SQL Server types in SSIS data flow
48. Parallel Mode What does the SSIS Oracle Destination Parallel property mean in Fast Load mode? The Parallel Load option is available in the Oracle Destination component when choosing the Fast Load mode.This setting tells Oracle not to lock the target table exclusively for this Fast Load session. This allows running additional fast load destination components to load the same target table in parallel. Does this mean that the Oracle Destination component will load the target table in parallel sessions? No, it means that the table will not be locked exclusively by the Oracle Destination therefore the user can design multiple data flow tasks that loads the same table.Here is an example of two tasks loading the same table: A Source component reads records 1-500 and the Fast Load Destination component loads them to the target Oracle table. A Source component reads records 501-1000 and the Fast Load Destination component loads them to the target Oracle table in parallel. Note that both Destination components should have the Parallel property checked.
49. Support : http://www.attunity.com/forums/microsoft-ssis-connectors-attunity/ Error Message: Failed to load OCI DLLSUMMARYThe error indicates a failure when loading the Oracle oci.dll. The error could occur when having multiple Oracle homes or permission problems.SCENARIO 1The user is working on Microsoft Windows 64Bit operating system and have installed the following components: Oracle Client 32Bit (Required for Design) Oracle Client 64Bit Microsoft SSIS Oracle Connector 64Bit This is the required list of components for Developing on 64Bit Windows machine. On some machines the following error was reported when running the Oracle component in 32Bit (Design time): Error at Package [Connection manager "Oracle Connector 1"]: Failed to load OCI DLL.SOLUTIONFirst check that the environments are working and configured properly, and that SQLPlus is responding on both installations of the Oracle Client.On the machine which we encountered the error, it was noticed that the Oracle Home for the 64Bit Oracle Client was defined in the 32Bit registry portion (Wow6432Node), this was causing the problem of loading the wrong oci.dll.To workaround the problem, the user should define a dummy registry entry (Z_SSIS) as follows: 1. Open the regedit utility.2. Locate the following Key: HKEY_LOCAL_MACHINEOFTWAREow6432NodeRACLE.ssis_oracle1.png3. Right-Click on ORACLE node and click on New -> Key.4. Call the new key Z_SSIS (to make sure it's the last entry).5. Right-Click on the Z_SSIS node and click on New -> String.6. Name the property ORACLE_HOME.7. Double-Click on the ORACLE_HOME and set it to the location of the Oracle 32Bit installation home directory.
50. Attunity Oracle-CDC for SSIS Complete solution for integrating changes made to Oracle tables using SSIS, efficiently and in real-time Minimal impact on Oracle (log-based CDC, no install on Oracle) Efficient and reduced use or resources (process only changes) Fully integrated into SSIS (2005, 2008) Synchronizes Bulk-Load and CDC Monitoring & control console Change Stream Oracle SQL Server log SSIS 34
51. Additional Resources Download Connectors Oracle SSIS Connector http://www.microsoft.com/downloads/details.aspx?FamilyID=d9cb21fe-32e9-4d34-a381-6f9231d84f1e&DisplayLang=en Teradata Connector http://www.microsoft.com/downloads/details.aspx?FamilyID=d9cb21fe-32e9-4d34-a381-6f9231d84f1e&DisplayLang=en DB2 Connector http://download.microsoft.com/download/0/E/6/0E67502A-22B4-4C47-92D3-0D223F117190/DB2OLEDB.exe SAP BI http://www.microsoft.com/DownLoads/details.aspx?familyid=B33D2C78-1059-4CE2-B80D-2343C099BCB4&displaylang=en SAP Theobald http://www.theobald-software.com/cms/en/haupt/evaluation.html Others Links http://www.codeplex.com/ssisctc 35
