• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
BI Apps ETL-SSIS 2008 & 2012, Pentaho & Talend
 

BI Apps ETL-SSIS 2008 & 2012, Pentaho & Talend

on

  • 393 views

 

Statistics

Views

Total Views
393
Views on SlideShare
393
Embed Views
0

Actions

Likes
0
Downloads
5
Comments
1

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft Word

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • hi sir .. pls i need this slide ..pls pls pls for God sake ..pls pls i need this slide email me pls saqib.ganatra@gmail.com ..thank sir if you emailed me ..
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    BI Apps ETL-SSIS 2008 & 2012, Pentaho & Talend BI Apps ETL-SSIS 2008 & 2012, Pentaho & Talend Document Transcript

    • Business Intelligence Applications -ETL Microsoft SSIS 2008 & 2012, Talend Open Studio & Pentaho Data Integration By Sunny Okoro
    • 1
    • 2 Contents ETL Applications............................................................................................................................................3 Applications...................................................................................................................................................5 Database Platform ........................................................................................................................................8 Pentaho Enterprise Data Integration Edition.............................................................................................11 Talend Open Studio.....................................................................................................................................25 Microsoft SQL Server Integration Services 2008 ........................................................................................35 Oracle SQL Developer Data Migration Utility .............................................................................................48 Microsoft SQL Server Integration Services 2012 ........................................................................................56
    • 3 ETL Applications Pentaho Enterprise Data Integration Edition Talend Open Studio
    • 4 Microsoft SQL Server Integration Services 2012&2008 Oracle SQL Developer Data Migration Utility
    • 5 Applications MySQL Workbench 1. 2. 3. 4. Oracle SQL Developer
    • 6 Microsoft Excel 2010 Oracle JDeveloper
    • 7 IBM Data Studio Microsoft Visual Studio Microsoft Visual Studios 2010
    • 8 Database Platform MYSQL 5.5 IBM DB2 10
    • 9 Microsoft SQL Server 2012 Microsoft SQL Server 2008R2
    • 10 Sybase SQL AnyWhere12 Oracle 11g
    • 11 Pentaho Enterprise Data Integration Edition Database Connection
    • 12 Tasks
    • 13 1. Extract data from Sybase SQL Anywhere12 Adventure Works database server and dump the data into a XML, text and Excel File. The task will contain five components which are Table Input, Microsoft Excel Output, Text File Output, Add XML and XML Output as illustrated below. Table Input The table input contains the query to retrieve the data from the database. I have renamed the step name to Customer Table to reflect the name of the data being extracted. The connection has a drop
    • 14 down arrow which allows me to select from lists of established database or data source. I placed a limit to increase the processing time. The preview options allow developers to preview the data in real time. They can specify the number of rows returned for preview. I am new to this ETL application and is much different compared to SSIS. Text FileOuput
    • 15 The text file output works like the Microsoft SSIS File component. The only difference is that the user does not used variables to specify extensions to the file name as illustrated below to distinguish each generated files from old files. The developer has to check one of the boxes to specify their criteria for file naming standard.
    • 16 The fields tab allows developer to format the destination output Add XML Allows developers to specify their encoding format, root xml element like auto, root etc and output structure
    • 17 XML Ouput This components works like the text output file. The developer has to specify the filename and other criteria.
    • 18 Microsoft Excel Output This component has multiple parts to format the data according to specifications that matches to Excel destination.
    • 19
    • 20
    • 21
    • 22
    • 23 Textile Output Extract DueDate;OrderDate;ShipDate;CountryRegionCode;CUSTOMERID;FREIGHT;SUBTOTAL;TAXAMT;TOTALDUE;TerritoryID;TerritoryName 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29825;616.10;24643.94;1971.51;27231.55;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29672;38.83;1553.10;124.25;1716.18;5;Southeast 07/13/2001;07/01/2001;07/08/2001;US;29672;38.83;1553.10;124.25;1716.18;5;Southeast Results Abridged Excel Output File Extract SalesOrderNumber OrderDate DueDate ShipDate ProductID CustomerID SubTotal TaxAmt Freight TotalDue ProductName TerritoryName CountryRegionCode TerritoryGroup TerritoryID SO43659 07/01/2001 07/13/2001 07/08/2001 776 29825 24643.94 1971.51 616.10 27231.55 Mountain-100 Black, 42 Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 777 29825 24643.94 1971.51 616.10 27231.55 Mountain-100 Black, 44 Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 778 29825 24643.94 1971.51 616.10 27231.55 Mountain-100 Black, 48 Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 771 29825 24643.94 1971.51 616.10 27231.55 Mountain-100 Silver, 38 Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 772 29825 24643.94 1971.51 616.10 27231.55 Mountain-100 Silver, 42 Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 773 29825 24643.94 1971.51 616.10 27231.55 Mountain-100 Silver, 44 Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 774 29825 24643.94 1971.51 616.10 27231.55 Mountain-100 Silver, 48 Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 714 29825 24643.94 1971.51 616.10 27231.55 Long-Sleeve Logo Jersey, M Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 716 29825 24643.94 1971.51 616.10 27231.55 Long-Sleeve Logo Jersey, XL Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 709 29825 24643.94 1971.51 616.10 27231.55 Mountain Bike Socks, M Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 712 29825 24643.94 1971.51 616.10 27231.55 AWC Logo Cap Southeast US North America 5 SO43659 07/01/2001 07/13/2001 07/08/2001 711 29825 24643.94 1971.51 616.10 27231.55 Sport-100 Helmet, Blue Southeast US North America 5 Results Abridged XML Output File Extract <?xml version="1.0" encoding="UTF-8"?> <Rows> <Row><CUSTOMERID>29825</CUSTOMERID><OrderDate>07&#47;01&#47;01</OrderDate><DueDate>07&#47;13&#47;01</DueDate><ShipDate>07&#47;08&#47;01</ShipDate><CountryRegionCode>US</CountryRegionCode><Sales OrderNumber>SO43659</SalesOrderNumber><ProductID>776</ProductID><ProductName>Mountain-100 Black, 42</ProductName><FREIGHT> 616.0984</FREIGHT><TAXAMT> 1971.5149</TAXAMT><SUBTOTAL> 24643.9362</SUBTOTAL><TOTALDUE> 27231.5495</TOTALDUE><TerritoryID> 5</TerritoryID><TerritoryName>Southeast</TerritoryName><TerritoryGroup>North America</TerritoryGroup></Row> <Row><CUSTOMERID>29825</CUSTOMERID><OrderDate>07&#47;01&#47;01</OrderDate><DueDate>07&#47;13&#47;01</DueDate><ShipDate>07&#47;08&#47;01</ShipDate><CountryRegionCode>US</CountryRegionCode><Sales OrderNumber>SO43659</SalesOrderNumber><ProductID>777</ProductID><ProductName>Mountain-100 Black, 44</ProductName><FREIGHT> 616.0984</FREIGHT><TAXAMT> 1971.5149</TAXAMT><SUBTOTAL> 24643.9362</SUBTOTAL><TOTALDUE> 27231.5495</TOTALDUE><TerritoryID> 5</TerritoryID><TerritoryName>Southeast</TerritoryName><TerritoryGroup>North America</TerritoryGroup></Row> <Row><CUSTOMERID>29825</CUSTOMERID><OrderDate>07&#47;01&#47;01</OrderDate><DueDate>07&#47;13&#47;01</DueDate><ShipDate>07&#47;08&#47;01</ShipDate><CountryRegionCode>US</CountryRegionCode><Sales OrderNumber>SO43659</SalesOrderNumber><ProductID>778</ProductID><ProductName>Mountain-100 Black, 48</ProductName><FREIGHT> 616.0984</FREIGHT><TAXAMT> 1971.5149</TAXAMT><SUBTOTAL> 24643.9362</SUBTOTAL><TOTALDUE> 27231.5495</TOTALDUE><TerritoryID> 5</TerritoryID><TerritoryName>Southeast</TerritoryName><TerritoryGroup>North America</TerritoryGroup></Row>
    • 24 <Row><CUSTOMERID>29825</CUSTOMERID><OrderDate>07&#47;01&#47;01</OrderDate><DueDate>07&#47;13&#47;01</DueDate><ShipDate>07&#47;08&#47;01</ShipDate><CountryRegionCode>US</CountryRegionCode><Sales OrderNumber>SO43659</SalesOrderNumber><ProductID>771</ProductID><ProductName>Mountain-100 Silver, 38</ProductName><FREIGHT> 616.0984</FREIGHT><TAXAMT> 1971.5149</TAXAMT><SUBTOTAL> 24643.9362</SUBTOTAL><TOTALDUE> 27231.5495</TOTALDUE><TerritoryID> 5</TerritoryID><TerritoryName>Southeast</TerritoryName><TerritoryGroup>North America</TerritoryGroup></Row> <Row><CUSTOMERID>29825</CUSTOMERID><OrderDate>07&#47;01&#47;01</OrderDate><DueDate>07&#47;13&#47;01</DueDate><ShipDate>07&#47;08&#47;01</ShipDate><CountryRegionCode>US</CountryRegionCode><Sales OrderNumber>SO43659</SalesOrderNumber><ProductID>772</ProductID><ProductName>Mountain-100 Silver, 42</ProductName><FREIGHT> 616.0984</FREIGHT><TAXAMT> 1971.5149</TAXAMT><SUBTOTAL> 24643.9362</SUBTOTAL><TOTALDUE> 27231.5495</TOTALDUE><TerritoryID> 5</TerritoryID><TerritoryName>Southeast</TerritoryName><TerritoryGroup>North America</TerritoryGroup></Row> <Row><CUSTOMERID>29825</CUSTOMERID><OrderDate>07&#47;01&#47;01</OrderDate><DueDate>07&#47;13&#47;01</DueDate><ShipDate>07&#47;08&#47;01</ShipDate><CountryRegionCode>US</CountryRegionCode><Sales OrderNumber>SO43659</SalesOrderNumber><ProductID>773</ProductID><ProductName>Mountain-100 Silver, 44</ProductName><FREIGHT> 616.0984</FREIGHT><TAXAMT> 1971.5149</TAXAMT><SUBTOTAL> 24643.9362</SUBTOTAL><TOTALDUE> 27231.5495</TOTALDUE><TerritoryID> 5</TerritoryID><TerritoryName>Southeast</TerritoryName><TerritoryGroup>North America</TerritoryGroup></Row> <Row><CUSTOMERID>29825</CUSTOMERID><OrderDate>07&#47;01&#47;01</OrderDate><DueDate>07&#47;13&#47;01</DueDate><ShipDate>07&#47;08&#47;01</ShipDate><CountryRegionCode>US</CountryRegionCode><Sales OrderNumber>SO43659</SalesOrderNumber><ProductID>774</ProductID><ProductName>Mountain-100 Silver, 48</ProductName><FREIGHT> 616.0984</FREIGHT><TAXAMT> 1971.5149</TAXAMT><SUBTOTAL> 24643.9362</SUBTOTAL><TOTALDUE> 27231.5495</TOTALDUE><TerritoryID> 5</TerritoryID><TerritoryName>Southeast</TerritoryName><TerritoryGroup>North America</TerritoryGroup></Row> <Row><CUSTOMERID>29825</CUSTOMERID><OrderDate>07&#47;01&#47;01</OrderDate><DueDate>07&#47;13&#47;01</DueDate><ShipDate>07&#47;08&#47;01</ShipDate><CountryRegionCode>US</CountryRegionCode><Sales OrderNumber>SO43659</SalesOrderNumber><ProductID>714</ProductID><ProductName>Long-Sleeve Logo Jersey, M</ProductName><FREIGHT> 616.0984</FREIGHT><TAXAMT> 1971.5149</TAXAMT><SUBTOTAL> 24643.9362</SUBTOTAL><TOTALDUE> 27231.5495</TOTALDUE><TerritoryID> 5</TerritoryID><TerritoryName>Southeast</TerritoryName><TerritoryGroup>North America</TerritoryGroup></Row> Results Abridged I am new to this application, I would later discover how to resolve the date mapping for XML.
    • 25 Talend Open Studio Database Systems: MYSQL 5.6 Task: Extract data from the database with customer information and payment history which will include sum and countinformation of each amount paid. Step 1 Create view in the database.
    • 26 Step 2 Using Talend Open Studio design the ETL job with components that will connect to the database and extract the data tMYSQLInput Component Contains the database connection to MYSQL 5.6 Database Connection
    • 27 Guess schema allows the mapping of the columns to the database source. The SQL Builder allows the developer to view SQL Statements.
    • 28 The connector between Tmysqlinput_1 and the aggregation component allows developers to create new columns that would hold the aggregation data as illustrated below.
    • 29 tAggregaateSoredRow_1 Component Contains the aggregation details of the ETL process
    • 30 The connector between the destinations contains the columns required to complete the mapping process after the aggregation process. Using the Edit Schema maps the columns respectively as illustrated below.
    • 31 tFileOutputExcel_1 Component
    • 32 This component allows developers to specify requirements for the Excel destination including column mapping using edit schema as illustrated below. Customer sales file extract
    • 33 customer_id store_id Cust_Name address address2 phone city postal_code country amount payment_date Sum_Payment Count_Payment 2 0 $0.00 $128.73 27 4 0 $0.00 $81.78 22 1 0 $0.00 $118.68 32 3 0 $0.00 $135.74 26 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $4.99 27-05-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $2.99 17-06-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $2.99 10-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $6.99 10-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $4.99 27-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $5.99 27-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $5.99 27-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $5.99 29-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $2.99 29-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $5.99 29-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $4.99 30-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $10.99 30-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $0.99 30-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $6.99 30-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $6.99 30-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $2.99 31-07-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $0.99 01-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $0.99 02-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $5.99 02-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $6.99 02-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $2.99 02-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $2.99 17-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $2.99 19-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $4.99 21-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $5.99 21-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $4.99 22-08-2005 $0.00 2 1 JOHNSON,PATRICIA 1121 Loja Avenue 838635286649 San Bernardino 17886 United States $4.99 23-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $4.99 15-06-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $0.99 16-06-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $2.99 16-06-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $0.99 16-06-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $0.99 17-06-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $5.99 19-06-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $2.99 28-07-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $2.99 28-07-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $3.99 29-07-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $5.99 30-07-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $5.99 30-07-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $0.99 02-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $2.99 02-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $4.99 17-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $2.99 18-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $8.99 18-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $1.99 19-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $2.99 20-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $6.99 20-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $4.99 21-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $2.99 22-08-2005 $0.00 4 2 JONES,BARBARA 1566 Inegl Manor 705814003527 Myingyan 53561 Myanmar $1.99 23-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $2.99 25-05-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 28-05-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $5.99 15-06-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 15-06-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $9.99 15-06-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $4.99 16-06-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $4.99 18-06-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 18-06-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $3.99 21-06-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $5.99 08-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $5.99 08-07-2005 $0.00
    • 34 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $4.99 09-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $4.99 09-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $7.99 11-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $2.99 27-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $4.99 28-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $4.99 28-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 28-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 28-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $2.99 29-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $2.99 31-07-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $4.99 01-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $3.99 02-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 02-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $4.99 17-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 18-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 19-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $2.99 19-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $0.99 21-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $1.99 22-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $2.99 22-08-2005 $0.00 1 1 SMITH,MARY 1913 Hanoi Way 28303384290 Sasebo 35200 Japan $5.99 22-08-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $1.99 27-05-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $2.99 29-05-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $8.99 16-06-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $6.99 16-06-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $6.99 17-06-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $2.99 19-06-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $4.99 07-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $4.99 08-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $5.99 27-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $10.99 27-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $7.99 28-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $6.99 28-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $4.99 28-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $4.99 28-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $2.99 29-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $1.99 30-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $3.99 30-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $2.99 31-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $4.99 31-07-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $5.99 01-08-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $4.99 18-08-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $8.99 19-08-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $2.99 20-08-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $8.99 21-08-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $0.99 22-08-2005 $0.00 3 1 WILLIAMS,LINDA 692 Joliet Street 448477190408 Athenai 83579 Greece $2.99 23-08-2005 $0.00 Result Abridged
    • 35 Microsoft SQL Server Integration Services 2008 Database Systems: Oracle 11g, Microsoft SQL Server 2008 R2 and 2012 Task Create view to contain customer information relating to sales history (sales order number, tax, amount paid etc). The data will be dumped into an Oracle Database using Oracle Client for Microsoft Visual Studio, text file and SQL Server 2012.
    • 36 View Creation View Verification SSIS Package Few technical problems with Microsoft Visual Studio 2010 forced me to switch to Microsoft Visual Studio 2008 for Business Intelligence Development.
    • 37 Microsoft SSIS Package Execution Phase Design Stage SQL Server Database 2008 R2 This component contains the database configuration needed to extract the data
    • 38 FlatFile Destination Component This components contains all the configuration details for the flat file destination
    • 39 Multicast Component This component acts like a copy machine that directs the output data from the aggregation component to various components. Derived Column Component This component allows developers to add new columns to the package. In this task, a created date column would be added to output that would allow users to know when a particular data is loaded into the database.
    • 40 DataConversion Component This component allows developers to map the Input and Output columns to a specific data type and sizes based on the database specification. Both the Oracle data conversion were identical with exception of the created date for SQL Server 2012. Database Destination Components This component contains the database destination. Oracle and SQL Server uses different connection parameters to establish connection with exception of dbo prefix in SQL Server. That is the major
    • 41 difference between Oracle and SQL Server Database Destination components. I have maintained the same names for the tables in both databases.
    • 42 Variables and File Naming Standard
    • 43 Each time the variables passed in the expression will be used to create the file using the connection string as illustrated. I did not add any constraints to the connector or error handling to this package since is not a production based package SISS Execution SISS can be executed using Microsoft Visual Studio, SQL Agent, Command Line or Execute Page Utility. Execute Page Utility
    • 44
    • 45 Validation Oracle 11G The file is created per design with the date encoded as illustrated below Data Validation Text File Extract SQL Server 2012
    • 46 Customer_ID,Cust_Name,ProductKey,SalesOrderNumber,OrderQuantity,UnitPrice,TotalProductCost,Sal esAmount,TaxAmt,Freight 11003,Christy,Zhu,346,SO43701,1,3399.99,1912.15,3399.99,272.00,85.00 14501,Ruben,Prasad,336,SO43700,1,699.10,413.15,699.10,55.93,17.48 21768,Cole,Watson,310,SO43697,1,3578.27,2171.29,3578.27,286.26,89.46 25863,Sydney,Wright,346,SO43699,1,3399.99,1912.15,3399.99,272.00,85.00 28389,Rachael,Martinez,346,SO43698,1,3399.99,1912.15,3399.99,272.00,85.00 11005,Julio,Ruiz,351,SO43704,1,3374.99,1898.09,3374.99,270.00,84.37 11011,Curtis,Lu,344,SO43705,1,3399.99,1912.15,3399.99,272.00,85.00 16624,Albert,Alvarez,310,SO43703,1,3578.27,2171.29,3578.27,286.26,89.46 27645,Colin,Anand,311,SO43702,1,3578.27,2171.29,3578.27,286.26,89.46 16351,Martha,Xu,313,SO43709,1,3578.27,2171.29,3578.27,286.26,89.46 16517,Katrina,Raji,314,SO43710,1,3578.27,2171.29,3578.27,286.26,89.46 20042,Brad,Deng,330,SO43708,1,699.10,413.15,699.10,55.93,17.48 27616,Emma,Brown,312,SO43707,1,3578.27,2171.29,3578.27,286.26,89.46 27621,Edward,Brown,312,SO43706,1,3578.27,2171.29,3578.27,286.26,89.46 Results Abridged
    • 47 SELECT* FROMmaster.dbo.CustomerSales Results Abridged SELECT "Customer_ID", "Cust_Name", "ProductKey", "SalesOrderNumber", "OrderQuantity", "UnitPrice", "TotalProductCost", "SalesAmount", "TaxAmt", "Freight" FROM "CustomerSales" Results Abridged Createad_Date Customer_ID Cust_Name ProductKey SalesOrderNumber OrderQuantity UnitPrice TotalProductCost SalesAmount TaxAmt Freight 5/12/2013 20:24 17440 Richard,Cox 477 SO63903 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 18130 Kaitlyn,Mitchell 477 SO63880 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 18221 Marc,Alvarez 477 SO63887 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 18933 Kelli,Huang 477 SO63904 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 20645 Xavier,Stewart 477 SO63941 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 21166 Tamara,Yuan 477 SO63936 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 21550 Kara,Deng 477 SO63910 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 11724 Jason,Carter 477 SO63965 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 12599 Jillian,Fernandez 477 SO63981 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 13858 Melanie,Richardson 477 SO63969 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 14671 Cristina,Lal 477 SO63994 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 15063 Kari,Malhotra 477 SO63955 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 15369 Connor,Perez 477 SO63980 1 4.99 1.87 4.99 0.4 0.12 5/12/2013 20:24 15931 Micah,Xu 477 SO63958 1 4.99 1.87 4.99 0.4 0.12 Customer_ID Cust_Name ProductKey SalesOrderNumber OrderQuantity UnitPrice TotalProductCost SalesAmount TaxAmt Freight 11245 Ricky,Vazquez 477 SO51178 1 4.99 1.87 4.99 0.4 0.12 16313 Hailey,Russell 477 SO51180 1 4.99 1.87 4.99 0.4 0.12 11006 Janet,Alvarez 477 SO51198 1 4.99 1.87 4.99 0.4 0.12 11448 Kyle,Patterson 477 SO51197 1 4.99 1.87 4.99 0.4 0.12 12390 Denise,Martinez 477 SO51191 1 4.99 1.87 4.99 0.4 0.12 18906 Adrienne,Gutierrez 477 SO51196 1 4.99 1.87 4.99 0.4 0.12 21440 Ian,Morgan 477 SO51199 1 4.99 1.87 4.99 0.4 0.12 11117 April,Deng 477 SO51216 1 4.99 1.87 4.99 0.4 0.12
    • 48 Oracle SQL Developer Data Migration Utility In this task I would migrate MYSQL Database Sakila and DB2GSDB Database into Oracle database. I have created the repository named Oracle System using Oracle System account. The actual data will be stored in new schema named sakila default password of sakila for MYSQL migration. For IBM DB2 each schema will be created with the schema name as the user name and password for that schema. I would connect to the Oracle GOSALES schema for demonstration .
    • 49
    • 50
    • 51 Repository explorer shows the three migration projects including the DB2 and MYSQL Data Validation- MYSQL Migragation To connect to the new created schema sakila I have to use the connection parameters as illustrated below.
    • 52 Oracle SQL Developer MySQL Workbench
    • 53 Data Validation- IBM DB2 Migragation Oracle SQL Developer IBM Data Studio
    • 54 File Location Oracle SQL Developer generates several output files which can be used to reconstruct the database as illustrated below MYSQL Output Migragation File
    • 55 DB2 Migration Output Files
    • 56 Microsoft SQL Server Integration Services 2012
    • 57 Table Creation Table verification
    • 58 Stored Procedures Creation Stored Procedure Verification
    • 59 SSIS Package Design and Implementation CustSales Package SSIS 2008 Format
    • 60 Solution This package will create reports associated with sales. The state report will contain total sales across the states associated with a country code provided in the parameter to the stored procedure. This report will contain total sales, average, maximum and minimum results of each state The customer report will contains aggregations associated with each customer of total sales, average, maximum and minimum of their sales history. The yearly report will contain aggregations associated with sales of specific year including total sales, average, maximum and minimum of sales history. The customer sales pivot report will contain total sales of each customer sales information in pivot format associated with a yearly format. Each reports generated are stored in oracle database and text files. The text files extensions include the file name, date and country code provided in the variable to the stored procedure. Design The package was designed using Microsoft 2008 Business Intelligence Development and was later converted to the 2012 SSIS format using Microsoft Visual Studio 2010 as illustrated below
    • 61 CustSales Package SSIS 2012 Version Design Details The stored procedure is executed using the value received from the variable which is mapped to the stored procedure parameter to retrieve specific data.
    • 62
    • 63
    • 64
    • 65
    • 66
    • 67 Execution
    • 68 Pivot Report View Verification
    • 69 Database Verifications
    • 70
    • 71
    • 72 Execution with SQL Agent Project Conversion Wizard - Allows developers to store packages in SQL Server Database
    • 73 Integration Service Deployment
    • 74 SQL Server Agent Configuration
    • 75
    • 76 File Verification