Prepared By:  Agnes TetterSETFOCUS, LLC07/31/2009SSIS Student ProjectALLWORKS Database Documentation<br /> I. INTRODUCTION...
The following row counts:</li></ul>Total numbers of rows in the source file (Employee Spreadsheet)<br />Total number of ne...
EmployeePK (Employee ID) and EffectiveDate(rate effective date) is not Null,
If both EmployeePK (Employee ID) and EffectiveDate(rate effective date)  don’t exist in the Employee Rate Table.</li></ul>...
The following row counts:</li></ul>Total number of new rows inserted to the Employee Rate Table<br />Total number of exist...
It will not also be inserted if the County Key (CountyPK) does not exist in the County Table. The package will add the cou...
The following row counts:
Total number of rows read from the source file (Client Listing Spreadsheet)</li></ul>Total number of new rows inserted to ...
ClientGroupingPK (Grouping#) from the Special Grouping Sheet is not null</li></ul>Update: <br />The Client Grouping Table ...
The following row counts:
Total number of rows read from the Special Grouping Spreadsheet</li></ul>Total number of new rows inserted to the Client G...
DivisionPK (Division) from the Divison Definition Sheet is not null</li></ul>Update: <br />The Division Table can only be ...
The following row counts:
Total number of rows read from the Division Definition Spreadsheet</li></ul>Total number of new rows inserted to the Divis...
The following row counts:
Total number of rows read from the Special Grouping Spreadsheet</li></ul>Total number of new rows inserted to the ClientGr...
JobMasterPK exist in JobMaster Table and EmployeePK exist in Employee Table</li></ul>Update: <br />The JobTimeSheets Table...
Upcoming SlideShare
Loading in...5
×

Agnes's SSIS Project Documentation

1,573

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,573
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Agnes's SSIS Project Documentation

  1. 1. Prepared By: Agnes TetterSETFOCUS, LLC07/31/2009SSIS Student ProjectALLWORKS Database Documentation<br /> I. INTRODUCTION – Project Mission<br />ALLWORKS is a fictitious construction company. In this project we design and build an SQL Server 2005 Database to track employee and customer information, timesheet and labor rates data, as well as a job order information, job materials, and customer invoices. ALLWORKS stored its information in Excel Spreadsheets, XML, and CSV files. In this project we will make some improvements to the existing data sources in order to support more flexible business practice for customer invoicing, use SQL 2005 Integration Services to integrate these external data sources into SQL Server Database.<br />II. LIST OF ALL PACKAGES<br />PACKAGE 3A: EmployeeMasterPackage.dtsx<br />This package will read the contents of the Employee.xls (under employee’s sheet) that is located from C:SetFocusBISourceData and add them to the AllWorksDBStudent database (Employee Table). <br />Employee sheet contains roaster of employees and flag for whether the employee is a contractor or a regular.<br />Source File: C:SetFocusBISourceDataEmployees.XLS, Sheet: Employees<br />Target Table: dbo.Employees<br />Assumption: <br />Insert: Loading a NULL value in EmployeePK(Employee ID) field is prohibited in Employee Table.<br />Update: The Employee table can only be updated if the Employee ID in Employee Spreadsheet is equal to the EmployeeID (EmployeePK in the Employee Table) and other data are not equal<br />Notification:<br />These include two sets of information and this email sent to biproject@setfocus.com. The email server for SMTP connection is Sfexch0003.setfocus.com<br /><ul><li>Successful email notification will include:
  2. 2. The following row counts:</li></ul>Total numbers of rows in the source file (Employee Spreadsheet)<br />Total number of new rows inserted to the Employee Table<br />Total number of existing rows updated from the Employee Table<br /><ul><li>Failure email notification will include:</li></ul>The name of the package that processing fails.<br />PACKAGE 3b: EmployeeRatePackage.dtsx<br />This package will read the contents of the Employee.xls (under employee rate sheet) that is located from C:SetFocusBISourceData and add them to the AllWorksDBStudent database (Employee Rates Table). <br />Employee Rate Sheet includes Employee Hourly Rates for each employee and the rate effective date.<br />Source File: C:SetFocusBISourceDataEmployees.XLS, Sheet: Employees Rates<br />Target Table: dbo.EmployeeRates<br />Assumption: <br />Insert: <br />You can only insert new rows to the Employee Rate Table if the following conditions are TRUE:<br /><ul><li>EmployeePK exist in the Employee Table
  3. 3. EmployeePK (Employee ID) and EffectiveDate(rate effective date) is not Null,
  4. 4. If both EmployeePK (Employee ID) and EffectiveDate(rate effective date) don’t exist in the Employee Rate Table.</li></ul>Update: <br />The Employee Rate table can only be updated if the TransformedEmployeePK(Employee ID in Employee.XLS) is equal to the TargetEmployeePK (EmployeePK in the Employee Table).<br />Notification:<br />These include two sets of information and this email is sent to biproject@setfocus.com. The email server for SMTP connection is Sfexch0003.setfocus.com<br /><ul><li>Successful email notification will include:
  5. 5. The following row counts:</li></ul>Total number of new rows inserted to the Employee Rate Table<br />Total number of existing rows updated in the Employee Rate Table<br />Total number of invalid rows (those employee that don’t exist in the Employee Table)<br />Failure email notification will include:<br />The name of the package that processing fails.<br />Error Handling:<br />One file is created for all Employees from the Employee Rate Spreadsheet that don’t exist in the Employee Table. This file is save in the C:SetFocusBIProjectsStudentVersionSSISStudentProjectLog<br /> File Name: MissingEmployee_EmployeeRatePackage.csv<br />PACKAGE 3c: ClientMasterPackage.dtsx<br />This package will read the contents of the ClientGeographies.xls (under client listing sheet) that is located from C:SetFocusBISourceData and add them to the AllWorksDBStudent database (Clients Table). <br />Client Listing Sheet includes client for AllWorks, along with a CountyKey to associate the client with a county. The information that needed from the Client Listing spreadsheet are the Client No (ClientPK), Account No, Client Name, Project Manager, County Key (CountyPK), Client Image.<br />This package also loads the data from County Definition Sheet to County Table.<br />Source File: C:SetFocusBISourceData ClientGeographies.xls, Sheet: Client Listing and County Definition<br />Target Table: dbo.Clients and dbo.County<br />Assumption: <br />Insert: <br />You can only insert new rows to the Clients Table if the following conditions are TRUE:<br /><ul><li>The Client No(ClientPK) from the Client Listing Spreadsheet is not empty and if it has no same value in the Client Table (SQL Server DB).
  6. 6. It will not also be inserted if the County Key (CountyPK) does not exist in the County Table. The package will add the county that doesn’t exist in the County first in the County Table.</li></ul>Update: <br />The Client table can only be updated if the Client No (ClientPK) from the Client Listing Spreadsheet is equal to the ClientPK (ClientNo) and if it exist in the Clients Table and the other information that need to be inserted are not equal.<br />Notification:<br />These include two sets of information and this email is sent to biproject@setfocus.com. The email server for SMTP connection is Sfexch0003.setfocus.com<br /><ul><li>Successful email notification will include:
  7. 7. The following row counts:
  8. 8. Total number of rows read from the source file (Client Listing Spreadsheet)</li></ul>Total number of new rows inserted to the Client Table<br />Total number of existing rows updated to the Client Table<br />Total number of invalid rows (those county that don’t exist in the County Table)<br />Failure email notification will include:<br />The name of the package that processing fails.<br />Error Handling:<br />One file is created for all Counties from the Client Listing Spreadsheet that don’t exist in the County Table. This file is save in the C:SetFocusBIProjectsStudentVersionSSISStudentProjectLog<br /> File Name: MissingCounty_ClientPackage.csv<br />PACKAGE 3d: ClientGroupingPackage.dtsx<br />This package will read the contents of the Clientgeographies.xls (under special grouping sheet) that is located from C:SetFocusBISourceData and add them to the AllWorksDBStudent database (Client Grouping Table). <br />Special Grouping Sheet includes Grouping# (ClientGroupingPK), GroupingName, and Account Key. In this package we only need the first two data. In this package we use aggregrate data flow task to group the repeating grouping# and GroupingName.<br />Source File: C:SetFocusBISourceData Clientgeographies.xls, Sheet: Special Grouping<br />Target Table: dbo.ClientGroupings <br />Assumption: <br />Insert: <br />You can only insert new rows to the Client Grouping Table if the following conditions are TRUE:<br /><ul><li>ClientGroupingPK (Grouping#) don’t exist in the Client Grouping Table
  9. 9. ClientGroupingPK (Grouping#) from the Special Grouping Sheet is not null</li></ul>Update: <br />The Client Grouping Table can only be updated if the Grouping#(ClientGroupingPK) from the Special Grouping sheet is equal to the ClientGroupingPK in the Client Grouping Table and the GroupingName is different.<br />Notification:<br />These include two sets of information and this email is sent to biproject@setfocus.com. The email server for SMTP connection is Sfexch0003.setfocus.com<br /><ul><li>Successful email notification will include:
  10. 10. The following row counts:
  11. 11. Total number of rows read from the Special Grouping Spreadsheet</li></ul>Total number of new rows inserted to the Client Grouping Table<br />Total number of existing rows updated to the Client Grouping Table<br />Failure email notification will include:<br />The name of the package that processing fails.<br />PACKAGE 3e: DivisionMasterPackage.dtsx<br />This package will read the contents of the Clientgeographies.xls (under Division Definition sheet) that is located from C:SetFocusBISourceData and add them to the AllWorksDBStudent database (Division Table). <br />Division Definition Sheet includes Division (DivisionPK), Description (DivisionName), and Account Key. In this package we only need the first two data. In this package we use aggregrate data flow task to group the repeating Division and Description.<br />Source File: C:SetFocusBISourceData Clientgeographies.xls, Sheet: Division Definition<br />Target Table: dbo.Division<br />Assumption: <br />Insert: <br />You can only insert new rows to the Division Table if the following conditions are TRUE:<br /><ul><li>DivisionPK (Division) don’t exist in the Division Table
  12. 12. DivisionPK (Division) from the Divison Definition Sheet is not null</li></ul>Update: <br />The Division Table can only be updated if the Division ( DivisionPK) from the Division Definition sheet is equal to the DivisionPK in the Client Grouping Table and the Description(DivisionName) is different.<br />Notification:<br />These include two sets of information and this email is sent to biproject@setfocus.com. The email server for SMTP connection is Sfexch0003.setfocus.com<br /><ul><li>Successful email notification will include:
  13. 13. The following row counts:
  14. 14. Total number of rows read from the Division Definition Spreadsheet</li></ul>Total number of new rows inserted to the Division Table<br />Total number of existing rows updated to the Division Table<br />Failure email notification will include:<br />The name of the package that processing fails.<br />PACKAGE 3f: ClientGroupingXrefPackage.dtsx<br />This package will read the contents of the Clientgeographies.xls (under Special Grouping sheet) that is located from C:SetFocusBISourceData and add them to the AllWorksDBStudent database (ClientGroupingXClients Table). <br />Special Grouping Sheet includes Grouping# (ClientGroupingPK), GroupingName, and Account Key. In this package we use three lookups one for the Client, ClientGrouping, and ClientGroupingXClients Tables. The first ttwo lookups will validate that each ClientNo(ClientPK) and Grouping#(ClientGroupingPK) exist in Client Table and ClientGroupings Table respectively before it will be inserted in the ClientGroupingXClients.<br />Source File: C:SetFocusBISourceData Clientgeographies.xls, Sheet: Special Grouping<br />Target Table: dbo.ClientGroupingsXClients<br />Assumption: <br />Insert: <br />You can only insert new rows to the ClientGroupingsXClients Table if the following conditions are TRUE:<br /><ul><li>ClientGroupingPK (Grouping#) exist in Client Grouping Table and the ClientPK(ClientNo) exist in Client Table.</li></ul>Notification:<br />These include two sets of information and this email is sent to biproject@setfocus.com. The email server for SMTP connection is Sfexch0003.setfocus.com<br /><ul><li>Successful email notification will include:
  15. 15. The following row counts:
  16. 16. Total number of rows read from the Special Grouping Spreadsheet</li></ul>Total number of new rows inserted to the ClientGroupingsXCLients<br />Failure email notification will include:<br />The name of the package that processing fails.<br />PACKAGE 3g: ProjectJobMasterPackage.dtsx<br />This package will read the contents of the ProjectMaster.xls (under Project Master sheet) that is located from C:SetFocusBISourceData and add them to the AllWorksDBStudent database (JobMaster Table). All the columns from Project Master Sheet will be loaded to the JobMaster Table. <br />Source File: C:SetFocusBISourceData ProjectMaster.xls, Sheet: Project Master<br />Target Table: dbo.JobMaster<br />Assumption: <br />Insert: <br />You can only insert new rows to the JobMaster Table if the JobMasterPK (ProjectID) from ProjectMaster.xls is not null.<br />Update: <br />The JobMaster Table can only be updated if the Project ID (JobMasterPK) from the ProjectMaster sheet is equal to the JobMasterPK in the JobMaster Table and other rows are not equal.<br />Error Handling:<br />One file is created for all Clients from the Project Master Spreadsheet that don’t exist in the Clients Table. This file is save in the C:SetFocusBIProjectsStudentVersionSSISStudentProjectLog<br /> File Name: MissingClient_ProjectJobMasterPackage.csv<br />Notification:<br />These include two sets of information and this email is sent to biproject@setfocus.com. The email server for SMTP connection is Sfexch0003.setfocus.com<br /><ul><li>Successful email notification will include:</li></ul>The following row counts:<br />Total number of new rows inserted to the JobMaster Table<br />Total number of existing rows updated to the JobMasterTable<br />Total number of invalid rows (those client that don’t exist in the Client Table)<br />Failure email notification will include:<br />The name of the package that processing fails.<br />PACKAGE 3h: JobTimeSheetPackage.dtsx<br />This package used a ForEachLoop Container to read EmpTime####.csv files that were stored at C:SetFocusBISourceData ime and add them to the AllWorksDBStudent database (JobTimeSheets Table). This package also used Script Task to count the total rows that being processed from each file.<br />Source Folder: C:SetFocusBISourceData ime FileNames: EmpTime500.csv, EmpTime1000.csv, EmpTime2000.csv, EmpTime3000.csv, EmpTime4000.csv<br />Target Table: dbo.JobTimeSheet<br />Assumption: <br />Insert: <br />You can only insert new rows to the JobTimeSheets Table if the following conditions are TRUE:<br /><ul><li>JobMasterPK (Job Number), EmployeePK(Employee ID), and WorkDate(Date) from EmpTime####.csv is not null.
  17. 17. JobMasterPK exist in JobMaster Table and EmployeePK exist in Employee Table</li></ul>Update: <br />The JobTimeSheets Table can only be updated if the JobMasterPK (Job Number), EmployeePK(Employee ID), and WorkDate(Date) from Emptime####.csv is equal to the JobMasterPK, EmployeePK and WorkDate in the JobTimeSheets Table and other rows are not equal.<br />Error Handling:<br />Three file s are created:<br /><ul><li>MissingEmployee_JobTimeSheetPackage.csv for all Employees that don’t exist in Employee Table.
  18. 18. MissingJob_JobTimeSheetPackage.csv for all Jobs that don’t exist in JobMaster Table
  19. 19. InvalidTimeSheet_JobTimeSheetPackage.csv for all the job that has been closed (workdate exceeds with the jobclosedate</li></ul>These files were saved in C:SetFocusBIProjectsStudentVersionSSISStudentProjectLog<br />Notification:<br />These include two sets of information and this email is sent to biproject@setfocus.com. The email server for SMTP connection is Sfexch0003.setfocus.com<br /><ul><li>Successful email notification will include:</li></ul>The following row counts:<br /><ul><li>Total Rows that were read from the EmpTime####.csv files under Time Folder
  20. 20. Total Rows that were inserted to the JobTimeSheets Table from EmpTime####.csv files under Time Folder
  21. 21. Total Rows that were updated to the JobTimeSheets Table from EmpTime####.csv Files under Time Folder
  22. 22. Total Rows that were invalid from EmpTime####.csv Files under Time Folder</li></ul> (These are the Timesheet that WorkDate is greater than the JobClosedDate)<br />Failure email notification will include:<br />The name of the package that processing fails.<br />PACKAGE 3i: MasterPackage.dtsx<br />This package used Execute Package Task and grouped all the eight packages by using the sequence container. The packages were joined based on their dependencies. <br />The following four tasks have been setup to run after the MasterPackage.dtsx has successfully completed. <br /><ul><li>Back Up Database Task
  23. 23. Shrink Database Task
  24. 24. Rebuild Index Task
  25. 25. Send Mail Notification
  26. 26. Failure Email Notification if either of the database maintenance fails.
  27. 27. Success Email Notification if the database maintenance has been successfully performs.</li></ul>The MasterPackage used the SQL Agent Job to scheduled and run daily at 12 midnight.<br />

×