Your SlideShare is downloading. ×
  • Like
  • Save
Bilir's Business Intelligence Portfolio SSIS Project
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Bilir's Business Intelligence Portfolio SSIS Project

  • 1,163 views
Published

Business Intelligence Portfolio SSIS Project

Business Intelligence Portfolio SSIS Project

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,163
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. 3/9/2010
    Figen Bilir ©
    1
  • 2. Project Overview: AllWorks
    The SSIS project was for a fictitious construction company called AllWorks. Project was designed and build a SQL Server 2005 database to track employee and customer information, timesheet and labor rates data, as well as job order information, job materials, and customer invoices. In the client project scenario, AllWorks currently stores this information in Excel Spreadsheets, XML files, and CSV files.
    Extract data from diverse files (*.csv, *.xls, etc.)
    Transform data as required by business and SQL database requirements
    Load data into a local SQL AllWorksDBStudent database
    3/9/2010
    Figen Bilir ©
    2
  • 3. Database Diagram
    3/9/2010
    Figen Bilir ©
    3
  • 4. Source Data Review
    3/9/2010
    Figen Bilir ©
    4
  • 5. Employee Master Package
    This package loads the employee data from an Excel Spreadsheet, Employees.XLS. Data conversion takes place to verify that the data is in a useable format. A log is created to quantify the number of updates and inserts which are sent to a specific destination.
    The records are sent to an email recipient who will tell them if the package was a success along with the package name, user who performed action and record results. In the event of a failure of the package during execution an error would be sent to specific address alerting them of the failure.
    3/9/2010
    Figen Bilir ©
    5
  • 6. Sendmail Setup & Outcome
    3/9/2010
    Figen Bilir ©
    6
  • 7. Employee Rate Package
    This package loads the employee rate data from an Excel Spreadsheet, Employees.XLS. Data conversion takes place to verify that the data is in a useable format. All employee records are validated in Lookup task, invalid ones are logged to a CSV file. A log is created to quantify the number of updates, inserts and invalid records which are sent to a specific destination.
    The records are sent to an email recipient who will tell them if the package was a success along with the package name, user who performed action and record results. In the event of a failure of the package during execution an error would be sent to specific address alerting them of the failure.
    3/9/2010
    Figen Bilir ©
    7
  • 8. Client Master Package
    Due to the constraint between dbo.Client and dbo.County tables based on CountyPK as a FK on Clients table, first County table has been populated to the database. That’s why Data Flow for County Data was generated first, and then Data Flow Client Data has been achieved.
    From the source file for the current package, County Definitions Worksheet within the same file has been utilized in order to populate dbo.County table in the database.
    3/9/2010
    Figen Bilir ©
    8
  • 9. Client Master Package Cont’d
    Invalidated CountyIDs has been set up to be written to the Log File above based on check from the County table since it already runs prior to Data Flow Client Data.
    3/9/2010
    Figen Bilir ©
    9
  • 10. Client Groupings Master Package
    The Clientgeographies.XLS spreadsheet has been in normalized form, after the data conversion, the incoming data has been aggregated based on groupingno and groupingname.
    3/9/2010
    Figen Bilir ©
    10
  • 11. Client Groupings to Client Xref Table Package
    There are 3 lookups to validate the incoming data. First, ClientID is checked to validate the AccountKey, then, GroupingNo is validated in the ClientsGrouping and lastly GroupingID and ClientID is validated against ClientGroupingsXClients table in order to handle to insert in the next step.
    3/9/2010
    Figen Bilir ©
    11
  • 12. Project Job Master Package
    Due to the FK relationship between Clients and JobMaster Table, ClientPK has to be validated with a lookup. If there are any invalid Clients, they are written to the CSV Log file specified in Flat File Connection Manager.
    3/9/2010
    Figen Bilir ©
    12
  • 13. Project Job Time Sheets (Labor) Package
    3/9/2010
    Figen Bilir ©
    13
    This package loads data from several CSV JobTimeSheet files into SQL Server 2005 database. The content of its Data Flow task is shown on the next two slides.
  • 14. Project Job Time Sheets (Labor) Package Cont’d
    In the data flow, data is read from the CSV files each time and EmployeeID and JobMasterID is validated with the Lookups from Employee and JobMaster Tables respectively after data conversion take places.
    3/9/2010
    Figen Bilir ©
    14
  • 15. Project Job Time Sheets (Labor) Package Cont’d
    Package reads the TimeSheet data from several CSV files and inserts new rows or update existing row if data is different.
    Data Flow for Load Job Time Sheet control flow has a Foreach Loop Container that loops through each file and processes the job time sheet files. In order to accumulate the row count a script has been written to for the total row counts and file counts.
    3/9/2010
    Figen Bilir ©
    15
  • 16. Project Job Time Sheets (Labor) Package Cont’d
    The script is written to accumulate the total count for given variables in ReadOnly and ReadWrite Variables are used in the definition of the mail message.
    3/9/2010
    Figen Bilir ©
    16
  • 17. Master Package
    This is the main package that launches the execution of all ETL packages from SQL Server 2005.
    After the successful execution of all ETL packages, it launches the database maintenance tasks starting with the database shrinking, indexes building, statistics update and database backup task.
    Upon completion the whole package a notification of the successful email is sent. If any maintenance task fails, “Unsuccessful Email” is sent from each task.
    3/9/2010
    Figen Bilir ©
    17
  • 18. SQL Server Agent Job
    All packages were deployed to the (local) SQL Server and a job -Execute SSIS Student Project- in SQL Server Agent was set up to run this Master Package nightly at 12:00AM.
    3/9/2010
    Figen Bilir ©
    18