SQL Server Integration Services –
       Enterprise Manageability


                       David Pendleton
Dan English
   ...
Who are we? – Dan and David
             Dan English                                            David Pendleton
          ...
Who are we? – Magenic
 Founded in 1995, Magenic is a technical consulting firm
    focused exclusively on Microsoft techn...
Today‟s Agenda
• SQL Server Integration Services Introduction and Architecture
   •8:30 to 9 (Dan English)

• Team Develop...
Quick Audience Poll
•How many DBAs? SQL/BI Developers? Data Architects? Others?

•How many are currently evaluating SSIS?
...
Today‟s Agenda
• SQL Server Integration Services Introduction and Architecture
    •8:30 to 9 (Dan English)

• Team Develo...
SSIS Introduction

SQL Server 6.5
                                    SQL Server                       SQL Server
• Transf...
Microsoft BI Tool Offerings
                                   DELIVERY

                                SharePoint Server...
SSIS Introduction – BIDS
SSIS Introduction
What can you use SSIS for?

1. Extract, Transform, Load data from heterogeneous sources
2. Combine, sani...
SSIS Architecture
SSIS is made up of two core components:

1. Data Transformation Run-time engine
     • Handles the contr...
SSIS Architecture
Data Transformation Run-time engine

• Provides services to the packages and the
  components

    •   H...
SSIS Architecture

 •The „package‟ is the core component of SSIS. SSIS
 packages are the basic unit of deployment and exec...
SSIS Architecture

What makes up a
package?

1.Control Flow
2.Precedence constraints
3.Data Flow


• Control Flow
   • Con...
SSIS Architecture
• Precedence constraints
    • Controls order of execution
    • Success, Failure,
      Completion
    ...
SSIS Architecture

• Data Flow
   • Source adapters
   • Transformations
   • Destination adapters
SSIS Architecture
SSIS Architecture Enhancements
• What‟s new in SSIS 2008?
  – Improved Scripting with VSTA including support for C#
  – Da...
SSIS Architecture Enhancements



     Data Profiling Demo
Today‟s Agenda
• SQL Server Integration Services Introduction and Architecture
    •8:30 to 9 (Dan English)

• Team Develo...
SSIS Enterprise Manageability
 • SSIS Package Deployment
   – The SSIS runtime environment
   – Package deployment
   – Se...
The SSIS runtime environment

 • Does not require SQL Server to run*
   – Although SQL Server is probably involved somewhe...
Package Deployment

 • In general, there are two ways to deploy a
   package to a server
   – File System
   – SQL Server ...
Package Deployment

 • Deployment option advantages compared
   – File System
      • OS-based file encryption and ACLs
  ...
Package Deployment
 • Creating the Package Deployment Utility
   – Accessed via Project Properties in BIDS
Securing your packages
 • SSIS offers these options for securing packages
   – Setting the ProtectionLevel property of a p...
Securing your packages
 • The package ProtectionLevel property
   – Determines the way package content is secured
      • ...
Securing your packages
 • DontSaveSensitive
   – Removes sensitive information from the package during the
     save, e.g....
Securing your packages
 • EncryptAllWithUserKey
   – Encrypts the entire package by applying a user-specific key
     secu...
Securing your packages
 • EncryptSensitiveWithUserKey
   – Encrypts sensitive information in the package using a user-
   ...
Securing your packages
 • EncryptAllWithPassword
   – Encrypts the entire package with a user-supplied password
   – Passw...
Securing your packages
 • EncryptSensitiveWithPassword
   – Encrypts sensitive in formation in the package using a user-
 ...
SSIS package Configurations
 • Configurations allow you to modify package
   property and variable values at load time
 • ...
SSIS Package Configurations
 • SQL configuration
   –   Configurations are stored in a SQL Server table
   –   You may cal...
SSIS Package Configurations
 • Registry configuration
   – Configurations are stored in a registry key
   – Restricted to ...
SSIS Package Configurations
 • Understanding how configurations are applied
   – SSIS 2005
      1.   The package is loade...
SSIS Package Configurations
 • Understanding how configurations are applied
   – SSIS 2008
      1.   The package is loade...
Team Development and SSIS
 • Use a source control system
   – SSIS packages are stored as XML, but are effectively binary ...
Team Development and SSIS
 • Design your ETL process with team
   development in mind.
   – One package; one table
   – Pa...
Team Development and SSIS
 • Create and use package templates
   – A template is simply an SSIS package
   – Any package c...
Team Development and SSIS
 • Security considerations
   – Use Windows Authentication whenever possible, both in
     devel...
Team Development and SSIS
 • Deploy your packages to the File System
 • Use a common folder structure
   – Provides a cons...
Team Development and SSIS
 • Common folder structure: the SSIS Root Folder
   ..SSIS
   ..SSISCheckpointFiles
   ..SSISCon...
Team Development and SSIS
 • Machine-specific Configuration file
   – XML configuration
      • Accessed via indirect conf...
Team Development and SSIS
 • SQL Server Configuration
   – Easily managed using DDL
   – Adding an identity column helps w...
Demo: Tying it all together
 • Creating the common folder structure
 • Redirecting the SSIS Package Store
 • Implementing ...
Today‟s Agenda
• SQL Server Integration Services Introduction and Architecture
    •8:30 to 9 (Dan English)

• Team Develo...
SSIS Security
 •   Security in SQL Server Integration Services consists of several layers that provide a
     rich and fle...
SSIS Security – database roles
 •   SQL Server Integration Services includes the three fixed database-
     level roles fo...
SSIS Manageability - SSMS

 •   The Integration Services server
     node presents two folders for
     working with SSIS ...
SSIS Manageability - SSMS

 •   The Stored Packages folder lists
     the saved SSIS packages that
     have been register...
SSIS Manageability - SSMS
•   Right-clicking a package displays a shortcut menu
    that enables you to perform a number o...
SSIS Manageability - dtutil
•   SSIS Package Management
    Utility (dtutil)

    SSIS contains the Package
    Management...
SSIS Manageability - dtutil
 •   To copy a package that is stored in the msdb database on a local instance of SQL Server u...
SSIS Manageability - dtutil
 •   To determine whether a package exists in the msdb database on a local instance of SQL Ser...
SSIS Manageability



       Stored Package
      Configuration Demo
SSIS Manageability - package
 •   Checkpoint Restart – recover from a package failure; information about the packages exec...
SSIS Manageability - package



    Event Handlers Demo
SSIS Manageability - package

Create smaller packages
•   Pros
     – less complex child
       packages
     – call them ...
SSIS Manageability – package execution

 Scheduling Package Execution
   – SQL Server Agent
      •   Requires SQL Server ...
SSIS Manageability – package execution



    SQL Server Agent Job
           Demo
SSIS Manageability – package execution

 Remote Package Execution
   –   SQL Server Agent job (sp_start_job)
   –   Execut...
SSIS Manageability – package execution



      Remote Package
      Execution Demo
SSIS Manageability – package logging
•   The SQL Profiler Log Provider - The SQL Profiler log provider
    enables you to ...
SSIS Manageability – package logging
Events                   Description

OnError                  Writes a log entry whe...
SSIS Manageability – package logging
SSIS Monitoring

  •SQL Server Management Studio
  •Logging
  •Notifications
  •Performance Monitor
    •SSIS Service
    ...
Today‟s Agenda
• SQL Server Integration Services Introduction and Architecture

• Team Development, Deployment and Configu...
Q&A



      Any Questions?
Resources
 SQL Server 2008 – Integration Services Portal Page
 http://msdn.microsoft.com/en-us/sqlserver/cc511477.aspx

 S...
Contact Information – Thank You!

    Contact us to find out how your business can benefit
    from a complimentary strate...
Upcoming SlideShare
Loading in...5
×

SQL Server Integration Services – Enterprise Manageability

4,290

Published on

Talking about SQL Server Integration Services (SSIS) overview, deployment, security, and manageability

Published in: Technology, Business
0 Comments
15 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,290
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
15
Embeds 0
No embeds

No notes for slide

SQL Server Integration Services – Enterprise Manageability

  1. 1. SQL Server Integration Services – Enterprise Manageability David Pendleton Dan English Senior Consultant Principal Consultant DavidPe@magenic.com DanE@magenic.com
  2. 2. Who are we? – Dan and David Dan English David Pendleton http://denglishbi.spaces.live.com/ • Developing with Microsoft technologies for over 10 years • 20+ years experience as a systems analyst, • database developer, designer and software Over 5 years experience with Data Warehousing and Business Intelligence engineer • • Experienced in ETL and Analysis Services Experienced technician with strengths in development, requirements gathering and data database design and development, systems modeling architecture and software development • • Microsoft Certified IT Professional (MCITP) and Specialized in ETL, database modeling and Microsoft Certified Technology Specialist (MCTS) design, SQL development, and change management
  3. 3. Who are we? – Magenic  Founded in 1995, Magenic is a technical consulting firm focused exclusively on Microsoft technologies and has designed and delivered more than 500 Microsoft-based applications  Headquartered in Minneapolis, with offices in Chicago, Boston, Atlanta and San Francisco  2005 Microsoft Partner of the Year, Custom Development Solutions – Technical Innovation  2007 Microsoft Partner of the Year Finalist, Data Management  Microsoft Gold Certified Partner and National Systems Integrator  40 Enterprise Data Services (EDS) consultants
  4. 4. Today‟s Agenda • SQL Server Integration Services Introduction and Architecture •8:30 to 9 (Dan English) • Team Development, Deployment and Configuration •9:15 to 10:30 (David Pendleton) • Security and Manageability •10:45 to 11:30 (Dan English) • Summary and Q&A
  5. 5. Quick Audience Poll •How many DBAs? SQL/BI Developers? Data Architects? Others? •How many are currently evaluating SSIS? •How many are currently using SSIS in production? •How many are currently using SSIS 2008? •Is SSIS being used for something besides DB Maintenance or ETL? •If so what? •Anyone still using DTS?
  6. 6. Today‟s Agenda • SQL Server Integration Services Introduction and Architecture •8:30 to 9 (Dan English) • Team Development, Deployment and Configuration •9:15 to 10:30 (David Pendleton) • Security and Manageability •10:45 to 11:30 (Dan English) • Summary and Q&A
  7. 7. SSIS Introduction SQL Server 6.5 SQL Server SQL Server • Transfer 2000 2008 Manager • DTS • SSIS • Bulk Copy Enhancements Enhancements Program SQL Server SQL Server 7.0 2005 • Data • Integration Transformation Services Services
  8. 8. Microsoft BI Tool Offerings DELIVERY SharePoint Server Analytic Excel Scorecards Plans Reports Dashboards Views Workbooks END USER TOOLS & PERFORMANCE MANAGEMENT APPS Excel PerformancePointServer BI PLATFORM SQL Server SQL Server Reporting Services Analysis Services SQL Server DBMS SQL Server Integration Services
  9. 9. SSIS Introduction – BIDS
  10. 10. SSIS Introduction What can you use SSIS for? 1. Extract, Transform, Load data from heterogeneous sources 2. Combine, sanitize, standardize, split, and summarize data 3. Convert data to different data types and put into different formats and sources 4. Database maintenance and transfer operations • Backup, check integrity, execute jobs, cleanup, rebuild indexes, update statistics, etc. 5. Transfer SQL Server database objects 6. Workflow engine supporting complex logic and precedence constraints 7. Perform Analysis Services processing and Data Mining querying 8. Use as a data source for Reporting Services report 9. Perform FTP operations 10. Extensible to create your own components (sources, transforms, control flow, etc.) 11. Utilize scripting tasks and access .NET code with VSTA (Visual Basic .NET or C#) 12. Send email notifications 13. Access Web Services 14. File archiving and maintenance tasks 15. And much, much more…
  11. 11. SSIS Architecture SSIS is made up of two core components: 1. Data Transformation Run-time engine • Handles the control flow of a package 2. Data Flow engine • Data Transformation Pipeline engine • Data Sources • Transformations • Destination Targets
  12. 12. SSIS Architecture Data Transformation Run-time engine • Provides services to the packages and the components • Handle the storage • Execution • Logging • Debugging • Event handling • Deployment • Variable management • Transactions • Connections
  13. 13. SSIS Architecture •The „package‟ is the core component of SSIS. SSIS packages are the basic unit of deployment and execution. •An SSIS package is XML. •Packages can be created using BIDS or programmatically.
  14. 14. SSIS Architecture What makes up a package? 1.Control Flow 2.Precedence constraints 3.Data Flow • Control Flow • Containers • Tasks
  15. 15. SSIS Architecture • Precedence constraints • Controls order of execution • Success, Failure, Completion • Expressions • AND/OR logic
  16. 16. SSIS Architecture • Data Flow • Source adapters • Transformations • Destination adapters
  17. 17. SSIS Architecture
  18. 18. SSIS Architecture Enhancements • What‟s new in SSIS 2008? – Improved Scripting with VSTA including support for C# – Data Flow Task performance improvements with smarter thread allocation and processing of execution trees – ADO NET source and destination adapters – Import/Export Wizard enhancements • ADO.Net support, data conversions (XML file), performance improvements – Cache Transform and Cache Connection Manager for Lookup Transform (Lookup Transform improved also – no match output) – Utilize CDC and Merge for incremental updates – Data Profiling Task (and Viewer)
  19. 19. SSIS Architecture Enhancements Data Profiling Demo
  20. 20. Today‟s Agenda • SQL Server Integration Services Introduction and Architecture •8:30 to 9 (Dan English) • Team Development, Deployment and Configuration •9:15 to 10:30 (David Pendleton) • Security and Manageability •10:45 to 11:30 (Dan English) • Summary and Q&A
  21. 21. SSIS Enterprise Manageability • SSIS Package Deployment – The SSIS runtime environment – Package deployment – Securing packages • SSIS Configurations – Understanding and using SSIS package configurations • Team Development using SSIS – SSIS package development in a team environment • Demo: Tying it all together
  22. 22. The SSIS runtime environment • Does not require SQL Server to run* – Although SQL Server is probably involved somewhere • Does not require the SSIS Service to run – The SSIS service enables package management and monitoring via SSMS • Any server can host SSIS packages – Provided the minimum requirements are met • .NET Framework, SQL Native Client, Setup support files • Installed using SQL Server Setup
  23. 23. Package Deployment • In general, there are two ways to deploy a package to a server – File System – SQL Server (msdb..sysdtspackages90) • Package Deployment Utility – Configured in BIDS, created by the build process – Uses the .SSISDeploymentManifest file extension • Deployment can also be done “manually”, e.g. Xcopy, SMSS import
  24. 24. Package Deployment • Deployment option advantages compared – File System • OS-based file encryption and ACLs • Not subject to SQL Server availability • Easier direct access for viewing or editing • Generally easier on the developer – SQL Server • Easier access by multiple individuals • Benefits from database security, roles and Agent interaction • Packages get backed up with normal database backup processes • Generally easier on the DBA
  25. 25. Package Deployment • Creating the Package Deployment Utility – Accessed via Project Properties in BIDS
  26. 26. Securing your packages • SSIS offers these options for securing packages – Setting the ProtectionLevel property of a package – Setting the ProtectionLevel and PackagePassword properties of a package – Controlling access by using SQL Server database-level roles – Securing the operational environment by controlling access to file locations – Guaranteeing the integrity of a package by signing with a certificate.
  27. 27. Securing your packages • The package ProtectionLevel property – Determines the way package content is secured • DontSaveSensitive • EncryptSensitiveWithUserKey • EncryptSensitiveWithPassword • EncryptAllWithPassword • EncryptAllWithUserKey • ServerStorage
  28. 28. Securing your packages • DontSaveSensitive – Removes sensitive information from the package during the save, e.g. passwords – When the package is opened, the removed items will have to be provided – Scheduled or unattended execution will likely fail – Actually your best option for most development
  29. 29. Securing your packages • EncryptAllWithUserKey – Encrypts the entire package by applying a user-specific key securely stored in the user‟s profile – The package can only be opened by the same user with access to the same user profile. – Guaranteed to make you unpopular with your team members
  30. 30. Securing your packages • EncryptSensitiveWithUserKey – Encrypts sensitive information in the package using a user- specific key securely stored in the user‟s profile – The package may be edited by other users, but the sensitive information will be automatically removed – Execution without access to the encryption key will fail – This is the default level
  31. 31. Securing your packages • EncryptAllWithPassword – Encrypts the entire package with a user-supplied password – Password is required to open or edit – Requires the /De[crypt] password option to run with DTEXEC
  32. 32. Securing your packages • EncryptSensitiveWithPassword – Encrypts sensitive in formation in the package using a user- supplied password – Package may be edited without the password, but sensitive information is automatically removed – Requires the /De[crypt] password option to run with DTEXEC
  33. 33. SSIS package Configurations • Configurations allow you to modify package property and variable values at load time • Available in several types – SQL – XML – Registry – Environment Variables – Parent Package • You can mix-and-match configurations in a package
  34. 34. SSIS Package Configurations • SQL configuration – Configurations are stored in a SQL Server table – You may call the table anything you like – May reside in any database, on any SQL Server – Contains one or more configurations • XML configuration – Plain-text XML file – Contains one or more configurations – Uses the file extension .dtsConfig by default – Easily deployed, easily shared
  35. 35. SSIS Package Configurations • Registry configuration – Configurations are stored in a registry key – Restricted to the HKEY_CURRENT_USER hive – Property path is stored in the package • Environment Variable configuration – Package property value is stored in an environment variable – Property path is stored in the package – Not the same as an Indirect Configuration* • Parent Package configuration – Used to fetch a value from a variable in a calling package
  36. 36. SSIS Package Configurations • Understanding how configurations are applied – SSIS 2005 1. The package is loaded 2. Configurations specified at design time are applied in the order they appear in the package 3. Command-line configuration values are applied 4. Parent Package variable configuration values are applied 5. The package is run
  37. 37. SSIS Package Configurations • Understanding how configurations are applied – SSIS 2008 1. The package is loaded 2. The configurations specified at design time are applied in the order they appear in the package 3. Command-line configuration values are applied 4. The configurations specified at design time are reloaded in the order specified in the package 5. Parent Package variable configuration values are applied 6. The package is run
  38. 38. Team Development and SSIS • Use a source control system – SSIS packages are stored as XML, but are effectively binary files – Merging packages is difficult, if not impossible – Only one developer can work on a package at a time • Require exclusive check-outs – Both Visual SourceSafe and Team Foundation Server will permit shared check-outs; this should be disabled • Perform regular source control operations – Ensure you are always working with the latest version of a package, check-in incrementally
  39. 39. Team Development and SSIS • Design your ETL process with team development in mind. – One package; one table – Package functionality should be as granular as is practical – Use Parent packages and the Execute Package Task to group packages together for end-to-end execution – Spread the work around • Your packages • Your team
  40. 40. Team Development and SSIS • Create and use package templates – A template is simply an SSIS package – Any package can be used as a template – Templates can be directly visible to BIDS, or not • A template in the DataTransformationItems template folder will appear in the Add|New Item dialog – Implement and reinforce standard practices in your templates • Logging • Checkpointing • Configurations • Annotations
  41. 41. Team Development and SSIS • Security considerations – Use Windows Authentication whenever possible, both in development and production – Manage the package ProtectionLevel property • ProtectionLevel = DontSaveSensitive – Maximum flexibility – Sensitive information can be managed in SQL configurations
  42. 42. Team Development and SSIS • Deploy your packages to the File System • Use a common folder structure – Provides a consistent, known file location regardless of the server or environment – All files of a certain type are stored in the same place – Allows you to make all locations dynamic within your package – Easier to manage
  43. 43. Team Development and SSIS • Common folder structure: the SSIS Root Folder ..SSIS ..SSISCheckpointFiles ..SSISConfigurations ..SSISErrorFiles ..SSISLogFiles ..SSISPackages ..SSISRawFiles • Use project-specific sub-folders to group related files below the root folders • Edit MsDtsSrvr.ini.xml to use Packages as the SSIS Package Store
  44. 44. Team Development and SSIS • Machine-specific Configuration file – XML configuration • Accessed via indirect configuration – Only one configuration file per machine – Contains two (and only two) configurations • RootFolder – Contains the path to the root level of the custom folder structure • ConfigurationServer – Contains the connection string to the SQL configuration server
  45. 45. Team Development and SSIS • SQL Server Configuration – Easily managed using DDL – Adding an identity column helps with DML tasks • ConfigurationID int IDENTITY(1,1) NOT NULL – Store multiple configurations in the same table – Allows multiple packages to access the same configurations – All the benefits of SQL Server security • Not only sensitive information, but all information can be protected – Offers the maximum flexibility
  46. 46. Demo: Tying it all together • Creating the common folder structure • Redirecting the SSIS Package Store • Implementing the machine-specific configuration file • Implementing the SQL configuration database and table(s) • Leverage the common folder structure in a template using variables and expressions • Making a template available to BIDS
  47. 47. Today‟s Agenda • SQL Server Integration Services Introduction and Architecture •8:30 to 9 (Dan English) • Team Development, Deployment and Configuration •9:15 to 10:30 (David Pendleton) • Security and Manageability •10:45 to 11:30 (Dan English) • Summary and Q&A
  48. 48. SSIS Security • Security in SQL Server Integration Services consists of several layers that provide a rich and flexible security environment. – These security layers include the use of: • digital signatures • package properties • SQL Server database roles • and operating system permissions • Security categories - Identity or Access control – Identity control goal: Ensure that you only open and run packages from trusted sources. (digital signatures) – Access control goal: Ensure that only authorized users open and run packages. (protection levels, encryption, msdb storage, db roles, file permissions)
  49. 49. SSIS Security – database roles • SQL Server Integration Services includes the three fixed database- level roles for controlling access to packages db_ssisadmin (sysadmin) – can do anything related to all packages 1. db_ssisltduser – can import packages; view, delete, execute, export, and 2. change own package roles db_ssisoperator – can view, execute, export all packages (including SQL 3. Server Agent) and they cannot import packages And Windows administrators – can view execution details and stop all 4. running packages • Roles can be implemented only on packages that are saved to the msdb database in SQL Server. You assign roles to a package using SQL Server Management Studio. The role assignments are saved to the msdb database.
  50. 50. SSIS Manageability - SSMS • The Integration Services server node presents two folders for working with SSIS packages: – the Running Packages folder • The Running Packages folder displays the SSIS packages that are currently executing on the local server – and the Stored Packages Folder.
  51. 51. SSIS Manageability - SSMS • The Stored Packages folder lists the saved SSIS packages that have been registered on the local server. By default this folder contains two subfolders: – the File System Folder – and the MSDB folder
  52. 52. SSIS Manageability - SSMS • Right-clicking a package displays a shortcut menu that enables you to perform a number of task including: • New Folder. Creates a new folder in Object Explorer for displaying packages saved in the file system or in the sysdtapackages90 table. • Import Package. Imports the package from the file system to the msdb database • Export Package. Exports the package from the msdb database to the file system. • Run Package. Executes the package using dtexecui. • Delete. Deletes the package. • Rename. Renames the package.
  53. 53. SSIS Manageability - dtutil • SSIS Package Management Utility (dtutil) SSIS contains the Package Management Utility (dtutil) that you can run from a command line. The dtutil command prompt allows you to manage SSIS packages that are stored in the database or the file system. You may specify to copy, move, delete, or verify the existence of a package using the appropriate command prompt options.
  54. 54. SSIS Manageability - dtutil • To copy a package that is stored in the msdb database on a local instance of SQL Server using Windows Authentication to the SSIS Package Store, use the following syntax: dtutil /SQL srcPackage /COPY DTS;destFolderdestPackage • To copy a package from a location on the File system to another location and give the copy a different name, use the following syntax: dtutil /FILE c:myPackagesmypackage.dtsx /COPY FILE;c:myTestPackagesmynewpackage.dtsx • To copy a package on the local file system to an instance of SQL Server hosted on another computer, use the following syntax: dtutil /FILE c:sourcepkg.dtsx /DestServer <servername> /COPY SQL;destpkgname • To create a new ID for a package after it is copied, use the following syntax: dtutil /I /FILE copiedpkg.dtsx • To create a new ID for all the packages in a specific folder, use the following syntax: for %%f in (C:testSSISPackages*.dtsx) do dtutil.exe /I /FILE %%f
  55. 55. SSIS Manageability - dtutil • To determine whether a package exists in the msdb database on a local instance of SQL Server that uses SQL Server Authentication, use the following syntax: dtutil SQL srcPackage /SOURCEUSER srcUserName /SOURCEPASSWORD *hY$d56b /EXISTS • To move a package that is stored in the msdb database on a local instance of SQL Server that uses SQL Server Authentication to the msdb database on another local instance of SQL Server that uses SQL Server Authentication, use the following syntax: dtutil /SQL srcPackage /SOURCEUSER srcUserName /SOURCEPASSWORD $Hj45jhd@X /MOVE SQL;destPackage /DESTUSER destUserName /DESTPASSWORD !38dsFH@v • To sign a package that is stored in a SQL Server database on a local instance of SQL Server that uses Windows Authentication, use the following syntax: dtutil /FILE srcPackage.dtsx /SIGN FILE;destpkg.dtsx;1767832648918a9d989fdac9819873a91f919 • The following sample encrypts the file-based PackageToEncrypt.dtsx to the file-based EncryptedPackage.dtsx using full package encryption, with a password. The password that is used for the encryption is EncPswd. • dtutil /FILE PackageToEncrypt.dtsx /ENCRYPT file;EncryptedPackage.dtsx;3;EncPswd
  56. 56. SSIS Manageability Stored Package Configuration Demo
  57. 57. SSIS Manageability - package • Checkpoint Restart – recover from a package failure; information about the packages execution and status is written to a checkpoint file. The Data Transformation Run-time engine can process the checkpoint file and the package can be restarted from the container that was being processed at the point of failure • Transaction Support – SSIS provides full support for database transactions. If enabled and the package fails that contains data flow tasks, all of the database update actions performed will be rolled back. • Transformation Error Handling – you can choose to ignore bad data or you can choose to redirect the row to a file or table for review and later processing; setup a central database for a central point of auditing and review. • Event Handlers – provide the ability to raise and handle events raised by containers and tasks. These could be used to send e-mail notifications, create logging information, or remove temporary files or database objects. • Execute Package Tasks – provide ability to run one package from another package; pass information from parent package to child package and setup a master package to control the flow of other packages.
  58. 58. SSIS Manageability - package Event Handlers Demo
  59. 59. SSIS Manageability - package Create smaller packages • Pros – less complex child packages – call them from multiple parent packages – simplifies development process – promotes reusability and manageability
  60. 60. SSIS Manageability – package execution Scheduling Package Execution – SQL Server Agent • Requires SQL Server installation • Setup credentials • Setup proxies • Create SQL job – Windows Task Scheduler • Setup user account for task – 3rd party scheduler
  61. 61. SSIS Manageability – package execution SQL Server Agent Job Demo
  62. 62. SSIS Manageability – package execution Remote Package Execution – SQL Server Agent job (sp_start_job) – Execute SQL Server Agent Job task (SSIS task) – .NET Code (SSIS Object Model) – Web Service (IIS – ASP.NET) – HTTP Endpoint – WCF application
  63. 63. SSIS Manageability – package execution Remote Package Execution Demo
  64. 64. SSIS Manageability – package logging • The SQL Profiler Log Provider - The SQL Profiler log provider enables you to create logs that can be opened using the SQL Profiler. • The SQL Server Log Provider - Using the SQL Server log provider enables you to capture SSIS package run-time data in the sysdtslog90 or sysssislog table of the database. • The Windows Event Log Provider - writes entries to the Application log in the Windows Event log on the local computer. • The XML File Log Provider - writes log files to an XML file and the default file extension is .xml. • The Text File Log Provider - writes log entries to ASCII text files in a comma-separated value (CSV) format. The default file name extension for this provider is .log. • Custom Log Provider - You can create your own custom log provider to obtain information to accommodate your business needs. Using a custom log provider allows you to integrate SSIS logs within your applications.
  65. 65. SSIS Manageability – package logging Events Description OnError Writes a log entry when an error occurs. OnExecStatusChanged Writes a log entry when the execution status of the executable changes. OnInformation Writes a log entry during the validation and execution of an executable to report information. OnPostExecute Writes a log entry immediately after the executable has finished running. OnPostValidate Writes a log entry when the validation of the executable finishes. OnPreExecute Writes a log entry immediately before the executable runs. OnPreValidate Writes a log entry when the validation of the executable starts. OnProgress Writes a log entry when measurable progress is made by the executable. OnQueryCancel Writes a log entry at any juncture in the task processing where it is feasible to cancel execution. OnTaskFailed Writes a log entry when a task fails. OnVariableValueChanged Writes a log entry when the value of a variable changes. OnWarning Writes a log entry when a warning occurs. For each data flow component, writes a log entry for each phase of validation and execution. The log entry specifies PipelineComponentTime the processing time for each phase. Writes a log entry that provides diagnostic information. Diagnostic For example, you can log a message before and after every call to an external data provider.
  66. 66. SSIS Manageability – package logging
  67. 67. SSIS Monitoring •SQL Server Management Studio •Logging •Notifications •Performance Monitor •SSIS Service •SSIS Pipeline •PowerShell •WMI
  68. 68. Today‟s Agenda • SQL Server Integration Services Introduction and Architecture • Team Development, Deployment and Configuration • Security and Manageability • Summary and Q&A
  69. 69. Q&A Any Questions?
  70. 70. Resources SQL Server 2008 – Integration Services Portal Page http://msdn.microsoft.com/en-us/sqlserver/cc511477.aspx SQLIS.com - SQL Server Integration Services - SSIS http://sqlis.com Jamie Thomson (UK) – SQL Server MVP – SSIS and forum moderator http://blogs.conchango.com/jamiethomson/ SQL Server 2008 Upgrade Technical Reference Guide http://www.microsoft.com/downloads/details.aspx?FamilyID=66d3e6f5-6902-4fdd-af75-9975aea5bea7&displaylang=en Channel9 MSDN BI Screencasts http://channel9.msdn.com/Showforum.aspx?forumid=38&tagid=277 SQL Server Best Practices http://msdn.microsoft.com/en-us/sqlserver/bb671432.aspx Microsoft Virtual Labs (TechNet and MSDN) http://www.microsoft.com/events/vlabs/default.mspx Microsoft BI Site http://www.microsoft.com/bi/ Magenic Blogs http://blog.magenic.com/blogs
  71. 71. Contact Information – Thank You! Contact us to find out how your business can benefit from a complimentary strategy session with one of our consultants and look into one of our BI quickstart engagements. Dan - http://denglishbi.spaces.live.com David - Magenic - info@magenic.com

×