You can watch the replay for this Geek Sync webcast in the IDERA Resource Center: http://ow.ly/f22750A5ptU
SQL Server 2012 redefined how SSIS packages are stored and executed. The advent of the SSISDB catalog gives a central point for working with SSIS projects deployed to the server. From this catalog we can set project and package parameter, configure environments, and monitor execution. There is no need to build in package logging because Microsoft had done it for us.
New Repository Model introduced in SQL Server 2012. Instead of deploying packages we deploy projects. (Changes in SQL Server 2016, will allow deployment of package only)
Previous options:
Deploy to file system.
Deploy to package store. (Which is nothing more than storing it on the file system under the SQL Server directory)
Deploy to SQL Server (Stored in MSDB syssispackages)
Previous Ways to deploy:
Create deployment utility from the project. (Run it an install to file system or SQL Server)
Use DTUTIL command line (run from command prompt to manage packages)
Use SSMS
You can upgrade packages to the new deployment model. (Upgrade Wizard available in Data Tools/Visual Studio)
You lose some capabilities you previously had.
For example, configurations used to configure SSIS packages across multiple environments are no longer available.
Project parameters now replace configurations along with Environments, which live on the Integration Serves Catalog.
“DEVIN KNIGHT has a good blog on upgrading”
Catalog Creation
After install, right click on Integration Services and select “Create Catalog”.
Follow the wizard.
“Open SSMS and view properties”
Catalog consists of folders.
Each folder can have one or more projects and environments.
Packages live in projects.
Projects
Projects are deployed to the “Projects” folder.
Projects consist of:
Parameters (project, package level)
Packages deployed with project
References to environment variables
Version history is stored at the time the project is deployed. By default settings, 10 versions back are maintained.
Environments
Allows you to run projects in multiple environments. Replaces need for configuration files. Example, test vs. prod.
You can assign different values to parameters in your project based on environments.
Environments can be assigned to projects or packages at execution time.
Open SSMS
Show catalog properties.
Clean logs
Retention Period
Default logging level
Max number of versions
Click on project and go to configure
Show parameters
Show references
Show how parameter values are based on references to an environment
Click on Environments
Go to properties
Go to variables
Add environment example
Parameters can be assigned at time of execution using environment variables.
Parameters can be defined using expressions only at the package level.
Package level parameters are scoped at the package level only.
Project parameters are shared across all packaged in the project.
Literal values only, cannot use expressions.
Will show Visual Studio deployment in example.
“Open SSMS”
Show project versions example.
Parameters
Environments
Environment Variables
Example
Open Visual Studio
Show project parameters
Parameterized connection managers
Execution Methods
Right click on package in SSMS and execute
Use dtexec (calls stored procedures in SSISDB to create execution, set execution parameters and start execution)
SQL Server Agent Job
TSQL using stored procedures residing in SSISDB
Previous to 2012 logging was only at the package level and had to be turned on and the events selected for logging, along with any custom messages created. The events were logged in the SYSSSISLOG table in MSDB. This option is still available in current versions of SQL Server.
Basic:
OnPreValidate
OnPostValidate
OnPreExecute
OnPostExecute
OnInformation
OnWarning
OnError
Basic
It captures important events (like errors and warnings), as well as enough progress information to display what the currently active task is, and how long each task has.
Performance
used when you are doing benchmarking and performance tuning for your packages.
While it actually logs less, it captures a lot more events internally to analyze the performance of the data flow components.
A bit more overhead during execution – packages run with Basic will actually run a little faster than Performance
Verbose
The Verbose logging level captures a lot of events (especially when using OLE DB connections, as they generate many DIAGNOSTIC events).
Verbose should only be used when trying to debug or diagnose package failures.
“Go to SSMS”
Execute by “right click” example.
SQL Agent job example.
“Open SSMS and show TSQL example”
Execution reports are available in the catalog by default. Amount of data is dependent on the logging level.
You can write your own reports as the data is available through the SSISDB.
“Show reporting example in SSMS”
SSIDB catalog database contains all tables, views , stored procedures gear around:
Execution information of packages
Configuration of packages and projects
Administration of projects and packages as well as execution of packages through stored procedures.
These tables are centered around information regarding package execution.
The views collapse and consolidate data.
“Go to examples in SSMS”
In execution_parameter_values
When the value is 20, the parameter is a project parameter.
When the value is 30, the parameter is a package parameter.
When the value is 50, the parameter is one of the following.
LOGGING_LEVEL
DUMP_ON_ERROR
DUMP_ON_EVENT
DUMP_EVENT_CODE
CALLER_INFO
SYNCHRONIZED
The operations tables contain information around all operations performed on the SSIS Catalog.
Operations
Object_type values
101 – deploy
200 – create execution, start execution
202 – stop execution
300 – validate project
301 – validate package
Status
(1), running (2), canceled (3), failed (4), pending (5), ended unexpectedly (6), succeeded (7), stopping (8), and completed (9).
Event_message_context
Context_type
10 Task State of a task when an error occurred.
20 Pipeline Error from a pipeline component: source, destination, or transformation component.
30 Sequence State of a sequence.
40 For Loop State of a For Loop.
50 Foreach Loop State of a Foreach Loop
60 Package State of the package when an error occurred.
70 Variable Variable value
80 Connection manager Properties of a connection manager.