2. ContentContent
Recap and Q&A
Data Flow Task
Pipeline Architecture
Data Sources
◦ Demo: ADO.NET Source
◦ Demo: Excel Source
◦ Demo: Flat File Source
◦ Demo: OLE DB Source
◦ Demo: XML Source
◦ Demo: Raw File Destination
◦ Demo: Raw File Source
Data Destinations
◦ Demo: OLE DB Destination
◦ Demo: DataReader Destination
◦ Demo: Excel Destination
◦ Demo: Flat File Destination
◦ Demo: SQL Server Destination
Analysis Services Destinations
◦ Demo: Dimension Processing
◦ Demo: Partition Processing
◦ Demo: Data Mining Model Training
Summary
3. Recap and Q&ARecap and Q&A
Variables
◦ Demo: Introducing the Variables Window
◦ Demo: Working with Variables
◦ Demo :Using variables in Execute SQL task
Expressions
◦ Demo: Creating Time stamped log file
◦ Demo: Operators and More Variables
Expression Usage
◦ Demo: Using a Variable in an Expression
◦ Demo: Using an Expression with a Connection Manager
◦ Demo: Using Expressions with Precedence Constraints
4. Data Flow TaskData Flow Task
Encapsulates the data flow
engine that Extract, transform, and
load data
A Data Flow task can include multiple
data flows
When order is important, use
multiple Data Flow tasks, each task
containing one data flow.
Diagram includes the inputs,
outputs, ,error outputs external
columns.
5. Pipeline ArchitecturePipeline Architecture
The run-time engine implements
the control flow and package
management infrastructure
The data flow engine
is a specialized, high performance
engine that is exclusively
dedicated to extracting,
transforming, and loading data.
The data flow engine provides the
in-memory buffers
Arrow in dataflow are called
Service path
6. ADO.NET SourceADO.NET Source
The ADO NET source consumes data from a .NET provider and makes the
data available to the data flows
SSIS 2012 onwards, you can use the ADO NET source to connect to
Microsoft Windows Azure SQL Database
The ADO.net source provides two different data access modes for
extracting data:
◦ A table or view.
◦ The results of an SQL statement.
7. Excel SourceExcel Source
The Excel source extracts data
from worksheets or ranges in
Microsoft Excel workbooks
The Excel source provides four
different data access modes
for extracting data:
◦ A table or view.
◦ A table or view specified in a
variable.
◦ The results of an SQL statement.
◦ The results of an SQL statement
stored in a variable.
8. Flat File SourceFlat File Source
The Flat File source reads data from a
text file. The text file can be in
delimited, fixed width, or mixed format.
Delimited format uses column and row
delimiters to define columns and rows.
Fixed width format uses width to define
columns and rows. This format also
includes a character for padding fields to
their maximum width.
Ragged right format uses width to define
all columns, except for the last column,
which is delimited by the row delimiter.
9. OLE DB SourceOLE DB Source
The OLE DB source extracts
data from a variety of OLE DB-
compliant relational databases
The OLE DB source provides
four different data access
modes for extracting data:
◦ A table or view.
◦ A table or view specified in a
variable.
◦ The results of an SQL statement. The
query can be a parameterized query.
◦ The results of an SQL statement
stored in a variable.
10. XML SourceXML Source
The XML Source does not
require a Connection Manager,
but instead references a XML
data source either as a
◦ direct reference to a file
◦ a variable containing a file reference
or
◦ a variable containing XML data.
11. Raw File DestinationRaw File Destination
The Raw File destination writes
raw data to a file. Because the
format of the data is native to
the destination
write data more quickly than
other destinations
You use the Raw File source
to retrieve raw data that was
previously written by the
destination
This source does not use a
connection manager.
12. Raw File SourceRaw File Source
The Raw File source reads raw
data from a file
Reads data more quickly than
other sources
Retrieve raw data that was
previously written by the Raw
File destination
This source does not use a
connection manager.
13. OLEDB DestinationOLEDB Destination
The OLE DB destination loads data into a variety of OLE DB-compliant databases
The OLE DB destination provides five different data access modes for loading
data:
◦ A table or view. You can specify an existing table or view, or you create a new table.
◦ A table or view using fast-load options. You can specify an existing table or create a new table.
◦ A table or view specified in a variable.
◦ A table or view specified in a variable using fast-load options.
◦ The results of an SQL statement.
Does not support parameters. To execute a parameterized INSERT statement,
use OLE DB Command transformation
If OLE DB destination uses fast-load data access mode, specify following
options:
◦ Keep identity values from the imported data file or use unique values assigned by SQL Server.
◦ Retain a null value during the bulk load operation.
◦ Check constraints on the target table or view during the bulk import operation.
◦ Acquire a table-level lock for the duration of the bulk load operation.
◦ Specify the number of rows in the batch and the commit size.
15. Excel DestinationExcel Destination
The Excel destination loads data into
worksheets or ranges in Microsoft
Excel workbooks
Can create new sheet based on input
In Excel, a worksheet or range is the
equivalent of a table or view.
The lists of available tables in the
Excel Source and Destination editors
display only existing worksheets
16. Flat File DestinationFlat File Destination
The Flat File destination writes data to a text file
This destination uses a Flat File connection manager to
access the text file
Specify whether to overwrite a data in a destination file
that has the same name.
17. SQL Server DestinationSQL Server Destination
Connects to a local SQL Server database and bulk loads data
Cannot access a SQL Server database on a remote server
Users who execute packages that include the SQL Server destination require
the "Create global objects" permission.
Assigning "Create Global Objects" user right to that account:
Click Start, point to Administrative Tools, and then click Local Security Policy.
Expand Local Policies, and then click User Rights Assignment.
In the right pane, double-click Create global objects
In the Local Security Policy Setting dialog box, click Add.
In the Select Users or Group dialog box, click the user accounts that you want to add,
click Add, and then click OK.
Click OK.
If you are running the package from either SQL Server Management
Studio (SSMS) or Business Intelligence Development Studio (BIDS)
or DTExecUI.exe, launch those tools under the elevated Administrator
account
18. Dimension ProcessingDimension Processing
The Dimension Processing
destination loads and processes an
SQL Server Analysis Services
dimension
The Dimension Processing
destination includes the following
features:
◦ Options to perform incremental, full, or update
processing.
◦ Error configuration, to specify whether
dimension processing ignores errors or stops
after a specified number of errors.
◦ Mapping of input columns to columns in
dimension tables.
19. Partition ProcessingPartition Processing
The Partition Processing destination loads and processes an SQL Server Analysis
Services partition.
The Partition Processing destination includes the following features:
◦ Options to perform incremental, full, or update processing.
◦ Error configuration, to specify whether processing ignores errors or stops after a specified number of errors.
◦ Mapping of input columns to partition columns.
20. SummarySummary
Data Flow Task
Pipeline Architecture
Data Sources
◦ Demo: ADO.NET Source
◦ Demo: Excel Source
◦ Demo: Flat File Source
◦ Demo: OLE DB Source
◦ Demo: XML Source
◦ Demo: Raw File Destination
◦ Demo: Raw File Source
Data Destinations
◦ Demo: OLE DB Destination
◦ Demo: DataReader Destination
◦ Demo: Excel Destination
◦ Demo: Flat File Destination
◦ Demo: SQL Server Destination
Analysis Services Destinations
◦ Demo: Dimension Processing
◦ Demo: Partition Processing