Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

5\9 SSIS 2008R2_Training - DataFlow Basics

372 views

Published on

This PPT explains SSIS data flow task basics that exists in SQL Server 2008 R2.

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

5\9 SSIS 2008R2_Training - DataFlow Basics

  1. 1. 5.5. Data Flow BasicsData Flow Basics Presented by: Pramod Singla Pramod_singla@yahoo.co.in .
  2. 2. ContentContent  Recap and Q&A  Data Flow Task  Pipeline Architecture  Data Sources ◦ Demo: ADO.NET Source ◦ Demo: Excel Source ◦ Demo: Flat File Source ◦ Demo: OLE DB Source ◦ Demo: XML Source ◦ Demo: Raw File Destination ◦ Demo: Raw File Source  Data Destinations ◦ Demo: OLE DB Destination ◦ Demo: DataReader Destination ◦ Demo: Excel Destination ◦ Demo: Flat File Destination ◦ Demo: SQL Server Destination  Analysis Services Destinations ◦ Demo: Dimension Processing ◦ Demo: Partition Processing ◦ Demo: Data Mining Model Training  Summary
  3. 3. Recap and Q&ARecap and Q&A Variables ◦ Demo: Introducing the Variables Window ◦ Demo: Working with Variables ◦ Demo :Using variables in Execute SQL task Expressions ◦ Demo: Creating Time stamped log file ◦ Demo: Operators and More Variables Expression Usage ◦ Demo: Using a Variable in an Expression ◦ Demo: Using an Expression with a Connection Manager ◦ Demo: Using Expressions with Precedence Constraints
  4. 4. Data Flow TaskData Flow Task  Encapsulates the data flow engine that Extract, transform, and load data  A Data Flow task can include multiple data flows  When order is important, use multiple Data Flow tasks, each task containing one data flow.  Diagram includes the inputs, outputs, ,error outputs external columns.
  5. 5. Pipeline ArchitecturePipeline Architecture  The run-time engine implements the control flow and package management infrastructure  The data flow engine is a specialized, high performance engine that is exclusively dedicated to extracting, transforming, and loading data.  The data flow engine provides the in-memory buffers  Arrow in dataflow are called Service path
  6. 6. ADO.NET SourceADO.NET Source  The ADO NET source consumes data from a .NET provider and makes the data available to the data flows  SSIS 2012 onwards, you can use the ADO NET source to connect to Microsoft Windows Azure SQL Database  The ADO.net source provides two different data access modes for extracting data: ◦ A table or view. ◦ The results of an SQL statement.
  7. 7. Excel SourceExcel Source  The Excel source extracts data from worksheets or ranges in Microsoft Excel workbooks  The Excel source provides four different data access modes for extracting data: ◦ A table or view. ◦ A table or view specified in a variable. ◦ The results of an SQL statement. ◦ The results of an SQL statement stored in a variable.
  8. 8. Flat File SourceFlat File Source  The Flat File source reads data from a text file. The text file can be in delimited, fixed width, or mixed format.  Delimited format uses column and row delimiters to define columns and rows.  Fixed width format uses width to define columns and rows. This format also includes a character for padding fields to their maximum width.  Ragged right format uses width to define all columns, except for the last column, which is delimited by the row delimiter.
  9. 9. OLE DB SourceOLE DB Source  The OLE DB source extracts data from a variety of OLE DB- compliant relational databases  The OLE DB source provides four different data access modes for extracting data: ◦ A table or view. ◦ A table or view specified in a variable. ◦ The results of an SQL statement. The query can be a parameterized query. ◦ The results of an SQL statement stored in a variable.
  10. 10. XML SourceXML Source  The XML Source does not require a Connection Manager, but instead references a XML data source either as a ◦ direct reference to a file ◦ a variable containing a file reference or ◦ a variable containing XML data.
  11. 11. Raw File DestinationRaw File Destination  The Raw File destination writes raw data to a file. Because the format of the data is native to the destination  write data more quickly than other destinations  You use the Raw File source to retrieve raw data that was previously written by the destination  This source does not use a connection manager.
  12. 12. Raw File SourceRaw File Source  The Raw File source reads raw data from a file  Reads data more quickly than other sources  Retrieve raw data that was previously written by the Raw File destination  This source does not use a connection manager.
  13. 13. OLEDB DestinationOLEDB Destination  The OLE DB destination loads data into a variety of OLE DB-compliant databases  The OLE DB destination provides five different data access modes for loading data: ◦ A table or view. You can specify an existing table or view, or you create a new table. ◦ A table or view using fast-load options. You can specify an existing table or create a new table. ◦ A table or view specified in a variable. ◦ A table or view specified in a variable using fast-load options. ◦ The results of an SQL statement.  Does not support parameters. To execute a parameterized INSERT statement, use OLE DB Command transformation  If OLE DB destination uses fast-load data access mode, specify following options: ◦ Keep identity values from the imported data file or use unique values assigned by SQL Server. ◦ Retain a null value during the bulk load operation. ◦ Check constraints on the target table or view during the bulk import operation. ◦ Acquire a table-level lock for the duration of the bulk load operation. ◦ Specify the number of rows in the batch and the commit size.
  14. 14. DataReader DestinationDataReader Destination  Exposes the data in a data flow by using the ADO.NET DataReader interface  The data can then be consumed by other application
  15. 15. Excel DestinationExcel Destination  The Excel destination loads data into worksheets or ranges in Microsoft Excel workbooks  Can create new sheet based on input  In Excel, a worksheet or range is the equivalent of a table or view.  The lists of available tables in the Excel Source and Destination editors display only existing worksheets
  16. 16. Flat File DestinationFlat File Destination  The Flat File destination writes data to a text file  This destination uses a Flat File connection manager to access the text file  Specify whether to overwrite a data in a destination file that has the same name.
  17. 17. SQL Server DestinationSQL Server Destination  Connects to a local SQL Server database and bulk loads data  Cannot access a SQL Server database on a remote server  Users who execute packages that include the SQL Server destination require the "Create global objects" permission.  Assigning "Create Global Objects" user right to that account:  Click Start, point to Administrative Tools, and then click Local Security Policy.  Expand Local Policies, and then click User Rights Assignment.  In the right pane, double-click Create global objects  In the Local Security Policy Setting dialog box, click Add.  In the Select Users or Group dialog box, click the user accounts that you want to add, click Add, and then click OK.  Click OK.  If you are running the package from either SQL Server Management Studio (SSMS) or Business Intelligence Development Studio (BIDS) or DTExecUI.exe, launch those tools under the elevated Administrator account
  18. 18. Dimension ProcessingDimension Processing  The Dimension Processing destination loads and processes an SQL Server Analysis Services dimension  The Dimension Processing destination includes the following features: ◦ Options to perform incremental, full, or update processing. ◦ Error configuration, to specify whether dimension processing ignores errors or stops after a specified number of errors. ◦ Mapping of input columns to columns in dimension tables.
  19. 19. Partition ProcessingPartition Processing  The Partition Processing destination loads and processes an SQL Server Analysis Services partition.  The Partition Processing destination includes the following features: ◦ Options to perform incremental, full, or update processing. ◦ Error configuration, to specify whether processing ignores errors or stops after a specified number of errors. ◦ Mapping of input columns to partition columns.
  20. 20. SummarySummary  Data Flow Task  Pipeline Architecture  Data Sources ◦ Demo: ADO.NET Source ◦ Demo: Excel Source ◦ Demo: Flat File Source ◦ Demo: OLE DB Source ◦ Demo: XML Source ◦ Demo: Raw File Destination ◦ Demo: Raw File Source  Data Destinations ◦ Demo: OLE DB Destination ◦ Demo: DataReader Destination ◦ Demo: Excel Destination ◦ Demo: Flat File Destination ◦ Demo: SQL Server Destination  Analysis Services Destinations ◦ Demo: Dimension Processing ◦ Demo: Partition Processing
  21. 21. DEMO 5_DataFlowBasics.dtsx 5_DataFlowBasics_OnlinDemo.dtsx
  22. 22. Resources & QuestionsResources & Questions Contact me : ◦ Pramod_singla@yahoo.co.in ◦ http://pramodsingla.wordpress.com/ Microsoft Resources: ◦ http://pluralsight.com/training/courses/TableOfContents?courseName=ssis-basic&highlight=stacia-misn ◦ http://technet.microsoft.com/en- us/library/microsoft.sqlserver.dts.pipeline.wrapper(v=sql.105).aspx ◦ http://technet.microsoft.com/en-us/library/bb522498(v=sql.105).aspx ◦ http://www.mssqltips.com/sqlservertip/2557/ssis-architecture-and-internals-interview- questions/ ◦ http://msdn.microsoft.com/en-us/library/ms141122.aspx ◦ http://www.bimonkey.com/support/ssis-component-samples-list/ ◦ http://technet.microsoft.com/en-us/library/ms137622.aspx ◦ http://support.microsoft.com/kb/2009672 ◦ https://www.simple-talk.com/sql/reporting-services/using-sql-server-integration-services-to- bulk-load-data/

×