• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
SQL Server Integration Services
 

SQL Server Integration Services

on

  • 3,036 views

This session provides an introduction to using SSIS. This is an update to my older presentation on the topic: http://www.slideshare.net/rmaclean/sql-server-integration-services-2631027

This session provides an introduction to using SSIS. This is an update to my older presentation on the topic: http://www.slideshare.net/rmaclean/sql-server-integration-services-2631027

Statistics

Views

Total Views
3,036
Views on SlideShare
2,142
Embed Views
894

Actions

Likes
0
Downloads
94
Comments
0

4 Embeds 894

http://www.sadev.co.za 890
http://webcache.googleusercontent.com 2
http://sadev.co.za 1
http://www.slashdocs.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Create a new packageAdd a data flow componentAdd a flat file connection – set it to the exercise.csv data. So suggested data times.Add a flat file source – bind to connectionAdd conditional splitLink to flat fileSplit on distance > 0Add sortLink to main split outputSort on dateAdd ADO.NET data destination connection manager – to spacedata.exerciseAdd ADO.NET DestinationLink to sort and connection managerDo mappingAdd Variable-Make sure scope is packageAdd Row countLink to conditional split elseLink to variableOn Control flowAdd SMTP taskSMTP connection to webmail.bbd.co.za and windows auth-Message body expression from file and change variableSave -> Run -> CrashChange distance to float on flat file connection, trickle changesSave -> Run -> Email
  • SSIS adds layers of support, logging etc... These all add a performance hit and you should not waste time using it for things where a quick BCP or even c# code would be better.SSIS is perfect for batch solutions, it is BAD for near real time solutions. There are tools built into SQL server (linked servers) and other tools (BizTalk) which handle real time well.SSIS is not a SOA/ESB/B2B tool it is a ETL tool.
  • Use management studio script to clean data outAdd Excel SourceDelete flat file sourceConnect Excel source to split inputOpen excel source, create new OLE DB connectionTrickle changesSave -> Run -> CrashProject -> Properties -> Debugging -> Run64bitRuntime -> FalseSave -> Run
  • Record set info: http://blogs.conchango.com/jamiethomson/archive/2006/06/28/SSIS_3A00_-Comparing-performance-of-a-raw-file-against-a-recordset-destination.aspxMemory – Make sure you have enoughSELECT * - all that meta data and unused columns need processing!Small packages – You can call one package from another. Allows work to be broken up, which means a team can easily work on the solution, makes fault finding easier, lowers over headsComments – DUH!Understand the components is key to super usage – Many can be used to do the same thing. Lookup and Merge Join for instance can both be used to lookup data. Lookup has three modes which imposed performance vs. Memory trade offs where merge join does not. Merge join requires sorted input while lookups don’t. Execute SQL allows any SQL dialect while execute T-SQL allows only T-SQLBecause things can be put into parallel that doesn’t mean they execute in parallel. Some are async and some aren’t!

SQL Server Integration Services SQL Server Integration Services Presentation Transcript

  • SSIS
  • Microsoft’s ETL solution bundled with SQL Server
    E – Extract
    T – Transform
    L – Load
    What is it?
    Destination
    Source
    Source
    Write
    Read
    SSIS
  • A simple ETL
  • When to use Script
    YES
    NO
    Can one or more components
    do the same thing?
    NO
    YES
    Use those
    Unique situation
    Build custom component
    Use script
  • Confusion
  • Performance
    • Consider SQL or BCP for simple imports
    • File system performance
    Data Latency
    • SSIS is not a near real time solution
    SOA, ESB, B2B Integration
    • No business rules support
    • Very basic queue support
    • XML support limited
    Reasons not to consider SSIS
  • Merging Data from Heterogeneous Data Stores
    Populating Data Warehouses
    Cleaning and Standardizing Data
    Building Business Intelligence into a Data Transformation Process
    Automating Administrative Functions and Data Loading
    Areas SSIS is strong
  • Command line tools (dtexec, dtutil) cannot co-exist with 32bit versions.
    No DTS support.
    Limitations on data providers – No Access, Excel or SQL Compact
    IA64 has more limitations including no designer support
    X64 Limitations
  • Running 32bit on 64bit
  • RecordSet Destination
    • SLOW (5 times than a raw file!)
    Memory
    • SSIS is an in memory process.
    SELECT *
    • Exceptionally bad in SSIS
    Use many small packages
    Comments!!!
    Understand the components
    • Many do the same things in different ways with different trade offs
    • Lookup vs. Merge Join or Execute SQL vs. Execute T-SQL
    • Understand which components run asynchronously and which run synchronously
    What to watch out for
  • Parallelism Example I
  • Parallelism Example II
  • Parallelism Example III
  • Source Control
    Highest Encryption
    Easy loading into designer
    Access by multiple users
    DB Roles, DTS Roles
    SQL Backups
    Able to filter packages
    Files
    Server
    Storage Options
    Details http://www.sqljunkies.com/WebLog/knight_reign/archive/2005/05/05/13523.aspx
  • Scheduling a package
  • No more often than 3x avg. execution time.
    Settings in configuration files.
    Enable logging (step) and notifications (job).
    Execute signed packages only.
    Do not make packages which execute themselves.
    Scheduling Guidelines
  • DTS Upgrade Options
    Run DTS in 2005 or 2008
    Missing the package logs
    Runs under 32 bit
    Upgrade using MS Wizard
    Not compatible with most package
    Upgrade using DTS xChange
    Minutes per package
    Starting from scratch
    About 3-6 hrs per package conservatively
  • Package Upgrade Wizard
    Built into SQL Server 2008
    Pros:
    Free
    Works on simple packages
    Cons:
    Does not handle ODBC
    Only handles a few types of text file use cases
    No Dynamic Properties Task
    No UDL or legacy database support in data pump
    Packages only have about a 20% chance of working
  • Profiles DTS packages to help with a conversion project plan
    Rapidly converts DTS Packages to SSIS (2005 or 2008) and applies SSIS best practices
    Converts tasks that are not handled by the existing SQL Server conversion wizard
    Includes a SSIS logging repository and reports for trending and alerting
    Includes BI xPress for new SSIS packages
    DTS xChangeUpgrade
  • ActiveX Script Upgrade
    Both tools mentioned migrate DTS ActiveX to ActiveX in SSIS
    ActiveX migrates to SSIS but you would not want to keep it there and it may not run
    Need for ActiveX Script Task has been replaced with built-in, easy to maintain SSIS tasks
    File System Object = File System Task
    Mail objects = Send Mail Task (now has SMTP)
    ADO objects = Execute SQL Task
  • http://www.microsoft.com/technet/prodtechnol/sql/2005/technologies/ssisperfstrat.mspx
    http://ssisblog.replicationanswers.com/2008/01/04/when-to-not-use-ssis--directory--file-iterations.aspx
    http://marcusrosen.blogspot.com/2008/04/sql-server-2005-integration-service.html
    http://www.eggheadcafe.com/software/aspnet/32465151/linked-servers-versus-ssi.aspx
    http://blogs.conchango.com/jamiethomson/archive/2006/06/28/SSIS_3A00_-Comparing-performance-of-a-raw-file-against-a-recordset-destination.aspx
    http://www.microsoft.com/technet/prodtechnol/sql/2005/ssisperf.mspx
    References