Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Embrace and extend first-class activity and 3rd party ecosystem for ssis in adf

This session focuses on the deeper integration of SQL Server Integration Services (SSIS) in Azure Data Factory (ADF) and the broad extensibility of Azure-SSIS Integration Runtime (IR). We will first show you how to provision Azure-SSIS IR – dedicated ADF servers for lifting & shifting SSIS packages – and extend it with custom/3rd party components. Preserving your skillsets, you can then use the familiar SQL Server Data Tools (SSDT)/SQL Server Management Studio (SSMS) to design/deploy/configure/execute/monitor your SSIS packages in the cloud just like you do on premises. Next, we will guide you to trigger/schedule SSIS package executions as first-class activities in ADF pipelines and combine/chain them with other activities, allowing you to inject/splice built-in tasks/data transformations in your ETL/ELT workflows, automatically provision Azure-SSIS IR on demand/just in time, etc. And finally, you will learn about the licensing model for ISVs to develop paid components/extensions and join the growing 3rd party ecosystem for SSIS in ADF with a few examples from our partners.

  • Login to see the comments

  • Be the first to like this

Embrace and extend first-class activity and 3rd party ecosystem for ssis in adf

  1. 1. On-premises data sources SQL Server  OS: Windows/Linux  SCALABILITY: Scale-Out feature  EDITION: Standard/Enterprise  TOOLS: SSDT/SSMS to design/deploy/ manage/execute/monitor packages  EXTENSIBILITY: ISVs can build components/extensions on SSIS  PRICING: Bundled w/ on-prem SQL Server
  2. 2. On-premises data sources Azure SQL DB/Managed InstanceAzure Data FactoryCloud data sources Cloud On-premises SQL Server  LIFT & SHIFT: Use Azure SQL DB/Managed Instance (MI) to host SSISDB  SCALABILITY: Use ADF to provision a managed cluster of Azure VMs dedicated to run your packages – Azure-SSIS Integration Runtime (IR)  EDITION: Standard/Enterprise  TOOLS: SSDT/SSMS + ADF app to design/deploy/manage/execute/monitor packages (activities)  EXTENSIBILITY: ISVs can build components/extensions + SaaS on SSIS in ADF via custom setup + 3rd party licensing  PRICING: Pay per hour + Azure Hybrid Benefit (AHB) to Bring Your Own License (BYOL)
  3. 3. On-premises data sources Azure SQL DB/Managed Instance VNet Azure Data FactoryCloud data sources Cloud On-premises SQL Server  HYBRID: Join Azure-SSIS IR to a VNet that is connected to your on-prem network to enable on-prem data access  MODERNIZATION: Schedule first-class SSIS activities in ADF pipelines via SSMS and chain/group them w/ other activities via ADF app  COMPLEMENTARY: Splice/inject built- in/custom/Open Source/3rd party SSIS tasks and transformations in ADF pipelines  READINESS: General Availability (GA) w/ 24/7 live-site support
  4. 4. Azure-SSIS IR node Container ISV Setup1. Specify Product Key in setup script ISV Activation Server 2. Get Activation Key by submitting Cluster ID + Product Key Local Store (e.g. Registry) 3. Write Activation Key SSIS Executor ISV Extension 4. Read Activation Key and validate it with Cluster ID Setup Runtime 4. Get Cluster ID 4. Report on Node Count (Optional) SSIS Runtime 2. Activation Key
  5. 5. Azure SQL DB SQLPhonetics.NET TempDB SSIS Data source SSIS Data source Azure SQL DB Target Server SQLPhonetics.NET for SSIS
  6. 6. Set-AzureRmDataFactoryV2IntegrationRuntime $ResourceGroupName ` $DataFactoryName ` $AzureSSISName $MySetupScriptContainerSasUri $SSISDBServerEndpoint $serverCreds $SSISDBPricingTier $AzureSSISDescription $Location $AzureSSISNodeSize $AzureSSISNodeNumber $AzureSSISMaxParallelExecutionsPerNode
  7. 7. oh22is.match.setup.msi "C:Program Files (x86)oh22information services GmbHSQLPhonetics.NET for SSISoh22is.Licensing.exe" "<LICENSE ID>" "<PASSWORD>" "<CONTAINER>" "<BLOB STORAGE CONNECTION STRING>"
  8. 8. msiexec /i SSISPlus-x64.msi /quiet /l %CUSTOM_SETUP_SCRIPT_LOG_DIR%install.log "C:Program Files (x86)CozyRocSSISLicense.2017.exe" /azure /v /u {{Licensee}} /d /l "%CUSTOM_SETUP_SCRIPT_LOG_DIR%license.log"
  9. 9. KingswaySoft.LicenseManager.exe /a /k XXXXX-XXXXX-XXXXX-XXXXX-XXXXX-XXXXX /e /l "Company Name" msiexec /i IntegrationToolkit-Dynamics365-x64.msi /qn /lv %CUSTOM_SETUP_SCRIPT_LOG_DIR%install.log ACCEPT_EULA=1
  10. 10. @echo off REM Xtract IS for Azure Installation script REM (C) Theobald Software GmbH echo Running XtractISSetup.exe /S... XtractISSetup.exe /S echo %ERRORLEVEL%
  11. 11. DECLARE @execution_id bigint EXEC [SSISDB].[catalog].[create_execution] @package_name=N'TugaSample.dtsx', @execution_id=@execution_id OUTPUT, @folder_name=N'TUGA', @project_name=N'SQLPhonetics', @use32bitruntime=False, @reference_id=1, @useanyworker=True, @runinscaleout=True SELECT @execution_id DECLARE @var0 smallint = 1 EXEC [SSISDB].[catalog].[set_execution_parameter_value] @execution_id, @object_type=50, @parameter_name=N'LOGGING_LEVEL', @parameter_value=@var0 EXEC [SSISDB].[catalog].[start_execution] @execution_id, @retry_count=0 GO