Presentation about automating repeatable steps during SSIS projects deployment using PowerShell and msbuild

1. #SQLSatDenmark
Automate your SSIS
deployment process
Bartosz Ratajczyk

2. #SQLSatDenmark
BIG Thanks to SQLSat Denmark sponsors

3. #SQLSatDenmark
What is he talking about?
About making deployments easier
removing manual tasks
Things tested in practice
starting with small steps
From multiple scripts to one script deployment
or using one/two steps in TFS
How to build your own solution
About different options and hurdles

4. #SQLSatDenmark
I will not cover
Single package deployments
Best practices
DevOps

5. #SQLSatDenmark
Bartosz Ratajczyk
SQL Server Consultant @ 7N
Speaker
SQL Saturday volunteer
Advisory Board Member in Data
Community Poland
MCSE, MCT
http://bartekr.net b.ratajczyk@gmail.com

6. #SQLSatDenmark
All deployment steps
Client side
1. Compile project
2. Deploy project
Server side
1. Create folder
2. Create environment
3. Reference environment
4. Create environment variables
5. Configure project
6. Validate project
7. Verify validation
8. Test run
Testing (NBi, LegiTest)
Naming convention compliance (ie. Jamie Thomson)

7. #SQLSatDenmark
Raising the bar
Version control
DEV, TEST, PROD branches
Different servers
DEV, TEST, QA, PREPROD, PROD
Different configurations
Drive letters, folder names
Connection strings
We have more than one project

8. #SQLSatDenmark
MAKING THE WORK EASIER
Let’s start
SQLDay 2017

9. #SQLSatDenmark
Compile project – first steps

10. #SQLSatDenmark
Compilation problems
EncryptAllWithPassword, EncryptSensitiveWithPassword
Manual password entry
The devenv.com process can hang

11. #SQLSatDenmark
Handcrafting the package

12. #SQLSatDenmark
MSBuild

13. #SQLSatDenmark
SSISMSBuild
SSIS compilation and deployment tasks for MSBuild
DeploymentFileCompilerTask
DeployProjectToCatalogTask
Source code, build yourself the .dll
https://www.simple-talk.com/sql/ssis/deployment-automation-for-sql-server-integration-services-ssis/
https://seddryck.wordpress.com/2016/12/27/building-the-msbuild-tasks-for-ssis-2016/
http://sqlsrvintegrationsrv.codeplex.com

14. #SQLSatDenmark
Compilation: Build.proj

15. #SQLSatDenmark
Deployment – IsDeploymentWizard
IsDeploymentWizard.exe
/Silent

16. #SQLSatDenmark
IsDeploymentWizard /Silent
/Silent == fire and forget
There might be a situation, where /Silent has problems
e.g. external library in Script Task

17. #SQLSatDenmark
MSBuild, again

18. #SQLSatDenmark
DEMO
Compilation and deployment
SQLDay 2017

19. #SQLSatDenmark
Other methods
PowerShell script using SMO
http://muxtonmumbles.blogspot.dk/2013/11/ssis-project-deployment-and-visual.html
SSISBuild
http://roman.tumaykin.com/tools/2017/01/10/Continuous-integration-of-SSIS-projects-Part-1-build.html
PowerShell Continuous Integration (PSCI)
https://github.com/ObjectivityLtd/PSCI
SSIS Build & Deploy
https://marketplace.visualstudio.com/items?itemName=TG.VSTS-SSIS

20. #SQLSatDenmark
What’s with the name?

21. #SQLSatDenmark
Compilation: Build.proj (again)

22. #SQLSatDenmark
Handling the Environments
Visual Studio Configuration Manager
PowerShell + SMO
Array
CSV
Database
JSON

23. #SQLSatDenmark
Configuration Manager

24. #SQLSatDenmark
Configuration Manager - usage
Different parameter values for different configurations
(think: local environments)
Contained in .dtproj files
No single window to configure them all

25. #SQLSatDenmark

26. #SQLSatDenmark
Create the Environment
Environment-<NAME>.csv

27. #SQLSatDenmark
Reference the Environment

28. #SQLSatDenmark
Configure the project
Environment-<NAME>-Configuration.csv

29. #SQLSatDenmark
Data sensitivity
Encryption
http://www.codewrecks.com/blog/index.php/2014/07/12/store-secure-password-in-tfs-build-definition/
Certificates
http://www.codewrecks.com/blog/index.php/2014/08/09/encrypt-your-password-in-tfs-build-using-certificates/

30. #SQLSatDenmark
DEMO
Create the environment and configure the project
SQLDay 2017

31. #SQLSatDenmark
Validation
Validation is an asynchronous process
We start the process and track the progress

32. #SQLSatDenmark
Validation from code
::UseAllReferences
::SpecifyReference, $reference

33. #SQLSatDenmark
Test run – the job

34. #SQLSatDenmark
DEMO
Validation and test
SQLDay 2017

35. #SQLSatDenmark
What else?
NBi tests
Jamie Thomson’s naming convention
http://www.nbi.io/
http://microsoft-ssis.blogspot.dk/2016/12/ssis-naming-conventions.html

36. #SQLSatDenmark
And how about TFS?
Custom proces for building and deployment
You can expand it with PowerShell steps to
control environments, validation, …

37. #SQLSatDenmark
DEMO
Automatic proces using TFS
SQLDay 2017

38. #SQLSatDenmark
So simple!
The work is still boring, but this time the computer does it
For automation use
MSBuild
Powershell
SMO
Use additional software for CI/CD process

39. #SQLSatDenmark
BIG Thanks to SQLSat Denmark sponsors