Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Script and Automate SSIS Development
Cathrine Wilhelmsen, Senior BI Consultant, Inmeta
Moderated By: Narinder Sharma
Biml ...
If you require assistance
during the session, type
your inquiry into the
question pane on the right
side.
Maximize your sc...
Quest helps IT Professionals simplify administration tasks so they can
focus on the evolving needs of their businesses. Co...
PASS’ flagship event
November 6-9
Seattle, Washington
Free 1-day local
training events
Local user groups
around the world
...
Cathrine Wilhelmsen
Senior BI Consultant, Inmeta
CathrineW.net
Sci-fi, chocolate, coffee, craft beers,
ciders, cat gifs, a...
Script and Automate SSIS Development
Cathrine Wilhelmsen, Senior BI Consultant, Inmeta
Biml for Beginners:
Presenting Spon...
Basic Biml Tools & Code
SSIS from Metadata
…the next 45 minutes…
once upon a time…
Meet Sandra,
the SSIS developer
Sandra has to create a
new staging project with
100+ SSIS packages
Sandra gets quite bored of
creating the same package
over and over and over
Yay, Sandra has finally
completed her project!
But Sandra's manager
has just a few new
requirements…
Sandra gets to update
every single SSIS package
one… more… time…
…yay
It's time for a change!
Business Intelligence Markup Language
Easy to read and write XML language
Describes business intelligence objects:
• Datab...
Why use Biml?
SSIS: Plumbing Biml: Business Logic
Spend time on dragging, dropping, connecting, aligning
Create the same package over and over again with minor changes
Stan...
Spend time on what is unique in a package
Create a pattern once and reuse for all similar packages
Handle scope and requir...
How does SSIS development work?
How does Biml development work?
Biml is a tool for generating SSIS packages
Biml is not a pre-defined Data Warehouse framework
Biml is not a tool for auto...
Timesaving: Many SSIS packages from one Biml file
Reusable: Write once and run on any platform
Flexible: Start simple, exp...
Source Control: Keep track of actual logic changes
Simplify: Untangle complex packages for quick edits
Scripts: Use Biml i...
What do you need?
Free add-in for Visual Studio
Code editor with syntax highlighting and Biml Intellisense
More frequent updates than BIDS H...
Licensed full-featured development environment for Biml
Visual designer and metadata modeling
Full-stack automation and tr...
Biml syntax
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Package>
<Pack...
Biml syntax: Root Element
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></...
Biml syntax: Collections of Elements
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyP...
Biml syntax: Elements
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Pack...
Biml syntax: Attributes
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="EmptyPackage1"></Pa...
Biml syntax: Full vs. Shorthand Syntax
<Biml xmlns="http://schemas.varigence.com/biml.xsd">
<Packages>
<Package Name="Empt...
DEMO
Let's generate
some packages!
Ok, so we can go from Biml to SSIS…
…can we go from SSIS to Biml?
Yes!
DEMO
Let's convert some
SSIS to Biml!
The magic is in the…
Extend Biml with C# or VB code blocks
Import database structure and metadata
Loop over tables and columns
Expressions repl...
How does it work?
Yes, but how does it work?
BimlScript Code Blocks
<# … #> Control Block (Variables and logic)
<#= … #> Text Block (Returns string)
<#* … *#> Comment ...
BimlScript Syntax
<# var con = SchemaManager.CreateConnectionNode(...); #>
<# var metadata = con.GetDatabaseSchema(); #>
<...
BimlScript Syntax: Import metadata
<# var con = SchemaManager.CreateConnectionNode(...); #>
<# var metadata = con.GetDatab...
BimlScript Syntax: Loop over tables
<# var con = SchemaManager.CreateConnectionNode(...); #>
<# var metadata = con.GetData...
BimlScript Syntax: Replace static values
<# var con = SchemaManager.CreateConnectionNode(...); #>
<# var metadata = con.Ge...
Yes, but how does it actually work?
<# foreach (var table in RootNode.Tables) { #>
<Package Name="Load_<#=table.Name#>" />...
PreviewPane
The power is in the…
DEMO
Let's generate
many packages!
New project
Of course I can create
200 SSIS Packages!
…what do you need me
to do after lunch?
Where can you learn more?
Free online training
bimlscript.com
Get things done
Start small
Start simple
Start with ugly code
Keep going
Expand
Improve
Deliver often
Biml on Monday…
…BimlBreak the rest
of the week!
@cathrinew
cathrinew.net
hi@cathrinew.net
Biml resources and demo files:
cathrinew.net/biml
Questions?
Coming up next…
Improve App Performance Using Async Messaging with Service Broker
Allen White
Thank you for attending!
@sqlpass #sqlpass @PASScommunity
Presenting Sponsor
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018)
Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018)
Upcoming SlideShare
Loading in …5
×

of

Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 1 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 2 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 3 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 4 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 5 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 6 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 7 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 8 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 9 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 10 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 11 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 12 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 13 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 14 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 15 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 16 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 17 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 18 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 19 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 20 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 21 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 22 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 23 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 24 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 25 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 26 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 27 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 28 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 29 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 30 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 31 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 32 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 33 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 34 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 35 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 36 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 37 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 38 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 39 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 40 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 41 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 42 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 43 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 44 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 45 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 46 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 47 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 48 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 49 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 50 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 51 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 52 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 53 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 54 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 55 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 56 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 57 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 58 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 59 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 60 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 61 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 62 Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018) Slide 63
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

0 Likes

Share

Download to read offline

Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018)

Download to read offline

Biml for Beginners: Script and Automate SSIS development (Presented for 24 Hours of PASS: Summit Preview on June 13th 2018)

  • Be the first to like this

Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018)

  1. 1. Script and Automate SSIS Development Cathrine Wilhelmsen, Senior BI Consultant, Inmeta Moderated By: Narinder Sharma Biml for Beginners: Presenting Sponsor
  2. 2. If you require assistance during the session, type your inquiry into the question pane on the right side. Maximize your screen with the zoom button on the top of the presentation window. Please fill in the short evaluation following the session. It will appear in your web browser. Technical Assistance
  3. 3. Quest helps IT Professionals simplify administration tasks so they can focus on the evolving needs of their businesses. Combined with its commitment to help companies protect today’s investment while planning for the future, Quest continues to deliver the most comprehensive solutions to monitor, manage, protect and replicate database environments. Thank you to our Presenting Sponsor
  4. 4. PASS’ flagship event November 6-9 Seattle, Washington Free 1-day local training events Local user groups around the world Online special interest user groups Business analytics training Get involved Explore everything PASS has to offer Free Online Resources Newsletters PASS.org
  5. 5. Cathrine Wilhelmsen Senior BI Consultant, Inmeta CathrineW.net Sci-fi, chocolate, coffee, craft beers, ciders, cat gifs, and smilies :) Community Speaker, organizer, blogger, author, mentor, and chronic volunteer Data and Biml Microsoft Data Platform MVP BimlHero Certified Expert /cathrinewilhelmsen @cathrinew hi@cathrinew.net
  6. 6. Script and Automate SSIS Development Cathrine Wilhelmsen, Senior BI Consultant, Inmeta Biml for Beginners: Presenting Sponsor
  7. 7. Basic Biml Tools & Code SSIS from Metadata …the next 45 minutes…
  8. 8. once upon a time…
  9. 9. Meet Sandra, the SSIS developer
  10. 10. Sandra has to create a new staging project with 100+ SSIS packages
  11. 11. Sandra gets quite bored of creating the same package over and over and over
  12. 12. Yay, Sandra has finally completed her project!
  13. 13. But Sandra's manager has just a few new requirements…
  14. 14. Sandra gets to update every single SSIS package one… more… time…
  15. 15. …yay
  16. 16. It's time for a change!
  17. 17. Business Intelligence Markup Language Easy to read and write XML language Describes business intelligence objects: • Databases, Schemas, Tables, Views, Columns • SSIS Packages • SSAS and Azure Data Factory * What is Biml?
  18. 18. Why use Biml? SSIS: Plumbing Biml: Business Logic
  19. 19. Spend time on dragging, dropping, connecting, aligning Create the same package over and over again with minor changes Standards, patterns and templates must be defined up-front Changes must be done in every single package Higher risk of manual errors More packages, more time SSIS: Plumbing
  20. 20. Spend time on what is unique in a package Create a pattern once and reuse for all similar packages Handle scope and requirement changes quickly and easily Changes can be applied to all packages at once Lower risk of manual errors Longer time to start, but then automate and reuse Biml: Business Logic
  21. 21. How does SSIS development work?
  22. 22. How does Biml development work?
  23. 23. Biml is a tool for generating SSIS packages Biml is not a pre-defined Data Warehouse framework Biml is not a tool for automating deployment Will Biml solve all your challenges?
  24. 24. Timesaving: Many SSIS packages from one Biml file Reusable: Write once and run on any platform Flexible: Start simple, expand as you learn How can Biml help you?
  25. 25. Source Control: Keep track of actual logic changes Simplify: Untangle complex packages for quick edits Scripts: Use Biml instead of dynamic T-SQL What if you don't have 100+ packages?
  26. 26. What do you need?
  27. 27. Free add-in for Visual Studio Code editor with syntax highlighting and Biml Intellisense More frequent updates than BIDS Helper varigence.com/bimlexpress BimlExpress
  28. 28. Licensed full-featured development environment for Biml Visual designer and metadata modeling Full-stack automation and transformers varigence.com/bimlstudio BimlStudio
  29. 29. Biml syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  30. 30. Biml syntax: Root Element <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  31. 31. Biml syntax: Collections of Elements <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  32. 32. Biml syntax: Elements <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  33. 33. Biml syntax: Attributes <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  34. 34. Biml syntax: Full vs. Shorthand Syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  35. 35. DEMO Let's generate some packages!
  36. 36. Ok, so we can go from Biml to SSIS…
  37. 37. …can we go from SSIS to Biml?
  38. 38. Yes!
  39. 39. DEMO Let's convert some SSIS to Biml!
  40. 40. The magic is in the…
  41. 41. Extend Biml with C# or VB code blocks Import database structure and metadata Loop over tables and columns Expressions replace static values Generate, control and manipulate Biml code What is BimlScript?
  42. 42. How does it work?
  43. 43. Yes, but how does it work?
  44. 44. BimlScript Code Blocks <# … #> Control Block (Variables and logic) <#= … #> Text Block (Returns string) <#* … *#> Comment Block (Disable BimlScript) <#@ … #> Directive (Compiler instructions) <#+ … #> Class Block (C# classes and methods)
  45. 45. BimlScript Syntax <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  46. 46. BimlScript Syntax: Import metadata <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  47. 47. BimlScript Syntax: Loop over tables <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  48. 48. BimlScript Syntax: Replace static values <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  49. 49. Yes, but how does it actually work? <# foreach (var table in RootNode.Tables) { #> <Package Name="Load_<#=table.Name#>" /> <# } #> <Package Name="Load_Customer" /> <Package Name="Load_Product" /> <Package Name="Load_Sales" />
  50. 50. PreviewPane The power is in the…
  51. 51. DEMO Let's generate many packages!
  52. 52. New project
  53. 53. Of course I can create 200 SSIS Packages! …what do you need me to do after lunch?
  54. 54. Where can you learn more? Free online training bimlscript.com
  55. 55. Get things done Start small Start simple Start with ugly code Keep going Expand Improve Deliver often
  56. 56. Biml on Monday… …BimlBreak the rest of the week!
  57. 57. @cathrinew cathrinew.net hi@cathrinew.net Biml resources and demo files: cathrinew.net/biml
  58. 58. Questions?
  59. 59. Coming up next… Improve App Performance Using Async Messaging with Service Broker Allen White
  60. 60. Thank you for attending! @sqlpass #sqlpass @PASScommunity
  61. 61. Presenting Sponsor

Biml for Beginners: Script and Automate SSIS development (Presented for 24 Hours of PASS: Summit Preview on June 13th 2018)

Views

Total views

1,061

On Slideshare

0

From embeds

0

Number of embeds

6

Actions

Downloads

11

Shares

0

Comments

0

Likes

0

×