Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018)

213 views

Published on

Biml for Beginners: Script and Automate SSIS development (Presented for 24 Hours of PASS: Summit Preview on June 13th 2018)

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Biml for Beginners: Script and Automate SSIS development (24 Hours of PASS: Summit Preview 2018)

  1. 1. Script and Automate SSIS Development Cathrine Wilhelmsen, Senior BI Consultant, Inmeta Moderated By: Narinder Sharma Biml for Beginners: Presenting Sponsor
  2. 2. If you require assistance during the session, type your inquiry into the question pane on the right side. Maximize your screen with the zoom button on the top of the presentation window. Please fill in the short evaluation following the session. It will appear in your web browser. Technical Assistance
  3. 3. Quest helps IT Professionals simplify administration tasks so they can focus on the evolving needs of their businesses. Combined with its commitment to help companies protect today’s investment while planning for the future, Quest continues to deliver the most comprehensive solutions to monitor, manage, protect and replicate database environments. Thank you to our Presenting Sponsor
  4. 4. PASS’ flagship event November 6-9 Seattle, Washington Free 1-day local training events Local user groups around the world Online special interest user groups Business analytics training Get involved Explore everything PASS has to offer Free Online Resources Newsletters PASS.org
  5. 5. Cathrine Wilhelmsen Senior BI Consultant, Inmeta CathrineW.net Sci-fi, chocolate, coffee, craft beers, ciders, cat gifs, and smilies :) Community Speaker, organizer, blogger, author, mentor, and chronic volunteer Data and Biml Microsoft Data Platform MVP BimlHero Certified Expert /cathrinewilhelmsen @cathrinew hi@cathrinew.net
  6. 6. Script and Automate SSIS Development Cathrine Wilhelmsen, Senior BI Consultant, Inmeta Biml for Beginners: Presenting Sponsor
  7. 7. Basic Biml Tools & Code SSIS from Metadata …the next 45 minutes…
  8. 8. once upon a time…
  9. 9. Meet Sandra, the SSIS developer
  10. 10. Sandra has to create a new staging project with 100+ SSIS packages
  11. 11. Sandra gets quite bored of creating the same package over and over and over
  12. 12. Yay, Sandra has finally completed her project!
  13. 13. But Sandra's manager has just a few new requirements…
  14. 14. Sandra gets to update every single SSIS package one… more… time…
  15. 15. …yay
  16. 16. It's time for a change!
  17. 17. Business Intelligence Markup Language Easy to read and write XML language Describes business intelligence objects: • Databases, Schemas, Tables, Views, Columns • SSIS Packages • SSAS and Azure Data Factory * What is Biml?
  18. 18. Why use Biml? SSIS: Plumbing Biml: Business Logic
  19. 19. Spend time on dragging, dropping, connecting, aligning Create the same package over and over again with minor changes Standards, patterns and templates must be defined up-front Changes must be done in every single package Higher risk of manual errors More packages, more time SSIS: Plumbing
  20. 20. Spend time on what is unique in a package Create a pattern once and reuse for all similar packages Handle scope and requirement changes quickly and easily Changes can be applied to all packages at once Lower risk of manual errors Longer time to start, but then automate and reuse Biml: Business Logic
  21. 21. How does SSIS development work?
  22. 22. How does Biml development work?
  23. 23. Biml is a tool for generating SSIS packages Biml is not a pre-defined Data Warehouse framework Biml is not a tool for automating deployment Will Biml solve all your challenges?
  24. 24. Timesaving: Many SSIS packages from one Biml file Reusable: Write once and run on any platform Flexible: Start simple, expand as you learn How can Biml help you?
  25. 25. Source Control: Keep track of actual logic changes Simplify: Untangle complex packages for quick edits Scripts: Use Biml instead of dynamic T-SQL What if you don't have 100+ packages?
  26. 26. What do you need?
  27. 27. Free add-in for Visual Studio Code editor with syntax highlighting and Biml Intellisense More frequent updates than BIDS Helper varigence.com/bimlexpress BimlExpress
  28. 28. Licensed full-featured development environment for Biml Visual designer and metadata modeling Full-stack automation and transformers varigence.com/bimlstudio BimlStudio
  29. 29. Biml syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  30. 30. Biml syntax: Root Element <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  31. 31. Biml syntax: Collections of Elements <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  32. 32. Biml syntax: Elements <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  33. 33. Biml syntax: Attributes <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  34. 34. Biml syntax: Full vs. Shorthand Syntax <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <Package Name="EmptyPackage1"></Package> <Package Name="EmptyPackage2"/> </Packages> </Biml>
  35. 35. DEMO Let's generate some packages!
  36. 36. Ok, so we can go from Biml to SSIS…
  37. 37. …can we go from SSIS to Biml?
  38. 38. Yes!
  39. 39. DEMO Let's convert some SSIS to Biml!
  40. 40. The magic is in the…
  41. 41. Extend Biml with C# or VB code blocks Import database structure and metadata Loop over tables and columns Expressions replace static values Generate, control and manipulate Biml code What is BimlScript?
  42. 42. How does it work?
  43. 43. Yes, but how does it work?
  44. 44. BimlScript Code Blocks <# … #> Control Block (Variables and logic) <#= … #> Text Block (Returns string) <#* … *#> Comment Block (Disable BimlScript) <#@ … #> Directive (Compiler instructions) <#+ … #> Class Block (C# classes and methods)
  45. 45. BimlScript Syntax <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  46. 46. BimlScript Syntax: Import metadata <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  47. 47. BimlScript Syntax: Loop over tables <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  48. 48. BimlScript Syntax: Replace static values <# var con = SchemaManager.CreateConnectionNode(...); #> <# var metadata = con.GetDatabaseSchema(); #> <Biml xmlns="http://schemas.varigence.com/biml.xsd"> <Packages> <# foreach (var table in metadata.TableNodes) { #> <Package Name="Load_<#=table.Name#>"></Package> <# } #> </Packages> </Biml>
  49. 49. Yes, but how does it actually work? <# foreach (var table in RootNode.Tables) { #> <Package Name="Load_<#=table.Name#>" /> <# } #> <Package Name="Load_Customer" /> <Package Name="Load_Product" /> <Package Name="Load_Sales" />
  50. 50. PreviewPane The power is in the…
  51. 51. DEMO Let's generate many packages!
  52. 52. New project
  53. 53. Of course I can create 200 SSIS Packages! …what do you need me to do after lunch?
  54. 54. Where can you learn more? Free online training bimlscript.com
  55. 55. Get things done Start small Start simple Start with ugly code Keep going Expand Improve Deliver often
  56. 56. Biml on Monday… …BimlBreak the rest of the week!
  57. 57. @cathrinew cathrinew.net hi@cathrinew.net Biml resources and demo files: cathrinew.net/biml
  58. 58. Questions?
  59. 59. Coming up next… Improve App Performance Using Async Messaging with Service Broker Allen White
  60. 60. Thank you for attending! @sqlpass #sqlpass @PASScommunity
  61. 61. Presenting Sponsor

×