Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BI Markup Language - BI to the next level

3,903 views

Published on

BIML is an XML-based language that allows us to completely model a BI solution . It's particularly interesting for the automatic creation of ETL processes , for which it can be used free of charge via the BIDS Helper, a free tool that should be known to all those who develop BI solutions with the platform Microsoft. In this session we will learn the basics and some advanced trick , how to use it and how it can help to significantly reduce the development time of an ETL solution and at the same time increase the quality

Published in: Data & Analytics
  • Be the first to comment

BI Markup Language - BI to the next level

  1. 1. BI Markup Language - BI to the next level Davide Mauri SolidQ – Founder & Mentor @mauridb www.solidq.com dmauri@solidq.com #sqlsatParma #sqlsat355 November 22nd, 2014
  2. 2. Sponsors #sqlsatParma #sqlsat355 November 22nd, 2014
  3. 3. Organizers #sqlsatParma #sqlsat355 November 22nd, 2014
  4. 4. Davide Mauri  20 Years of experience on the SQL Server Platform  Specialized in Data Solution Architecture, Database Design, Performance Tuning, Business Intelligence  Projects, Consulting, Mentoring & Training  Regular Speaker @ SQL Server events  Microsoft SQL Server MVP  President of UGISS (Italian SQL Server UG)  Mentor @ SolidQ  Find me here:  Blog: http://sqlblog.com/blogs/davide_mauri/default.aspx  Twitter: @mauridb #sqlsatParma #sqlsat355 November 22nd, 2014
  5. 5. Agenda  BI, DWH & ETL today  BIML – What’s that?  Tools  BIDS Helper  MIST  BIML Kick Off  Basic Concepts  Automation With “Embedded” C#  BIML & PowerShell #sqlsatParma #sqlsat355 November 22nd, 2014
  6. 6. Building a BI/DWH/ETL in 2014 Is still a (almost) manual process A *lot* of repetitive low-value work No (or very few) standard tools available #sqlsatParma #sqlsat355 November 22nd, 2014
  7. 7. How it should be Semi-automatic process  “develop by intent” CREATE DIMENSION Customer FROM SourceCustomerTable MAP USING CustomerMetadata ALTER DIMENSION Customers ADD ATTRIBUTE LoyaltyLevel AS TYPE 1 Define the mapping logic from a semantic perspective  Source to Dimensions / Measures  (Metadata anyone?) CREATE FACT Orders FROM SourceOrdersTable MAP USING OrdersMetadata ALTER FACT Orders ADD DIMENSION Customer Design the model and let the tool build it for you #sqlsatParma #sqlsat355 November 22nd, 2014
  8. 8. The perfect BI process & architecture Iterative! #sqlsatParma #sqlsat355 November 22nd, 2014
  9. 9. Design Pattern “A general reusable solution to a commonly occurring problem within a given context” #sqlsatParma #sqlsat355 November 22nd, 2014
  10. 10. No Monkey Work! Let the people think and let the machines do the «monkey» work. #sqlsatParma #sqlsat355 November 22nd, 2014
  11. 11. BIML – What’s that?  Business Intelligence Markup Language  XML-Based  BI Object-Generator  SQL Server  Integration Services  Analysis Services #sqlsatParma #sqlsat355 November 22nd, 2014
  12. 12. BIML – What’s that?  Allows a declarative approach to define BI objects  Favor automation  Human-readable  Compatible with SCC system  Allows merge of files  Born from “Project Vulcan”  Still available on CodePlex #sqlsatParma #sqlsat355 November 22nd, 2014
  13. 13. BIML – What’s that?  Once objects are generated, BIML is not needed anymore  No additional runtime/components required  Support for SQL Server 2008 & 2012 objects  2014 is coming very soon  Anyway generated packages are automatically converted into 2014 packages by Visual Studio 2013, without problems  #sqlsatParma #sqlsat355 November 22nd, 2014
  14. 14. BIML – What’s that?  BIMLScript is based on the T4 templating engine. Allows the generation of BIML using  C#  VB.NET  BIML + BIMLScript = HTML + ASP Classic #sqlsatParma #sqlsat355 November 22nd, 2014
  15. 15. Tools  BIDS Helper  http://bidshelper.codeplex.com/  Visual Studio Plugin  Free  Source Code Available  Not all BIML features are supported  “Limited” to SSIS generation only #sqlsatParma #sqlsat355 November 22nd, 2014
  16. 16. Tools  MIST  Full-Featured BIML IDE  Buy or Rent  Made by BIML creator: Varigence #sqlsatParma #sqlsat355 November 22nd, 2014
  17. 17. DEMO  First Contact with BIML #sqlsatParma #sqlsat355 November 22nd, 2014
  18. 18. Basic Concepts  A single BIML file can generate more than one SSIS Package  All SSIS object and related properties are exposed via BIML elements  Additional specific attribute: ConstraintMode  Linear or Parallel #sqlsatParma #sqlsat355 November 22nd, 2014
  19. 19. Basic Concepts  Parallel Constraint Mode  “Inputs” connected to “Outputs”  SSIS Variables: “dot” notation #sqlsatParma #sqlsat355 November 22nd, 2014
  20. 20. Basic Concepts  Control Flow and Data Flow 100% supported  In Data Flow it’s mandatory to define input and outputs  Since they aren’t constraints, but data streams  #sqlsatParma #sqlsat355 November 22nd, 2014
  21. 21. DEMO  BIML is your friend! #sqlsatParma #sqlsat355 November 22nd, 2014
  22. 22. Automation with BIMLScript  BIML Script (T4) Directives: <#@ e #> #sqlsatParma #sqlsat355 November 22nd, 2014
  23. 23. Automation with BIMLScript  BIML Script Tags: <# e #>  Allows usage of C# / VB.Net within BIML  The “=” symbol allows “in-line” code injection #sqlsatParma #sqlsat355 November 22nd, 2014
  24. 24. Automation with BIMLScript  Metadata is needed in order to define what to generate  Metadata can be extracted by  BIML Engine (Hadron)  T-SQL Queries #sqlsatParma #sqlsat355 November 22nd, 2014
  25. 25. Automation with BIMLScript  Metadata via BIML Engine (Hadron) #sqlsatParma #sqlsat355 November 22nd, 2014
  26. 26. Automation with BIMLScript  Metadata via SQL Query #sqlsatParma #sqlsat355 November 22nd, 2014
  27. 27. DEMO  BIMLScript RULES! #sqlsatParma #sqlsat355 November 22nd, 2014
  28. 28. BIML & PowerShell  What if you don’t like C#...  Or if BIMLScript looks to complex?  Especially without MIST?  One option is to generate BIML using PowerShell  Is just XML after all…. #sqlsatParma #sqlsat355 November 22nd, 2014
  29. 29. DEMO  BIML & PowerShell #sqlsatParma #sqlsat355 November 22nd, 2014
  30. 30. Conclusions  Use BIML!!!!  It helps to  Make better use of your time  Create your own ETL patterns  http://www.slideshare.net/davidemauri/automating-dwh-patterns-through- metadata  Support Refactoring & Enables Agility  Favor use of metadata #sqlsatParma #sqlsat355 November 22nd, 2014
  31. 31. References  BIML Script  http://bimlscript.com/  Stairway to BIML  http://www.sqlservercentral.com/stairway/100550/  Automating DWH Patterns Through Metadata  http://www.slideshare.net/davidemauri/automating-dwh-patterns- through-metadata #sqlsatParma #sqlsat355 November 22nd, 2014
  32. 32. Q&A  Questions? #sqlsatParma #sqlsat355 November 22nd, 2014
  33. 33. THANKS! #sqlsatParma #sqlsat355 #sqlsatParma #sqlsat355 November 22nd, 2014

×