BIML is an XML-based language that allows us to completely model a BI solution . It's particularly interesting for the automatic creation of ETL processes , for which it can be used free of charge via the BIDS Helper, a free tool that should be known to all those who develop BI solutions with the platform Microsoft. In this session we will learn the basics and some advanced trick , how to use it and how it can help to significantly reduce the development time of an ETL solution and at the same time increase the quality

1. BI Markup Language:
BI to the next level
Davide Mauri
SolidQ – Founder & Mentor
@mauridb
www.solidq.com
dmauri@solidq.com
13.12.2014

2. Thank you to all our SPONSORS!
13.12.2014

3. Davide Mauri
 20 Years of experience on the SQL Server Platform
 Specialized in Data Solution Architecture, Database Design,
Performance Tuning, Business Intelligence
 Projects, Consulting, Mentoring & Training
 Regular Speaker @ SQL Server events
 Microsoft SQL Server MVP
 President of UGISS (Italian SQL Server UG)
 Mentor @ SolidQ
 Find me here:
 Blog: http://sqlblog.com/blogs/davide_mauri/default.aspx
 Twitter: @mauridb
13.12.2014

4. Agenda
 BI, DWH & ETL today
 BIML – What’s that?
 Tools
 BIDS Helper
 MIST
 BIML Kick Off
 Basic Concepts
 Automation With “Embedded” C#
 BIML & PowerShell
13.12.2014

5. Building a BI/DWH/ETL in 2014
Is still a (almost) manual process
A *lot* of repetitive low-value work
No (or very few) standard tools available
13.12.2014

6. How it should be
Semi-automatic process
 “develop by intent”
Define the mapping logic from a semantic
perspective
 Source to Dimensions / Measures
 (Metadata anyone?)
Design the model and let the tool build it for you
13.12.2014

7. How it should be
Semi-automatic process
 “develop by intent”
Define the mapping logic from a semantic
perspective
 Source to Dimensions / Measures
 (Metadata anyone?)
Design the model and let the tool build it for you
13.12.2014
CREATE DIMENSION Customer
FROM SourceCustomerTable
MAP USING CustomerMetadata
ALTER DIMENSION Customers
ADD ATTRIBUTE LoyaltyLevel
AS TYPE 1
CREATE FACT Orders
FROM SourceOrdersTable
MAP USING OrdersMetadata
ALTER FACT Orders
ADD DIMENSION Customer

8. The perfect BI process & architecture
13.12.2014
Iterative!

9. Design Pattern
“A general reusable
solution to a
commonly occurring
problem within a
given context”
13.12.2014

10. No Monkey Work!
13.12.2014
Let the people think
and let the machines
do the «monkey»
work.

11. BIML – What’s that?
 Business Intelligence Markup Language
 XML-Based
 BI Object-Generator
 SQL Server
 Integration Services
 Analysis Services
13.12.2014

12. BIML – What’s that?
 Allows a declarative approach to define BI
objects
 Favor automation
 Human-readable
 Compatible with SCC system
 Allows merge of files
 Born from “Project Vulcan”
 Still available on CodePlex
13.12.2014

13. BIML – What’s that?
 Once objects are generated, BIML is not needed
anymore
 No additional runtime/components required
 Support for SQL Server 2008 & 2012 objects
 2014 is coming very soon
 Anyway generated packages are automatically
converted into 2014 packages by Visual Studio 2013,
without problems 
13.12.2014

14. BIML – What’s that?
 BIMLScript is based on the T4 templating engine.
Allows the generation of BIML using
 C#
 VB.NET
 BIML + BIMLScript = HTML + ASP Classic
13.12.2014

15. Tools
 BIDS Helper
 http://bidshelper.codeplex.com/
 Visual Studio Plugin
 Free
 Source Code Available
 Not all BIML features are supported
 “Limited” to SSIS generation only
13.12.2014

16. Tools
 MIST
 Full-Featured BIML IDE
 Buy or Rent
 Made by BIML creator: Varigence
13.12.2014

17. DEMO
 First Contact with BIML
13.12.2014

18. Basic Concepts
 A single BIML file can generate more than one
SSIS Package
 All SSIS object and related properties are exposed
via BIML elements
 Additional specific attribute: ConstraintMode
 Linear or Parallel
13.12.2014

19. Basic Concepts
 Parallel Constraint Mode
 “Inputs” connected to “Outputs”
 SSIS Variables: “dot” notation
13.12.2014

20. Basic Concepts
 Control Flow and Data Flow 100% supported
 In Data Flow it’s mandatory to define input and
outputs
 Since they aren’t constraints, but data streams 
13.12.2014

21. DEMO
 BIML is your friend!
13.12.2014

22. Automation with BIMLScript
 BIML Script (T4) Directives: <#@ and #>
13.12.2014

23. Automation with BIMLScript
 BIML Script Tags: <# e #>
 Allows usage of C# / VB.Net within BIML
 The “=” symbol allows “in-line” code injection
13.12.2014

24. Automation with BIMLScript
 Metadata is needed in order to define what to
generate
 Metadata can be extracted by
 BIML Engine (Hadron)
 T-SQL Queries
13.12.2014

25. Automation with BIMLScript
 Metadata via BIML Engine (Hadron)
13.12.2014

26. Automation with BIMLScript
 Metadata via SQL Query
13.12.2014

27. DEMO
 BIMLScript RULES!
13.12.2014

28. Debugging BIML
 With complex BIML Script it can be difficult to
“see” what’s happening behind the scenes
 With MIST you can see the resulting BIML
generated from BIML Script
 But with BIDS Helper you’re on your own.
 NLog can help a lot here!
13.12.2014

29. Debugging BIML
 Nlog: free, powerful & flexible logging
infrastructure
 http://nlog-project.org/
 Stand-Alone Assembly
 Just reference the assembly in the BIML Script (T4)
template and then use Nlog objects & methods
 http://sqlblog.com/blogs/davide_mauri/default.aspx
13.12.2014

30. DEMO
 BIMLScript & NLog
13.12.2014

31. BIML & PowerShell
 What if you don’t like C#...
 Or if BIMLScript looks to complex?
 Especially without MIST?
 One option is to generate BIML using PowerShell
 Is just XML after all….
13.12.2014

32. DEMO
 BIML & PowerShell
13.12.2014

33. Conclusions
 Use BIML!!!!
 It helps to
 Make better use of your time
 Create your own ETL patterns
 http://www.slideshare.net/davidemauri/automating-dwh-patterns-
13.12.2014
through-metadata
 Support Refactoring & Enables Agility
 Favor use of metadata

34. References
 BIML Script
 http://bimlscript.com/
 Stairway to BIML
 http://www.sqlservercentral.com/stairway/100550/
 Automating DWH Patterns Through Metadata
 http://www.slideshare.net/davidemauri/automating-dwh-
13.12.2014
patterns-through-metadata

35. Q&A
Questions?
13.12.2014

36. Thanks!
13.12.2014