--session donnée lors du SQL Saturday Edinburgh 2015-- http://www.sqlsaturday.com/388/EventHome.aspx Data Management Gateway (and also AS Connector) is what make modern Microsoft BI stack hybrid. Power BI and Azure Data Factory use that component to interact with On-Prem Data assets. That session is a Deep dive into the DMG and the hybrid architecture involved by Power BI and ADF. How does it work ? Security, Firewall, Certificates, Multiple gateways, Admin delegation, Scale out, Disaster Recovery…. All that topics will be covered during that technical session.

1. Data Management Gateway
Deep Dive

2. Thanks to our Main Sponsors:

3. Say Thank you to Volunteers:
 They spend their FREE time to give you this
event.
 Because they are crazy. 
 Because they want YOU
to learn from the BEST IN THE WORLD.

4. Jean-Pierre Riehl
Practice Manager Data & BI
@djeepy1 - http://blog.djeepy1.net
Who am I ?
Pure-Player Microsoft
 Self-Service BI experts
 « Agile BI » projects
 Data Strategy
 Hybrid-BI
MVP SQL Server
French PASS
Chapter Leader

5. La communauté Data Microsoft
Webcasts, Conférences, Afterworks
.Pro
Save the Date
September, 5th
Campus SUPINFO
Tour Montparnasse, Paris
Self-promotion

6. An hybrid story…

7. Microsoft Cloud-First strategy…

8. But your data remains On-Prem…

9. Here it comes DMG

10. DMG-aware tools
Power BI for O365
PowerBI.com
Azure Data Factory

11. Power BI
Excel
Power Query Power Pivot
Power View Power Map
Power BI Tenant SQL
Cloud On-Prem
Oracle …
Power BI
Admin Center
Data
Catalog
Q&A
Power BI Sites
Data Refresh
Indexing / Search
OData
Access

12. PowerBI.com
Live Power BI
reports &
dashboards
Live
Query
Scheduled
Refresh
Cloud
On-premises

13. Azure Data Factory

14. Hybrid features
 Data Refresh
 Data Movement
 Live Query
 Indexing Sources
 OData Access (Data Virtualization)

15. One DMG to rule them all
DMZ
SQL
Cloud On-Prem
Oracle
…
Power BI for O365
Data
Management
Gateway
OnPrem Data Sources
PowerBI.com
Azure Data Factory
Excel

16. DMG DETAILS

17. Versions of DMG
 There are multiple versions of DMG
Data Management Gateway
Power BI
ADF
1.4.5549.1 24/3/2015
Analysis Services Connector Power BI v2 1.4.5576.2 17/4/2015
Breaking News: 1.4.5612.1 (released 26/5)

18. What’s that
 A Windows Service
 DIAHostService
 A configuration application
 Managed by the cloud
 Power BI Admin Center
 Azure Portal (new version)

20. FIRST LOOK AT DMG

21. Data Sources Supported
Source Authentification
SQL Server / Oracle
Sybase, Teradata, DB2, MySQL
Windows, Basic
Files (CSV, XML, Text, Excel, Access)
Folders
Windows
SharePoint List (Online)
SharePoint List (On-prem)
Anonymous
Anonymous, Windows
OData Feed
Anonymous, Windows, Basic
Key (Data Market)
Azure Marketplace Key (Data Market)
Azure HDInsight
Azure Blob Storage
Azure Table Storage
Key (Azure Account)
SQL Azure
PostgreSQL
Basic
Teradata, Sybase, DB2, MySQL Windows, Basic
SSAS Tabular (Live Query) Windows
+SSIS Data
Feed

22. Data Sources supported
 Details on data sources supported
 https://support.office.com/en-us/article/Supported-Data-Sources-
and-Data-Types
 Prequisites
 Beware of versions
 Beware of types
 Drivers installed
 https://support.office.com/en-us/article/Data-source-prerequisites-
Power-Query

23. HOW DOES IT WORK

24. Cloud On-Prem
Data Refresh explained
Power BI
OnPrem
Sources
DMG
Azure Service
Bus
0
Register DMG
Register Data Sources
1
Ask for Data Refresh
Check permissions
Find Gateway
4 Execute Query
5 Get Data
6 Compress, Chunk
Queue
request2
3 Read
request
Gateway
Cloud Service
7 Update Model

25. Cloud On-Prem
OData access explained
Power BI
OnPrem
Sources
DMG
0 Index Sources
2
Resolve Data Source
Redirect to local Gateway
4 Execute Query
3 Request Data
1 Search Source
Excel
Power Query
Hybrid Proxy

26. DMG IN ACTION

27. Data Source Discovery
 Data Sources are declared on « Admin
Center »
 They are associated to a Gateway
 Gateway Cloud Service resolves Data
Source and find right Gateway
 Resolution = Provider + Server + Catalog
 Warning: Case sensitive

28. DATA SOURCE DISCOVERY

29. SECURITY

30. Excel
Power Query
Power BI
DMG
Data sources
• SQL (1433)
• FileShare
• Etc.
+DNS resolution
Azure Service Bus (outbound)
• 9350-9354 / 5671,5672
Power BI Admin Center (outbound)
• 80/443
Direct HTTP
(inbound)
• HTTPS (8050)
• HTTP (8051)
HTTPS
(443)
Cloud
On-Prem
Network Security
Credential
Manager

31. Identity Management
2 flavors :
 Impersonation by DMG
 Impersonation by Data Source
How is query authentified on Data Source ?

32. Impersonation by DMG
 Connection identity is impersonated by DMG
 Peer-to-peer connection
 Credentials are stored by DMG for each
source
 Credentials can be stored encrypted locally
 Credentials can be stored encrypted on Cloud
 Credentials are set up locally
 Credential Manager is a Click Once application
 Credential Manager discuss with DMG

33. CREDENTIAL MANAGER

34. Impersonation by Data Source
 Used by Live Query
 Keep identity of original user
 Use the EffectiveUsername method
 DMG authenticates to SSAS
 DMG passes EffectiveUsername property
 SSAS executes the query as if it was original user
 DMG must be SSAS Admin

35. EffectiveUsername

36. HIGH AVAILABILITY &
DISASTER RECOVERY

37. As easy as scale-out
DMGs
Cloud
On-Prem
Power BI
…
Instance 1 Instance 2
Server A Server B

38. HA & DR
 Gateways and Data sources are managed by
Cloud
 Credentials are encrypted by MDG
 Use of Certificate, self-generated or yours
 Need Certificate when registering a new instance
 Warning when stored localy
 Load Balancing

39. HA & DR

40. Going further
 Express Route
 Mashup Container (aka Power Query)
 Monitoring & Logging

41. Resources
 Getting Started : https://support.office.com/en-us/article/Power-BI-for-Office-365-Admin-Center-Help-
5e391ecb-500c-47a3-bd0f-a6173b541044
 Data Factory Reference : https://msdn.microsoft.com/en-us/library/azure/dn834987.aspx
 Supported Data sources : https://support.office.com/en-us/article/Supported-Data-Sources-and-Data-
Types
 DMG Prerequisites : https://support.office.com/en-us/article/Data-source-prerequisites-Power-Query
 SSIS Data Feed : http://www.microsoft.com/en-us/download/details.aspx?id=39931
 How does DMG work - sequence diagram :
 https://support.office.com/en-us/article/An-Inside-look-at-Microsoft-Data-Management-Gateway-
078D306D-2D28-409C-962B-D80B69A1D0B2
 Azure IP adresses : http://www.microsoft.com/en-us/download/details.aspx?id=41653
 Store encrypted credentials in the cloud for business continuity : https://support.office.com/en-
us/article/Business-Continuity-for-a-Data-Management-Gateway-00106c58-a0a1-4cea-8886-
da51b8cadc5b
 Matt Masson Session at SPC : http://channel9.msdn.com/Events/SharePoint-
Conference/2014/SPC397
 Mini Nay & Luming Han Session at PASS Summit :
http://www.sqlpass.org/summit/2014/Sessions.aspx#sid7231

42. @djeepy1 - http://blog.djeepy1.net
Any questions ?
Thank You!