SlideShare a Scribd company logo
EmPower your BI
Architecture
Riccardo Perico
Riccardo Perico
rperico@solidq.com | @R1k91 |
Data Platform & BI Specialist @
10 years in Microsoft «Data Realm»
MCP: MTA, MCSA
Speaker at Azure Saturday, SQL Saturday and DIAD tutor
Agenda? The Right Questions
1. What’s Storage Mode and which one best suits to me?
2. Do I need an On-premises Data Gateway?
3. How to refresh my data?
4. Is it just on-cloud solution?
5. Can I «encapsulate» PowerBI?
6. Do I need to care about dataflows?
7. Can I have some advisable scenarios?
What’s Storage Mode and
which one best suits to me?
Storage Mode: Import Mode
Data Model Cached into RAM
Storage Mode: Import Mode
• Model Cached inside Power BI Service
• High Compression thanks to VertiPaq (*)
• Full Power BI functionalities and DAX functions support
• Multiple Data Sources mashup (all available)
• 10 GB model size for PREMIUM / 1 GB for PRO
Large models for Power BI Premium
Storage Mode: Direct Query
Schema only
Storage Mode: Direct Query
• Limited data sources
• Not full Power Query and DAX support
• Pay attention to performance (1 render = 1 query)
• Overcome model size limits
• Real-time / Near Real-time scenarios
Direct Query: Quick Tip
Storage Mode: Live Connection
PowerBI acts just like an
interface
Storage Mode: Live Connection
• Data Sources: AAS, SSAS Tabular/Multidimensional and PBI dataset
• Data Preparation and Data Modeling done in the source
• Interaction with report = DAX query sent to source
• Overcome model size limits
• Partitioned Load
• Tends to be faster than DQ
• Never prefer «Import» over «Live Connection» for supported sources
Storage Mode: Composite Mode
Direct Query
Import Mode
Dual Mode
Do I need an On-premises
Data Gateway?
Let’s draw a typical scenario
Publish
On-Prem Data Gateway is the way
Azure Service BusOn-prem Data Gateway
Data Movement Service
Refresh Scheduler Service
2 Gateway types
On-premises data gateway
(Enterprise)
On-premises data gateway
(Personal mode)
All Storage Mode Support
Scheduled Refresh Support
Import Mode Support
Scheduled Refresh Support
Gateway: Architecture security
• Credentials stored locally and in Power BI Service
• Data Source credentials encrypted and stored in the cloud only gateway
can decrypt
• Gateway polling  No inbound port to open
• Outbound port to open TCP: 443, 5671, 5672, 9530...9534
Gateway: Considerations
• Keep gateways’ name/recovery key in a safe place (migration, take over
and recovery)
• Do not install Personal and then Enterprise
• Sometimes 2 gateways are needed: Import + DQ/Live
• Create a cluster for High-Availability: round robin or balanced load
• On-premises data gateway app helps you
• New Powershell 6 cmdlets to manage Gateway
Demo Preview
On-prem & On-Cloud Mashup
?
Azure Service BusOn-prem Data Gateway
1 Mashup Query On-Prem
1 Mashup Query On-Cloud
X1 Mashup Query for both On-Prem and On-
Cloud
Demo: Mashup and
Gateway
How to refresh my data?
Data are old… let’s refresh!
Storage Mode Refresh Needed
Import Mode Pro
Import Mode Premium
Direct Query
Live Connection
Max Refresh Per Day Max Refresh Time
8 2 hours
48 5 hours
- -
- -
+
Too much data?
FOR PRO IS cOMIng
Is it just on-cloud solution?
Cloud is not the only option
Publish / Upload
PBI Service Vs PBI Report Server
PBI Service PBI Report Server
Deploy Cloud On-Prem
Licensing Power BI Pro and/or Premium PBI Premium or SQL Server EE + SA
Release Cycle Every month Every 4 months
Gateway Required For on-prem source No
Realtime Streaming Yes No
Dashboard & Quick Insights Yes No
Apps Yes No
Q&A Yes No
Analyze in Excel Yes No
Paginated reports Yes Yes
https://docs.microsoft.com/it-it/power-bi/report-server/compare-report-server-service
Can I «encapsulate»
PowerBI?
Encapsulate Power BI: 2 options
Opt. 1: Embedded in SharePoint Online
Pros
• Centralized UX
• Easy setup
Cons
• Only reports
• Users won’t use service
• Permissions management
Opt. 2: Power BI Embedded
Pros
• Full custom application
• Reports, Dashboards, Datasets and
also Q&A
• End user do not need a Power BI
Pro License
Cons
• Licensing: buckets of range
renders
• Registration and management
• Not all service’s features
• Web developer needed
Do I need to care about
dataflows?
dataflows: as-is
Power Query
Sales.pbix
Customer
Date
Sales
Power Query
Purchases.pbix
Vendor
Date
Purchases
dataflows: a new paradigm
Sales.pbix
Customer
Date
Sales
PQ in dataflows
Purchases.pbix
Vendor
Purchases
PQ in dataflows
Azure Data Lake
Storage V2 (CDM)
dataflows: 2 steps forward
Customer
Date
Sales
PQ in dataflows
Vendor
Purchases
PQ in dataflows
Azure Data Lake
Storage V2 (CDM)
Power BI Dataset
Data Preparation Data Modeling Data Presentation
dataflows: pros
• Centralization of data preparation (reusability and standardization)
• Isolating refresh (1 table a time)
• Attach an existing ADLS v2 (preview)
• Speed up dataset refresh
dataflows: summary pt. 1
Storage Mode: Live Connection
https://ssbipolar.com/2019/10/14/power-bite-dataflows-vs-datasets/
Demo: dataflows, datasets,
refresh... all in!
Can I have some advisable
scenarios?
Some guidelines
Relational DWH AAS/SSAS Model Size Advice
Yes No Under 1 GB Storage Mode: Import Data
License: Pro
Data Preparation: Power Query or dataflows
No No Under 1 GB Storage Mode: Import Data (also shared dataset)
License: Pro
Data Preparation: dataflows
Yes Yes - Storage Mode: Live Query Connection
License: Pro
Yes/No No Over 1 GB Storage Mode: Import Data
License: Premium
Data Preparation: Power Query or dataflows
Yes/No No Over 1 GB Storage Mode: Direct Query or Dual Mode with Aggregations
License: Pro
Data Preparation: Power Query
Useful Links
• Live or Import? http://tiny.cc/fa9uaz
• Gateway http://tiny.cc/lb9uaz
• PowerShell Cmdlets for data gateway: http://tiny.cc/ebia1
• Incremental Refresh: http://tiny.cc/4c9uaz
• dataflows & CDM: http://tiny.cc/5d9uaz
• Dataflows best practices: http://tiny.cc/dmgebi
• Shared Datasets replace SSAS: http://tiny.cc/8imtbz
Riccardo Perico
rperico@solidq.com | @R1k91 |
Data Platform & BI Specialist @
10 years in Microsoft «Data Realm»
MCP: MTA, MCSA
Speaker at Azure Saturday, SQL Saturday and DIAD tutor
Q&A
Thank You
Riccardo Perico

More Related Content

What's hot

How Orwell built a geo-distributed Bank-as-a-Service with microservices
How Orwell built a geo-distributed Bank-as-a-Service with microservicesHow Orwell built a geo-distributed Bank-as-a-Service with microservices
How Orwell built a geo-distributed Bank-as-a-Service with microservices
MariaDB plc
 
Scaling Your Database in the Cloud
Scaling Your Database in the CloudScaling Your Database in the Cloud
Scaling Your Database in the Cloud
RightScale
 
SQL Server 2014 New Features
SQL Server 2014 New FeaturesSQL Server 2014 New Features
SQL Server 2014 New Features
Onomi
 
Power BI: Tips and Tricks
Power BI: Tips and TricksPower BI: Tips and Tricks
Power BI: Tips and Tricks
GlobalLogic Ukraine
 
How to power microservices with MariaDB
How to power microservices with MariaDBHow to power microservices with MariaDB
How to power microservices with MariaDB
MariaDB plc
 
What'sNnew in 3.0 Webinar
What'sNnew in 3.0 WebinarWhat'sNnew in 3.0 Webinar
What'sNnew in 3.0 Webinar
MongoDB
 
Caching for Microservices Architectures: Session I
Caching for Microservices Architectures: Session ICaching for Microservices Architectures: Session I
Caching for Microservices Architectures: Session I
VMware Tanzu
 
Maximizing performance via tuning and optimization
Maximizing performance via tuning and optimizationMaximizing performance via tuning and optimization
Maximizing performance via tuning and optimization
MariaDB plc
 
CCV: migrating our payment processing system to MariaDB
CCV: migrating our payment processing system to MariaDBCCV: migrating our payment processing system to MariaDB
CCV: migrating our payment processing system to MariaDB
MariaDB plc
 
JBoss Data Virtualization (JDV) Sample Physical Deployment Architecture
JBoss Data Virtualization (JDV) Sample Physical Deployment ArchitectureJBoss Data Virtualization (JDV) Sample Physical Deployment Architecture
JBoss Data Virtualization (JDV) Sample Physical Deployment Architecture
ejlp12
 
What's new in SQL Server 2016
What's new in SQL Server 2016What's new in SQL Server 2016
What's new in SQL Server 2016
Onomi
 
Temporal Tables, Transparent Archiving in DB2 for z/OS and IDAA
Temporal Tables, Transparent Archiving in DB2 for z/OS and IDAATemporal Tables, Transparent Archiving in DB2 for z/OS and IDAA
Temporal Tables, Transparent Archiving in DB2 for z/OS and IDAA
Cuneyt Goksu
 
Scalable relational database with SQL Azure
Scalable relational database with SQL AzureScalable relational database with SQL Azure
Scalable relational database with SQL Azure
Shy Engelberg
 
ETL with Clustered Columnstore - PASS Summit 2014
ETL with Clustered Columnstore - PASS Summit 2014ETL with Clustered Columnstore - PASS Summit 2014
ETL with Clustered Columnstore - PASS Summit 2014
Niko Neugebauer
 
Azure SQL Database
Azure SQL DatabaseAzure SQL Database
Azure SQL Database
rockplace
 
Webinar: Introduction to MongoDB 3.0
Webinar: Introduction to MongoDB 3.0Webinar: Introduction to MongoDB 3.0
Webinar: Introduction to MongoDB 3.0
MongoDB
 
Things Every Oracle DBA Needs to Know About the Hadoop Ecosystem 20170527
Things Every Oracle DBA Needs to Know About the Hadoop Ecosystem 20170527Things Every Oracle DBA Needs to Know About the Hadoop Ecosystem 20170527
Things Every Oracle DBA Needs to Know About the Hadoop Ecosystem 20170527
Zohar Elkayam
 
What SQL DBAs need to know about SharePoint
What SQL DBAs need to know about SharePointWhat SQL DBAs need to know about SharePoint
What SQL DBAs need to know about SharePointJ.D. Wade
 
VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld
 
4. (mjk) extreme performance 2
4. (mjk) extreme performance 24. (mjk) extreme performance 2
4. (mjk) extreme performance 2
Doina Draganescu
 

What's hot (20)

How Orwell built a geo-distributed Bank-as-a-Service with microservices
How Orwell built a geo-distributed Bank-as-a-Service with microservicesHow Orwell built a geo-distributed Bank-as-a-Service with microservices
How Orwell built a geo-distributed Bank-as-a-Service with microservices
 
Scaling Your Database in the Cloud
Scaling Your Database in the CloudScaling Your Database in the Cloud
Scaling Your Database in the Cloud
 
SQL Server 2014 New Features
SQL Server 2014 New FeaturesSQL Server 2014 New Features
SQL Server 2014 New Features
 
Power BI: Tips and Tricks
Power BI: Tips and TricksPower BI: Tips and Tricks
Power BI: Tips and Tricks
 
How to power microservices with MariaDB
How to power microservices with MariaDBHow to power microservices with MariaDB
How to power microservices with MariaDB
 
What'sNnew in 3.0 Webinar
What'sNnew in 3.0 WebinarWhat'sNnew in 3.0 Webinar
What'sNnew in 3.0 Webinar
 
Caching for Microservices Architectures: Session I
Caching for Microservices Architectures: Session ICaching for Microservices Architectures: Session I
Caching for Microservices Architectures: Session I
 
Maximizing performance via tuning and optimization
Maximizing performance via tuning and optimizationMaximizing performance via tuning and optimization
Maximizing performance via tuning and optimization
 
CCV: migrating our payment processing system to MariaDB
CCV: migrating our payment processing system to MariaDBCCV: migrating our payment processing system to MariaDB
CCV: migrating our payment processing system to MariaDB
 
JBoss Data Virtualization (JDV) Sample Physical Deployment Architecture
JBoss Data Virtualization (JDV) Sample Physical Deployment ArchitectureJBoss Data Virtualization (JDV) Sample Physical Deployment Architecture
JBoss Data Virtualization (JDV) Sample Physical Deployment Architecture
 
What's new in SQL Server 2016
What's new in SQL Server 2016What's new in SQL Server 2016
What's new in SQL Server 2016
 
Temporal Tables, Transparent Archiving in DB2 for z/OS and IDAA
Temporal Tables, Transparent Archiving in DB2 for z/OS and IDAATemporal Tables, Transparent Archiving in DB2 for z/OS and IDAA
Temporal Tables, Transparent Archiving in DB2 for z/OS and IDAA
 
Scalable relational database with SQL Azure
Scalable relational database with SQL AzureScalable relational database with SQL Azure
Scalable relational database with SQL Azure
 
ETL with Clustered Columnstore - PASS Summit 2014
ETL with Clustered Columnstore - PASS Summit 2014ETL with Clustered Columnstore - PASS Summit 2014
ETL with Clustered Columnstore - PASS Summit 2014
 
Azure SQL Database
Azure SQL DatabaseAzure SQL Database
Azure SQL Database
 
Webinar: Introduction to MongoDB 3.0
Webinar: Introduction to MongoDB 3.0Webinar: Introduction to MongoDB 3.0
Webinar: Introduction to MongoDB 3.0
 
Things Every Oracle DBA Needs to Know About the Hadoop Ecosystem 20170527
Things Every Oracle DBA Needs to Know About the Hadoop Ecosystem 20170527Things Every Oracle DBA Needs to Know About the Hadoop Ecosystem 20170527
Things Every Oracle DBA Needs to Know About the Hadoop Ecosystem 20170527
 
What SQL DBAs need to know about SharePoint
What SQL DBAs need to know about SharePointWhat SQL DBAs need to know about SharePoint
What SQL DBAs need to know about SharePoint
 
VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right VMworld 2013: Virtualizing Databases: Doing IT Right
VMworld 2013: Virtualizing Databases: Doing IT Right
 
4. (mjk) extreme performance 2
4. (mjk) extreme performance 24. (mjk) extreme performance 2
4. (mjk) extreme performance 2
 

Similar to PPWT2019 - EmPower your BI architecture

Datapolis Guest Expert Presentation: Top 15 SharePoint Server Configuration M...
Datapolis Guest Expert Presentation: Top 15 SharePoint Server Configuration M...Datapolis Guest Expert Presentation: Top 15 SharePoint Server Configuration M...
Datapolis Guest Expert Presentation: Top 15 SharePoint Server Configuration M...
Datapolis
 
Le novità di SQL Server 2022
Le novità di SQL Server 2022Le novità di SQL Server 2022
Le novità di SQL Server 2022
Gianluca Hotz
 
Deep Dive - Usage of on premises data gateway for hybrid integration scenarios
Deep Dive - Usage of on premises data gateway for hybrid integration scenariosDeep Dive - Usage of on premises data gateway for hybrid integration scenarios
Deep Dive - Usage of on premises data gateway for hybrid integration scenarios
Sajith C P Nair
 
Sql Start! 2020 - SQL Server Lift & Shift su Azure
Sql Start! 2020 - SQL Server Lift & Shift su AzureSql Start! 2020 - SQL Server Lift & Shift su Azure
Sql Start! 2020 - SQL Server Lift & Shift su Azure
Marco Obinu
 
CCI2018 - Real-time dashboard whatif analysis
CCI2018 - Real-time dashboard whatif analysisCCI2018 - Real-time dashboard whatif analysis
CCI2018 - Real-time dashboard whatif analysis
walk2talk srl
 
Data exposure in Azure - production use-case
Data exposure in Azure - production use-caseData exposure in Azure - production use-case
Data exposure in Azure - production use-case
Alexander Laysha
 
How Data Drives Business at Choice Hotels
How Data Drives Business at Choice HotelsHow Data Drives Business at Choice Hotels
How Data Drives Business at Choice Hotels
Cloudera, Inc.
 
Feature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine LearningFeature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine Learning
Provectus
 
Serverless SQL
Serverless SQLServerless SQL
Serverless SQL
Torsten Steinbach
 
New enhancements for security and usability in EDB 13
New enhancements for security and usability in EDB 13New enhancements for security and usability in EDB 13
New enhancements for security and usability in EDB 13
EDB
 
SQL PASS Taiwan 七月份聚會-1
SQL PASS Taiwan 七月份聚會-1SQL PASS Taiwan 七月份聚會-1
SQL PASS Taiwan 七月份聚會-1
SQLPASSTW
 
SQL Server Lift & Shift on Azure - SQL Saturday 921
SQL Server Lift & Shift on Azure - SQL Saturday 921SQL Server Lift & Shift on Azure - SQL Saturday 921
SQL Server Lift & Shift on Azure - SQL Saturday 921
Marco Obinu
 
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio JourneyModernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Alluxio, Inc.
 
GPPB2020 - Milan - Power BI dataflows deep dive
GPPB2020 - Milan - Power BI dataflows deep diveGPPB2020 - Milan - Power BI dataflows deep dive
GPPB2020 - Milan - Power BI dataflows deep dive
Riccardo Perico
 
Introducing Azure SQL Data Warehouse
Introducing Azure SQL Data WarehouseIntroducing Azure SQL Data Warehouse
Introducing Azure SQL Data Warehouse
James Serra
 
Exploring sql server 2016
Exploring sql server 2016Exploring sql server 2016
Exploring sql server 2016
Antonios Chatzipavlis
 
SQL Server 2019 CTP2.4
SQL Server 2019 CTP2.4SQL Server 2019 CTP2.4
SQL Server 2019 CTP2.4
Gianluca Hotz
 
Planning a Power BI deployment?
Planning a Power BI deployment?Planning a Power BI deployment?
Planning a Power BI deployment?
Senturus
 
Should I move my database to the cloud?
Should I move my database to the cloud?Should I move my database to the cloud?
Should I move my database to the cloud?
James Serra
 
What we learned at pass summit in 2018
What we learned at pass summit in 2018What we learned at pass summit in 2018
What we learned at pass summit in 2018
Red Gate Software
 

Similar to PPWT2019 - EmPower your BI architecture (20)

Datapolis Guest Expert Presentation: Top 15 SharePoint Server Configuration M...
Datapolis Guest Expert Presentation: Top 15 SharePoint Server Configuration M...Datapolis Guest Expert Presentation: Top 15 SharePoint Server Configuration M...
Datapolis Guest Expert Presentation: Top 15 SharePoint Server Configuration M...
 
Le novità di SQL Server 2022
Le novità di SQL Server 2022Le novità di SQL Server 2022
Le novità di SQL Server 2022
 
Deep Dive - Usage of on premises data gateway for hybrid integration scenarios
Deep Dive - Usage of on premises data gateway for hybrid integration scenariosDeep Dive - Usage of on premises data gateway for hybrid integration scenarios
Deep Dive - Usage of on premises data gateway for hybrid integration scenarios
 
Sql Start! 2020 - SQL Server Lift & Shift su Azure
Sql Start! 2020 - SQL Server Lift & Shift su AzureSql Start! 2020 - SQL Server Lift & Shift su Azure
Sql Start! 2020 - SQL Server Lift & Shift su Azure
 
CCI2018 - Real-time dashboard whatif analysis
CCI2018 - Real-time dashboard whatif analysisCCI2018 - Real-time dashboard whatif analysis
CCI2018 - Real-time dashboard whatif analysis
 
Data exposure in Azure - production use-case
Data exposure in Azure - production use-caseData exposure in Azure - production use-case
Data exposure in Azure - production use-case
 
How Data Drives Business at Choice Hotels
How Data Drives Business at Choice HotelsHow Data Drives Business at Choice Hotels
How Data Drives Business at Choice Hotels
 
Feature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine LearningFeature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine Learning
 
Serverless SQL
Serverless SQLServerless SQL
Serverless SQL
 
New enhancements for security and usability in EDB 13
New enhancements for security and usability in EDB 13New enhancements for security and usability in EDB 13
New enhancements for security and usability in EDB 13
 
SQL PASS Taiwan 七月份聚會-1
SQL PASS Taiwan 七月份聚會-1SQL PASS Taiwan 七月份聚會-1
SQL PASS Taiwan 七月份聚會-1
 
SQL Server Lift & Shift on Azure - SQL Saturday 921
SQL Server Lift & Shift on Azure - SQL Saturday 921SQL Server Lift & Shift on Azure - SQL Saturday 921
SQL Server Lift & Shift on Azure - SQL Saturday 921
 
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio JourneyModernizing Global Shared Data Analytics Platform and our Alluxio Journey
Modernizing Global Shared Data Analytics Platform and our Alluxio Journey
 
GPPB2020 - Milan - Power BI dataflows deep dive
GPPB2020 - Milan - Power BI dataflows deep diveGPPB2020 - Milan - Power BI dataflows deep dive
GPPB2020 - Milan - Power BI dataflows deep dive
 
Introducing Azure SQL Data Warehouse
Introducing Azure SQL Data WarehouseIntroducing Azure SQL Data Warehouse
Introducing Azure SQL Data Warehouse
 
Exploring sql server 2016
Exploring sql server 2016Exploring sql server 2016
Exploring sql server 2016
 
SQL Server 2019 CTP2.4
SQL Server 2019 CTP2.4SQL Server 2019 CTP2.4
SQL Server 2019 CTP2.4
 
Planning a Power BI deployment?
Planning a Power BI deployment?Planning a Power BI deployment?
Planning a Power BI deployment?
 
Should I move my database to the cloud?
Should I move my database to the cloud?Should I move my database to the cloud?
Should I move my database to the cloud?
 
What we learned at pass summit in 2018
What we learned at pass summit in 2018What we learned at pass summit in 2018
What we learned at pass summit in 2018
 

Recently uploaded

OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamOpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
takuyayamamoto1800
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
Tier1 app
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
Globus
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Globus
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
informapgpstrackings
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Globus
 
How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
wottaspaceseo
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
AMB-Review
 
RISE with SAP and Journey to the Intelligent Enterprise
RISE with SAP and Journey to the Intelligent EnterpriseRISE with SAP and Journey to the Intelligent Enterprise
RISE with SAP and Journey to the Intelligent Enterprise
Srikant77
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
Ortus Solutions, Corp
 
Lecture 1 Introduction to games development
Lecture 1 Introduction to games developmentLecture 1 Introduction to games development
Lecture 1 Introduction to games development
abdulrafaychaudhry
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Natan Silnitsky
 
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Globus
 
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Mind IT Systems
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
Globus
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
Globus
 
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxTop Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
rickgrimesss22
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
e20449
 
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Anthony Dahanne
 

Recently uploaded (20)

OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamOpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
 
How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
 
RISE with SAP and Journey to the Intelligent Enterprise
RISE with SAP and Journey to the Intelligent EnterpriseRISE with SAP and Journey to the Intelligent Enterprise
RISE with SAP and Journey to the Intelligent Enterprise
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
 
Lecture 1 Introduction to games development
Lecture 1 Introduction to games developmentLecture 1 Introduction to games development
Lecture 1 Introduction to games development
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
 
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
 
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
 
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxTop Features to Include in Your Winzo Clone App for Business Growth (4).pptx
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptx
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
 
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...
 

PPWT2019 - EmPower your BI architecture

  • 1.
  • 3. Riccardo Perico rperico@solidq.com | @R1k91 | Data Platform & BI Specialist @ 10 years in Microsoft «Data Realm» MCP: MTA, MCSA Speaker at Azure Saturday, SQL Saturday and DIAD tutor
  • 4. Agenda? The Right Questions 1. What’s Storage Mode and which one best suits to me? 2. Do I need an On-premises Data Gateway? 3. How to refresh my data? 4. Is it just on-cloud solution? 5. Can I «encapsulate» PowerBI? 6. Do I need to care about dataflows? 7. Can I have some advisable scenarios?
  • 5. What’s Storage Mode and which one best suits to me?
  • 6. Storage Mode: Import Mode Data Model Cached into RAM
  • 7. Storage Mode: Import Mode • Model Cached inside Power BI Service • High Compression thanks to VertiPaq (*) • Full Power BI functionalities and DAX functions support • Multiple Data Sources mashup (all available) • 10 GB model size for PREMIUM / 1 GB for PRO
  • 8. Large models for Power BI Premium
  • 9. Storage Mode: Direct Query Schema only
  • 10. Storage Mode: Direct Query • Limited data sources • Not full Power Query and DAX support • Pay attention to performance (1 render = 1 query) • Overcome model size limits • Real-time / Near Real-time scenarios
  • 12. Storage Mode: Live Connection PowerBI acts just like an interface
  • 13. Storage Mode: Live Connection • Data Sources: AAS, SSAS Tabular/Multidimensional and PBI dataset • Data Preparation and Data Modeling done in the source • Interaction with report = DAX query sent to source • Overcome model size limits • Partitioned Load • Tends to be faster than DQ • Never prefer «Import» over «Live Connection» for supported sources
  • 14. Storage Mode: Composite Mode Direct Query Import Mode Dual Mode
  • 15. Do I need an On-premises Data Gateway?
  • 16. Let’s draw a typical scenario Publish
  • 17. On-Prem Data Gateway is the way Azure Service BusOn-prem Data Gateway Data Movement Service Refresh Scheduler Service
  • 18. 2 Gateway types On-premises data gateway (Enterprise) On-premises data gateway (Personal mode) All Storage Mode Support Scheduled Refresh Support Import Mode Support Scheduled Refresh Support
  • 19. Gateway: Architecture security • Credentials stored locally and in Power BI Service • Data Source credentials encrypted and stored in the cloud only gateway can decrypt • Gateway polling  No inbound port to open • Outbound port to open TCP: 443, 5671, 5672, 9530...9534
  • 20. Gateway: Considerations • Keep gateways’ name/recovery key in a safe place (migration, take over and recovery) • Do not install Personal and then Enterprise • Sometimes 2 gateways are needed: Import + DQ/Live • Create a cluster for High-Availability: round robin or balanced load • On-premises data gateway app helps you • New Powershell 6 cmdlets to manage Gateway
  • 22. On-prem & On-Cloud Mashup ? Azure Service BusOn-prem Data Gateway 1 Mashup Query On-Prem 1 Mashup Query On-Cloud X1 Mashup Query for both On-Prem and On- Cloud
  • 24. How to refresh my data?
  • 25. Data are old… let’s refresh! Storage Mode Refresh Needed Import Mode Pro Import Mode Premium Direct Query Live Connection Max Refresh Per Day Max Refresh Time 8 2 hours 48 5 hours - - - - +
  • 27. FOR PRO IS cOMIng
  • 28. Is it just on-cloud solution?
  • 29. Cloud is not the only option Publish / Upload
  • 30. PBI Service Vs PBI Report Server PBI Service PBI Report Server Deploy Cloud On-Prem Licensing Power BI Pro and/or Premium PBI Premium or SQL Server EE + SA Release Cycle Every month Every 4 months Gateway Required For on-prem source No Realtime Streaming Yes No Dashboard & Quick Insights Yes No Apps Yes No Q&A Yes No Analyze in Excel Yes No Paginated reports Yes Yes https://docs.microsoft.com/it-it/power-bi/report-server/compare-report-server-service
  • 33. Opt. 1: Embedded in SharePoint Online Pros • Centralized UX • Easy setup Cons • Only reports • Users won’t use service • Permissions management
  • 34. Opt. 2: Power BI Embedded Pros • Full custom application • Reports, Dashboards, Datasets and also Q&A • End user do not need a Power BI Pro License Cons • Licensing: buckets of range renders • Registration and management • Not all service’s features • Web developer needed
  • 35. Do I need to care about dataflows?
  • 36. dataflows: as-is Power Query Sales.pbix Customer Date Sales Power Query Purchases.pbix Vendor Date Purchases
  • 37. dataflows: a new paradigm Sales.pbix Customer Date Sales PQ in dataflows Purchases.pbix Vendor Purchases PQ in dataflows Azure Data Lake Storage V2 (CDM)
  • 38. dataflows: 2 steps forward Customer Date Sales PQ in dataflows Vendor Purchases PQ in dataflows Azure Data Lake Storage V2 (CDM) Power BI Dataset Data Preparation Data Modeling Data Presentation
  • 39. dataflows: pros • Centralization of data preparation (reusability and standardization) • Isolating refresh (1 table a time) • Attach an existing ADLS v2 (preview) • Speed up dataset refresh
  • 41. Storage Mode: Live Connection https://ssbipolar.com/2019/10/14/power-bite-dataflows-vs-datasets/
  • 43. Can I have some advisable scenarios?
  • 44. Some guidelines Relational DWH AAS/SSAS Model Size Advice Yes No Under 1 GB Storage Mode: Import Data License: Pro Data Preparation: Power Query or dataflows No No Under 1 GB Storage Mode: Import Data (also shared dataset) License: Pro Data Preparation: dataflows Yes Yes - Storage Mode: Live Query Connection License: Pro Yes/No No Over 1 GB Storage Mode: Import Data License: Premium Data Preparation: Power Query or dataflows Yes/No No Over 1 GB Storage Mode: Direct Query or Dual Mode with Aggregations License: Pro Data Preparation: Power Query
  • 45. Useful Links • Live or Import? http://tiny.cc/fa9uaz • Gateway http://tiny.cc/lb9uaz • PowerShell Cmdlets for data gateway: http://tiny.cc/ebia1 • Incremental Refresh: http://tiny.cc/4c9uaz • dataflows & CDM: http://tiny.cc/5d9uaz • Dataflows best practices: http://tiny.cc/dmgebi • Shared Datasets replace SSAS: http://tiny.cc/8imtbz
  • 46. Riccardo Perico rperico@solidq.com | @R1k91 | Data Platform & BI Specialist @ 10 years in Microsoft «Data Realm» MCP: MTA, MCSA Speaker at Azure Saturday, SQL Saturday and DIAD tutor
  • 47. Q&A