SlideShare a Scribd company logo
Powered by
BI and AI in the
Microsoft data
platform
universe
(with a dash of cloud)
Ivan Donev
Agenda
• What’s new in SQL 2019 for BI
• Important updates on Azure for Data platform
• AI and ML for the masses
What is new in 2019 for BI
• SQL Server AS
• MD nothing
• Tabular – Calc groups, M:M relationships, dynamic formatting
• SQL Server IS
• Nothing
• SQL Server RS
• Nothing
• SQL Server MDM and DQS
• Almost nothing
DEMO with AS 2019
Calculation groups
Dynamic formatting
M:M in tabular
Important Azure Milestones
Updates in Azure SQL DB
• Azure SQL DB – Hyperscale• Azure SQL DB – Serverless
Azure SQL DB – Serverless
• Single DB Serverless compute tier
• Billed on compute used per SECOND
• Used only in the vCore model
• Parametrize the min/max vCores
• Scenarios
• Intermittent usage
• Frequently rescaled DBs
• New deployments prior historical usage data
Azure SQL DB – Hyperscale
• Up to 100TB
• Fast backups (filesystem snapshots)
• Up to a minute restores
• Faster throughput
• Fast scale out and scale up
• Distributed architecture
Updates in the DWH and Big Data world
The modern datawarehouse layout
Modern DWH – important updates in Ingest
• Azure Data Factory v2
• Integration runtimes to run SSIS
as-is
• Storing SSIS catalogue in SQL DB
• Mapping workflow
• Wrangling workflow
Data factory v2 – Mapping workflow
Data factory v2 – wrangling dataflow
Modern DWH – important updates in Store
• Azure Data Lake Gen 2
• Hierarchical file system
• Security
• Performance
• Much easier to integrate with
other services
Modern DWH – important updates in Prep and
Train
• Azure Databricks Delta
• Spark engine with RDBMS features
Databricks Delta
• ACID transactions
• Versioned PARQUET files
• Streaming writes to a table (i.e.
Kafka)
• Batch upserts
• High performance reads
• Schema enforcement
Modern DWH – important updates in Model and
Serve
• Changes in Azure DWH
• Concurrency increased to 128
• Adaptive caching (NVMe !!!)
• Unlimited Columnstore storage
capacity
• Workload classification and
importance improvements
• Changes in PowerBI
PowerBI Updates worth noting
• PowerBI Dataflows
• Self-service data transformation
• Shared and certified datasets
(preview)
• Paginated Reports (SSRS)
• Premium
• XMLA Endpoints
• Premium
• Auto ML
• Premium
• CDS integration
The AI in BI
• Options to use in DWH and BI
The AI in BI
• Options to use in DWH and BI
The ML in BI
Not scalableSelf-service AI
•Prototyping
•Do not need additional configuration or tuning
•Options are
•Microsoft Cognitive services with PowerBI (demo)
•AutoML in Dataflows in PowerBI Premium
•R/Python visuals
Scalable, configurable, needs specialized staffEnterprise AI
•Mandatory to run ML and store it in the Store/Serve model
•Options are
•Databricks
•Azure ML
•R/Python in Dataflow as data sources
How to choose?
• The aim?
• Prototype/Test/Verify
• Production/O16N
• The knowledge
• R/Python/Scala/Java/…
• The task
• Image processing/Text analytics/Prediction/Classification
• The post-production support
• Can you support the solution afterwards?
Demo
• PowerBI Dataflows
• Using External Cognitive Services APIs
THANK YOU
All my demos will be described and uploaded on our blog:
http://sqlmasteracademy.com/techblog/

More Related Content

What's hot

Modern ETL: Azure Data Factory, Data Lake, and SQL Database
Modern ETL: Azure Data Factory, Data Lake, and SQL DatabaseModern ETL: Azure Data Factory, Data Lake, and SQL Database
Modern ETL: Azure Data Factory, Data Lake, and SQL DatabaseEric Bragas
 
Customer migration to Azure SQL database, December 2019
Customer migration to Azure SQL database, December 2019Customer migration to Azure SQL database, December 2019
Customer migration to Azure SQL database, December 2019George Walters
 
AliCloud Object Storage Service (OSS) Core Features
AliCloud Object Storage Service (OSS) Core FeaturesAliCloud Object Storage Service (OSS) Core Features
AliCloud Object Storage Service (OSS) Core FeaturesAlibaba Cloud
 
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...HostedbyConfluent
 
Netflix's Big Leap from Oracle to Cassandra
Netflix's Big Leap from Oracle to CassandraNetflix's Big Leap from Oracle to Cassandra
Netflix's Big Leap from Oracle to CassandraRoopa Tangirala
 
Running Scylla on Kubernetes with Scylla Operator
Running Scylla on Kubernetes with Scylla OperatorRunning Scylla on Kubernetes with Scylla Operator
Running Scylla on Kubernetes with Scylla OperatorScyllaDB
 
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Lucas Jellema
 
Big Data Quickstart Series 3: Perform Data Integration
Big Data Quickstart Series 3: Perform Data IntegrationBig Data Quickstart Series 3: Perform Data Integration
Big Data Quickstart Series 3: Perform Data IntegrationAlibaba Cloud
 
Logging infrastructure for Microservices using StreamSets Data Collector
Logging infrastructure for Microservices using StreamSets Data CollectorLogging infrastructure for Microservices using StreamSets Data Collector
Logging infrastructure for Microservices using StreamSets Data CollectorCask Data
 
Beyond Relational
Beyond RelationalBeyond Relational
Beyond RelationalLynn Langit
 
Cloud-based Linked Data Management for Self-service Application Development
Cloud-based Linked Data Management for Self-service Application DevelopmentCloud-based Linked Data Management for Self-service Application Development
Cloud-based Linked Data Management for Self-service Application DevelopmentPeter Haase
 
Organizational compliance and security SQL 2012-2019 by George Walters
Organizational compliance and security SQL 2012-2019 by George WaltersOrganizational compliance and security SQL 2012-2019 by George Walters
Organizational compliance and security SQL 2012-2019 by George WaltersGeorge Walters
 
Microsoft ignite 2018 SQL Server 2019 big data clusters - intro session
Microsoft ignite 2018  SQL Server 2019 big data clusters - intro sessionMicrosoft ignite 2018  SQL Server 2019 big data clusters - intro session
Microsoft ignite 2018 SQL Server 2019 big data clusters - intro sessionTravis Wright
 
Building a data warehouse with AWS Redshift, Matillion and Yellowfin
Building a data warehouse with AWS Redshift, Matillion and YellowfinBuilding a data warehouse with AWS Redshift, Matillion and Yellowfin
Building a data warehouse with AWS Redshift, Matillion and YellowfinLynn Langit
 
Benchmarking Aerospike on the Google Cloud - NoSQL Speed with Ease
Benchmarking Aerospike on the Google Cloud - NoSQL Speed with EaseBenchmarking Aerospike on the Google Cloud - NoSQL Speed with Ease
Benchmarking Aerospike on the Google Cloud - NoSQL Speed with EaseLynn Langit
 
Sql connections germany - migration considerations when migrating your on pre...
Sql connections germany - migration considerations when migrating your on pre...Sql connections germany - migration considerations when migrating your on pre...
Sql connections germany - migration considerations when migrating your on pre...Charley Hanania
 
Personalization Journey: From Single Node to Cloud Streaming
Personalization Journey: From Single Node to Cloud StreamingPersonalization Journey: From Single Node to Cloud Streaming
Personalization Journey: From Single Node to Cloud StreamingDatabricks
 
Automated Metadata Management in Data Lake – A CI/CD Driven Approach
Automated Metadata Management in Data Lake – A CI/CD Driven ApproachAutomated Metadata Management in Data Lake – A CI/CD Driven Approach
Automated Metadata Management in Data Lake – A CI/CD Driven ApproachDatabricks
 
10 Things Learned Releasing Databricks Enterprise Wide
10 Things Learned Releasing Databricks Enterprise Wide10 Things Learned Releasing Databricks Enterprise Wide
10 Things Learned Releasing Databricks Enterprise WideDatabricks
 
Vitalii Bondarenko "Machine Learning on Fast Data"
Vitalii Bondarenko "Machine Learning on Fast Data"Vitalii Bondarenko "Machine Learning on Fast Data"
Vitalii Bondarenko "Machine Learning on Fast Data"DataConf
 

What's hot (20)

Modern ETL: Azure Data Factory, Data Lake, and SQL Database
Modern ETL: Azure Data Factory, Data Lake, and SQL DatabaseModern ETL: Azure Data Factory, Data Lake, and SQL Database
Modern ETL: Azure Data Factory, Data Lake, and SQL Database
 
Customer migration to Azure SQL database, December 2019
Customer migration to Azure SQL database, December 2019Customer migration to Azure SQL database, December 2019
Customer migration to Azure SQL database, December 2019
 
AliCloud Object Storage Service (OSS) Core Features
AliCloud Object Storage Service (OSS) Core FeaturesAliCloud Object Storage Service (OSS) Core Features
AliCloud Object Storage Service (OSS) Core Features
 
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
Data in Motion: Building Stream-Based Architectures with Qlik Replicate & Kaf...
 
Netflix's Big Leap from Oracle to Cassandra
Netflix's Big Leap from Oracle to CassandraNetflix's Big Leap from Oracle to Cassandra
Netflix's Big Leap from Oracle to Cassandra
 
Running Scylla on Kubernetes with Scylla Operator
Running Scylla on Kubernetes with Scylla OperatorRunning Scylla on Kubernetes with Scylla Operator
Running Scylla on Kubernetes with Scylla Operator
 
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
 
Big Data Quickstart Series 3: Perform Data Integration
Big Data Quickstart Series 3: Perform Data IntegrationBig Data Quickstart Series 3: Perform Data Integration
Big Data Quickstart Series 3: Perform Data Integration
 
Logging infrastructure for Microservices using StreamSets Data Collector
Logging infrastructure for Microservices using StreamSets Data CollectorLogging infrastructure for Microservices using StreamSets Data Collector
Logging infrastructure for Microservices using StreamSets Data Collector
 
Beyond Relational
Beyond RelationalBeyond Relational
Beyond Relational
 
Cloud-based Linked Data Management for Self-service Application Development
Cloud-based Linked Data Management for Self-service Application DevelopmentCloud-based Linked Data Management for Self-service Application Development
Cloud-based Linked Data Management for Self-service Application Development
 
Organizational compliance and security SQL 2012-2019 by George Walters
Organizational compliance and security SQL 2012-2019 by George WaltersOrganizational compliance and security SQL 2012-2019 by George Walters
Organizational compliance and security SQL 2012-2019 by George Walters
 
Microsoft ignite 2018 SQL Server 2019 big data clusters - intro session
Microsoft ignite 2018  SQL Server 2019 big data clusters - intro sessionMicrosoft ignite 2018  SQL Server 2019 big data clusters - intro session
Microsoft ignite 2018 SQL Server 2019 big data clusters - intro session
 
Building a data warehouse with AWS Redshift, Matillion and Yellowfin
Building a data warehouse with AWS Redshift, Matillion and YellowfinBuilding a data warehouse with AWS Redshift, Matillion and Yellowfin
Building a data warehouse with AWS Redshift, Matillion and Yellowfin
 
Benchmarking Aerospike on the Google Cloud - NoSQL Speed with Ease
Benchmarking Aerospike on the Google Cloud - NoSQL Speed with EaseBenchmarking Aerospike on the Google Cloud - NoSQL Speed with Ease
Benchmarking Aerospike on the Google Cloud - NoSQL Speed with Ease
 
Sql connections germany - migration considerations when migrating your on pre...
Sql connections germany - migration considerations when migrating your on pre...Sql connections germany - migration considerations when migrating your on pre...
Sql connections germany - migration considerations when migrating your on pre...
 
Personalization Journey: From Single Node to Cloud Streaming
Personalization Journey: From Single Node to Cloud StreamingPersonalization Journey: From Single Node to Cloud Streaming
Personalization Journey: From Single Node to Cloud Streaming
 
Automated Metadata Management in Data Lake – A CI/CD Driven Approach
Automated Metadata Management in Data Lake – A CI/CD Driven ApproachAutomated Metadata Management in Data Lake – A CI/CD Driven Approach
Automated Metadata Management in Data Lake – A CI/CD Driven Approach
 
10 Things Learned Releasing Databricks Enterprise Wide
10 Things Learned Releasing Databricks Enterprise Wide10 Things Learned Releasing Databricks Enterprise Wide
10 Things Learned Releasing Databricks Enterprise Wide
 
Vitalii Bondarenko "Machine Learning on Fast Data"
Vitalii Bondarenko "Machine Learning on Fast Data"Vitalii Bondarenko "Machine Learning on Fast Data"
Vitalii Bondarenko "Machine Learning on Fast Data"
 

Similar to Bi and AI updates in the Microsoft Data Platform stack

Migrating to Amazon RDS with Database Migration Service
Migrating to Amazon RDS with Database Migration ServiceMigrating to Amazon RDS with Database Migration Service
Migrating to Amazon RDS with Database Migration ServiceAmazon Web Services
 
Microsoft Cloud BI Update 2012 for SQL Saturday Philly
Microsoft Cloud BI Update 2012 for SQL Saturday PhillyMicrosoft Cloud BI Update 2012 for SQL Saturday Philly
Microsoft Cloud BI Update 2012 for SQL Saturday PhillyMark Kromer
 
Amazon Redshift with Full 360 Inc.
Amazon Redshift with Full 360 Inc.Amazon Redshift with Full 360 Inc.
Amazon Redshift with Full 360 Inc.Amazon Web Services
 
DAT201 Migrating Databases to AWS - AWS re: Invent 2012
DAT201 Migrating Databases to AWS - AWS re: Invent 2012DAT201 Migrating Databases to AWS - AWS re: Invent 2012
DAT201 Migrating Databases to AWS - AWS re: Invent 2012Amazon Web Services
 
Migrate SQL Server 2008 R2 to Azure Cloud
Migrate SQL Server 2008 R2 to Azure CloudMigrate SQL Server 2008 R2 to Azure Cloud
Migrate SQL Server 2008 R2 to Azure CloudRavi Yadav
 
OSS DB on Azure
OSS DB on AzureOSS DB on Azure
OSS DB on Azurerockplace
 
Directions NA Choosing the best possible Azure platform for NAV
Directions NA Choosing the best possible Azure platform for NAVDirections NA Choosing the best possible Azure platform for NAV
Directions NA Choosing the best possible Azure platform for NAVAleksandar Totovic
 
DBaaS with EDB Postgres on AWS
DBaaS with EDB Postgres on AWSDBaaS with EDB Postgres on AWS
DBaaS with EDB Postgres on AWSEDB
 
Introduction to Azure SQL DB
Introduction to Azure SQL DBIntroduction to Azure SQL DB
Introduction to Azure SQL DBChristopher Foot
 
Should I move my database to the cloud?
Should I move my database to the cloud?Should I move my database to the cloud?
Should I move my database to the cloud?James Serra
 
SQL Server Lift & Shift on Azure - SQL Saturday 921
SQL Server Lift & Shift on Azure - SQL Saturday 921SQL Server Lift & Shift on Azure - SQL Saturday 921
SQL Server Lift & Shift on Azure - SQL Saturday 921Marco Obinu
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure Antonios Chatzipavlis
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure Antonios Chatzipavlis
 
Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...
Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...
Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...Tokyo Azure Meetup
 
Mainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft AzureMainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft AzurePrecisely
 
ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...
ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...
ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...Amazon Web Services
 
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part20812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2Raul Chong
 
Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...
Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...
Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...Amazon Web Services
 

Similar to Bi and AI updates in the Microsoft Data Platform stack (20)

Migrating to Amazon RDS with Database Migration Service
Migrating to Amazon RDS with Database Migration ServiceMigrating to Amazon RDS with Database Migration Service
Migrating to Amazon RDS with Database Migration Service
 
Microsoft Cloud BI Update 2012 for SQL Saturday Philly
Microsoft Cloud BI Update 2012 for SQL Saturday PhillyMicrosoft Cloud BI Update 2012 for SQL Saturday Philly
Microsoft Cloud BI Update 2012 for SQL Saturday Philly
 
Amazon Redshift with Full 360 Inc.
Amazon Redshift with Full 360 Inc.Amazon Redshift with Full 360 Inc.
Amazon Redshift with Full 360 Inc.
 
DAT201 Migrating Databases to AWS - AWS re: Invent 2012
DAT201 Migrating Databases to AWS - AWS re: Invent 2012DAT201 Migrating Databases to AWS - AWS re: Invent 2012
DAT201 Migrating Databases to AWS - AWS re: Invent 2012
 
Migrate SQL Server 2008 R2 to Azure Cloud
Migrate SQL Server 2008 R2 to Azure CloudMigrate SQL Server 2008 R2 to Azure Cloud
Migrate SQL Server 2008 R2 to Azure Cloud
 
OSS DB on Azure
OSS DB on AzureOSS DB on Azure
OSS DB on Azure
 
Exploring sql server 2016
Exploring sql server 2016Exploring sql server 2016
Exploring sql server 2016
 
Directions NA Choosing the best possible Azure platform for NAV
Directions NA Choosing the best possible Azure platform for NAVDirections NA Choosing the best possible Azure platform for NAV
Directions NA Choosing the best possible Azure platform for NAV
 
DBaaS with EDB Postgres on AWS
DBaaS with EDB Postgres on AWSDBaaS with EDB Postgres on AWS
DBaaS with EDB Postgres on AWS
 
Introduction to Azure SQL DB
Introduction to Azure SQL DBIntroduction to Azure SQL DB
Introduction to Azure SQL DB
 
Should I move my database to the cloud?
Should I move my database to the cloud?Should I move my database to the cloud?
Should I move my database to the cloud?
 
SQL Server Lift & Shift on Azure - SQL Saturday 921
SQL Server Lift & Shift on Azure - SQL Saturday 921SQL Server Lift & Shift on Azure - SQL Saturday 921
SQL Server Lift & Shift on Azure - SQL Saturday 921
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
 
Designing a modern data warehouse in azure
Designing a modern data warehouse in azure   Designing a modern data warehouse in azure
Designing a modern data warehouse in azure
 
Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...
Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...
Tokyo Azure Meetup #7 - Introduction to Serverless Architectures with Azure F...
 
Mainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft AzureMainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft Azure
 
ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...
ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...
ENT305 Migrating Your Databases to AWS: Deep Dive on Amazon Relational Databa...
 
IBM - Introduction to Cloudant
IBM - Introduction to CloudantIBM - Introduction to Cloudant
IBM - Introduction to Cloudant
 
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part20812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
0812 2014 01_toronto-smac meetup_i_os_cloudant_worklight_part2
 
Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...
Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...
Migrate from SQL Server or Oracle into Amazon Aurora using AWS Database Migra...
 

More from Ivan Donev

Power bi - enterprise cloud reporting platform Azure Bootcamp 19
Power bi - enterprise cloud reporting platform Azure Bootcamp 19Power bi - enterprise cloud reporting platform Azure Bootcamp 19
Power bi - enterprise cloud reporting platform Azure Bootcamp 19Ivan Donev
 
Tips and tricks to optimiza SQL Server Backup and Restore
Tips and tricks to optimiza SQL Server Backup and RestoreTips and tricks to optimiza SQL Server Backup and Restore
Tips and tricks to optimiza SQL Server Backup and RestoreIvan Donev
 
Get the most out of your Windows Azure VMs
Get the most out of your Windows Azure VMsGet the most out of your Windows Azure VMs
Get the most out of your Windows Azure VMsIvan Donev
 
Develop your database with Visual Studio
Develop your database with Visual StudioDevelop your database with Visual Studio
Develop your database with Visual StudioIvan Donev
 
Windows Azure Bootcamp - Microsoft BI in Azure VMs
Windows Azure Bootcamp - Microsoft BI in Azure VMsWindows Azure Bootcamp - Microsoft BI in Azure VMs
Windows Azure Bootcamp - Microsoft BI in Azure VMsIvan Donev
 
Building your first AS solution
Building your first AS solutionBuilding your first AS solution
Building your first AS solutionIvan Donev
 
Sql server consolidation and virtualization
Sql server consolidation and virtualizationSql server consolidation and virtualization
Sql server consolidation and virtualizationIvan Donev
 
Self-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerViewSelf-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerViewIvan Donev
 
Is "the bigger the beter" valid in the database world
Is "the bigger the beter" valid in the database worldIs "the bigger the beter" valid in the database world
Is "the bigger the beter" valid in the database worldIvan Donev
 

More from Ivan Donev (9)

Power bi - enterprise cloud reporting platform Azure Bootcamp 19
Power bi - enterprise cloud reporting platform Azure Bootcamp 19Power bi - enterprise cloud reporting platform Azure Bootcamp 19
Power bi - enterprise cloud reporting platform Azure Bootcamp 19
 
Tips and tricks to optimiza SQL Server Backup and Restore
Tips and tricks to optimiza SQL Server Backup and RestoreTips and tricks to optimiza SQL Server Backup and Restore
Tips and tricks to optimiza SQL Server Backup and Restore
 
Get the most out of your Windows Azure VMs
Get the most out of your Windows Azure VMsGet the most out of your Windows Azure VMs
Get the most out of your Windows Azure VMs
 
Develop your database with Visual Studio
Develop your database with Visual StudioDevelop your database with Visual Studio
Develop your database with Visual Studio
 
Windows Azure Bootcamp - Microsoft BI in Azure VMs
Windows Azure Bootcamp - Microsoft BI in Azure VMsWindows Azure Bootcamp - Microsoft BI in Azure VMs
Windows Azure Bootcamp - Microsoft BI in Azure VMs
 
Building your first AS solution
Building your first AS solutionBuilding your first AS solution
Building your first AS solution
 
Sql server consolidation and virtualization
Sql server consolidation and virtualizationSql server consolidation and virtualization
Sql server consolidation and virtualization
 
Self-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerViewSelf-service BI with PowerPivot and PowerView
Self-service BI with PowerPivot and PowerView
 
Is "the bigger the beter" valid in the database world
Is "the bigger the beter" valid in the database worldIs "the bigger the beter" valid in the database world
Is "the bigger the beter" valid in the database world
 

Recently uploaded

IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIES VE
 
Transforming The New York Times: Empowering Evolution through UX
Transforming The New York Times: Empowering Evolution through UXTransforming The New York Times: Empowering Evolution through UX
Transforming The New York Times: Empowering Evolution through UXUXDXConf
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomCzechDreamin
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...CzechDreamin
 
Agentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfAgentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfChristopherTHyatt
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Julian Hyde
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsStefano
 
Motion for AI: Creating Empathy in Technology
Motion for AI: Creating Empathy in TechnologyMotion for AI: Creating Empathy in Technology
Motion for AI: Creating Empathy in TechnologyUXDXConf
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoTAnalytics
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastUXDXConf
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxJennifer Lim
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyUXDXConf
 
The architecture of Generative AI for enterprises.pdf
The architecture of Generative AI for enterprises.pdfThe architecture of Generative AI for enterprises.pdf
The architecture of Generative AI for enterprises.pdfalexjohnson7307
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessUXDXConf
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeCzechDreamin
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKUXDXConf
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
 
Introduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG EvaluationIntroduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG EvaluationZilliz
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaRTTS
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...CzechDreamin
 

Recently uploaded (20)

IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Transforming The New York Times: Empowering Evolution through UX
Transforming The New York Times: Empowering Evolution through UXTransforming The New York Times: Empowering Evolution through UX
Transforming The New York Times: Empowering Evolution through UX
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Agentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfAgentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdf
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Motion for AI: Creating Empathy in Technology
Motion for AI: Creating Empathy in TechnologyMotion for AI: Creating Empathy in Technology
Motion for AI: Creating Empathy in Technology
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
The architecture of Generative AI for enterprises.pdf
The architecture of Generative AI for enterprises.pdfThe architecture of Generative AI for enterprises.pdf
The architecture of Generative AI for enterprises.pdf
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Connector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a buttonConnector Corner: Automate dynamic content and events by pushing a button
Connector Corner: Automate dynamic content and events by pushing a button
 
Introduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG EvaluationIntroduction to Open Source RAG and RAG Evaluation
Introduction to Open Source RAG and RAG Evaluation
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 

Bi and AI updates in the Microsoft Data Platform stack

  • 1.
  • 2. Powered by BI and AI in the Microsoft data platform universe (with a dash of cloud) Ivan Donev
  • 3. Agenda • What’s new in SQL 2019 for BI • Important updates on Azure for Data platform • AI and ML for the masses
  • 4. What is new in 2019 for BI • SQL Server AS • MD nothing • Tabular – Calc groups, M:M relationships, dynamic formatting • SQL Server IS • Nothing • SQL Server RS • Nothing • SQL Server MDM and DQS • Almost nothing
  • 5. DEMO with AS 2019 Calculation groups Dynamic formatting M:M in tabular
  • 7. Updates in Azure SQL DB • Azure SQL DB – Hyperscale• Azure SQL DB – Serverless
  • 8. Azure SQL DB – Serverless • Single DB Serverless compute tier • Billed on compute used per SECOND • Used only in the vCore model • Parametrize the min/max vCores • Scenarios • Intermittent usage • Frequently rescaled DBs • New deployments prior historical usage data
  • 9. Azure SQL DB – Hyperscale • Up to 100TB • Fast backups (filesystem snapshots) • Up to a minute restores • Faster throughput • Fast scale out and scale up • Distributed architecture
  • 10. Updates in the DWH and Big Data world
  • 12. Modern DWH – important updates in Ingest • Azure Data Factory v2 • Integration runtimes to run SSIS as-is • Storing SSIS catalogue in SQL DB • Mapping workflow • Wrangling workflow
  • 13. Data factory v2 – Mapping workflow
  • 14. Data factory v2 – wrangling dataflow
  • 15. Modern DWH – important updates in Store • Azure Data Lake Gen 2 • Hierarchical file system • Security • Performance • Much easier to integrate with other services
  • 16. Modern DWH – important updates in Prep and Train • Azure Databricks Delta • Spark engine with RDBMS features
  • 17. Databricks Delta • ACID transactions • Versioned PARQUET files • Streaming writes to a table (i.e. Kafka) • Batch upserts • High performance reads • Schema enforcement
  • 18. Modern DWH – important updates in Model and Serve • Changes in Azure DWH • Concurrency increased to 128 • Adaptive caching (NVMe !!!) • Unlimited Columnstore storage capacity • Workload classification and importance improvements • Changes in PowerBI
  • 19. PowerBI Updates worth noting • PowerBI Dataflows • Self-service data transformation • Shared and certified datasets (preview) • Paginated Reports (SSRS) • Premium • XMLA Endpoints • Premium • Auto ML • Premium • CDS integration
  • 20. The AI in BI • Options to use in DWH and BI
  • 21.
  • 22. The AI in BI • Options to use in DWH and BI
  • 23. The ML in BI Not scalableSelf-service AI •Prototyping •Do not need additional configuration or tuning •Options are •Microsoft Cognitive services with PowerBI (demo) •AutoML in Dataflows in PowerBI Premium •R/Python visuals Scalable, configurable, needs specialized staffEnterprise AI •Mandatory to run ML and store it in the Store/Serve model •Options are •Databricks •Azure ML •R/Python in Dataflow as data sources
  • 24. How to choose? • The aim? • Prototype/Test/Verify • Production/O16N • The knowledge • R/Python/Scala/Java/… • The task • Image processing/Text analytics/Prediction/Classification • The post-production support • Can you support the solution afterwards?
  • 25. Demo • PowerBI Dataflows • Using External Cognitive Services APIs
  • 26. THANK YOU All my demos will be described and uploaded on our blog: http://sqlmasteracademy.com/techblog/

Editor's Notes

  1. Serverless - https://docs.microsoft.com/en-us/azure/sql-database/sql-database-serverless Scenarios well-suited for serverless compute Single databases with intermittent, unpredictable usage patterns interspersed with periods of inactivity and lower average compute utilization over time. Single databases in the provisioned compute tier that are frequently rescaled and customers who prefer to delegate compute rescaling to the service. New single databases without usage history where compute sizing is difficult or not possible to estimate prior to deployment in SQL Database. Hyperscale - https://docs.microsoft.com/en-us/azure/sql-database/sql-database-service-tier-hyperscale Support for up to 100 TB of database size Nearly instantaneous database backups (based on file snapshots stored in Azure Blob storage) regardless of size with no IO impact on Compute Fast database restores (based on file snapshots) in minutes rather than hours or days (not a size of data operation) Higher overall performance due to higher log throughput and faster transaction commit times regardless of data volumes Rapid scale out - you can provision one or more read-only nodes for offloading your read workload and for use as hot-standbys Rapid Scale up - you can, in constant time, scale up your compute resources to accommodate heavy workloads as and when needed, and then scale the compute resources back down when not needed.
  2. What exactly is the Azure DWH landscape at its core. All other services like Logic apps, functions, etc. are auxiliary to the main concept
  3. What exactly is the Azure DWH landscape at its core. All other services like Logic apps, functions, etc. are auxiliary to the main concept
  4. What exactly is the Azure DWH landscape at its core. All other services like Logic apps, functions, etc. are auxiliary to the main concept
  5. What exactly is the Azure DWH landscape at its core. All other services like Logic apps, functions, etc. are auxiliary to the main concept
  6. https://azure.microsoft.com/en-us/blog/adaptive-caching-powers-azure-sql-data-warehouse-performance-gains/ https://docs.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-workload-classification https://docs.microsoft.com/en-us/azure/sql-data-warehouse/sql-data-warehouse-workload-importance