Make your data AI ready
with Microsoft Fabric and
Azure Databricks (and, now
Snowflake!)
George Walters
george.walters@microsoft.com
https://www.linkedin.com/in/georgewalters/
AI is changing the world
CIOs are scrambling to bring
AI to their data estate
But they face immense complexity
Needing to manually integrate across the stack
Every project has many
subsystems
Every subsystem need a different
class of product
Products often comes from
multiple vendors
Integration is complex, fragile
and expensive
With siloes created everywhere
Applications are built by
different teams
With diverse technology stacks
and architectures, each designed
as a standalone system
Making cross-app integration, and
leveraging analytics in operations
very hard
This chaotic complexity
imperils the transition to the
age of AI
“
Unify,
I am the Chief Information Officer and don’t
want to be the Chief Integration Officer.
Help me unleash AI in my data estate.”
Every CIO, Every Enterprise
Microsoft Fabric
The data platform for the era of AI
From
One component
Single database
Isolated application
Gen AI wired in
To
Unified stack
All the data
Entire estate
Gen AI built in
Microsoft Fabric
Data analytics for the era of AI
Data
Factory
Synapse Data
Engineering
Synapse Data
Science
Synapse Data
Warehouse
Synapse Real
Time Analytics
Power BI
Data
Activator
OneLake
Purview
Public Preview May 2023
Microsoft Fabric
Data analytics for the era of AI
25k+
active organizations
(TPIDs)
Customers are using
Fabric end to end
3+
Workloads
84%
of Customers
4+
Workloads
44%
of Customers
# of Active Workloads 1 2 3 4 6 7 8
84%
3K (12.7%)
3K (11.7%)
3K (12%)
1K (5.5%) 0K (2.1%)
0K (2.1%)
3.6K (14%)
10K (39.8%)
Microsoft Fabric
Generally Available, Nov 15, 2023
aka.ms/IgniteFabric2023
Weekly Fabric Releases
Rapid pace of innovation
Aug 2023 Sept 2023 Oct 2023
Weekly Fabric Releases
Rapid pace of innovation
Jul 2023 Aug 2023 Sep 2023
Explore the Fabric Roadmap
Rapid pace of innovation
Jul 2023 Aug 2023 Sep 2023
aka.ms/FabricRoadmap
Microsoft Fabric
Data analytics for the era of AI
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower
Every Business
User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights
Microsoft Fabric
Data analytics for the era of AI
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower
Every Business
User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Microsoft Fabric
Data
Factory
Synapse Data
Engineering
Synapse Data
Science
Synapse Data
Warehouse
Synapse Real
Time Analytics
Power BI
Data
Activator
AI Assisted
Shared Workspaces
Universal Compute Capacities
One Security
OneLake
Intelligent data foundation
Single…
• Onboarding and trials
• Sign-on
• Navigation model
• UX model
• Workspace organization
• Collaboration experience
• Data Lake
• Storage format
• Data copy for all engines
• Security model
• CI/CD
• Monitoring hub
• Data Hub
• Governance & compliance
Persona
optimized
experiences
Demo: Every data workload in one SaaS
experience
https://www.youtube.com/watch?v=fz5fBgww0rE&t=77s
Microsoft Fabric
Data analytics for the era of AI
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower
Every Business
User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights
Lake Centric
and Open
OneLake
One copy
Open at every tier
OneLake for all Data
“The OneDrive for Data”
Data
Factory
Synapse Data
Engineering
Synapse Data
Science
Synapse Data
Warehouse
Synapse Real
Time Analytics
Power BI
Data
Activator
OneLake
Intelligent data foundation
A single SaaS lake for the whole
organization
Provisioned automatically with the tenant
All workloads automatically store their
data in the OneLake workspace folders
All the data is organized in an intuitive
hierarchical namespace
The data in OneLake is automatically
indexed for discovery, MIP labels, lineage,
PII scans, sharing, governance and
compliance
One Copy for all computes
Real separation of compute and storage
Data
Factory
Synapse Data
Engineering
Synapse Data
Science
Synapse Data
Warehouse
Synapse Real
Time Analytics
Power BI
Data
Activator
Spark T-SQL
Serverless
Compute
KQL
Analysis
Services
Customers
360
Delta – Parquet
Format
Finance
Delta – Parquet
Format
Service
Telemetry
Delta – Parquet
Format
Business
KPIs
Delta – Parquet
Format
All the compute engines store their data
automatically in OneLake
The data is stored in a single common format
Delta – Parquet, an open standards format,
is the storage format for all tabular data in
Analytics vNext
Once data is stored in the lake, it is directly
accessible by all the engines without needing
any import/export
All the compute engines have been fully
optimized to work with Delta Parquet as their
native format
Shared universal security model is enforced
across all the engines
Demo: Every workload works with
OneCopy and open formats
https://youtu.be/fz5fBgww0rE?si=2Nxk9imOmks2ULhW&t=135
Better together through OneLake
Open and Governed Lakehouse
Microsoft Fabric Azure Databricks
All Fabric engines have been redesigned and
optimized for delta/parquet as their storage
format for tabular data
Azure Databricks can directly read/write data
from any Fabric data item stored in OneLake
through external link.
Existing non-Unity Catalog managed data in
ADLS gen2 storage can be a shortcut into
OneLake.
Any Fabric engine including Power BI with direct
lake in OneLake or with shortcut.
Coming soon: Unity Catalog and Fabric
integration, Unity Catalog data will be available
through Azure Databricks data item enhanced
shortcuts experience.
Unifying data in OneLake
Shortcuts
Data
Factory
Synapse Data
Engineering
Synapse Data
Science
Synapse Data
Warehouse
Synapse Real
Time Analytics
Power BI
Data
Activator
Customers
360
Finance
OneLake
Service
Telemetry
Business
KPIs
Azure Storage Databricks Amazon S3 Google Dataverse
Sharing data in OneLake is as easy as
sharing files in OneDrive, removing the
needs for data duplication
With shortcuts, data throughout OneLake
can be composed together without any
data movement
Shortcuts also allow instant linking of
data already existing in Azure and in
other clouds, without any data duplication
and movement, making OneLake the first
multi-cloud data lake
With support for industry standard APIs,
OneLake data can be directly accessed by
any application or service
Snowflake
Unifying data in OneLake
Frictionless Mirroring of External Databases
OneLake
Data
Factory
Synapse Data
Engineering
Synapse Data
Science
Synapse Data
Warehouse
Synapse Real
Time Analytics
Power BI
Data
Activator
Amazon Google
Azure
Announcing
Mirroring
aka.ms/IgniteFabric2023
Mirroring in Fabric
Near real-time replication of proprietary
database stores with zero-ETL
XMLA
(Data Plane)
Delta
(Data Plane)
Fabric
Spark
Power
BI
TDS
(Data Plane)
Fabric DW
CDC
Consumption Replica
Delta Parquet in OneLake
High throughput, low latency
Mirrored
Database
Replicator
Cosmos DB & SQL DB
Proprietary Store
Demo: Fabric Mirroring, and Join across data
https://www.youtube.com/watch?v=fz5fBgww0rE&t=198s
https://www.youtube.com/watch?v=fz5fBgww0rE&t=329s
Microsoft Fabric
Data analytics for the era of AI
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower
Every Business
User
Familiar and intuitive
Built into Microsoft 365
Insight to action
Empower
Every Business
User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights
Microsoft Fabric
Data analytics for the era of AI
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower
Every Business
User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights
Announcing
Copilot in Microsoft Fabric
aka.ms/IgniteFabric2023
Copilot in Microsoft Fabric
Generative AI for every step of your data journey
Unlock
Your Data
Intelligent data prep
and integration for every
data source
Enhance
Productivity
Harness the power of AI
to turn your ideas
into solutions
Accelerate
Insight Discovery
Analyze and explore
your data faster
Copilot in Microsoft Fabric
Generative AI for every step of your data journey
Unlock
Your Data
Intelligent data prep
and integration for every
data source
Enhance
Productivity
Harness the power of AI
to turn your ideas
into solutions
Accelerate
Insight Discovery
Analyze and explore
your data faster
Demo: Microsoft Fabric Copilot, and
Power BI Copilot
https://youtu.be/fz5fBgww0rE?si=2TNF4ofVDnBbdq9J&t=437
https://youtu.be/fz5fBgww0rE?si=QYEA21mpyxkZaFtK&t=605
Customer Momentum
In developing and launching EY Intelligence, Microsoft
Fabric has been a game changer. Our unique analytics-as-a-
service offering gives the C-suite at our client
organizations cross-functional transparency and on-demand
insights to make better and quicker decisions.
Swen Gehring
Director, Strategy and Transactions, Ernst & Young​
Microsoft Fabric would have been our choice from the
beginning – had it only existed then. Building a
democratized data platform was too big of a distraction
from our focus on supporting actuaries in managing risk.
Tom Peplow
Principal and Senior Director of Technology Strategy for Life Technology Solutions, Milliman
Customer Video – 3 Mins
(Milliman)
Fabric featured partners
The AI era has arrived, and our clients need a strong data foundation to ensure
they can successfully achieve their AI aspirations securely and responsibly. At
Accenture and Avanade, we are ready to securely unify data, analytics and AI
for our clients with Microsoft Fabric. We’ve launched the Microsoft Fabric
University that already has trained 3,000 data professionals to be fluent in
Fabric, and we’re taking lessons learned from building our own operations on
Fabric to accelerate AI opportunities with our clients across the globe.
Simon Thomas
Data & AI Global Lead, Avanade ​
Through our strategic partnership and the transformative capabilities of
Microsoft Fabric, we are accelerating data value creation for our customers.
Capgemini's investment in scaling Fabric competency, paving the way for
excellence through our Fabric-specific assessment and certification via
Capgemini's OCEANS Tool, provides enterprises with the opportunity to
unleash unified data and drive their organizations forward.
Niraj Parihar
Insights & Data CEO​
Microsoft Fabric – An Open Platform
+100s of partners
Next steps
Attend Fabric sessions
aka.ms/IgniteData2023
Try Microsoft Fabric
aka.ms/try-Fabric
Register for the Fabric
Community Conference
aka.ms/FabricConference
Azure. Limitless innovation.
Session Resources
Next steps
Resources
Related sessions
aka.ms/Ignite2023/BRK221H
© Copyright Microsoft Corporation. All rights reserved.
George Walters
george.walters@microsoft.com
https://www.linkedin.com/in/georgewalters/
Thank you!

Make your data AI ready with Microsoft Fabric and Azure Databricks pitch deck 2024 August.pdf

  • 1.
    Make your dataAI ready with Microsoft Fabric and Azure Databricks (and, now Snowflake!) George Walters george.walters@microsoft.com https://www.linkedin.com/in/georgewalters/
  • 2.
    AI is changingthe world
  • 3.
    CIOs are scramblingto bring AI to their data estate
  • 4.
    But they faceimmense complexity
  • 5.
    Needing to manuallyintegrate across the stack Every project has many subsystems Every subsystem need a different class of product Products often comes from multiple vendors Integration is complex, fragile and expensive
  • 6.
    With siloes createdeverywhere Applications are built by different teams With diverse technology stacks and architectures, each designed as a standalone system Making cross-app integration, and leveraging analytics in operations very hard This chaotic complexity imperils the transition to the age of AI
  • 7.
    “ Unify, I am theChief Information Officer and don’t want to be the Chief Integration Officer. Help me unleash AI in my data estate.” Every CIO, Every Enterprise
  • 8.
    Microsoft Fabric The dataplatform for the era of AI From One component Single database Isolated application Gen AI wired in To Unified stack All the data Entire estate Gen AI built in
  • 9.
    Microsoft Fabric Data analyticsfor the era of AI Data Factory Synapse Data Engineering Synapse Data Science Synapse Data Warehouse Synapse Real Time Analytics Power BI Data Activator OneLake Purview
  • 10.
    Public Preview May2023 Microsoft Fabric Data analytics for the era of AI
  • 11.
  • 12.
    Customers are using Fabricend to end 3+ Workloads 84% of Customers 4+ Workloads 44% of Customers # of Active Workloads 1 2 3 4 6 7 8 84% 3K (12.7%) 3K (11.7%) 3K (12%) 1K (5.5%) 0K (2.1%) 0K (2.1%) 3.6K (14%) 10K (39.8%)
  • 13.
    Microsoft Fabric Generally Available,Nov 15, 2023 aka.ms/IgniteFabric2023
  • 14.
    Weekly Fabric Releases Rapidpace of innovation Aug 2023 Sept 2023 Oct 2023
  • 15.
    Weekly Fabric Releases Rapidpace of innovation Jul 2023 Aug 2023 Sep 2023
  • 16.
    Explore the FabricRoadmap Rapid pace of innovation Jul 2023 Aug 2023 Sep 2023 aka.ms/FabricRoadmap
  • 17.
    Microsoft Fabric Data analyticsfor the era of AI Complete Analytics Platform Everything, unified SaaS-ified Secured and governed Lake Centric and Open OneLake One copy Open at every tier Empower Every Business User Familiar and intuitive Built into Microsoft 365 Insight to action AI Powered Copilot accelerated Gen AI on your data AI-driven insights
  • 18.
    Microsoft Fabric Data analyticsfor the era of AI Lake Centric and Open OneLake One copy Open at every tier Empower Every Business User Familiar and intuitive Built into Microsoft 365 Insight to action AI Powered Copilot accelerated Gen AI on your data AI-driven insights Complete Analytics Platform Everything, unified SaaS-ified Secured and governed Complete Analytics Platform Everything, unified SaaS-ified Secured and governed
  • 19.
    Microsoft Fabric Data Factory Synapse Data Engineering SynapseData Science Synapse Data Warehouse Synapse Real Time Analytics Power BI Data Activator AI Assisted Shared Workspaces Universal Compute Capacities One Security OneLake Intelligent data foundation Single… • Onboarding and trials • Sign-on • Navigation model • UX model • Workspace organization • Collaboration experience • Data Lake • Storage format • Data copy for all engines • Security model • CI/CD • Monitoring hub • Data Hub • Governance & compliance
  • 20.
  • 21.
    Demo: Every dataworkload in one SaaS experience https://www.youtube.com/watch?v=fz5fBgww0rE&t=77s
  • 23.
    Microsoft Fabric Data analyticsfor the era of AI Complete Analytics Platform Everything, unified SaaS-ified Secured and governed Lake Centric and Open OneLake One copy Open at every tier Empower Every Business User Familiar and intuitive Built into Microsoft 365 Insight to action AI Powered Copilot accelerated Gen AI on your data AI-driven insights Lake Centric and Open OneLake One copy Open at every tier
  • 24.
    OneLake for allData “The OneDrive for Data” Data Factory Synapse Data Engineering Synapse Data Science Synapse Data Warehouse Synapse Real Time Analytics Power BI Data Activator OneLake Intelligent data foundation A single SaaS lake for the whole organization Provisioned automatically with the tenant All workloads automatically store their data in the OneLake workspace folders All the data is organized in an intuitive hierarchical namespace The data in OneLake is automatically indexed for discovery, MIP labels, lineage, PII scans, sharing, governance and compliance
  • 25.
    One Copy forall computes Real separation of compute and storage Data Factory Synapse Data Engineering Synapse Data Science Synapse Data Warehouse Synapse Real Time Analytics Power BI Data Activator Spark T-SQL Serverless Compute KQL Analysis Services Customers 360 Delta – Parquet Format Finance Delta – Parquet Format Service Telemetry Delta – Parquet Format Business KPIs Delta – Parquet Format All the compute engines store their data automatically in OneLake The data is stored in a single common format Delta – Parquet, an open standards format, is the storage format for all tabular data in Analytics vNext Once data is stored in the lake, it is directly accessible by all the engines without needing any import/export All the compute engines have been fully optimized to work with Delta Parquet as their native format Shared universal security model is enforced across all the engines
  • 26.
    Demo: Every workloadworks with OneCopy and open formats https://youtu.be/fz5fBgww0rE?si=2Nxk9imOmks2ULhW&t=135
  • 28.
    Better together throughOneLake Open and Governed Lakehouse Microsoft Fabric Azure Databricks All Fabric engines have been redesigned and optimized for delta/parquet as their storage format for tabular data Azure Databricks can directly read/write data from any Fabric data item stored in OneLake through external link. Existing non-Unity Catalog managed data in ADLS gen2 storage can be a shortcut into OneLake. Any Fabric engine including Power BI with direct lake in OneLake or with shortcut. Coming soon: Unity Catalog and Fabric integration, Unity Catalog data will be available through Azure Databricks data item enhanced shortcuts experience.
  • 30.
    Unifying data inOneLake Shortcuts Data Factory Synapse Data Engineering Synapse Data Science Synapse Data Warehouse Synapse Real Time Analytics Power BI Data Activator Customers 360 Finance OneLake Service Telemetry Business KPIs Azure Storage Databricks Amazon S3 Google Dataverse Sharing data in OneLake is as easy as sharing files in OneDrive, removing the needs for data duplication With shortcuts, data throughout OneLake can be composed together without any data movement Shortcuts also allow instant linking of data already existing in Azure and in other clouds, without any data duplication and movement, making OneLake the first multi-cloud data lake With support for industry standard APIs, OneLake data can be directly accessed by any application or service Snowflake
  • 31.
    Unifying data inOneLake Frictionless Mirroring of External Databases OneLake Data Factory Synapse Data Engineering Synapse Data Science Synapse Data Warehouse Synapse Real Time Analytics Power BI Data Activator Amazon Google Azure
  • 32.
  • 33.
    Mirroring in Fabric Nearreal-time replication of proprietary database stores with zero-ETL XMLA (Data Plane) Delta (Data Plane) Fabric Spark Power BI TDS (Data Plane) Fabric DW CDC Consumption Replica Delta Parquet in OneLake High throughput, low latency Mirrored Database Replicator Cosmos DB & SQL DB Proprietary Store
  • 34.
    Demo: Fabric Mirroring,and Join across data https://www.youtube.com/watch?v=fz5fBgww0rE&t=198s https://www.youtube.com/watch?v=fz5fBgww0rE&t=329s
  • 36.
    Microsoft Fabric Data analyticsfor the era of AI Complete Analytics Platform Everything, unified SaaS-ified Secured and governed Lake Centric and Open OneLake One copy Open at every tier Empower Every Business User Familiar and intuitive Built into Microsoft 365 Insight to action Empower Every Business User Familiar and intuitive Built into Microsoft 365 Insight to action AI Powered Copilot accelerated Gen AI on your data AI-driven insights
  • 38.
    Microsoft Fabric Data analyticsfor the era of AI Complete Analytics Platform Everything, unified SaaS-ified Secured and governed Lake Centric and Open OneLake One copy Open at every tier Empower Every Business User Familiar and intuitive Built into Microsoft 365 Insight to action AI Powered Copilot accelerated Gen AI on your data AI-driven insights AI Powered Copilot accelerated Gen AI on your data AI-driven insights
  • 39.
    Announcing Copilot in MicrosoftFabric aka.ms/IgniteFabric2023
  • 40.
    Copilot in MicrosoftFabric Generative AI for every step of your data journey Unlock Your Data Intelligent data prep and integration for every data source Enhance Productivity Harness the power of AI to turn your ideas into solutions Accelerate Insight Discovery Analyze and explore your data faster
  • 41.
    Copilot in MicrosoftFabric Generative AI for every step of your data journey Unlock Your Data Intelligent data prep and integration for every data source Enhance Productivity Harness the power of AI to turn your ideas into solutions Accelerate Insight Discovery Analyze and explore your data faster
  • 42.
    Demo: Microsoft FabricCopilot, and Power BI Copilot https://youtu.be/fz5fBgww0rE?si=2TNF4ofVDnBbdq9J&t=437 https://youtu.be/fz5fBgww0rE?si=QYEA21mpyxkZaFtK&t=605
  • 46.
  • 47.
    In developing andlaunching EY Intelligence, Microsoft Fabric has been a game changer. Our unique analytics-as-a- service offering gives the C-suite at our client organizations cross-functional transparency and on-demand insights to make better and quicker decisions. Swen Gehring Director, Strategy and Transactions, Ernst & Young​
  • 48.
    Microsoft Fabric wouldhave been our choice from the beginning – had it only existed then. Building a democratized data platform was too big of a distraction from our focus on supporting actuaries in managing risk. Tom Peplow Principal and Senior Director of Technology Strategy for Life Technology Solutions, Milliman
  • 49.
    Customer Video –3 Mins (Milliman)
  • 50.
  • 51.
    The AI erahas arrived, and our clients need a strong data foundation to ensure they can successfully achieve their AI aspirations securely and responsibly. At Accenture and Avanade, we are ready to securely unify data, analytics and AI for our clients with Microsoft Fabric. We’ve launched the Microsoft Fabric University that already has trained 3,000 data professionals to be fluent in Fabric, and we’re taking lessons learned from building our own operations on Fabric to accelerate AI opportunities with our clients across the globe. Simon Thomas Data & AI Global Lead, Avanade ​
  • 52.
    Through our strategicpartnership and the transformative capabilities of Microsoft Fabric, we are accelerating data value creation for our customers. Capgemini's investment in scaling Fabric competency, paving the way for excellence through our Fabric-specific assessment and certification via Capgemini's OCEANS Tool, provides enterprises with the opportunity to unleash unified data and drive their organizations forward. Niraj Parihar Insights & Data CEO​
  • 53.
    Microsoft Fabric –An Open Platform +100s of partners
  • 54.
    Next steps Attend Fabricsessions aka.ms/IgniteData2023 Try Microsoft Fabric aka.ms/try-Fabric Register for the Fabric Community Conference aka.ms/FabricConference
  • 55.
  • 56.
    Session Resources Next steps Resources Relatedsessions aka.ms/Ignite2023/BRK221H
  • 57.
    © Copyright MicrosoftCorporation. All rights reserved. George Walters george.walters@microsoft.com https://www.linkedin.com/in/georgewalters/ Thank you!