The document outlines the Microsoft Fabric Fabric Engineering Deep Dive event schedule and key topics such as migration pathways, real-time intelligence, and the unified data platform for AI. It emphasizes the integration capabilities of the platform, including onelake, generative AI, and a structured approach to data engineering. Additionally, it discusses migration offerings and best practices for utilizing Microsoft Fabric and Power BI.

1. Microsoft Fabric
Fabric Engineering
Deep Dive
Welcome & Introduction

2. Microsoft Fabric
Fabric Roadshow
Agenda
Time Sessions Duration Speaker
09:00 Welcome & Introduction 15 min Gabi
09:15 Keynote 45 min Gabi & Wolf
10:00 Migration pathway – Datawarehouse 60 min Artur
11:00 Coffee break 15 mins -
11:15 Everything you need to know about DirectLake 60 min Gabi
12:15 What’s new in Fabric Real-Time Intelligence? 60 min Devang
13:15 Lunch break 60 min -
14:15 Roadmap 45 min All
15:00 AMA 60 min All
16:00 Closing

3. Microsoft Fabric
Meet the speakers
Artur Vieira
Fabric CAT Principal PM

4. Microsoft Fabric
Meet the speakers
Devang Shah
Fabric CAT RTI Principal PM

5. Microsoft Fabric
Meet the speakers
Wolf Biber
Practice lead for analytics

6. Microsoft Fabric
Meet the speakers
Gabi Münster
Fabric CAT Senior PM

7. Microsoft Fabric

8. Microsoft Fabric
Fabric Engineering
Deep Dive
Keynote

9. The unified data platform for the era of AI

10. The unified data platform for the era of AI
Data
Engineering
Data
Warehouse
Data
Science
Real-Time
Intelligence
Power BI
Data
Factory

11. Rapid pace
of innovation:

12. Rapid pace
of innovation:

13. The unified data platform for the era of AI
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower Every
Business User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights

14. Scales to the most demanding projects

15. Scales to the most demanding projects

16. Taskflows

17. The unified data platform for the era of AI
Data
Engineering
Data
Warehouse
Data
Science
Real-Time
Intelligence
Power BI
Data
Factory

18. The unified data platform for the era of AI
Data
Engineering
Data
Warehouse
Data
Science
Real-Time
Intelligence
Power BI
Partner
workloads
Data
Factory

19. The unified data platform for the era of AI
Data
Engineering
Data
Warehouse
Data
Science
Real-Time
Intelligence
Power BI
Data Factory
and more…

21. Your data application engine
embedded as a native Fabric
experience customers know and love
Rich component framework to for
building native Fabric experience
Single view to see all data processing
operations
Leverage Fabric ingestion and
connectivity to get data into your
workload
Unmatched integration flexibility

22. The unified data platform for the era of AI
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower Every
Business User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights

23. A single SaaS lake for the whole
organization
Provisioned automatically with the tenant
All workloads automatically store their
data in the OneLake workspace folders
All the data is organized in an intuitive
hierarchical namespace
The data in OneLake is automatically
indexed for discovery, MIP labels, lineage,
PII scans, sharing, governance and
compliance
“The OneDrive for Data”

24. Data Factory
Seamlessly connect to
data stores
Data Factory

25. Mirroring of External Databases
linking of external
databases, with full replicas
created with a couple of clicks
Available for both multi-cloud
and on-premises databases
Real time updates of the replicas
using the CDC feeds of the
database
Data is stored in Delta Parquet
tables, with all Fabric services
instantly available
Amazon Google
Azure

26. OneLake

27. Sharing data in OneLake is as easy as
sharing files in OneDrive, removing the
needs for data duplication
With , data throughout OneLake
can be composed together without any
data movement
Shortcuts also allow instant linking of
data already existing in Azure and in
other clouds, without any data duplication
and movement, making
With support for industry standard APIs,
OneLake data can be directly accessed by
any application or service
Multi-Cloud Shortcuts
Customers
360
Finance
Service
Telemetry
Business
KPIs
Data
Factory
Data
Engineering
Data
Warehouse
Data
Science
Real-Time
Intelligence
Power BI Partner
workloads

28. OneLake

29. OneLake

30. OneLake

31. Iceberg support in OneLake
Bi-directional data access
Seamless access from M365
and Copilot

32. Unify your data in OneLake
Bi-directional data access
Seamless access from M365
and Copilot

33. Open and
Governed Lakehouse

35. Developer friendly
to all data in OneLake
OneLake

36. The API for Accessing Fabric Data
Single endpoint for querying data from
OneLake
GraphQLensures consistent and
predictable results. Clients control the
shape of the response, eliminating
surprises and making it easier to build
stable applications
Request exactly the information needed,
streamlining data access and reducing
unnecessary round trips
Flexibility empowering teams to tailor
data retrieval to their specific
requirements, improving efficiency

37. Reusable Custom Business
Logic in Microsoft Fabric

38. User Data
Functions
Reflex
Triggers T-SQL
Data Flows
Notebooks
Pipeline
Activities
Event-Driven
Actions
Warehouses Lakehouses Mirrored DBs
Invocable from many Fabric items
Simple programming model
Access to Fabric data sources
Developer-friendly experience
Reusable, discoverable via Hub
Security, governance, capacity
consumption
User Data Functions in Microsoft Fabric

39. The unified data platform for the era of AI
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower Every
Business User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights

42. The unified data platform for the era of AI
Data
Engineering
Data
Warehouse
Data
Science
Real-Time
Intelligence
Power BI
Partner
workloads
Data
Factory

43. The unified data platform for the era of AI
Complete
Analytics
Platform
Everything, unified
SaaS-ified
Secured and governed
Lake Centric
and Open
OneLake
One copy
Open at every tier
Empower Every
Business User
Familiar and intuitive
Built into Microsoft 365
Insight to action
AI
Powered
Copilot accelerated
Gen AI on your data
AI-driven insights

44. Gen AI accelerates your data journey in Fabric
AI-driven
insights
Copilot accelerated
experiences
Custom
generative AI
for your data

46. AI Powered

47. on your data
Deliver custom
generative AI experiences
for
Enable custom Q&A on your
data in Fabric
Define custom business semantics
and grounding
Scale the custom experiences to
, , and

48. 4 days of
Microsoft Fabric
Learning, Connection
and Inspiration
24-27 September 2024
Stockholm, Sweden
Learn more:
https://aka.ms/FabCon-Europe

49. CSU Migration Factory
Explained
Lakehouse Migration
Offerings
Power BI Migrations
Real-Time Intelligence
01
02
03
04

50. CSU Migration
Factory Explained

51. A Microsoft CSU delivery model to provide
rehost* migration of Apps, Infra, and Data
workloads at Zero cost to customers.
Modernize your applications and data to
accelerate time to market and deliver new
experiences.
* 7 Options To Modernize Legacy System s (gartner.com )
Migrate or m odernize first? - Cloud Adoption Fram ework | Microsoft Learn
Data & AI Global Solution Architecture
52
What is the CSU
Migration
Factory?

52. Get your first
workloads running
in Azure in weeks
Expert Guidance and Delivery
Zero Cost to Customer
Accelerated Migrations
No Minimum Requirements
53 Data & AI Global Solution Architecture

53. CMF | Workloads & Execution Focus
54 Data & AI Global Solution Architecture
Jumpstart Azure journey for Apps, Infra, and Data workloads through Microsoft-owned delivery at Zero Cost
WS + SQL +
Linux
(including Arc
enabled)
Native
AVD
NoSQL & OSS
Databases
App
Migration
Analytics AI Security
• Rehost/Refactor
migration From On-
prem, AWS, GCP,
Hosters; To: Azure SQL,
Azure VMs
• Upgrade Win OS if
applicable
• Automated scripts for
Arc Enabled
deployment
• Azure VMware Solution:
Factory to migrate
servers, apps, and DBs
into AVS
• Modernize on-
prem RDS
to AVD
• Migrate on-
prem Citrix to
AVD (currently
in incubation)
NoSQL:
• On-prem Cassandra
to Azure MI for
Apache Cassandra
• On-prem MongoDB
to Azure Cosmos DB
for MongoDB (vCore)
OSS Databases:
• MySQL/PostgreSQL
to Azure DB for
MySQL/PostgreSQL
• Single
Server to FlexServer
• .NET, Java Apps
On-Prem to PaaS
• Non .NET
workloads
(Containerized)
workloads On-
Prem to AKS/ACA
• Apps/Self Hosted
K8’s running on
Azure VM’s to
PaaS
• WordPress
migration to
App Service
• Lakehouse
deployment (Data
migration, Build
MVP for initial use
case)
• SQL Server
Reporting Services
to Power BI
• SSAS/Analysis
Services to Power
BI
• P SKU to F SKU
migration
• Real-Time
Analytics
• Deployment of
AOAI use cases:
Conversational
AI/Search,
Virtual Assistant,
Doc Intelligence,
Personalized
content, Image
Analysis
• POC, Landing
Zone for AOAI,
Prod
deployment,
Solution
Optimization
• Defender for Cloud
deployment – cloud
security posture
management
• Deployment of Cloud
workload protection:
Defender for Servers,
Azure SQL, Storage;
• Configuration of
monitoring
components for
automated data
collection
Current localized coverage:
All Time Zones: English
ASIA: Chinese, Japanese
EMEA: Germany, French
LATAM: Spanish, Portuguese
• Customer Sponsorship
secured
• Scope is confirmed and
aligned with CMF scope
• Active MSX Opportunity –
Workload aligned Milestones
(for Managed accounts)
• All customers, any size
Migration (no minimum size)
• Execution Method (Hands-
on-Keyboard or Screen-Share
guidance)
• Nomination form:
https://aka.ms/CMF
Nomination Acceptance Criteria:

54. CSU Migration Factory for Analytics Offerings
SSAS/AAS to PBI Premium
SSRS to PBI Premium
Lakehouse - Fabric
Fabric Real-Time Intelligence
Lakehouse - Databricks
55 Data & AI Global Solution Architecture
P SKU to F SKU

55. Accelerating Adoption through CSU Migration Factory for Analytics
Fabric Databricks
Data & AI Global Solution Architecture
56
Lakehouse
• Offerings:
• Fabric Lakehouse
• Fabric Lakehouse + DW
• Scope:
• Lakehouse medallion architecture with bronze, silver, and
gold layer.
• Transformations with Spark notebooks.
• Orchestration of notebooks with Azure Data Factory or
Fabric Data Factory.
• Silver and/or Gold layer can be built in Fabric DW
• One basic Power BI report to demonstrate how to connect
Power BI reports to Gold layer
• How: Leverage repeatable IP to accelerate establishing Lakehouse
environment, migrate data and rewrite scripts leveraging
repeatable components
• Offerings:
• Lakehouse
• Unity Catalog
• Scope:
• Lakehouse medallion architecture with bronze, silver, and
gold layer.
• Transformations with Spark notebooks.
• Orchestration of notebooks with Azure Data Factory or
Delta Live Tables.
• How: Leverage repeatable IP to accelerate establishing Lakehouse
environment, migrate data and rewrite scripts leveraging repeatable
components

56. Accelerating Adoption through CSU Migration Factory for Analytics
Power BI
• Offerings:
• ADX
• Fabric
• Scope: Migrate data to Fabric using scripts, pipelines, streaming
features or agents. Big Data workloads such as Telemetry, IoT,
Cyber/App Logs, Timeseries, Metrics, Geospatial, Graph,
Embedding Vectors, High-granular, Discrete analytics.
• How: Analyze requirements and help you determine the optimal
alignment. Assess business needs, current platform and existing
architecture.
Real-Time Intelligence
• Offerings:
• SSRS to PBI
• SSAS/AAS to PBI
• P SKU to F SKU
• Scope: SQL Server Reporting Services(SSRS) & Analysis
Services (SSAS/AAS) can be migrated easily to Power BI.
Migration of P SKU to F SKU workspaces in the same region
or another region with considerations.
• How: Leverage 1st party tooling to migrate customers out of
legacy solutions like SSRS & SSAS/AAS into Power BI &
Fabric
Data & AI Global Solution Architecture
57
Additional Fabric Offerings

57. Lakehouse
Migrations

58. Lakehouse
Medallion Architecture
The Medallion Architecture describes a
series of data layers that denotes the quality
of data stored in the Lakehouse. This
architecture guarantees atomicity,
consistency, isolation and durability as data
passes through multiple layers of validation
and transformations being stored in a
layout optimized for efficient analytics.
Key Capabilities:
• Ingest raw data to the Bronze layer
• Validate and deduplicate data in the Silver layer
• Power analytics with the Gold layer
Lakehouses are a single location for data engineers, data scientists, and data analysts to
access and use data.
Data & AI Global Solution Architecture
60

59. Lakehouse – In Scope
Data Sources Orchestration and Transformation
Azure Data Factory
Fabric Pipelines FO
Notebooks
PySpark
SparkSQL
Stored Procedures
Delta Live T
ables DO
Unity Catalog DO
FO Fabric only
Azure SQLDB and SQLMI
PostgreSQL
MySQL
Oracle
SQLServer (on-premises)
Flat Files
Hadoop
AWS Redshift
Dedicated SQLPoolFO*
Google Big Query FO
Fabric Shortcuts FO
DO Databricks only
* Lift and Shift is not currently supported for Dedicated SQLPool. See details in Appendix
Data & AI Global Solution Architecture
61

60. Lakehouse Project Timeline
1
Requirements:Local CSA, Corp
Factory T
eam Lead and Customer will
meet to discuss requirements of the
program
2
Design:Local CSA, Corp Factory
T
eam Lead and Customer will work
together to design a high value use
case
3
Implementation:Development team
will be doing the work
Key Phases /
Milestones
Start
Date
End
Date
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
Requirements Week 1 Week 2
Design Week 2 Week 3
Implementation Week 3 Week 8
Testing Week 5 Week 8
Handover Week 7 Week 8
Requirements
Design
Implementation
Testing
Handover
T
esting:Development team and
Customer should be doing iterative
testing
Handover:Development team will do
knowledge transfer sessions
4
5
Data & AI Global Solution Architecture
66

61. Power BI
Migrations

62. Migrate .rdl reports and SSRS PBI interactive reports
from SSRS servers to PBI Premium
Migrate SSAS/AAS models to Power BI Semantic
Models
Publish reports that pass checks as PBI Paginated
Reports
Automation and Business Operations, such as .Net
Code, SSIS Packages and Azure Data Factory
Governance and Security such as PBI capacity
governance, workspace config and role membership
Setup and training on Optimization methodology,
tenant management, release management,
monitoring, alerting, post deployment oversight
Out of Scope
In Scope
Migrate P SKU (PBI Premium) to F SKU (Fabric)
Power BI
Scope
Data & AI Global Solution Architecture
68

63. Data & AI Global Solution Architecture
72
P SKU to F SKU Migration Scenarios
Migration Scenario Supported
Migrating workspaces having only Power BI items
- within the same region
Y
es
Migrating workspaces having only Power BI items
- T
o a different region
Y
es
Migrating workspaces having Fabric items
- within the same region
Y
es
Migrating workspaces having Fabric items
- T
o a different region
No - you must delete all the Fabric items
from the workspace first.
Cross T
enant Migration No

64. Data & AI Global Solution Architecture
73
Power BI Project Timeline
1
Requirements and Design: Local CSA,
Corp Factory T
eam Lead and Customer
will meet to discuss requirements and
design of the program
2 Deployment and Implementation:
Development team will be doing the work
3
T
esting and Handover: Development team
and Customer should be doing iterative
testing and completing the handover
Key Phases /
Milestones
Start
Date
End
Date
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6
Requirements &
Design
Week 1 Week 2
Deployment &
Implementation
Week 2 Week 5
Testing &
Handover
Week 5 Week 6
Requirements & Design
Deployment & Implementation
Testing & Handover

65. Real-Time
Intelligence

66. Real-Time Intelligence
Eventhouse
RT Dashboard
KQL Queryset
Power BI
Analyze &
Transform
Eventstream
Ingest & Process
Reflex
Act
Real-Time Hub
OneLake
Digital Operations, Observational, (I)IoT+
high-granular, discrete analytics
Streaming, minimal-latency, data in-motion, predictive analytics
Real-Time Intelligence

67. Real-Time Intelligence – In Scope
IoT
SignalR websockets
REST-APIs
Kafka, Flink, Redpanda, Druid
Splunk can forward to Fabric for analytics
Elasticsearch
Sentinel using continuous export or setup parallel-ingestion
Azure Database Watcher
InfluxDB by leveraging telegraph kusto connector
Aveva OSI-PI
AWS Kinesis, AWS Timestream, Confluent, Google Pubsub, Spark streaming
Azure Time Series Insights (retires July 7, 2024)
Azure AI Metrics Advisor - Anomaly Detection (retires October 1, 2026)
Snowflake, Google BigQuery, IBM DB2 when data is timeseries, logs or telemetry
KSQL, Singlestore, Clickhouse, Datadog, Newrelic, Dynatrace & Pinot
GraphDBs such as Neo4j & Tigergraph
VectorstoreDBs such as Weaviate, Qdrant, Chroma, Milvus
CDC scenarios
Azure PostgreSQL
Cosmos DB
Azure MySQL
Azure SQLDatabase
Oracle Goldengate via EH connector
Data Sources Interface Patterns
Data & AI Global Solution Architecture
76