Azure Purview Microsoft's answer to Data Governance and Data Lineage

1. InSpark
Azure Purview
Microsoft's answer to Data Governance and Data Lineage
Erwin de Kreuk
@erwindekreuk
https://erwindekreuk.com

2. InSpark
We help organizations
accelerating their digital
transformation with impactful
Microsoft solutions & expertise
We Are InSpark

3. InSpark
Unified data governance to
maximize the business value of data
Azure Purview

4. InSpark
History
Private
Previews
Azure Purview
BlueTalon
Acquisition
ADC Gen 2
ADC Gen 1

5. Data is your most strategic asset
GARTNER
Of corporate strategies will
cite information as a critical
enterprise asset by 2022
IDC
Expected global volume
of data generated
annually by 2025
Why Data and Analytics are key to digital
transformation. Christy Pettey. Mar, 2019
IDC Data Age 2025, Dave Reinsel,

6. InSpark
Data governance is becoming increasingly
interdisciplinary
DISCOVERY COMPLIANCE
ChiefDataOfficer

7. InSpark
Data governance is becoming increasingly
interdisciplinary
What data do I have?
Where did the data originate?
Can I trust it?
What’s my exposure to risk?
Is my usage compliant?
How do I control access & use?
What is required by regulation
X?
Data Compliancy
Data Access
Data Security
Data Privacy
Data Management
Data Stewardship
Data Engineering
Data Science
Data Analysist
ChiefDataOfficer

8. InSpark
Overcome
operational silos
Manage growing
data landscape
Elements of successful
data governance
Increase data
agility
Comply with
industry
regulations

9. InSpark
Data Map
Multicloud
On-prem
Data Insights
Azure Purview
Data Catalog
SaaS
Data Map
 Automate and manage metadata at scale
 Elastic data map
 Resource sets
 Scans and ingestion
 Data lineage
Data Catalog
 Enable effortless discovery for data consumers
 Catalog search
 Business glossary
 Asset normalization
Data Insights
 Assess data usage across your organization

10. InSpark
Unified data governance to
maximize the business
value of data
Azure Purview
Reimagine data
governance in the cloud
Set the foundation for
effective data governance
Maximize business value
of data for data
consumers
Gain insight into data use
across the estate

11. InSpark
 Manage and govern operational,
transactional and analytical data
 Cloud-native, purpose-built
service to address discovery and
compliance needs
 Fully managed, serverless, PaaS
service
 Eliminate manual, ad-hoc and
homegrown solutions
Reimagine data
governance in the cloud

12. InSpark
 Automate discovery of data in on-
premises, multicloud and SaaS
sources
 Classify data at scale to specify
sensitivity, compliance, industry,
business and company-specific
value
 Know where data came from and
what was derived from it with
data lineage
Set the foundation for
effective data governance

13. InSpark
 Connect business and technical
data analysts, data scientists, and
data engineers to a trusted data
catalog
 Enable users to quickly find data
and view its lineage and
sensitivity
 Deliver a curated and consistent
glossary of business terms and
definitions
Maximize business value
of data for data
consumers

14. InSpark
 Understand at a glance how data
is being created and used across
your data estate
 Visually assess the state of data
assets, scans, business glossary
and sensitive data
Gain insight into data use
across the estate

15. InSpark
Azure Purview Features
Azure Purview
Azure Purview Platform
Azure Purview Studio
Automated Scanning & Classification
• Dedicated per customer on shared infra
• Provisioned default capacity with option to add-on capacity
Data Map
• Serverless, pay per use
• Includes connectors, scanning of sources, processing into data assets, lineage capture, classification
• Search, browse, asset details
• Automated meta-data and lineage extraction
• Automated classification based on content inspection
• Private Endpoint
• Management center
On-prem & Multi-cloud Operational, Analytical, SaaS
Azure Purview Catalog included with Platform (C0)
Power BI
SQL Server on-prem
Azure Synapse
Azure Data Services
M365 Compliance Cen
Open APIs
(Apache Atlas 2.0)

16. InSpark
Azure Purview Features
Azure Purview
Azure Purview Platform
Azure Purview Studio
Azure Purview Catalog (C1)
Automated Scanning & Classification
• Dedicated per customer on shared infra
• Provisioned default capacity with option to add-on capacity
Data Map
• Serverless, pay per use
• Includes connectors, scanning of sources, processing into data assets, lineage capture, classification
• Search, browse, asset details
• Automated meta-data and lineage extraction
• Automated classification based on content inspection
• Private Endpoint
• Management center
On-prem & Multi-cloud Operational, Analytical, SaaS
• Business Glossary templates
• Lineage visualization & workflows
Azure Purview Catalog included with Platform (C0)
Data Producers &
Consumers
Open APIs
(Apache Atlas 2.0)
Power BI
SQL Server on-prem
Azure Synapse
Azure Data Services
M365 Compliance Cen

17. InSpark
Azure Purview Features
Azure Purview
Azure Purview Platform
Azure Purview Studio
Azure Purview Catalog (C1)
Automated Scanning & Classification
• Dedicated per customer on shared infra
• Provisioned default capacity with option to add-on capacity
Data Map
• Serverless, pay per use
• Includes connectors, scanning of sources, processing into data assets, lineage capture, classification
• Search, browse, asset details
• Automated meta-data and lineage extraction
• Automated classification based on content inspection
• Private Endpoint
• Management center
On-prem & Multi-cloud Operational, Analytical, SaaS
Azure Purview Data Insights (D1)
• Business Glossary templates
• Lineage visualization & workflows
Azure Purview Catalog included with Platform (C0)
• Catalog Insights (Asset, Scan, Glossary)
• Sensitive Information Types & Labeling insights
Data Producers &
Consumers
Data Officers &
Security Officers
Open APIs
(Apache Atlas 2.0)
Power BI
SQL Server on-prem
Azure Synapse
Azure Data Services
M365 Compliance Cen

18. InSpark
Azure Purview – Access Control
Collection Admins • Can edit the collection, its details, and add subcollections.
• Can also add data curators, data readers, and other Purview roles to a
collection scope.
• Collection admins that are automatically inherited from a parent collection
can't be removed.

19. InSpark
Azure Purview – Access Control
Data Source Admins
Collection Admins • can manage data sources and data scans

20. InSpark
Azure Purview – Access Control
Data Source Admins
Data Curator
Collection Admins • can perform create, read, modify, and delete actions on catalog data objects
and establish relationships between objects.

21. InSpark
Azure Purview – Access Control
Data Source Admins
Data Reader
Data Curator
Collection Admins • can access but not modify catalog data objects.

22. InSpark
Azure Purview – Access Control
Data Source Administrator Data Reader Data Curator

23. InSpark
Azure Purview - Pricing
• Capacity Unit
• €0.353 per 1 Capacity Unit Hour
• 1 Capacity Unit supports requests of up to 25 data map operations per second
and includes storage of up to 2 GB of metadata about data assets.
Azure Purview Data Map

24. InSpark
Azure Purview - Pricing
• Power BI Online
• Free in Preview
• SQL Server On Prem
• Free in Preview
• Other Data Sources
• Free in Preview
• €0.540 per 1 vCore Hour
Azure Purview Data Map
Scanning and Classification

25. InSpark
Azure Purview - Pricing
• Power BI Online
• Free in Preview
• SQL Server On Prem
• Free in Preview
• Other Data Sources
• Free in Preview
• €0.540 per 1 vCore Hour
Azure Purview Data Map
Scanning and Classification

26. InSpark
Azure Purview – Pricing-Example
• Resource Set
• €0.18 per 1 vCore Hour
Azure Purview Data Map
Scanning and Classification
Other Features

27. InSpark
Azure Purview – Pricing Example
Azure Purview Data Map
1 CPU x €0.353 X 730 hours
€ 257,69
Scanning and Classification
4 scans x 2 hours x 32 VCore x
€0.540 per vCore per hour
€ 138,24
Other Features
4 scans x 1 hour x €0.18 per
vCore per hour
€ 0,72
€396,65

28. InSpark
Azure Purview Studio Updates Accounts Notifications
Feedback
Metrics
Search Bar
Usefull Links
Recently
Accessed Entities
Search Bar
Key Actvities

29. InSpark
• Quick Actions, recently accessed items, owned Items, search bar and
Documentation
Azure Purview Studio - Activity hubs
• Create collections, register data sources, setup Scans, Integration runtime
• Manage Glossary Items, search, manage terms templates and custom
attributes, import and export Terms using csv
• Insights on your data
• Meta Data Management Security, ADF and data share Connections

30. InSpark

31. InSpark
DEMO

32. InSpark
Innovate
to
accelerate

33. InSpark
Purview Data Map
Unify and make data meaningful
 A graph describing the data assets
and their relationships across your
data estate
 Automated data scanning,
classification and lineage
extraction of hybrid data stores
 200+ built-in data classifiers
 Support for 35 data sources and
growing

34. InSpark
Purview Data Map
Unify and make data meaningful
 A graph describing the data assets
and their relationships across your
data estate
 Automated data scanning,
classification and lineage
extraction of hybrid data stores
 200+ built-in data classifiers
 Support for 35 data sources and
growing

35. InSpark
Azure Purview Data Catalog
Enable effortless discovery
 Semantic search and
browse
 Business glossary and
workflows
 Data lineage with sources,
owners, transformations,
and lifecycle

36. InSpark
Azure Purview Data Catalog
Unify and make data meaningful

37. InSpark
Unify and make data meaningful
Azure Purview Data Catalog
•Azure Data Factory lineage
•Azure Data Share lineage
•Azure Synapse Analytics lineage
•Cassandra lineage
•Erwin lineage
•Looker lineage
•Google BigQuery lineage
•Oracle lineage
•Power BI lineage
•SAP S/4HANA lineage
•SAP ECC lineage
•Spark lineage
•SQL Server Integration Services lineage
•Teradata lineage

38. InSpark
Unify and make data meaningful
Azure Purview Data Catalog

39. InSpark
Insights
 Data Asset distribution
 Sensitive Data
 Data Scanning Coverage
 Business Glossary Utilization
Get a bird’s-eye view of sensitive data

40. InSpark

41. InSpark
Register and Scan a Power BI Tenant
Discover data registered and scanned by Azure Purview

42. InSpark
Register and Scan a Power BI Tenant
Discover data registered and scanned by Azure Purview
 Allow service principals to use read-only Power BI admin APIs
 Enhance admin APIs responses with detailed metadata
 Enhance admin APIs responses with DAX and mashup expressions (Preview)Enabled for a subset of the organization

43. InSpark
Register and Scan a Power BI Tenant
Discover data registered and scanned by Azure Purview
 Allow service principals to use read-only Power BI admin APIs
 Enhance admin APIs responses with etailed metadata
 Enhance admin APIs responses with DAX and mashup expressions (Preview)

44. InSpark

45. InSpark
Integrate Azure Purview in Azure Synapse Analytics
Discover data registered and scanned by Azure Purview

46. InSpark
Integrate Azure Purview in Azure Synapse Analytics
Discover data registered and scanned by Azure Purview

47. InSpark
Integrate Azure Purview in Azure Synapse Analytics
Discover data registered and scanned by Azure Purview

48. InSpark
Integrate Azure Purview in Azure Synapse Analytics
Discover data registered and scanned by Azure Purview

49. InSpark

50. InSpark
DEMO

51. InSpark
Innovate
to
accelerate

52. InSpark
Roadmap /Preview features
 Data Quality
 Data Privacy
 Master Data Management
 Data Policy
 Microsoft Defender for Cloud
 Managed Vnet’s

53. InSpark
Policy in Azure Purview
Data access policy provisioning for Azure Storage - Azure Purview | Microsoft Docs

54. InSpark
Managed Vnets in Azure Purview
Azure Purview Managed Vnet, Vnet Integration Runtime and Managed Private Endpoints - Microsoft Tech Community

55. InSpark

56. InSpark
@erwindekreuk
https://www.linkedin.com/in/erwindekreuk/
https://erwindekreuk.com
Slides will be available on my blog

57. InSpark