This document discusses self-service analytics and introduces the concepts of data mesh and data products. It explains that as data sources and formats grow exponentially, the traditional centralized approach to data management faces challenges in terms of capacity, integration, and governance. Data mesh is presented as a decentralized approach where domain experts own, manage, and publish discoverable, addressable, self-describing data products for self-service access. Key principles of data mesh include distributed ownership of data products, a self-serve data platform, and federated computational governance. The document contrasts how data mesh differs from traditional use case-driven approaches and is not just a technical solution or process, but rather a socio-technical model for managing analytical data at scale
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
Data Con LA 2022 - Self-Service Success and Data Products
1. By Chirag Katbamna
Senior Manager, Accenture
Self-Service Analytics
Success - Data Mesh
and Data Products
August 13, 2022
2. Agenda
• Self-Service Analytics
• The Case for the Change
• What is Data Mesh
• What is Data Product
• Why you need to consider Data Mesh
2
3. Self-Service Analytics
Gartner:
Self-Service Analytics is a form of business intelligence
(BI) in which line-of-business professionals are enabled
and encouraged to perform queries and generate reports
on their own, with nominal IT support.
3
https://www.gartner.com/en/information-technology/glossary/self-service-analytics
4. Changing Landscape
• The way Business uses data is
changing
• Number of data sources have grown
exponentially
• Data format are numerous
• Complexity of Data Integration
pipelines
• Size of Data grows daily
• Changing business rules and
consolidation in EDW
• Centralized Team capacity
challenges
4
6. Data mesh is a decentralized socio-technical approach
to managing data at scale, specially as the number of
producers & consumers proliferate.
It applies purpose-driven product thinking to data where
ownership is distributed to domain experts. These
experts own, manage & publish their data products for
self-service access by consumers and require a
federated governance model.
6
7. Domain Ownership Data-as-a-Product
Self-Serve Data Platform Federated Computational
Governance
• Independent and
autonomous teams own all
data products in the mesh.
• The data is owned by those
who understand it best
• Make Data Sharable
• Easy to Discover
• Trustworthy
• Secure
• Interoperable
• Governed
• Data products are
published
• Available everywhere in
the organization
• Self serve through the Data
Marketplace
• Data is governed where it
is, at the domain level
• Shared Responsibility of
governance enforcement
(standards and regulations)
– Domain + IT/Data Mesh
Data Mesh Principles
DATA MESH
7
8. Data Mesh – What its NOT
It is not a tool and not just a process It is a socio-technical approach to managing
analytical data at scale
It is not just a mechanism to produce
data products
Applying product thinking of data is one of
the pillars of data mesh
It is not just an architectural pattern It is beyond architectures and involves
people & processes as well
It is not a technology focused solution
or even a subset of technologies
Data Mesh requires technology, but there
isn’t just one technical solution
It doesn’t mean everything must be
Decentralized
Mesh needs an equilibrium on the spectrum
of a centralized vs decentralized approach
8
9. Data
Product
DISCOVERABLE
SELF-DESCRIBING
Data Product Characteristics (Governed)
➢ Discoverable [I can find it]
➢ Addressable [I can consume it]
➢ Self-Describing [I understand it]
➢ Trustworthy [I can trust its quality]
➢ Secure [It is protected from unauthorized access]
➢ Interoperable [I can use it with other data products]
Data Product
10. 10
Approach Self Service Differently
Acquire Data Acquire Data Acquire Data
Prepare Data Prepare Data Prepare Data
Analyze Data Analyze Data Analyze Data
UC UC UC
Data Sources
Biz Function Biz Function Biz Function
D
A
T
A
P
L
A
T
F
O
R
M
D
A
T
A
P
L
A
T
F
O
R
M Acquire Data
Data Sources
Build Product
Data Product - A
Build Product
Data Product - B
Biz Function Biz Function Biz Function
UC UC UC
UC UC UC
• Data as Products: Company data is viewed as a “PRODUCT”
• Decentralized Data Ownership: Data Teams (engineers, analysts,
data scientists) provide the data that the company needs for different
purposes
• Self-Service Infrastructure: Business Teams use / re-use data for
decision making, application building, etc.
• Federated & Global Governance
• Use case driven approach – tackle very specific problems using data
• End deliverable is an insight
• Low “re-usability” across business functions
• Stakeholder-driven mindset
FUTURE - DATA as a PRODUCT
NOW - Use Case Driven