SlideShare a Scribd company logo
Tristan Baker - linkedin.com/in/tristanbaker
Suresh Raman - linkedin.com/in/ramansuresh
Allison Bellah (in absentia) - linkedin.com/in/allisonbellah
Data Mesh at Intuit
May 13, 2021
©2021 Intuit Inc. All rights reserved. 2
Arriving at data mesh
Our vision and four part strategy
Now with 25% more parts!
Q&A
Fire away!
Agenda
Arriving at data mesh
©2021 Intuit Inc. All rights reserved. 4
A brief history of data infrastructure
©2021 Intuit Inc. All rights reserved. 5
A brief history of data infrastructure
©2021 Intuit Inc. All rights reserved. 6
A brief history of data infrastructure
©2021 Intuit Inc. All rights reserved. 7
Today
©2021 Intuit Inc. All rights reserved. 8
What “we cannot scale” sounds like from our users
Discovering Data
● Where can I find data about a particular thing (customer,
company, etc)?
● Where can I find the data sourced from a particular product
or service?
Understanding Data
● Who can approve my access so that I can see samples of the
data?
● What is the schema of the data?
● What is the business meaning and context of the data?
● Is this data related to other concepts? Is it joinable to other
data? What is the meaning of the relationship?
Trusting Data
● What system produces this data and at what latency?
● What other systems use this data?
● What is the quality of this data? Is it ‘clean’?
● Which team supports this data if it breaks?
Publishing Data
● How do I describe my data so that others understand what it
means and how to use it?
● Where do I host my data so that other systems can access it?
● Data systems are complicated, how can I build and operate
my process on top of one?
● What are my operational responsibilities once my
process/data is in production?
● How do I meet my compliance requirements for
processing/storing/publishing data?
● Am I duplicating processing/data that already exists?
Consuming Data
● How is this table/topic partitioned?
● Who can approve my production system to access it?
● Will I get alerted if the schema changes?
©2021 Intuit Inc. All rights reserved. 9
The future of data infrastructure
● Data treated as code
● Data service as a facet of a product
● Data responsibility decentralized
● Producers take responsibility for data
● Producers serve consumers
● Data platform provides the ecosystem to
govern and manage the lifecycle of data and
machine learning
The provocation
Data Mesh is born
Our vision and four part
strategy
©2021 Intuit Inc. All rights reserved. 11
Enable more Intuit teams
to more easily use and
create data
©2021 Intuit Inc. All rights reserved. 12
Four part strategy
• Stewardship
– ensures accountability for a set of defined responsibilities in building and managing their solutions; including
adherence to a set of defined best practices to produce only high quality data.
• Organizing people, code and data
– A systematic approach to organizing the people, code and data which clearly identifies the owners of a business problem and its
solution.
• Self serve products
– A rich suite of self serve products that enable teams to more easily author, deploy, govern and operate their own solutions, aided
by automation and processes that support best practices and high quality as a precondition for deployment.
• Rationalizing data definitions
– A process for rationalizing all critical data definitions at the company so that data concepts like Customer, Product and
Entitlement are unique, re-usable and non-conflicting.
Stewardship
©2021 Intuit Inc. All rights reserved. 14
©2021 Intuit Inc. All rights reserved. 15
Stewardship goals for next year
Organizing People,
Code, and Data
©2021 Intuit Inc. All rights reserved. 17
Raw information about physical systems that describes where the data is stored and where code is
executing. This describes where data is physically located so that it can be accessed.
©2021 Intuit Inc. All rights reserved. 18
Basic dependency, ownership and classification information provides additional context about physical
data and code locations so that data can be better governed, secured and operated by the owning
teams.
©2021 Intuit Inc. All rights reserved. 19
Why organizing people, code and data matters
19
Private vs Public
~50% tables are either
temp/sandbox/staging/test/backu
p tables
- Messes up search & discovery
- Teams consume data not meant
for external use
Data Ownership
~50% tables don’t have clearly
identified owners
- Erodes Trust
- Copies proliferate
- Operational, Governance risk
Self Serve Products
©2021 Intuit Inc. All rights reserved. 21
Data Processing Capabilities Data Serving Capabilities
©2021 Intuit Inc. All rights reserved. 22
Self Serve goals for next year
100% of Top 20 tasks in the Data lifecycle are Self Serve
Infra Provisioning
● Transactional Persistence
● Compute for stream, batch
processing
● Monitor, Debug Infra
● Cost
Data Authoring
● Events, Schemas
● Ingestion
● Transformations
● Entities
● ML Features
● Data Quality,
Observability
● Orchestration
Data Governance
● Access Management
● Key management
● Compliance Controls &
Audit
● Privacy
Rationalizing Data
Definitions
©2021 Intuit Inc. All rights reserved. 24
Clean entity information with formally defined meaning and relationships enables better data understanding. This
is the purpose of entity definitions. They ensure that data is clean, organized, connected, discoverable and
documented in a formal way.
©2021 Intuit Inc. All rights reserved. 25
When you bring it all together, you get Intuit’s Data Mesh
©2021 Intuit Inc. All rights reserved. 26
©2021 Intuit Inc. All rights reserved. 27
©2021 Intuit Inc. All rights reserved. 28
©2021 Intuit Inc. All rights reserved. 29
©2021 Intuit Inc. All rights reserved. 30
Capturing meaning,
relationship, ownership, and
system dependencies builds a
full, rich picture for everyone.
No tribal knowledge needed!
In this example, the clean
information describes entities
OII Account and Intuit Product
and the Entitled To relationship
between them.
The basic information describes
how the data for these entities
are sourced from the Identity
Universal Service and the
Entitlement Reference Service.
The raw information describes
which Event Bus topic and Data
Lake table the data for these
entities can be found in.
Q&A
Tristan Baker - linkedin.com/in/tristanbaker
Suresh Raman - linkedin.com/in/ramansuresh
Allison Bellah (in absentia) - linkedin.com/in/allisonbellah
©2021 Intuit Inc. All rights reserved. 32
32

More Related Content

What's hot

Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?
DATAVERSITY
 
Data Catalog for Better Data Discovery and Governance
Data Catalog for Better Data Discovery and GovernanceData Catalog for Better Data Discovery and Governance
Data Catalog for Better Data Discovery and Governance
Denodo
 
How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...
How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...
How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...
HostedbyConfluent
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
Databricks
 
Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?
DATAVERSITY
 
Data Mesh in Practice - How Europe's Leading Online Platform for Fashion Goes...
Data Mesh in Practice - How Europe's Leading Online Platform for Fashion Goes...Data Mesh in Practice - How Europe's Leading Online Platform for Fashion Goes...
Data Mesh in Practice - How Europe's Leading Online Platform for Fashion Goes...
Dr. Arif Wider
 
Improving Data Literacy Around Data Architecture
Improving Data Literacy Around Data ArchitectureImproving Data Literacy Around Data Architecture
Improving Data Literacy Around Data Architecture
DATAVERSITY
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
James Serra
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business Goals
DATAVERSITY
 
Gartner: Master Data Management Functionality
Gartner: Master Data Management FunctionalityGartner: Master Data Management Functionality
Gartner: Master Data Management Functionality
Gartner
 
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
Databricks
 
Enterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data ArchitectureEnterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data Architecture
DATAVERSITY
 
Databricks: A Tool That Empowers You To Do More With Data
Databricks: A Tool That Empowers You To Do More With DataDatabricks: A Tool That Empowers You To Do More With Data
Databricks: A Tool That Empowers You To Do More With Data
Databricks
 
Modernizing to a Cloud Data Architecture
Modernizing to a Cloud Data ArchitectureModernizing to a Cloud Data Architecture
Modernizing to a Cloud Data Architecture
Databricks
 
Business Intelligence & Data Analytics– An Architected Approach
Business Intelligence & Data Analytics– An Architected ApproachBusiness Intelligence & Data Analytics– An Architected Approach
Business Intelligence & Data Analytics– An Architected Approach
DATAVERSITY
 
Data Modeling, Data Governance, & Data Quality
Data Modeling, Data Governance, & Data QualityData Modeling, Data Governance, & Data Quality
Data Modeling, Data Governance, & Data Quality
DATAVERSITY
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
Databricks
 
Activate Data Governance Using the Data Catalog
Activate Data Governance Using the Data CatalogActivate Data Governance Using the Data Catalog
Activate Data Governance Using the Data Catalog
DATAVERSITY
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business Goals
DATAVERSITY
 
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Databricks
 

What's hot (20)

Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?
 
Data Catalog for Better Data Discovery and Governance
Data Catalog for Better Data Discovery and GovernanceData Catalog for Better Data Discovery and Governance
Data Catalog for Better Data Discovery and Governance
 
How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...
How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...
How to Build the Data Mesh Foundation: A Principled Approach | Zhamak Dehghan...
 
Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4Data Lakehouse Symposium | Day 4
Data Lakehouse Symposium | Day 4
 
Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?
 
Data Mesh in Practice - How Europe's Leading Online Platform for Fashion Goes...
Data Mesh in Practice - How Europe's Leading Online Platform for Fashion Goes...Data Mesh in Practice - How Europe's Leading Online Platform for Fashion Goes...
Data Mesh in Practice - How Europe's Leading Online Platform for Fashion Goes...
 
Improving Data Literacy Around Data Architecture
Improving Data Literacy Around Data ArchitectureImproving Data Literacy Around Data Architecture
Improving Data Literacy Around Data Architecture
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business Goals
 
Gartner: Master Data Management Functionality
Gartner: Master Data Management FunctionalityGartner: Master Data Management Functionality
Gartner: Master Data Management Functionality
 
Architect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh ArchitectureArchitect’s Open-Source Guide for a Data Mesh Architecture
Architect’s Open-Source Guide for a Data Mesh Architecture
 
Enterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data ArchitectureEnterprise Architecture vs. Data Architecture
Enterprise Architecture vs. Data Architecture
 
Databricks: A Tool That Empowers You To Do More With Data
Databricks: A Tool That Empowers You To Do More With DataDatabricks: A Tool That Empowers You To Do More With Data
Databricks: A Tool That Empowers You To Do More With Data
 
Modernizing to a Cloud Data Architecture
Modernizing to a Cloud Data ArchitectureModernizing to a Cloud Data Architecture
Modernizing to a Cloud Data Architecture
 
Business Intelligence & Data Analytics– An Architected Approach
Business Intelligence & Data Analytics– An Architected ApproachBusiness Intelligence & Data Analytics– An Architected Approach
Business Intelligence & Data Analytics– An Architected Approach
 
Data Modeling, Data Governance, & Data Quality
Data Modeling, Data Governance, & Data QualityData Modeling, Data Governance, & Data Quality
Data Modeling, Data Governance, & Data Quality
 
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
5 Critical Steps to Clean Your Data Swamp When Migrating Off of Hadoop
 
Activate Data Governance Using the Data Catalog
Activate Data Governance Using the Data CatalogActivate Data Governance Using the Data Catalog
Activate Data Governance Using the Data Catalog
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business Goals
 
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
 

Similar to Intuit's Data Mesh - Data Mesh Leaning Community meetup 5.13.2021

Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA
 
inSis Suite - Process Data Analytics, Dashboards, Portal & Historian
inSis Suite - Process Data Analytics, Dashboards, Portal & HistorianinSis Suite - Process Data Analytics, Dashboards, Portal & Historian
inSis Suite - Process Data Analytics, Dashboards, Portal & Historian
Kondapi V Siva Rama Brahmam
 
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Denodo
 
RWDG Slides: Using Tools to Advance Your Data Governance Program
RWDG Slides: Using Tools to Advance Your Data Governance ProgramRWDG Slides: Using Tools to Advance Your Data Governance Program
RWDG Slides: Using Tools to Advance Your Data Governance Program
DATAVERSITY
 
How often do Your Machines and People talk? Humanizing the IoT - AWS IoT Web Day
How often do Your Machines and People talk? Humanizing the IoT - AWS IoT Web DayHow often do Your Machines and People talk? Humanizing the IoT - AWS IoT Web Day
How often do Your Machines and People talk? Humanizing the IoT - AWS IoT Web Day
AWS Germany
 
Data Virtualization Accelerating Your Data Strategy
Data Virtualization Accelerating Your Data StrategyData Virtualization Accelerating Your Data Strategy
Data Virtualization Accelerating Your Data Strategy
Denodo
 
Rethink Your 2021 Data Management Strategy with Data Virtualization (ASEAN)
Rethink Your 2021 Data Management Strategy with Data Virtualization (ASEAN)Rethink Your 2021 Data Management Strategy with Data Virtualization (ASEAN)
Rethink Your 2021 Data Management Strategy with Data Virtualization (ASEAN)
Denodo
 
The value of big data analytics
The value of big data analyticsThe value of big data analytics
The value of big data analytics
Marc Vael
 
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
Denodo
 
Modernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your DataModernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your Data
Precisely
 
SG Data Mgt - Findings and Recommendations.pptx
SG Data Mgt - Findings and Recommendations.pptxSG Data Mgt - Findings and Recommendations.pptx
SG Data Mgt - Findings and Recommendations.pptx
ssuser57f752
 
Tdwi march 2015 presentation
Tdwi march 2015 presentationTdwi march 2015 presentation
Tdwi march 2015 presentation
Alison Macfie
 
GDPR Noncompliance: Avoid the Risk with Data Virtualization
GDPR Noncompliance: Avoid the Risk with Data VirtualizationGDPR Noncompliance: Avoid the Risk with Data Virtualization
GDPR Noncompliance: Avoid the Risk with Data Virtualization
Denodo
 
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
TigerGraph
 
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data VirtualizationKASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
Denodo
 
A Real World Case Study for Implementing an Enterprise Scale Data Fabric
A Real World Case Study for Implementing an Enterprise Scale Data FabricA Real World Case Study for Implementing an Enterprise Scale Data Fabric
A Real World Case Study for Implementing an Enterprise Scale Data Fabric
Neo4j
 
Webinar #2 - Transforming Challenges into Opportunities for Credit Unions
Webinar #2 - Transforming Challenges into Opportunities for Credit UnionsWebinar #2 - Transforming Challenges into Opportunities for Credit Unions
Webinar #2 - Transforming Challenges into Opportunities for Credit Unions
Denodo
 
China data-mngnt-solution-market-report
China data-mngnt-solution-market-reportChina data-mngnt-solution-market-report
China data-mngnt-solution-market-report
ssuser7709011
 
Data Governance for End-User Computing
Data Governance for  End-User ComputingData Governance for  End-User Computing
Data Governance for End-User Computing
DATAVERSITY
 
Go from data to decision in one unified platform.pdf
Go from data to decision in one unified platform.pdfGo from data to decision in one unified platform.pdf
Go from data to decision in one unified platform.pdf
webmaster553228
 

Similar to Intuit's Data Mesh - Data Mesh Leaning Community meetup 5.13.2021 (20)

Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
Data Con LA 2022 - Modernizing Analytics & AI for today's needs: Intuit Turbo...
 
inSis Suite - Process Data Analytics, Dashboards, Portal & Historian
inSis Suite - Process Data Analytics, Dashboards, Portal & HistorianinSis Suite - Process Data Analytics, Dashboards, Portal & Historian
inSis Suite - Process Data Analytics, Dashboards, Portal & Historian
 
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
Implementar una estrategia eficiente de gobierno y seguridad del dato con la ...
 
RWDG Slides: Using Tools to Advance Your Data Governance Program
RWDG Slides: Using Tools to Advance Your Data Governance ProgramRWDG Slides: Using Tools to Advance Your Data Governance Program
RWDG Slides: Using Tools to Advance Your Data Governance Program
 
How often do Your Machines and People talk? Humanizing the IoT - AWS IoT Web Day
How often do Your Machines and People talk? Humanizing the IoT - AWS IoT Web DayHow often do Your Machines and People talk? Humanizing the IoT - AWS IoT Web Day
How often do Your Machines and People talk? Humanizing the IoT - AWS IoT Web Day
 
Data Virtualization Accelerating Your Data Strategy
Data Virtualization Accelerating Your Data StrategyData Virtualization Accelerating Your Data Strategy
Data Virtualization Accelerating Your Data Strategy
 
Rethink Your 2021 Data Management Strategy with Data Virtualization (ASEAN)
Rethink Your 2021 Data Management Strategy with Data Virtualization (ASEAN)Rethink Your 2021 Data Management Strategy with Data Virtualization (ASEAN)
Rethink Your 2021 Data Management Strategy with Data Virtualization (ASEAN)
 
The value of big data analytics
The value of big data analyticsThe value of big data analytics
The value of big data analytics
 
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?¿En qué se parece el Gobierno del Dato a un parque de atracciones?
¿En qué se parece el Gobierno del Dato a un parque de atracciones?
 
Modernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your DataModernize your Infrastructure and Mobilize Your Data
Modernize your Infrastructure and Mobilize Your Data
 
SG Data Mgt - Findings and Recommendations.pptx
SG Data Mgt - Findings and Recommendations.pptxSG Data Mgt - Findings and Recommendations.pptx
SG Data Mgt - Findings and Recommendations.pptx
 
Tdwi march 2015 presentation
Tdwi march 2015 presentationTdwi march 2015 presentation
Tdwi march 2015 presentation
 
GDPR Noncompliance: Avoid the Risk with Data Virtualization
GDPR Noncompliance: Avoid the Risk with Data VirtualizationGDPR Noncompliance: Avoid the Risk with Data Virtualization
GDPR Noncompliance: Avoid the Risk with Data Virtualization
 
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
How to Build An AI Based Customer Data Platform: Learn the design patterns fo...
 
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data VirtualizationKASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
KASHTECH AND DENODO: ROI and Economic Value of Data Virtualization
 
A Real World Case Study for Implementing an Enterprise Scale Data Fabric
A Real World Case Study for Implementing an Enterprise Scale Data FabricA Real World Case Study for Implementing an Enterprise Scale Data Fabric
A Real World Case Study for Implementing an Enterprise Scale Data Fabric
 
Webinar #2 - Transforming Challenges into Opportunities for Credit Unions
Webinar #2 - Transforming Challenges into Opportunities for Credit UnionsWebinar #2 - Transforming Challenges into Opportunities for Credit Unions
Webinar #2 - Transforming Challenges into Opportunities for Credit Unions
 
China data-mngnt-solution-market-report
China data-mngnt-solution-market-reportChina data-mngnt-solution-market-report
China data-mngnt-solution-market-report
 
Data Governance for End-User Computing
Data Governance for  End-User ComputingData Governance for  End-User Computing
Data Governance for End-User Computing
 
Go from data to decision in one unified platform.pdf
Go from data to decision in one unified platform.pdfGo from data to decision in one unified platform.pdf
Go from data to decision in one unified platform.pdf
 

Recently uploaded

Training on CSPro and step by steps.pptx
Training on CSPro and step by steps.pptxTraining on CSPro and step by steps.pptx
Training on CSPro and step by steps.pptx
lenjisoHussein
 
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
NABLAS株式会社
 
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
bhupeshkumar0889
 
Solution Manual for First Course in Abstract Algebra A, 8th Edition by John B...
Solution Manual for First Course in Abstract Algebra A, 8th Edition by John B...Solution Manual for First Course in Abstract Algebra A, 8th Edition by John B...
Solution Manual for First Course in Abstract Algebra A, 8th Edition by John B...
rightmanforbloodline
 
DU degree offer diploma Transcript
DU degree offer diploma TranscriptDU degree offer diploma Transcript
DU degree offer diploma Transcript
uapta
 
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
sheetal singh$A17
 
History and Application of LLM Leveraging Big Data
History and Application of LLM Leveraging Big DataHistory and Application of LLM Leveraging Big Data
History and Application of LLM Leveraging Big Data
Jongwook Woo
 
Histology of Muscle types histology o.ppt
Histology of Muscle types histology o.pptHistology of Muscle types histology o.ppt
Histology of Muscle types histology o.ppt
SamanArshad11
 
VVIP Girls Call Noida 9873940964 Provide Best And Top Girl Service And No1 in...
VVIP Girls Call Noida 9873940964 Provide Best And Top Girl Service And No1 in...VVIP Girls Call Noida 9873940964 Provide Best And Top Girl Service And No1 in...
VVIP Girls Call Noida 9873940964 Provide Best And Top Girl Service And No1 in...
Ak47
 
Semantic Web and organizational data .pptx
Semantic Web and organizational data .pptxSemantic Web and organizational data .pptx
Semantic Web and organizational data .pptx
Kanchana Weerasinghe
 
the potential of the development of the Ford–Fulkerson algorithm to solve the...
the potential of the development of the Ford–Fulkerson algorithm to solve the...the potential of the development of the Ford–Fulkerson algorithm to solve the...
the potential of the development of the Ford–Fulkerson algorithm to solve the...
huseindihon
 
AWS re:Invent 2023 - Deep dive into Amazon Aurora and its innovations DAT408
AWS re:Invent 2023 - Deep dive into Amazon Aurora and its innovations DAT408AWS re:Invent 2023 - Deep dive into Amazon Aurora and its innovations DAT408
AWS re:Invent 2023 - Deep dive into Amazon Aurora and its innovations DAT408
Grant McAlister
 
Oracle Database Desupported Features on 23ai (Part B)
Oracle Database Desupported Features on 23ai (Part B)Oracle Database Desupported Features on 23ai (Part B)
Oracle Database Desupported Features on 23ai (Part B)
Alireza Kamrani
 
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
tanupasswan6
 
The Rise of Python in Finance,Automating Trading Strategies: _.pdf
The Rise of Python in Finance,Automating Trading Strategies: _.pdfThe Rise of Python in Finance,Automating Trading Strategies: _.pdf
The Rise of Python in Finance,Automating Trading Strategies: _.pdf
Riya Sen
 
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeliveryBDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
erynsouthern
 
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
6459astrid
 
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
45unexpected
 
Cyber Insurance Mathematical Model & Pricing
Cyber Insurance Mathematical Model & PricingCyber Insurance Mathematical Model & Pricing
Cyber Insurance Mathematical Model & Pricing
BaraDaniel1
 
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
revolutionary575
 

Recently uploaded (20)

Training on CSPro and step by steps.pptx
Training on CSPro and step by steps.pptxTraining on CSPro and step by steps.pptx
Training on CSPro and step by steps.pptx
 
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
社内勉強会資料_TransNeXt: Robust Foveal Visual Perception for Vision Transformers
 
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...🚂🚘 Premium Girls Call Bangalore  🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
🚂🚘 Premium Girls Call Bangalore 🛵🚡000XX00000 💃 Choose Best And Top Girl Serv...
 
Solution Manual for First Course in Abstract Algebra A, 8th Edition by John B...
Solution Manual for First Course in Abstract Algebra A, 8th Edition by John B...Solution Manual for First Course in Abstract Algebra A, 8th Edition by John B...
Solution Manual for First Course in Abstract Algebra A, 8th Edition by John B...
 
DU degree offer diploma Transcript
DU degree offer diploma TranscriptDU degree offer diploma Transcript
DU degree offer diploma Transcript
 
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
Exclusive Girls Call Noida 🎈🔥9873940964 🔥💋🎈 Provide Best And Top Girl Service...
 
History and Application of LLM Leveraging Big Data
History and Application of LLM Leveraging Big DataHistory and Application of LLM Leveraging Big Data
History and Application of LLM Leveraging Big Data
 
Histology of Muscle types histology o.ppt
Histology of Muscle types histology o.pptHistology of Muscle types histology o.ppt
Histology of Muscle types histology o.ppt
 
VVIP Girls Call Noida 9873940964 Provide Best And Top Girl Service And No1 in...
VVIP Girls Call Noida 9873940964 Provide Best And Top Girl Service And No1 in...VVIP Girls Call Noida 9873940964 Provide Best And Top Girl Service And No1 in...
VVIP Girls Call Noida 9873940964 Provide Best And Top Girl Service And No1 in...
 
Semantic Web and organizational data .pptx
Semantic Web and organizational data .pptxSemantic Web and organizational data .pptx
Semantic Web and organizational data .pptx
 
the potential of the development of the Ford–Fulkerson algorithm to solve the...
the potential of the development of the Ford–Fulkerson algorithm to solve the...the potential of the development of the Ford–Fulkerson algorithm to solve the...
the potential of the development of the Ford–Fulkerson algorithm to solve the...
 
AWS re:Invent 2023 - Deep dive into Amazon Aurora and its innovations DAT408
AWS re:Invent 2023 - Deep dive into Amazon Aurora and its innovations DAT408AWS re:Invent 2023 - Deep dive into Amazon Aurora and its innovations DAT408
AWS re:Invent 2023 - Deep dive into Amazon Aurora and its innovations DAT408
 
Oracle Database Desupported Features on 23ai (Part B)
Oracle Database Desupported Features on 23ai (Part B)Oracle Database Desupported Features on 23ai (Part B)
Oracle Database Desupported Features on 23ai (Part B)
 
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
New Girls Call Delhi 🎈🔥9711199171 🔥💋🎈 Provide Best And Top Girl Service And N...
 
The Rise of Python in Finance,Automating Trading Strategies: _.pdf
The Rise of Python in Finance,Automating Trading Strategies: _.pdfThe Rise of Python in Finance,Automating Trading Strategies: _.pdf
The Rise of Python in Finance,Automating Trading Strategies: _.pdf
 
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDeliveryBDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
BDSM Girls Call Mumbai 👀 9820252231 👀 Cash Payment With Room DeliveryDelivery
 
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
Premium Girls Call Navi Mumbai 🎈🔥9920725232 🔥💋🎈 Provide Best And Top Girl Ser...
 
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
Female Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service A...
 
Cyber Insurance Mathematical Model & Pricing
Cyber Insurance Mathematical Model & PricingCyber Insurance Mathematical Model & Pricing
Cyber Insurance Mathematical Model & Pricing
 
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
Celebrity Girls Call Andheri 9930245274 Unlimited Short Providing Girls Servi...
 

Intuit's Data Mesh - Data Mesh Leaning Community meetup 5.13.2021

  • 1. Tristan Baker - linkedin.com/in/tristanbaker Suresh Raman - linkedin.com/in/ramansuresh Allison Bellah (in absentia) - linkedin.com/in/allisonbellah Data Mesh at Intuit May 13, 2021
  • 2. ©2021 Intuit Inc. All rights reserved. 2 Arriving at data mesh Our vision and four part strategy Now with 25% more parts! Q&A Fire away! Agenda
  • 4. ©2021 Intuit Inc. All rights reserved. 4 A brief history of data infrastructure
  • 5. ©2021 Intuit Inc. All rights reserved. 5 A brief history of data infrastructure
  • 6. ©2021 Intuit Inc. All rights reserved. 6 A brief history of data infrastructure
  • 7. ©2021 Intuit Inc. All rights reserved. 7 Today
  • 8. ©2021 Intuit Inc. All rights reserved. 8 What “we cannot scale” sounds like from our users Discovering Data ● Where can I find data about a particular thing (customer, company, etc)? ● Where can I find the data sourced from a particular product or service? Understanding Data ● Who can approve my access so that I can see samples of the data? ● What is the schema of the data? ● What is the business meaning and context of the data? ● Is this data related to other concepts? Is it joinable to other data? What is the meaning of the relationship? Trusting Data ● What system produces this data and at what latency? ● What other systems use this data? ● What is the quality of this data? Is it ‘clean’? ● Which team supports this data if it breaks? Publishing Data ● How do I describe my data so that others understand what it means and how to use it? ● Where do I host my data so that other systems can access it? ● Data systems are complicated, how can I build and operate my process on top of one? ● What are my operational responsibilities once my process/data is in production? ● How do I meet my compliance requirements for processing/storing/publishing data? ● Am I duplicating processing/data that already exists? Consuming Data ● How is this table/topic partitioned? ● Who can approve my production system to access it? ● Will I get alerted if the schema changes?
  • 9. ©2021 Intuit Inc. All rights reserved. 9 The future of data infrastructure ● Data treated as code ● Data service as a facet of a product ● Data responsibility decentralized ● Producers take responsibility for data ● Producers serve consumers ● Data platform provides the ecosystem to govern and manage the lifecycle of data and machine learning The provocation Data Mesh is born
  • 10. Our vision and four part strategy
  • 11. ©2021 Intuit Inc. All rights reserved. 11 Enable more Intuit teams to more easily use and create data
  • 12. ©2021 Intuit Inc. All rights reserved. 12 Four part strategy • Stewardship – ensures accountability for a set of defined responsibilities in building and managing their solutions; including adherence to a set of defined best practices to produce only high quality data. • Organizing people, code and data – A systematic approach to organizing the people, code and data which clearly identifies the owners of a business problem and its solution. • Self serve products – A rich suite of self serve products that enable teams to more easily author, deploy, govern and operate their own solutions, aided by automation and processes that support best practices and high quality as a precondition for deployment. • Rationalizing data definitions – A process for rationalizing all critical data definitions at the company so that data concepts like Customer, Product and Entitlement are unique, re-usable and non-conflicting.
  • 14. ©2021 Intuit Inc. All rights reserved. 14
  • 15. ©2021 Intuit Inc. All rights reserved. 15 Stewardship goals for next year
  • 17. ©2021 Intuit Inc. All rights reserved. 17 Raw information about physical systems that describes where the data is stored and where code is executing. This describes where data is physically located so that it can be accessed.
  • 18. ©2021 Intuit Inc. All rights reserved. 18 Basic dependency, ownership and classification information provides additional context about physical data and code locations so that data can be better governed, secured and operated by the owning teams.
  • 19. ©2021 Intuit Inc. All rights reserved. 19 Why organizing people, code and data matters 19 Private vs Public ~50% tables are either temp/sandbox/staging/test/backu p tables - Messes up search & discovery - Teams consume data not meant for external use Data Ownership ~50% tables don’t have clearly identified owners - Erodes Trust - Copies proliferate - Operational, Governance risk
  • 21. ©2021 Intuit Inc. All rights reserved. 21 Data Processing Capabilities Data Serving Capabilities
  • 22. ©2021 Intuit Inc. All rights reserved. 22 Self Serve goals for next year 100% of Top 20 tasks in the Data lifecycle are Self Serve Infra Provisioning ● Transactional Persistence ● Compute for stream, batch processing ● Monitor, Debug Infra ● Cost Data Authoring ● Events, Schemas ● Ingestion ● Transformations ● Entities ● ML Features ● Data Quality, Observability ● Orchestration Data Governance ● Access Management ● Key management ● Compliance Controls & Audit ● Privacy
  • 24. ©2021 Intuit Inc. All rights reserved. 24 Clean entity information with formally defined meaning and relationships enables better data understanding. This is the purpose of entity definitions. They ensure that data is clean, organized, connected, discoverable and documented in a formal way.
  • 25. ©2021 Intuit Inc. All rights reserved. 25 When you bring it all together, you get Intuit’s Data Mesh
  • 26. ©2021 Intuit Inc. All rights reserved. 26
  • 27. ©2021 Intuit Inc. All rights reserved. 27
  • 28. ©2021 Intuit Inc. All rights reserved. 28
  • 29. ©2021 Intuit Inc. All rights reserved. 29
  • 30. ©2021 Intuit Inc. All rights reserved. 30 Capturing meaning, relationship, ownership, and system dependencies builds a full, rich picture for everyone. No tribal knowledge needed! In this example, the clean information describes entities OII Account and Intuit Product and the Entitled To relationship between them. The basic information describes how the data for these entities are sourced from the Identity Universal Service and the Entitlement Reference Service. The raw information describes which Event Bus topic and Data Lake table the data for these entities can be found in.
  • 31. Q&A Tristan Baker - linkedin.com/in/tristanbaker Suresh Raman - linkedin.com/in/ramansuresh Allison Bellah (in absentia) - linkedin.com/in/allisonbellah
  • 32. ©2021 Intuit Inc. All rights reserved. 32 32