MODERN DATA MANAGEMENT AT SCALE
JOURNEY TO AI ENABLED DATA
Data Management Office, ANZ Australia
Graph Summit Melbourne 2024
Classification: Public
METADATA ENABLES TRUST & UNDERSTANDING
Untrusted
Incomplete
+ Metadata
TRUST!
+ Lineage
THE COMPLEX DATA LANDSCAPE LARGE ORGANISATIONS
FACE TODAY
Culture
Volume
Modernisation efforts coexisting with legacy
Regulatory landscape
DATA MANAGEMENT: MAPPING THE JOURNEY TO MATURITY
Organised
Data
Catalogued
Data
Managed
Data
Unorganised
Data
Intelligent
Data
Low data
maturity
Foundational
data maturity
Modern data
maturity
Transformational
data maturity
Innovative
data maturity
+ common language = + consistent = + context =
+ structure =
MODERN DATA MANAGEMENT;
EMPOWERING A DATA-DRIVEN FUTURE
Traditional
Data
Management
Limited Scalability
Dashboards
Batch
Static & Controlled
Only Reg Data
Siloed Data
Designed to Scale
Knowledge Graphs
Real-time
Agile, flexible
Data Innovation
AI-ready Data
Modern
Data
Management
Data
Capabilities
Defense Offense
OUR APPROACH TO DRIVE MODERN DATA MANAGEMENT
Culture
• Purpose Led
• Invest in people & skills
• Build a foundation
Focus
• Connecting our data
• Bold, Modern but with empathy
• Engagement = Adoption
Capabilities
• Investment in purpose-build
capabilities:
• Metadata Management
• Semantic Modelling
• Knowledge graph / AI
Discovery
CONNECTED DATA FUELS VALUE, DRIVES SCALE
Siloed Data is:
• Incomplete
• Expensive
Connected Data is:
• Contextual
• Value-driven
Data in Cloud
Data in Lakes
Data in Staging Areas
Data in Streams
Data in Warehouses
DATA FABRIC: AN EMERGING DATA MANAGEMENT DESIGN TO
ENABLE CONNECTED & AI READY DATA
External
Reference
Models
Enterprise Data
Taxonomy
Semantic Rules
Data Quality
Rules
Conceptual
Data Model
Business Terms
& Context
Business Rules Reference Data
Metadata
Catalogues
Data Products
Metadata
Context
Mapping
Knowledge Graph
GenAI
Semantics,
Consistency
& Innovation
Data Fabric
BRINGING THIS TO LIFE. A STORY AND AN EXAMPLE
Data Lineage can be time & resource consuming.​
​We’re investing in Data Fabric, Knowledge Graph & Gen AI to innovate & experiment.​
​As a result, we are now able to semantically connect data, visualise and generate insights without
needing deep technical knowledge every time.​
Let’s take a look at one of the capabilities that we are working on.
CONNECTED DATA LINEAGE WITH AI
Data
Sources
Staging
Area
Data Lakes
on Cloud
Data Lakes
on Prem
Data
Users
Data Lineage RAG (Retrieval-Augmented)
Architecture
Chatbot
Enabled
1.
Ask the right
questions, using
the Chatbot
2.
Access the
correct Data
Lineage
3.
Enable tracking
of the correct
information
Converted into Knowledge Graph
DEMO
RECAP - METADATA MATTERS – DATA MUST HAVE CONTEXT TO BE
USEFUL
8 sliced Rosacea
fruit of the Prunus
Persica suspended
in a solution of
polysaccharides
from Barbados and
H2O in a Zinc-Alloy
container and
Bishphenol-A
An indication of the
Producer and Quality
Regulatory required
information (ingredients
and nutritional values)
Easy to identify
Provenance
(Lineage)
Instructions
(how to use)
Serving suggestions
(may go well with)
Best Before Date
(freshness)
It could be anything
No Metadata Technical Metadata
May be true but how it is
useful?
Business & Technical Metadata
Labelled to be found,
understood & used
THANK YOU

ANZ Presentation: GraphSummit Melbourne 2024

  • 1.
    MODERN DATA MANAGEMENTAT SCALE JOURNEY TO AI ENABLED DATA Data Management Office, ANZ Australia Graph Summit Melbourne 2024 Classification: Public
  • 2.
    METADATA ENABLES TRUST& UNDERSTANDING Untrusted Incomplete + Metadata TRUST! + Lineage
  • 3.
    THE COMPLEX DATALANDSCAPE LARGE ORGANISATIONS FACE TODAY Culture Volume Modernisation efforts coexisting with legacy Regulatory landscape
  • 4.
    DATA MANAGEMENT: MAPPINGTHE JOURNEY TO MATURITY Organised Data Catalogued Data Managed Data Unorganised Data Intelligent Data Low data maturity Foundational data maturity Modern data maturity Transformational data maturity Innovative data maturity + common language = + consistent = + context = + structure =
  • 5.
    MODERN DATA MANAGEMENT; EMPOWERINGA DATA-DRIVEN FUTURE Traditional Data Management Limited Scalability Dashboards Batch Static & Controlled Only Reg Data Siloed Data Designed to Scale Knowledge Graphs Real-time Agile, flexible Data Innovation AI-ready Data Modern Data Management Data Capabilities Defense Offense
  • 6.
    OUR APPROACH TODRIVE MODERN DATA MANAGEMENT Culture • Purpose Led • Invest in people & skills • Build a foundation Focus • Connecting our data • Bold, Modern but with empathy • Engagement = Adoption Capabilities • Investment in purpose-build capabilities: • Metadata Management • Semantic Modelling • Knowledge graph / AI Discovery
  • 7.
    CONNECTED DATA FUELSVALUE, DRIVES SCALE Siloed Data is: • Incomplete • Expensive Connected Data is: • Contextual • Value-driven Data in Cloud Data in Lakes Data in Staging Areas Data in Streams Data in Warehouses
  • 8.
    DATA FABRIC: ANEMERGING DATA MANAGEMENT DESIGN TO ENABLE CONNECTED & AI READY DATA External Reference Models Enterprise Data Taxonomy Semantic Rules Data Quality Rules Conceptual Data Model Business Terms & Context Business Rules Reference Data Metadata Catalogues Data Products Metadata Context Mapping Knowledge Graph GenAI Semantics, Consistency & Innovation Data Fabric
  • 9.
    BRINGING THIS TOLIFE. A STORY AND AN EXAMPLE Data Lineage can be time & resource consuming.​ ​We’re investing in Data Fabric, Knowledge Graph & Gen AI to innovate & experiment.​ ​As a result, we are now able to semantically connect data, visualise and generate insights without needing deep technical knowledge every time.​ Let’s take a look at one of the capabilities that we are working on.
  • 10.
    CONNECTED DATA LINEAGEWITH AI Data Sources Staging Area Data Lakes on Cloud Data Lakes on Prem Data Users Data Lineage RAG (Retrieval-Augmented) Architecture Chatbot Enabled 1. Ask the right questions, using the Chatbot 2. Access the correct Data Lineage 3. Enable tracking of the correct information Converted into Knowledge Graph
  • 11.
  • 12.
    RECAP - METADATAMATTERS – DATA MUST HAVE CONTEXT TO BE USEFUL 8 sliced Rosacea fruit of the Prunus Persica suspended in a solution of polysaccharides from Barbados and H2O in a Zinc-Alloy container and Bishphenol-A An indication of the Producer and Quality Regulatory required information (ingredients and nutritional values) Easy to identify Provenance (Lineage) Instructions (how to use) Serving suggestions (may go well with) Best Before Date (freshness) It could be anything No Metadata Technical Metadata May be true but how it is useful? Business & Technical Metadata Labelled to be found, understood & used
  • 14.