Neo4j GraphTalks München Jesus Barrasa, Consultant

1. Graph-based Metadata Management &
Data Governance
Dr. Jesús Barrasa - @BarrasaDV
Leverage the power of Graphs for GDPR
8 May 2018
GraphTalks - Munich

2. WHAT IS A GRAPH?

3. A way of representing data
DATA DATA

4. Relational
Database
Good for:
Well-understood data structures
that don’t change too frequently
A way of representing data
Known problems involving discrete
parts of the data, or minimal
connectivity

5. Graph
Database
Relational
Database
Good for:
Well-understood data structures
that don’t change too frequently
Known problems involving discrete
parts of the data, or minimal
connectivity
A way of representing data
Good for:
Dynamic systems: where the data
topology is difficult to predict
Dynamic requirements:
the evolve with the business
Problems where the relationships in
data contribute meaning & value

6. The answer is GRAPHS… but what was
the question?

7. Capture Complexity
Allow Flexibility
Performance for Automated
processing
Bridge the gap between
business and IT
Rich Graph Model
Dynamic Graph Model
Graph Native Storage
Humane (intuitive) Model

8. GRAPH THINKING

9. Look at this data…

10. Swap glasses…

11. … now look at it again, this time as a graph

12. Audience Experiment:
Dependency modelling

13. Element Depends On
A B
A C
A D
C H
D J
E F
E G
F J
G L
H I
J N
J M
L M
Look at this data…

14. Element Depends On
A B
A C
A D
C H
D J
E F
E G
F J
G L
H I
J N
J M
L M
Time challenge #1: Does A depend on F ?
?

15. Look at the data again…

16. Time challenge #2: Does E depend on M ?
?
M
E

17. MATCH (a:Element { id: “A”})
MATCH p = (a)-[:DEPENDS_ON*]->(n { id: “N”})
RETURN p
SELECT
d1.ElementId, d2.ElementId, d3.ElementId
FROM dpndncs AS d1
INNER JOIN dpndncs AS d2
ON d1.dependsOnElemId = d2.ElemId
INNER JOIN dpndncs AS d3
ON d2.dependsOnElemId = d3.ElemId
… <arbitrary number of joins>…
WHERE d1.ElementId = “A”
AND d3.ElementId = “N”
Element Depends On
A B
A C
A D
C H
D J
E F
E G
F J
G L
H I
J N
J M
L M
Does X depend on Y ?

18. Things get more complicated: Detecting SPOFs

19. Detect the SPOF for Component E

20. (spof)<-[:DEPENDS_ON*]-(x:Element)-[:DEPENDS_ON*]->(spof)
Detect the SPOF on a graph?

21. SPOF on tables anyone ?
ElemId dependsOnElemId
A B
A C
A D
C H
D J
E F
E G
F J
G L
H I
J N
J M
L M

22. Use of Graphs has created some of the most successful companies in the world
C
34,3%B
38,4%A
3,3%
D
3,8%
1,8%
1,8%
1,8%
1,8%
1,8%
E
8,1%
F
3,9%

23. Neo4j is the world’s first “off the shelf” graph platform
designed to derive value from data relationships

24. Finance HR &
Recruiting
Manufacturing
& Logistics
Health CareTelco Retail
Today we see graph-projects in virtually every industry
Government

25. NEO4j USE CASES
Real Time Recommendations
Fraud Detection
Identity & Access Mgmnt
Dependency Modelling in Telecoms
Knowledge Graph

26. VALUE FROM GRAPHS
IN DATA GOVERNANCE AND
METADATA MANAGEMENT

27. Data Modelling and Definition
1
Data Lineage2
Consent Management 3
Entitlement 4
GRAPHS IN
METADATA
MANAGEMENT

28. #1 Data Modelling and
Definition

29. Party
CUST_SCHEMA
Party
First NameHAS_LOGICAL_ATT
CUST_SCH
EMA.ROLE
CUST_SCHEM
A_PARTY.FIRST_
NM
CUST_SCH
EMA.PARTY
SHEMA_HAS_TABLE
#1 Data Modelling and Definition
Party
Last Name
HAS_LOGICAL_ATT
SHEMA_HAS_TABLE
GENERATES
CONCEPT_FOR
CONCEPT_FOR
CONCEPT_FOR
HAS_NAME
CUSTOMER
NAMECUSTOMER
HAS_PHONE
TABLE_HAS_COL
CUST_SCHEM
A_PARTY.LAST_
NM
TABLE_HAS_COL
GENERATES
Enterprise Ontology
Application Logical
Model
Physical Schema

30. m
#2 Data Lineage

31. ETL_PROC
_1
SALES_SCHEMA
Normaliz
e_Date
SLS_SCHEM
A.PRODUCT
SLS_SCHEMA.S
ALES.DATE
SLS_SCHE
MA.SALES
SHEMA_HAS_TABLE
#2 Data Lineage
Channel_No
rmalization
SHEMA_HAS_TABLE
HAS_COL
SLS_SCHEMA.S
ALES.CHANNELHAS_COL
BusinessView
Integration MiddlewareOperational Systems
HAS_INPUT
HAS_INPUT
Time.time_key
Time.day_of_
week
Enterprise DWH
HAS_OUTPUT
HAS_OUTPUT
HAS_COL
HAS_COL
TechnicalView
Billing
System
EDWH
CDE:
Transaction
_Date
SENDS RECEIVES
CONCEPTUAL_ELEMENT_FOR
Star_Schema
Star_Sch
ema.Time

32. #3 Consent management

33. #3 Consent Management + MDM
2 Blackfriars
Bridge rd.
J.Barrasa
+44776611…
jesus@neo4j
.com
PHO
NE_FO
R
EMAIL_FOR
ADDRESS_FOR
jb@outlook.
com
EMAIL_FOR
{ contrib: ‘XYZ’,
permittedFor: [UC1,UC4],
consentUntil : 31-12-19 }
{ contrib: ‘internal’,
permittedFor: [UC3],
consentUntil : 31-12-20 }
{ contrib: ‘internal’,
permittedFor: [UC3],
consentUntil : 31-12-20 }
{ contrib: ‘internal’,
permittedFor: [UC3],
consentUntil : 31-12-20 }
EMAIL_FOR
{ contrib: ‘LMN’,
permittedFor: [UC2,UC6],
consentUntil : 31-12-20 }

34. #4 Entitlement

35. #4 Entitlement
User 1 User 3User 2
Exclusion
List G1
Resource 1
Group 1
Resource 2
Group 3
D
MEMBER
CRUD
MEMBER
R
M
EM
BER
MEMBER
CRUD

38. Recent Projects on GDPR & Governance

39. Big 4 Bank in London: Using Neo4j for Metadata
Lineage for GDPR

40. • GDPR: Protecting,
Categorising and Auditing
personal data.
• Lineage
• Change tracking
• Impact analysis
APM +
ITOA
Graph
Database
APM +
ITOA
Data
Classification

41. Global Information Services Group: Using Neo4j for
Identity and Consent Management

42. Identity and consent management
• Sales want to…
“… know which people I can compliantly contact with a mailing
in vertical market x ?”
• Client wants to…
“… see the statement that allowed to contact me”
”… see a full statement of all the information you hold on me”
”… have my contact details suppressed from future mailings”
• Prod Man wants to…
“… assess the impact of removing data from supplier x”
“… test the effects of adjusting consent parameters”

43. 2 Blackfriars
Bridge rd.
J.Barrasa
+44776611…
jesus@neo4j
.com
PHO
NE_FO
R
EMAIL_FOR
ADDRESS_FOR
jb@outlook.
com
EMAIL_FOR
{ contrib: ‘XYZ’,
permittedFor: [UC1,UC4],
consentUntil : 31-12-19 }
{ contrib: ‘internal’,
permittedFor: [UC3],
consentUntil : 31-12-20 }
{ contrib: ‘internal’,
permittedFor: [UC3],
consentUntil : 31-12-20 }
{ contrib: ‘internal’,
permittedFor: [UC3],
consentUntil : 31-12-20 }
EMAIL_FOR
{ contrib: ‘LMN’,
permittedFor: [UC2,UC6],
consentUntil : 31-12-20 }

44. DEMO

45. THANK YOU!