Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Neo4j GraphTalks Munich - Graph-based Metadata Managament & Data Governance

102 views

Published on

Neo4j GraphTalks München
Jesus Barrasa, Consultant

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Neo4j GraphTalks Munich - Graph-based Metadata Managament & Data Governance

  1. 1. Graph-based Metadata Management & Data Governance Dr. Jesús Barrasa - @BarrasaDV Leverage the power of Graphs for GDPR 8 May 2018 GraphTalks - Munich
  2. 2. WHAT IS A GRAPH?
  3. 3. A way of representing data DATA DATA
  4. 4. Relational Database Good for: Well-understood data structures that don’t change too frequently A way of representing data Known problems involving discrete parts of the data, or minimal connectivity
  5. 5. Graph Database Relational Database Good for: Well-understood data structures that don’t change too frequently Known problems involving discrete parts of the data, or minimal connectivity A way of representing data Good for: Dynamic systems: where the data topology is difficult to predict Dynamic requirements: 
 the evolve with the business Problems where the relationships in data contribute meaning & value
  6. 6. The answer is GRAPHS… but what was the question?
  7. 7. Capture Complexity Allow Flexibility Performance for Automated processing Bridge the gap between business and IT Rich Graph Model Dynamic Graph Model Graph Native Storage Humane (intuitive) Model
  8. 8. GRAPH THINKING
  9. 9. Look at this data…
  10. 10. Swap glasses…
  11. 11. … now look at it again, this time as a graph
  12. 12. Audience Experiment: Dependency modelling
  13. 13. Element Depends On A B A C A D C H D J E F E G F J G L H I J N J M L M Look at this data…
  14. 14. Element Depends On A B A C A D C H D J E F E G F J G L H I J N J M L M Time challenge #1: Does A depend on F ? ?
  15. 15. Look at the data again…
  16. 16. Time challenge #2: Does E depend on M ? ? M E
  17. 17. MATCH (a:Element { id: “A”}) MATCH p = (a)-[:DEPENDS_ON*]->(n { id: “N”})
 RETURN p SELECT d1.ElementId, d2.ElementId, d3.ElementId 
 FROM dpndncs AS d1 INNER JOIN dpndncs AS d2 ON d1.dependsOnElemId = d2.ElemId INNER JOIN dpndncs AS d3 ON d2.dependsOnElemId = d3.ElemId … <arbitrary number of joins>…
 WHERE d1.ElementId = “A” AND d3.ElementId = “N” Element Depends On A B A C A D C H D J E F E G F J G L H I J N J M L M Does X depend on Y ?
  18. 18. Things get more complicated: Detecting SPOFs
  19. 19. Detect the SPOF for Component E
  20. 20. (spof)<-[:DEPENDS_ON*]-(x:Element)-[:DEPENDS_ON*]->(spof) Detect the SPOF on a graph?
  21. 21. SPOF on tables anyone ? ElemId dependsOnElemId A B A C A D C H D J E F E G F J G L H I J N J M L M
  22. 22. Use of Graphs has created some of the most successful companies in the world C 34,3%B 38,4%A 3,3% D 3,8% 1,8% 1,8% 1,8% 1,8% 1,8% E 8,1% F 3,9%
  23. 23. Neo4j is the world’s first “off the shelf” graph platform designed to derive value from data relationships
  24. 24. Finance HR & Recruiting Manufacturing & Logistics Health CareTelco Retail Today we see graph-projects in virtually every industry Government
  25. 25. NEO4j USE CASES Real Time Recommendations Fraud Detection Identity & Access Mgmnt Dependency Modelling in Telecoms Knowledge Graph
  26. 26. VALUE FROM GRAPHS IN DATA GOVERNANCE AND METADATA MANAGEMENT
  27. 27. Data Modelling and Definition 1 Data Lineage2 Consent Management 3 Entitlement 4 GRAPHS IN METADATA MANAGEMENT
  28. 28. #1 Data Modelling and Definition
  29. 29. Party CUST_SCHEMA Party First NameHAS_LOGICAL_ATT CUST_SCH EMA.ROLE CUST_SCHEM A_PARTY.FIRST_ NM CUST_SCH EMA.PARTY SHEMA_HAS_TABLE #1 Data Modelling and Definition Party Last Name HAS_LOGICAL_ATT SHEMA_HAS_TABLE GENERATES CONCEPT_FOR CONCEPT_FOR CONCEPT_FOR HAS_NAME CUSTOMER NAMECUSTOMER HAS_PHONE TABLE_HAS_COL CUST_SCHEM A_PARTY.LAST_ NM TABLE_HAS_COL GENERATES Enterprise Ontology Application Logical Model Physical Schema
  30. 30. m #2 Data Lineage
  31. 31. ETL_PROC _1 SALES_SCHEMA Normaliz e_Date SLS_SCHEM A.PRODUCT SLS_SCHEMA.S ALES.DATE SLS_SCHE MA.SALES SHEMA_HAS_TABLE #2 Data Lineage Channel_No rmalization SHEMA_HAS_TABLE HAS_COL SLS_SCHEMA.S ALES.CHANNELHAS_COL BusinessView Integration MiddlewareOperational Systems HAS_INPUT HAS_INPUT Time.time_key Time.day_of_ week Enterprise DWH HAS_OUTPUT HAS_OUTPUT HAS_COL HAS_COL TechnicalView Billing System EDWH CDE: Transaction _Date SENDS RECEIVES CONCEPTUAL_ELEMENT_FOR Star_Schema Star_Sch ema.Time
  32. 32. #3 Consent management
  33. 33. #3 Consent Management + MDM 2 Blackfriars Bridge rd. J.Barrasa +44776611… jesus@neo4j .com PHO NE_FO R EMAIL_FOR ADDRESS_FOR jb@outlook. com EMAIL_FOR { contrib: ‘XYZ’, permittedFor: [UC1,UC4], consentUntil : 31-12-19 } { contrib: ‘internal’, permittedFor: [UC3], consentUntil : 31-12-20 } { contrib: ‘internal’, permittedFor: [UC3], consentUntil : 31-12-20 } { contrib: ‘internal’, permittedFor: [UC3], consentUntil : 31-12-20 } EMAIL_FOR { contrib: ‘LMN’, permittedFor: [UC2,UC6], consentUntil : 31-12-20 }
  34. 34. #4 Entitlement
  35. 35. #4 Entitlement User 1 User 3User 2 Exclusion List G1 Resource 1 Group 1 Resource 2 Group 3 D MEMBER CRUD MEMBER R M EM BER MEMBER CRUD
  36. 36. Recent Projects on GDPR & Governance
  37. 37. Big 4 Bank in London: Using Neo4j for Metadata Lineage for GDPR
  38. 38. • GDPR: Protecting, Categorising and Auditing personal data. • Lineage • Change tracking • Impact analysis APM + ITOA Graph Database APM + ITOA Data Classification
  39. 39. Global Information Services Group: Using Neo4j for Identity and Consent Management
  40. 40. Identity and consent management • Sales want to… “… know which people I can compliantly contact with a mailing in vertical market x ?” • Client wants to… “… see the statement that allowed to contact me” ”… see a full statement of all the information you hold on me” ”… have my contact details suppressed from future mailings” • Prod Man wants to… “… assess the impact of removing data from supplier x” “… test the effects of adjusting consent parameters”
  41. 41. 2 Blackfriars Bridge rd. J.Barrasa +44776611… jesus@neo4j .com PHO NE_FO R EMAIL_FOR ADDRESS_FOR jb@outlook. com EMAIL_FOR { contrib: ‘XYZ’, permittedFor: [UC1,UC4], consentUntil : 31-12-19 } { contrib: ‘internal’, permittedFor: [UC3], consentUntil : 31-12-20 } { contrib: ‘internal’, permittedFor: [UC3], consentUntil : 31-12-20 } { contrib: ‘internal’, permittedFor: [UC3], consentUntil : 31-12-20 } EMAIL_FOR { contrib: ‘LMN’, permittedFor: [UC2,UC6], consentUntil : 31-12-20 }
  42. 42. DEMO
  43. 43. THANK YOU!

×