These are the presentation materials from our lunch and learn: Tackling GenAI Challenges with Knowledge Graphs, Graph Data Science and LLMs. Watch the full recording here: https://www.youtube.com/watch?v=Dlz3bAssKSU

1. Tackling GenAI
Challenges
How Knowledge Graphs, Graph Data Science, and
LLMs Combine for Impactful AI
Yizhi Yin, Ph.D.
Solution Engineer
Katie Roberts, Ph.D.
Data Science
Solution Architect

2. Why Ground your LLM?
2 Neo4j Inc. All rights reserved 2023
2
Hurricane Calvin
caused minor
flooding in
Hawaii….
What was the
impact of
Hurricane Calvin?

3. Why Ground your LLM?
3 Neo4j Inc. All rights reserved 2023
3
50 policyholders
filed for property
damage due to
Hurricane Calvin.
What was the
impact of
Hurricane Calvin?

4. Why Ground your LLM?
4 Neo4j Inc. All rights reserved 2023
4
50 policyholders
filed for property
damage due to
Hurricane Calvin.
Accurate, Relevant
Responses
What was the
impact of
Hurricane Calvin?

5. Grounding with Knowledge Graphs
Connect Enrich Consume
Context rich,
connected view of
your data that
enables easier
decision making
Enhance your data
with graph data
science, text
embeddings, and
additional derived
context
Ground responses
with information and
context in the graph
Improve search
relevance combining
vector search and
graph traversals
Neo4j Inc. All rights reserved 2023
5
Capture
Log, visualize, and
analyze LLM
interactions to
improve application
deployments

6. Connected Data
With Knowledge Graphs
Neo4j Inc. All rights reserved 2023
6

7. Knowledge Graphs
Neo4j Inc. All rights reserved 2023
7
Structured
Unstructured
Ontologies
Graph Algorithms and
Graph Queries
Semantics,
Derived Relationships and
Additional Context
Natural
Relationships

8. Neo4j Inc. All rights reserved 2023
8
What can you do with a knowledge graph?
Finance
How many flagged accounts are
in the applicant’s network 4+
hops out?
How many login / account
variables in common?
Add these metrics to your
approval process

9. Neo4j Inc. All rights reserved 2023
9
What can you do with a knowledge graph?
Finance
How many flagged accounts are
in the applicant’s network 4+
hops out?
How many login / account
variables in common?
Add these metrics to your
approval process
What completes the
connections from genes to
diseases to targets?
What genes can be reached 4+
hops out from a known drug
target?
What mechanisms in common
are there between two drugs?
Life Sciences

10. Neo4j Inc. All rights reserved 2023
10
What can you do with a knowledge graph?
Finance
How many flagged accounts are
in the applicant’s network 4+
hops out?
How many login / account
variables in common?
Add these metrics to your
approval process
What completes the
connections from genes to
diseases to targets?
What genes can be reached 4+
hops out from a known drug
target?
What mechanisms in common
are there between two drugs?
Collaborative filtering: users
who bought X, also bought Y
What items make you more
likely to buy additional items in
subsequent transactions?
Traverse hierarchies-what items
are similar 4+ hops out?
Life Sciences
Marketing &
Recommendations

11. Enrich
With Cypher Patterns, Graph Data Science & Text Embeddings
Neo4j Inc. All rights reserved 2023
11

12. Elements of High-Quality Grounding Data
Neo4j Inc. All rights reserved 2023
12
1
2
3
Relevant
Augmenting
Reliable
3 Clean
3 Efficient
4
5

13. Elements of High-Quality Grounding Data
Neo4j Inc. All rights reserved 2023
13
1
2
3
Relevant
Augmenting
Reliable
3 Clean
3 Efficient
4
5

14. Graph Enrichment
Neo4j Inc. All rights reserved 2023
14
Human-crafted query, human-readable result
MATCH (p1:Person)-[:ENEMY]->(:Person)<-[:ENEMY]-(p2:PERSON)
MERGE (p1)-[:FRIEND]->(p2)
AI-learned formula, machine-readable result
Predefined formula, human-readable result
PageRank(Emil) = 13.25
PageRank(Amy) = 4.83
PageRank(Alicia) = 4.75
Node2Vec(Emil) =[5.4 5.1 2.4 4.5 3.1]
Node2Vec(Amy) =[2.8 1.8 7.2 0.9 3.0]
Node2Vec(Alicia)=[1.4 5.2 4.4 3.9 3.2]
Queries
Algorithms
Embeddings
Machine
Learning
Workflows
Train ML models
based on results

15. What Are Graph Algorithms?
Neo4j Inc. All rights reserved 2023
15

16. Insights From Graph Algorithms
Neo4j Inc. All rights reserved 2023
16
Outliers, Influencers, Vulnerabilities, ...
Recommendations, Homophily, Outliers, ...
Recommendations, What-if Analysis, Disambiguation, ...
Shortest Path, Optimal Path, Route Optimization, ...
Link prediction, Recommendations, Next-Best Action, ...
Centrality
Pathfinding
Community
Detection
Similarity
Embeddings
Link Prediction
Dimensionality Reduction, Representation Learning, ...

17. Node Embedding
Neo4j Inc. All rights reserved 2023
17
Similarity of embeddings between nodes is
reflective of the similarity in the actual graph
A
B
zA
zB
ENC(A)
ENC(B)

18. Graph Embeddings & KNN Similarity
Neo4j Inc. All rights reserved 2023
18
KNN is a similarity algorithm that compares the
properties of nodes to find the k most similar
neighbors.
How: Infer relationships based on embedding
similarity or other node properties.

19. Entity Resolution
Capture relationships
between entities across
data sources using a
knowledge graph
Construct node embeddings and
resolve entities based on
weighted pairwise similarity between
various entities
Create additional weighted
relationships based on similar
text description and/or other
similar metadata
Identify
communities of
entities based on
distance between node
embeddings
Neo4j Inc. All rights reserved 2023
19

20. Graph Data Science Journey
Knowledge Graphs
Graph Algorithms
Graph Native ML
Find the patterns you’re
looking for in connected data
Identify associations,
anomalies, and trends using
unsupervised machine learning
Learn features in your graph
that you don’t even know
are important yet
→
→
Knowledge Graphs
Graph Algorithms
Graph Native ML
→
→
Neo4j Inc. All rights reserved 2023
20

21. Consume
With LLMs and Semantic Search
Neo4j Inc. All rights reserved 2023
21

22. Text Embedding Vectors for Semantic Search
Given a question, find the most relevant documents based on a similarity metric (such
as Cosine Similarity) between vector of the question and vectors of contents.
Moving from keyword search to similarity (semantic) search.
Neo4j Inc. All rights reserved 2023
22
Q: what is a text
embedding?
abstractId similarity
456 0.923445
22 0.892114
… ...
Top K by similarity

23. Semantic Search Journey
Neo4j Inc. All rights reserved 2023
23
Vector Similarity
Search
Graph Traversals &
Pattern Matching
Knowledge Graph
Inference & ML
Find relevant documents and
content for user queries.
Find people, places, and
things associated to content.
Identify patterns in connected
data.
Further improve search
relevance and insights by
enhancing your Knowledge
Graph.
Use graph algorithms and ML
to discover new relationships,
entities, and groups.
Vector Search
HNSW
Graph Database Graph Data Science

24. Combine Structured & Unstructured Knowledge
Neo4j Inc. All rights reserved 2023
24
PRODUCT
PRODUCT
PRODUCT
PRODUCT
[0.2322,0.3321,….,0.0021]
[0.3233,0.3543,….,0.0047]
[0.5674,0.2134,….,0.0054]
[0.4565,0.2345,….,0.0342]
Unstructured text stored as
node properties

25. Neo4j Inc. All rights reserved 2023
25
PRODUCT
PRODUCT
PRODUCT
PRODUCT
[0.2322,0.3321,….,0.0021]
[0.3233,0.3543,….,0.0047]
[0.5674,0.2134,….,0.0054]
[0.4565,0.2345,….,0.0342]
CATEGORY
CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
Unstructured text stored as
node properties
Combine Structured & Unstructured Knowledge
Structured
information stored
as a graph

26. Neo4j Inc. All rights reserved 2023
26
PRODUCT
PRODUCT
PRODUCT
PRODUCT
[0.2322,0.3321,….,0.0021]
[0.3233,0.3543,….,0.0047]
[0.5674,0.2134,….,0.0054]
[0.4565,0.2345,….,0.0342]
BRAND
BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
CATEGORY
CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
Unstructured text stored as
node properties
Combine Structured & Unstructured Knowledge
Structured
information stored
as a graph

27. Neo4j Inc. All rights reserved 2023
27
PRODUCT
PRODUCT
PRODUCT
PRODUCT
[0.2322,0.3321,….,0.0021]
[0.3233,0.3543,….,0.0047]
[0.5674,0.2134,….,0.0054]
[0.4565,0.2345,….,0.0342]
CATEGORY
CATEGORY
BRAND
BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
CUSTOMER
CUSTOMER
:BOUGHT
:BOUGHT
:BOUGHT
Unstructured text stored as
node properties
Combine Structured & Unstructured Knowledge
Structured
information stored
as a graph

28. Neo4j Inc. All rights reserved 2023
28
PRODUCT
PRODUCT
PRODUCT
PRODUCT
[0.2322,0.3321,….,0.0021]
[0.3233,0.3543,….,0.0047]
[0.5674,0.2134,….,0.0054]
[0.4565,0.2345,….,0.0342]
CATEGORY
CATEGORY
BRAND
BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
CUSTOMER
CUSTOMER
:BOUGHT
:BOUGHT
:BOUGHT
:RATED rating: 5
Unstructured text stored as
node properties
Combine Structured & Unstructured Knowledge
Structured
information stored
as a graph

29. Neo4j Inc. All rights reserved 2023
29
PRODUCT
PRODUCT
PRODUCT
PRODUCT
[0.2322,0.3321,….,0.0021]
[0.3233,0.3543,….,0.0047]
[0.5674,0.2134,….,0.0054]
[0.4565,0.2345,….,0.0342]
CATEGORY
CATEGORY
BRAND
BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
CUSTOMER
CUSTOMER
:BOUGHT
:BOUGHT
:BOUGHT
REVIEW
REVIEW
:WROTE
:WROTE
[0.2322,0.3321,….,0.0021]
[0.5674,0.2134,….,0.0054]
:RATED rating: 5
Unstructured text stored as
node properties
Combine Structured & Unstructured Knowledge
Structured
information stored
as a graph

30. Neo4j Inc. All rights reserved 2023
30
PRODUCT
PRODUCT
PRODUCT
PRODUCT
[0.2322,0.3321,….,0.0021]
[0.3233,0.3543,….,0.0047]
[0.5674,0.2134,….,0.0054]
[0.4565,0.2345,….,0.0342]
CATEGORY
CATEGORY
BRAND
BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:BY_BRAND
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
:HAS_CATEGORY
CUSTOMER
CUSTOMER
:BOUGHT
:BOUGHT
:BOUGHT
:HAS_REVIEW
:HAS_REVIEW
REVIEW
REVIEW
:WROTE
:WROTE
[0.2322,0.3321,….,0.0021]
[0.5674,0.2134,….,0.0054]
:RATED rating: 5
Unstructured text stored as
node properties
Combine Structured & Unstructured Knowledge
Structured
information stored
as a graph

31. Neo4j Inc. All rights reserved 2023
31
Land with
unstructured
text
Enrich & Refine
with structured
data
Combine Structured & Unstructured Knowledge

32. RAG (Retrieval Augmented Generation) Pattern with Neo4j
Neo4j
LLM API
User
Cypher Prompt + Relevant
Information
Prompt Response
Relevant Results
Retrieve relevant results from Neo4j
using LLM to generate embeddings
and/or Cypher
2
3
1
1
2
3
Combine relevant results with prompt
Instruct LLM to only use the relevant
results to generate response
LLM API
Vector and/or Cypher
Generation
Improved ACCURACY and
RELEVANCE of responses
E.g. What is the impact of
Hurricane Calvin?
Hurricane Calvin caused minor
flooding in Hawaii….
vs…
50 policyholders may be at risk of
property damage due to Hurricane
Calvin.
Neo4j Inc. All rights reserved 2023
32

33. Capture
LLM interactions for Analysis
Neo4j Inc. All rights reserved 2023
33

34. Graphs Enable Explainable AI
34
How do you ensure a high-quality production environment with LLMs?
Knowledge Graphs and Graph Data Science enable:
● Logging user interactions in the same database as the context
● Visualizing conversations with context
● Analysing LLM performance and identifying opportunities for
improvement
Neo4j Inc. All rights reserved 2023

35. Example Data Model
35 Neo4j Inc. All rights reserved 2023
User OR Assistant

36. Grounded LLM Conversations are Graphs
Neo4j Inc. All rights reserved 2023
36
Graphs enable logging of LLM conversations in the same database as the context
documents and with defined relationships.

37. Grounded LLM Conversations are Graphs
Neo4j Inc. All rights reserved 2023
37
Graphs enable logging of LLM conversations in the same database as the context
documents and with defined relationships.

38. Elements of High-Quality Grounding Data
Neo4j Inc. All rights reserved 2023
38
1
2
3
Relevant
Augmenting
Reliable
3 Clean
3 Efficient
4
5

39. Elements of High-Quality Grounding Data
Neo4j Inc. All rights reserved 2023
39
1
2
3
Relevant
Augmenting
Reliable
3 Clean
3 Efficient
4
5

40. Neo4j Database
40
API
Knowledge Graph
Graph Data Science
Graph DB
Knowledge Apps
Information
Extraction
& Ingestion
Structured
Unstructured
Ontologies
GCP Vertex AI
Data
Sources API Layer
Customer Service
Ticket Triaging
Recommendations
News Content &
Discovery
Enterprise Knowledge
Search
Patient Prioritization
Clinical Decision
Support Systems
Pharmacovigilance
Health Assistants
FAQ Bots
LLMs
Amazon
SageMaker
AzureML
Bloom
Aura Enterprise Edition LLMs
Neo4j Inc. All rights reserved 2023
API

41. Neo4j Database
41
API
Knowledge Graph
Graph Data Science
Graph DB
Knowledge Apps
Information
Extraction
& Ingestion
Structured
Unstructured
Ontologies
GCP Vertex AI
Data
Sources API Layer
Customer Service
Ticket Triaging
Recommendations
News Content &
Discovery
Enterprise Knowledge
Search
Patient Prioritization
Clinical Decision
Support Systems
Pharmacovigilance
Health Assistants
FAQ Bots
LLMs
Amazon
SageMaker
AzureML
Bloom
Aura Enterprise Edition LLMs
Step 1: Capture Knowledge Step 2: Enrich
API
Step 3: Semantic & Contextual Search
Neo4j Inc. All rights reserved 2023
Step 4: Monitor & Improve

42. Demo
Neo4j Inc. All rights reserved 2023
42
Yizhi Yin, Ph.D.
Solution Engineer

43. Consuming Graph
Enriched Data
Demo 1 : Semantic Search
Neo4j Inc. All rights reserved 2023
43

44. Demo 1
Neo4j Inc. All rights reserved 2023
44
https://msnews.github.io/
750K users and their MSN news click behaviors
made publically available by Microsoft

45. Grounding with
Knowledge Graphs
Demo 2: LLM Chatbot
Neo4j Inc. All rights reserved 2023
45

46. RAG (Retrieval Augmented Generation) Pattern with Neo4j
Neo4j
LLM API
User
Cypher Prompt + Relevant
Information
Prompt Response
Relevant Results
Retrieve relevant results from Neo4j
using LLM to generate embeddings
and/or Cypher
2
3
1
1
2
3
Combine relevant results with prompt
Instruct LLM to only use the relevant
results to generate response
LLM API
Vector and/or Cypher
Generation
Neo4j Inc. All rights reserved 2023
46

47. Typical Business Resilience Data
Analyze business impact of
● software & OS vulnerabilities,
● hardware & software upgrades,
● building/geographic disasters
● changes to business data formats
…across mission critical applications and
business locations
Neo4j Inc. All rights reserved 2023
47
hierarchies, flows,
relationships…

48. Actual Data Model
Neo4j Inc. All rights reserved 2023
48
CVE Data
Business Data Elements
Vendors,Software
Business Tasks
Application Instances
Data Transfers
People/Roles
Locations
IT Assets

49. LangChain Demo Application
● Translates English to
Cypher
● Consumption using LLM
model with few shot
prompting
● Data augmentation from
Neo4j response
Neo4j Inc. All rights reserved 2023
49

50. Demo
Neo4j Inc. All rights reserved 2023
50

51. 51
Resources
Upcoming Event
Neo4j Inc. All rights reserved 2023
Neo4j Bookshelf
neo4j.com/books/
r.neo4j.com/GenAIWebinar
r.neo4j.com/DecConnections
Learn More

52. Thank you!
Neo4j Inc. All rights reserved 2023
52