Knowledge Graphs are becoming the de-facto solution for managing complex aggregated knowledge and Neo4j is the leading platform for storing and querying connected data. Christophe Willemsen describes a graph-centric cognitive computing pipeline and detail the process from the ingestion of unstructured text up to the generation of a knowledge graph, queryable using natural language through chatbots built with IBM Watson Conversation.

1. Knowledge Graphs and
Chatbots with Neo4j
GraphAware, world’s #1 Neo4j consultancy
graphaware.com
@graph_aware

2. ● Christophe Willemsen
● From Belgium but living in Southern Italy
● Principal Consultant at GraphAware
● Expert in Graphs and Search
● Currently researching in Natural Language Understanding
and its implementation in chatbots
About me
@ikwattro

3. ● How to represent Knowledge in a Graph
○ Text Understanding
■ Processing Text
■ Enrichment
■ Information Extraction
■ Facts
○ Building the Knowledge Graph
● Querying Knowledge
○ Intent detection and Entity Extraction
○ Query representation
○ Finding Responses
○ Context
Outline

4. ● Original domain model,
for eg authors of news,
news, views, original
topics, etc.
● Standard text
processing features
● Information Extraction
● Enrichment
● Facts
● Knowledge Entities
and Semantic
relationships
● Built from aggregation
and validation of layer
1 and 2
● Offers Multi-View
(Perspectives)
DOMAIN MODEL
LAYER 1
TEXT UNDERSTANDING
LAYER 2
KNOWLEDGE GRAPH
LAYER 3
USER CONTEXT
LAYER 4
● Keeps track of the user
conversation
● Relates conversation
steps to entry points in
the Knowledge Graph
● Use distance to
determine out of
context situations

5. How to represent
Knowledge in a Graph?

6. Ahmed Fathi, Director of Marketing for ABC Arabia Bank in
Dubai, said on Monday 12th of January that in the next two
years investment banks in Dubai will almost certainly use
blockchains technology for corporate lending

7. ● Original domain model,
for eg authors of news,
news, views, original
topics, etc.
● Standard text
processing features
● Information Extraction
● Enrichment
● Facts
● Knowledge Entities
and Semantic
relationships
● Built from aggregation
and validation of layer
1 and 2
● Offers Multi-View
(Perspectives)
DOMAIN MODEL
LAYER 1
TEXT UNDERSTANDING
LAYER 2
KNOWLEDGE GRAPH
LAYER 3
USER CONTEXT
LAYER 4
● Keeps track of the user
conversation
● Relates conversation
steps to entry points in
the Knowledge Graph
● Use distance to
determine out of
context situations

8. ● Natural Language Processing
○ Sentence segmentation
○ Tokenization
○ Stopwords Removal
○ Part of Speech Tagging
○
Text Understanding

9. ● Natural Language Processing
○ Sentence segmentation
○ Tokenization
○ Stopwords Removal
○ Part of Speech Tagging
○ Named Entity Recognition
Text Understanding

10. CALL ga.nlp.annotate({text: n.text, id: id(n)})
Text Understanding

13. ● Natural Language Processing
○ Sentence segmentation
○ Tokenization
○ Stopwords Removal
○ Part of Speech Tagging
○ Named Entity Recognition
○ Syntactic Dependencies Parsing
Text Understanding

16. ● Enrichment
○ Uses external tools to enrich the original knowledge, for eg :
director of marketing -> leadership position
○ Makes implicit relationships now explicit between original
documents, for eg :
Doc 1 refers to Los Angeles, Doc 2 to San Francisco, nothing
relates them with pure text, a new “California” node will now
relate explicitly those 2 documents thanks to external
knowledge bases
Text Understanding

18. CALL ga.nlp.enrich.concept({node: n})
Text Understanding

21. ● Enrichment
○ Uses third-party services like IBM Watson Visual Recognition
to gather metadata from images present in our documents
Text Understanding

23. ● Facts
○ Using Semantic Parsers to extract facts from sentences in a
Subject-Root-Object form
Text Understanding

26. ● Quick Summary
○ Original text has been analysed with natural language
processing tools
○ We have extracted entities
○ We gathered new knowledge with external tools
○ We have facts about sentences
○ We have a semantic linguistic graph of the original text
Text Understanding

27. The Knowledge Graph

28. ● Original domain model,
for eg authors of news,
news, views, original
topics, etc.
● Standard text
processing features
● Information Extraction
● Enrichment
● Facts
● Knowledge Entities
and Semantic
relationships
● Built from aggregation
and validation of layer
1 and 2
● Offers Multi-View
(Perspectives)
DOMAIN MODEL
LAYER 1
TEXT UNDERSTANDING
LAYER 2
KNOWLEDGE GRAPH
LAYER 3
USER CONTEXT
LAYER 4
● Keeps track of the user
conversation
● Relates conversation
steps to entry points in
the Knowledge Graph
● Use distance to
determine out of
context situations

29. ● How to build it ?
○ Aggregate all the information from the previous steps
○ Use external tools to validate some assumptions, e.g. :
Director of marketing -> corporate position
○ Create a new graph from those informations
The Knowledge Graph

30. ● Analyse what informations we have
○ A Person named Ahmed Fathi
○ This person has a position as director of marketing
○ This person has a position for ABC Arabia Bank
○ ABC Arabia Bank is located in Dubai
○ A Person say something on a certain date
○ This something is related to a sentence that has keywords,
keywords can also be enriched
○ The position of the person can also be enriched
○ ...
The Knowledge Graph

33. Querying Knowledge

34. ● Original domain model,
for eg authors of news,
news, views, original
topics, etc.
● Standard text
processing features
● Information Extraction
● Enrichment
● Facts
● Knowledge Entities
and Semantic
relationships
● Built from aggregation
and validation of layer
1 and 2
● Offers Multi-View
(Perspectives)
DOMAIN MODEL
LAYER 1
TEXT UNDERSTANDING
LAYER 2
KNOWLEDGE GRAPH
LAYER 3
USER CONTEXT
LAYER 4
● Keeps track of the user
conversation
● Relates conversation
steps to entry points in
the Knowledge Graph
● Use distance to
determine out of
context situations

35. ● Intent detection
○ What did Ahmed Fathi say about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position think about
investment banks ?
Querying Knowledge

36. ● Intent detection
○ What did Ahmed Fathi say about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position think about
investment banks ?
Understand what we want out of the knowledge graph
Querying Knowledge

37. ● Intent detection
○ What did Ahmed Fathi say about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position thinks about
investment banks ?
Understand what we want out of the knowledge graph
Querying Knowledge
Looking for facts ?

38. ● Intent detection
○ What did say Ahmed Fathi about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position thinks about
investment banks ?
Understand what we want out of the knowledge graph
Querying Knowledge
Looking for a Location entity

39. ● Intent detection
○ What did say Ahmed Fathi about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position think about
investment banks ?
Understand what we want out of the knowledge graph
Querying Knowledge
If you “say” something, do you
“think” it ?

40. ● Entity Extraction
○ What did Ahmed Fathi say about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position think about
investment banks ?
Querying Knowledge

41. ● Entity Extraction
○ What did Ahmed Fathi say about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position think about
investment banks ?
Determine Entry Points in the Knowledge Graph as well as
semantic constraints
Querying Knowledge

42. ● Entity Extraction
○ What did Ahmed Fathi say about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position thinks about
investment banks ?
Determine Entry Points in the Knowledge Graph as well as
semantic constraints
Querying Knowledge
Person Entity : Ahmed Fathi
Semantic: SAY
Topic: blockchain

43. ● Entity Extraction
○ What did say Ahmed Fathi about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position thinks about
investment banks ?
Determine Entry Points in the Knowledge Graph as well as
semantic constraints
Querying Knowledge
Company Entity : ABC Arabia Bank

44. ● Entity Extraction
○ What did say Ahmed Fathi about blockchains ?
○ Where is ABC Arabia Bank located ?
○ What does a person with a leader position thinks about
investment banks ?
Determine Entry Points in the Knowledge Graph as well as
semantic constraints
Querying Knowledge
Company Position : Leader position
Person: related to position
Semantic: think
Topic: investment bank

45. ● Query representation
○ Queries should be treated as original corpus and pass the
same set of processing
Querying Knowledge

47. Can this be used to match against a Person label in the
Knowledge Graph ?

48. How to match “do think about” with a “SAY” relationship in the Knowledge
Graph?

49. How to match “do think about” to a “SAY” relationship in the Knowledge Graph?

50. Probabilistic Traversals

51. ● Probabilistic Traversals
○ Use a probability based classifier like Naive Bayes for
determining the type of the relationship to traverse
○ Avoid to return non-relevant results in a “Always return
something architecture”

52. ● Dynamic Query Stack
○ Depending on the intent, the graph should be queried
differently, there is no rocket science out of the box answer to
how, just knowing your domain and lot lot lot of failures and
tests
○ The Knowledge Graph is queried but not only that, you could
also query the NLP graph for tf-idf enforcement and score the
results with different weights at the end

53. ● Queries made
○ EntityLinking()
○ EntitiesSimilarity()
○ SemanticSimilarity()
○ TraversalProbability()
○ KeywordSensitivePageRank()
○ TopicSensitivePageRank()
○ TF-IDF()

54. ● Lot of techniques
○ Minimum subgraph matching
○ Sequence pattern recognition for SVO generation when
queries are not parsed fully
○ Voice adaptations aka soundex
○ Deep learning
○ ...

56. User Query and Context

57. ● Conversation
○ After all, chatbots are considered as Conversational Interfaces,
it wouldn’t have this name if the end goal of such systems is
having a conversation with a machine
○ Keeping track of where the user is in the conversation can help
to add more constraints to the queries
User Context

59. ● When to go out ?
○ A user can quickly go out of context during a conversation, for
example :
* How is the weather in San Francisco ?
-- it is 25 degrees
* what size?
-- ?? Are you really gonna try to find a response ?
○ We use distance calculation in the graph based to trigger
Signals if the user is out of context
User Context

60. ● Some more thoughts
○ Embrace failures
○ Monitor
○ Humans are still a thing
○ ..

61. www.graphaware.com
@graph_aware
Thank you !
world’s #1 Neo4j consultancy
Christophe Willemsen
@ikwattro