1
Apply Knowledge
Graphs & Search
F O R R E A L - W O R L D D E C I S I O N I N T E L L I G E N C E
Justin Sears | VP Product Marketing | Lucidworks
Karl Hampson | Director of AI, CTO | Solstice
January 21, 2020
2
Speakers
Justin Sears
VP Product Marketing
Lucidworks
Karl Hampson
Director of AI, CTO
Solstice
Justin heads product
marketing and analyst
relations at Lucidworks,
creating strategies and
content to position
Lucidworks products.
Justin has led product
marketing teams in venture-
backed Big Data and
analytics companies, and he
lives with his family south of
San Francisco.
Karl is CTO of AI and Data
at Kin+Carta Solstice. He
has been working with
Search for over two
decades with experience
across many use cases and
technologies.
Of particular interest is how
search delivers successful
outcomes for end users and
the developing role of AI in
this.
3
W HAT A R E KNOW LEDGE
GR A PHS ?
C R E AT E A H U M A N - R E A D A B L E
N E T W O R K O F F A C T S
D E S C R I B E R E A L - W O R L D
E N T I T I E S A N D T H E I R
R E L AT I O N S H I P S ( E . G .
O B J E C T S , P L A C E S , E V E N T S )
“ T H I N G S N O T S T R I N G S ”
I N T R O D U C E D B Y G O O G L E I N
2 0 1 2
M A N Y A P P L I C AT I O N S , W E ’ L L
F O C U S O N S E A R C H
44
KNOW LEDGE GR A PHS &
S EA RCH
A S E A R C H I N D E X C O N TA I N S
M A N Y M E N T I O N S O F
T H I N G S , P E O P L E , P L A C E S ,
E V E N T S , E T C . W I T H
I M P L I E D R E L AT I O N S H I P S
S E M A N T I C K N O W L E D G E
G R A P H ( S K G ) T E C H
E X T R A C T S & N AV I G AT E S
T H E M D Y N A M I C A L LY
T H I N K O F O P E R AT I N G AT
T H E “ T H I N G ” L E V E L I N
Y O U R D O C S
5
S E MA NTIC KNOW LE DGE
GR A PH A PI
C O R E S I M I L A R I T Y
E N G I N E , E X P O S E D V I A
A P I
F U L L D O M A I N S U P P O R T
M A N Y L E V E L S O F
I N T E R S E C T I O N S ,
O V E R L A P S &
R E L AT I O N S H I P
S C O R I N G
6
How can we create a Knowledge
Graph?
TR A DITIO NA LLY, NATUR A L LA NGUAGE PRO CES S ING ( A I) CA N HE LP A NA LYZE
TE X T TO PO PULATE KG’S AUTO MATICA LLY W ITH FAC TS IN THE FO R M
SUBJEC T -PRE DICATE - OBJECT. THIS CAN BE A HIGHLY COMPLE X PROCESS.
TOWER BRIDGE IS LOCATED IN LONDON
A S E MA NTIC KNOW LEDGE GR A PH ( S KG) O F CO NCEPTUA LLY S IMILA R
THINGS , DE R IV E D AUTOMAT I CAL LY F RO M INDE X ED DATA BY A NA LYZING
THE IMPLICIT LINKS BE TWE EN THE ‘ THINGS’ ACROSS THE CORPUS
TOWER BRIDGE, THE THAMES, TOWER OF LONDON, THE
SHARD
7
What are some of the benefits of
SKGs?
FO R A LL T HI NGS : E .G. PEO PLE , PLACES , E V E NTS , CO MPA NIES , PRO DUC TS ,
E NTITIES ...
MORE T HAN KEYWORD SEARCH . MOV E BE YO ND MATCHING JUST WO R DS IN
S EA RCHES TO THINGS & R E LATE D THINGS .
AUTOMAT I CAL LY DERI V E I NSI GHTS . DIS COV E R CO NNEC TIONS & PROV IDE
THING -BA S E D NAV IGATIO N O F UNSTR UC TUR E D DATA .
AUTOMAT I C RECOMMENDAT I ONS . FO R A GIV E N THING, S UGGEST OTHE R
THINGS .
PREDI C TI VE ANALYTI CS . FOR A GIVE N THING, PRE DIC T THE IMPORTANCE OF
OTHE R THINGS A ND THE LIKE LIHO OD O F S PECIF IC O UTCO MES .
8
How do SKGs aid decision
intelligence?
DATA GENERATES INSIGHTS, BUT DECISIONS ARE HOW WE
ENGAGE WITH AND IMPACT THE WORLD
KGS CAN HELP CONNECT YOUR USERS AND EMPLOYEES TO THE
MOST RELEVANT INFORMATION THEY NEED TO MAKE BETTER &
FASTER DECISIONS IN THE REAL WORLD.
AT SOLSTICE , WE CONNECT DESI GN THI NKI NG WITH MODERN DATA
PL AT FORMS TO CREAT E DI GI TAL PRODUC TS W HICH TUR N INFO R MATIO N
INTO BE T TE R , FA STE R A ND F E W E R DECIS IONS AT A NY S CA LE .
9
Lucidworks
Fusion
F O R A N E W T Y P E O F K N O W L E D G E M A N A G E M E N T
Photo by James L.W on Unsplash
10
FILTER
VISUALIZATION
ACTIVITY
CONTENT
INDEX
NATURAL
LANGUAGE
TARGETED
RESULTS
MACHINE
LEARNING
QUERY RULE
MATCHING
USER
SIGNALS
FACET, TOPIC
& CLUSTER
D ATA
Human
Generated
System
Generated
Application
Generated
S O L U T I O N
Digital
Commerce
Digital
Workplace
11
Advanced connectors and AI enrichment,
delivered by intuitive applications created with App Studio,
deployed on-prem or as a multi-tenant cloud managed
service.
D ATA
Any format,
any platform
S O L U T I O N
Personalized
to meet needs of
each unique user
FUSION
Server
FUSION
Search AI
FUSION
App Studio
FUSION
Data AI
F U S I O N P L AT F O R M
Human
Generated
System
Generated
Application
Generated
Digital
Commerce
Digital
Workplace
12
FUSION
Server
FUSION
Search AI
FUSION
App Studio
FUSION
Data AI
NLP: NER, phrases, POS
Document classification
Anomaly detection
Clustering
Topic detection
Search engine &
data processing
Connectors
ETL pipelines
Scheduling & alerting
SQL engine
Rules engine
Query pipelines
Query intent detector
Automatic relevancy
Signals & query analytics
Recommenders
A/B testing
Modular components
Stateless architecture
User-focused experience
Geospatial mapping
Results preview
Rapid prototyping
S C A L A B L E O P E R AT I O N S
SECURITYCDCRCLOUDSCALABLEEXTENSIBLE
13
Knowledge Graphs help predict user
intent
Content
Understanding
Keyword
Search
Domain
Understanding
Knowledge
Graph
User
Understanding
Collaborative
Recommendations
USER
INTENT
Semantic
Search
Personalized
Search
Domain-aware
Matching
14
SKG Use Case
#1
Q U E R Y E X PA N S I O N
Some overly simplistic definitions
Alternative Labels: Substitute words with identical meanings
[ CTO => Chief Technology Officer; specialise => specialize ]
Synonyms List: Provides substitute words that can be used to represent
the same or very similar things
[ human => homo sapien, mankind; food => sustenance, meal ]
Taxonomy: Classifies things into Categories
[ John is Human; Human is Mammal; Mammal is Animal ]
Ontology: Defines relationships between types of things
[ animal eats food; human is animal ]
Knowledge Graph: Instantiation of an
Ontology (contains the things that are related)
[ john is human; john eats food ]
In practice, there is significant overlap…
16
Example Query
Places (also includes geonames database)
Entities (includes search commands)
Text Content
[ Web crawl of restaurant and product reviews sites ]
18
SKG Use Case
#2
Q U E R Y D I S A M B I G U AT I O N
19
Example Query
20
What BBQ?
Disambiguation by Category Example
Meaning 1: Restaurant => bbq, brisket, ribs, pork, …
Meaning 2: Outdoor Equipment => bbq, grill, charcoal, propane, …
22Source: M. Korayem, C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015.
Example Related Keywords (representing multiple
meanings)
restaurant coffee shop, diner, deli, chophouse, grill, pizzeria,
barbeque, cafeteria, bistro, barbecue, eatery,
charcuterie, sushi bar, food truck, bbq, ribs, brisket,
salad bar, sausage
cooking
device
BBQ, steamer, hibachi, pit, fryer, crock pot, barbecue,
oven, toaster, stove, barbeque, rotisserie, brasserie, grill
… …
23Source: M. Korayem, C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015.
Example Related Keywords (representing multiple
meanings)
restaurant coffee shop, diner, deli, chophouse, grill, pizzeria,
barbeque, cafeteria, bistro, barbecue, eatery,
charcuterie, sushi bar, food truck, bbq, ribs, brisket,
salad bar, sausage
cooking
device
BBQ, steamer, hibachi, pit, fryer, crock pot, barbecue,
oven, toaster, stove, barbeque, rotisserie, brasserie, grill
… …
24
Red Hat Example
25
26
R E DHAT R ES ULTS FO R
CUSTO ME R PO RTA L
> 2 0 0 % I N C R E A S E I N
C L I C K T H R O U G H R AT E S
9 1 % R E D U C T I O N I N T C O
5 0 , 0 0 F E W E R S U P P O R T
T I C K E T S
I M P R O V E D C U S T O M E R
S AT I S FA C T I O N
27
How they did it: signal-driven
relevancy
M E A S U R I N G I M P R E S S I O N S W H E N C O N T E N T A P P E A RS
I N S E A R C H R E S U LT S ( E .G . , AV E R A G E P O S I T I O N ,
C L I C K - T H R O U G H R AT E , T I M E O N PA G E )
B O O S T I N G D O C U M E N T S W I T H H I G H E R - T H A N -
AV E R A G E R E A D S P E E D S A S A N I N D I C ATO R O F
R E L E VA N C Y
N OT E V E RY C L I C K I S A G O O D C L I C K – B U I L D I N G A
M O D E L I N F U S I O N TO I D E N T I F Y D O C S T H AT P E O P L E
D O N ’ T R E A D TO I M P R OV E C O N T E N T Q UA L I T Y
2828
Questions
29
THANK YOU

Apply Knowledge Graphs and Search for Real-World Decision Intelligence

  • 1.
    1 Apply Knowledge Graphs &Search F O R R E A L - W O R L D D E C I S I O N I N T E L L I G E N C E Justin Sears | VP Product Marketing | Lucidworks Karl Hampson | Director of AI, CTO | Solstice January 21, 2020
  • 2.
    2 Speakers Justin Sears VP ProductMarketing Lucidworks Karl Hampson Director of AI, CTO Solstice Justin heads product marketing and analyst relations at Lucidworks, creating strategies and content to position Lucidworks products. Justin has led product marketing teams in venture- backed Big Data and analytics companies, and he lives with his family south of San Francisco. Karl is CTO of AI and Data at Kin+Carta Solstice. He has been working with Search for over two decades with experience across many use cases and technologies. Of particular interest is how search delivers successful outcomes for end users and the developing role of AI in this.
  • 3.
    3 W HAT AR E KNOW LEDGE GR A PHS ? C R E AT E A H U M A N - R E A D A B L E N E T W O R K O F F A C T S D E S C R I B E R E A L - W O R L D E N T I T I E S A N D T H E I R R E L AT I O N S H I P S ( E . G . O B J E C T S , P L A C E S , E V E N T S ) “ T H I N G S N O T S T R I N G S ” I N T R O D U C E D B Y G O O G L E I N 2 0 1 2 M A N Y A P P L I C AT I O N S , W E ’ L L F O C U S O N S E A R C H
  • 4.
    44 KNOW LEDGE GRA PHS & S EA RCH A S E A R C H I N D E X C O N TA I N S M A N Y M E N T I O N S O F T H I N G S , P E O P L E , P L A C E S , E V E N T S , E T C . W I T H I M P L I E D R E L AT I O N S H I P S S E M A N T I C K N O W L E D G E G R A P H ( S K G ) T E C H E X T R A C T S & N AV I G AT E S T H E M D Y N A M I C A L LY T H I N K O F O P E R AT I N G AT T H E “ T H I N G ” L E V E L I N Y O U R D O C S
  • 5.
    5 S E MANTIC KNOW LE DGE GR A PH A PI C O R E S I M I L A R I T Y E N G I N E , E X P O S E D V I A A P I F U L L D O M A I N S U P P O R T M A N Y L E V E L S O F I N T E R S E C T I O N S , O V E R L A P S & R E L AT I O N S H I P S C O R I N G
  • 6.
    6 How can wecreate a Knowledge Graph? TR A DITIO NA LLY, NATUR A L LA NGUAGE PRO CES S ING ( A I) CA N HE LP A NA LYZE TE X T TO PO PULATE KG’S AUTO MATICA LLY W ITH FAC TS IN THE FO R M SUBJEC T -PRE DICATE - OBJECT. THIS CAN BE A HIGHLY COMPLE X PROCESS. TOWER BRIDGE IS LOCATED IN LONDON A S E MA NTIC KNOW LEDGE GR A PH ( S KG) O F CO NCEPTUA LLY S IMILA R THINGS , DE R IV E D AUTOMAT I CAL LY F RO M INDE X ED DATA BY A NA LYZING THE IMPLICIT LINKS BE TWE EN THE ‘ THINGS’ ACROSS THE CORPUS TOWER BRIDGE, THE THAMES, TOWER OF LONDON, THE SHARD
  • 7.
    7 What are someof the benefits of SKGs? FO R A LL T HI NGS : E .G. PEO PLE , PLACES , E V E NTS , CO MPA NIES , PRO DUC TS , E NTITIES ... MORE T HAN KEYWORD SEARCH . MOV E BE YO ND MATCHING JUST WO R DS IN S EA RCHES TO THINGS & R E LATE D THINGS . AUTOMAT I CAL LY DERI V E I NSI GHTS . DIS COV E R CO NNEC TIONS & PROV IDE THING -BA S E D NAV IGATIO N O F UNSTR UC TUR E D DATA . AUTOMAT I C RECOMMENDAT I ONS . FO R A GIV E N THING, S UGGEST OTHE R THINGS . PREDI C TI VE ANALYTI CS . FOR A GIVE N THING, PRE DIC T THE IMPORTANCE OF OTHE R THINGS A ND THE LIKE LIHO OD O F S PECIF IC O UTCO MES .
  • 8.
    8 How do SKGsaid decision intelligence? DATA GENERATES INSIGHTS, BUT DECISIONS ARE HOW WE ENGAGE WITH AND IMPACT THE WORLD KGS CAN HELP CONNECT YOUR USERS AND EMPLOYEES TO THE MOST RELEVANT INFORMATION THEY NEED TO MAKE BETTER & FASTER DECISIONS IN THE REAL WORLD. AT SOLSTICE , WE CONNECT DESI GN THI NKI NG WITH MODERN DATA PL AT FORMS TO CREAT E DI GI TAL PRODUC TS W HICH TUR N INFO R MATIO N INTO BE T TE R , FA STE R A ND F E W E R DECIS IONS AT A NY S CA LE .
  • 9.
    9 Lucidworks Fusion F O RA N E W T Y P E O F K N O W L E D G E M A N A G E M E N T Photo by James L.W on Unsplash
  • 10.
    10 FILTER VISUALIZATION ACTIVITY CONTENT INDEX NATURAL LANGUAGE TARGETED RESULTS MACHINE LEARNING QUERY RULE MATCHING USER SIGNALS FACET, TOPIC &CLUSTER D ATA Human Generated System Generated Application Generated S O L U T I O N Digital Commerce Digital Workplace
  • 11.
    11 Advanced connectors andAI enrichment, delivered by intuitive applications created with App Studio, deployed on-prem or as a multi-tenant cloud managed service. D ATA Any format, any platform S O L U T I O N Personalized to meet needs of each unique user FUSION Server FUSION Search AI FUSION App Studio FUSION Data AI F U S I O N P L AT F O R M Human Generated System Generated Application Generated Digital Commerce Digital Workplace
  • 12.
    12 FUSION Server FUSION Search AI FUSION App Studio FUSION DataAI NLP: NER, phrases, POS Document classification Anomaly detection Clustering Topic detection Search engine & data processing Connectors ETL pipelines Scheduling & alerting SQL engine Rules engine Query pipelines Query intent detector Automatic relevancy Signals & query analytics Recommenders A/B testing Modular components Stateless architecture User-focused experience Geospatial mapping Results preview Rapid prototyping S C A L A B L E O P E R AT I O N S SECURITYCDCRCLOUDSCALABLEEXTENSIBLE
  • 13.
    13 Knowledge Graphs helppredict user intent Content Understanding Keyword Search Domain Understanding Knowledge Graph User Understanding Collaborative Recommendations USER INTENT Semantic Search Personalized Search Domain-aware Matching
  • 14.
    14 SKG Use Case #1 QU E R Y E X PA N S I O N
  • 15.
    Some overly simplisticdefinitions Alternative Labels: Substitute words with identical meanings [ CTO => Chief Technology Officer; specialise => specialize ] Synonyms List: Provides substitute words that can be used to represent the same or very similar things [ human => homo sapien, mankind; food => sustenance, meal ] Taxonomy: Classifies things into Categories [ John is Human; Human is Mammal; Mammal is Animal ] Ontology: Defines relationships between types of things [ animal eats food; human is animal ] Knowledge Graph: Instantiation of an Ontology (contains the things that are related) [ john is human; john eats food ] In practice, there is significant overlap…
  • 16.
  • 17.
    Places (also includesgeonames database) Entities (includes search commands) Text Content [ Web crawl of restaurant and product reviews sites ]
  • 18.
    18 SKG Use Case #2 QU E R Y D I S A M B I G U AT I O N
  • 19.
  • 20.
  • 21.
    Disambiguation by CategoryExample Meaning 1: Restaurant => bbq, brisket, ribs, pork, … Meaning 2: Outdoor Equipment => bbq, grill, charcoal, propane, …
  • 22.
    22Source: M. Korayem,C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015. Example Related Keywords (representing multiple meanings) restaurant coffee shop, diner, deli, chophouse, grill, pizzeria, barbeque, cafeteria, bistro, barbecue, eatery, charcuterie, sushi bar, food truck, bbq, ribs, brisket, salad bar, sausage cooking device BBQ, steamer, hibachi, pit, fryer, crock pot, barbecue, oven, toaster, stove, barbeque, rotisserie, brasserie, grill … …
  • 23.
    23Source: M. Korayem,C. Ortiz, K. AlJadda, T. Grainger. "Query Sense Disambiguation Leveraging Large Scale User Behavioral Data". IEEE Big Data 2015. Example Related Keywords (representing multiple meanings) restaurant coffee shop, diner, deli, chophouse, grill, pizzeria, barbeque, cafeteria, bistro, barbecue, eatery, charcuterie, sushi bar, food truck, bbq, ribs, brisket, salad bar, sausage cooking device BBQ, steamer, hibachi, pit, fryer, crock pot, barbecue, oven, toaster, stove, barbeque, rotisserie, brasserie, grill … …
  • 24.
  • 25.
  • 26.
    26 R E DHATR ES ULTS FO R CUSTO ME R PO RTA L > 2 0 0 % I N C R E A S E I N C L I C K T H R O U G H R AT E S 9 1 % R E D U C T I O N I N T C O 5 0 , 0 0 F E W E R S U P P O R T T I C K E T S I M P R O V E D C U S T O M E R S AT I S FA C T I O N
  • 27.
    27 How they didit: signal-driven relevancy M E A S U R I N G I M P R E S S I O N S W H E N C O N T E N T A P P E A RS I N S E A R C H R E S U LT S ( E .G . , AV E R A G E P O S I T I O N , C L I C K - T H R O U G H R AT E , T I M E O N PA G E ) B O O S T I N G D O C U M E N T S W I T H H I G H E R - T H A N - AV E R A G E R E A D S P E E D S A S A N I N D I C ATO R O F R E L E VA N C Y N OT E V E RY C L I C K I S A G O O D C L I C K – B U I L D I N G A M O D E L I N F U S I O N TO I D E N T I F Y D O C S T H AT P E O P L E D O N ’ T R E A D TO I M P R OV E C O N T E N T Q UA L I T Y
  • 28.
  • 29.