SlideShare a Scribd company logo
HOW TO BUILD A
SEARCH ENGINE
(IN TWO DAYS)
SEARCH IS
NOT SEXY
I don’t have anything
against the search idea,
but it just doesn’t click for
me at this point.
SEARCH IS
SUPERSEXY
HOW TO BUILD A
SEARCH ENGINE
(IN TWO DAYS)
SEMANTIC
FAST
RELIABLE
USER-FRIENDLY
SEMANTIC
SEARCH?
Semantic search seeks to improve search
accuracy by understanding the searcher's intent
and the contextual meaning of search terms
HANDLING
- synonyms
- generalizations
- concept matching
- natural language queries and questions
SEMANTIC
QUERY PARSING
ILLNESS
I’m sick
=
I’m ill
=
What is the policy for
sickness?
PARKING
Where can I park?
=
Where can I park my car?
=
I’m looking for a parking
spot
HANDLING
- synonyms
- generalizations
- concept matching
- natural language queries and questions
PARKING
Where can I park?
=
Where can I park my car?
=
I’m looking for a
parking spot
PARKING
Where can I park?
=
Where can I park my car?
=
I’m looking for a
parking spot
VB
VB NN
NN NN
ILLNESS
I’m sick
=
I’m ill
=
What is the policy for
sickness
NN
NN
NN
NN
HANDLING
- synonyms
- generalizations
- concept matching
- natural language queries and questions
CONCEPT
MATCHING
SYNONYMY
INTERMEZZO
Funny fuckup
SEMANTIC
DOCUMENT ANALYSIS
FULL-TEXT?
Master_WIT_Master_1_Sem2_Crypto_Readings_Pickpocketing.Mifare.pdf
Master
WIT -> Wiskundige Ingenieurstechnieken
Master
1
Sem -> Semester
2
Crypto -> Cryptography
Readings
Pickpocketing
Mifare
[pdf]
+ NER
MAKING IT
FAST
MAKING IT
USER-FRIENDLY
TESTING IT’S
RELIABLE
OVERVIEW
ARCHITECTURE
Flask
Web Framework
Query
Spacy
Query Parsing
Concepts
Gensim & Fasttext
Word Embedding Documents
Word
Clouds
Results
ARCHITECTURE
Feedback?
Questions?
Want to start a study group? :-)
hello@willworkfor.coffee
twitter.com/@lienmichiels
medium.com/@Paliendroom

More Related Content

More from Data Science Leuven

Computing power and algorithms? In people we trust
Computing power and algorithms? In people we trustComputing power and algorithms? In people we trust
Computing power and algorithms? In people we trust
Data Science Leuven
 
Trumania , a realistic scenario-based data-generator
Trumania , a realistic scenario-based data-generatorTrumania , a realistic scenario-based data-generator
Trumania , a realistic scenario-based data-generator
Data Science Leuven
 
Recommender systems, optimizing least squares or user experience
Recommender systems, optimizing least squares or user experienceRecommender systems, optimizing least squares or user experience
Recommender systems, optimizing least squares or user experience
Data Science Leuven
 
Replicability and questionable research practices
Replicability and questionable research practicesReplicability and questionable research practices
Replicability and questionable research practices
Data Science Leuven
 
Predicting Eurosong with Google Predicting Eurosong with Google and data visu...
Predicting Eurosong with Google Predicting Eurosong with Google and data visu...Predicting Eurosong with Google Predicting Eurosong with Google and data visu...
Predicting Eurosong with Google Predicting Eurosong with Google and data visu...
Data Science Leuven
 
Storytelling for impactful predictive models - Gert De Geyter
Storytelling for impactful predictive models - Gert De GeyterStorytelling for impactful predictive models - Gert De Geyter
Storytelling for impactful predictive models - Gert De Geyter
Data Science Leuven
 
Lessons from driving analytics projects
Lessons from driving analytics projectsLessons from driving analytics projects
Lessons from driving analytics projects
Data Science Leuven
 
Geospatial visual analytics
Geospatial visual analyticsGeospatial visual analytics
Geospatial visual analytics
Data Science Leuven
 
Open-Source Data Science Crossing The Chasm
Open-Source Data Science Crossing The ChasmOpen-Source Data Science Crossing The Chasm
Open-Source Data Science Crossing The Chasm
Data Science Leuven
 
Probabilistic machine learning for optimization and solving complex
Probabilistic machine learning for optimization and solving complexProbabilistic machine learning for optimization and solving complex
Probabilistic machine learning for optimization and solving complex
Data Science Leuven
 
Closing
ClosingClosing
Welcome
WelcomeWelcome
Pl data science october 2017
Pl data science october 2017Pl data science october 2017
Pl data science october 2017
Data Science Leuven
 
Leuven city 21092017
Leuven city 21092017Leuven city 21092017
Leuven city 21092017
Data Science Leuven
 

More from Data Science Leuven (14)

Computing power and algorithms? In people we trust
Computing power and algorithms? In people we trustComputing power and algorithms? In people we trust
Computing power and algorithms? In people we trust
 
Trumania , a realistic scenario-based data-generator
Trumania , a realistic scenario-based data-generatorTrumania , a realistic scenario-based data-generator
Trumania , a realistic scenario-based data-generator
 
Recommender systems, optimizing least squares or user experience
Recommender systems, optimizing least squares or user experienceRecommender systems, optimizing least squares or user experience
Recommender systems, optimizing least squares or user experience
 
Replicability and questionable research practices
Replicability and questionable research practicesReplicability and questionable research practices
Replicability and questionable research practices
 
Predicting Eurosong with Google Predicting Eurosong with Google and data visu...
Predicting Eurosong with Google Predicting Eurosong with Google and data visu...Predicting Eurosong with Google Predicting Eurosong with Google and data visu...
Predicting Eurosong with Google Predicting Eurosong with Google and data visu...
 
Storytelling for impactful predictive models - Gert De Geyter
Storytelling for impactful predictive models - Gert De GeyterStorytelling for impactful predictive models - Gert De Geyter
Storytelling for impactful predictive models - Gert De Geyter
 
Lessons from driving analytics projects
Lessons from driving analytics projectsLessons from driving analytics projects
Lessons from driving analytics projects
 
Geospatial visual analytics
Geospatial visual analyticsGeospatial visual analytics
Geospatial visual analytics
 
Open-Source Data Science Crossing The Chasm
Open-Source Data Science Crossing The ChasmOpen-Source Data Science Crossing The Chasm
Open-Source Data Science Crossing The Chasm
 
Probabilistic machine learning for optimization and solving complex
Probabilistic machine learning for optimization and solving complexProbabilistic machine learning for optimization and solving complex
Probabilistic machine learning for optimization and solving complex
 
Closing
ClosingClosing
Closing
 
Welcome
WelcomeWelcome
Welcome
 
Pl data science october 2017
Pl data science october 2017Pl data science october 2017
Pl data science october 2017
 
Leuven city 21092017
Leuven city 21092017Leuven city 21092017
Leuven city 21092017
 

Recently uploaded

Tabula.io Cheatsheet: automate your data workflows
Tabula.io Cheatsheet: automate your data workflowsTabula.io Cheatsheet: automate your data workflows
Tabula.io Cheatsheet: automate your data workflows
alex933524
 
一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单
enxupq
 
Predicting Product Ad Campaign Performance: A Data Analysis Project Presentation
Predicting Product Ad Campaign Performance: A Data Analysis Project PresentationPredicting Product Ad Campaign Performance: A Data Analysis Project Presentation
Predicting Product Ad Campaign Performance: A Data Analysis Project Presentation
Boston Institute of Analytics
 
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
axoqas
 
Adjusting primitives for graph : SHORT REPORT / NOTES
Adjusting primitives for graph : SHORT REPORT / NOTESAdjusting primitives for graph : SHORT REPORT / NOTES
Adjusting primitives for graph : SHORT REPORT / NOTES
Subhajit Sahu
 
一比一原版(TWU毕业证)西三一大学毕业证成绩单
一比一原版(TWU毕业证)西三一大学毕业证成绩单一比一原版(TWU毕业证)西三一大学毕业证成绩单
一比一原版(TWU毕业证)西三一大学毕业证成绩单
ocavb
 
tapal brand analysis PPT slide for comptetive data
tapal brand analysis PPT slide for comptetive datatapal brand analysis PPT slide for comptetive data
tapal brand analysis PPT slide for comptetive data
theahmadsaood
 
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
axoqas
 
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
yhkoc
 
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
vcaxypu
 
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Subhajit Sahu
 
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Subhajit Sahu
 
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
ukgaet
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
AbhimanyuSinha9
 
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
nscud
 
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
ewymefz
 
Business update Q1 2024 Lar España Real Estate SOCIMI
Business update Q1 2024 Lar España Real Estate SOCIMIBusiness update Q1 2024 Lar España Real Estate SOCIMI
Business update Q1 2024 Lar España Real Estate SOCIMI
AlejandraGmez176757
 
Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)
TravisMalana
 
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
ewymefz
 
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
ewymefz
 

Recently uploaded (20)

Tabula.io Cheatsheet: automate your data workflows
Tabula.io Cheatsheet: automate your data workflowsTabula.io Cheatsheet: automate your data workflows
Tabula.io Cheatsheet: automate your data workflows
 
一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单
 
Predicting Product Ad Campaign Performance: A Data Analysis Project Presentation
Predicting Product Ad Campaign Performance: A Data Analysis Project PresentationPredicting Product Ad Campaign Performance: A Data Analysis Project Presentation
Predicting Product Ad Campaign Performance: A Data Analysis Project Presentation
 
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
 
Adjusting primitives for graph : SHORT REPORT / NOTES
Adjusting primitives for graph : SHORT REPORT / NOTESAdjusting primitives for graph : SHORT REPORT / NOTES
Adjusting primitives for graph : SHORT REPORT / NOTES
 
一比一原版(TWU毕业证)西三一大学毕业证成绩单
一比一原版(TWU毕业证)西三一大学毕业证成绩单一比一原版(TWU毕业证)西三一大学毕业证成绩单
一比一原版(TWU毕业证)西三一大学毕业证成绩单
 
tapal brand analysis PPT slide for comptetive data
tapal brand analysis PPT slide for comptetive datatapal brand analysis PPT slide for comptetive data
tapal brand analysis PPT slide for comptetive data
 
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
做(mqu毕业证书)麦考瑞大学毕业证硕士文凭证书学费发票原版一模一样
 
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
 
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
一比一原版(ArtEZ毕业证)ArtEZ艺术学院毕业证成绩单
 
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
 
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...
 
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
一比一原版(UVic毕业证)维多利亚大学毕业证成绩单
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
 
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
 
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
一比一原版(UofM毕业证)明尼苏达大学毕业证成绩单
 
Business update Q1 2024 Lar España Real Estate SOCIMI
Business update Q1 2024 Lar España Real Estate SOCIMIBusiness update Q1 2024 Lar España Real Estate SOCIMI
Business update Q1 2024 Lar España Real Estate SOCIMI
 
Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)Malana- Gimlet Market Analysis (Portfolio 2)
Malana- Gimlet Market Analysis (Portfolio 2)
 
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
 
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
一比一原版(UMich毕业证)密歇根大学|安娜堡分校毕业证成绩单
 

How to build a search engine in 2 days