Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Beyond search
queries
Michal Barla
searchd.co
About me
● researcher and teacher at
Slovak University of Technology in
Bratislava
● developer @ synopsi.tv, searchd.co
● ...
Search
as seen by developers
{
"query": {
"query_string": {
"query": "elasticsearch book"
}
}
}
return response.hits.hits
Search
as experienced by users
query: elasticsarch
Typo in query.
No results.
query: elasticsearch
Too many hits.
Not rele...
Measuring
search quality
Cpt. Obvious:
“Hits, clicks and order
do matter.”
Accurately interpreting clickthrough
data as implicit feedback
Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke...
Accurately interpreting clickthrough
data as implicit feedback
Search quality metrics
● Mean Average Precision @ N
○ probability of target result being in top N items
● Mean Reciprocal ...
Search KPIs
● CTR trend
● # of queries w/o results or clicks
● # of searches per session
● Search engine latency
Search quality
optimization
Optimizing search engines using
clickthrough data
Thorsten Joachims. Optimizing search engines using clickthrough data. In...
Optimizing search engines using
clickthrough data
Query chains: learning to rank from
implicit feedback
Filip Radlinski and Thorsten
Joachims. Query chains: learning
to ran...
Fighting Search Engine Amnesia:
Reranking Repeated Results
Milad Shokouhi, Ryen W. White, Paul Bennett, and Filip Radlinsk...
searchd.co
Search Analytics
searchd.co dashboard
searchd.co
Search Analytics
● Identify and fix key search problems
● KPIs for site search
● Actionable tips for search tun...
Bad search experience is a lost
opportunity. Let's fix it.
searchd.co
Search Analytics
www.searchd.co
info@searchd.co
Michal Barla: Beyond search queries @ ElasticSearch Vienna Meetup #1
Michal Barla: Beyond search queries @ ElasticSearch Vienna Meetup #1
Michal Barla: Beyond search queries @ ElasticSearch Vienna Meetup #1
Upcoming SlideShare
Loading in …5
×

Michal Barla: Beyond search queries @ ElasticSearch Vienna Meetup #1

911 views

Published on

How do users interact with search engines? What can we learn from this behavior? How can we make search engines better? How do we measure quality of search results and what are the key metrics? Do you even measure the quality of your search? Let's take a walk standing on the shoulders of giants like Google, Yahoo or Yandex and learn about the recent advances in search research.

Published in: Technology, Design
  • Be the first to comment

Michal Barla: Beyond search queries @ ElasticSearch Vienna Meetup #1

  1. 1. Beyond search queries Michal Barla searchd.co
  2. 2. About me ● researcher and teacher at Slovak University of Technology in Bratislava ● developer @ synopsi.tv, searchd.co ● co-owner of minio, s.r.o. ○ otvorenezmluvy.sk, govdata.sk
  3. 3. Search as seen by developers { "query": { "query_string": { "query": "elasticsearch book" } } } return response.hits.hits
  4. 4. Search as experienced by users query: elasticsarch Typo in query. No results. query: elasticsearch Too many hits. Not relevant. query: elasticsearch book Click! Success! Or?
  5. 5. Measuring search quality
  6. 6. Cpt. Obvious: “Hits, clicks and order do matter.”
  7. 7. Accurately interpreting clickthrough data as implicit feedback Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in Information retrieval, SIGIR ’05, pages 154–161, New York, NY, USA, 2005. ACM.
  8. 8. Accurately interpreting clickthrough data as implicit feedback
  9. 9. Search quality metrics ● Mean Average Precision @ N ○ probability of target result being in top N items ● Mean Reciprocal Rank ○ 1 / rank of target result ● Normalized Discounted Cumulative Gain ● Expected Reciprocal Rank
  10. 10. Search KPIs ● CTR trend ● # of queries w/o results or clicks ● # of searches per session ● Search engine latency
  11. 11. Search quality optimization
  12. 12. Optimizing search engines using clickthrough data Thorsten Joachims. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, KDD ’02, pages 133–142, New York, NY, USA, 2002. ACM.
  13. 13. Optimizing search engines using clickthrough data
  14. 14. Query chains: learning to rank from implicit feedback Filip Radlinski and Thorsten Joachims. Query chains: learning to rank from implicit feedback. In KDD ’05: Proceeding of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, pages 239–248, New York, NY, USA, 2005. ACM.
  15. 15. Fighting Search Engine Amnesia: Reranking Repeated Results Milad Shokouhi, Ryen W. White, Paul Bennett, and Filip Radlinski. Fighting search engine amnesia: reranking repeated results. In Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval, SIGIR ’13, pages 273–282, New York, NY, USA, 2013. ACM. In this paper, we observed that the same results are often shown to users multiple times during search sessions. We showed that there are a number of effects at play, which can be leveraged to improve information retrieval performance. In particular, previously skipped results are much less likely to be clicked, and previously clicked results may or may not be re-clicked depending on other factors of the session.
  16. 16. searchd.co Search Analytics
  17. 17. searchd.co dashboard
  18. 18. searchd.co Search Analytics ● Identify and fix key search problems ● KPIs for site search ● Actionable tips for search tuning ● Easy setup a. Add our hosted JavaScript b. Annotate search results with HTML5 tags c. Done. ● Currently in private beta
  19. 19. Bad search experience is a lost opportunity. Let's fix it. searchd.co Search Analytics www.searchd.co info@searchd.co

×