This document discusses improving site search through search analytics. It begins by outlining the basic steps to be as good as Google for site search: index content, design a search form, and show the best results first. It then discusses analyzing search query logs from a site to understand user behavior and improve search performance, content, navigation, and user experience. Specific techniques recommended include analyzing the most common queries, randomly selecting "long tail" queries, measuring relevance of results, removing irrelevant results from the index, using metadata and synonyms, adjusting the ranking algorithm, and boosting or reducing the score of individual results.

1. Improving site search
with search analytics
Audun Rundberg
June 22nd, 2013

2. #uxce13
@audunru

6. Be just as good as
Google - in 3 small steps:
1. Index everything you want to search
2. Design a search form:
3. Show the best results first when people search on your site
Search!

7. Is it that
simple?

8. Photo: Google

9. Photo: Google

10. Photo: Google

11. Source: http://en.wikipedia.org/wiki/PageRank

12. B
38,4%
C
34,3%
F
3,9%
E
8,1%
D
3,9%
A
3,3%
1,6% 1,6% 1,6%
1,6%
1,6%
Source: http://en.wikipedia.org/wiki/PageRank

13. Content
Links
Domain history
User behaviour
Social media sharing

15. Google vs You
Google
• Billions of documents
• User behaviour data from
hundreds of millions of users
• 50.000+ employees
Your search engine
• 100-100.000 documents
• Probably not used very much
• 0-1 employees

16. Big country
Big websites

17. Bigger country
Even
bigger
websites

18. #1 Check your
search results

19. your site, they’re telling you in their own words
m your organization. In Search Analytics for Your Site,
ws you how to harvest and analyze search query
ur site’s search performance, content, navigation,
ness, and overall user experience.
dback will improve any site’s user experience, why aren’t we spending
ords our audience uses when asking us for stuff? I can’t imagine a more
Rosenfeld to help us put this amazing data to work.”
te lays out pretty much everything you need to know to mine your query
sitive customer experience.”
Demystified
lytics for Your Site is a superlative work from the initial story to the final
lytics and UX practice. Great book!”
oks that makes me pound the table with my fist and yell, ‘Yes! Exactly!
g it.”
nt Strategy for the Web
OR YOUR SITE
ks/searchanalytics/
SE ARCH ANALY TICS FOR YOUR SITE
Conversations with Your Customers
by LOUIS ROSENFELD
forewords by Steve Krug and Avinash Kaushik
SEARCHANALYTICSFORYOURSITEbyLOUISROSENFELD

20. Most
common
queries

21. 0
62,5
125
187,5
250
1 9 17 25 33 41 49 57 65 73 81 89 97 105 113 121 129 137 145 153 161 169 177 185 193 201 209 217 225 233 241 249
Top 50 queries: 10% of searches
50 of 23,252 total queries: 0,2%

23. PPM

24. PPM
5. place... Good enough?

25. Query Best result
vacancies 1
sia 2
ppm 4
plane tickets 24
travel 3
Average 6,8

26. Old – Jan 31 New – Jan 31 New – Feb 22 New – Apr 17
0
2,5
5
7,5
10
4,2
9,3
1,74
1,27
Average position of best result

27. Most common queries
1. Find the 50 most used queries in your search statistics, or as
many as you have time for
2. Perform a search for each query
3. Decide which result is the best
4. Write down the position of the best result
5. Calculate the average
6. Repeat from time to time

28. Google Docs spreadsheet
http://bit.ly/uxce2013-search

29. Long tail
queries

31. Take 100 randomly
selected queries

32. «Long tail» queries
1. Take 100 randomly selected queries from the «long tail»
2. Categorise them
3. Perform a search for each query
4. Decide which result is the best
5. Write down the position of the best result
6. Calculate the average
7. Repeat from time to time

33. Relevance

34. How to measure
1. Find the 50 most used queries in your search statistics, or as
many as you have time for
2. Perform a search for each query
3. Rate the five first results from A to C
A. Perfect result
B. Not perfect, but the user will get to the right page
C. Irrelevant

35. Relevance
Query Result #1 Result #2 Result #3 Result #4 Result #5
Relevance
(strict)
Relevance
(less strict)
mesys A B B B B 20 % 100 %
glasses A B C C C 20 % 40 %
cell phone A C B B B 20 % 80 %
Average 20 % 73 %

36. #2 Improve your search
results

38. Photo: flickr.com/kendo26

39. «hytte»
«company cabin»
Result 1 - 66.762 points:
Date: 923 points
IA: 60.000 points
Word matches: 66 points
Title matches etc.: 5773 points
Result 2 - 62.637 points:
Date: 946 points
IA: 56.000 points
Word matches: 31 points
Title matches etc.: 5660 points

40. Remove
irrelevant
results
• Front page
• Admin pages
• Different views of the same
page (e.g. screen and print
versions)
• Duplicates

41. Metadata
matches
• Results that match the user
profile property «Location» get
a higher score
• Also good for filters

42. Rewrite
content
• «training» instead of
«competence-increasing
measures»

43. Use
synonyms
• Create a company-specific list
of synonyms that your search
engine doesn’t already know
about

44. Adjust the
algorithm
• Title match is more important
than a text match

45. Boost
individual
results
• Quick solution if a common
search query gives bad results
• You need a plan for how to
review manual changes

46. Give
unimportant
content a
negative
boost
• Older news articles or archived
content receives a negative
boost

47. Buy a new and more
expensive search
engine that will solve all
your problems
Let me know if you find one!

48. Thank you!