Search analytics comperio

1. OSLO STOCKHOLM LONDON BOSTON SINGAPORE
Search Analytics
Comperio - Seminar on Searchdriven
Websites and Analytics of Searchlogs
Stockholm Digital Days 2013-05-22
Bo Engren

2. Agenda
• What is Search(log) Analytics?
• Improving Search
• Best Practices & Administration
• QA

3. Web Analytics vs. Search Analytics
The difference between Web
Analytics and Search Analytics is
that Web shows what the users
actually have been doing, Search
shows their intent.
(and btw Search Analytics isn’t SEO either)

4. The challenges with Search
I can’t find
what I’m
looking for
Content
is old
Duplicates
and
versions
Not
maintained
Too many
choices
Language
and
domain
vocabulary
Poor user
experience
etc…

5. The relevancy threshold
By raising the
relevance with
40%, we can
move the
search solution
from low to
high trust.

6. Tuning relevancy - toolboxes

7. The search team

8. Best Practices & Administration

9. Operational steps for good search
DEFINE
SCOPE
IMPLEMENT
RELEASE
MAINTAIN
Understand business
needs
• Understand what you are
trying to achieve
• Plan and define goals
• Identify good trends, ROI
Measure and refine
• Monitor and use query
information
• Mine query logs
• Measure effectiveness
of search towards a
target
Output and benefits
• Better search
• Better results
• Enhanced usability
• Enhanced revenues
Search
customer

10. Analyzing search logs – fundamentals
When you have defined your business needs
Monitor your search logs...
...again and again and again
Look for
• Specific queries
• General queries
• Queries with zero results
• Filter away junk!

11. Know your search distribution
350 10.0000
0
500
20%
80%
Similar
searches
Unique
searches
Frequency
Query term
Can we find
patterns in
this type of
searches?
Take good
care of your
top queries

12. Frequent queries
Visualized Search history. Most frequent query terms

13. Unique queries example
A lot of
product code
searches

14. Sample Query report

15. Zero Result Queries
9.95% of today’s queries
return no results
Create a synonym for the query
Select time period

16. Empty result sets
How do we fix empty result sets?
• Investigate why!
– Spelling errors?
– Semantics?
– UI difficulties?
• Correct the underlying causes

17. Create Synonyms

18. Top/Frequent queries
How do we serve frequent queries best?
• Ensure good relevance
• Apply best bets
• If ambient, present options to narrow results
• If specific, make sure user get to the goal

19. Content Search - Refiners
• Filters are based on words in documents
• Words are used to tag the document with predefined set of Filter
names
Result Refiners
Enables filtering

20. Boosts and Blocks
• Boosting is the process of changing the
“natural” rank to alter the position of a document
within the result set

21. Apply selected Linguistic Features
• Automatic language detection
• Approximate matching (spell checking) “cort”, “court”
• Lemmatization Noun: “car”  “cars”
Verb: “break”  “break”, “breaks”, “broke”
•
• Synonyms “color” = “colour”
“car” = “automobile”
• Proper Name and Phrasing /Spellcheck “Venus Williams”, “French Open”
• Anti-phrasing (Stopwords) “[I want a] Nikon camera”
• Character Normalization “Molière -> Moliere”
• Tokenization (CJK support) “market-shares” -> “market shares”
• Phonetic Search “Eyvind”, “Oyvind” -> “Eyvind”
• Automatic spelltuning Based on index contents
When implemented properly can drastically improve the
usefulness of a search

22. Search statistics – several tools available
• Start with the searchlogs:
– Use the built in tools
– Loggparsers (IIS loggparser etc.)
– Webanalytics tools (Google Analytics,
Webtrends etc.)
– Log management (logstash, kibana)
– Big data (Hadoop, pig)

23. Visual searchresults
Comperio internal Knowledge Management DB February 2013

24. Statistic analysis – Best Practice
• Zero hit results  key to monitor and remove
• Analyze the Top queries
• Trends over time – group by day/week/month
• Separate internal and external searches
• Group the queries for better understanding (for
example products, documents, persons)

25. Examples of Metrics for Search
Analytics – select a few initally
Search perspective
Measures Definition
Metric
type
Total queries Total number of search queries #
Clicks Total number of clicks that goes from search results to final file or page #
Satisfied queries Percentage of search results with at least one click %
Opportunity queries Percentage of search results with no click %
Visits with keyword searches Percentage of web visitors that use search %
Visits with guided product search Percentage of web visitors that use guided product search %
Visits with browsing searches Percentage of web visitors that use browseing searches e.g. listings %
Search result exits Percentage of web visitors that exit the website on the search result page %
Searches with zero results Percentage of searches that end up with zero results %
Search depth Depth after search result page #
Refined searches Number of searches refined with new query text after result view #
Result relevancy Relevancy of search results, based on recall/precision test model and test set #
Query suggestion use Number of searches performed with suggested queries #
Related queries Number of searches with related queries used #
Filtered queries Number of searches with query refinement filters #
Time to destination Time spent from search to final result Time
Result sidebar use Percentage of clicks on sidebar results on result page views %
Advanced queries Number of advanced queries performed with boolean or filter operators #
Best bets use Percentage of clicks on manual top results when displayed %

26. Improve results of searches - Best
Practice
Improve similar searches (fat head)
• Autocomplete
• Best bets
Improve uniqe searches (long tail)
• Spellchecking
• Synonyms
• Adjust your content

27. Internal searches – do we understand
the context of the user?
• Start with the User
– Study/test your User Stories.
Example: You are going to start a new project.
Do you find what you need to get started?
– Use Online surverys for deeper insights

28. All search platforms need maintenance
• A team that specializes in search
and related technologies
– Front end search specialists
– Search analysts
• Examples of Tasks
– Sounding board for proposed projects or reported
problems
– Cataloguing agreed search best practice
– Control vocabularies and taxonomies
– Monitoring and tuning
– In-house training

29. Search Analytics – Summary 1
• Make someone responsible for search - Appoint a
Search Manager
• Set a search strategy which enables the business
strategy and is in line with overall IT-strategy
• Make the Business Case
• Measure and Monitor Search Queries = Search
Analytics
• Enable User Feedback
• Raise quality of information by adding metadata and
doing content lifecycle management
• Add metadata - manual, mandatory or automatic?

30. Search Analytics - Summary 2
• Establish processes to deliver feedback to your
Stakeholders regarding the search logs
– Separate External and Internal sites?
• Educate information creators - simple handouts and
sit-downs
• Apply spelling suggestions, key-matches and auto-
complete
• What can we do as Editors and what do we need
Techies to do?
– You can do more than you think!

31. Thanks for listening
and time for QA!