This paper is about the challenge of using analytics to drive your findability strategy and measure how well it’s achieving on its objectives.

1. Site Search Analytics 1
Site Search Analytics 1
Site Search Analytics
White paper by SearchBroker -­‐ Colbenson
John Tomlinson
This paper is about the challenge of using analytics to drive your findability strategy and
measure how well it’s achieving on its objectives.

2. Contents
Introduction ....................................................................................................................................................................3
Putting the customer first ..........................................................................................................................................3
The long neck or the long tail? ..................................................................................................................................5
Search analytics to drive SEO........................................................................................................................................5
Site search is different ...................................................................................................................................................5
Findability: the foundation of conversion..............................................................................................................6
Findability..........................................................................................................................................................................6
Search analytics: the long neck .................................................................................................................................9
Content or search terms?...............................................................................................................................................9
Calculating opportunity ...............................................................................................................................................10
The findability ceiling ...................................................................................................................................................12
Search analytics: the long tail..................................................................................................................................14
Step one: create clusters .............................................................................................................................................14
Step two: low findability analysis...............................................................................................................................14
Step three: take action.................................................................................................................................................14
Low findability due to no content..............................................................................................................................14
Low findability due to non-indexed content..........................................................................................................15
Low findability due to customer behaviour...........................................................................................................15
Low findability for other reasons............................................................................................................................15
Step four: don't stop.....................................................................................................................................................16
Conclusion......................................................................................................................................................................17

3. 3
Introduction
People have one thing in common: they are all
unique.
This means that people will search in lots of
different ways, and your onsite search engine
must be able to deal with it.
This paper is about that challenge. The challenge
of using analytics to drive your findability strategy
and measure how well it’s achieving on its
objectives.
Putting the customer first
The proper place to start is with customer-focussed search analytics.
This means going through all the words (terms) and phrases (queries) and working out what people are
talking about.
Putting each terms or query into a graph, with quantity of searches on the vertical y-axis and each search
term (in quantity order) on the horizontal x-axis, you get something that looks a bit like this:

4. Roughly speaking, about 80% of your searches will fall into the top 20% of terms (we call this the long
neck), and then the other 80% of terms will be spread across about 20% of your traffic (the long tail).
This "long tail" is a broad range of search queries that tend to be a mix of words and phrases that will
usually have poor findability rates (ie. they don't connect the customer to any site content).
The tactics for dealing wth the long neck are different
from those of the long tail – but which is more
important? Where should you start?

5. 5
The long neck or the long tail?
Search analytics advice is usually focussed on the importance of the "long tail" in driving conversion.
Whilst the "long tail" is important, please be aware, a lot of this advice is focussed on Search Engine
Optimisation (SEO) and not on internal site search.
Search analytics to drive SEO
The argument runs that using high volume queries like "TV" is less effective in driving conversion than a
much more specific - but much less common - query like "LCD smart HD TV Samsung" for example.
This makes sense, the more specific the search is, the more likely it is that the customer will connect to
the data they are actually looking for, and the more likely it is that the customer is clear about what they
want and serious about finding it.
Many of us search this way. We try a fairly general query then, when faced with too much generic
content, we refine the queries getting more and more specific until we find links that connect us directly
to the content we're looking for.
The challenge of SEO is to get seen in external search engines like Google. In this case, the long-tail is a
great source of data to help you focus your advertising, make use of key words that convert, drive a
content marketing strategy, and other useful techniques for attracting valuable traffic.
Site search is different
It is not the same challenge in site search. In site search the customer is already on the site, so the same
argument simply doesn't apply and a different approach is needed.
In site search, the challenge is to ensure that the customer makes the right connections to the right
data.
We may see a "long tail" query like "Samsung HD Smart LCD TV" with findability at zero, but this is much
less likely. If this is a product (or product type) you stock, then if it is not connecting it is probably due to
data quality or design issues rather than search. If you stock other types of HD TVs, a good search engine
- even a poor search engine - should make that connection!
The long-tail in site search is often more idiosyncratic: variations on terms, common mistakes or off-
catalogue items that you haven't indexed.

6. This is why search analytics always starts with the customer.
It is vital to use the customer's own language to drive how the search and category navigation functions
deliver. If you don't, you'll be building connections from a business perspective, not from a customer
perspective.
This is why the head of the graph, the long neck that are about 80% of the traffic, becomes hugely
important: that's where the customers are and in site search analytics, that’s where you need to start!
Findability: the foundation of conversion
Before discussing the specific approaches you should take when dealing with the long neck and long tail
of site search, we need to define the term “findability”.
Findability is mapped on the vertical y-axis of the graph where each term or query has a findability value
associated with it. This is driving the analytics process, and so we must understand the term to be sure
we are building our methodology on solid foundations.
Findability
The term "findability" is a cumbersome and inelegant word, but one that captures an important concept.
The first formal definition is often credited to Peter Morville:
"the ability of users to identify an appropriate Web site and navigate the pages of the site to
discover and retrieve relevant information resources"
(Peter Morville, 2005
1
)
He wasn't the first to use the term, but his definition is most widely accepted.
This definition includes two ideas:
• The ability of the website and its content to be found by customers searching externally.
• The ability of the content of the website to be found by customers internally (i.e. already on the
site).
1
Source: http://findability.org/

7. 7
The first part of the definition is now the field known as SEO: (Search Engine Optimization). This is about
how to ensure your site ranks in Google and other major search engines, and has become a vital tool for
online organisations and a complex science with its own devoted experts.
The second part of the definition is the focus of this paper. Can people who are already on your site
actually find what they're looking for?
This is increasingly important as big data becomes ever more commonplace, and especially so in e-
commerce. The ability of your e-commerce customers to find your products is the different between
making a sale and losing a customer.
This is why we at Colbenson define "findability" as:
"the measure for how easily your customers can make the right connections to your data"
This is findability, and it is the fundamental component of online conversion. Can your customers
connect to your data correctly - if not, they're not going to buy anything.
This also goes beyond site search.
Findability is not just about how onsite search works, it applies generally to how your customers connect
to your data. This could be via promotional advertising, category navigation, social media or any other
channel.
How customers make these connections will vary from sector to sector and from business to business.

8. This is why “findability” is such a useful concept, it doesn't focus on the tool (e.g. the onsite search
engine), it focuses on the outcome: customers making the right connections to your data, or, to put it
another way, it focuses on conversion.
Reducing Site Search abandons increases Conversion

9. 9
Search analytics: the long neck
In site search, 80% of your customers are at the long neck; that's where you need to be!
The great thing about long neck search terms is that they are predictable. It doesn't take too much
effort to scan through the top bundle of search terms used, see what's happening, and create a plan for
the kind of connections you want to see made between customers using those terms and your site
content.
The best way to convert the long neck is by creating promotional links to either successful searches or
high-performing content. Or both.
But what links should we create?
The best way to decide is to let the customer decide by using A/B Testing on various different solutions.
A/B Testing
A/B Testing involves trying two or more options and randomly applying these to your customers at
the same times of day. You can then compare which is the most successful at building connections.
Content or search terms?
A search engine like Google will link you to popular search terms by suggesting popular searches as you
type. This is called Google Suggest. In this case, the search is still performed and a page of results shown.
The customer needs to browse these results and select the content they want.
This is great when there is no obvious standout link between the term and the content.

10. If there is a strong relationship between a search term and site content, you can bypass the search and
results process and directly suggest that content to the customer is the autocomplete space. This means
the customer clicks and is taken directly to that content.
A great example of this is Apple:
Be careful with this, you can overdo it, and end up thinking all of your customers want to make the same
connections to the same content. You must always allow the customer to feel in control of the search,
and allow them to make the connections they want to make.
Calculating opportunity
A useful way to measure this is to consider the concept of “opportunity”. This process allows you to
prioritise your actions when managing site search.
Opportunity
Opportunity is the measure of what you’re not converting – how many opportunities to sell are you
missing because your search is not making the right connections.
Sticking with the TV example from earlier in this paper, if we have a "long-tail" item like "telvision" that
has a findability of zero, and a frequency of 5, then we are potentially missing out on 5 connections and
therefore have 5 opportunites.

11. 11
At the same time, if a thousand people are searching "TV" and this has a findability of 20%, then that's
800 people (1000 x 80%) not making connections to your products.
Looked at this way, you have a much greater opportunity to make an impact on your overall conversion
rates if you work out why people are not finding what they want when they search "TV" than connecting
"telvision" that had only 5 incidences.
The calculation for working out the number of opportunities (o) each search query presents us with is
simple: the number of incidences (n) multiplied by the percentage not found (1-f%):
o = n x (1-f%)
There is a problem with this method.
We are assuming that it is possible to get a findability percentage (f%) of 100%.
Sadly, we must reluctantly accept that there is a ceiling somewhat lower than perfection for most
queries. Some individual queries with very low frequency (usually pointing at specific content) may have
unusually high findability, but this anayisis uses he more numerous terms found in the "head" of the
graph.
This will vary from industry to industry, and from sector to
sector, but it is rare to see 100% findability - especially if your
results page (or autocomplete) has so much information that
customers need not click on anything to get the information!

12. The findability ceiling
To estimate your findability ceiling, first isolate your top search queries. These should be the top ten (or
so) that have the best findability percentage, but that have a similar success rate. It needn't be exactly
ten queries, but you should be able to see a number of queries that cluster at the top with similar
percentages.
Now you need to drill down on the part that is not converting.
Staying customer-focussed, there are only two things that
can be happening if the customer is not connecting to
content. Either they are (a) giving up and abandoning the
search, or (b) they try again and search with new terms or
use filters to refine the results.
This is shown in this diagram. The green channel shows
conversion, the red is abandonment and the yellow is
customers trying again.
Let us assume that our top performing search queries
have an average findability of 40%, and that of the 60%
that's not connecting, it is split equally into 30%
abandoning the search and 30% refining or redoing.
We can now make an assumption that we should be able
to see a 30% improvement in (b) and a 10% improvement
in (a).
The difference is because redos usually signal that the person is not finding what they want to find (i.e. a
search problem), whereas abandonments may signal the customer being put off by a non-search-related
reason such as price, terms and conditions, payment methods, poor reviews, lack of information, or
simply poor page design.
So, adding this up we get the original 40%, plus 10% of abandonments (3%) plus 30% of redos (9%) and
a ceiling target to aim for of 52%. This is much more realistic than 100%. For those who like equations:
o = n x (fc%-f%)
(The new term is fc% which is the percentage of the findability ceiling).

13. 13
The queries should then be listed in order of opportunity, giving us a quick and simple priority list for
actions we need to take to drive up conversion.
In this way we are able to focus our efforts in the places where we will have the most impact.

14.
Search analytics: the long tail
Although it makes sense to start with the long neck, it is still useful to look at the "long tail”.
This contrasts with the long tail where the search terms are more numerous and less predictable. For the
long tail we need the search engine to be really clever at ranking results organically - but in the long
neck, because we're working with fewer terms and we know what they are, we can take control and
create the links we want.
There is a four-step process for dealing with long tail analytics.
Step one: create clusters
The first step is to examine the data, and where possible, create clusters of terms. If, for example, the
are a lot of terms like "postage", "delivery", "shipping", "packaging" or even errors such as "pakaging"
and "dliver" which are recognisable, then you can see there is demand for content about how products
are delivered.
Step two: low findability analysis
There are only four reasons why findability is zero or very low:
1. You don't have content on your site that relates to what they are looking for.
2. You do have content, but didn't index it so it cannot be found with the search engine.
3. The customer misspelled, mistyped or described the content in a way that wasn't recognised
4. None of the above. For some other reason they did not choose to click on any result that was
offered.
For each cluster created in Step One (or each term where no clusters could be created), work out what
the reason for the low score is.
Step three: take action
Each of the above four reasons requires a different approach.
Low findability due to no content
You have three choices:
1. Create content that includes these terms. This can mean adding a blog post, an FAQ, a technical
description ... it depends on the nature of the query which type of content makes sense.

15. 15
2. Stock the item. If the customer is searching for an item you don't stock, this a great view into
potential demand. If it makes no sense to stock it, you may still be able to create content that
may be of interest and may keep the customer on the site, or you can link the term to a similar
item - perhaps via the "zero results page" ("We don't stock X, but we do have Y").
3. Don't worry about it. Sometimes people will search for things that just don't make any sense ...
but beware, sometimes what might seem to make no sense to you, does make sense to the
customer. Why would people suddenly search for Brad Pitt on the Chanel site? Or search for
Kate Moss on Mango’s site? Before dismissing the search term, ensure you're aware of all
promotional activity your organisation is involved in - there may be a connection there and the
right move is to create some content or direct the search to the relevant product.
Low findability due to non-­‐indexed content
This one is fairly obvious. All site content should be indexed, not just your product catalogue. Often
customers will search for store location, returns policies, how to make complaints, how the delivery
process works and other off-catalogue information. This should be there and should be indexed.
Low findability due to customer behaviour
This data is a gold mine of common mistakes that your customers will make and allows you to be
proactive about anticipating future errors. If they are often misspelling a word, you will find it here. If
they don't understand how your product is categorized, you'll find their version of the truth represented
in the words and phrases in this data.
Where it is possible to link these words and phrases to content, you should do so by including them in
product descriptions, keywords or synonyms.
The most important action you can take here is to ensure that your autocomplete function offers search
suggestions and content that avoid mistakes in the first place.
An alternative is to link them to similar words or phrases that have good rates of conversion. This is
popular with Google where they will use "Did you mean X?" and perform the search on X which they
know has some good results. If you do this, you must be clear that the results relate to X, and not what
the customer searched for.
A last point on this. Don't assume these are all customer errors, they could just as easily be your errors.
Challenge your own category structures, product names and descriptions: are you the one who is making
the mistakes?
Low findability for other reasons
This is a very broad category, and could include many factors, not all of which may be negative.

16. Here are ten of the more common reasons:
• Poor results page design.
• Lack of, or poorly designed, autocomplete function.
• Poor zero results page.
• Poor data quality - the right content is not being offered.
• Lack of, or poorly designed, filters.
• Ineffective dynamic ranking of results.
• Poor or confusing product descriptions so that the customer cannot correctly recognise the
product.
• The lack of appropriate use of images and other visuals.
• So much information shown in the results page or autocomplete that it becomes unnecessary
for the customer to click on anything (not necessarily a bad thing!)
• Non-search related reasons such as wrong pricing, confusing descriptions, generally poor design
destroying customer confidence.
Step four: don't stop
This process is not a straight line, it's a circle. Once these actions have been taken, the process starts
again.

17. 17
Conclusion
A good search engine succeeds because it is part of a customer-centric findability strategy. It is focussed
on helping your customers connect with your content, and measured by how well it achieves this
objective.
The process for acting on these analytics involves taking the long neck and the long tail of search terms
separately. Each is then dealt with differently, and always with the objective of driving up findability and
so increasing the connections your customers make to your content.
This is the foundation of conversion. If your customers can’t connect to the right content, they can’t buy
it.
15 Queen Square, LEEDS, LS2 8AJ United Kingdom
0113 335 0791 / 08454 757 973
http://www.colbenson.com