Case Study BBCi Search Why
search is not a technology problem

me. <ul><li>What do I do?
</li></ul><ul><ul><li>I work for the BBC </li></ul></ul><ul><ul><ul><li>Q -What’s your perception of what the BBC do? </li></ul></ul></ul><ul><ul><li>I work in the New Media dept. </li></ul></ul><ul><ul><li>Information Architect? </li></ul></ul><ul><ul><li>Interference Architect! </li></ul></ul><ul><li>Previously: </li></ul><ul><ul><li>Creative director at Sapient London </li></ul></ul><ul><ul><li>Creative director / launch designer of BBC News Online, London Times and Sunday Times </li></ul></ul><ul><li>Even further back </li></ul><ul><ul><li>Wasn’t an IA – I was just an ‘A’ </li></ul></ul><ul><li>What am I going to talk about? </li></ul><ul><ul><li>Why search is not a technology problem. </li></ul></ul>

Some context

Search is strategic to us.
business imperative

Henley centre study: online life

Project context The “BBCi” initiative

Pre-BBCi – “geocities”

User perception

BBCi perceptual model aimed for

technology /content context <ul><li>Existing, disparate
search engines </li></ul><ul><ul><li>Muscat </li></ul></ul><ul><ul><ul><li>All around the site in different implementations, different user interfaces </li></ul></ul></ul><ul><ul><ul><li>Often “bolted-on” </li></ul></ul></ul><ul><ul><li>Autonomy </li></ul></ul><ul><ul><ul><li>News and Sport </li></ul></ul></ul><ul><ul><ul><ul><li>heavy duty “research” driven use. </li></ul></ul></ul></ul><ul><ul><li>Not possible to search all BBC web content. </li></ul></ul>

Business advantages of shared search
<ul><li>Shared development means we could invest in a long-term shared solution </li></ul><ul><li>Research and resulting improvements shared by all </li></ul><ul><li>Possible Licensing advantages/economies </li></ul>

User advantages of shared search
<ul><li>Able to search across all of the BBC for the first time </li></ul><ul><li>One common experience at the core of any search within the BBC </li></ul>

How we started

Project process

Market research driven positioning <ul><li>We
currently over-serve a 20-45 male techno-savvy audience </li></ul><ul><li>Want to reach ‘newbies’, mainstream consumers, especially young mums* </li></ul><ul><li>Our market research told us they often used the search engine on their default homepage (as set by ISP) </li></ul><ul><li>Or often “Ask Jeeves” </li></ul><ul><li>* moms ;-) </li></ul>

persona driven design <ul><li>Introducing Gayle
& Steve </li></ul>

“ Use-models”

use-models Where the BBC and
it’s brand can provide value

use-models Hideous Assumptions Ahoy !!!!

use-models

Which turned into…

use-models

use-models

early encouragement!!!

Building taxonomy from user-centred seeds
<ul><li>Limited time and resources to build a taxonomy </li></ul><ul><li>Focussed on analysing the most entered search terms from our logs as a starting point to give most value. </li></ul>

Taxonomy tool

Taxonomy tool – top level
screen Top level of the hierarchical tree structure taxonomy. Shows all our subject areas plus a test node for playing around and a xyzxyz node which reflects our most popular query – nothing! i.e. &quot;&quot;. We couldn't have an entry for nothing so we had to equate a search for no string with something to give us a node. We then had to have that node as top level node because otherwise it would inherit context from its parent.

Taxonomy tool – node level
screen Shows screen for “ Question of Sport ” which has a parent - BBC programmes - and children - the presenter's names. Each node also has urls and synonyms attached. Each url has a label (title), a description, score (crude method to indicate which url is most relevant to the node). Each url has the possibility of having an expiry date which can be reviewed and the url can be archived after expiring.

Taxonomy tool – synonyms Shows
a node with numerous urls attached and scored Plus numerous synonyms The children are all aspects of WW2 – often these are based on programme content on BBC TV or radio (e.g. documentary on Monte Cassino being shown, therefore people likely to enter as a search term)

Taxonomy building process

Taxonomy/editorial team

UI design and testing Round
and round and round…

Iterative design development <ul><li>5 rounds
of user-testing and iterative design over a 2 month period </li></ul><ul><li>We probably could have cut it down if we had done more research upfront </li></ul><ul><li>We started getting ‘diminishing returns’ </li></ul><ul><li>Avoiding Non-designer Kneejerk </li></ul><ul><ul><li>Great to invite marketers and managers to tests, gets them bought into UCD brilliantly </li></ul></ul><ul><ul><li>Manage their ‘breakthroughs’ – “sleep on it!” </li></ul></ul>

What we tested: paper prototypes

What we tested

What we tested

What we tested

What we were looking for
<ul><li>The search is supposed to allow users to… </li></ul><ul><ul><li>Zoom in (refine search) </li></ul></ul><ul><ul><li>Step sideways (to another BBC content area “scope”) </li></ul></ul><ul><ul><li>Zoom out (widen to BBCi or Web scope) </li></ul></ul><ul><li>Do users understand these options? Can they navigate through them? Do they want them? </li></ul>

Key findings

“Best links” area <ul><li>The title
was clear. The www icon was better. But still not perfect. </li></ul>“ A site that's not to do with the BBC. Not run by them. It would be nice to see the whole name: www.NIassembly.gov.uk” User no. 4 , Pippa Key findings

“Best links” area <ul><li>The position
of the icons confused one user (novice). </li></ul>“ Don't know what they are doing there. You'd expect to just click on the words on the left and get what you want. Is it just a link to return to the BBC easily?” User no. 4 , Pippa Key findings

Zooming in <ul><li>Some users understood
that searching is a dialogue… </li></ul>“ Finding the right search word is very frustrating. I think one thing, but the computer thinks differently.” User no. 4 , Pippa Key findings

Zooming in <ul><li>… going from
general to specific. </li></ul>“ I'd expect to be asked to be more specific. It would prompt you type in more words -- recipes? flowers? boat show? Whatever…” User no. 5, Laura Key findings

Zooming in <ul><li>So encourage dialogue:
be polite and positive, however general the initial query was </li></ul>“ It would need to make you feel that you got it right, even though you weren't specific enough. It should make you feel you're on the right track.” User no. 2 , Gillian Key findings

Zooming in or stepping sideways?
<ul><li>Users tended to view the “specific sections” links as a way to zoom in. </li></ul><ul><li>They thought about the links in that section in terms of their search domain, not in terms of BBC organisational structure. </li></ul><ul><li>They don’t know anything much about BBC organisational structure. </li></ul>“ See all the regions -- all the different divisions they have. Listed A-Z... Regions of Northern Ireland” User no. 1, Fred Key findings

Zooming in or stepping sideways?
<ul><li>The “specific section links” area didn’t seem specific enough… </li></ul>“ I'm normally quite rushed on the web. I haven't got time to concentrate. So the specifics section should say all the main headings for the subject (Vikings). Boats, gods, battles, food… The specifics section would not help you out. It doesn't tell you that it can look for specific specifics.” User no. 3 , Emma Key findings

Stepping sideways <ul><li>The planet of
the apes search worked best because the sideways steps (film, cult, news) clearly refined the search. </li></ul>“ You'd get used to that idea of the buttons…” User no. 3 , Emma Key findings

Zooming out <ul><li>Users can’t articulate
what a search engine does but they know what to expect when they see the term. </li></ul>“ Oh! I didn't think it as a search engine. Yes I know I said it would search the web. But I didn't click!” User no. 2 , Gillian Key findings

Live!

Searching across all the BBC

Searching just BBC News

Searching the UK Web

More Lou!

lessons learnt... post-live assessment <ul><li>Marketing
campaign starts April 20 th </li></ul><ul><li>Started a V1.5 testing and design blitz in early February </li></ul><ul><li>2 weeks, 3 user-tests with 2 days to iterate design in-between each! </li></ul>

V1.5

v1.5 <ul><li>Whole team contributed to
the design. </li></ul><ul><li>Posted out new designs every evening </li></ul><ul><li>Covered walls </li></ul><ul><li>Fashion! </li></ul><ul><ul><li>Operating system ‘trends’ getting more important. </li></ul></ul>

What we learnt from the
post-live testing <ul><li>People did NOT see the best links!!! </li></ul><ul><ul><li>All that effort, all the value… useless if UI hides it… </li></ul></ul><ul><li>Numbering is a curse </li></ul><ul><ul><li>It was contributing to skipping the good stuff. </li></ul></ul><ul><li>“Relevance” is irrelevant </li></ul><ul><li>People loved the tabs once they found them. </li></ul><ul><ul><li>But they only found them on the 3 rd or 4 th visit. </li></ul></ul>

Design lessons <ul><li>The tyranny of
the search engine idiom </li></ul><ul><li>Migrate habits </li></ul><ul><li>Get team to collaborate on use-models early – generates understanding early </li></ul><ul><ul><li>Gives whole team something to go back and question fundamentals usefully </li></ul></ul><ul><li>The whole team and the business understands where the value is </li></ul><ul><ul><li>The taxonomy </li></ul></ul><ul><ul><li>The human factor </li></ul></ul>

Conclusion

My Soylent Green Moment… Oh
my God, Search IS PEOPLE!!!

Q&A

Thanks! Hope it was useful/interesting…
http://www.blackbeltjones.com/presentations/asist2002/