Slideshare.net (beta)

 
Post: 
Myspace Hi5 Friendster Xanga LiveJournal Facebook Blogger Tagged Typepad Freewebs BlackPlanet gigya icons

All comments

Add a comment on Slide 1

If you have a SlideShare account, login to comment; else you can comment as a guest


Showing 1-50 of 24 (more)

Site Search Analytics Workshop Presentation

From lrosenfeld, 1 month ago

Taught by Louis Rosenfeld (Boston: April 4, 2008; Sunnyvale, CA: A more

5773 views  |  0 comments  |  24 favorites  |  197 downloads  |  15 embeds (Stats)
 

Tags

experience design user experience louis rosenfeld search log search web analytics site search analytics analytics ia search analytics

more

 
 

Groups/Events

Not added to any group/event

 
 

Privacy InfoNew!

This slideshow is Public

 

Slideshow transcript

Slide 1: Site Search Analytics for a Better User Experience Louis Rosenfeld lou@louisrosenfeld.com April / May, 2008 ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 2: 0 Welcome! ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 3: About us Me You? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 4: Agenda 1.  What is Site Search Analytics (SSA)? 2.  SSA Demonstration 3.  The Point of SSA 4.  Technical Stuff: the nuts and bolts of SSA 5.  Exercise/Discussion: Pattern Analysis 6.  Improving Metadata and Navigation 7.  Exercise/Discussion: Failure Analysis 8.  Improving Content 9.  Exercise/Discussion: Session Analysis 10.  Improving Search 11.  SSA and UX methodology 12.  Advanced Topics/Discussion ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 5: 1 What is Site Search Analytics? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 6: Anatomy of a search log (from Google Search Appliance) Critical elements in bold: IP address, time/date stamp, query, and # of results: XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxystyleshe et=www&q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02 XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www&q=license+plate &ud=1&site=AllSites&spell=1&oe=UTF-8&proxystylesheet=www&ip=XXX.XXX. X.104 HTTP/1.1" 200 8283 146 0.16 XXX.XXX.XX.130 - - [10/Jul/2006:10:24:38 -0800] "GET /search? access=p&entqr=0&output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8&client=www&oe=UTF-8&proxystyleshe et=www&q=regional+transportation+governance +commission&ip=XXX.XXX.X.130 HTTP/1.1" 200 9718 62 0.17 ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 7: ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 8: ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 9: Querying the queries: Generic questions to help get started 1.  What are the most frequent unique queries? 2.  Are frequent queries retrieving quality results? 3.  Click-through rates per frequent query? 4.  Most frequently clicked result per query? 5.  Which frequent queries retrieve zero results? 6.  What are the referrer pages for frequent queries? 7.  Which queries retrieve popular documents? 8.  What interesting patterns emerge in general? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 10: Tuning your questions: From generic to specific Netflix asks 1.  Which movies most frequently searched? 2.  Which of them most frequently clicked through? 3.  Which of them least frequently added to queue? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 11: 2 A brief demonstration ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 12: Session report sample (homegrown; data from WW Norton) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 13: Frequency report sample (from ISYS) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 14: Failure report sample (from WebSideStory/HBX) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 15: 3 What’s the point of Site Search Analytics? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 16: What do users want? in an academic setting ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 17: How we determine what users want Qualitative research (observation)  Field studies  Focus groups  Card sorting  Usability testing and task analysis Quantitative research (data-driven)  Help/reference desk/switchboard  Web analytics (SEO/SEM, clickstream, SSA) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 18: How is SSA different? Data that’s:   Real   High volume   Readily available Analysis that’s:   Quantitative   Inexpensive   Scalable   Complementary to other user research methods ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 19: What’s SSA good for? Profiling users’ needs: what, not why Identifying and diagnosing problems with:   Content   Navigation and metadata   Search: interface design and configuration Designing more effectively through:   More effective prioritization of your work   Asking better, clearer questions   Complementing and improving your user research and evaluation ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 20: 4 Nuts and Bolts: The technical stuff ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 21: Clicks and queries: Often stored separately ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 22: Formats for query capture Flat file search log: often produced by local search engines; requires parsing Local search database: queries automatically stored in database; greater reporting flexibility Turnkey search appliance: local or remotely hosted; often provide web based reporting tools Analytics application: integrated with clickstream data ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 23: Flat file query storage ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 24: Flat file record example (from Google Appliance) 123.45.67.89 - 25/Mar/2006 10:15:32 – 
 http://www.ipodstuff.com/search?q=nanotubes - Firefox 1.0.7; Windows NT 5.1 - 740674ce2123e969 •  123.45.67.89 is the Internet Protocol (IP) address for the computer that originated the search. An IP address may be static – that is, it’s assigned permanently (more or less) to one computer or it may be dynamic; i.e. the Internet service provider assigns the address randomly from a pool of addresses. In any event, this is the address of your customer’s computer. •  25/Mar/2006 10:15:32 is the date and time of the query. •  http://www.ipodstuff.com/search?q=nanotubes is the request URL, including the search query. In this example, the user typed “nanotubes” into the search box. •  Firefox 1.0.7; Windows NT 5.1 is the Web browser and operating system of the computer used by the customer entering the query •  740674ce2123e969 is a “cookie.” A cookie is a unique file stored on a computer accessing a particular function on the Web. Cookies can be used to track user behavior across time, including search activity. ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 25: Local search database ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 26: Local search database: Benefits over flat file approach Easier to store, especially over time Already parsed Better report generation abilities, including ad hoc Can be used in combination with best bets, automated indices (great example at msu.edu) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 27: Turnkey search software appliance Typically include local search database with built-in reports Provide minimal access to query data (some built-in reports, few if any ad hoc reports) SSA might mean breaking open the “black box” (or undercutting vendors’ professional service offerings) Examples: Verity Ultraseek, Mondosoft, Google Appliance ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 28: Analytics applications Javascript inserted into pages to intercept queries and other user activity Analytics vendors typically not focused on site SSA, so reports may require custom development Closest to delivering “Holy Grail” of integrated data (search + browse) Examples: Google Analytics, Omniture ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 29: Analytics applications: Sample report from Google Analytics ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 30: How do you do it? Pros and cons… Capturing queries Analyzing queries ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 31: 5 Exercise/Discussion: Pattern analysis ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 32: 6 Improving metadata and navigation ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 33: SSA techniques for tuning metadata and navigation Pattern analysis: looking for clusters that suggest metadata values and attributes Session analysis: to understand term granularity Query testing: search a query and use “reverse engineering” to find ways to improve metadata ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 34: Why analyze for metadata values (AKA terms, keywords)? Determine synonyms, select preferred terms Improve tone of metadata value  “lorry” versus “truck”  “im” versus “intramural building” Determine term granularity  “cat” and “dog” versus “persian” and “yorkipoo” Inform taxonomies ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 35: Determining metadata values through clustering One week’s top 50 queries, clustered by topic Represents 20% of all search activity < 1 hour of work Can help suggest preferred terms ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 36: Determining metadata values through similar queries BBC explores similar queries (i.e., collaborative filtering) Helps with misspellings, synonyms, best bets ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 37: Determining term granularity through session analysis Steps 1.  Sample a variety of sessions 2.  Compare “starting points” (initial queries) with “end points” 3.  Any patterns in terms of specificity? Example   “HP 1012 printer” -> “HP 1012 printer driver” ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 38: Testing and tuning metadata values Approaches  Tracking appearances, disappearances, and trends among terms  Testing terms by querying them  Deriving terms through “reverse lookup” ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 39: Term trend tracking Which term deserves your attention? Flash in pan, or something important? Next steps and implications: add new term, or replace/delete existing term ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 40: Term trend tracking (sample report from ISYS) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 41: Testing your metadata values: Steps 1.  Choose a manageable number of common queries (e.g., top 25) 2.  Check for matches among metadata values (e.g., query “campus map” matches metadata term “map”) 3.  Queries without corresponding metadata values may indicate gaps in your metadata 4.  You might find new metadata values among your queries with 0 or few results ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 42: Term derivation from reverse lookup: Steps 1.  Identify “important” documents (e.g., popular, or strategic, or new) 2.  Generate list of queries that retrieve those important documents 3.  Determine if there are metadata terms that correspond to those queries 4.  If not, you’ve identified metadata gaps or opportunities to tweak existing terms ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 43: Why analyze for metadata attributes? Determine which attributes to support (each = $$$) Determine which are highest priority ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 44: Determining metadata attributes (AKA types, fields) Top queries clustered and prioritized by attributes:   Place   Dept./ Program   Service   Task 1-2 hours work ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 45: Better metadata leads to better navigation Indirect improvements to any navigation system driven by metadata:   Site-wide taxonomies   Local taxonomies   Thesauri Direct improvements to other navigation systems   Contextual navigation   A-Z indices   Main pages and other major pages ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 46: Identifying and addressing navigation failures/goal changes Do this if you have referrer data Steps: 1.  Identify navigation pages (i.e., pages in site hierarchy) where many queries are initiated (aside from main page/search page) 2.  Cluster most frequent queries for each page to determine common information needs that aren’t being met 3.  Then improve contextual navigation for each navigation page ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 47: Navigation failures/goal changes: (from Google Analytics) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 48: Supporting contextual navigation through embedded queries SLI Systems can display other queries related to a particular query… …and queries related to a particular document ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 49: Identifying search failures to address with better navigation Do this if you have combined query and clickstream data Steps: 1.  Determine top failed queries 2.  Determine to which documents users navigate once they give up on search 3.  Build those results into these queries’ result pages OR extrapolate and present those documents’ categories (if available) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 50: A-Z indices: Three SSA-based approaches 1.  Manual: grab top queries and insert in index (embedding queries in links) 2.  Automated: generate index of queries on the fly (embedding queries in links) 3.  Hybrid: repurpose individual best bets results as single alphabetized index (see Michigan State University example) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 51: Hybrid A-Z index example: Michigan State University ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 52: Main pages: The obvious should be obvious Making sure “maps” were on the MSU main page ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 53: Improving metadata and navigation: The SSA checklist 1.  Analyze for metadata value tone, granularity, and trends 2.  Determine (and invest in) important metadata attributes 3.  Study queries that follow navigation failures to create better contextual navigation 4.  Look to improve “top-down” navigation (e.g., indices, taxonomy, main page) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 54: 7 Exercise/Discussion: Failure analysis ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 55: 8 Improving content ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 56: Major benefits of applying SSA to content Content creation Content positioning  What to view when (in search results, on navigational pages)  Highlighting, demotion, deletion Content prioritization  What to do when  Content authoring, content models, content inventories, content migrations ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 57: SSA techniques for tuning content Simple yet effective  Failure analysis  Trend monitoring Couple with “reverse engineering”:  Use SSA to identify major problems in authoring process  Demonstrate to authors how they impact their content’s findability (e.g., by violating authoring guidelines) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 58: Fixing crawling: Plug indexing gaps Problem  The content was available on the site  But it hadn’t been properly indexed Two approaches 1.  Use SSA to test top queries (requires knowledge of appropriate results) 2.  User server logs to test top and random documents in the clickstream (or automate comparison); are they searchable? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 59: Placeholders/pass-throughs: Acknowledge an unmet content need “freedom of information” a frequent query at MSU; pass through to FOIA officer record ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 60: Developing new content: Addressing emergent content needs Using SSA to plug gaps 0 results report (from behaviortracking.com)   Failure analysis suggests unmet content needs by topic (e.g., frequent queries for “iPod wristband” and “iPod holder for jogging”)   Content typing (covered later) suggests what kind of content to create (e.g., “product comparison” and “specifications”) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 61: Best bets: Going beyond meeting basic needs Ensure useful results for top X (50? 100?) most frequent queries Usually determined manually (guided by documented logic) Technically straightforward; editorial and political issues can be difficult ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 62: Best bets: NCI example (clearly marked) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 63: Best bets: HP example (subtle; exposes content model) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 64: Monitoring trends: Tuning content to meet demand How might such trends impact your content? Search engine? Navigation? From behaviortracking.com ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 65: ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 66: Monitoring trends Short head and long tail generally predictable “Middle torso” is interesting area to monitor ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 67: The shape of your content: Determining content types “What kind of page would users want when they searched this term?” Top queries clustered and prioritized by content types:   Application   News/ Announce- ments   Main page   Contact info   Instructions 1-2 hours work ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 68: Content types help expose content models, improve navigation album pages artist descriptions TV listings album reviews discography artist bios ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved. 68

Slide 69: Exposing content models within search ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 70: Engaging content owners: Showing how users find their content Connecting pages that are found through search… ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 71: Engaging content owners: Showing how users find their content …with how those pages were found. ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 72: Engaging content owners: Comparing time periods Showing trends (from behaviortracking.com) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 73: Improving content: The SSA checklist 1.  Verify content spidering 2.  Plug content gaps (placeholders, new content, appropriate types, best bets) 3.  Improve result positioning (highlighting, demotion, deletion) 4.  Prioritize content development (invest in appropriate topics and types, more efficient migration and inventory) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 74: 9 Exercise/Discussion: Session analysis ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 75: 10 Improving search ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 76: Search systems: Know your anatomy …and know your own system; its functionality will impact your query data ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 77: Search systems: Yours, specifically Vivian Bliss recommends investigating these:  Tokens:  Is the search term/phrase preserved or broken up?  Can the user preserve a phrase?  NLP (Natural Language Processing)  Does the system have any natural language processing capabilities?  Does it stem?  Does it support morphological variants (i.e., gerunds, plurals)? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 78: Search systems: Yours, specifically (continued) Vivian Bliss recommends investigating these:   Ranking: In simple terms, how is this done?   Weighting  Can certain elements such as title or personal name be weighted more heavily?  If the search term is found in a title it is considered more relevant, and ranked higher, than if it is found in the full text?   Concept searching  Does the system support this type of searching?  Are the concepts built and maintained manually or automatically? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 79: SSA techniques for tuning search Pattern analysis  Simple stuff: looking for oddities (e.g., URLs, SKUs, syntax, dates)  Not-so-simple stuff: assessing information- seeking behaviors (up the head, down the tail) Session analysis  Determining what users will do in specific situations (e.g., 0 results, too many results) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 80: Improving “the box”: Initial query entry UI Validate width of query entry field  Use query data to calculate “high number” (e.g., 90th percentile) for query length in characters  Does your entry field accommodate that length? Amazon: 65 characters Microsoft: 25 characters ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 81: Improving “the box”: Search suggestions Populate query entry with common queries (Ask) or reuse keywords from Best Bets development Alternative: use controlled vocabulary (Netflix movie titles) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 82: Improving “the box”: Preset text Are presets showing up frequently in your query data? Consider multivariate testing…   The copy: is there a clearer alternative?   The interaction design: should submit actions (clicking “search” or <return>) work when no query is entered?   The sequence: Flickr displays more explanatory search UI ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 83: Query enhancement: Syntax fixes Examples:   Booleans (AND, OR, NOT)   Wildcards (wom*n)   Fielded searches (site:involution.com) Does your search engine support these queries? Solutions:   Support syntax in search UI   Surface its availability in “advanced search” or “revise your search” UI ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 84: Query enhancement: Handling unique query types Little fixes can add up  Spell-checking  Proper names (people, places)  Unique identifiers (e.g., SKUs, ISBNs) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 85: Query enhancement: Handling unique query types (cont.) Little fixes can add up  Acronyms (glossary lookup) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 86: Search results design: What to show per individual result Help searchers choose most appropriate documents by showing most appropriate information per result Dependent on a sense of common information-seeking behaviors  Known-item: show author, title, date  Open-ended: show keywords, description SSA can help determine common information-seeking behaviors ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 87: Search results design: What to show per individual result …more descriptive information …less information for known- for open-ended searchers item searchers ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 88: Search results design: SSA helps determine sorting Financial Times found a high level of dates within queries Solution: support chronological sorting and filtering ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 89: Search results design: SSA helps determine typing Content typing can enable:   Filtering by exposing types (HP)   Faceted classification/ navigation (WebMD) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 90: Search results design: Which results where? #10 result is clicked through more often than #s 6, 7, 8, and 9 (ten results per page) From SLI Systems (www.sli-systems.com) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 91: Rethinking advanced search: “Revise your Search” Support contextualization of search “help,” rather than grab-bag “advanced” UI Analyze session data to determine how to:  Narrow results  How do users revise after large retrievals?  Build those functions into a "revise your search" UI  Broaden results  How do users revise after small/0 retrievals?  Build those functions into a "revise your search" UI ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 92: Rethinking advanced search: Which functions go where (or away)? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 93: Rethinking advanced search: “Revise your Search” in action ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 94: Surfacing common queries: Valuable or curiosity? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 95: What does SSA have to do with actual search engines? 1. Basic configuration   New features (e.g., spell-checking)   Weighting (e.g., Financial Times and proper names) 2. Tool selection/reconfiguration   Develop functional specs for new tool   Better yet, help tweak existing tool   Justify investment: Deloitte and Vanguard use SSA to demonstrate that improvements that don’t involve engine (e.g., Best Bets) are insufficient ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 96: Improving search: The SSA checklist 1.  Understand how search works (and what can be changed/fixed) 2.  Validate “the box” UI design 3.  Look for opportunities to “soup up” queries (e.g., syntax, spell-checking) 4.  Design better search results (individual and groups) 5.  Blow up “advanced search” and contextualize functions 6.  Buy a better engine—or improve your own ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 97: 11 SSA and your UX methodology ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 98: The power of data: Why UX needs SSA 1.  Balance: UX methodologies/practitioners are often quantitatively weak 2.  Legitimacy: Data-driven analyses make an impression on people hard to impress 3.  Cost: The data is available; the analysis scales well with available time 4.  Fidelity: The data is behavioral, real, and voluminous 5.  Comprehensiveness: Quantitative methods complement and help improve qualitative methods (and vice versa) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 99: Where and when does SSA fit? Or, the what and why of user research Generic What questions lead to specific What questions   Use quantitative methods (e.g., SSA) to determine the contextual diagnostics/KPIs   Example: Netflix What questions lead to new Why questions   Use SSA to prioritize investments in qualitative exploration   Example: clothing retailer following SSA with a field study ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 100: Comparing SSA with other quantitative user research methods Data-driven methods   Clickstream/server log analysis: tells you where more than what users wants (complementary)   Event logging: comparable but often less data and usually at a different point in customer/user lifecycle   SEO/SEM: Different query types (finding where the answer is versus finding the answer); often focused on commerce conversions ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 101: SSA and qualitative user research methods What questions lead to Why questions Examples  Personas: SSA supplements fictitious construct with real data (i.e., likely queries a persona might have)  Task analysis: SSA helps identify common tasks to evaluate  Field studies: SSA suggests issues to monitor in the field ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 102: Augmenting personas and audience profiles with frequent queries Persona example (from Adaptive Path) Frequent queries added (in green) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 103: Using common queries to enhance/fuel task analyses Use for tasks that are more what than how (i.e., testing information needs rather than functions) Examples •  Can you find a map of the campus? •  What study abroad options are available to MSU students? •  When is the last home football game of the season? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 104: Identifying trends to monitor during field studies Example: catalog clothing retailer SSA showed:   Preponderance of SKUs, even though they were not displayed on the site   What were they doing there? Subsequent field study showed:   Customers preferred print catalog to web catalog to identify products   Customers preferred web catalog for ordering Action: add SKUs to web catalog ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 105: Combining quantitative and qualitative Exploring the needs of a particular audience: users of BBC children’s content ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 106: 12 “Advanced” topics/ discussion/conclusion ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 107: Comparing referral queries with local queries ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 108: Analyzing the long tail: Why and how? Why  Synonym mining and term disambiguation (e.g., if “transfer” is a short head query, do variants show up in long tail?)  The data is different (more esoteric queries)  Therefore query categories are different  When you’ve already nailed the short head How: regular random sampling ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 109: Long tail queries: Longer, more complex (from Vanguard) Short head: common queries Long tail: common queries Beneficiary form 403(b)(7) account asset transfer 401(k) authorization beneficiary automatic investing career Wire transfer instructions forms adoption agreement amt international wire transfers money market socially responsible investing location Vanguard tax identification number loans IRA Asset Transfer form calculator fdic insured account early withdrawal penalties ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 110: SSA and the organization: Adoption by UX practitioners Quantitative, data-driven approach that gets UX “taken seriously” Requires getting hands dirty with data, negotiating with IT Needs to scale up gradually (“SSA on a budget”) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 111: SSA and the organization: Adoption by management Appeal of numbers/metrics Viral nature of analytics reports Appeal of scalable approach and quick, measurable wins Tuning, rather than replacing, search engine (and other enterprise–class applications) ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 112: SSA and the organization: Marrying analytics and UX 1.  Beyond diagnostics: SSA as a predictive tool (example: Financial Times) 2.  The business of not converting: moving beyond transactions toward holistic user experience ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 113: Discussion Did I answer your burning question? What three things will you try out at work tomorrow? ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 114: What we’ve covered 1.  What is Site Search Analytics (SSA)? 2.  SSA Demonstration 3.  The Point of SSA 4.  Technical Stuff: the nuts and bolts of SSA 5.  Exercise/Discussion: Pattern Analysis 6.  Improving Metadata and Navigation 7.  Exercise/Discussion: Failure Analysis 8.  Improving Content 9.  Exercise/Discussion: Session Analysis 10.  Improving Search 11.  SSA and UX methodology 12.  Advanced Topics/Discussion ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 115: Please Share Your SSA Knowledge: Visit our book in progress site Search Analytics for Your Site: Conversations with your Customers by Louis Rosenfeld and Richard Wiggins (Rosenfeld Media, 2008) Site URL: www.rosenfeldmedia.com/books/searchanalytics/ Feed URL: feeds.rosenfeldmedia.com/searchanalytics/ ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.

Slide 116: Contact Information Louis Rosenfeld Rosenfeld Media, LLC 705 Carroll Street, #2L Brooklyn, NY 11215 USA lou@louisrosenfeld.com www.louisrosenfeld.com www.rosenfeldmedia.com ©2008 Louis Rosenfeld, LLC (www.louisrosenfeld.com). All rights reserved.