OSLO STOCKHOLM LONDON BOSTON SINGAPORESearch AnalyticsComperio - Seminar on SearchdrivenWebsites and Analytics of SearchlogsStockholm Digital Days 2013-05-22Bo Engren
Agenda• What is Search(log) Analytics?• Improving Search• Best Practices & Administration• QA
Web Analytics vs. Search AnalyticsThe difference between WebAnalytics and Search Analytics isthat Web shows what the usersactually have been doing, Searchshows their intent.(and btw Search Analytics isn’t SEO either)
The challenges with SearchI can’t findwhat I’mlooking forContentis oldDuplicatesandversionsNotmaintainedToo manychoicesLanguageanddomainvocabularyPoor userexperienceetc…
The relevancy thresholdBy raising therelevance with40%, we canmove thesearch solutionfrom low tohigh trust.
Operational steps for good searchDEFINESCOPEIMPLEMENTRELEASEMAINTAINUnderstand businessneeds• Understand what you aretrying to achieve• Plan and define goals• Identify good trends, ROIMeasure and refine• Monitor and use queryinformation• Mine query logs• Measure effectivenessof search towards atargetOutput and benefits• Better search• Better results• Enhanced usability• Enhanced revenuesSearchcustomer
Analyzing search logs – fundamentalsWhen you have defined your business needsMonitor your search logs......again and again and againLook for• Specific queries• General queries• Queries with zero results• Filter away junk!
Know your search distribution350 10.0000050020%80%SimilarsearchesUniquesearchesFrequencyQuery termCan we findpatterns inthis type ofsearches?Take goodcare of yourtop queries
Frequent queriesVisualized Search history. Most frequent query terms
Unique queries exampleA lot ofproduct codesearches
Top/Frequent queriesHow do we serve frequent queries best?• Ensure good relevance• Apply best bets• If ambient, present options to narrow results• If specific, make sure user get to the goal
Content Search - Refiners• Filters are based on words in documents• Words are used to tag the document with predefined set of FilternamesResult RefinersEnables filtering
Boosts and Blocks• Boosting is the process of changing the“natural” rank to alter the position of a documentwithin the result set
Apply selected Linguistic Features• Automatic language detection• Approximate matching (spell checking) “cort”, “court”• Lemmatization Noun: “car” “cars”Verb: “break” “break”, “breaks”, “broke”•• Synonyms “color” = “colour”“car” = “automobile”• Proper Name and Phrasing /Spellcheck “Venus Williams”, “French Open”• Anti-phrasing (Stopwords) “[I want a] Nikon camera”• Character Normalization “Molière -> Moliere”• Tokenization (CJK support) “market-shares” -> “market shares”• Phonetic Search “Eyvind”, “Oyvind” -> “Eyvind”• Automatic spelltuning Based on index contentsWhen implemented properly can drastically improve theusefulness of a search
Search statistics – several tools available• Start with the searchlogs:– Use the built in tools– Loggparsers (IIS loggparser etc.)– Webanalytics tools (Google Analytics,Webtrends etc.)– Log management (logstash, kibana)– Big data (Hadoop, pig)
Visual searchresultsComperio internal Knowledge Management DB February 2013
Statistic analysis – Best Practice• Zero hit results key to monitor and remove• Analyze the Top queries• Trends over time – group by day/week/month• Separate internal and external searches• Group the queries for better understanding (forexample products, documents, persons)
Examples of Metrics for SearchAnalytics – select a few initallySearch perspectiveMeasures DefinitionMetrictypeTotal queries Total number of search queries #Clicks Total number of clicks that goes from search results to final file or page #Satisfied queries Percentage of search results with at least one click %Opportunity queries Percentage of search results with no click %Visits with keyword searches Percentage of web visitors that use search %Visits with guided product search Percentage of web visitors that use guided product search %Visits with browsing searches Percentage of web visitors that use browseing searches e.g. listings %Search result exits Percentage of web visitors that exit the website on the search result page %Searches with zero results Percentage of searches that end up with zero results %Search depth Depth after search result page #Refined searches Number of searches refined with new query text after result view #Result relevancy Relevancy of search results, based on recall/precision test model and test set #Query suggestion use Number of searches performed with suggested queries #Related queries Number of searches with related queries used #Filtered queries Number of searches with query refinement filters #Time to destination Time spent from search to final result TimeResult sidebar use Percentage of clicks on sidebar results on result page views %Advanced queries Number of advanced queries performed with boolean or filter operators #Best bets use Percentage of clicks on manual top results when displayed %
Improve results of searches - BestPracticeImprove similar searches (fat head)• Autocomplete• Best betsImprove uniqe searches (long tail)• Spellchecking• Synonyms• Adjust your content
Internal searches – do we understandthe context of the user?• Start with the User– Study/test your User Stories.Example: You are going to start a new project.Do you find what you need to get started?– Use Online surverys for deeper insights
All search platforms need maintenance• A team that specializes in searchand related technologies– Front end search specialists– Search analysts• Examples of Tasks– Sounding board for proposed projects or reportedproblems– Cataloguing agreed search best practice– Control vocabularies and taxonomies– Monitoring and tuning– In-house training
Search Analytics – Summary 1• Make someone responsible for search - Appoint aSearch Manager• Set a search strategy which enables the businessstrategy and is in line with overall IT-strategy• Make the Business Case• Measure and Monitor Search Queries = SearchAnalytics• Enable User Feedback• Raise quality of information by adding metadata anddoing content lifecycle management• Add metadata - manual, mandatory or automatic?
Search Analytics - Summary 2• Establish processes to deliver feedback to yourStakeholders regarding the search logs– Separate External and Internal sites?• Educate information creators - simple handouts andsit-downs• Apply spelling suggestions, key-matches and auto-complete• What can we do as Editors and what do we needTechies to do?– You can do more than you think!