Site Search Analytics: 8 things you can do

Search Analyticsfor Your Site:8 things you can doLouis Rosenfeldlou@louisrosenfeld.com • @louisrosenfeldeMetrics • San Francisco • March 6, 2012

Hello, my name is Lou www.louisrosenfeld.com | www.rosenfeldmedia.com

Agenda

Agenda1.The basics of Site Search Analytics (SSA)

Agenda1.The basics of Site Search Analytics (SSA)2.8 things you can do with SSA

Agenda1.The basics of Site Search Analytics (SSA)2.8 things you can do with SSA3.Some things you can do today

Agenda1.The basics of Site Search Analytics (SSA)2.8 things you can do with SSA3.Some things you can do today4.A short plea for peace, love and harmony

Let’s look at the data

No, let’s look at the real dataCritical elements in bold: IP address, time/date stamp, query, and # of results:XXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?access=p&entqr=0 &output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8 &client=www&oe=UTF-8&proxystylesheet=www& q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /searchaccess=p&entqr=0 &output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www& q=license+plate&ud=1&site=AllSites &spell=1&oe=UTF-8&proxystylesheet=www& ip=XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16

No, let’s look at the real dataCritical elements in bold: IP address, time/date stamp, query, and # of results: What are usersXXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?access=p&entqr=0 searching? &output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8 &client=www&oe=UTF-8&proxystylesheet=www& q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /searchaccess=p&entqr=0 &output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ie=UTF-8&client=www& q=license+plate&ud=1&site=AllSites &spell=1&oe=UTF-8&proxystylesheet=www& ip=XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16

No, let’s look at the real dataCritical elements in bold: IP address, time/date stamp, query, and # of results: What are usersXXX.XXX.X.104 - - [10/Jul/2006:10:25:46 -0800] "GET /search?access=p&entqr=0 searching? &output=xml_no_dtd&sort=date%3AD%3AL %3Ad1&ud=1&site=AllSites&ie=UTF-8 &client=www&oe=UTF-8&proxystylesheet=www& q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1" 200 971 0 0.02XXX.XXX.X.104 - - [10/Jul/2006:10:25:48 -0800] "GET /searchaccess=p&entqr=0 &output=xml_no_dtd&sort=date%3AD%3AL How often are %3Ad1&ie=UTF-8&client=www& users failing? q=license+plate&ud=1&site=AllSites &spell=1&oe=UTF-8&proxystylesheet=www& ip=XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16

SSA is semantically rich data, and...

SSA is semantically rich data, and... Queries sorted by frequency

...what users want--in their own words

A handful of queries/tasks/ways to navigate/features/ documents A little goes a long waymeet the needs of your most important audiences

A handful of queries/tasks/ways to navigate/features/ documents A little goes a long waymeet the needs of your most important audiences Not all queries are distributed equally

A handful of queries/tasks/ways to navigate/features/ documents A little goes a long waymeet the needs of your most important audiences Nor do they diminish gradually

A handful of queries/tasks/ways to navigate/features/ documents A little goes a long waymeet the needs of your most important audiences 80/20 rule isn’t quite accurate

(and the tail is quite long)

The Long Tail is(and the tail is quite long) much longer than you’d suspect

The Zipf Distribution, textually

8 things you can do with SSA 1.Make it harder to get lost in deep content 2.Make search smarter 3.Reduce jargon 4.Learn how your audiences differ 5.Know when to publish what 6.Own and enjoy your failures 7.Avoid disaster 8.Predict the future

#1Make it harder to get lost

Start with basic SSA data:queries and query frequency Percent: volume of search activity for a unique query during a particular time period Cumulative Percent: running sum of percentages

Tease out common content types

Tease out common content types Took an hour to... • Analyze top 50 queries (20% of all search activity) • Ask and iterate: “what kind of content would users be looking for when they searched these terms?” • Add cumulative percentages Result: prioritized list of potential content types #1) application: 11.77% #2) reference: 10.5% #3) instructions: 8.6% #4) main/navigation pages: 5.91% #5) contact info: 5.79% #6) news/announcements: 4.27%

Clear content types lead tobetter contextual navigation album pages artist descriptions TV listingsalbum reviews discography artist bios

Clear content types improvesearch performance

Clear content types improvesearch performance Content objects related to products

Clear content types improvesearch performance Content objects related to products Raw search results

#2Make search smarter

Contextualizing “advanced” features

Session data suggestprogression and context

Session data suggestprogression and context search session patterns 1. solar energy 2. how solar energy works

Session data suggestprogression and context search session patterns 1. solar energy 2. how solar energy works search session patterns 1. solar energy 2. energy

Session data suggestprogression and context search session patterns search session patterns 1. solar energy 1. solar energy 2. solar energy charts 2. how solar energy works search session patterns 1. solar energy 2. energy

Session data suggestprogression and context search session patterns search session patterns 1. solar energy 1. solar energy 2. solar energy charts 2. how solar energy works search session patterns search session patterns 1. solar energy 1. solar energy 2. explain solar energy 2. energy

Session data suggestprogression and context search session patterns search session patterns 1. solar energy 1. solar energy 2. solar energy charts 2. how solar energy works search session patterns search session patterns 1. solar energy 1. solar energy 2. explain solar energy 2. energy search session patterns 1. solar energy 2. solar energy news

Recognizingproper nouns,dates, andunique ID#s

#3Reduce jargon

Saving the brand by killing jargonat a community collegeJargon related to online education: FlexEd, COD, College on DemandMarketing’s solution: expensive campaign to educate public (via posters, brochures)The Numbers query rank query (from SSA): #22 online* #101 COD #259 College on Demand #389 FlexTrack * “online” part of 213 queriesResult: content relabeled, money saved

#4Learn how your audiences differ

Who cares about what?

Why analyze queries by audience?Fortify your personas with dataLearn about differences between audiences • Open University “Enquirers”: 16 of 25 queries are for subjects not taught at OU • Open University Students: search for course codes, topics dealing with completing programDetermine what’s commonly important to all audiences (these queries better work well)

#5Know when to publish what

Interest in the football team: going...

Interest in the football team: going... ...going...

Interest in the football team: going... ...going... gone

Time toInterest in the study! football team: going... ...going... gone

BeforeTax Day

AfterTax Day

#6Own and enjoy your failures

Failed navigation?Examining unexpected searching Look for places searches happen beyond main page What’s going on? • Navigational failure? • Content failure? • Something else?

Where navigation is failing (“Professional Resources” page)Do users andAIGA meandifferentthings by“ProfessionalResources”?

Comparing what users ﬁndand what they want

Failed business goals?Developing custom metrics Netﬂix asks 1. Which movies most frequently searched? (query count) 2. Which of them most frequently clicked through? (MDP views) 3. Which of them least frequently added to queue? (queue adds)

#7Avoid disasters

The new and improved search enginethat wasn’tVanguard used SSA to help benchmark existing search engine’s performance and help select new engineNew search engine “performed” poorlyBut IT needed Information Architect & Dev Team Meeting Where’s the proof? convincing You can’t tell to delay for sure. Search seems to have a few problems… Nah launch .

What to do?Test performance of common queries“Before and after” testing using two sets of metrics 1.Relevance: how reliably the search engine returns the best matches ﬁrst 2.Precision: proportion of relevant results clustered at the top of the list

Old engine (target) and new compared Note: low relevance and high precision scores are optimal More on Vanguard case study: http://bit.ly/D3B8c

Old engine (target) and new compared uh-oh Note: low relevance and high precision scores are optimal More on Vanguard case study: http://bit.ly/D3B8c

Old engine (target) and new compared uh-oh better Note: low relevance and high precision scores are optimal More on Vanguard case study: http://bit.ly/D3B8c

#8Predict the future

Shaping theFinancial Times’ editorial agendaFT compares these • Spiking queries for proper nouns (i.e., people and companies) • Recent editorial coverage of people and companiesDiscrepancy? • Breaking story?! • Let the editors know!

8 things you can do with SSA 1.Make it harder to get lost in deep content 2.Make search smarter 3.Reduce jargon 4.Learn how your audiences differ 5.Know when to publish what 6.Own and enjoy your failures 7.Avoid disaster 8.Predict the future

3 things you can do today

3 things you can do today1.Set up SSA in Google Analytics

3 things you can do today1.Set up SSA in Google Analytics2.Query your queries

3 things you can do today1.Set up SSA in Google Analytics2.Query your queries3.Start developing a site report card

Turn on SSA in Google AnalyticsSet up GA for your site if you haven’t alreadyThen teach it to parse and capture your search engine’s queries (not set by default)References • http://is.gd/cR0qr • http://is.gd/cR0qP

Seed your analysis byquerying your queriesStarter questions 1. What are the most frequent unique queries? 2. Are frequent queries retrieving quality results? 3. Click-through rates per frequent query? 4. Most frequently clicked result per query? 5. Which frequent queries retrieve zero results? 6. What are the referrer pages for frequent queries? 7. Which queries retrieve popular documents? 8. What interesting patterns emerge in general?

Use SSA to start workon a site report card

Use SSA to start work SSA helps determine commonon a site report card information needs

Can SSA bring us together?

Lou’s TABLE OF OVERGENERALIZED Web Analytics User Experience DICHOTOMIES Users intentions and Users behaviors (whatsWhat they analyze happening) motives (why those things happen) Qualitative methods forWhat methods they Quantitative methods to explaining why things employ determine whats happening happen Helps users achieve goalsWhat theyre trying Helps the organization meet (expressed as tasks or to achieve goals (expressed as KPI) topics of interest) Uncover patterns and Measure performance (goal-How they use data driven analysis) surprises (emergent analysis) Statistical data ("real" data Descriptive data (in smallWhat kind of data in large volumes, full of volumes, generated in lab they use errors) environment, full of errors)

Now on sale Search Analytics for Your Site: Conversations with Your Customers by Louis Rosenfeld (Rosenfeld Media, 2011) www.rosenfeldmedia.com Use code eMetrics12 for 25% off all Rosenfeld Media products

Say helloLouis Rosenfeldlou@louisrosenfeld.comwww.louisrosenfeld.comwww.rosenfeldmedia.com@louisrosenfeld@rosenfeldmediaThis presentation available on SlideShare:http://slidesha.re/otzE2t

http://louisrosenfeld.com	7104
http://www.rosenfeldmedia.com	2516
http://rosenfeldmedia.com	2404
http://www.louisrosenfeld.com	103
http://translate.googleusercontent.com	30
http://tomfisher.posterous.com	22
http://thinkery.me	16
http://nicholasthomas.wordpress.com	15
http://2145238431.nvmodules.netvibes.com	14
http://lanyrd.com	13
http://adamtaplin.com	12
http://a0.twimg.com	8
http://www.samsungsxswi.com	6
http://feeds.feedburner.com	4
http://uxbooks.com	4
http://webcache-exp-test.googleusercontent.com	4
http://us-w1.rockmelt.com	3
url_unknown	3
http://10.1.12.249:15871	3
http://tweetedtimes.com	3
http://subodh737.blogspot.com	2
http://www.hanrss.com	2
http://infoalmogiacelia2011.blogspot.com	2
http://dashboard.bloglines.com	2
http://www.zero2geek.com	2
http://fanyi.youdao.com	1
http://translate.google.com.br	1
http://www.netvibes.com	1
http://www.onlydoo.com	1
https://p.yammer.com	1
https://pramati1.pcmk-2.pramati.com	1
http://www3.prisma.edu.pe	1
http://www.rentallect.com	1
http://blackboks.posterous.com	1
http://static.slidesharecdn.com	1
http://www.ofertadeviajes.eu	1
http://www.ofertadeviaje.org	1
http://www.slideshare.net	1
http://twitter.com	1
http://adamhq.tumblr.com	1

Site Search Analytics: 8 things you can do

by Louis Rosenfeld on Mar 13, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

40 Embeds 12,312

Statistics

Site Search Analytics: 8 things you can do — Presentation Transcript