Workshop presented at Webdagene 2013 (http://webdagene.no/en/) September 9, 2013; UX Lisbon (http://www.ux-lx.com), May 12, 2011; UX Hong Kong (http://www.uxhongkong.com/), February 17, 2011.
Presented at EuroIA17, September 2017; World IA Day NYC, February 2017; Interact, October 2016 (London, UK); earlier versions in 2014 at UXPA Boston (Boston, MA, USA); in 2013 at Interaction S.A. (Recife, Brasil), Intuit (Mountain View, CA, USA), Designers + Geeks (New York, USA); in 2012 at UX Russia (Moscow, Russia), UX Hong Kong (Hong Kong, China), WebVisions NYC (New York, NY, USA); in 2011 at the IA Summit (Denver, CO, USA), UX-LX (Lisbon, Portugal), Love at First Website (Portland, OR, USA).
This is something of a successor to my talk "Marrying Web Analytics and User Experience" (http://is.gd/vK34zS)
Slides for my full-day information architecture workshop. Will teach in Minneapolis, MN (November 12, 2012) and Toronto, ON (November 29, 2012) Details: http://rosenfeldmedia.com/workshops/
Fast, Cheap, and Actionable: Creating an Affordable User Research ProgramMichael Powers
Done a usability study? Ready for the next step? Today we have an abundance of fast, affordable website user research methods, many of which can be done remotely with real users. Learn about available user research options and how IUP runs successful research projects that lead to actionable insights.
Slides from a workshop at the 2019 Service Design in Government conference, Edinburgh, March 2019.
The workshop challenged participants to consider:
what happens after you've done some user research for your service? Decisions made, do you move on and forget it? Or do you preserve that research for re-use and future team members? The session was an opportunity for user researchers in government to describe, compare and improve ResearchOps activities.
Presented at EuroIA17, September 2017; World IA Day NYC, February 2017; Interact, October 2016 (London, UK); earlier versions in 2014 at UXPA Boston (Boston, MA, USA); in 2013 at Interaction S.A. (Recife, Brasil), Intuit (Mountain View, CA, USA), Designers + Geeks (New York, USA); in 2012 at UX Russia (Moscow, Russia), UX Hong Kong (Hong Kong, China), WebVisions NYC (New York, NY, USA); in 2011 at the IA Summit (Denver, CO, USA), UX-LX (Lisbon, Portugal), Love at First Website (Portland, OR, USA).
This is something of a successor to my talk "Marrying Web Analytics and User Experience" (http://is.gd/vK34zS)
Slides for my full-day information architecture workshop. Will teach in Minneapolis, MN (November 12, 2012) and Toronto, ON (November 29, 2012) Details: http://rosenfeldmedia.com/workshops/
Fast, Cheap, and Actionable: Creating an Affordable User Research ProgramMichael Powers
Done a usability study? Ready for the next step? Today we have an abundance of fast, affordable website user research methods, many of which can be done remotely with real users. Learn about available user research options and how IUP runs successful research projects that lead to actionable insights.
Slides from a workshop at the 2019 Service Design in Government conference, Edinburgh, March 2019.
The workshop challenged participants to consider:
what happens after you've done some user research for your service? Decisions made, do you move on and forget it? Or do you preserve that research for re-use and future team members? The session was an opportunity for user researchers in government to describe, compare and improve ResearchOps activities.
This slidedeck is from our surfray webinar on Search Analytics in SharePoint 2010. The presentation contains some search theory and an introduction to search analytics reports in Sharepoint 2010. It also covers simple techniques for improving search based on the analytics.
Clare Corthell: Learning Data Science Onlinesfdatascience
Clare Corthell, Data Scientist and Designer at Mattermark, and author of the Open Source Data Science Masters, shares her experience teaching herself data science with online resources. http://datasciencemasters.org/
Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...Dawn Anderson MSc DigM
We are headed to the age of assistive task driven search where the user needs help to 'do' things as well as learn things. Smart speakers, mobile phones, assistive systems and conversational search and action devices are where the buck is headed for now. Where are we at in this wave? What are the challenges? What are the opportunities right now? Here we look at some of the ways we can start to prepare our tactics and strategy to be pioneering search marketers with conversation search and conversation action.
Claudia Gold: Learning Data Science Onlinesfdatascience
Claudia Gold, author of the Data Analysis Learning path on SlideRule, talks about why she wrote it and how to approach learning data science on your own. https://www.mysliderule.com/learning-paths/data-analysis/
Get Better Content with Analytics and User TestingMichael Powers
So you're going to Confab Higher Ed. You're already pretty excited about content strategy. But your boss and colleagues? Not so much. To outsiders, content strategy is just another buzzword. And as more schools move to become "data-driven" organizations, talking about content can sound hopelessly qualitative.
So don't say "content strategy": do it. This session will look at content strategy practices you can introduce to show even your most quantitatively-oriented colleagues the value of content strategy: content analytics, social media analytics, and user testing techniques. Rack up successes first—then start talking content strategy.
• Introduce content strategy practices into your organization when your organization doesn't care about content strategy.
• Use analytics to identify what needs improvement.
• Learn how user-testing techniques can improve your content.
Cloud of Knowing MRS 2010 conference slides - the award winning paperJohn Griffiths
The actual presentation I gave at the Market Research Society conference 2010. There is an earlier preview uploaded here on slideshare from I think the Cloud 2 meetup when I did a dummy run in front of the group. The presentation was shortlisted for best presentation and won best new thinking the last time the award was given funnily enough
HackerEarth is pleased to announce its next session to help you understand what it really takes to become a data scientist.
Agenda of this session will include answers to the following questions:
- Why is it the best time to take up Data Science as a career?
- How can you take the first step in Data Science? (After all, first step is always the hardest!)
- How can you become better and progress fast?
- How is life after becoming a Data Scientist?
Speaker:
Jesse Steinweg-Woods is soon-to-be a Senior Data Scientist at tronc, working on recommender systems for articles and understanding customer behavior. Previously, he worked at Argo Group Insurance on new pricing models that took advantage of machine learning techniques. He received his PhD in Atmospheric Science from Texas A&M University, and his research focused on numerical weather and climate prediction.
How to Become an Internet Research SpecialistChinedum Azuh
Money-Spinning Opportunity: Become an Internet Research Specialist
Earn Fabulous Income in Naira and Foreign Currencies Every Hour Doing Internet Research. No Experience Necessary! No Capital Required!!
Do you like to surf the Web?
Ever used Google?
Would you like to get paid just browsing the internet the way you are used to?
I'm talking about $50 or N5000 an hour at a minimum. It could be a great part-time way to make extra money. If you go full-time, you could make a very nice living -- more than some bankers do. And in either case, you don't have to worry about going to an office, a 9-to-5 schedule, or any of that corporate hassle.
You work where you want, with whom you want, when you want it.
The Internet provides a vast amount of information. If you can quickly and efficiently navigate through that wealth of information, and essentially find something useful, you can create a very successful business as an Internet researcher.
Let me give you some type of research businesses are looking for. This list is not all inclusive. But it should help to get your wheels turning to figure out how you want to position yourself in the research business.
What type of internet research can I do for companies and agencies?
Here is a list of examples:
Market Research
Keyword Research
Content/Information Research
Fact Checking
Background Checking
Competitive Analysis
Business Research
Product Research
Medical Research
Image/Audio research
Social Media Research
Many people assume that they have no marketable skills that other people would want to use, but in the vast majority of cases that simply isn’t true.
You might think that because you can’t write, you can’t make any money online or offline. Not true. Can’t design or draw anything either? Not a problem.
The solution for many people is to take on simple research jobs for other people who don’t have the time to do it themselves. The word ‘research’ tends to bring to mind stuffy libraries and reams of paperwork but that isn’t the case nowadays. Thanks to the internet. Research can be a quick and easy task, and the fact that some people still don’t have the time to do it themselves means big money making opportunities for you.
The great thing about offering up your skills as a researcher is that you don’t necessarily have to specialize in any one particular area. One client might need information on keywords that other websites use so they can determine how to structure their own site content. Someone else might want to get information about a particular subject so they can write a book about it.
As an Internet researcher you'll work with writers, marketers, authors, website owners, publishing companies and other businesses to find information they need for their blogs, websites, articles, books, e-books, products, special reports etc.
Had a great pleasure and honor to give a lecture about the Current and Future Challenges in Data Science at the Nextech 2019 conference alongside an impressive list of other speakers
Originally presented at SXSW March 13, 2011, on panel with Fred Beecher and Austin Govella. Modified and updated for Web 2.0 Expo talk, October 12, 2011, UX Web Summit September 26, 2012; Webdagene September 10, 2013.
Search Analytics: Conversations with Your Customersrichwig
Did you know that the search box on your home page handles half or more of all your visitors requests? What do people search for most often when they visit your Web site? How can you tune your site search -- and your site -- to perform better?
Rich Wiggins presents a talk that he and co-author Lou Rosenfeld prepared, covering the topis of search analytics, Best Bets, and tuning your Web site to match what your customers seek.
This slidedeck is from our surfray webinar on Search Analytics in SharePoint 2010. The presentation contains some search theory and an introduction to search analytics reports in Sharepoint 2010. It also covers simple techniques for improving search based on the analytics.
Clare Corthell: Learning Data Science Onlinesfdatascience
Clare Corthell, Data Scientist and Designer at Mattermark, and author of the Open Source Data Science Masters, shares her experience teaching herself data science with online resources. http://datasciencemasters.org/
Voice Search and Conversation Action Assistive Systems - Challenges & Opportu...Dawn Anderson MSc DigM
We are headed to the age of assistive task driven search where the user needs help to 'do' things as well as learn things. Smart speakers, mobile phones, assistive systems and conversational search and action devices are where the buck is headed for now. Where are we at in this wave? What are the challenges? What are the opportunities right now? Here we look at some of the ways we can start to prepare our tactics and strategy to be pioneering search marketers with conversation search and conversation action.
Claudia Gold: Learning Data Science Onlinesfdatascience
Claudia Gold, author of the Data Analysis Learning path on SlideRule, talks about why she wrote it and how to approach learning data science on your own. https://www.mysliderule.com/learning-paths/data-analysis/
Get Better Content with Analytics and User TestingMichael Powers
So you're going to Confab Higher Ed. You're already pretty excited about content strategy. But your boss and colleagues? Not so much. To outsiders, content strategy is just another buzzword. And as more schools move to become "data-driven" organizations, talking about content can sound hopelessly qualitative.
So don't say "content strategy": do it. This session will look at content strategy practices you can introduce to show even your most quantitatively-oriented colleagues the value of content strategy: content analytics, social media analytics, and user testing techniques. Rack up successes first—then start talking content strategy.
• Introduce content strategy practices into your organization when your organization doesn't care about content strategy.
• Use analytics to identify what needs improvement.
• Learn how user-testing techniques can improve your content.
Cloud of Knowing MRS 2010 conference slides - the award winning paperJohn Griffiths
The actual presentation I gave at the Market Research Society conference 2010. There is an earlier preview uploaded here on slideshare from I think the Cloud 2 meetup when I did a dummy run in front of the group. The presentation was shortlisted for best presentation and won best new thinking the last time the award was given funnily enough
HackerEarth is pleased to announce its next session to help you understand what it really takes to become a data scientist.
Agenda of this session will include answers to the following questions:
- Why is it the best time to take up Data Science as a career?
- How can you take the first step in Data Science? (After all, first step is always the hardest!)
- How can you become better and progress fast?
- How is life after becoming a Data Scientist?
Speaker:
Jesse Steinweg-Woods is soon-to-be a Senior Data Scientist at tronc, working on recommender systems for articles and understanding customer behavior. Previously, he worked at Argo Group Insurance on new pricing models that took advantage of machine learning techniques. He received his PhD in Atmospheric Science from Texas A&M University, and his research focused on numerical weather and climate prediction.
How to Become an Internet Research SpecialistChinedum Azuh
Money-Spinning Opportunity: Become an Internet Research Specialist
Earn Fabulous Income in Naira and Foreign Currencies Every Hour Doing Internet Research. No Experience Necessary! No Capital Required!!
Do you like to surf the Web?
Ever used Google?
Would you like to get paid just browsing the internet the way you are used to?
I'm talking about $50 or N5000 an hour at a minimum. It could be a great part-time way to make extra money. If you go full-time, you could make a very nice living -- more than some bankers do. And in either case, you don't have to worry about going to an office, a 9-to-5 schedule, or any of that corporate hassle.
You work where you want, with whom you want, when you want it.
The Internet provides a vast amount of information. If you can quickly and efficiently navigate through that wealth of information, and essentially find something useful, you can create a very successful business as an Internet researcher.
Let me give you some type of research businesses are looking for. This list is not all inclusive. But it should help to get your wheels turning to figure out how you want to position yourself in the research business.
What type of internet research can I do for companies and agencies?
Here is a list of examples:
Market Research
Keyword Research
Content/Information Research
Fact Checking
Background Checking
Competitive Analysis
Business Research
Product Research
Medical Research
Image/Audio research
Social Media Research
Many people assume that they have no marketable skills that other people would want to use, but in the vast majority of cases that simply isn’t true.
You might think that because you can’t write, you can’t make any money online or offline. Not true. Can’t design or draw anything either? Not a problem.
The solution for many people is to take on simple research jobs for other people who don’t have the time to do it themselves. The word ‘research’ tends to bring to mind stuffy libraries and reams of paperwork but that isn’t the case nowadays. Thanks to the internet. Research can be a quick and easy task, and the fact that some people still don’t have the time to do it themselves means big money making opportunities for you.
The great thing about offering up your skills as a researcher is that you don’t necessarily have to specialize in any one particular area. One client might need information on keywords that other websites use so they can determine how to structure their own site content. Someone else might want to get information about a particular subject so they can write a book about it.
As an Internet researcher you'll work with writers, marketers, authors, website owners, publishing companies and other businesses to find information they need for their blogs, websites, articles, books, e-books, products, special reports etc.
Had a great pleasure and honor to give a lecture about the Current and Future Challenges in Data Science at the Nextech 2019 conference alongside an impressive list of other speakers
Originally presented at SXSW March 13, 2011, on panel with Fred Beecher and Austin Govella. Modified and updated for Web 2.0 Expo talk, October 12, 2011, UX Web Summit September 26, 2012; Webdagene September 10, 2013.
Search Analytics: Conversations with Your Customersrichwig
Did you know that the search box on your home page handles half or more of all your visitors requests? What do people search for most often when they visit your Web site? How can you tune your site search -- and your site -- to perform better?
Rich Wiggins presents a talk that he and co-author Lou Rosenfeld prepared, covering the topis of search analytics, Best Bets, and tuning your Web site to match what your customers seek.
Large language models in higher educationPeter Trkman
Discussing the possibilities of large language models for the automatic generation of academic content by the students (e.g. master thesis), and the related need for changes in the way in which to educate and evaluate students.
Users are Losers! They’ll Like Whatever we Make! and Other Fallacies.Carol Smith
Presented at CodeMash 2013.
If this sounds familiar it is time to make big changes or look for a new job. Failing your users will only end badly. In this session we look at the assumptions that are all-too-often made about users, usability and the User Experience (UX). In response to each of these misguided statements Carol will provide a quick method you can conduct with little or no resources to debunk these myths.
A Brief (and Practical) Introduction to Information ArchitectureLouis Rosenfeld
Keynote presentation by Louis Rosenfeld at the Usability and Accessibility for the Web International Seminar; 26 July 2007, Monterrey, Nuevo Leon, Mexico
Information Architecture: Putting the "I" back in ITLouis Rosenfeld
Presentation by Lou Rosenfeld that introduces information architecture to senior IT managers. Covers perceived problems faced by IT managers, strategic value of information, IA basics, tangible IA benefits, and how IT and IA are natural allies in making information truly strategic to enterprises.
Book Formatting: Quality Control Checks for DesignersConfidence Ago
This presentation was made to help designers who work in publishing houses or format books for printing ensure quality.
Quality control is vital to every industry. This is why every department in a company need create a method they use in ensuring quality. This, perhaps, will not only improve the quality of products and bring errors to the barest minimum, but take it to a near perfect finish.
It is beyond a moot point that a good book will somewhat be judged by its cover, but the content of the book remains king. No matter how beautiful the cover, if the quality of writing or presentation is off, that will be a reason for readers not to come back to the book or recommend it.
So, this presentation points designers to some important things that may be missed by an editor that they could eventually discover and call the attention of the editor.
Between Filth and Fortune- Urban Cattle Foraging Realities by Devi S Nair, An...Mansi Shah
This study examines cattle rearing in urban and rural settings, focusing on milk production and consumption. By exploring a case in Ahmedabad, it highlights the challenges and processes in dairy farming across different environments, emphasising the need for sustainable practices and the essential role of milk in daily consumption.
Expert Accessory Dwelling Unit (ADU) Drafting ServicesResDraft
Whether you’re looking to create a guest house, a rental unit, or a private retreat, our experienced team will design a space that complements your existing home and maximizes your investment. We provide personalized, comprehensive expert accessory dwelling unit (ADU)drafting solutions tailored to your needs, ensuring a seamless process from concept to completion.
Hello everyone! I am thrilled to present my latest portfolio on LinkedIn, marking the culmination of my architectural journey thus far. Over the span of five years, I've been fortunate to acquire a wealth of knowledge under the guidance of esteemed professors and industry mentors. From rigorous academic pursuits to practical engagements, each experience has contributed to my growth and refinement as an architecture student. This portfolio not only showcases my projects but also underscores my attention to detail and to innovative architecture as a profession.
Can AI do good? at 'offtheCanvas' India HCI preludeAlan Dix
Invited talk at 'offtheCanvas' IndiaHCI prelude, 29th June 2024.
https://www.alandix.com/academic/talks/offtheCanvas-IndiaHCI2024/
The world is being changed fundamentally by AI and we are constantly faced with newspaper headlines about its harmful effects. However, there is also the potential to both ameliorate theses harms and use the new abilities of AI to transform society for the good. Can you make the difference?
Transforming Brand Perception and Boosting Profitabilityaaryangarg12
In today's digital era, the dynamics of brand perception, consumer behavior, and profitability have been profoundly reshaped by the synergy of branding, social media, and website design. This research paper investigates the transformative power of these elements in influencing how individuals perceive brands and products and how this transformation can be harnessed to drive sales and profitability for businesses.
Through an exploration of brand psychology and consumer behavior, this study sheds light on the intricate ways in which effective branding strategies, strategic social media engagement, and user-centric website design contribute to altering consumers' perceptions. We delve into the principles that underlie successful brand transformations, examining how visual identity, messaging, and storytelling can captivate and resonate with target audiences.
Methodologically, this research employs a comprehensive approach, combining qualitative and quantitative analyses. Real-world case studies illustrate the impact of branding, social media campaigns, and website redesigns on consumer perception, sales figures, and profitability. We assess the various metrics, including brand awareness, customer engagement, conversion rates, and revenue growth, to measure the effectiveness of these strategies.
The results underscore the pivotal role of cohesive branding, social media influence, and website usability in shaping positive brand perceptions, influencing consumer decisions, and ultimately bolstering sales and profitability. This paper provides actionable insights and strategic recommendations for businesses seeking to leverage branding, social media, and website design as potent tools to enhance their market position and financial success.
2. Hello, my name is Lou
www.louisrosenfeld.com | www.rosenfeldmedia.com
3. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
5. No, let’s look at the real data
Critical elements in bold: IP address, time/date stamp, query, and # of
results:
XXX.XXX.X.104 - - [10/Jul/2011:10:25:46 -0800]
"GET /search?access=p&entqr=0
&output=xml_no_dtd&sort=date%3AD%3AL
%3Ad1&ud=1&site=AllSites&ie=UTF-8
&client=www&oe=UTF-8&proxystylesheet=www&
q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1"
200 971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2011:10:25:48 -0800]
"GET /searchaccess=p&entqr=0
&output=xml_no_dtd&sort=date%3AD%3AL
%3Ad1&ie=UTF-8&client=www&
q=license+plate&ud=1&site=AllSites
&spell=1&oe=UTF-8&proxystylesheet=www&
ip=XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
6. No, let’s look at the real data
Critical elements in bold: IP address, time/date stamp, query, and # of
results:
XXX.XXX.X.104 - - [10/Jul/2011:10:25:46 -0800]
"GET /search?access=p&entqr=0
&output=xml_no_dtd&sort=date%3AD%3AL
%3Ad1&ud=1&site=AllSites&ie=UTF-8
&client=www&oe=UTF-8&proxystylesheet=www&
q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1"
200 971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2011:10:25:48 -0800]
"GET /searchaccess=p&entqr=0
&output=xml_no_dtd&sort=date%3AD%3AL
%3Ad1&ie=UTF-8&client=www&
q=license+plate&ud=1&site=AllSites
&spell=1&oe=UTF-8&proxystylesheet=www&
ip=XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
What are users
searching?
7. No, let’s look at the real data
Critical elements in bold: IP address, time/date stamp, query, and # of
results:
XXX.XXX.X.104 - - [10/Jul/2011:10:25:46 -0800]
"GET /search?access=p&entqr=0
&output=xml_no_dtd&sort=date%3AD%3AL
%3Ad1&ud=1&site=AllSites&ie=UTF-8
&client=www&oe=UTF-8&proxystylesheet=www&
q=lincense+plate&ip=XXX.XXX.X.104 HTTP/1.1"
200 971 0 0.02
XXX.XXX.X.104 - - [10/Jul/2011:10:25:48 -0800]
"GET /searchaccess=p&entqr=0
&output=xml_no_dtd&sort=date%3AD%3AL
%3Ad1&ie=UTF-8&client=www&
q=license+plate&ud=1&site=AllSites
&spell=1&oe=UTF-8&proxystylesheet=www&
ip=XXX.XXX.X.104 HTTP/1.1" 200 8283 146 0.16
What are users
searching?
How often are
users failing?
11. A little goes a long wayA handful of queries/tasks/ways to navigate/features/ documents
meet the needs of your most important audiences
12. A little goes a long wayA handful of queries/tasks/ways to navigate/features/ documents
meet the needs of your most important audiences
Not all queries are
distributed equally
13. A little goes a long wayA handful of queries/tasks/ways to navigate/features/ documents
meet the needs of your most important audiences
14. A little goes a long wayA handful of queries/tasks/ways to navigate/features/ documents
meet the needs of your most important audiences
Nor do they
diminish gradually
15. A little goes a long wayA handful of queries/tasks/ways to navigate/features/ documents
meet the needs of your most important audiences
16. A little goes a long wayA handful of queries/tasks/ways to navigate/features/ documents
meet the needs of your most important audiences
80/20 rule isn’t
quite accurate
25. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
26. Exercise 1 (pattern analysis)
Work in pairs
• Each pair should have a laptop with
Microsoft Excel
• Laptop platform (Mac, PC) doesn’t matter
Download data files: 2005-October.xls
Refer to exercise sheet
No right answers
Have fun!
27. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
32. Start with basic SSA data:
queries and query frequency
Percent: volume
of search activity
for a unique
query during a
particular time
period
Cumulative
Percent:
running sum of
percentages
35. Tease out common content types
Took an hour to...
• Analyze top 50 queries (20% of all search activity)
• Ask and iterate: “what kind of content would users be
looking for when they searched these terms?”
• Add cumulative percentages
Result: prioritized list of potential content types
#1) application: 11.77%
#2) reference: 10.5%
#3) instructions: 8.6%
#4) main/navigation pages: 5.91%
#5) contact info: 5.79%
#6) news/announcements: 4.27%
36. Clear content types lead to
better contextual navigation
artist descriptions
album reviews
album pages
artist biosdiscography
TV listings
46. Session data suggest
progression and context
search session patterns
1. solar energy
2. how solar energy works
search session patterns
1. solar energy
2. energy
47. Session data suggest
progression and context
search session patterns
1. solar energy
2. how solar energy works
search session patterns
1. solar energy
2. energy
search session patterns
1. solar energy
2. solar energy charts
48. Session data suggest
progression and context
search session patterns
1. solar energy
2. how solar energy works
search session patterns
1. solar energy
2. energy
search session patterns
1. solar energy
2. solar energy charts
search session patterns
1. solar energy
2. explain solar energy
49. Session data suggest
progression and context
search session patterns
1. solar energy
2. how solar energy works
search session patterns
1. solar energy
2. energy
search session patterns
1. solar energy
2. solar energy charts
search session patterns
1. solar energy
2. explain solar energy
search session patterns
1. solar energy
2. solar energy news
62. Why analyze queries by audience?
Fortify your personas with data
Learn about differences between audiences
• Open University “Enquirers”: 16 of 25 queries
are for subjects not taught at OU
• Open University Students: search for course
codes, topics dealing with completing program
Determine what’s commonly important to all
audiences (these queries better work well)
64. Save the brand by killing jargon
Jargon related to online education: FlexEd, COD,
College on Demand
Marketing’s solution: expensive campaign to
educate public (via posters, brochures)
Result: content relabeled, money saved
query rank query
#22 online*
#101 COD
#259 College on Demand
#389 FlexTrack
*“online”part of 213 queries
65. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
66. Exercise 2 (longitudinal analysis)
Work in pairs
• Each pair should have a laptop with
Microsoft Excel
• Laptop platform (Mac, PC) doesn’t matter
Download data files: 2006-February.xls +
2006-June.xls
Refer to exercise sheet
No right answers
Have fun!
67. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
85. Failed business goals?
Developing custom metrics
Netflix asks
1. Which movies most frequently searched? (query count)
2. Which of them most frequently clicked through? (MDP views)
3. Which of them least frequently added to queue? (queue adds)
86. Failed business goals?
Developing custom metrics
Netflix asks
1. Which movies most frequently searched? (query count)
2. Which of them most frequently clicked through? (MDP views)
3. Which of them least frequently added to queue? (queue adds)
87. Failed business goals?
Developing custom metrics
Netflix asks
1. Which movies most frequently searched? (query count)
2. Which of them most frequently clicked through? (MDP views)
3. Which of them least frequently added to queue? (queue adds)
96.
1.Choose a
content type (e.g.,
events)
2.Ask:“Where
should users go
from here?”
3.Analyze the
frequent queries
from this content
type
from aiga.org
99. Sandia National Labs
• Regularly record which documents came up
at position #1 for 50 most frequent queries
• If and when that top document falls out of
position #1, document's owner is alerted
• Result: healthy dialogue (often about
following policies and procedures and their
value)
103. Shaping the
FinancialTimes’ editorial agenda
FT compares these
• Spiking queries
for proper nouns
(i.e., people and
companies)
• Recent editorial
coverage of
people and
companies
Discrepancy?
• Breaking story?!
• Let the editors
know!
104. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
105. Avoiding a disaster atVanguard
Vanguard used SSA to help benchmark
existing search engine’s performance and
help select new engine
New search engine “performed” poorly
But IT needed
convincing
to delay
launch
Information Architect &
Dev Team Meeting
Search seems
to have a few
problems… Nah
.
Where’s the
proof?
You can’t tell
for sure.
106. What to do?
Test performance of most frequent queries
Measure using original two sets of metrics
1.relevance: how reliably the search engine
returns the best matches first
2.precision: proportion of relevant and
irrelevant results clustered at the top of the list
107. Relevance: 5 metrics
(queries tested have “best” result)
Mean: Average distance from the top
Median: Less sensitive to outliers, but not useful once at
least half are ranked #1
Count - Below 1st: How
often is the best target
something other than
1st?
Count – Below 5th: How
often is the best target
outside the critical area?
Count – Below 10th: How
often is the best target
beyond the first page?
108. Relevance: 5 metrics
(queries tested have “best” result)
Mean: Average distance from the top
Median: Less sensitive to outliers, but not useful once at
least half are ranked #1
Count - Below 1st: How
often is the best target
something other than
1st?
Count – Below 5th: How
often is the best target
outside the critical area?
Count – Below 10th: How
often is the best target
beyond the first page?
OK!
109. Relevance: 5 metrics
(queries tested have “best” result)
Mean: Average distance from the top
Median: Less sensitive to outliers, but not useful once at
least half are ranked #1
Count - Below 1st: How
often is the best target
something other than
1st?
Count – Below 5th: How
often is the best target
outside the critical area?
Count – Below 10th: How
often is the best target
beyond the first page?
OK!
Hmmm...
110. Relevance: 5 metrics
(queries tested have “best” result)
Mean: Average distance from the top
Median: Less sensitive to outliers, but not useful once at
least half are ranked #1
Count - Below 1st: How
often is the best target
something other than
1st?
Count – Below 5th: How
often is the best target
outside the critical area?
Count – Below 10th: How
often is the best target
beyond the first page?
OK!
Hmmm...
Uh oh
111. Precision:
rating scale
Evaluate frequent queries’ top search results on this scale
• r / Relevant: Based on the information the user provided, the page's
ranking is completely relevant
• n / Near: The page is not a
perfect match, but it’s clearly
reasonable for it to be ranked
highly
• m / Misplaced: You can see
why the search engine
returned it, but it should not
be ranked highly
• i / Irrelevant: The result has
no apparent relationship to
the user’s search
112. Precision:
three metrics
Metrics based on degrees of permissiveness
1. strict: only counts completely relevant results
2. loose: counts relevant and near results
3. permissive: counts relevant, near, and misplaced results
113. Putting it all together:
old engine (target) and new
Note: low relevance and high precision scores are optimal
More on Vanguard case study: http://bit.ly/D3B8c
114. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
116. Search Metrics: general examples
(Lee Romero, blog.leeromero.org)
• Total searches for a given time period
• Total distinct search terms for a given time period
• Total distinct words for a given time period
• Average words per search
• Top searches for a given time period
• Top Searches over time
• Not found searches
• Error searches
• Ratio of searches performed each reporting period to the
number of visits for that same time period
117. Search Metrics: search engine tuning
(Jeannine Bartlett, earley.com)
Do users not find what they want because the search engine
and its ranking and relevance algorithms have not been
adequately tuned?
Example Benchmarks and Metrics
• # of valid queries returning no results / total unique queries
• Relative % search results per data source
• Relative % click throughs per data source
• Pass/fail % for queries using stemming
• Pass/fail % for queries with misspellings
• Precision measures of“seed”documents sent through the tagging
process
118. Search Metrics: query entry
(Jeannine Bartlett, earley.com)
Do users not find what they want because the UI for
expressing search terms is inadequate or unintuitive?
Example Benchmarks and Metrics
• % queries in the bottom set of the Zipf Curve (flat vs. hockey-stick
distribution)
• % queries with no click throughs
• % queries using syntactic metadata filtering (date, author, source,
document type, geography, etc.)
• % queries using Boolean search grammar
• % queries using type-ahead against taxonomy terms and synonyms
• % queries using faceted semantic refinement
• % pages from which search is available
119. Search Metrics: result sets
(Jeannine Bartlett, earley.com)
Do users not find what they want because the UI for
visualizing result sets is inadequate or unintuitive?
Example Benchmarks and Metrics
• % queries utilizing multiple results views
• % queries with drill-down through clusters
• % queries using iterative syntactic metadata filtering (date range,
sorting, type or source inclusion/exclusion, etc.)
• % queries suggesting broader/narrower terms
• % queries suggesting“Best Bets”or“See Also”
• % queries using iterative semantic term filtering, inclusion or
exclusion
120. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
121. Things to do today
1.Set up SSA in Google Analytics
2.Query your queries
3.Start developing a site report card
4.Start incorporating SSA into your
user research program
122. Turn on SSA in Google Analytics
Set up GA for your site if you haven’t already
Then teach it to parse and capture your
search engine’s queries (not set by default)
References
• http://is.gd/cR0qr
• http://is.gd/cR0qP
123. Seed your analysis by
querying your queries
Starter questions
1. What are the most frequent unique queries?
2. Are frequent queries retrieving quality results?
3. Click-through rates per frequent query?
4. Most frequently clicked result per query?
5. Which frequent queries retrieve zero results?
6. What are the referrer pages for frequent queries?
7. Which queries retrieve popular documents?
8. What interesting patterns emerge in general?
128. Agenda
1.The basics of Site Search Analytics (SSA)
2.Exercise 1 (pattern analysis)
3.Things you can do with SSA
4.Exercise 2 (longitudinal analysis
5.More things you can do with SSA
6.A case study
7.More on metrics
8.Things you can do today
9.Discussion
130. Long tail queries:
Longer, more complex (fromVanguard)
Short head: common queries Long tail: common queries
Beneficiary form
401(k)
beneficiary
career
forms
amt
money market
location
loans
calculator
403(b)(7) account asset transfer authorization
automatic investing
Wire transfer instructions
adoption agreement
international wire transfers
socially responsible investing
Vanguard tax identification number
IRA Asset Transfer form
fdic insured account
early withdrawal penalties
131. Now on sale
Search Analytics forYour Site:
Conversations with
Your Customers
by Louis Rosenfeld
(Rosenfeld Media, 2011)
www.rosenfeldmedia.com
Use code
WEBDAGENE2013
for 20% off all
Rosenfeld Media books
We get two major things out of this data: SESSIONS and FREQUENT QUERIES\n
Your brain on data: what will it do?\n
Your brain on data: what will it do?\n
\n
\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
Amazing drawing by Eva-Lotta Lamm: www.evalotta.net\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
Personas: http://www.uie.com/images/blog/YahooExamplePersona.gif\nTable: From Jarrett, Quesenbery, Stirling, and Allen’s report “Search Behaviour at OU;” April 6, 2007.\n
Personas: http://www.uie.com/images/blog/YahooExamplePersona.gif\nTable: From Jarrett, Quesenbery, Stirling, and Allen’s report “Search Behaviour at OU;” April 6, 2007.\n
Personas: http://www.uie.com/images/blog/YahooExamplePersona.gif\nTable: From Jarrett, Quesenbery, Stirling, and Allen’s report “Search Behaviour at OU;” April 6, 2007.\n
Personas: http://www.uie.com/images/blog/YahooExamplePersona.gif\nTable: From Jarrett, Quesenbery, Stirling, and Allen’s report “Search Behaviour at OU;” April 6, 2007.\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
Examples\n “OO7” versus “007”\n Porn-related (not carried by Netflix)\n “yoga”: not stocking enough? Or not indexing enough record content? Some other problem?\n
Examples\n “OO7” versus “007”\n Porn-related (not carried by Netflix)\n “yoga”: not stocking enough? Or not indexing enough record content? Some other problem?\n