Tony Russell-Rose, UXLabs
Stephann Makri, UCL
   The information-seeking process
     Models and frameworks
     Search strategies and tactics
   Patterns of search behaviour
     Enterprise Search
     Site Search
   Design patterns
   Conclusions

              UXLabs - User Experience Research and Design - www.uxlabs.co.uk   2
Models and frameworks




              UXLabs - User Experience Research and Design - www.uxlabs.co.uk   3
Query
                                                                                     Refinement

   Standard Model
     Broder 2002                 Task
                                              Information
                                                  Need
                                                            Verbal form     Query
                                                                                       Search
                                                                                       Engine
                                                                                                  Results



     Sutcliffe & Ennis 1998
                                                                                    Documents

   Cognitive Model
     Norman
      1988, Marchioni 1989

   Dynamic Model                                                 External World

     Bates 1989, O‘ Day
      and Jeffries 1993

               UXLabs - User Experience Research and Design - www.uxlabs.co.uk                              4
• to keep a search on track and
 Monitoring                 efficient

Information               • for navigating through the file or link
  structure                 structure of the individual resources

    Query                 • to support the overall query or
reformulation               search specification)

                          • to aid the selection of words or
    Term                    phrases in the current query
           UXLabs - User Experience Research and Design - www.uxlabs.co.uk   5
Search Modes
 • Monitoring a known topic or set of variables over time
 • Following a specific plan for information gathering
 • Exploring a topic in an undirected fashion

Analysis Techniques
 • Looking for trends or correlations
 • Making comparisons
 • Experimenting with different aggregations/scaling
 • Identifying critical subsets
 • Making assessments
 • Interpreting data to find meaning


                  UXLabs - User Experience Research and Design - www.uxlabs.co.uk   6
Original Model
•   Starting: activities associated with the initial search for information
•   Chaining: following citation chains
•   Browsing: semi-directed searches
•   Differentiating: filtering the material examined
•   Monitoring: maintaining awareness
•   Extracting: working thru a source to locate material of interest

Additions
• Verifying: checking accuracy of information
• Ending: activities associated with the end of a project

                  UXLabs - User Experience Research and Design - www.uxlabs.co.uk   7
Navigational            • to reach a particular site


                        • to acquire information assumed to be
Informational             present on one or more web pages

                        • to perform some web-mediated
Transactional             activity


  Resource              • to get access to an online resource

         UXLabs - User Experience Research and Design - www.uxlabs.co.uk   8
UXLabs - User Experience Research and Design - www.uxlabs.co.uk   9
Search modes and chains




               UXLabs - User Experience Research and Design - www.uxlabs.co.uk   10
Academic
                    • Bates
                    • Ellis
                    • Marchionini




Practitioner
                                       Data-driven
• Morville
• Spencer                              • Enterprise search
• Lamantia                             • Site search




 UXLabs - User Experience Research and Design - www.uxlabs.co.uk   11
   104 scenarios gathered thru professional
    services engagements
     e.g. I need to understand the cost drivers for a
      commodity so I can negotiate better terms with suppliers
   Generated by end users / stakeholders
     Moderated and prioritised in requirements workshops
   Represent user needs
     Not limited by current systems
     Self-reported (not inferred from interview transcripts)
   Consistent analysis: Grounded Theory
     Iterative coding / re-coding
     Excluded ‘non-discovery’ scenarios
                  UXLabs - User Experience Research and Design - www.uxlabs.co.uk   12
Locating Verifying     Monitoring      Comparing      Comprehending Exploring




Scenarios




                  UXLabs - User Experience Research and Design - www.uxlabs.co.uk           13
Lookup
                    • Locating
                    • Verifying
                    • Monitoring



Investigate                         Learn
• Analyzing
                                    • Comparing
• Evaluating
                                    • Comprehending
• Synthesizing
                                    • Exploring




 UXLabs - User Experience Research and Design - www.uxlabs.co.uk   14
• Find a specific (possibly known) item
Locate                • Find a new part with particular technical attributes –
                        Engineering



                    • Confirm that an item meets some specific, objective

Verify                criterion
                      • Ensure all data in the fund attributes table is accurate and
                        up to date – Quantitative Analyst



                    • Maintain awareness of the status of an item for purposes

Monitor               of management or control
                      • Monitor how well we are tracking to revenue and margin
                        targets by division – District Manager


     UXLabs - User Experience Research and Design - www.uxlabs.co.uk                   15
• To identify similarities & differences between a set of
                        items
 Compare                • Compare a lead's performance claims with relevant
                           benchmarks – Portfolio Manager



                      • To generate independent insight by understanding the
                        patterns within a data set
Comprehend              • Understand the cost drivers for this commodity so I can
                          negotiate better terms with my suppliers - Procurement



                      • To investigate an item or data set for the purpose of
                        knowledge discovery
  Explore               • Identify sales opportunities and targets – District
                          Manager


        UXLabs - User Experience Research and Design - www.uxlabs.co.uk             16
• To examine an item or data set to identify patterns
                       & relationships
 Analyze               • Analyze expected cost impact of proposed
                         strategies/actions – Cost Estimator


                     • To use judgement to determine the value of an item
                       with respect to a specific goal
 Evaluate              • Assess leads for potential to build portfolio –
                         Department Lead


                     • To create a novel or composite artefact from
                       diverse inputs
Synthesize             • Formulate scope & strategy for sourcing and gap
                         closure – Core Buyer

       UXLabs - User Experience Research and Design - www.uxlabs.co.uk       17
   Emphasis on Analyze (72) and Evaluate (67)
     Locate represents a tiny fraction of all needs (2)
     (Baseline = 242 instances)
   Search behaviour is not random
     Framework captures common patterns
     Repeating sequences of 2 or 3 modes
     Composite chains (nested)



              UXLabs - User Experience Research and Design - www.uxlabs.co.uk   18
Verify



Analyze                       Comprehend                              Synthesize



                                Compare                                 Evaluate



Monitor                          Analyze



Explore

          UXLabs - User Experience Research and Design - www.uxlabs.co.uk          19
   Comparison-driven search (analyze-compare-evaluate)
     Engineering: Compare our module set teardowns with competitive
      teardown information to see if we’re staying competitive
   Exploration-driven search (explore-analyze-evaluate)
     Core Engineer: Identify opportunities to optimize use of tooling
      capacity for my commodity/parts
   Strategic insight (analyze-comprehend-evaluate)
     Engineering: Track module cost versus functionality over time to
      determine trends
   Strategic oversight (monitor-analyze-evaluate)
     Financial Analyst: Monitor & assess commodity status against
      strategy/plan/target



                 UXLabs - User Experience Research and Design - www.uxlabs.co.uk   20
   Methodology as before
     277 site search scenarios
   Emphasis on Locate (121) and Explore (109)
     Synthesize represents a tiny fraction (2)
     (Baseline = 454 instances)
   Search behaviour is not random



              UXLabs - User Experience Research and Design - www.uxlabs.co.uk   21
Explore                         Analyze
                              Comprehend                               Comprehend



                                  Locate                               Evaluate



                                                                            Verify




          UXLabs - User Experience Research and Design - www.uxlabs.co.uk            22
   Explore -> Analyze -> Comprehend (44/97)
     Assess the proper market value for my car
     Understand what my competitors are selling
   Explore -> Locate -> Evaluate (31/97)
     Find useful stuff on my subject topic
     I want to find best offers before the others do so I can
      have a high margin
   Locate -> Verify (29/56)
     Find out if a specific book is available in the library
     Find a van that meets a customer's requirements

                UXLabs - User Experience Research and Design - www.uxlabs.co.uk   23
   Greater emphasis on ‘Lookup’ modes
     Locate, Verify
   Less emphasis on ‘Investigate’ modes
     Analyze, Evaluate
   Much flatter distribution
     Shorter chains
     Less embedding



              UXLabs - User Experience Research and Design - www.uxlabs.co.uk   24
Support for search modes




               UXLabs - User Experience Research and Design - www.uxlabs.co.uk   25
26
27
28
29
30
31
32
33
34
35
Summary and future work




                          36
Search modes and patterns
• A lens for understanding information-seeking behaviour
• A framework for design

Site Search vs. Enterprise Search
• Same modes but different patterns
• Different priorities: lookup vs. investigate
Further Work
• Refine and validate process
• Extend framework to other data sources
               UXLabs - User Experience Research and Design - www.uxlabs.co.uk   37
   Coming soon …
   Morgan Kaufmann,
    late 2012
   M.J. Bates. Information search tactics. Journal of the American Society for
    Information Science, 30(4):205–214, 1979
   A. Broder. A taxonomy of web search. SIGIR Forum, 36(2):3–10, 2002
   D. Ellis. A behavioural model for information retrieval system design.
    Journal of Information Science, 15:237–247, 1989
   Gary Marchionini. Information Seeking in Electronic Environments.
    Cambridge University Press, 1995
   D.A. Norman. The Psychology of Everyday Things. Basic Books, New
    York, 1988
   Vicki L. O'Day and Robin Jeffries. Orienteering in an information
    landscape: how information seekers get from here to there. In
    Proceedings of the INTERCHI Conference on Human Factors in Computing
    Systems (CHI'93), Amsterdam, April 1993
   D.E. Rose and D. Levinson. Understanding User Goals In Web Search.
    Proceedings of the 13th International Conference on World Wide Web
    (WWW'04), pages 13–19, 2004
   A.G. Sutcliffe and M. Ennis. Towards a cognitive theory of information
    retrieval. Interacting with Computers, 10:321–351, 1998

                   UXLabs - User Experience Research and Design - www.uxlabs.co.uk   39
Tony Russell-Rose, PhD
Director, UXLabs

   Web:     http://uxlabs.co.uk
   Email: tgr@uxlabs.co.uk
   Blog:    http://isquared.wordpress.com
   LinkedIn: http://uk.linkedin.com/in/tonyrussellrose
   Twitter: @tonygrr



              UXLabs - User Experience Research and Design - www.uxlabs.co.uk   40

A Taxonomy of Site Search

  • 1.
  • 2.
    The information-seeking process  Models and frameworks  Search strategies and tactics  Patterns of search behaviour  Enterprise Search  Site Search  Design patterns  Conclusions UXLabs - User Experience Research and Design - www.uxlabs.co.uk 2
  • 3.
    Models and frameworks UXLabs - User Experience Research and Design - www.uxlabs.co.uk 3
  • 4.
    Query Refinement  Standard Model  Broder 2002 Task Information Need Verbal form Query Search Engine Results  Sutcliffe & Ennis 1998 Documents  Cognitive Model  Norman 1988, Marchioni 1989  Dynamic Model External World  Bates 1989, O‘ Day and Jeffries 1993 UXLabs - User Experience Research and Design - www.uxlabs.co.uk 4
  • 5.
    • to keepa search on track and Monitoring efficient Information • for navigating through the file or link structure structure of the individual resources Query • to support the overall query or reformulation search specification) • to aid the selection of words or Term phrases in the current query UXLabs - User Experience Research and Design - www.uxlabs.co.uk 5
  • 6.
    Search Modes •Monitoring a known topic or set of variables over time • Following a specific plan for information gathering • Exploring a topic in an undirected fashion Analysis Techniques • Looking for trends or correlations • Making comparisons • Experimenting with different aggregations/scaling • Identifying critical subsets • Making assessments • Interpreting data to find meaning UXLabs - User Experience Research and Design - www.uxlabs.co.uk 6
  • 7.
    Original Model • Starting: activities associated with the initial search for information • Chaining: following citation chains • Browsing: semi-directed searches • Differentiating: filtering the material examined • Monitoring: maintaining awareness • Extracting: working thru a source to locate material of interest Additions • Verifying: checking accuracy of information • Ending: activities associated with the end of a project UXLabs - User Experience Research and Design - www.uxlabs.co.uk 7
  • 8.
    Navigational • to reach a particular site • to acquire information assumed to be Informational present on one or more web pages • to perform some web-mediated Transactional activity Resource • to get access to an online resource UXLabs - User Experience Research and Design - www.uxlabs.co.uk 8
  • 9.
    UXLabs - UserExperience Research and Design - www.uxlabs.co.uk 9
  • 10.
    Search modes andchains UXLabs - User Experience Research and Design - www.uxlabs.co.uk 10
  • 11.
    Academic • Bates • Ellis • Marchionini Practitioner Data-driven • Morville • Spencer • Enterprise search • Lamantia • Site search UXLabs - User Experience Research and Design - www.uxlabs.co.uk 11
  • 12.
    104 scenarios gathered thru professional services engagements  e.g. I need to understand the cost drivers for a commodity so I can negotiate better terms with suppliers  Generated by end users / stakeholders  Moderated and prioritised in requirements workshops  Represent user needs  Not limited by current systems  Self-reported (not inferred from interview transcripts)  Consistent analysis: Grounded Theory  Iterative coding / re-coding  Excluded ‘non-discovery’ scenarios UXLabs - User Experience Research and Design - www.uxlabs.co.uk 12
  • 13.
    Locating Verifying Monitoring Comparing Comprehending Exploring Scenarios UXLabs - User Experience Research and Design - www.uxlabs.co.uk 13
  • 14.
    Lookup • Locating • Verifying • Monitoring Investigate Learn • Analyzing • Comparing • Evaluating • Comprehending • Synthesizing • Exploring UXLabs - User Experience Research and Design - www.uxlabs.co.uk 14
  • 15.
    • Find aspecific (possibly known) item Locate • Find a new part with particular technical attributes – Engineering • Confirm that an item meets some specific, objective Verify criterion • Ensure all data in the fund attributes table is accurate and up to date – Quantitative Analyst • Maintain awareness of the status of an item for purposes Monitor of management or control • Monitor how well we are tracking to revenue and margin targets by division – District Manager UXLabs - User Experience Research and Design - www.uxlabs.co.uk 15
  • 16.
    • To identifysimilarities & differences between a set of items Compare • Compare a lead's performance claims with relevant benchmarks – Portfolio Manager • To generate independent insight by understanding the patterns within a data set Comprehend • Understand the cost drivers for this commodity so I can negotiate better terms with my suppliers - Procurement • To investigate an item or data set for the purpose of knowledge discovery Explore • Identify sales opportunities and targets – District Manager UXLabs - User Experience Research and Design - www.uxlabs.co.uk 16
  • 17.
    • To examinean item or data set to identify patterns & relationships Analyze • Analyze expected cost impact of proposed strategies/actions – Cost Estimator • To use judgement to determine the value of an item with respect to a specific goal Evaluate • Assess leads for potential to build portfolio – Department Lead • To create a novel or composite artefact from diverse inputs Synthesize • Formulate scope & strategy for sourcing and gap closure – Core Buyer UXLabs - User Experience Research and Design - www.uxlabs.co.uk 17
  • 18.
    Emphasis on Analyze (72) and Evaluate (67)  Locate represents a tiny fraction of all needs (2)  (Baseline = 242 instances)  Search behaviour is not random  Framework captures common patterns  Repeating sequences of 2 or 3 modes  Composite chains (nested) UXLabs - User Experience Research and Design - www.uxlabs.co.uk 18
  • 19.
    Verify Analyze Comprehend Synthesize Compare Evaluate Monitor Analyze Explore UXLabs - User Experience Research and Design - www.uxlabs.co.uk 19
  • 20.
    Comparison-driven search (analyze-compare-evaluate)  Engineering: Compare our module set teardowns with competitive teardown information to see if we’re staying competitive  Exploration-driven search (explore-analyze-evaluate)  Core Engineer: Identify opportunities to optimize use of tooling capacity for my commodity/parts  Strategic insight (analyze-comprehend-evaluate)  Engineering: Track module cost versus functionality over time to determine trends  Strategic oversight (monitor-analyze-evaluate)  Financial Analyst: Monitor & assess commodity status against strategy/plan/target UXLabs - User Experience Research and Design - www.uxlabs.co.uk 20
  • 21.
    Methodology as before  277 site search scenarios  Emphasis on Locate (121) and Explore (109)  Synthesize represents a tiny fraction (2)  (Baseline = 454 instances)  Search behaviour is not random UXLabs - User Experience Research and Design - www.uxlabs.co.uk 21
  • 22.
    Explore Analyze Comprehend Comprehend Locate Evaluate Verify UXLabs - User Experience Research and Design - www.uxlabs.co.uk 22
  • 23.
    Explore -> Analyze -> Comprehend (44/97)  Assess the proper market value for my car  Understand what my competitors are selling  Explore -> Locate -> Evaluate (31/97)  Find useful stuff on my subject topic  I want to find best offers before the others do so I can have a high margin  Locate -> Verify (29/56)  Find out if a specific book is available in the library  Find a van that meets a customer's requirements UXLabs - User Experience Research and Design - www.uxlabs.co.uk 23
  • 24.
    Greater emphasis on ‘Lookup’ modes  Locate, Verify  Less emphasis on ‘Investigate’ modes  Analyze, Evaluate  Much flatter distribution  Shorter chains  Less embedding UXLabs - User Experience Research and Design - www.uxlabs.co.uk 24
  • 25.
    Support for searchmodes UXLabs - User Experience Research and Design - www.uxlabs.co.uk 25
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
    Search modes andpatterns • A lens for understanding information-seeking behaviour • A framework for design Site Search vs. Enterprise Search • Same modes but different patterns • Different priorities: lookup vs. investigate Further Work • Refine and validate process • Extend framework to other data sources UXLabs - User Experience Research and Design - www.uxlabs.co.uk 37
  • 38.
    Coming soon …  Morgan Kaufmann, late 2012
  • 39.
    M.J. Bates. Information search tactics. Journal of the American Society for Information Science, 30(4):205–214, 1979  A. Broder. A taxonomy of web search. SIGIR Forum, 36(2):3–10, 2002  D. Ellis. A behavioural model for information retrieval system design. Journal of Information Science, 15:237–247, 1989  Gary Marchionini. Information Seeking in Electronic Environments. Cambridge University Press, 1995  D.A. Norman. The Psychology of Everyday Things. Basic Books, New York, 1988  Vicki L. O'Day and Robin Jeffries. Orienteering in an information landscape: how information seekers get from here to there. In Proceedings of the INTERCHI Conference on Human Factors in Computing Systems (CHI'93), Amsterdam, April 1993  D.E. Rose and D. Levinson. Understanding User Goals In Web Search. Proceedings of the 13th International Conference on World Wide Web (WWW'04), pages 13–19, 2004  A.G. Sutcliffe and M. Ennis. Towards a cognitive theory of information retrieval. Interacting with Computers, 10:321–351, 1998 UXLabs - User Experience Research and Design - www.uxlabs.co.uk 39
  • 40.
    Tony Russell-Rose, PhD Director,UXLabs  Web: http://uxlabs.co.uk  Email: tgr@uxlabs.co.uk  Blog: http://isquared.wordpress.com  LinkedIn: http://uk.linkedin.com/in/tonyrussellrose  Twitter: @tonygrr UXLabs - User Experience Research and Design - www.uxlabs.co.uk 40

Editor's Notes

  • #14 c/o Grounded Theory
  • #30 Also List Inlay (e.g. Google reader)? How about hit highlighting?
  • #35 Citation maps, Concept maps, summariesTyler learning styles docGeospatial / water example
  • #36 Also addGrouping of similar results: structured results, a la NCSU libraries?