Recommended
More Related Content
Similar to Exploring session search
Similar to Exploring session search (20)
More from Gene Golovchinsky
More from Gene Golovchinsky (6)
Recently uploaded
Recently uploaded (20)
Exploring session search
- 1. Exploring Session Search Gene Golovchinsky FX Palo Alto Laboratory, Inc. @HCIR_GeneG
- 2. Thanks to: Jeremy Pickens, Abdigani Diriye, Tony Dunnigan
- 3. Exploratory search Interactive Information seeking Anomalous state of knowledge Evolving information need Often recall-oriented
- 4. One Query to Rule Them All No single query satisfies a typical exploratory search information need Search strategies involve many queries Queries return overlapping results
- 5. Why we’re here 1. How do we know what’s a session? 2. How do we help people deal with this complex task? 3. How do we evaluate systems and algorithms?
- 6. Warning THIS TALK CONTAINS EXPLICIT CONTENT
- 7. Explicit vs. implicit sessions Explicit sessions 1. We ask the person 2. We infer it from structural aspects of the search context Task context may provide strong organizing queues For example, genealogical searches are often tied to a person in a family tree What about implicit sessions?
- 8. Implicit section detection is based on implicit assumptions How do we detect a session? – Time heuristics – Client connection heuristics – Query similarity heuristics What are we assuming? – Person works continuously – Person does not switch tasks – Enough overlap in queries How good are these assumptions?
- 9. Tradeoffs Implicit sessions Explicit sessions Pros Pros No explicit user input required Accurate Needed for collaboration Cons Durable over time Effectiveness relies on precision- oriented information needs and inter-query similarity, i.e., on Cons redundancy Requires manual input in some cases More difficult to connect recurring or ongoing instances of the same information need
- 11. Strategies Ignore it The traditional approach Manage redundancy in the UI Ancestry.com, Querium Increase diversity through scoring Some algorithmic evaluation, but are such interactive systems deployed?
- 12. Manage redundancy in the UI COPING WITH REDUNDANCY
- 13. Some UI examples Google +1 but no session awareness & no good persistent visual feedback Bing Visible query history but no help with documents Ancestry.com Flags previously saved records for current person Querium user interface Variety of document- and query-centric displays
- 14. Ancestry.com: Query overlap How can we help people make sense of search results? What’s new? What’s redundant? What’s useful? What’s not useful?
- 15. Querium: Filtering by process metadata History of interaction during a search can be projected onto current results
- 16. Querium: Visualizing re-retrieval Document-centered retrieval history can be projected onto each search result Indicates “important” documents Indicates new documents
- 21. Some (cor)related metrics Novelty Precision Diversity Recall The exact relationship Redundancy is hard to pin down
- 22. Increasing {diversity} with scoring Pros Query – Can incorporate prior explicit and implicit relevance assessments Black box – More focused queries may retrieve more pertinent documents at a given Rank Session docs state cutoff Cons – Relies on accurate assessment of relevance Displayed User – No way to recover “organic” results, feedback ranking so hard for people to understand effect of personalization Stop
- 23. Increasing {diversity} with post-processing Rank Query Pros docs – Can recover “organic” results – Supports feedback on incorrect inference “organic” If user selects demoted doc ranking – Accommodates shifting info needs better Session – Can be applied interactively state Re-rank Cons docs – Limited document set Displayed User feedback ranking Stop
- 25. Vague generalities Session-based search must be evaluated as a human- machine system Hard to account for real human behavior through simulations only Recall and precision do not tell the whole story Exploratory search is inherently a learning process Effort, knowledge gain, frustration, serendipity important Look at patterns of interaction that led to discovery Hard to evaluate marginal contribution of each query due to negative results, learning, information need drift
- 26. Some thoughts on evaluating algorithms Small gains in retrieval effectiveness will be swamped by interaction, good or bad Small statistically-significant effects are meaningless in practice Evaluation “in the wild” relies on users for ground truth Use post-hoc analysis to test how algorithms predicted users’ choices Look at system’s ability to help people recognize useful documents How many times was a document retrieved before it was seen? This works for lab and naturalistic studies
- 27. In closing… Information needs evolve Queries are approximations Knowledge is uncertain Design challenge: Help people plan future actions by understanding the present in the context of the past
- 28. While I have your attention… There is a pending proposal to create a StackExchange site for information retrieval. Think of it as Stack Overflow for IR geeks. We need more people to vote & promote. http://area51.stackexchange.com/proposals/39142/informatio n-retrieval-and-search-engines
- 29. Do I still have your attention? IIiX 2012 August 21-24, 2012, Nijmegen, The Netherlands Deadline for papers April 9, 2012 EuroHCIR 2012 Same place, August 25 Deadline for papers is June 22, 2012 HCIR 2012: The 6th Symposium on Human Computer Information Retrieval October 4-5, 2012, Boston, Massachusetts, USA Submission deadline mid-summer Will publish works in progress and archival, full-length papers