WWW 2018 tutorial on Understanding User Needs & Tasks Details: https://task-ir.github.io/Task-based-Search/

1. Inferring User Tasks and Needs
Rishabh Mehrotra1, Emine Yilmaz2, Ahmed Hassan Awadallah3
1Spotify, London
2University College London
3Microsoft Research

2. Outline of the Tutorial
• Section 1: Introduction
• Section 2: Characterizing Tasks
• Section 3: Tasks Extraction Algorithms
• Section 4: Task based Evaluation
• Section 5: Applications

3. Section 5: Applications
– Task based personalization
–Task based recommendations
–Task Tours
–Predicting Task Continuation
–Task Completion Dialogue Systems

4. Task Based Personalization
• Users tend to be interested in certain tasks when they use
search engines
• Represent users in terms of tasks they are interested in
• Personalize search results based on that
– Query recommendation, re-ranking search results, …

5. Traditional Approach: Topic Based Personalization
• Topics commonly
constructed in two ways
– Manual topical
categories (e.g. ODP)
– Topic Modelling Based
(e.g. LDA)
Topics

6. Personalization: Topics versus Tasks
Topics Tasks
Finance,
Basketball, Jazz
Finance,
Basketball, Pop
music
Basketball, Pop
music

7. Task based Personalization
[Mehrotra and Yilmaz, ACM RecSys Posters ’14]
– Given N users and M tasks
– Construct an NxM user-task association matrix R
• Cosine similarity between user profiles and task representations
– Find the Nxd user feature matrix U, and Mxd task feature matrix
T s.t.
R ≈ U TT
– Discover the d latent features underlying the interactions
between users and tasks

8. Representing Users in the Task-space
• Existing user modeling methods fail to differentiate between
users having similar topical interests
– User curious about "search engines" and an experienced IR researcher
– a stockbroker and a normal investor
• The objective is to leverage user's topical interest profiles
along with user's task associations.
Topics Tasks
Finance,
Basketball, Jazz
Finance,
Basketball, Pop
music
Basketball, Pop
music

9. Coupling Topics and Tasks
[Mehrotra and Yilmaz, ACM ICTIR’15]

10. • Construct a 3-mode tensor to jointly model the user's topical
and task preferences:
– <users, topics, tasks>
• Define each tensor component as:
• A user's participation in a certain task gets weighted by her
topical affinity.
Coupling Topics and Tasks
[Mehrotra and Yilmaz, ACM ICTIR’15]

11. • Tensor decomposition to leverage connections between different
users across different topics and different tasks
• PARAFAC Tensor Decomposition [Stegeman and Sidiropolous, Linear Algebra and
Applications, ‘07]
• Ui, Vj, Tk are D-dimensional vectors representing users, topics, and
tasks, respectively
• Discover the D latent features underlying the interactions between users, topics and tasks
Coupling Topics and Tasks
[Mehrotra and Yilmaz, ACM ICTIR’15]

12. Evaluation: Collaborative Query Recommendation
• Identify user cohorts based on user
preferences
• Personalize search results based on
recommendations from similar users
Number of Similar Users

13. Section 5: Applications
– Task based personalization
–Task based recommendations
–Task Tours
–Predicting Task Continuation
–Task Completion Dialogue Systems

14. • Provide heterogeneous recommendations during users’
browsing process
• Define tasks as demand sequences embedded in user browsing
sessions
Task-based Recommendation on a Web-Scale

15. • Step 1: Collaborative Task Mining:
– extract frequent demand sequences from large scale browser logs
– achieved via frequent sequence mining problem
• Step 2: Task-based Demand Prediction
– predict the upcoming demand of a user given the current browsing session
– estimate the probability of each demand d ∈ D being the follow-on demand of
the current session
• Step 3: Task-based Recommendation
– Provide site-level recommendations (based on predicted demands)
– Provide link-level recommendations (heterogeneous recommendations
based on browsing behavior)
Task-based Recommendation on a Web-Scale

16. Task Tours: Helping Users Tackle Complex Search Tasks
Automatically create multi-step task tours:
– URL labeling with topical category to identify tasks
– construction of the task graph that relates tasks to
each other
– building of the tours using task graph
– identification of triggers
Task tours help users:
– understand the required steps to complete a task,
– find URLs related to the active task
– alert users to activities they may have missed
Task Tours: Helping Users Tackle Complex Search Tasks, CIKM 2012

17. Predicting Task Continuation
– Understand, characterize and
detect tasks which will be
continued
– Bing logs used to identify intent,
topics & search behavior
associated with long running
tasks
– Prediction model using various
features
Search, Interrupted: Understanding and Predicting Search Task Continuation; SIGIR 2012
Task continuation for broad search intent

18. Task Completion Dialogue Systems
– Reinforcement learning based
model
– Goal directed conversations
– Accesses external knowledge
base
– Slot filling to form a semantic
frame
End-to-End Task-Completion Neural Dialogue Systems, arXiv 2017

19. Section 5: Applications
– Task based personalization
–Task based recommendations
–Task Tours
–Predicting Task Continuation
–Task Completion Dialogue Systems

20. Summary - I
– Query intent understanding
• Classification based (ODP, LDA)
• Cluster based (Random walks, reformulations)
• Session based techniques
– Time based segmentation
– Content based segmentation
– Hybrid segmentation
– Extracting search tasks
– Evaluating task extraction algorithms
– Applications

21. – Query intent understanding
– Extracting search tasks
• Task Extraction
– Clustering based approaches
– Entity oriented task extraction
– Structured SVM based bestlinks structures
– LDA topics with Hawkes process
• Tasks à Subtasks
– dd-CRP with embeddings model
– BRT Hierarchical Subtask segmentation
– Evaluating task extraction algorithms
– Applications
Summary - II

22. – Query intent understanding
– Extracting search tasks
– Evaluating task extraction algorithms
• Gold standard dataset
• User study based evaluation
• Alternative techniques
• TREC Tasks Tracks
– Applications
Summary - III

23. –Query intent understanding
–Extracting search tasks
–Evaluating task extraction algorithms
–Applications
• Task based user modeling
• Related Search suggestions
• Task based ecommerce recommendations
Summary - IV

24. Ongoing/Future Work
• Task based user satisfaction prediction
• Digital assistants
– Task understanding
– Task completion
• Book Uber
• Deliver food
• Task based recommendations
• Beyond search – web tasks

25. Questions?
• Rishabh Mehrotra
Research Scientist
Spotify, London
rishabhm@spotify.com
• Emine Yilmaz
Associate Professor, UCL
Faculty Fellow, The Alan Turing Institute
Research Consultant, Microsoft Research
emine.yilmaz@ucl.ac.uk
• Ahmed Hassan Awadallah
Research Lead
Microsoft Research, Redmond
hassanam@microsoft.com

26. References
[1] Ahmed, White, Pantel, Dumais, and Wang. Supporting complex search
tasks. In Proceedings of the ACM CIKM 2014.
[2] Baeza-Yates, Hurtado, and Mendoza. Query recommendation using query
logs in search engines. In Current Trends in
Database Technology-EDBT 2004 Workshops, 2005.
[3] D. M. Blei and Griths. The nested chinese restaurant process and bayesian
nonparametric inference of topic hierarchies.
Journal of the ACM (JACM) 2010.
[4] Blundell and Teh. Bayesian hierarchical community discovery. In NIPS 2013.
[5] C. Blundell, Y. W. Teh, and K. A. Heller. Bayesian rose trees. In UAI 2010.
[6] B. Carterette, E. Kanoulas, M. Hall, and P. Clough. Overview of the trec 2014
session track. 2013.
[7] L. D. Catledge. Characterizing browsing strategies in the world-wide web.
Computer Networks and ISDN systems, 1995.
[8] Chuang and Chien. Towards automatic generation of query taxonomy: A
hierarchical clustering approach. In ICDM 2003.
[9] N. Craswell and M. Szummer. Random walks on the click graph. In ACM
SIGIR 2007.
[10] Donato, Bonchi, and Chi. Do you want to take notes?: identifying missions
in yahoo! search pad. In WWW 2010.

27. [11] D. He. Combining evidence for automatic web session identication.
Information Processing & Management, 2002.
[12] Heller and Ghahramani. Bayesian hierarchical clustering. In ICML 2005.
[13] Hua, Song, and Wang. Identifying users' topical tasks in web search. In
ACM WSDM 2013.
[14] Jones, Rey, Madani, and Greiner. Generating query substitutions. In
WWW 2006.
[15] R. Jones and K. L. Klinkner. Beyond the session timeout: automatic
hierarchical segmentation of search topics in query logs. In CIKM 2008.
[16] J. H. Kim and Kim. Modeling topic hierarchies with the recursive
chinese restaurant process. In CIKM 2012.
[17] Kotov, Bennett, White, Dumais, and Teevan. Modeling and analysis of
cross-session search tasks. In SIGIR 2011.
[18] L. Li and H. Deng. Identifying and labeling search tasks via query-based
hawkes processes. In KDD 2014.
[19] Y. Li. Relationships among work tasks, search tasks, and interactive
information searching behavior. ProQuest, 2008.
[20] Y. Li and N. J. Belkin. A faceted approach to conceptualizing tasks in
information seeking. Information Processing & Management, 2008.
References

28. [21] H. Liao, Song. Evaluating the effectiveness of search task trails. In WWW
2012.
[22] J. Liu and N. J. Belkin. Personalizing information retrieval for multi-
session tasks: The roles of task stage and task type. In SIGIR 2010.
[23] Lucchese, Orlando, Perego, Silvestri, and Tolomei. Discovering tasks from
search engine query logs. ACM Transactions on Information Systems, 2013.
[24] C. Lucchese, S. Orlando, R. Perego, F. Silvestri, and G. Tolomei. Identifying
task-based sessions in search engine query logs. In WSDM 2011.
[25] R. Mehrotra, P. Bhattacharya, and E. Yilmaz. Characterizing users' multi-
tasking behavior in web search. In CHIIR 2016.
[26] Mei, Zhou, and Church. Query suggestion using hitting time. In ACM
CIKM 2008.
[27] Q. Mei, H. Fang, and C. Zhai. A study of poisson query generation model
for information retrieval. In SIGIR 2007.
[28] D. Morris. Searchbar: a search-centric web history for task resumption
and information re-nding. In CHI 2008.
[29] D. Newman, J. H. Lau, K. Grieser, and T. Baldwin. Automatic evaluation of
topic coherence. In NAACL 2010.
[30] O'Connor, Krieger, and Ahn. Tweetmotif: Exploratory search and topic
summarization for twitter. In ICWSM 2010.
References

29. [31] P. Pecina. Lexical association measures and collocation extraction.
Language resources and evaluation, 2010.
[32] F. Radlinski and T. Joachims. Query chains: learning to rank from implicit
feedback. In KDD 2005.
[33] Segal, Koller, and Ormoneit. Probabilistic abstraction hierarchies. NIPS
2002.
[34] Silverstein and Marais. Analysis of a very large web search engine query
log. In SIGIR Forum 1999.
[35] A. Singla, R. White, and J. Huang. Studying trailnding algorithms for
enhanced web search. In SIGIR 2010.
[36] Song, Liu, and Wang. Automatic taxonomy construction from keywords.
In Proceedings of the 18th ACM SIGKDD 2012.
[37] Spink, Koshman, Park, Field, and Jansen. Multitasking web search on
vivisimo. com. In ITCC 2005.
[38] P. Vakkari. Task-based information searching. Annual review of
information science and technology, 2003.
[39] H. Wang, X. Song, R. W. White, and W. Chu. Learning to extract cross-
session search tasks. In WWW 2013.
[40] White, Bennett, and Dumais. Predicting short-term interests using
activity-based search context. In CIKM 2010.
References

30. •Deadline: 30th November 2017
•Notification: 15th December 2017
•Workshop: 9th February 2018
aka.ms/wsdm2018-learnir-workshop