Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
1 of 229

Related Books

Free with a 30 day trial from Scribd

See all

The User is the Query - The Rise of Predictive Proactive Search

  1. 1. The User is The Query: The Rise of Predictive Proactive Search
  2. 2. “Today you are you! That is truer than true! There is no one alive who is you-er than you!” (Dr Seuss)
  3. 3. Said Dr Seuss…& Google
  4. 4. When introducing Google Feed (now Discover)
  5. 5. Today’s topic…‘The User is The Query’
  6. 6. WHO IS DAWN ANDERSON @dawnieando dawn.anderson@@move-it- Meet Bert & Tedward
  7. 7. There’s a problem with queries, content & users too
  8. 8. “In 1998 the web consisted of just 25 million pages…” (Ben Gomez, Google, 2018)
  9. 9. “… That’s roughly the equivalent number of those in a small library” (Ben Gomez, Google, 2018)
  10. 10. In 2019… we know the web is huge… billions of web pages (Netcraft, 2019)
  11. 11. App usage is huge too - By 2018 – App Store has 20 million registered developers. (Techcrunch, 2018)
  12. 12. 42% of the global population use social media (Emarsys, 2019)
  13. 13. We are competing with programmatic solutions spraying content & information EVERYWHERE
  14. 14. Over-connection & Information Overload
  15. 15. Too much choice often has negative impacts
  16. 16. Almost 98% of visits are people window shopping Average ecommerce conversion +/- 2%
  17. 17. Despite this… users are still seeking even more information
  18. 18. Humans forage (like bears) all over the place seeking information… we are informavores
  19. 19. Researching ALL THE THINGS… before making final decisions
  20. 20. The number of Google searches increases year on year (Internetlivestat, 2018, curation from various sources)
  21. 21. 15% of queries every day are new (Google)
  22. 22. We have become very good at filtering out things which are NOT interesting enough (8 second filter)
  23. 23. It’s not a short attention- span thing
  24. 24. Otherwise we would not binge on ‘Stranger Things’
  25. 25. This is cognitive load management & information filtering
  26. 26. AT THE SAME TIME words are problematic. Ambiguous… polysemous… synonymous
  27. 27. In spoken word it is even worse because of homophones and prosody
  28. 28. Like “four candles” and “fork handles”
  29. 29. Which does not bode well for the likes of conversational search
  30. 30. In query understanding sometimes users don’t know what they want either
  31. 31. Sometimes exactly the same users express an information need in a different way
  32. 32. Sometimes different users use lots of different ways to mean exactly the same thing
  33. 33. 'The Vocabulary Problem’ Furnas, G.W., Landauer, T.K., Gomez, L.M. and Dumais, S.T., 1987. The vocabulary problem in human-system communication. Communicatio ns of the ACM, 30(11), pp.964- 971. 1987
  34. 34. One of the inventors of ‘Latent Semantic Indexing’, created to solve ‘The Vocabulary Problem’ whilst researching at Bellcore (1990)
  35. 35. BTW… No-one said LSI was used by Google (aside)
  36. 36. Often words have multiple meanings. Like “like” can be 5 possible parts of speech (POS)
  37. 37. Sometimes the searcher query is a ‘cold start’ query
  38. 38. Broad or cold start queries might call for result diversification due to lack of intent detection
  39. 39. Search engines may return a broad blend of results to match these queries Freshness Serendipity Novelty Diversity
  40. 40. AKA Result Diversification
  41. 41. The searcher has to click around to provide feedback on their intent or reformulate the query by entering something else (‘query refinement’)
  42. 42. To then deliver sequential queries with greater intent understanding
  43. 43. Human in the loop
  44. 44. Query refinement says… “Your move next”
  45. 45. And maybe… MARKOV CHAINS Where the results are dependent on the previous step ov_chain
  46. 46. Like RPG (Role Playing Games) Where choices made determine the next choices given
  47. 47. People also ask / related queries are a special kind of ‘Query Refinement’
  48. 48. A kind of ‘probability-driven fork in the road’ sadikov.pdf (Sadikov et al, 2010) CLUSTERING QUERY REFINEMENTS BY USER INTENT
  49. 49. BUT word’s meaning & user intent /context combined are still very hard to understand for search engines
  50. 50. Despite huge leaps forward in query classification & natural language understanding
  51. 51. Accelerated mostly by Google’s BERT
  52. 52. Glue Benchmark Leaderboard
  53. 53. Superglue Benchmarks
  54. 54. MS MARCO
  55. 55. Stanford Question And Answer Dataset 2.0 • Rajpurkar, P., Zhang, J., Lopyrev, K. and Liang, P., 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
  56. 56. Time (& Space) context shifts query intent further
  57. 57. The exact same queries have different intent at different times & different locations
  58. 58. Let’s Take The Query [Easter]
  59. 59. Modeling & Predicting Behavioural Dynamics on The Web (Radinsky et al, 2012)
  60. 60. “When users’ information needs change over time, the ranking of results should also change to accommodate these needs.” (Radinsky, 2013)
  61. 61. This is ‘Query Intent Shift’
  62. 62. Your ranking flux might well be shifting query intents at scale
  63. 63. The intent of queries changes over time
  64. 64. Thought: The Mobile-First Index is built For mobile first and mobile is probably ‘VERY’ query dynamic, local & temporal (shifty)
  65. 65. The passage of time adds new meaning to queries sometimes too
  66. 66. The rise and fall of the Blackberry?
  67. 67. At certain times far more intents will be transactional
  68. 68. And sometimes only reasons a particular audience would understand spike temporal queries
  69. 69. [Four candles] + [fork handles] interest over time
  70. 70. Sometimes it is other events which trigger unexpected queries
  71. 71. What… A… Nightmare Queries Are
  72. 72. Maybe It’s Time For A Change?
  73. 73. Enter… The Next 20 Years of Search
  74. 74. Hmm… That sounds big Google… This is HUGE
  75. 75. Three FUNDAMENTAL shifts in search
  76. 76. 1. The shift from answers to journeys 2. The shift from queries to query-less 3. The shift from text to visual information
  77. 77. The shift from text to more visual information is simple
  78. 78. This is cognitive load management & information filtering
  79. 79. Images are much easier to mentally consume than text & audio
  80. 80. Images & video engage… Images & video entertain Images & video provoke emotion
  81. 81. Photography app usage had a 210% increase between 2016 and 2018 according to App Annie
  82. 82. People spend on average 2.6x more time on pages with video
  83. 83. Image search is curation. Totally different to text-based search
  84. 84. Think accessibility first with images & videos. You won’t go far wrong (alt / caption / file / description / title)
  85. 85. Go nuts with quality images & video
  86. 86. This feels like a huge UX / accessibility shift… Hoorah
  87. 87. What about the switch to journeys and query-less?
  88. 88. “Easier if we can model: who is asking, what they have done in the past, where they are, when it is, etc.” (Susan Dumais, CIKM, 2016)
  89. 89. “Queries Are Difficult To Understand in Isolation” (Susan Dumais, Microsoft Research, 2016)
  90. 90. AKA - Contextual Search = User + Time + Location + Device + Task
  91. 91. Better still… what about predicting the user’s informational needs to proactively make suggestions
  92. 92. “Nevertheless, as the world is becoming more mobile-centric, this old-fashioned query-driven search scenario and clickbased evaluation mechanism can no longer catch up with the rapid evolution of user demand on mobile devices.” (Song and Guo,2016 (Microsoft Research))
  93. 93. “”Therefore,a more user-friendly, mobile-centric and scenario driven search paradigm that requires minimal user inputs is ready to come out” (Song and Guo,2016 (Microsoft Research))
  94. 94. Zero-Query Queries – No Query Required
  95. 95. Personalising Search via Interests & Activities 2005 paper awarded the 2017 SIGIR Test of Time Award. Cited 1029 times to date Teevan, J., Dumais, S.T. and Horvitz, E., 2005, August. Personalizing search via automated analysis of interests and activities. In Proceedings of the 28th annual international ACM
  96. 96. QueryLess: Next Gen Proactive Search And Recommender Engines (2016)
  97. 97. It kind of sounds like Google Discover
  98. 98. At last announcement Google Discover had 800 million users (May, 2018)
  99. 99. And it’s on the home page of the mobile browser now too... How many users now?
  100. 100. It knows you… and the things you do… where you’ve been… where you’re going
  101. 101. Mobile Device Sensors (14 sensors or more) Proximity sensors GPS sensor Ambient light sensor Accelerometer Compass Gyroscope Back illuminated sensor
  102. 102. ‘The User (needs) is The Query’
  103. 103. This is Task-Driven Search & Recommender Systems
  104. 104. Google’s Recommender Systems
  105. 105. Google Scholar is now a Recommender System Too
  106. 106. QueryLess: Next Gen Proactive Search And Recommender Engines
  107. 107. “Patterns were spotted about repetitive task driven search behaviours – predictable” (Song & Guo, 2016)
  108. 108. “Predictable task timeline patterns are more prevalent on mobile devices” (Song & Guo, 2016)
  109. 109. Like e.g. ‘checking the stock market’ every morning if you’re interested in stocks and shares
  110. 110. People are creatures of habit it seems
  111. 111. “In many cases predicting informational needs removes the need for the query & reactive search engine” (Song & Guo, 2016)
  112. 112. Google Discover looks to be focusing on hobbies, interests, news and social activities
  113. 113. An information need is rarely a task with a single finite item
  114. 114. It’s more like a series of little chunks (sub-tasks)
  115. 115. Tasks & timelines go hand in hand… it seems
  116. 116. Many tasks & intents can be modelled according to predicted patterns
  117. 117. Very Recent Microsoft Research
  118. 118. The Ideal is Personalisation • Not easy to achieve fully • Sparsity of data • Privacy concerns • Broken sequences
  119. 119. In the absence of personalization… collaborative Filtering
  120. 120. There are other people nearly like you
  121. 121. You (and me) are unique… but may be similar
  122. 122. Matrix Factorisation (Netflix Recommendation System) + Matrix Factorisation (WALS Algorithm, Tensorflow)
  123. 123. Tensorflow Matrix Factorisation
  124. 124. Based on users liking the same things (with hidden common preferences)
  125. 125. Those sharing similar interests likely share other hidden interests too (i.e. the system does not know of them yet)
  126. 126. Understand the user, understand their cohort… Understand other similar informational needs
  127. 127. Google Discover ‘Topics’
  128. 128. Modelling cohorts
  129. 129. Advances in Machine Learning
  130. 130. Reinforcement learning thrives from rewards (implicit feedback)
  131. 131. Contextual Bandit Algorithms
  132. 132. YouTube is a Recommender System
  133. 133. YouTube Feedback Controls is ‘The Human in The Loop’
  134. 134. Progressive personalisation
  135. 135. Toward a Personal Knowledge Graph
  136. 136. The two sides of assistant will both be proactive Provide answers / search Conversation Search Help with activities / tasks Conversation Actions
  137. 137. Extend Actions on Google using Machine Learning
  138. 138. Understand your customers to assist with AI Perceived Information need Micro-task Micro-task Micro-task Micro-task Micro-task Task Micro-task Micro-task Micro-task Micro-task Task Micro-task Micro-task Task Micro-task Micro-task Micro-task Task Micro-task Micro-task Task Micro-task Task We can identify the user’s probable top tasks & subtasks Identify their needs & what info they need along the way
  139. 139. Tell us about the tasks, order and steps involved in booking a hotel
  140. 140. Many built-in intents & many ‘coming soon’
  141. 141. Connections Between Things
  142. 142. Multi-platforming • Switching between search and video • Between search and a recommender system
  143. 143. Connecting Tasks Across Devices & Applications
  144. 144. Truly PERSONAL AI is not possible without a PERSONAL KNOWLEDGE GRAPH (Krisztian Balog, ECIR 2019)
  145. 145. Building a Personal Knowledge Graph
  146. 146. A Recent Microsoft Personal Knowledge Graph Patent
  147. 147. Assistant + Home + Discover + Search App + Desktop + Location Tracker + Calendar + Gmail + YouTube
  148. 148. In your car
  149. 149. In your console
  150. 150. Carrier’s for Recommender Systems
  151. 151. Where the user is truly ‘the query’
  152. 152. Toward An Audience of One
  153. 153. What Can SEOs Do About This?
  154. 154. Realise… your ranking tools are mostly wrong
  155. 155. Identify interests & affinity groups
  156. 156. Map every single informational need sub- task you can think of to the sections of a model like the RACE model
  157. 157. Build task timeline clusters
  158. 158. Map & cluster ‘Related’ content by task type. Categories are too broad, and topics may be too
  159. 159. Contextual Order Matters
  160. 160. Instead of continually creating Moooaaaarrr content, make what you have better
  161. 161. Continually update and improve on solid URL evergreen content
  162. 162. Continually update and improve on solid URL seasonal temporal content
  163. 163. Map content clearly to tasks & task timelines
  164. 164. Maximise the ‘Local’ Opportunity
  165. 165. Think “CRM for SEO”
  166. 166. Identify predictable patterns of user behavior
  167. 167. The QueryLess change will not come overnight… things move slowly
  168. 168. Take-aways
  169. 169. Go • Go big on evergreen content & keep updated Optimise • Optimise images well – think curation / collections Map • Map user journeys to content plans Optimise • video well – enhance with markup / transcription Get • Get personal – keep refining segments / personas Identify • Identify & cluster content around task timelines Use • Use relatedness across content, tasks & temporality
  170. 170. Keep in Touch •@dawnieando •@BeBertey
  171. 171. References
  172. 172. You may need a dual or multi- armed content strategy 2%
  173. 173. Book hotel intent When do you want to stay? dates dates How many nights? 3 nights 2 nights Overnight A week Single or double room? Single room Double room Programme your own expected questions and answers
  174. 174. References • Broder, A., 2002, September. A taxonomy of web search. In ACM Sigir forum (Vol. 36, No. 2, pp. 3-10). ACM. • Chuklin, A., Severyn, A., Trippas, J., Alfonseca, E., Silen, H. and Spina, D., 2018. Prosody Modifications for Question-Answering in Voice-Only Settings. arXiv preprint arXiv:1806.03957. • HigherVisibility. 2018. How Popular is Voice Search? | HigherVisibility. [ONLINE] Available at: • Filippova, K., Alfonseca, E., Colmenares, C.A., Kaiser, L. and Vinyals, O., 2015. Sentence compression by deletion with lstms. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 360-368). • Filippova, K. and Alfonseca, E., 2015. Fast k-best sentence compression. arXiv preprint arXiv:1510.08418. • Google Developers. 2018. Content-based Actions | Actions on Google | Google Developers. [ONLINE] Available at: actions/. [Accessed 18 June 2018]
  175. 175. References • Mitkov, R., 2014. Anaphora resolution. Routledge. • NLP Department - Stanford University - Imran Q Sayed. 2018. Issues in Anaphora Resolution. [ONLINE] Available at: [Accessed 28 June 2018]. • Radlinski, F. and Craswell, N., 2017, March. A theoretical framework for conversational search. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (pp. 117-126). ACM. • Schalkwyk, J., Beeferman, D., Beaufays, F., Byrne, B., Chelba, C., Cohen, M., Kamvar, M. and Strope, B., 2010. “Your word is my command”: Google search by voice: a case study. In Advances in speech recognition (pp. 61-90). Springer, Boston, MA. • SISTRIX. 2018. Stepping out of the SEO Bubble - SISTRIX. [ONLINE] Available at: [Accessed 16 June 2018]. • Presentation at ESSIR2017 on work by Radinsky, K., Svore, K.M., Dumais, S.T., Shokouhi, M., Teevan, J., Bocharov, A. and Horvitz, E., 2013. Behavioral dynamics on the web: Learning, modeling, and prediction. ACM Transactions on Information Systems (TOIS), 31(3), p.16.
  176. 176. References • The Stanford Question Answering Dataset. 2018. The Stanford Question Answering Dataset. [ONLINE] Available at: • Trippas, J.R., Spina, D., Cavedon, L., Joho, H. and Sanderson, M., 2018. Informing the Design of Spoken Conversational Search. • reinforcement-learning-over-knowledge-graphs-a8af155e716c • Jansen, B.J., Booth, D.L. and Spink, A., 2008. Determining the informational, navigational, and transactional intent of Web queries. Information Processing & Management, 44(3), pp.1251- 1266.
  177. 177. References Radinsky, K., Svore, K.M., Dumais, S.T., Shokouhi, M., Teevan, J., Bocharov, A. and Horvitz, E., 2013. Behavioral dynamics on the web: Learning, modeling, and prediction. ACM Transactions on Information Systems (TOIS), 31(3), p.16 Sadikov, E., Madhavan, J. and Halevy, A., Google LLC, 2013. Clustering query refinements by inferred user intent. U.S. Patent 8,423,538. Official Google Webmaster Central Blog. 2019. Official Google Webmaster Central Blog: Rolling out mobile-first indexing . [ONLINE] Available at: indexing.html. [Accessed 25 September 2019]. Zhou, S., Cheng, K. and Men, L., 2017, April. The survey of large-scale query classification. In AIP Conference Proceedings (Vol. 1834, No. 1, p. 040045). AIP Publishing.
  178. 178. References Search Engine Land. 2019. Starting July 1, all new sites will be indexed using Google's mobile-first indexing - Search Engine Land. [ONLINE] Available at: first-indexing-317490. [Accessed 25 September 2019]. Teevan, J., Dumais, S.T. and Horvitz, E., 2005, August. Personalizing search via automated analysis of interests and activities. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 449-456). ACM. Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R. and Deng, L., 2016. MS MARCO: A Human-Generated MAchine Reading COmprehension Dataset.
  179. 179. Appendix
  180. 180. The task (query mission) tied to the page type matters hugely
  181. 181. Understand the shared preferences, learn the hidden preferences
  182. 182. Bias & Reproducibility is a Challenge
  183. 183. Bias Considerations Presentation Bias Programming Bias Audience Manipulated Bias (e.g fake reviews) Machine Learning / AI Bias (Black box algorithms) Matthew’s Law Zipfian Distribution of Web Content
  184. 184. Spotify add novelty items to home page to avoid biased personalisation
  185. 185. Do yourself a favour and follow Mounia Lalmas @mounialalmas
  186. 186. And this polar bear
  187. 187. Bias on the web and recommender systems
  188. 188. NoBIAS Project
  189. 189. Reproducibility problems in research & RecSys (very high)
  190. 190. MS MARCO Paper • Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R. and Deng, L., 2016. MS MARCO: A Human- Generated MAchine Reading COmprehension Dataset.
  191. 191. Query Classifications perhaps? - There are some we know of already
  192. 192. A Taxonomy of Web Search (Broder, 2002) Informational Navigational Transactional
  193. 193. Google’s Quality Raters Guide simplifies & extends these Know query == Informational Website query == Navigational Do query == Transactional Visit in person == Local intent
  194. 194. There are also several types of queries too (Krisztian Balog, ECIR, 2019) Keyword queries (Normal keyword queries) Keyword++ queries (Faceted / filtered queries) Zero-Query queries (User is the query) Natural language queries Structured queries (e.g. SQL)
  195. 195. 80% of all queries are information al in nature (Jansen et al, 2008) 80% 10% 10% Query Intent Split Informational Transactional Navigational
  196. 196. Temporal Dynamic Intent (Burstiness) is a huge factor for intent
  197. 197. Broder et al 39 - 48% Informational 20 – 25% Navigational 30 – 36% Transactional
  198. 198. “dresses”, “shoes”, “bags” “buy dresses”, “buy shoes”, “buy bags”, “dress sales”, “shoe sales” Really means
  199. 199. Multi-armed Bandit Algorithms
  200. 200. Google now processes over 40,000 search queries every second on average (Source: Dubious Internetlivestats estimation)
  201. 201. And local intent considerations
  202. 202. Another Great ‘Ronnies’ Sketch BTW
  203. 203. ‘iPhone’ – Query Example (Google Quality Raters Guidelines)
  204. 204. IR researchers competing over natural language understanding
  205. 205. What did you really mean when you searched for ‘Easter’? • Radinsky, K., Svore, K.M., Dumais, S.T., Shokouhi, M., Teevan, J., Bocharov, A. and Horvitz, E., 2013. Behavioral dynamics on the web: Learning, modeling, and prediction. ACM Transactions on Information Systems (TOIS), 31(3), p.16. When did you search for ‘Easter’? A few weeks before Easter A few days before Easter During Easter What you mostly meant When is Easter? Things to do at Easter What is the meaning of Easter?
  206. 206. The problem is consistent high precision is nowhere in sight
  207. 207. And if we are to move into multi-device ubiquitious search then…
  208. 208. Accuracy of results is more important than quantity
  209. 209. Paraphrase handling on ‘Actions’ appears to be programmable
  210. 210. Gets Round The Vocabulary & Disambiguation Problem
  211. 211. Hotel Booking Dialogflow
  212. 212. 2017 2017 2017
  213. 213. Understand your customers to assist with AI Customer Service Data Customer Panels Email questions FAQs Build Assistant App
  214. 214. Dialogflow
  215. 215. Extend Actions on Google using Machine Learning
  216. 216. Think about the different features which matter to users more dependent on the domain News (freshness) Jobs (salary, job title, location) Restaurants (location, cuisine) Shopping (price)
  217. 217. By 2022 PCs will account for only 19 percent of IP traffic (Comscore, 2019)
  218. 218. We know there are various versions of query intent classifications (Broder, 2002; Rose; Jansen, 2008)
  219. 219. Interest over time for Google Home & Amazon Alexa
  220. 220. App usage is also huge