Web search queries for which there are no clicks are referred to as abandoned queries and are usually considered
as leading to user dissatisfaction. However, there are many
cases where a user may not click on any search result page
(SERP) but still be satised. This scenario is referred to
as good abandonment and presents a challenge for most ap-
proaches measuring search satisfaction, which are usually
based on clicks and dwell time. The problem is exacerbated
further on mobile devices where search providers try to in-
crease the likelihood of users being satised directly by the
SERP. This paper proposes a solution to this problem us-
ing gesture interactions, such as reading times and touch
actions, as signals for dierentiating between good and bad
abandonment. These signals go beyond clicks and charac-
terize user behavior in cases where clicks are not needed to
achieve satisfaction. We study different good abandonment
scenarios and investigate the dierent elements on a SERP
that may lead to good abandonment. We also present an
analysis of the correlation between user gesture features and
satisfaction. Finally, we use this analysis to build models to
automatically identify good abandonment in mobile search
achieving an accuracy of 75%, which is significantly better
than considering query and session signals alone. Our fundings have implications for the study and application of user
satisfaction in search systems.
๐คCall On 7877925207 ๐ค Ahmedguda Call Girls Hot Model With Sexy Bhabi Ready Fo...
ย
Detecting Good Abandonment in Mobile Search
1. Detecting Good Abandonment
in Mobile Search
Kyle Williams Julia Kiseleva Aidan C. Crook
Imed Zitouni Ahmed Hassan Awadallah Madian Khabsa
Pennsylvania State University
Eindhoven University of Technology
Microsoft
WWWโ16, Montrรฉal, Quรฉbec, Canada
3. Mobile Search
โข More and more popular: 2008 ๏ 31% 2013 ๏ 63%
โข Mobile Search differs from traditional search [Human et. al, 2009]
โข On Mobiles users are satisfied by the SERP [Li et. al, 2009]
โข Mobiles screen is much smaller
โข Mobiles are used on the way
4. Mobile Search
โข More and more popular: 2008 ๏ 31% 2013 ๏ 63%
โข Mobile Search differs from traditional search [Human et. al, 2009]
โข On Mobiles users are satisfied by the SERP [Li et. al, 2009]
โข Mobiles screen is much smaller
โข Mobiles are used on the way
Search Engines need to adapt
And to Evaluate!
9. Evaluating User Satisfaction
โข We need metrics to evaluate user satisfaction
โข Good abandonment [Human et. al, 2009]:
Mobile: 36% of abandoned queries in were likely good
Desktop: 14.3%
โข Traditional methods use implicit signals: clicks and dwell time
10. Evaluating User Satisfaction
โข We need metrics to evaluate user satisfaction
โข Good abandonment [Human et. al, 2009]:
Mobile: 36% of abandoned queries in were likely good
Desktop: 14.3%
โข Traditional methods use implicit signals: clicks and dwell time
Donโt work
11. Our Main Research Problem
In the absence of clicks, what is the relationship
between a user's gestures and satisfaction and can we
use gestures to detect satisfaction and good
abandonment?
12. Research Questions
โข RQ1: What SERP elements are the sources of good
abandonment in mobile search?
โข RQ2: Do a user's gestures provide signals that can be used
to detect satisfaction and good abandonment in mobile
search?
โข RQ3: Which user gestures provide the strongest signals for
satisfaction and good abandonment?
13. Research Questions
โข RQ1: What SERP elements are the sources of good
abandonment in mobile search?
โข RQ2: Do a user's gestures provide signals that can be used
to detect satisfaction and good abandonment in mobile
search?
โข RQ3: Which user gestures provide the strongest signals for
satisfaction and good abandonment?
USERSTUDY
14. Research Questions
โข RQ1: What SERP elements are the sources of good
abandonment in mobile search?
โข RQ2: Do a user's gestures provide signals that can be used
to detect satisfaction and good abandonment in mobile
search?
โข RQ3: Which user gestures provide the strongest signals for
satisfaction and good abandonment?
USERSTUDY
CROWDSOURCING
15. User Study Participants
75%
25%
GENDER
Male Female
55%
45%
LANGUAGE
English Other
82%
8%
2%
8%
EDUCATION
Computer Science Electrical Engineering
Mathematics Other
โข 60 Participants
โข 25.53 +/- 5.42 years
16. User Study Design
โข Video Instructions (same for all participants)
โข Tasks:
1. A conversion between the imperial and metric systems
2. Determining if it was a good time to phone a friend in another
part of the world
3. Finding the score from a recent game of the userโs favorite
sports team
4. Finding the user's favorite celebrity's hair color
5. Finding the CEO of a company that lost most of its value in the
last 10 years
17. Find out what is
the hair color of
your favorite
celebrity
18. Questionnaire
โข Were you able to complete the task?
o Yes/No
โข Where did you find the answer?
o Answer Box, Image, SERP, Visited Website
โข Which query led you to finding the answer?
o First, Second, Third, >= Fourth
โข How satisfied are you with your experience in this task?
o 5-point Likert scale
โข Did you put in a lot of effort to complete the task?
o 5-point Likert scale
19. Questionnaire
โข Were you able to complete the task?
o Yes/No
โข Where did you find the answer?
o Answer Box, Image, SERP, Visited Website
โข Which query led you to finding the answer?
o First, Second, Third, >= Fourth
โข How satisfied are you with your experience in this task?
o 5-point Likert scale
โข Did you put in a lot of effort to complete the task?
o 5-point Likert scale
5 Tasks
~20 Minutes
20. User Study Data
โข Total queries โ 607 ๏ 563
โข Abandoned queries โ 576 ๏ 461
โข Potential abandonment tasks โ 274
21. User Study Data
โข Total queries โ 607 ๏ 563
โข Abandoned queries โ 576 ๏ 461
โข Potential abandonment tasks โ 274
Binary
Labels
22. Crowdsourcing Procedure
Random sample of abandoned queries from the search logs of a
personal digital assistant during one week in June 2015 (no query
suggestion)
24. Crowdsourcing Data
โข Total amount of queries โ 3,895
โข Judgments agreement (3 per one query) โ 73%
โข After filtering: SAT โ 1,565 and DSAT โ 1,924
27. Query and Session Features
โข Session duration
โข Number of queries in session
Session
Features
28. Query and Session Features
โข Session duration
โข Number of queries in session
โข Index of query within session
โข Time to next query
โข Query length (number of words)
โข Is this query a reformulation
โข Was this query reformulated
Session
Features
Query
Features
29. Query and Session Features
โข Session duration
โข Number of queries in session
โข Index of query within session
โข Time to next query
โข Query length (number of words)
โข Is this query a reformulation
โข Was this query reformulated
โข Click count
โข Number of SAT clicks (> 30 sec)
โข Number of back-click clicks (< 30 sec)
Session
Features
Query
Features
Click
Features
30. Baseline 1:Click & Dwell
โข Session duration
โข Number of queries in session
โข Index of query within session
โข Time to next query
โข Query length (number of words)
โข Is this query a reformulation
โข Was this query reformulated
โข Click count
โข Number of SAT clicks (> 30 sec)
โข Number of back-click clicks (< 30 sec)
Session
Features
Query
Features
Click
Features
Click >
30 sec
No
Refomul
ation
B1:Click,Dwellwith
noReformulation
31. Baseline 2: Optimistic
โข Session duration
โข Number of queries in session
โข Index of query within session
โข Time to next query
โข Query length (number of words)
โข Is this query a reformulation
โข Was this query reformulated
โข Click count
โข Number of SAT clicks (> 30 sec)
โข Number of back-click clicks (< 30 sec)
Session
Features
Query
Features
Click
Features
NO
Click
NO
Refomul
ation
B2:Optimistic
32. Baseline 3: Query-Session Model
โข Session duration
โข Number of queries in session
โข Index of query within session
โข Time to next query
โข Query length (number of words)
โข Is this query a reformulation
โข Was this query reformulated
โข Click count
โข Number of SAT clicks (> 30 sec)
โข Number of back-click clicks (< 30 sec)
Session
Features
Query
Features
Click
Features
B3:Query-SessionModel:
TrainingRandomForest
33. Gesture Features (1)
โข Viewport features swipes-related:
o up swipes and down swipes
o changes in swipe direction
o swiped distance in pixels and average swiped distance
o swipe distance divided by time spent on the SERP
34. Gesture Features (1)
โข Viewport features swipes-related:
o up swipes and down swipes
o changes in swipe direction
o swiped distance in pixels and average swiped distance
o swipe distance divided by time spent on the SERP
โข Time To Focus
o Time to focus on Answer
o Time to Focus on Organic Search Results
35. 3 seconds 6 seconds
33% of
ViewPort
66% of
ViewPort
ViewPortHeight
2 seconds
20% of
ViewPort
1s 4s 0.4s 5.4s+ + =
GF(2): Attributed Reading Time
37. Models: Detecting Good Abandonment
M1: Gesture Model:
Training Random Forest based on gesture features
M2: Gesture Model + Query and Session Features:
Training Random Forest based on gesture, query and session features
38. RQ2: Are gestures useful? (1)
On only abandoned user study data:
148 SAT queries and 313 DSAT queries
39. RQ2: Are gestures useful? (2)
On crowdsourced data:
1565 SAT queries and 1924 DSAT queries
40. RQ2: Are gestures useful? (3)
On all user study data:
179 SAT queries and 384 DSAT queries
Gestures Features are useful to detect user satisfaction
in general!
41. Conclusions
โข RQ1: What SERP elements are the sources of good abandonment in
mobile search?
Answer, Images and Snippet
โข RQ2: Do a user's gestures provide signals that can be used to detect
satisfaction and good abandonment in mobile search?
Yes
โข RQ3: Which user gestures provide the strongest signals for satisfaction
and good abandonment
Time spent interacting with Answers is positively correlated. Swipe
actions and time spent with SERP is negatively correlated
42. โข Answer, Images and Snippet are
potentially source of the good
abandonment
โข User gestures provide useful signals to
detect good abandonment
โข Time spent interacting with Answers is
positively correlated. Swipe actions
and time spent with SERP is
negatively correlated
Questions?
Editor's Notes
what will the weather be like tomorrow? What time does the movie start tonight?
Or what year was a celebrity born?
Many of these types of questions can be answered by search engines without users needing to click on search results
later. We nd
strong signicant negative correlation of -0.65 between sat-
isfaction and eort, and a negative correlation of -0.08 be-
tween completion and eort, indicating that less eort leads
to more satisfaction and higher completion rates.