http://www-users.cs.umn.edu/~qian/publication.html

1. Gaze Prediction for
Recommender Systems
Qian Zhao, Shuo Chang, F. Max Harper,
Joseph A. Konstan
1

2. Why gaze?
2

3. 3

4. Why gaze: users are thinking a lot!
» Go beyond normally logged ratings and
actions
» Understand what users are thinking
• Choice/decision making research
• Cognitive modeling
4

5. Why gaze?
» Understand user inaction
• Do users see the displayed items?
» Problem with machine learning
assumptions
• Are positive training instances really
positive?
• Are negative training instances really
negative?
5

6. However
» Eye tracking is not widely used (and in
the near future)
» Eye tracking may not be widely used.
6

7. Let’s model and predict gaze
7

8. Aggregated Fixation Prediction
» Consider browsing one page in a grid-
based interface (r rows * c columns)
» Aggregating entire page browsing,
predict each displayed item’s
• fixation probability
• fixation time
» Given user browsing data
• item positions, page dwell time, user
actions (top-down vs. bottom-up)
8

9. Two scenarios
» Training models only based user
browsing data
» Training models with both
• user browsing data
• eye tracking data from a small number of
users
• Evaluation
• Extrapolation (across users)
9

10. Data Sets - MovieLens
» November 2015, user browsing data
• 102K page views
» 17 subjects’ eye tracking data, each
recorded for ~30 mins
• 452 page views, 10K data points
• Tasks: free using, rating, finding movies
etc.
10

11. Building Linear Models
» Logistic regression for fixation
probability
» Hurdle linear models for fixation time
» Features
• Position: row index and column index
• Dwell time
• 1/minActionDist
11

12. Building HMM
12
» Fixation (latent or observable): F
• r * c possible values
» Action (latent or observable): A
• r * c + 1 possible values

13. Building HMM
13
» Estimation
• with eye tracking data (MLE)
• with only browsing data (EM or Appr.)
» Prediction based on posterior of F

14. Evaluation
» Randomly pick 20% of the 17 subjects
for testing, others for training
» Repeat for 100 times but always using a
different set of testing subjects
(independence)
14

15. Fixation Probability – AUC
» Collecting eye tracking data greatly
helps and it extrapolates across users.
» HMM is better than linear models
15
Action stats Linear models HMM
Training with
only browsing
data
0.580 N.A. 0.693
Training with
eye tracking and
browsing data
- 0.757 0.823

16. Fixation Time – MAE
16
» Collecting eye tracking data significantly
helps.
» Hurdle linear model is better than HMM
» R-squared: 21%
Action stats Linear models HMM
Training with
only browsing
data
0.466 N.A. 0.520
Training with
eye tracking and
browsing data
- 0.332 0.488

17. F-pattern (vs. center effect)
17

18. Messages from this talk
» Gaze prediction extrapolates across
users!
• Collecting eye tracking data from a small
number of users greatly help.
» Applying the right models makes a
significant difference.
• HMMs for fixation probability.
• Hurdle linear models for fixation time.
» F-pattern instead of center effect
18

19. Thanks! Questions?
» Title: Gaze Prediction for Recommender
Systems
• See the paper for more results on other HMM
models and prediction for different user
tasks!
» Authors: Qian Zhao, Shuo Chang, F. Max
Harper, Joseph A. Konstan
» Contact
• zhaox331@umn.edu
• http://www-users.cs.umn.edu/~qian/
19