Slides from the presentation at ACM UbiComp 2015 When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage (best-paper award). Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver UbiComp’ 15: ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015.

1. When Attention is not Scarce
Detecting Boredom from Mobile Phone Usage
Research
UbiComp ‘15, Osaka, Japan
Martin
Pielot
Telefonica
Research
Tilman
Dingler
University of
Stuttgart
Jose
San Pedro
Telefonica
Research
Nuria
Oliver
Telefonica
Research

2. times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0
War on
Attention*
* http://www.forbes.com/sites/onmarketing/2012/10/19/the-attention-war/

3. Revenue per active user
$45 in Q1 2014 = 50 cents per day

4. SocialMediaCube. Yoel Ben-Avraham. Apr 8, 2013 via Flickr. CC BY-ND 2.0
The trade we make:
Our attention so they can pay their bills

6. Our engagement is now defined by push-driven
notifications rather than the traditional pull-driven
experience. We’re “hunting and pecking” through our
app grid a lot less; the apps that notify us (without over-
notifying to the point of uninstall) are rewarded with our
engagement (and our dollars).

7. Example: Push-Driven Notifications

8. ‚Attention is a limited resource—a person
has only so much of it ‘
[Matthew B. Crawford]
Attention Economy: treating human
attention as a scarce commodity
[Davenport and Beck, 2001]
times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0

9. Wild-West Land-Grab Phase
“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain

10. Overload
“Ahhhhhhh” by Kenny Louie, Jun 06, 2010, via Flickr, CC BY 2.0

11. From “Banner Blindness: New and Old Findings” by Jakob Nielsen on August 20, 2007
Banner Blindness

12. Overload
“Ahhhhhhh” by Kenny Louie, Jun 06, 2010, via Flickr, CC BY 2.0
Notification blindness

13. Wild-West Land-Grab Phase
“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain
If the trade
attention for free services
is to be sustained
we need to better
protect mobile phone users

14. Boredom as part of the solution

15. Attention is not always scarce
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

16. Attention is not always scarce
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
Boredom displeasure caused by “lack of stimulation”
[Fenichel, 1951]
“a bored person is not just someone who does not have anything
to do; it’s someone who is actively looking for stimulation”
[Eastwood, 2002]

17. Attention is not always scarce
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
Boredom displeasure caused by “lack of stimulation”
[Fenichel, 1951]
“a bored person is not just someone who does not have anything
to do; it’s someone who is actively looking for stimulation”
[Eastwood, 2002]
Mobile phones are a commonly used tool to kill time when bored
[Brown et al. 2014]

22. Attention is not always scarce
Mobile phones are a commonly used tool to
fill or kill time when bored [Brown et al. 2014]
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
Boredom displeasure caused by “lack of stimulation”
[Fenichel, 1951]
“a bored person is not just someone who does not have anything
to do; it’s someone who is actively looking for stimulation”
[Eastwood, 2002]
If phones knew when their users are killing time
maybe they could suggest them to make better use
of the moment

23. How well can we
detect boredom from
mobile phone usage
patterns?

24. Borapp – Sensor-Data Collection
Always
collected
Only
collected if
phone in use
Sensor Description
Battery Status Battery level ranging from 0-100%
Notifications Time and type (app) of notification
Screen Events Screen turned on, off, and unlocked
Phone Events Time of incoming and outgoing calls
Proximity Screen covered or not
Ringer Mode Silent, Vibration, Normal
SMS Time of receiving, reading, and sending SMS
Sensor Description
Airplane Mode Whether phone in airplane mode
Ambient Noise Noise in dB as sensed by the microphone
Audio Jack Phone connected to headphones or speakers
Cell Tower The cell tower the phone is connected to
Data Activity Number of bytes up/downloaded
Foreground app Package name of the app in foreground
Light Ambient light level in SI lux units
Screen Orient Portrait or Landscape mode
Wifi Infos The WiFi network the phone is connected to

25. Experience Sampling
“Right now, I feel bored”
[5-point Likert scale]
Min. 6 times per day
Preferably triggered when phone in use
Borapp – Experience Sampling

26. Data Collection
54 Participants
aged 21 – 46 (M = 30.6) years
11 female, 23male, 19 not disclosed
For two weeks in July 2014
Over 40M sensor log entries
4398 valid self-reports of boredom

27. 0
500
1000
1500
0 1 2 3 4
Frequency
Agreement to "Right now, I feel bored"
0 = disagree, 4 = agree

28. 0
500
1000
1500
0 1 2 3 4
Frequency
Agreement to "Right now, I feel bored"
0 = disagree, 4 = agree
Absolute ground truth
Bored: ratings 3, 4
446 (10.1%) instances

29. Absolute ground truth
Bored: ratings 3, 4
446 (10.1%) instances
Normalized ground truth
Z-score per person
Bored: z > 0.25
1518 (34.5%) instances
0
400
800
1200
1600
2000
-2 -1 0 1 2
Frequency
Normalized Subjective Boredom,
(higher number = more bored than
usual)

30. Category Example Feature Explanation
Context Semantic Location Home, work, other, unknown
Demographics Age, gender 38, female
Last Communication
Activity
Time last incoming call Time passed since somebody called the participants
Usage (intensity) Bytes received Number of bytes downloaded in the last 5 minutes
Usage (externally triggered) Number of notifications Number of notifications received in the last 5 minutes
Usage (idling) Number of apps Number of apps launched in the last 5 minutes
Usage (type) Most used app App used for the most time in the last 5 minutes.
35 Features, 7 Categories

31. RQ1: how well can phones detect killing-
time boredom events from these usage
patterns?
RQ2: which usage patterns are related to
killing time with the phone?
RQ3 is the model good enough to be
useful?

32. 74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance | Random Forest (AUCROC)

33. 76.5%
82.5%
74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance (AUCROC) Including Boredom Proneness
scores of 22 participants

34. 76.5%
82.5%
74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance (AUCROC) Including Boredom Proneness
scores of 22 participants
Primary data set

35. 34.7%
42.8%
48.3%
52.1%
56.6%
62.4%
66.2%
70.1%
74.3%
76.3%
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Precision
Recall
Precision: 70.1% for 30% recall,
62.4% for 50% recall

36. Boredom can be detected from
phone-usage patterns with an
accuracy of ca. 75% to 83% AUCROC
Take Away #1

37. RQ1: how well can phones detect killing-
time boredom events from these usage
patterns?
RQ2: which usage patterns are related to
killing time with the phone?
RQ3 is the model good enough to be
useful?

38.  Recency of communication activity
i.e., time since last incoming or outgoing
communication;
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower

39.  Recency of communication activity
i.e., time since last incoming or outgoing
communication;
 Phase of the day
i.e., hour of the day, ambient light
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower

40.  Recency of communication activity
i.e., time since last incoming or outgoing
communication;
 Phase of the day
i.e., hour of the day, ambient light
 Demographics,
i.e., gender and age;
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower

41.  Recency of communication activity
i.e., time since last incoming or outgoing
communication;
 Phase of the day
i.e., hour of the day, ambient light
 Demographics,
i.e., gender and age;
 General usage intensity
i.e, phone out of pocket, or time since
last phone use …;
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower

42.  Recency of communication activity
i.e., time since last incoming or outgoing
communication;
 Phase of the day
i.e., hour of the day, ambient light
 Demographics,
i.e., gender and age;
 General usage intensity
i.e, phone out of pocket, or time since
last phone use …;
 Intensity of recent usage
i.e. # of unlocks, or
# of apps launched in last 5 minutes, …
Feature Import Correlation The more bored, the ..
time_last_outgoing_call 0.0607 -0.143 less time passed
time_last_incoming_call 0.0580 0.088 more time passed
time_last_notif 0.0564 0.091 more time passed
time_last_SMS_received 0.0483 0.053 more time passed
time_last_SMS_sent 0.0405 -0.090 less time passed
time_last_SMS_read 0.0388 -0.013 more time passed
light 0.0537 -0.010 darker
hour_of_day 0.0411 0.038 later
proximity 0.0153 -0.186 less covered
gender (0=f, 1=m) 0.0128 0.099 more male (1)
age 0.0093 n.a. +20s/40s, -30s
num_notifs 0.0123 0.061 more notifs
time_last_notif_cntr_acc 0.0486 -0.015 less time passed
time_last_unlock 0.0400 -0.007 less time passed
apps_per_min 0.0199 0.024 more apps per minute
num_apps 0.0124 0.049 more apps
bytes_received 0.0546 -0.012 less bytes
bytes_transmitted 0.0500 0.039 more bytes
battery_level 0.0268 0.012 the higher
battery_drain 0.0249 -0.014 the lower

43. Apps
Co-occur with being bored Co-occur with NOT bored
… and uncategorized apps

44. Boredom was related to
 Regency of communication
 Phase of the day
 Demographics
 Intensity and type of phone usage
 Type of used apps
Take Away #2

45. RQ1: how well can phones detect killing-
time boredom events from these usage
patterns?
RQ2: which usage patterns are related to
killing time with the phone?
RQ3 is the model good enough to be
useful?

46. Borapp2
Model running on Mobile Phone
Using primary data set with
 Constantly predicts when user is
bored on the fly

47. Suggest Reading Buzzfeed Articles

48. Data Collection
16 Participants (different from 1st study)
aged 18 – 51(M = 39) years
13 male, 2 female, rest did not disclose
For two weeks in Feb 2015
941 Buzzfeed recommendations
48% when predicted bored

49. Click-ratio
Fraction of times people
clicked on notification (Mdn)
8% when not bored
20.5% when bored
(as inferred by the model)
Difference significant
z = -2.102, p = .018
Large effect
r = -.543

50. Engagement-ratio
Fraction of times people spent
more than 30 sec reading
(Mdn)
4% when not bored
15% when bored
(as inferred by the model)
Difference significant
z = -2.102, p = .018
Large effect
r = -.511

51. When predicted bored, participants were …
More likely to click
More likely to read
for > 30 seconds

52. The generic model was powerful enough to
create significant, large effects on click-
and engagement-ratios
Take Away #3

53. Application Scenarios
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY

54. Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful but not necessarily
boredom-curing activities
Encourage embracing boredom

55. Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful but not necessarily
boredom-curing activities
Encourage embracing boredom

56. Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful but not necessarily
boredom-curing activities
Encourage embracing boredom

57. Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful but not necessarily
boredom-curing activities
Encourage embracing boredom

58. Being bored is
good for you
Why don’t you
turn me off?
Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful but not necessarily
boredom-curing activities
Encourage embracing boredom

59. When Attention is not Scarce
Detection Boredom from Mobile Phone Usage
Research
Contact: martin.pielot@telefonica.com | @martinpielot | UbiComp ‘15, Osaka, Japan
Nuria Oliver
Jose San
Pedro
Tilman
Dingler
Martin Pielot
Motivation
In general, attention is scarce, hence valuable
Thread of overload / notification blindness
However, boredom is defined is state of seeking stimuli
Contributions
A machine learning model to predict boredom from mobile phone usage patterns
An analysis of usage patterns related to boredom
Evidence that people are more likely to engage with suggested content when bored
Application
Engage user with proactive recommendations – possibly to alleviate boredom
Shield from interruptions when not bored
Suggest useful, but not necessarily boredom-curing activities
Encourage to embrace boredom to foster creativity