Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
When Attention is not Scarce
Detecting Boredom from Mobile Phone Usage
Research
UbiComp ‘15, Osaka, Japan
Martin
Pielot
Te...
times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0
War on
Attention*
* http://www.forbes.com/si...
Revenue per active user
$45 in Q1 2014 = 50 cents per day
SocialMediaCube. Yoel Ben-Avraham. Apr 8, 2013 via Flickr. CC BY-ND 2.0
The trade we make:
Our attention so they can pay t...
Our engagement is now defined by push-driven
notifications rather than the traditional pull-driven
experience. We’re “hunt...
Example: Push-Driven Notifications
‚Attention is a limited resource—a person
has only so much of it ‘
[Matthew B. Crawford]
Attention Economy: treating human...
Wild-West Land-Grab Phase
“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administr...
Overload
“Ahhhhhhh” by Kenny Louie, Jun 06, 2010, via Flickr, CC BY 2.0
From “Banner Blindness: New and Old Findings” by Jakob Nielsen on August 20, 2007
Banner Blindness
Overload
“Ahhhhhhh” by Kenny Louie, Jun 06, 2010, via Flickr, CC BY 2.0
Notification blindness
Wild-West Land-Grab Phase
“Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administr...
Boredom as part of the solution
Attention is not always scarce
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. C...
Attention is not always scarce
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. C...
Attention is not always scarce
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. C...
Attention is not always scarce
Mobile phones are a commonly used tool to
fill or kill time when bored [Brown et al. 2014]
...
How well can we
detect boredom from
mobile phone usage
patterns?
Borapp – Sensor-Data Collection
Always
collected
Only
collected if
phone in use
Sensor Description
Battery Status Battery ...
Experience Sampling
“Right now, I feel bored”
[5-point Likert scale]
Min. 6 times per day
Preferably triggered when phone ...
Data Collection
54 Participants
aged 21 – 46 (M = 30.6) years
11 female, 23male, 19 not disclosed
For two weeks in July 20...
0
500
1000
1500
0 1 2 3 4
Frequency
Agreement to "Right now, I feel bored"
0 = disagree, 4 = agree
0
500
1000
1500
0 1 2 3 4
Frequency
Agreement to "Right now, I feel bored"
0 = disagree, 4 = agree
Absolute ground truth
B...
Absolute ground truth
Bored: ratings 3, 4
446 (10.1%) instances
Normalized ground truth
Z-score per person
Bored: z > 0.25...
Category Example Feature Explanation
Context Semantic Location Home, work, other, unknown
Demographics Age, gender 38, fem...
RQ1: how well can phones detect killing-
time boredom events from these usage
patterns?
RQ2: which usage patterns are rela...
74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance | Random Forest (AUCROC)
76.5%
82.5%
74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance (AUCROC) Including Bored...
76.5%
82.5%
74.6%
82.9%
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
normalized
absolute
Model Performance (AUCROC) Including Bored...
34.7%
42.8%
48.3%
52.1%
56.6%
62.4%
66.2%
70.1%
74.3%
76.3%
0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
Precision
Reca...
Boredom can be detected from
phone-usage patterns with an
accuracy of ca. 75% to 83% AUCROC
Take Away #1
RQ1: how well can phones detect killing-
time boredom events from these usage
patterns?
RQ2: which usage patterns are rela...
 Recency of communication activity
i.e., time since last incoming or outgoing
communication;
Feature Import Correlation T...
 Recency of communication activity
i.e., time since last incoming or outgoing
communication;
 Phase of the day
i.e., hou...
 Recency of communication activity
i.e., time since last incoming or outgoing
communication;
 Phase of the day
i.e., hou...
 Recency of communication activity
i.e., time since last incoming or outgoing
communication;
 Phase of the day
i.e., hou...
 Recency of communication activity
i.e., time since last incoming or outgoing
communication;
 Phase of the day
i.e., hou...
Apps
Co-occur with being bored Co-occur with NOT bored
… and uncategorized apps
Boredom was related to
 Regency of communication
 Phase of the day
 Demographics
 Intensity and type of phone usage
 ...
RQ1: how well can phones detect killing-
time boredom events from these usage
patterns?
RQ2: which usage patterns are rela...
Borapp2
Model running on Mobile Phone
Using primary data set with
 Constantly predicts when user is
bored on the fly
Suggest Reading Buzzfeed Articles
Data Collection
16 Participants (different from 1st study)
aged 18 – 51(M = 39) years
13 male, 2 female, rest did not disc...
Click-ratio
Fraction of times people
clicked on notification (Mdn)
8% when not bored
20.5% when bored
(as inferred by the ...
Engagement-ratio
Fraction of times people spent
more than 30 sec reading
(Mdn)
4% when not bored
15% when bored
(as inferr...
When predicted bored, participants were …
More likely to click
More likely to read
for > 30 seconds
The generic model was powerful enough to
create significant, large effects on click-
and engagement-ratios
Take Away #3
Application Scenarios
Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful ...
Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful ...
Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful ...
Recommend content to alleviate
boredom
Shield user from non-important
interruptions during non-bored
times
Suggest useful ...
Being bored is
good for you
Why don’t you
turn me off?
Recommend content to alleviate
boredom
Shield user from non-importa...
When Attention is not Scarce
Detection Boredom from Mobile Phone Usage
Research
Contact: martin.pielot@telefonica.com | @m...
When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage
When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage
When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage
When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage
When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage
Upcoming SlideShare
Loading in …5
×

When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage

11,170 views

Published on

Slides from the presentation at ACM UbiComp 2015

When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage (best-paper award).
Martin Pielot, Tilman Dingler, Jose San Pedro, and Nuria Oliver
UbiComp’ 15: ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015.

Published in: Technology
  • Be the first to comment

When Attention is not Scarce – Detecting Boredom from Mobile Phone Usage

  1. 1. When Attention is not Scarce Detecting Boredom from Mobile Phone Usage Research UbiComp ‘15, Osaka, Japan Martin Pielot Telefonica Research Tilman Dingler University of Stuttgart Jose San Pedro Telefonica Research Nuria Oliver Telefonica Research
  2. 2. times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0 War on Attention* * http://www.forbes.com/sites/onmarketing/2012/10/19/the-attention-war/
  3. 3. Revenue per active user $45 in Q1 2014 = 50 cents per day
  4. 4. SocialMediaCube. Yoel Ben-Avraham. Apr 8, 2013 via Flickr. CC BY-ND 2.0 The trade we make: Our attention so they can pay their bills
  5. 5. Our engagement is now defined by push-driven notifications rather than the traditional pull-driven experience. We’re “hunting and pecking” through our app grid a lot less; the apps that notify us (without over- notifying to the point of uninstall) are rewarded with our engagement (and our dollars).
  6. 6. Example: Push-Driven Notifications
  7. 7. ‚Attention is a limited resource—a person has only so much of it ‘ [Matthew B. Crawford] Attention Economy: treating human attention as a scarce commodity [Davenport and Beck, 2001] times square night 2013. chensiyuan. Apr 16, 2013 via Wikipedia. CC BY-SA 4.0
  8. 8. Wild-West Land-Grab Phase “Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain
  9. 9. Overload “Ahhhhhhh” by Kenny Louie, Jun 06, 2010, via Flickr, CC BY 2.0
  10. 10. From “Banner Blindness: New and Old Findings” by Jakob Nielsen on August 20, 2007 Banner Blindness
  11. 11. Overload “Ahhhhhhh” by Kenny Louie, Jun 06, 2010, via Flickr, CC BY 2.0 Notification blindness
  12. 12. Wild-West Land-Grab Phase “Wild West Hotel, Calamity Av., Perry, 0. T., Sept. 93”. National Archives and Records Administration. Public Domain If the trade attention for free services is to be sustained we need to better protect mobile phone users
  13. 13. Boredom as part of the solution
  14. 14. Attention is not always scarce Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
  15. 15. Attention is not always scarce Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY Boredom displeasure caused by “lack of stimulation” [Fenichel, 1951] “a bored person is not just someone who does not have anything to do; it’s someone who is actively looking for stimulation” [Eastwood, 2002]
  16. 16. Attention is not always scarce Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY Boredom displeasure caused by “lack of stimulation” [Fenichel, 1951] “a bored person is not just someone who does not have anything to do; it’s someone who is actively looking for stimulation” [Eastwood, 2002] Mobile phones are a commonly used tool to kill time when bored [Brown et al. 2014]
  17. 17. Attention is not always scarce Mobile phones are a commonly used tool to fill or kill time when bored [Brown et al. 2014] Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY Boredom displeasure caused by “lack of stimulation” [Fenichel, 1951] “a bored person is not just someone who does not have anything to do; it’s someone who is actively looking for stimulation” [Eastwood, 2002] If phones knew when their users are killing time maybe they could suggest them to make better use of the moment
  18. 18. How well can we detect boredom from mobile phone usage patterns?
  19. 19. Borapp – Sensor-Data Collection Always collected Only collected if phone in use Sensor Description Battery Status Battery level ranging from 0-100% Notifications Time and type (app) of notification Screen Events Screen turned on, off, and unlocked Phone Events Time of incoming and outgoing calls Proximity Screen covered or not Ringer Mode Silent, Vibration, Normal SMS Time of receiving, reading, and sending SMS Sensor Description Airplane Mode Whether phone in airplane mode Ambient Noise Noise in dB as sensed by the microphone Audio Jack Phone connected to headphones or speakers Cell Tower The cell tower the phone is connected to Data Activity Number of bytes up/downloaded Foreground app Package name of the app in foreground Light Ambient light level in SI lux units Screen Orient Portrait or Landscape mode Wifi Infos The WiFi network the phone is connected to
  20. 20. Experience Sampling “Right now, I feel bored” [5-point Likert scale] Min. 6 times per day Preferably triggered when phone in use Borapp – Experience Sampling
  21. 21. Data Collection 54 Participants aged 21 – 46 (M = 30.6) years 11 female, 23male, 19 not disclosed For two weeks in July 2014 Over 40M sensor log entries 4398 valid self-reports of boredom
  22. 22. 0 500 1000 1500 0 1 2 3 4 Frequency Agreement to "Right now, I feel bored" 0 = disagree, 4 = agree
  23. 23. 0 500 1000 1500 0 1 2 3 4 Frequency Agreement to "Right now, I feel bored" 0 = disagree, 4 = agree Absolute ground truth Bored: ratings 3, 4 446 (10.1%) instances
  24. 24. Absolute ground truth Bored: ratings 3, 4 446 (10.1%) instances Normalized ground truth Z-score per person Bored: z > 0.25 1518 (34.5%) instances 0 400 800 1200 1600 2000 -2 -1 0 1 2 Frequency Normalized Subjective Boredom, (higher number = more bored than usual)
  25. 25. Category Example Feature Explanation Context Semantic Location Home, work, other, unknown Demographics Age, gender 38, female Last Communication Activity Time last incoming call Time passed since somebody called the participants Usage (intensity) Bytes received Number of bytes downloaded in the last 5 minutes Usage (externally triggered) Number of notifications Number of notifications received in the last 5 minutes Usage (idling) Number of apps Number of apps launched in the last 5 minutes Usage (type) Most used app App used for the most time in the last 5 minutes. 35 Features, 7 Categories
  26. 26. RQ1: how well can phones detect killing- time boredom events from these usage patterns? RQ2: which usage patterns are related to killing time with the phone? RQ3 is the model good enough to be useful?
  27. 27. 74.6% 82.9% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0% normalized absolute Model Performance | Random Forest (AUCROC)
  28. 28. 76.5% 82.5% 74.6% 82.9% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0% normalized absolute Model Performance (AUCROC) Including Boredom Proneness scores of 22 participants
  29. 29. 76.5% 82.5% 74.6% 82.9% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0% normalized absolute Model Performance (AUCROC) Including Boredom Proneness scores of 22 participants Primary data set
  30. 30. 34.7% 42.8% 48.3% 52.1% 56.6% 62.4% 66.2% 70.1% 74.3% 76.3% 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Precision Recall Precision: 70.1% for 30% recall, 62.4% for 50% recall
  31. 31. Boredom can be detected from phone-usage patterns with an accuracy of ca. 75% to 83% AUCROC Take Away #1
  32. 32. RQ1: how well can phones detect killing- time boredom events from these usage patterns? RQ2: which usage patterns are related to killing time with the phone? RQ3 is the model good enough to be useful?
  33. 33.  Recency of communication activity i.e., time since last incoming or outgoing communication; Feature Import Correlation The more bored, the .. time_last_outgoing_call 0.0607 -0.143 less time passed time_last_incoming_call 0.0580 0.088 more time passed time_last_notif 0.0564 0.091 more time passed time_last_SMS_received 0.0483 0.053 more time passed time_last_SMS_sent 0.0405 -0.090 less time passed time_last_SMS_read 0.0388 -0.013 more time passed light 0.0537 -0.010 darker hour_of_day 0.0411 0.038 later proximity 0.0153 -0.186 less covered gender (0=f, 1=m) 0.0128 0.099 more male (1) age 0.0093 n.a. +20s/40s, -30s num_notifs 0.0123 0.061 more notifs time_last_notif_cntr_acc 0.0486 -0.015 less time passed time_last_unlock 0.0400 -0.007 less time passed apps_per_min 0.0199 0.024 more apps per minute num_apps 0.0124 0.049 more apps bytes_received 0.0546 -0.012 less bytes bytes_transmitted 0.0500 0.039 more bytes battery_level 0.0268 0.012 the higher battery_drain 0.0249 -0.014 the lower
  34. 34.  Recency of communication activity i.e., time since last incoming or outgoing communication;  Phase of the day i.e., hour of the day, ambient light Feature Import Correlation The more bored, the .. time_last_outgoing_call 0.0607 -0.143 less time passed time_last_incoming_call 0.0580 0.088 more time passed time_last_notif 0.0564 0.091 more time passed time_last_SMS_received 0.0483 0.053 more time passed time_last_SMS_sent 0.0405 -0.090 less time passed time_last_SMS_read 0.0388 -0.013 more time passed light 0.0537 -0.010 darker hour_of_day 0.0411 0.038 later proximity 0.0153 -0.186 less covered gender (0=f, 1=m) 0.0128 0.099 more male (1) age 0.0093 n.a. +20s/40s, -30s num_notifs 0.0123 0.061 more notifs time_last_notif_cntr_acc 0.0486 -0.015 less time passed time_last_unlock 0.0400 -0.007 less time passed apps_per_min 0.0199 0.024 more apps per minute num_apps 0.0124 0.049 more apps bytes_received 0.0546 -0.012 less bytes bytes_transmitted 0.0500 0.039 more bytes battery_level 0.0268 0.012 the higher battery_drain 0.0249 -0.014 the lower
  35. 35.  Recency of communication activity i.e., time since last incoming or outgoing communication;  Phase of the day i.e., hour of the day, ambient light  Demographics, i.e., gender and age; Feature Import Correlation The more bored, the .. time_last_outgoing_call 0.0607 -0.143 less time passed time_last_incoming_call 0.0580 0.088 more time passed time_last_notif 0.0564 0.091 more time passed time_last_SMS_received 0.0483 0.053 more time passed time_last_SMS_sent 0.0405 -0.090 less time passed time_last_SMS_read 0.0388 -0.013 more time passed light 0.0537 -0.010 darker hour_of_day 0.0411 0.038 later proximity 0.0153 -0.186 less covered gender (0=f, 1=m) 0.0128 0.099 more male (1) age 0.0093 n.a. +20s/40s, -30s num_notifs 0.0123 0.061 more notifs time_last_notif_cntr_acc 0.0486 -0.015 less time passed time_last_unlock 0.0400 -0.007 less time passed apps_per_min 0.0199 0.024 more apps per minute num_apps 0.0124 0.049 more apps bytes_received 0.0546 -0.012 less bytes bytes_transmitted 0.0500 0.039 more bytes battery_level 0.0268 0.012 the higher battery_drain 0.0249 -0.014 the lower
  36. 36.  Recency of communication activity i.e., time since last incoming or outgoing communication;  Phase of the day i.e., hour of the day, ambient light  Demographics, i.e., gender and age;  General usage intensity i.e, phone out of pocket, or time since last phone use …; Feature Import Correlation The more bored, the .. time_last_outgoing_call 0.0607 -0.143 less time passed time_last_incoming_call 0.0580 0.088 more time passed time_last_notif 0.0564 0.091 more time passed time_last_SMS_received 0.0483 0.053 more time passed time_last_SMS_sent 0.0405 -0.090 less time passed time_last_SMS_read 0.0388 -0.013 more time passed light 0.0537 -0.010 darker hour_of_day 0.0411 0.038 later proximity 0.0153 -0.186 less covered gender (0=f, 1=m) 0.0128 0.099 more male (1) age 0.0093 n.a. +20s/40s, -30s num_notifs 0.0123 0.061 more notifs time_last_notif_cntr_acc 0.0486 -0.015 less time passed time_last_unlock 0.0400 -0.007 less time passed apps_per_min 0.0199 0.024 more apps per minute num_apps 0.0124 0.049 more apps bytes_received 0.0546 -0.012 less bytes bytes_transmitted 0.0500 0.039 more bytes battery_level 0.0268 0.012 the higher battery_drain 0.0249 -0.014 the lower
  37. 37.  Recency of communication activity i.e., time since last incoming or outgoing communication;  Phase of the day i.e., hour of the day, ambient light  Demographics, i.e., gender and age;  General usage intensity i.e, phone out of pocket, or time since last phone use …;  Intensity of recent usage i.e. # of unlocks, or # of apps launched in last 5 minutes, … Feature Import Correlation The more bored, the .. time_last_outgoing_call 0.0607 -0.143 less time passed time_last_incoming_call 0.0580 0.088 more time passed time_last_notif 0.0564 0.091 more time passed time_last_SMS_received 0.0483 0.053 more time passed time_last_SMS_sent 0.0405 -0.090 less time passed time_last_SMS_read 0.0388 -0.013 more time passed light 0.0537 -0.010 darker hour_of_day 0.0411 0.038 later proximity 0.0153 -0.186 less covered gender (0=f, 1=m) 0.0128 0.099 more male (1) age 0.0093 n.a. +20s/40s, -30s num_notifs 0.0123 0.061 more notifs time_last_notif_cntr_acc 0.0486 -0.015 less time passed time_last_unlock 0.0400 -0.007 less time passed apps_per_min 0.0199 0.024 more apps per minute num_apps 0.0124 0.049 more apps bytes_received 0.0546 -0.012 less bytes bytes_transmitted 0.0500 0.039 more bytes battery_level 0.0268 0.012 the higher battery_drain 0.0249 -0.014 the lower
  38. 38. Apps Co-occur with being bored Co-occur with NOT bored … and uncategorized apps
  39. 39. Boredom was related to  Regency of communication  Phase of the day  Demographics  Intensity and type of phone usage  Type of used apps Take Away #2
  40. 40. RQ1: how well can phones detect killing- time boredom events from these usage patterns? RQ2: which usage patterns are related to killing time with the phone? RQ3 is the model good enough to be useful?
  41. 41. Borapp2 Model running on Mobile Phone Using primary data set with  Constantly predicts when user is bored on the fly
  42. 42. Suggest Reading Buzzfeed Articles
  43. 43. Data Collection 16 Participants (different from 1st study) aged 18 – 51(M = 39) years 13 male, 2 female, rest did not disclose For two weeks in Feb 2015 941 Buzzfeed recommendations 48% when predicted bored
  44. 44. Click-ratio Fraction of times people clicked on notification (Mdn) 8% when not bored 20.5% when bored (as inferred by the model) Difference significant z = -2.102, p = .018 Large effect r = -.543
  45. 45. Engagement-ratio Fraction of times people spent more than 30 sec reading (Mdn) 4% when not bored 15% when bored (as inferred by the model) Difference significant z = -2.102, p = .018 Large effect r = -.511
  46. 46. When predicted bored, participants were … More likely to click More likely to read for > 30 seconds
  47. 47. The generic model was powerful enough to create significant, large effects on click- and engagement-ratios Take Away #3
  48. 48. Application Scenarios Show them this photo if someone said technology … . Adam Rifkin. May 21, 2014 via Flickr. CC BY
  49. 49. Recommend content to alleviate boredom Shield user from non-important interruptions during non-bored times Suggest useful but not necessarily boredom-curing activities Encourage embracing boredom
  50. 50. Recommend content to alleviate boredom Shield user from non-important interruptions during non-bored times Suggest useful but not necessarily boredom-curing activities Encourage embracing boredom
  51. 51. Recommend content to alleviate boredom Shield user from non-important interruptions during non-bored times Suggest useful but not necessarily boredom-curing activities Encourage embracing boredom
  52. 52. Recommend content to alleviate boredom Shield user from non-important interruptions during non-bored times Suggest useful but not necessarily boredom-curing activities Encourage embracing boredom
  53. 53. Being bored is good for you Why don’t you turn me off? Recommend content to alleviate boredom Shield user from non-important interruptions during non-bored times Suggest useful but not necessarily boredom-curing activities Encourage embracing boredom
  54. 54. When Attention is not Scarce Detection Boredom from Mobile Phone Usage Research Contact: martin.pielot@telefonica.com | @martinpielot | UbiComp ‘15, Osaka, Japan Nuria Oliver Jose San Pedro Tilman Dingler Martin Pielot Motivation In general, attention is scarce, hence valuable Thread of overload / notification blindness However, boredom is defined is state of seeking stimuli Contributions A machine learning model to predict boredom from mobile phone usage patterns An analysis of usage patterns related to boredom Evidence that people are more likely to engage with suggested content when bored Application Engage user with proactive recommendations – possibly to alleviate boredom Shield from interruptions when not bored Suggest useful, but not necessarily boredom-curing activities Encourage to embrace boredom to foster creativity

×