Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Analyzing and Predicting Task Reminders

220 views

Published on

Slides of my talk at The 24th Conference on User Modeling, Adaptation and Personalization (UMAP 2016) in Halifax, Canada.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Analyzing and Predicting Task Reminders

  1. 1. Analyzing and Predicting Task Reminders David Graus, Paul N. Bennett, Ryen W. White, Eric Horvitz
  2. 2. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 2
  3. 3. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3
  4. 4. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3 When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes.
  5. 5. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3 When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes.
  6. 6. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3 When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes.
  7. 7. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3 When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes.
  8. 8. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3 When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes.
  9. 9. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3 When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes. When would you like to be reminded? Alright, remind you to do the laundry at 12:00pm on Sunday, is that right? Great, I’ll remind you! Sunday at noon. Remind me to do the laundry. Yes.
  10. 10. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4 Research questions
  11. 11. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4 Research questions 1. Is there a body of common tasks that underlie the reminder creation process?
  12. 12. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4 Research questions 1. Is there a body of common tasks that underlie the reminder creation process? 2. Can we identify patterns in the times at which people create reminders, and, via notification times, when the associated tasks are to be executed?
  13. 13. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4 Research questions 1. Is there a body of common tasks that underlie the reminder creation process? 2. Can we identify patterns in the times at which people create reminders, and, via notification times, when the associated tasks are to be executed? 3. Can we predict when certain tasks are most likely to happen?
  14. 14. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5 Is there a body of common tasks that underlie the reminder creation process?
  15. 15. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5 Is there a body of common tasks that underlie the reminder creation process? • Common reminder structure: • “Remind me to <verb> <object>” • take out laundry • call mom • pay bills
  16. 16. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5 Is there a body of common tasks that underlie the reminder creation process? • Common reminder structure: • “Remind me to <verb> <object>” • take out laundry • call mom • pay bills • Identify frequent verb/object-pairs in 2-month sample (~950k reminders)
  17. 17. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5 Is there a body of common tasks that underlie the reminder creation process? • Common reminder structure: • “Remind me to <verb> <object>” • take out laundry • call mom • pay bills • Identify frequent verb/object-pairs in 2-month sample (~950k reminders) • Manually group them into tasks
  18. 18. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 6 ~60% of reminders start with one of these (52) verbs take
 eat
 have
 send
 email
 text
 call
 tell
 set
 make
 schedule
 turn on
 turn off
 check
 leave
 come
 be
 go to
 stop by take out feed clean wash charge do write change cancel order renew book mail submit fill out print pay wake set
  19. 19. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 7 Manual labeling Ò Dimensions; Ò Interruption vs. continuation of user activity Ò Context of task (home, work, on the go) Ò (Expected) duration of task Ò Impact on user “availability” Ò …
  20. 20. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8 common tasks that underlie the reminder creation process
  21. 21. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8 Go somewhere Switch context Run errand common tasks that underlie the reminder creation process
  22. 22. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8 Go somewhere Switch context Run errand common tasks that underlie the reminder creation process Go somewhere Chores Switch context Run errand Recurring Standalone
  23. 23. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8 Go somewhere Switch context Run errand common tasks that underlie the reminder creation process Go somewhere Chores Switch context Run errand Recurring Standalone Go somewhere Chores Communicate Switch context Run errand Recurring Standalone General Coordinate
  24. 24. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8 Go somewhere Switch context Run errand common tasks that underlie the reminder creation process Go somewhere Chores Switch context Run errand Recurring Standalone Go somewhere Chores Communicate Switch context Run errand Recurring Standalone General Coordinate Go somewhere Chores Communicate Manage Ongoing External Process Manage Ongoing User Activity Eat/Consume Switch context Run errand Recurring Standalone General Coordinate Start Prepare Stop
  25. 25. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8 Go somewhere Switch context Run errand common tasks that underlie the reminder creation process Go somewhere Chores Switch context Run errand Recurring Standalone Go somewhere Chores Communicate Switch context Run errand Recurring Standalone General Coordinate Go somewhere Chores Communicate Manage Ongoing External Process Manage Ongoing User Activity Eat/Consume Switch context Run errand Recurring Standalone General Coordinate Start Prepare Stop
  26. 26. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9 RQ2: Can we identify patterns in the reminder creation process?
  27. 27. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9 RQ2: Can we identify patterns in the reminder creation process? Ò Creation time (CT) Ò When user creates the reminder (remembers task)
  28. 28. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9 RQ2: Can we identify patterns in the reminder creation process? Ò Creation time (CT) Ò When user creates the reminder (remembers task) Ò Notification time (NT) Ò When reminder is set to trigger (executes task)
  29. 29. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9 RQ2: Can we identify patterns in the reminder creation process? Ò Creation time (CT) Ò When user creates the reminder (remembers task) Ò Notification time (NT) Ò When reminder is set to trigger (executes task) Ò Time delta (NT – CT) Ò How far task is planned in advance
  30. 30. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9 RQ2: Can we identify patterns in the reminder creation process? Ò Creation time (CT) Ò When user creates the reminder (remembers task) Ò Notification time (NT) Ò When reminder is set to trigger (executes task) Ò Time delta (NT – CT) Ò How far task is planned in advance Ò Text/task description
  31. 31. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 10 00:00am – 04:00am 04:00am – 08:00am 08:00am – 12:00pm 12:00pm – 4:00pm 4:00pm – 8:00pm 8:00pm – 0:00am Sun Mon Tue Wed Thu Fri Sat
  32. 32. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 11 RQ2: Can we identify patterns in the reminder creation process? 1. Aggregate 2. Per task type 3. By Creation Time and Terms (in task description)
  33. 33. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12 Creation times Notification times 1: Aggregate patterns (across all tasks)
  34. 34. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12 Creation times Notification times 1: Aggregate patterns (across all tasks)
  35. 35. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12 Creation times Notification times 1: Aggregate patterns (across all tasks)
  36. 36. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12 Creation times Notification times 1: Aggregate patterns (across all tasks)
  37. 37. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13 Findings: Aggregate patterns
  38. 38. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13 Findings: Aggregate patterns Ò Creation/Notification Ò People create reminders in evenings (end of day), w/ notifications set in mornings (start of day).

  39. 39. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13 Findings: Aggregate patterns Ò Creation/Notification Ò People create reminders in evenings (end of day), w/ notifications set in mornings (start of day).
 Ò (not shown) Time deltas Ò Most reminders are for “short-term tasks”
  40. 40. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 14 2: Per task type patterns
  41. 41. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15 Allrem inders Chore (recurring) Chore (standalone) Com m .(coordinate) Com m .(general) Eat/Consum e Go (sw itch context)Go (errand) M anage activity M anage process Delay(hours) 72 60 48 36 24 12 0 Time deltas per task type Time deltas per task type
  42. 42. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15 Allrem inders Chore (recurring) Chore (standalone) Com m .(coordinate) Com m .(general) Eat/Consum e Go (sw itch context)Go (errand) M anage activity M anage process Delay(hours) 72 60 48 36 24 12 0 Time deltas per task type Time deltas per task type
  43. 43. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15 Allrem inders Chore (recurring) Chore (standalone) Com m .(coordinate) Com m .(general) Eat/Consum e Go (sw itch context)Go (errand) M anage activity M anage process Delay(hours) 72 60 48 36 24 12 0 Time deltas per task type Time deltas per task type
  44. 44. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15 Allrem inders Chore (recurring) Chore (standalone) Com m .(coordinate) Com m .(general) Eat/Consum e Go (sw itch context)Go (errand) M anage activity M anage process Delay(hours) 72 60 48 36 24 12 0 Time deltas per task type Time deltas per task type
  45. 45. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16 Findings: per task type patterns
  46. 46. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16 Findings: per task type patterns Ò Time deltas Ò Differ between task type.
  47. 47. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16 Findings: per task type patterns Ò Time deltas Ò Differ between task type. Ò (not shown) Creation/Notification Ò broadly two types of tasks: Ò Created/set to notify during office hours Ò e.g., communicate, go somewhere Ò Created/set to notify outside of office hours Ò e.g., manage ongoing process, chores
  48. 48. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17 Patterns 3: By creation time
  49. 49. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17 What is the relation between creation and notification time? Patterns 3: By creation time
  50. 50. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17 What is the relation between creation and notification time? Patterns 3: By creation time
  51. 51. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17 What is the relation between creation and notification time? Patterns 3: By creation time
  52. 52. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17 What is the relation between creation and notification time? Patterns 3: By creation time
  53. 53. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17 What is the relation between creation and notification time? Patterns 3: By creation time
  54. 54. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17 What is the relation between creation and notification time? Patterns 3: By creation time
  55. 55. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17 What is the relation between creation and notification time? Patterns 3: By creation time
  56. 56. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18 “call” Patterns 3: By terms
  57. 57. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18 “call” Patterns 3: By terms
  58. 58. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18 “call” Patterns 3: By terms
  59. 59. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18 “call” Patterns 3: By terms
  60. 60. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 19 “kids”
  61. 61. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 19 “kids”
  62. 62. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 20
  63. 63. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 20 “church”
  64. 64. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21 RQ2: Can we identify patterns in the reminder creation process?
  65. 65. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21 RQ2: Can we identify patterns in the reminder creation process? Ò On average people tend to set plans in the evening.
  66. 66. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21 RQ2: Can we identify patterns in the reminder creation process? Ò On average people tend to set plans in the evening. Ò Most tasks that drive reminder setting are for short-term tasks to be executed in the next 24 hours.
  67. 67. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21 RQ2: Can we identify patterns in the reminder creation process? Ò On average people tend to set plans in the evening. Ò Most tasks that drive reminder setting are for short-term tasks to be executed in the next 24 hours. Ò Patterns differ between task types (suggesting the distinctions are meaningful).
  68. 68. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21 RQ2: Can we identify patterns in the reminder creation process? Ò On average people tend to set plans in the evening. Ò Most tasks that drive reminder setting are for short-term tasks to be executed in the next 24 hours. Ò Patterns differ between task types (suggesting the distinctions are meaningful). Ò Terms and creation times are informative w.r.t. notification time.
  69. 69. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 22 RQ3: Can we predict when certain tasks are most likely to happen? Ò Motivation: Ò See if patterns are informative. Ò Applications: Ò improve resource scheduling, Ò detect collisions Ò tailored advertising, Ò developing systems to automatically terminate ongoing tasks/allocate time for task completion
  70. 70. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 23 Findings Ò Predict notification day with ~70% accuracy. Ò Creation time provides the most information. Ò Terms provides significant additional information.
  71. 71. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 24 In summary 1.Studied frequent reminders, developed task type taxonomy. 2.Shown reminders’ temporal patterns 3.Demonstrated a direction in harnessing patterns; predicting notification times.
  72. 72. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 25 Future work Ò User studies? Ò assumptions w.r.t. “wrapping up day, planning ahead” Ò validate/check task type taxonomy Ò Classify task types for improved predictions? Ò More sophisticated predictive models Ò Look at additional reminder types (e.g., location-based)
  73. 73. Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 26 Thanks! Ò Questions? d.p.graus@uva.nl @dvdgrs www.graus.co

×