Online Multitasking and User Engagement


Published on

Users often access and re-access more than one site during an online session, effectively engaging in multitasking. In this paper, we study the effect of online multitasking on two widely used engagement metrics designed to capture users browsing behavior with a site. Our study is based on browsing data of 2.5M users across 760 sites encompassing diverse types of services such as social media, news and mail. To account for multitasking we need to redefine how user sessions are represented and we need to adapt the metrics under study. We introduce a new representation of user sessions: tree-streams - as opposed to the commonly used click-streams - present a more accurate picture of the browsing behavior of a user that includes how users switch between sites (e.g., hyperlinking, teleporting, backpaging). We then discuss a number of insights on multitasking patterns, and show how these help to better understand how users engage with sites. Finally, we define metrics that characterize multitasking during online sessions and show how they provide additional insights to standard engagement metrics.

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Online Multitasking and User Engagement

  1. 1. ONLINE MULTITASKING AND USER ENGAGEMENT CIKM 2013 Jane%e  Lehmann   In  collabora*on  with:   Mounia  Lalmas,     Ricardo  Baeza-­‐Yates,     George  Dupret  
  2. 2. OUTLINE 1.  Mo%va%on   2.  Characteris%cs  of  online  mul%tasking   Ac2vity  during  and  between  visits       3.  Measuring  online  mul%tasking   Defini2on  of  new  metrics,  case  study     Lights  on  by  JC*+A!   How  do  users  browse  the  web  today?      
  3. 3. leC  by    [  embr  ]     How  do  users   browse  the  Web   today?  
  4. 4. ONLINE MULTITASKING Browsing  the  “old  way”   1min   facebook   news   2min   news   1min   news   3min   news   mail   news  site   Dwell  2me  during  a  visit  on  a  news  site:   7min  on  average   JaneGe  Lehmann   Mo2va2on   4  
  5. 5. ONLINE MULTITASKING Nowadays   1min   news   2min   facebook   news   3min   1min   news   mail   news   Dwell  2me  during  a  visit  on  a  news  site:   2.33min  on  average  (1min  |  3min  |  3min)   JaneGe  Lehmann   Mo2va2on   5  
  6. 6. ONLINE MULTITASKING •  Users  switch  between  sites,  to  do  related  or  totally  unrelated  tasks         •  E.  Herder  [1]:   »  75%  of  sites  are  visited  more  than  once   »  74%  of  revisits  are  performed  within  a  session     Measuring  browsing  behavior  can  lead  to  incorrect  conclusions.     [1]  E.  Herder.  Characteriza*ons  of  user  web  revisit  behavior.  In  LWA,  2005.   JaneGe  Lehmann   Mo2va2on   6  
  7. 7. Danboard's  Messy  Home  by  Mullenkedheim   Characteris%cs   of  online   mul%tasking  
  8. 8. DATA SET Interac%on  data   •  July  2012   •  2.5M  users   •  785M  page  views     •  We  defined  a  new  naviga2on  model                                             (see  paper  for  detail)       •  Categoriza2on  of  the  most  frequent  accessed  sites   (e.g.  mail,  news,  shopping)   »  11  categories  (news),  33  subcategories  (e.g.  news   finance,  news  society)   »  760  sites  from  70  countries/regions     JaneGe  Lehmann         Characteris2cs   8  
  9. 9. Visit activity Visit  frequency     #Visits (avg sd) news (finance) news (tech) social media mail JaneGe  Lehmann   2.09 1.76 2.28 2.09 4.65 1.59 4.78 4.61 Mul%tasking  depends  on  the  site  under   considera%on     •  Social  media  sites  are  revisited  the   most   •  News  (tech)  sites  are  the  least     revisited  sites   Characteris2cs   9  
  10. 10. Visit activity Ac%vity  between  visits   Cumulative probability     Differences  in  the  absence  %me     •  50%  of  sites  are  revisited  aCer  less   than  1min            -­‐  Interrup*on  of  a  task   1.00 0.75 0.50 news (finance) news (tech) social media mail 0.25 0.00 10 2 10 1 10 0 10 1 10 2 •  There  are  revisits  aCer  a  long  break               -­‐  Returning  to  a  site  to  perform  a  new   task   Absence time [min] v1   *   v2   *   v3   *  -­‐  absence  2me   JaneGe  Lehmann   Characteris2cs   10  
  11. 11. Visit activity Ac%vity  paLern       Proportion of total dwell time on site decreasing attention mail sites 0.33 p-value = 0.09 m = -0.01 social media sites •  Four  types  of  "aGen2on  shiCs”   p-value = 0.07 m = -0.02 0.28 0.23 constant attention news (finance) sites Proportion of total dwell time on site increasing attention complex attention •  Complex  cases  refer  to  no   specific  paGern  or  repeated   paGern   news (tech) sites 0.33 p-value = 0.79 m = 0.00 •  Successive  visits  can  belong   together  (i.e.,  to  the  same  task)   0.28 0.23 JaneGe  Lehmann   Characteris2cs   11  
  12. 12. Danboard  by  sⓘndy°   Measuring     online   mul%tasking    
  13. 13. Cumulative activity Cumula%ve  ac%vity     CumActm,k     n = log10 (v1 + ∑ ivik • vi ) i=2                   v1         iv2         v2                 iv3              vi   v  3            ivi  Browsing  ac2vity  between  the  (i-­‐1)th  and  ith  visit    k=3  Rescaling  factor  for  ivi    m  Browsing  ac2vity  (e.g.  dwell  2me,  page  views)    Browsing  ac2vity  during  the  ith  visit     Assump%on:   If  users  return  aCer  short  2me,  they  return  to  con2nue  with  same  task.   If  users  return  aCer  longer  2me,  they  return  to  perform  a  new  task  -­‐  an  indica2on  of  loyalty  to   the  site.     JaneGe  Lehmann   Metrics   13  
  14. 14. Activity pattern ALen%on  shiN  and  range     invm,n − min Invm,n   AttShiftm,n = | max Invm,n | − | min Invm,n   | σ (Vm,n ) AttRangem,n = µ (Vm,n )                    n=4    Number  of  visits  in  session                                                      σ  μ  inv  Variance  in  the  visit  ac2vity    Average  of  the  visit  ac2vity    Modifica2on  of  the  “Inversion  number”         Descrip%on:   AGShiC  models  the  shiC  of  aGen2on  in  the  browsing  ac2vity   AGRange  describes  fluctua2ons  in  the  browsing  ac2vity     JaneGe  Lehmann   Metrics   14  
  15. 15. Activity pattern ALen%on  shiN  and  range   ARen*on  shiS   ARen*on  range   -­‐1   JaneGe  Lehmann   0   1   constant   constant   constant   decreasing   complex   increasing   0   >  0   Metrics   15  
  16. 16. Comparing metrics Comparing  the  ranking  of  the  sites   •  Visitdt  –  Dwell  2me  during  a  visit   •  Sessiondt  –  Dwell  2me  during  a  session       Visitdt   Sessiondt   CumActdt   ALShiNdt       Sessiondt   0.57     CumActdt   -­‐0.04   0.24     ALShiNdt   0.09   0.22   0.02     ALRangedt   -­‐0.01   -­‐0.01   -­‐0.26   0.19       Ø  Visitdt  and  Sessiondt  correlate   Ø  Otherwise  no  correla2on  à  the  other  metrics  capture  different  aspects  of   browsing  behavior   JaneGe  Lehmann   Metrics   16  
  17. 17. Models of browsing behavior “Models”  of  browsing  behavior   •  Clustering  of  sites  using  mul2tasking  and  standard  engagement  metrics:   •  CumActdt,  AGShiCdt,  AGRangedt   •  Visitdt,  Sessiondt     •                We  iden2fied  five  cluster:   C1: 172 sites C2: 108 sites C3: 156 sites C4: 74 sites C5: 166 sites 0.75 0.75 0.75 0.75 0.75 0.25 0.25 0.25 0.25 0.25 -0.25 -0.25 -0.25 -0.25 -0.25 -0.75 -0.75 -0.75 -0.75 -0.75 JaneGe  Lehmann   Visitdt [min] Sessiondt [min] CumActdt,3 Metrics   AttShiftdt,4 AttRangedt,4 17  
  18. 18. Models of browsing behavior C1: 172 sites C2: 108 sites mail, maps, news, news (soc.) auctions, front page, shopping, dating 0.75 0.75 0.25 0.25 -0.25 -0.25 -0.75 -0.75 Visitdt [min] JaneGe  Lehmann   Sessiondt [min] One  task  during  a  session     §  High  dwell  2me  per  visit  and  during   the  whole  session     §  Users  return  to  con2nue  a  task  (short   absence  2me)     §  C1:  aGen2on  is  shiCing  to  another  site   §  C2:  aGen2on  is  shiCing  slowly  towards   the  site   CumActdt,3 Metrics   AttShiftdt,4 AttRangedt,4 18  
  19. 19. Models of browsing behavior C3: 156 sites C4: 74 sites auctions, search, front page, shopping front page, search, download Several  tasks  during  a  session     §  Users  perform  several  tasks  on  these   sites  during  a  session   0.75 0.75 §  No  simple  ac2vity  paGern     0.25 0.25 -0.25 -0.25 -0.75 -0.75 Visitdt [min] JaneGe  Lehmann   Sessiondt [min] §  C3:  Dwell  2me  per  visit  is  low,  but  the   dwell  2me  per  session  is  high     CumActdt,3 Metrics   AttShiftdt,4 AttRangedt,4 19  
  20. 20. Models of browsing behavior C5: 166 sites service, download, blogging, news (soc.) 0.75 0.25 -0.25 Sites  with  low  ac%vity     §  Users  do  not  spend  a  lot  of  2me  on   these  sites     §  Time  between  visits  is  short     §  AGen2on  is  shiCing  towards  the  site   -0.75 Visitdt [min] JaneGe  Lehmann   Sessiondt [min] CumActdt,3 Metrics   AttShiftdt,4 AttRangedt,4 20  
  21. 21. Models of browsing behavior C2: 108 sites auctions, front page, shopping, dating C3: 156 sites auctions, search, front page, shopping 0.75 0.75 0.25 0.25 -0.25 -0.25 -0.75 Browsing  behavior  can  differ  between   sites  of  the  same  category     §  C2:  users  visit  site  once  to  perform   their  task   -0.75 Visitdt [min] JaneGe  Lehmann   Sessiondt [min] §  C3:  users  visit  site  several  2mes  to   perform  task(s)   CumActdt,3 Metrics   AttShiftdt,4 AttRangedt,4 21  
  22. 22. SUMMARY and Future Work •  Online  mul2tasking  affects  the  way  users  access  sites  –  Standard  metrics   do  not  capture  this!!!   •  We  defined  metrics  that  describe  different  aspects  of  mul2tasking   •  CumAct  accounts  for  the  2me  between  visits   •  AGShiC,  AGRange  describe  aGen2on  shiCs   •  We  showed  that  mul2tasking  depends  on  the  site  under  considera2on     Future  work:   •  Can  we  improve  the  defini2on  of  a  task?   •  How  does  mul2tasking  affect  other  metrics,  such  as  bounce  rate  and  click-­‐ through  rate?   •  Does  mul2tasking  differ  in  different  countries?   JaneGe  Lehmann   Summary   22  
  23. 23. Ques%ons?   Online Multitasking + User Engagement JaneGe  Lehmann   Universitat  Pompeu  Fabra,  Spain     Mounia  Lalmas   Yahoo  Labs  London     George  Dupret   Yahoo  Labs  Sunnyvale   gdupret@yahoo-­‐     Ricardo  Baeza-­‐Yates   Yahoo  Labs  Barcelona