Networked User Engagement

1,245 views

Published on

Online providers frequently offer a variety of services, ranging from news to mail. These providers endeavour to keep users accessing and interacting with the sites offering these services, that is to engage users by spending time on these sites and returning regularly to them. The standard approach to evaluate engagement with a site is by measuring engagement metrics of each site separately. However, when assessing engagement within a network of sites, it is crucial to take into account the traffic between sites. This paper proposes a methodology for studying networked used engagement. We represent sites (nodes) and user traffic (edges) between them as a network, and apply complex network metrics to study networked user engagement. We demonstrate the value of these metrics when applied to 728 services offered by Yahoo! with a sample of 2M users and a total of 25M online sessions.

Published in: Technology, Design
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,245
On SlideShare
0
From Embeds
0
Number of Embeds
115
Actions
Shares
0
Downloads
8
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Networked User Engagement

  1. 1. Networked user engagement user engagement optimization workshop - CIKM 2013 Jane%e  Lehmann   In  collabora*on  with:   Mounia  Lalmas,     Ricardo  Baeza-­‐Yates,     Elad  Yom-­‐Tov  
  2. 2. outline How  to  measure  user  engagement?       2.  Network  metrics   From  site  to  network  engagement.       3.  Measuring  networked  user  engagement   Case  study  based  on  the  network  of  Yahoo  sites.   Lights  on  by  JC*+A!   1.  Mo%va%on  
  3. 3. 182-­‐365+1  by  meaganmakes   How  to  measure   User   Engagement?  
  4. 4. Measuring user engagement User  engagement  on  Yahoo!   Popularity Ac%vity   Loyalty   JaneHe  Lehmann   Web  traffic  reports:  Google  Analy%cs,  Alexa.com,  …    (#Users)    (DwellTime)    (AcJveDays)   MoJvaJon   4  
  5. 5. Measuring user engagement However,  Yahoo!  has  many  sites:     maps,  flickr,  weather,  news,  shine,  etc.     How  to  measure  engagement  within     a  network  of  sites?   JaneHe  Lehmann   MoJvaJon   5  
  6. 6. NETWORK OF SITES Large  online  providers     (AOL,  Google,  Yahoo!,  etc.)     offer  not  one  site,     but  a  network  of  sites   local   maps   mail   search   weather   news   daJng   frontpage   finance   jobs   sports   tv   autos   omg   shine   flickr   shopping   health   JaneHe  Lehmann   games   answer   travel   homes   MoJvaJon   groups   6  
  7. 7. NETWORK OF SITES Each  site  is  usually     op%mized  individually,     with  some  effort  to  direct     users  between  them   local   maps   mail   search   weather   news   daJng   frontpage   finance   jobs   sports   tv   autos   omg   shine   flickr   shopping   health   JaneHe  Lehmann   games   answer   travel   homes   MoJvaJon   groups   7  
  8. 8. NETWORK OF SITES local   maps   Online  mul%tasking   Users  switch  between  sites   within  one  online  session     (e.g.  emailing,  reading  news,     social  networking,   jobs   search)   mail   search   weather   news   daJng   frontpage   finance   sports   tv   autos   omg   shine   flickr   shopping   health   JaneHe  Lehmann   games   answer   travel   homes   MoJvaJon   groups   8  
  9. 9. NETWORK OF SITES local   maps   Measuring  user     engagement  should     account  for     user  traffic  between     sites...   omg   shine   mail   search   weather   news   daJng   frontpage   finance   jobs   sports   tv   autos   flickr   shopping   health   JaneHe  Lehmann   games   answer   travel   homes   MoJvaJon   groups   9  
  10. 10. NETWORK OF SITES local   maps   …  to  detect   missing   connec9ons   omg   shine   mail   search   weather   news   daJng   frontpage   finance   jobs   sports   tv   autos   flickr   shopping   health   JaneHe  Lehmann   games   answer   travel   homes   MoJvaJon   groups   10  
  11. 11. NETWORK OF SITES local   maps   …  to  detect   sites  that  are     used  together   omg   shine   mail   search   weather   news   daJng   frontpage   finance   jobs   sports   tv   autos   flickr   shopping   health   JaneHe  Lehmann   games   answer   travel   homes   MoJvaJon   groups   11  
  12. 12. NETWORK OF SITES local   maps   …  to  detect   isolated   sites   omg   shine   mail   search   weather   news   daJng   frontpage   finance   jobs   sports   tv   autos   flickr   shopping   health   JaneHe  Lehmann   games   answer   travel   homes   MoJvaJon   groups   12  
  13. 13. NETWORK OF SITES local   maps   …  to  detect   important   sites   omg   shine   mail   search   weather   news   daJng   frontpage   finance   jobs   sports   tv   autos   flickr   shopping   health   JaneHe  Lehmann   games   answer   travel   homes   MoJvaJon   groups   13  
  14. 14. Danboard  by  sⓘndy°   From  site  to  network   engagement     Looking  beyond  one           own  nose!  
  15. 15. ENGAGEMENT NETWORKS Provider network Nonprovider service Traffic Provider service Provider service under study Our  approach   State  of  the  art   G=(V,  E,  λ)   V:   n<i>  -­‐  Sites,  services,  funcJonaliJes,  etc.   E:   e<i>  -­‐  Traffic  between  internal  nodes   λ(e):   Traffic  volume  (#Users)   JaneHe  Lehmann   n<e>  -­‐  External  sites   Networked  User  Engagement   15  
  16. 16. Network-LEVEL METRICS Network  metrics       Size     9 22     30   90 15    TrafficVolume                Def.:  Sum  of  edge  weights    A  high  value  is  a  sign  of  a  high  networked  user  engagement;      users  navigate  oHen  between  the  sites  of  the  network.   45 78 ConnecJvity                              Density    Def.:  ProporJon  of  edges  to  maximum  possible    A  high  value  is  a  sign  of  a  high  networked  user  engagement;      users  navigate  between  many  different  sites.                                  Reciprocity    Def.:  RaJo  of  number  of  edges  in  both  direcJons    A  high  value  is  a  sign  of  a  high  networked  user  engagement;    users  do  not  only  navigate  from  one  site  to  another,  they  also  tend  to  return      to  previously  visited  sites.     JaneHe  Lehmann   Networked  User  Engagement   16  
  17. 17. Network-LEVEL METRICS Network  metrics     Modularity          Modularity  [subNWmod]                      Def.:  Captures  the  existence  of  modules  based  on  a  random  walk  approach    A  high  modularity  indicates  that  users  visit  many  sites  of  one  subnetwork,      but  hardly  navigate  to  other  subnetworks.                            Global  Clustering  Coefficient  [GCC]      Def.:  Captures  the  existence  of  Jghtly  connected  groups  of  nodes    A  high  value  is  a  sign  of  a  high  networked  user  engagement;    users  access  sites  directly,  instead  of  using  front  pages.               Low GCC High GCC   JaneHe  Lehmann   Networked  User  Engagement   17  
  18. 18. ENGAGEMENT NETWORKS Modularity  [subNWmod]   Yahoo!  Network  with  3  countries   low  modularity  è  low  engagement   JaneHe  Lehmann   high  modularity  è  high  engagement   Networked  User  Engagement   18  
  19. 19. ENGAGEMENT NETWORKS Modularity  [subNWmod]   Yahoo!  Network  with  3  countries   Country   low  modularity     è  high  engagement   low  modularity  è  low  engagement   JaneHe  Lehmann   high  modularity  è  high  engagement   Networked  User  Engagement   19  
  20. 20. lem  by    [  embr  ]     Case  Study     Measuring     Networked  user   engagement  
  21. 21. ENGAGEMENT NETWORKS Interac%on  data   •  July  2011   •  2M  user,  25M  sessions   •  728  Yahoo!  sites  (news,  mails,  search,  etc.)   returning  edge   external     Networks   •  12  weighted  directed  networks,  filtered  by   »  »  »  »  Edge  types   Time   User  loyalty   Country    (internal,  internal  +  returning)    (Wednesday,  Sunday)    (causal,  acJve,  VIP)    (5  EU  countries)     •  Applying  metrics  on  networks,  results  are   normalized  using  the  z-­‐score   z-­‐score   above average Country1 Country2 Country3 Country4 Country5 average below average trafficVolume JaneHe  Lehmann   Networked  User  Engagement   21  
  22. 22. Network-LEVEL METRICS Edge-­‐based  networks               Size Connectivity trafficVolume Edgeint Edgeint+ext density reciprocity Timewed Timesun Modularity subNWmod Usercasual Usernormal GCC UserVIP   Leaving  the  network  does  not  necessarily  entail  less  engagement   •  Traffic  increases  significant  when  accounJng  for  returning  traffic  [18.26%]   •  ConnecJvity  increases  à  User  use  new  navigaJon  paths   •  Modularity  increases  à  Users  stay  in  the  same  part  of  the  network     JaneHe  Lehmann   Networked  User  Engagement   22  
  23. 23. Network-LEVEL METRICS Time-­‐based  networks               Size Connectivity trafficVolume Edgeint Edgeint+ext density reciprocity Timewed Timesun Modularity subNWmod Usercasual Usernormal GCC UserVIP   Naviga%on  is  more  goal-­‐oriented  during  the  week   •  TrafficVolume  and  density  are  lower  on  Sunday  à  Less  acJvity   •  Reciprocity,  subNW,  and  GCC  are  higher  on  Sunday  à  User  return  more  omen  to  already   visited  sites  and  visit  more  sites  than  usual  (browsing  with  less  specific  goals)     JaneHe  Lehmann   Networked  User  Engagement   23  
  24. 24. Network-LEVEL METRICS User-­‐based  networks               Size Connectivity trafficVolume Edgeint Edgeint+ext density reciprocity Timewed Timesun Modularity subNWmod Usercasual Usernormal GCC UserVIP   Loyal  users  have  also  a  higher  networked  user  engagement   •  TrafficVolume  is  higher  for  VIP  user  à  More  acJvity   •  ConnecJvity  and  modularity  is  higher  for  VIP  user  à  Users  are  interested  in  more  sites   •  GCC  is  lower  for  causal  users  à  Users  access  sites  using  the  front  pages     JaneHe  Lehmann   Networked  User  Engagement   24  
  25. 25. Network-LEVEL METRICS Country-­‐based  networks               Size trafficVolume Country1 Connectivity density Modularity reciprocity Country2 subNWmod GCC Country4 Country5 Country3   Networked  user  engagement  differs  between  countries   •  Country1:  Highest  trafficVolume  +  GCC,  high  density  +  reciprocity,  lowest  subNWmod  à  high   engagement   •  Country5:  Low  trafficVolume,  but  high  density    +  GCC,  low  subNWmod  à  less  popular,  but  high   engagement   JaneHe  Lehmann     Networked  User  Engagement   25  
  26. 26. CONCLUSION AND FUTURE WORK •  Network  analysis  enhances  the  understanding  of  user  engagement.   It  allows:   •  To  analyze  users  browsing  behavior  on  a  global  scale   •  To  compare  networks  of  sites  (e.g.  of  different  countries)   •  To  observe  differences  over  Jme   •  Leaving  the  network  does  not  entail  less  engagement   •  Some  network  metrics  are  more  useful  than  others   Future  work:   •  DefiniJon  of  metrics  that  combine  network  traffic  and  engagement   •  Comparing  traffic-­‐  and  hyperlink-­‐networks   •  Studying  the  effect  of  changes  in  the  network   JaneHe  Lehmann   Networked  User  Engagement   26  
  27. 27. Janebe  Lehmann   Universitat  Pompeu  Fabra,  Spain   lehmannj@acm.org     Mounia  Lalmas   Yahoo  Labs,  London,  UK   mounia@acm.org     Ricardo  Baeza-­‐Yates   Yahoo  Labs,  Barcelona,  Spain   rbaeza@acm.org     Elad  Yom-­‐Tov   Microsog  Research,  Herzliya,  Israel   elad@ieee.org    

×