Your SlideShare is downloading. ×
The Songs of Our Past
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

The Songs of Our Past

3,944
views

Published on

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,944
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. THE SONGS OF OUR PAST DOMINIKUS BAUR WORKING WITH LISTENING HISTORIES UNIVERSITY OF MUNICH (LMU) GERMANYHi,  I’m  Dominikus  from  the  University  of  Munich.  I’m  a  fourth  year  Ph.D.  student  and  today  I  will  talk  about  some  of  the  work  that  I’ve  done  so  far.  
  • 2. In this talk:As  you’ve  probably  already  guessed  from  the  Btle,  my  focus  is  on  a  special  type  of  personal  histories,  namely  music  listening  histories.  In  this  talk  I  will  first  describe  what  listening  histories  are  and  what  we  mean  by  that.  Then  I’ll  show  you  some  projects  from  the  area  of  informaBon  visualizaBon  where  we  worked  with  listening  histories  from  single  or  mulBple  people  and  tried  to  make  them  understandable  for  them.  Finally  I  will  give  you  some  ideas  what  else  than  visualizing  we  could  do  with  this  type  of  data.  
  • 3. Photos,  be  they  analog  or  digital,  are  a  common  way  to  remember  the  past.  We  all  take  photos  while  on  vacaBon,  having  friends  over  or  for  all  these  other  occasions  and  aIerwards  look  at  them  (or  don’t)  and  think  about  the  past.  But  what  we  do  in  our  lives  is  oIenBmes  so  much  more  than  a  photo  can  capture  
  • 4. [click]  unfortunately  all  auditory  informaBon  is  lost  in  the  process.  The  music  we  made,  the  songs  we  heard.  Nowadays,  of  course,  there  is  oIen  a  structural  difference  between  both  acBviBes:  The  Bme  we  spend  acBvely  making  music  is  significantly  smaller  than  the  Bme  we  spend  listening  to  it.
  • 5. But  even  though  we  no  longer  make  the  music,  it  is  more  abundant  than  ever  before  thanks  to  our  mobile  gadgets.  And  so  these  songs  by  people  we  don’t  know  sBll  stand  for  parts  of  our  lives:
  • 6. The  song  you  heard  during  this  one  summer…
  • 7. Or  the  one  that  was  playing  when  you  met  a  special  someone…
  • 8. …  and  of  course  the  songs  we  hear  every  year  for  special  occasions.
  • 9. REMINISCINGSo,  an  account  of  all  the  music  we  listened  to,  a  listening  history,  can  serve  for  reminiscing  just  as  well  as  photos.  In  this  regard,  listening  histories  are  a  part  of  the  so-­‐called  lifelogging  data.
  • 10. Lifelog A digital representation of all aspects of one’s lifeLifelogs  are  digital  representaBons  of  aspects  of  one’s  life.  So,  via  this  definiBon,  every  facebook  status  and  blog  entry  already  stands  as  a  part  of  lifelog  data.  But  the  original  vision  of  lifelogging  consists  of  capturing  really  everything  that  you  experience.  And  the  original  visionaries  went  …
  • 11. …  to  great  lengths  to  reach  that  goal.  So  while  capturing  listening  histories  is  only  a  humble  secBon  of  a  complete  lifelog,  they  can  sBll  bring  many  of  the  benefits.
  • 12. REMINISCING Sellen, Whittaker: Beyond Total Capture: A Constructive Critique of Lifelogging, CACM, May 2010In  a  recent  paper,  Abigail  Sellen  and  Steve  WhiXaker  idenBfied  some  of  the  benefits  that  lifelogging  data  can  bring  and  summarized  them  as  the  ‘5  Rs’.  We’ve  already  seen  reminiscing,  as  re-­‐living  the  past  for  emoBonal  reasons.
  • 13. RECOLLECTINGREMINISCINGRETRIEVINGREFLECTINGREMEMBERING Sellen, Whittaker: Beyond Total Capture: A Constructive Critique of Lifelogging, CACM, May 2010But  there’s  more.
  • 14. RECOLLECTINGREMINISCINGRETRIEVINGREFLECTINGREMEMBERING Sellen, Whittaker: Beyond Total Capture: A Constructive Critique of Lifelogging, CACM, May 2010RecollecBng  is  the  more  general  (and  less  emoBonal)  case  of  reminiscing  and  can,  for  example,  mean  using  a  listening  history  to  find  a  song  whose  name  I  have  forgoXen.
  • 15. RECOLLECTINGREMINISCINGRETRIEVINGREFLECTINGREMEMBERING Sellen, Whittaker: Beyond Total Capture: A Constructive Critique of Lifelogging, CACM, May 2010Retrieving  is  more  appropriate  for  text-­‐  and  other  documents,  but  it  can  also  mean  that  I  can  immediately  listen  to  that  song.
  • 16. RECOLLECTINGREMINISCINGRETRIEVINGREFLECTINGREMEMBERING Sellen, Whittaker: Beyond Total Capture: A Constructive Critique of Lifelogging, CACM, May 2010ReflecBng  describes  the  process  of  thinking  about  your  life  using  the  lifelog.  Say,  I  listened  to  a  fair  share  of  pop  and  rock,  now  it’s  Bme  to  become  serious  and  listen  to  classical  music.
  • 17. RECOLLECTINGREMINISCINGRETRIEVINGREFLECTINGREMEMBERING Sellen, Whittaker: Beyond Total Capture: A Constructive Critique of Lifelogging, CACM, May 2010And  finally,  remembering  intenBons  describes  thinking  about  prospecBve  acBviBes,  such  as  regularly  checking  if  a  band  has  a  new  album  or  is  on  tour.
  • 18. RECOLLECTINGREMINISCINGRETRIEVINGREFLECTINGREMEMBERING Sellen, Whittaker: Beyond Total Capture: A Constructive Critique of Lifelogging, CACM, May 2010So,  these  ‘five  Rs’  provide  a  good  overview  of  the  possible  benefits  of  capturing  listening  histories.  
  • 19. Let  me  talk  a  liXle  bit  about  what  listening  histories  are  and  where  they  come  from.
  • 20. Listening history A complete chronological collection of musical items …In  my  understanding  an  ideal  listening  history  describes  all  songs  that  a  person  has  listened  to,  possibly  in  their  lifeBme.  What’s  important  here  is  that  …
  • 21. ... Each song: (1) pre-existing piece of music ...Each  song  is  a  pre-­‐exisBng  piece  of  music  that  has  aXributes  such  as  arBst,  Btle,  etc.
  • 22. … (2) has been heard at least partiallyAnd  second,  each  song  has  been  heard  by  the  owner  of  the  history  at  least  in  parts.  So,  the  quesBon  is,  where  do  we  get  such  data  from?
  • 23. Fortunately,  there’s  a  popular  service  called  ‘last.fm’  that’s  been  around  for  a  while  and  does  exactly  that.  Last.fm’s  actual  intenBon  for  capturing  a  person’s  listening  behavior  is  providing  beXer  recommendaBons  for  their  webradio,  but  the  resulBng  listening  histories  are  easily  accessible  through  their  API  which  makes  them  a  perfect  target  for  all  kinds  of  projects.
  • 24. Last.fm’s  tracking  technology  is  called  ‘Audioscrobbler’,  which  is  both  a  protocol  and  a  soIware.  Devices  and  media  players  can  either  use  the  protocol  directly  or  rely  on  the  background  audioscrobbler  process  running  on  the  user’s  machine.  And  in  the  end  we  arrive  at  a  chronological  list  of  /all/  the  songs  a  person  has  listened  to...
  • 25. + =But  while  this  could  be  used  to  hypotheBcally  capture  the  complete  listening  history  of  a  person  and  works  great  in  theory  [click]
  • 26. Reality  oIen  looks  a  bit  different.
  • 27. Real listening histories: - incomplete - noisyThe  actual  resulBng  listening  histories  are  both  incomplete  and  noisy.  Let  me  just  tell  you  what  I  mean  by  that.
  • 28. Real listening histories: - incomplete - noisyGaps  in  a  listening  history  can  come  from  various  places…
  • 29. One  common  source  is  that  the  listener  is  using  non-­‐supported  hardware  for  listening
  • 30. Another  that  music  comes  from  other  sources  like  when  shopping  or  being  at  a  friend’s  place.
  • 31. Real listening histories: - incomplete - noisyNoise,  i.e.,  too  many  songs  are  tracked  is  also  quite  common…
  • 32. The  user  might  leave  the  computer  while  the  music  keeps  on  playing…
  • 33. Or  someone  else  is  using  the  computer  while  the  audioscrobbler  is  sBll  running.
  • 34. > 50%Another Caveat: The audioscrobbler only tracks a song after the user has listened to atleast half of it. Again, keep in mind that the main incentive for last.fm to track listeningis improving the recommendations of their web radio: If a song is skipped then thelistener probably didn’t like it and it’s uninteresting for recommendations.
  • 35. 30 million users / month (March 2009) http://blog.last.fm/2009/03/24/lastfm-radio-announcementSBll,  despite  these  downsides,  last.fm’s  data  is  preXy  reliable  and  the  service  is  very  popular.  According  to  them,  30  million  people  visit  the  webpage  per  month.
  • 36. In  the  end  we  arrive  at  a  chronological  list  of  songs  and  that’s  all  we  get.  Each  secBon  of  Bme  either  contains  music  or  it  does  not.  So  we  have,  for  example,  no  informaBon  on  the  context  of  the  music  listening  (I’ll  get  back  to  that  aspect  later).  SBll,  to  make  it  easier  to  understand  this  data  we  can  then  start  to  analyze  it  and  e.g.,  to  extract  listening  sessions.
  • 37. Listening  sessions  are  characterized  by  the  gaps  between  the  songs,  so  a  gap  of  e.g.,  half  an  hour  between  two  songs  means  that  the  creator  of  the  history  stopped  listening  and  thus  ended  the  session.  To  make  these  histories  a  bit  more  meaningful  we  can  also  go  beyond  the  single  Bme  dimension…  
  • 38. Genre ……Sub-Genre Artists Albums Songs…  and  put  the  songs  into  the  musical  hierarchy  of  albums,  arBsts  and  genres.  While  this  classificaBon  is  not  perfect  and  oIenBmes  the  topic  of  heated  debates,  at  least  it’s  widely-­‐known  among  all  music  listeners.  One  more  step  to  overcome  the  downsides  of  a  strict  hierarchy  is  adding  user-­‐generated  keywords  into  the  mix…
  • 39. Genre ……Sub-Genre Artists Albums Songs Tags…  that  can  become  a  stand-­‐in  for  any  number  of  different  hierarchies  or  classificaBons.  You  will  see  some  of  these  aspects  in  the  prototypes  that  I’m  about  to  show  you.
  • 40. But  back  to  the  actual  benefits  to  the  creators  of  such  listening  histories,  think  of  the  ‘5  Rs’  of  reminiscing,  recollecBng  and  so  on.  Here  you  can  see  the  default  view  of  Last.fm  presenBng  this  data:  A  chronological  web-­‐based  list  which  is  not  that  helpful  for  any  of  these  tasks.  And  as  you’ve  just  seen,  listening  histories  can  become  quite  complex  once  you  dive  into  their  depths  which  makes  other  forms  of  presentaBon  more  useful.
  • 41. As  a  first  step  towards  understanding  this  data  and  also  making  their  owners  understand  them  I  started  building  visualizaBons  based  on  it.  To  make  maXers  not  overly  complicated,  …
  • 42. …  I  used  only  a  single  listening  history,  i.e.  a  possibly  long  list  of  possibly  repeated  songs.    
  • 43. My  first  approach  to  visualizing  this  type  of  data  was  a  node-­‐link  diagram:  The  idea  was  that  each  unique  song  would  be  represented  as  a  node  …
  • 44. …  while  each  pair  of  consecuBve  songs  would  form  an  edge  in  the  diagram.  And  while  this  concept  was  easy  to  understand,  the  result  wasn’t  –  necessarily.  And  you  might  also  understand  why  I  Btled  this  visualizaBon  ‘ Tangle’:
  • 45. Here  be  a  chaoBc  screenshot  of  tangle TangleWhile  it  certainly  looks  chaoBc,  there  are  sBll  several  aspects  that  you  can  draw  from  it:
  • 46. Here  be  a  chaoBc  screenshot  of  tangle TangleFor  one,  the  layout  of  the  nodes  is  force-­‐directed,  which  means  that  nodes  with  many  edges  (i.e.  songs  that  appear  repeatedly  within  the  history)  are  drawn  towards  the  center,  …
  • 47. Here  be  a  chaoBc  screenshot  of  tangle Tangle…  while  less  popular  songs  and  one-­‐hit-­‐wonders  are  on  the  outskirts.  
  • 48. Here  be  a  chaoBc  screenshot  of  tangle TangleAn  addiBonal  encoding  is  the  thickness  of  the  connecBng  arrows  that  represents  the  number  of  Bmes  this  two-­‐song-­‐sequence  was  played  which  shows  albums  and  pre-­‐defined  playlists.
  • 49. VIDEO Tangle[VIDEO]  And  finally,  Tangle’s  layout  is,  as  I  said,  force-­‐directed  which  means  that  the  user  is  able  to  interacBvely  explore  the  visualizaBon.  Zooming  and  panning  is  of  course  possible.  By  hovering  over  a  song  addiBonal  informaBon  is  shown.  And  the  user  can  drag  around  songs  at  will.
  • 50. As  it  was  not  easy  to  learn  much  from  the  Tangle  visualizaBon,  I  wanted  to  put  some  sense  into  it.  Filtering  or  splihng  the  data  seemed  promising,  so  I  focused  on  listening  sessions  this  Bme.  
  • 51. The  basic  idea  was  again  a  node-­‐link  diagram,  but  this  Bme  songs  could  appear  more  than  once.  
  • 52. This  Bme  the  more  important  factors  were  the  Bme  stamps  of  the  songs.  A  pause  of  in  this  case  1  hour  indicates  the  start  of  a  new  listening  session.
  • 53. StringsBy  sorBng  the  sessions  chronologically  we  arrived  at  this  visualizaBon,  called  ‘Strings’.  
  • 54. StringsZooming  out  gives  you  an  overview  of  the  length  of  your  listening  sessions,  shows  outliers  and  Bmes  when  you  didn’t  listen  to  music.  The  verBcal  Bme  line  is  very  important  in  this  regard.
  • 55. StringsFinally,  you  probably  wondered  about  the  blue-­‐ish  arcs:  The  problem  with  Strings  is  that  each  song  can  possibly  appear  several  Bmes  in  the  visualizaBon  as  single  songs  are  no  longer  represented  by  single  nodes.  Therefore,  we  draw  arcs  between  idenBcal  songs  which  makes  it  possible  to  gauge  the  importance  of  one  song  or  see  repeBBve  sequences  (at  the  boXom).
  • 56. ?So,  what  these  two  examples  had  in  common  that  they  were  both  restricted  visualizaBons  that  (1)  focussed  on  one  aspect  of  the  data  and  (2)  allowed  only  liXle  interacBon.
  • 57. playfulIf  you  want  to  put  a  label  on  them  it  would  probably  be  ‘playful’  which  means:  They  are  designed  for  one  specific  aspect  of  the  data  which  cannot  be  customized.  They’re  built  for  this  task  only.  But  they  can  sBll  engage  the  user  to  play  around  and  interact  (at  least  a  liXle).
  • 58. playful casual expertIf  you  want  to  put  this  into  an  infovis  perspecBve,  two  other  commonly  used  terms  are  useful:  ‘Casual’  describes  visualizaBons  that  are  a  liXle  more  interacBve  and  customizable  but  not  as  complex  as  ‘expert’  systems  that  allow  fine-­‐grained  customizaBon  but  require  solid  knowledge  in  the  respecBve  area.
  • 59. playful casual expertFor  visualizaBons:  A  type  of  playful  visualizaBon  would  be  Wordle,  engaging  but  with  a  single  purpose.  The  Many  Eyes  project  is  easy  to  use  but  has  much  more  ways  to  display  and  filter  the  data.  Finally,  programming  frameworks  such  as  protovis  or  processing  allow  utmost  flexibility  but  are  difficult  to  get  into  and  master.
  • 60. Interactivity expert casual playful ComplexityIf  we’re  inclined  to  put  these  three  concepts  into  relaBon  to  each  other,  we  can  use  interacBvity  and  complexity.  So,  playful  tools  aren’t  very  flexible,  but  also  not  very  complex.  Expert  tools  however  are  mulB-­‐purpose  and  highly  interacBve  but  also  difficult  to  master.  It  depends  on  the  user  populaBon  and  the  task  what  visualizaBon  concept  to  choose.  
  • 61. Interactivity expert casual playful ComplexityIn  our  case  with  listening  histories,  we  have  people  who  like  to  listen  to  music  and  are  not  necessarily  infovis-­‐experts.  Also,  analyzing  their  listening  behavior  is  something  they  don’t  do  regularly  so  forcing  them  to  learn  something  for  using  a  complex  visualizaBon  will  rather  put  them  off  than  engage  them.  Therefore  I  concentrated  on  the  playful/casual  corner  of  this  design  space.
  • 62. Ok,  so  back  to  the  visualizaBon.  Both  Strings  &  Tangle  were  very  single  purpose  and  liXle  customizable.  For  the  next  project,  I  wanted  to  give  users  more  freedom  in  analyzing  their  listening  histories  but  sBll  keep  the  tool  accessible.  Strings  &  Tangle  were  also  only  informally  evaluated  with  a  few  people  from  our  lab  so  I  wanted  to  see  if  real  people  would  actually  find  something  like  that  useful…
  • 63. LastHistory...  The  result  was  LastHistory,  a  /casual/  infovis  tool  for  analyzing  and  reminiscing  in  one’s  own  listening  history.  We  made  it  available  on  the  internet.  Several  thousand  people  downloaded  it  and  we  received  lots  of  feedback.  When  designing  LastHistory  we  first  wanted  to  make  sure  that  it  felt  easily  accessible  for  people.  The  visualizaBon  in  its  non-­‐interacBve  state  should  already  give  insights  to  the  user,  and  so  gradually  lure  them  into  exploring  the  more  sophisBcated  opBons.  
  • 64. LastHistorySo,  the  largest  part  of  the  applicaBon  is  taken  up  with  a  2D  Bmeline:  all  songs  are  represented  as  small  circles  and  mapped  horizontally  to  the  day  and  verBcally  to  the  Bme  of  day  of  their  Bmestamps.  This  way,  users  can  easily  see  daily  rhythms,  
  • 65. LastHistoryLike  at  what  Bme  this  person  usually  went  to  bed.
  • 66. LastHistoryAnd  here’s  another  example:  A  user  who  gets  up  at  the  same  Bme  everyday  and  listens  to  music  first  thing  in  the  morning.  
  • 67. classical jazz funk hip-hop electronic rock metal unknown/other LastHistoryEach  song’s  genre  is  color-­‐coded,  so  the  user  gets  an  immediate  overview  over  the  variety  of  songs.  We’re  of  course  restricted  in  the  number  of  colors  we  can  use  to  keep  them  disBnguishable.
  • 68. LastHistoryBeyond  staBc  visualizaBon,  users  can  navigate  within  the  visualizaBon  by  panning,  triggered  by  dragging  with  the  mouse  
  • 69. LastHistoryOne-­‐dimensional  zooming  by  using  the  mouse’s  zoom  wheel  or  the  slider  in  the  lower  right  corner  allows  them  to  focus  on  certain  secBons  of  the  history.
  • 70. LastHistoryHovering  over  a  song  shows  a  box  with  user-­‐generated  keywords  from  last.fm,  but  more  prominently:  connects  this  song  with  all  other  instances  of  it  throughout  the  history.  So,  users  can  easily  see  when  they  listened  to  this  one  song.
  • 71. LastHistoryPreceding  and  succeeding  repeated  songs  are  also  highlighted,  so  sequences  such  as  albums  or  other  predefined  playlists  are  automaBcally  highlighted.
  • 72. LastHistoryFinally,  in  the  upper  right  corner  of  the  applicaBon,  there’s  a  textbox  for  filtering  where  users  can  enter  freeform  terms.  It’s  possible  to  enter  song  or  album  Btles  or  arBst  names  to  filter  all  other  songs.  
  • 73. LastHistoryBut  the  filter  box  can  also  be  used  for  temporal  queries  by  entering  dates,  or  periods  of  Bme,  so  users  can,  for  example,  see  all  songs  that  they  listened  to  in  autumn  before  noon.  But  enough  with  the  default  infovis-­‐features.  One  interesBng  aspect  of  this  project  was  that  we  could  use  an  addiBonal  data  source  for  gaining  insights:  The  user’s  memories.
  • 74. To  access  these,  we  needed  memory  triggers.  Some  research  in  psychology  has  shown  that  personally  created  things  such  as  photos  can  be  useful  in  this  regard,  so  we  integrated  photos  and  calendar  entries  from  the  user’s  harddisk.
  • 75. Two usage modes: Analysis PersonalWe  split  these  into  two  different  usage  modes  and  called  them  ‘analysis’  (everybody  can  do  it)  and  ‘personal’  (with  memory  triggers  that  probably  are  only  useful  to  the  owner  of  the  history).  So  in  this  personal  mode  we  have  contextual  informaBon  that  makes  it  easier  to  remember  what  happened  at  what  Bme  and  understanding  the  listening  decisions.  Users  could  simply  switch  between  the  two  modes  with  the  buXon  in  the  upper  leI  corner.
  • 76. Ok,  so  much  for  the  tool.  As  I  said,  we  made  it  available  on  the  internet  and  a  lot  of  people  downloaded  it.  
  • 77. Praise on tech blogs 5,000 downloads 243 filled-out questionnairesSome  numbers:  First  we  got  a  good  amount  of  coverage  on  tech  blogs,  which  led  to  a  certain  popularity.  Right  now,  we  have  about  5,000  downloads.  We  also  included  a  link  to  a  quesBonnaire  that  pops  up  aIer  fiIeen  minutes  of  using  the  tool  and  around  250  people  answered  that  quesBonnaire.  We  kept  that  intenBonally  short,  in  order  not  to  put  off  people  as  a  short  answer  is  beXer  than  no  answer  at  all.  
  • 78. About the Personal Mode: “I like this mode the best, it should be the default mode!” “Clicking on a photo gallery and listening to what I was listening to at the time was very powerful.”People  who  had  photos  and  calendar  entries  available  enjoyed  using  the  personal  mode  and  also  features  like  the  possibility  to  create  a  slideshow  of  music  and  photos.  
  • 79. About the Analysis Mode: “I rarely listen to music between the hours of 9-11 a.m., even on weekends.” “I noted the … commuting pattern.” “Those ruts where you get stuck in listening to one particular song.” “I listened to music for 4 straight days!”And  in  general,  people  were  also  able  to  find  repeaBng  paXerns  and  liked  how  they  were  able  to  learn  interesBng  aspects  about  themselves.
  • 80. 75% found it easy to use and learn.Another  thing  we  learned  that  worked  really  well  was  puhng  a  five  minute  video  online  that  explained  how  to  use  the  tool:  There  was  no  online  help  or  something  like  that  available  and  sBll  75%  percent  found  it  easy  to  learn  and  use.
  • 81. Finally,  people  really  liked  to  share  the  results.  And  as  there  was  no  straighporward  way  to  do  it  within  the  applicaBon,  they  resorted  to  taking  screenshots  for  posBng  it  on  flickr  or  their  blogs.  So  a  future  version  should  definitely  take  that  into  account.
  • 82. AOk,  so  these  were  three  examples  for  visualizing  single  listening  histories.  But  it  gets  much  more  interesBng  when  we  have  not  one  history…
  • 83. E D C B A…  but  many  more.  And  as  music  has  an  intricate  social  funcBon  as  well,  comparing  one’s  taste  in  music  to  friends,  family  and  peers  can  be  an  interesBng  use  case.
  • 84. So  in  the  following,  I  will  give  you  two  examples  for  approaches  to  visualize  these  data.  Again,  I’ll  present  one  playful  and  one  casual  approach.
  • 85. B AWhile  mulBple  histories  can  mean  a  lot  of  histories,  for  a  first  approach,  I  decided  to  focus  on  just  two  histories.  One  will  usually  be  the  user’s  history  and  the  other  one  of  a  friend  or  another  person  that  he  or  she  knows.  
  • 86. Ok,  so  what’s  the  best  way  to  do  that:  Aligning  the  songs  to  a  Bme-­‐line  is  probably  a  good  idea,  to  allow  comparisons  for  the  number  of  songs,  regularity  of  listening  and  so  on.  But  users  are  especially  in  this  one-­‐on-­‐one  scenario  interested  in  also  comparing  their  taste:  Are  both  of  them  listening  to  the  same  songs  or  arBsts?  Or  is  there  no  similarity?
  • 87. An  easy  way  to  encode  that  is  using  the  distance  between  the  songs  from  each  history:  The  closer  a  song  gets  to  the  other  history,  the  more  similar  it  is  to  it,  resulBng  in  a  fever  chart  of  relatedness.
  • 88. LoomFMHere’s  an  example  from  the  resulBng  ‘LoomFM’  visualizaBon.  You  have  a  horizontal  Bmeline  and  two  listening  histories  from  user  red  and  purple.  The  closer  one  of  the  small  song  circles  comes  to  the  Bmeline,  the  more  related  it  is  to  the  other  user’s  taste  in  music.  
  • 89. LoomFMSome  more  things:  The  more  consecuBve  songs  share  the  same  genre  or  arBst,  the  larger  the  corresponding  label  gets.  By  doing  this,  important  arBsts  are  visible  even  when  zoomed  out.  Also,  labels  that  both  users  share  at  the  same  point  in  history  move  to  the  center  of  the  Bmeline.
  • 90. LoomFMAddiBonally,  the  yellow  arcs  connect  idenBcal  songs  –  the  same  principle  as  in  the  ‘Strings’  visualizaBon.  Using  this  approach,  you  can  get  a  sense  if  a  person  repeatedly  listens  to  the  same  songs  (as  user  red  in  this  example)  or  only  once.  Also,  songs  that  both  users  share  are  connected…
  • 91. LoomFM…  as  in  this  example,  where  a  new  album  of  ‘ Trail  of  Dead’  was  released  and  both  users  gradually  started  listening  to  it.  (you  can  also  clearly  see  the  sequence  here)
  • 92. VIDEO LoomFMHere  you  can  see  a  video  of  LoomFM.  As  always,  zooming  and  panning  are  possible  and  gehng  more  informaBon  by  hovering  over  songs  or  arcs.
  • 93. playful casual expertLoomFM  consBtutes  an  example  of  a  playful  visualizaBon  for  mulBple  histories.  The  tasks  are  clearly  defined,  interacBon  is  minimal  and  a  lot  of  informaBon  (e.g.  the  similarity  between  songs)  is  implicit  and  predefined…
  • 94. Screenshot Of  LastLoopplayful casual expert…  to  overcome  the  restricBons  and  also  to  integrate  more  than  two  histories  we  did  another  project  called  LastLoop  and  aimed  more  for  the  casual  area.
  • 95. The  basic  idea  was  to  have  a  cross  between  LastHistory  and  LoomFM,  to  give  users  the  chance  to  do  these  more  complex  analyses  using  filtering  and  things  like  that  while  also  being  able  to  connect  the  different  listening  histories  and  see  relaBons  between  them.
  • 96. LastLoopHere’s  the  result  that  we  called  ‘LastLoop’.  What  you  can  see  here  are  three  listening  histories  (you  can  have  an  unlimited  number  of  verBcally  stacked  histories),  arranged  to  the  same  Bmeline.
  • 97. LastLoopWe  used  the  2D  Bmeline  metaphor  from  LastHistory  once  more,  so  it’s  possible  to  see  daily  paXerns  across  all  histories.
  • 98. LastLoopAlso,  by  hovering  above  a  song,  all  other  occurences  within  this  one  history  and  the  others  are  highlighted  (re-­‐using  the  metaphor  from  the  other  projects).  
  • 99. LastLoopThe  user  can  also  select  a  whole  area  and  again,  see  where  else  the  songs  appear.
  • 100. LastLoopFinally,  to  make  the  informaBon  manageable,  users  can  also  search  for  songs,  arBsts,  albums  and  so  on…
  • 101. LastLoop…  or  filter  for  certain  genres.
  • 102. VIDEO LastLoopAnd  here’s  the  system  in  acBon:  You  can  pan  and  zoom  either  by  using  the  mouse  or  the  Bme  slider  at  the  boXom  of  the  screen,  select  screen  regions  to  see  other  occurences  of  the  selected  songs  (and  switch  between  all,  songs  from  the  selecBon  or  songs  within  the  selecBon  only).  Aaaaaand  you  can  also  highlight  songs  or  arBsts  …  and  filter  for  certain  genres.
  • 103. http://www.lastloop.deSo,  to  evaluate  the  system  we  followed  the  same  strategy  that  had  already  worked  with  LastHistory.  We  made  the  tool  available  on  the  web  (and  this  Bme  it  was  even  wriXen  in  Java  and  thus  plaporm-­‐independent,  while  LastHistory  was  Mac-­‐only).  You  could  -­‐  and  sBll  can  -­‐  run  it  easily  in  your  browser.
  • 104. For  learning  the  applicaBon,  we  provided  another  five-­‐minute-­‐video  that  explained  the  basics  of  interacBon  and  to  capture  the  users’  findings  we  had  another  short  quesBonnaire  …
  • 105. …  and  we  also  had  ‘feedback’  buXon  in  the  upper  leI  of  the  applicaBon  where  users  could  click  on,  provide  what  they  found  and  send  it  directly  back  to  us.
  • 106. 21 filled-out questionnaires (3 incomplete)So,  while  we  were  preXy  convinced  that  we  did  everything  right,  the  response  was  less  than  stellar.  AIer  one  month  we  had  21  responses  to  the  quesBonnaire  and  a  few  with  the  direct  feedback  buXon.  
  • 107. Insights gained: “That one user is also listening to a very infamous band from the 70s” “When did the other user hear my favorite song, have there been many connections lately, …”What  we  found  was  that  people  learned  about  themselves  and  others,  which  was  the  goal  of  the  visualizaBon  and  we  were  happy  that  it  worked.  But  we  wanted  to  find  out  what  went  wrong…
  • 108. Selecting a song was sketchy Results were cluttered and unclear…  and  the  problems  were  mostly  due  to  usability  issues  and  the  general  complexity  of  the  applicaBon.  People  found  it  difficult  to  accurately  select  a  song  as  the  selecBon  was  only  based  on  the  horizontal  posiBon  of  the  cursor  and  not  the  verBcal  (so  it  became  very  hard  to  select  a  specific  song  when  zoomed  out).  Also,  people  liked  how  the  results  looked  but  couldn’t  make  much  sense  of  them.  It  was  oIen  just  too  much  informaBon  in  too  liXle  space,  so  drawing  any  insights  other  than  very  superficial  ones  was  difficult.
  • 109. Screenshot Of  LastLoopplayful casual expertSo  what  we  learned  was  that  even  when  we  fixed  the  usability  issues,  LastLoop  would  probably  sBll  be  more  of  an  expert-­‐  than  a  casual  visualizaBon.  
  • 110. Screenshot Of  LastLoopplayful casual expertOk,  now  that  you’ve  seen  5  examples  for  visualizaBons  of  listening  histories  that  approached  different  aspects  of  the  topic,  where  do  we  go  from  here?
  • 111. RECOLLECTINGREMINISCINGRETRIEVINGREFLECTINGREMEMBERING Sellen, Whittaker: Beyond Total Capture: A Constructive Critique of Lifelogging, CACM, May 2010VisualizaBon  is  nice  and  all,  but  there  is  more  that  we  can  do  with  these  histories.  It’s  nice  to  give  the  creators  of  these  histories  the  chance  to  recollect,  reminisce  and  so  on,  but  we  can  also  use  them  to  make  their  day-­‐to-­‐day  interacBon  with  music  easier  and  more  convenient.  
  • 112. In  these  last  few  minutes  of  my  talk  I  will  show  you  two  examples  of  how  to  use  this  data  in  other  areas.
  • 113. One  problem  with  listening  to  music  is  that  there  a  mostly  only  two  ways  to  do  it:  You  either  manually  create  a  playlist  or  pick  an  album  or  have  it  done  fully  automaBcally.  The  former  makes  it  very  tedious  to  listen  to  music  (especially  on  the  go),  while  the  laXer  restricts  you  to  the  choice  of  the  machine  that  might  be  giving  you  the  same  songs  over  and  over  again  and  you  have  very  liXle  influence  on  that.  
  • 114. RushWith  our  Rush-­‐interacBon  technique  we  wanted  to  create  and  opBon  for  building  playlists  between  the  two  extremes  and  we  called  this  approach  ‘repeated  recommendaBons’…
  • 115. VIDEO RushYou  start  just  like  in  the  automaBc  case  with  a  hand-­‐picked  seed  song  and  receive  a  set  of  five  recommendaBons  for  this  item.  Once  you  choose  once  of  these  items,  you  get  another  set  of  five  and  so  on  and  so  forth.  The  great  thing  about  this  approach  is  that  you  do  not  have  the  large  overhead  of  going  through  your  whole  collecBon  to  create  a  playlist,  but  sBll  have  much  more  freedom  than  in  the  purely  automaBc  case.  
  • 116. So  where  do  listening  histories  come  in  here?  First,  we  can  of  course  use  them  to  shape  the  recommended  items.  In  our  study  we  used  a  pre-­‐defined  set  of  music  and  general  recommendaBons  from  last.fm  but  it  would  of  course  make  more  sense  to  adapt  the  recommendaBons  based  on  the  user’s  history….
  • 117. …  second:  Five  items  is  not  a  lot,  so  it  is  difficult  to  choose  the  right  ones  in  order  not  to  frustrate  the  user.  Having  his  or  her  listening  history  available  means  that  we  can  automaBcally  remove  candidates  that  the  user  does  not  know  (and  would  not  be  very  helpful  in  this  scenario).  
  • 118. RECOLLECTINGAnother  thing  that  you  can  do  when  working  with  listening  histories  is  use  them  for  rediscovery  of  music  that  you  forgot.  That  was  something  that  we  oIen  observed  when  people  used  one  of  the  visualizaBons  that  they  were  happy  to  find  some  song  or  arBst  that  they  had  forgoXen  about.  
  • 119. But  using  the  visualizaBons  is  an  explicit  acBvity  and  people  commonly  use  different  soIware  to  actually  listen  to  music.  So  in  this  last  project,  we  wanted  to  help  them  with  recollecBng  and  reminiscing  while  they  were  actually  listening  to  music.
  • 120. So  we  decided  to  make  a  plugin  for  a  media  player.  Because  we  wanted  to  keep  it  useful  for  as  many  people  as  possible  we  chose  Songbird,  an  open  source  media  player  with  an  acBve  community,  that’s  available  for  Mac  and  Windows  instead  of  iTunes  or  the  Windows  Media  Player.
  • 121. Our  idea  for  supporBng  rediscovery  was  based  on  the  idea  that  also  the  Tangle  visualizaBon  was  based  on:  Every  Bme  a  song  appears  in  a  listening  history  it  has  successors  and  predecessors.  And  this  order  of  songs  is  probably  important  for  the  listener,  not  always,  of  course,  but  at  least  someBmes.  So  the  idea  was  to  show  for  the  currently  playing  song  whatever  songs  appeared  before  and  aIer  it.
  • 122. SongSlopeThe  result  looks  like  this:  By  doing  what  they  would  have  done  anyway,  namely  listening  to  music,  users  automaBcally  receive  a  focused  glimpse  into  their  listening  past.  All  songs  before  and  aIer  are  displayed  and  they  can  switch  to  one  of  these  songs  simply  by  clicking  on  them.  
  • 123. SongSlope…  and  users  can  also  switch  to  a  view  of  the  underlying  listening  sessions,  browse  through  them  or  listen  to  them  as  a  new  playlist.
  • 124. Currently: 7,200 downloads 58 filled-out questionnaires (40 partial)We  had  a  lot  of  downloads  (as  I  said,  Songbird  has  a  very  acBve  community)  but  not  as  many  answers  to  the  quesBonnaire,  probably  because  we  had  no  pop-­‐up  or  email  reminder  to  fill  it  out.  We  also  logged  the  relevant  aspects  of  the  user’s  interacBon  with  the  plug-­‐in  (of  course,  only  aIer  they  agreed  to  that).
  • 125. Use cases: 44.8% Re-discovering music 31.0% Generating playlistsWe  were  especially  interested  in  what  people  used  it  for  and  found  that  almost  half  of  them  were  able  to  rediscover  music  with  it,  but  also  almost  a  third  used  it  for  creaBng  playlists  (or  relistening  to  old  playlists).  So  even  though  only  a  couple  of  people  answered  the  quesBonnaire  we  got  very  posiBve  feedback  from  them.
  • 126. Ok,  so  where  does  that  leave  us  and  what  can  you  take  away  from  this  talk:
  • 127. Listening  histories  are  today  mostly  used  for  recommendaBon.  But  as  they  are  a  type  of  personal  data  that  can  be  easily  collected  and  sBll  can  have  a  powerful  impact  into  people’s  lives  using  them  for  recommending  music  only  is  –  I  think  –  somewhat  of  a  waste.  We  can  do  much  more  with  them.  
  • 128. Screenshot Of  LastLoopplayful casual expert…  as  you’ve  seen:  We  can  visualize  this  informaBon  to  allow  people  to  reminisce  about  their  past  and  recollect  their  memories,  in  varying  degrees  of  complexity  and  for  various  approaches  to  the  topic…  
  • 129. And  beyond  navel-­‐gazing  we  can  also  use  this  data  for  helping  people  with  listening  to  music:    We  can  use  listening  histories  to  improve  the  usability  on  mobile  devcies  for  quickly  and  conveniently  creaBng  personalized  playlists  on  the  go  or  to  add  value  by  lehng  people  painlessly  rediscover  music  while  listening  to  it  anyway.
  • 130. Genre ……Sub-Genre Artists Albums Songs TagsSo,  for  three  more  concrete  results  that  I  learned  while  working  this  topic:  It’s  probably  a  good  idea  to  use  a  Bmeline  as  the  central  metaphor  for  represenBng  personal  histories,  as  the  temporal  aspect  is  very  important  for  filing  this  data  into  one’s  personal  life  story.  Also,  abstracBons  such  as  genre  hierarchies  are  great  for  reducing  the  complexity  of  the  data  while  preserving  the  access  to  single  items.
  • 131. 131Second,  for  collecBng  results  from  casual  users  several  approaches  can  be  helpful:  We  had  quesBonnaires  that  popped  up  aIer  a  while  in  LastHistory,  we  tracked  relevant  interacBon  with  the  user’s  consent  to  learn  about  how  an  applicaBon  is  used  and  where  it  fails  (in  SongSlope)  and  finally,  the  feedback-­‐buXon  that  we  had  in  LastLoop  allowed  for  impromptu  feedback  with  minimal  overhead.
  • 132. Finally,  one  very  interesBng  data  source  that  we  tapped  when  creaBng  LastHistory  were  the  user’s  memories.  These  memories  can  give  context  and  meaning  to  plain  lists  of  songs  and  by  using  suitable  memory  triggers  it’s  possible  to  unearth  great  stories  and  understand  these  histories.  Depending  on  the  use  case,  visualizaBon  shouldn’t  underesBmate  the  value  of  having  a  real  person  sihng  in  front  of  the  machine.
  • 133. I  think  the  central  part  is  that  these  histories  are  reflecBons  of  their  creators’  lives:  Music  accompanies  them  during  their  good  and  their  bad  Bmes,  their  triumphs  and  their  tragedies  and  forms  an  inseparable  bond  with  these  events.  But  what  they  are  lacking  are  the  tools  to  use  them  in  the  same  way  that  they  use  photos  for  reflecBng  about  their  past  and  making  sense  of  their  lives.  So  I  hope  my  work  is  a  first  step  towards  giving  this  data  back  to  the  people  who  created  it.
  • 134. DOMINIKUS BAUR UNIVERSITY OF dominikus.baur@ifi.lmu.de MUNICH (LMU), twitter: @dominikus GERMANYThank  you!
  • 135. Image credits I//
  • 136. Image credits II