This document proposes an interactive tool called MULHER for exploring music listening histories. It describes issues with existing visualizations like being static, lacking interactivity, and scaling issues. MULHER uses a timeline-based visualization combined with other synchronized views to allow interactive browsing, filtering, and detection of listening patterns. An evaluation with users found the timeline mechanism was effective for exploration. Context information and representing the "age" of songs also helped convey listening habits. The tool enabled users to analyze their profiles and infer hidden behaviors from their music listening data.

1. Interactive Exploration of
Music Listening Histories
Ricardo Dias, Manuel J. Fonseca,
Daniel Gonçalves

2. Context & Motivation

3. Proliferation of lifelogging services

4. Tracking Listening Habits

5. Music Listening Histories
User 1
Artist 1, Song 1, Timestamp
Artist 2, Song 3, Timestamp
Artist 1, Song 2, Timestamp Time
Artist 3, Song 5, Timestamp
Artist 4, Song 4, Timestamp
…
User 2
…

6. Profile Generation

7. Recommendation

8. Pattern detection in user habits
zachstern@flickr

9. Good at recording data…

10. … and allow direct access to it!

11. But Visualization is also important!
ottonassar@flickr

12. Related work

13. StreamGraph [Byron and Wattenber 2008]

14. Tangle, Strings and Knots [Baur2009]

15. LastHistory [Baur2011]

16. Fans effort to visualize data
Scrobbling Timeline Arc diagrams
LastGraph

17. Problems

18. Main issues
o More concerned about design and aesthetics
o Static visualizations and only overviews of
listening histories
o Lack of interactive browsing and filtering
o Scalability issues, regarding the number of
songs to represent

19. Main issues
o More concerned about design and aesthetics
o Static visualizations and only overviews of
listening histories
o Lack of interactive browsing and filtering
o Scalability issues, regarding the number of
songs to represent

20. Main issues
o More concerned about design and aesthetics
o Static visualizations and only overviews of
listening histories
o Lack of interactive browsing and filtering
o Scalability issues, regarding the number of
songs to represent

21. Main issues
o More concerned about design and aesthetics
o Static visualizations and only overviews of
listening histories
o Lack of interactive browsing and filtering
o Scalability issues, regarding the number of
songs to represent

22. Our solution

23. Design principles
Rationale

24. Exploration
thegiantvermin@flickr

25. Overview
[Dias and Fonseca2010]

26. Interactivity
Ultrastart

27. Details on demand
m4tik@flickr

28. Inference
mterraza@sxc

29. Analysis

30. Music Listening History Explorer
MULHER

31. MULHER

32. Main Visualization: Stacked dot

33. Main Visualization: Timeline

34. Main Visualization: Background

35. Filtering
o Genre
o Artist name Single selection
o Free Text
Multiple Selection
o Time
Combining Filters

36. Statistics
General Statistics Daily and Hourly Visualizations
With Context Without Context

37. Brushing & Highlighting

38. Implementation
Regular Web Application Backend Server
HTML, CSS, Javascript
Protovis Java backend
JSON to represent data

39. Demo

40. Evaluation

41. Two Experiments
swamibu@flickr

42. First experiment
Common listening history
Evaluate:
– UI usability
– Users satisfaction
and experience

43. Second Experiment: Case Studies
o Users exploring their own listening histories
o Listening Patterns Detection

44. Experiments Details
Test Procedure Users
Tasks Listening Logs
= ≠

45. Setup Procedure
o 45 min for each test
o User’s personal computers
mytudut@flickr

46. Test Session Procedure
1. Quick introduction
2. Application description
3. Practice Time
4. Tasks execution
5. Satisfaction survey & Informal Interview

47. Tasks
9 tasks
2 Categories
1. Explore and browse the listening history
2. Pattern detection and trends on music listening
habits

48. Exploring Tasks: examples
Indicate the most played song of the artist X
over the last three months
Describe the trend on the previous identified
song

49. Exploring Tasks: examples
Indicate the most played song of the artist X
over the last three months
Describe the trend on the previous identified
song

50. Pattern Detection Tasks: examples
Describe and try to justify the listening changes
that occurred over the last three months
Describe the listening habits on the selected
period

51. Pattern Detection Tasks: examples
Describe and try to justify the listening changes
that occurred over the last three months
Describe the listening habits on the selected
period

52. First Experiment

53. Common Log
o Listening Histories from one of the authors
o 5.000 records (11/2009 to 07/2011)
alaasafei@sxc

54. Participants
8 2
N = 10
Ages between 20-50 years
Listen to music almost every day

55. Occupations
4 CS students
2undergraduate CS students
2 SW engineers
1 journalist
1 HS student

56. Statistical Evaluation
o Effectiveness
o Error rate
o Satisfaction
iamwahid@sxc

57. First Experience
Results

58. Effectiveness
100%

59. Overall Task Success Rate
97% 3%
Successfull Unsuccessfull

60. Problem of Task 3
# Users
10
6
0
T1 T2 T3 T4 T5 T6 T7 T8 T9

61. Overall Tasks Easiness
1 8 1
0 0
Very Difficult Normal Easy Very easy
difficult

62. User Experience Satisfaction
Fantastic Rewarding Stimulant Easy Flexible
9
8
7
6
5
4
3
2
1
Horrible Frustrating Boring Hard Rigid

63. Second Experiment

64. Users’ logs
o Personal listening histories
o From 3.000 to 30.000 records
alaasafei@sxc

65. Participants: Second Experience
#N = 4
Ages between 20-40 years
All listen to music every day
Active Last.fm account

66. Occupations
2 CS Assistant Teachers
1 Senior Software Engineer
1 CS PhD Student

67. Results

68. Identical Statistical Results
lumaxart@flickr

69. Pattern Inference – Users interviews
whitebeard@sxc

70. Life Past Events
User 1
“Here I was working on a scientific paper,
because I was listening only classical music, and
I like to hear that kind of music when I'm
writing, But then I skipped listening to music,
because I had some project discussions, and no
time to listen to music”

71. Profile Inference
User 2
"Well, do you know why this part of the
visualization contains mostly recent music, even
though I just prefer to listen old music?“

72. Profile Inference
User 2
"Well, do you know why this part of the
visualization contains mostly recent music, even
though I just prefer to listen old music?“
 one of his favorite artists just released a
new album after years of absence

73. Hidden Time Habits
User 4
"I did not realized that I was listening too much
music in late night, but now that I think of this, I
usually listen to more rhythmic music at that
time to stay awake a little longer, mostly when
I'm working"

74. Listening Trends
User 3
"Looks like that through a regular day I keep
changing the genre of music I listen to. I start
with something stronger in the morning and
then end the day with more relaxing songs!"

75. Discussion

76. Timeline
o Timeline-based mechanism proved to be a
major asset:
– Main browsing and filtering technique
– Effectiveness and flexibility validated by
experimental results

77. Context Information
o Considered to be an important aspect of the
solution
– Info about most played elements
– Acts as a visual clue to start the exploration

78. “Age of Songs”
o Can effectively convey information about the
listening habits
– Different profiles by direct color inspection

79. Knowledge analysis and inference
o Performed by combining insights from the
different techniques
o Possible main based on time, absence of
music listening and context information

80. Conclusions

81. Conclusions
Novel solution for exploring and filtering
listening histories
1. Combines a timeline-based visualization with a
set of synchronized-views to perform direct
exploration
2. Introduces a new feature: the “age” of songs
3. Allows listening pattern detection, only based on
time and some textual metadata

82. Future Work
o Data mining on listening histories data
– Discover new hidden listening patterns
– New pattern examples:
• Users that always seek the tops
• Others that enjoy mostly female voices
• Listen to classic music in the morning but rhythmic
music in the afternoon
o Map the new patterns to this and other
visualizations

83. Thank you
Questions?
ricardo.dias@ist.utl.pt
web.ist.utl.pt/~ricardo.dias