Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Dominikus and I will present our work ‘The Streams of Our Lives: Visualizing Listening Histories in C
as a project I did t...
talk you will hear about the data space of music listening histories, see LastHistory, a tool for visual
mation plus conte...
An abundance of online services gives us the
chance to log almost all aspects of our lives:
Nike Plus captures your running behavior and
helps with exercising.
You can use wakoopa to track what software
you use on your computer and how much time
you waste on the internet.
And this new trend seems to have no
boundaries. You can track everything. Really…
everything.
One of the older services available is Last.fm.
While they initially tracked a person’s music
consumption to provide suita...
But once all this data has been collected,
making sense of it is hard, especially as last.fm
only provides chronologically...
Fan-made tools like Last.fm Explorer or
LastGraph but also Last.fm’s own “playground”
tools give users an entertaining but...
LastHistory, a casual infovis tool for analyzing
and reminiscing in one’s own listening history.
Several thousand people d...
First, let me talk a little bit about the data
space’s characteristics that we are talking
about here. To be able to provi...
A term that I’ve mentioned several times now
is listening history. In our understanding an
ideal listening history describ...
Each song is a pre-existing piece of music that
has attributes such as artist, title, etc.
...
Each song:
(1) pre-existing...
And second, each song has been heard by the
owner of the history at least in parts.
…
(2) has been heard at least partially
From the perspective of infovis, listening
histories are multivariate time series. We can
of course interpret it as univar...
as each section of time either contains music
or it does not and can then start to extract e.g.,
listening sessions
as eac...
But much more insight can be gained once we
go beyond this binary classification.
Listening SessionListening Session
Time
So, the first thing we can do about that is put
the songs into the musical hierarchy of albums,
artists and genres. While ...
… that can become a stand-in for any number
of different hierarchies or classifications. Ok, so
once we have mapped the sp...
Last.fm provides the so-called
“audioscrobbler”, a software that’s running in
the background and tracking all music files ...
Real listening histories:
- incomplete
- noisy
Real listening histories:
- incomplete
- noisy
One common source is that the listener is
using non-supported hardware for listening
Another that music comes from other sources
like when shopping or being at a friend’s place.
Real listening histories:
- incomplete
- noisy
The user might leave the computer while the
music keeps on playing…
Or someone else is using the computer while
the audioscrobbler is still running.
So much for the data space and its attributes.
Next, we have to think about who our users
are and what they want to do. Al...
Stroking your ego. It’s about learning about
yourself, understanding what you did, maybe
finding patterns that you were no...
We defined two larger types of tasks that
should be possible with one’s listening history:
First, the rather impersonal an...
First, the rather impersonal analysis where
users are looking for patterns within the data.
The nice thing about that is t...
To dive into the actual, underlying reasons we
have to ask the creators of the data
themselves, as they might be aware wha...
Much better memory triggers are photos or
other actively created items. Therefore, to
make sense of histories in this pers...
Ok, so with these two use cases of analysis and
personal mode in mind, let’s look at related
work from this area.
Related
...
Understanding how people listen to music is
the domain of music psychologists and music
sociologists. They have uncovered ...
The concept of the timeline is common in
Infovis, and for personal information it has
been repeatedly applied in projects ...
Two projects visualize listening histories: The
Stacked or Streamgraphs by Byron and
Wattenberg show overviews of prominen...
So, when we went about designing our own
tool, LastHistory, we wanted to create
something that allowed gaining an overview...
For our design requirements: We had non-
infovis experts as users. So we decided to keep
the interface as “non-threatening...
With these considerations we arrived at the
final version of LastHistory and I’ll try to give
you a quick rundown of it.
The largest part of the application is taken up
with a 2D timeline: all songs are represented
as small circles and mapped ...
Like at what time this user went to bed.
And here’s another example: A user who gets
up at the same time everyday and listens to
music first thing in the morning. ...
Each song’s genre is color-coded, so the user
gets an immediate overview over the variety of
songs. We’re of course restri...
We also separated the whole interaction into
two modes, ‘analysis’ and ‘personal’. The main
difference is that the persona...
Beyond static visualization, users can navigate
within the visualization by panning, triggered
by dragging with the mouse
...
One-dimensional zooming by using the
mouse’s zoom wheel or the slider in the lower
right corner allows them to focus on ce...
Hovering over a song shows a box with user-
generated keywords from last.fm, but more
prominently: connects this song with...
Preceding and succeeding repeated songs are
also highlighted, so sequences such as albums
or other predefined playlists ar...
Finally, in the upper right corner of the
application, there’s a textbox for filtering
where users can enter freeform term...
But the filter box can also be used for temporal
queries by entering dates, or periods of time,
so users can, for example,...
Having finished implementing LastHistory we
wanted to find out how well our
considerations had worked and if people
would ...
First, we gave LastHistory to four participants
in a lab study and let them play around with it
for an hour. They analyzed...
Results were insights about daily and other
rhythms: One participant stopped listening to
music during the week when she s...
But much more interesting than these case
studies was the feedback we received in the
wild: We made LastHistory available ...
In the end, about 5.000 people downloaded
the application. We included a link to an online
questionnaire that was shown af...
Our users were predominantly male (95.1% to
4.9%) and most of them (56%) had a
background in academia or technology. Age
r...
Regarding the available data, almost all users
had their own listening histories available
(97.1%), but only about one thi...
This, of course, led to a dominance of the
analysis mode over the personal mode: 75.7%
found it useful, while only 51.8% s...
Photos and calendars available they reach
about the same rating.
Analysis mode was useful:
75.7%
Personal mode was useful
...
Insights that people gained also represented
that: While some said about it: ‘I like this mode
the best, it should be the ...
The analysis mode was more open for all users,
and insights were mostly based on time and
musical taste. One user found ‘I...
But the most important finding for us was:
People actually wanted something like
LastHistory! The positive feedback just b...
And with that, I want to talk a little bit about
what we learned and where to go from here.
Beyond
music
6.
First, of course, our greatest fear as Infovis
researchers is that our work is labeled as
‘pretty pictures’. But if you ca...
One new aspect that this very personal data
type let us do was integrate a non-tangible
data type into the visualization: ...
Finally, lifelogging won’t go away and it will
probably only get worse. Especially as all
media consumption becomes digita...
The concepts that we used in LastHistory
would of course also work for these other
media and we could basically add all th...
Thank you!
> DOMINIKUS BAUR
FREDERIK SEIFFERT
MICHAEL SEDLMAIR
SEBASTIAN BORING UNIVERSITY OF MUNICH, GERMANY
Download Las...
Image credits:
Dear Diary: http://www.flickr.com/photos/hippie/2475835909/
Nike Plus: http://www.flickr.com/photos/irishey...
Upcoming SlideShare
Loading in …5
×

The Streams of Our Lives - Visualizing Listening Histories in Context

7,059 views

Published on

Slides to the talk I gave at the IEEE InfoVis conference on 29/10/10.

You can download LastHistory here:
http://www.frederikseiffert.de/lasthistory/

Here's the abstract of the paper:
The choices we take when listening to music are expressions of our personal taste and character. Storing and accessing our listening histories is trivial due to services like Last.fm, but learning from them and understanding them is not. Existing solutions operate at a very abstract level and only produce statistics. By applying techniques from information visualization to this problem, we were able to provide average people with a detailed and powerful tool for accessing their own musical past. LastHistory is an interactive visualization for displaying music listening histories, along with contextual information from personal photos and calendar entries. Its two main user tasks are (1) analysis, with an emphasis on temporal patterns and hypotheses related to musical genre and sequences, and (2) reminiscing, where listening histories and context represent part of one'€™s past. In this design study paper we give an overview of the field of music listening histories and explain their unique characteristics as a type of personal data. We then describe the design rationale, data and view transformations of LastHistory and present the results from both a lab- and a large-scale online study. We also put listening histories in contrast to other lifelogging data. The resonant and enthusiastic feedback that we received from average users shows a need for making their personal data accessible. We hope to stimulate such developments through this research.

Published in: Technology
  • Be the first to comment

The Streams of Our Lives - Visualizing Listening Histories in Context

  1. Dominikus and I will present our work ‘The Streams of Our Lives: Visualizing Listening Histories in C as a project I did together with Frederik Seiffert, Michael Sedlmair and Sebastian Boring. THE STREAMS OF OUR LIVES > DOMINIKUS BAUR FREDERIK SEIFFERT MICHAEL SEDLMAIR SEBASTIAN BORING UNIVERSITY OF MUNICH (LMU) GERMANY VISUALIZING LISTENING HISTORIES IN CONTEXT http://www.flickr.com/photos/natita2/2565850315/
  2. talk you will hear about the data space of music listening histories, see LastHistory, a tool for visual mation plus contextual data and finally, hear what we learned about aesthetics and user appreciation In this talk: • The music data space • • Large-scale study
  3. An abundance of online services gives us the chance to log almost all aspects of our lives:
  4. Nike Plus captures your running behavior and helps with exercising.
  5. You can use wakoopa to track what software you use on your computer and how much time you waste on the internet.
  6. And this new trend seems to have no boundaries. You can track everything. Really… everything.
  7. One of the older services available is Last.fm. While they initially tracked a person’s music consumption to provide suitable recommendations in their webradio, the resulting listening histories have become a use case on their own.
  8. But once all this data has been collected, making sense of it is hard, especially as last.fm only provides chronologically sorted lists. Fortunately, they also have an API that let all kinds of statistical and graphical tools appear.
  9. Fan-made tools like Last.fm Explorer or LastGraph but also Last.fm’s own “playground” tools give users an entertaining but ultimately superficial overview of their own listening habits. So, in this case study we present
  10. LastHistory, a casual infovis tool for analyzing and reminiscing in one’s own listening history. Several thousand people downloaded it and we received lots of feedback which I’ll be coming to in just a minute.
  11. First, let me talk a little bit about the data space’s characteristics that we are talking about here. To be able to provide a suitable visualization of this information we first have to be clear about the attributes of the visualized data. The space of music data 1.
  12. A term that I’ve mentioned several times now is listening history. In our understanding an ideal listening history describes all songs that a person has listened to, possibly in their lifetime. What’s important here is that Listening history (noun) A complete chronological collection of musical items …
  13. Each song is a pre-existing piece of music that has attributes such as artist, title, etc. ... Each song: (1) pre-existing piece of music ...
  14. And second, each song has been heard by the owner of the history at least in parts. … (2) has been heard at least partially
  15. From the perspective of infovis, listening histories are multivariate time series. We can of course interpret it as univariate and time- centric, Listening history (data type) Multivariate time series data
  16. as each section of time either contains music or it does not and can then start to extract e.g., listening sessions as each section of time either contains music or it does not and can then start to extract e.g., listening sessions Time
  17. But much more insight can be gained once we go beyond this binary classification. Listening SessionListening Session Time
  18. So, the first thing we can do about that is put the songs into the musical hierarchy of albums, artists and genres. While this classification is not perfect and oftentimes the topic of heated debates, at least it’s widely-known among all music listeners. One more step to overcome the downsides of a strict hierarchy is adding user-generated keywords into the mix… Listening SessionListening Session Time Genre Sub-Genre Artists Albums Songs ……
  19. … that can become a stand-in for any number of different hierarchies or classifications. Ok, so once we have mapped the space of ideal music listening into this neat format we’re good to go building a visualization on top of it, but, unfortunately, the real world is messy… Listening SessionListening Session Time Genre Sub-Genre Artists Albums Songs Tags ……
  20. Last.fm provides the so-called “audioscrobbler”, a software that’s running in the background and tracking all music files that are played on the computer. This procedure comes with its own limitations as the resulting listening histories
  21. Real listening histories: - incomplete - noisy
  22. Real listening histories: - incomplete - noisy
  23. One common source is that the listener is using non-supported hardware for listening
  24. Another that music comes from other sources like when shopping or being at a friend’s place.
  25. Real listening histories: - incomplete - noisy
  26. The user might leave the computer while the music keeps on playing…
  27. Or someone else is using the computer while the audioscrobbler is still running.
  28. So much for the data space and its attributes. Next, we have to think about who our users are and what they want to do. All lifelogging applications are first of all about User Requirements 2.
  29. Stroking your ego. It’s about learning about yourself, understanding what you did, maybe finding patterns that you were not aware of and remembering the past.
  30. We defined two larger types of tasks that should be possible with one’s listening history: First, the rather impersonal analysis where Lifelogging modes Analysis Personal
  31. First, the rather impersonal analysis where users are looking for patterns within the data. The nice thing about that is that all possible insights are hidden within the data, which means that people who haven’t “created” the listening history are able to understand what’s going on (or at least see the patterns). Still, these insights can only be on an abstract level – we see which songs are repeated are popular but don’t know why.
  32. To dive into the actual, underlying reasons we have to ask the creators of the data themselves, as they might be aware what intentions they had when picking a certain song. In this personal mode, a user’s memories form the second, complementary data source. The only thing is, we somehow have to reach the memories that the user’s have about a certain period in their lives and using songs with timestamps is not the best way to do it.
  33. Much better memory triggers are photos or other actively created items. Therefore, to make sense of histories in this personal mode, adding such contextual information can be really helpful.
  34. Ok, so with these two use cases of analysis and personal mode in mind, let’s look at related work from this area. Related Work 3.
  35. Understanding how people listen to music is the domain of music psychologists and music sociologists. They have uncovered fascinating aspects about human behavior in this regard but of course nothing about how to visualize that. .
  36. The concept of the timeline is common in Infovis, and for personal information it has been repeatedly applied in projects like LifeLines (for medical data), and PostHistory or TheMail (for email archives). LifeLines2 [1] PostHistory [2] TheMail [3]
  37. Two projects visualize listening histories: The Stacked or Streamgraphs by Byron and Wattenberg show overviews of prominent artists in listening histories. In our own former work, Pulling Strings from a Tangle, we presented two playful visualizations for this type of data, but they, too, could only paint an abstract picture of it. Strings & Tangle [5] Stacked Graphs [4]
  38. So, when we went about designing our own tool, LastHistory, we wanted to create something that allowed gaining an overview of a listening history, but also explore it in detail. Finally, we wanted to integrate contextual information as memory triggers. LastHistory4.
  39. For our design requirements: We had non- infovis experts as users. So we decided to keep the interface as “non-threatening” as possible and make more complex tasks not obligatory. Second, the tasks that users try to fulfill with the tool were nice to have, but not vital to them. So it was important to give them an immediate benefit and keep from frustrating them. Finally, we had to work with missing and imperfect data and users should be aware of this fact and not blame the visualization. Design requirements Non-expert users Non-vital tasks Missing and imperfect data
  40. With these considerations we arrived at the final version of LastHistory and I’ll try to give you a quick rundown of it.
  41. The largest part of the application is taken up with a 2D timeline: all songs are represented as small circles and mapped horizontally to the day and vertically to the time of day of their timestamps. This way, we can easily see daily rhythms,
  42. Like at what time this user went to bed.
  43. And here’s another example: A user who gets up at the same time everyday and listens to music first thing in the morning. The great thing about this mapping is that users have an immediate benefit even without interacting with the visualization.
  44. Each song’s genre is color-coded, so the user gets an immediate overview over the variety of songs. We’re of course restricted in the number of colors we can use to keep them distinguishable. classical jazz funk hip-hop electronic rock metal unknown/other
  45. We also separated the whole interaction into two modes, ‘analysis’ and ‘personal’. The main difference is that the personal mode also displays photos and calendar entries from the user’s computer to provide context and make it easier to remember what happened at what time and understanding the listening decisions. Users can simply switch between the two modes with the button in the upper left corner. Two usage modes: Analysis Personal
  46. Beyond static visualization, users can navigate within the visualization by panning, triggered by dragging with the mouse X Video of Pan
  47. One-dimensional zooming by using the mouse’s zoom wheel or the slider in the lower right corner allows them to focus on certain sections of the history. X Video of Zoom
  48. Hovering over a song shows a box with user- generated keywords from last.fm, but more prominently: connects this song with all other instances of it throughout the history. So, users can easily see when they listened to this one song.
  49. Preceding and succeeding repeated songs are also highlighted, so sequences such as albums or other predefined playlists are automatically highlighted.
  50. Finally, in the upper right corner of the application, there’s a textbox for filtering where users can enter freeform terms. It’s possible to enter song or album titles or artist names to filter all other songs.
  51. But the filter box can also be used for temporal queries by entering dates, or periods of time, so users can, for example, see all songs that they listened to in autumn before noon.
  52. Having finished implementing LastHistory we wanted to find out how well our considerations had worked and if people would actually find it useful. Reception5.
  53. First, we gave LastHistory to four participants in a lab study and let them play around with it for an hour. They analyzed their own listening history in combination with photos and calendar entries and told our experimenter what they were able to find out and what they liked or disliked about the application. 4 participants in lab study Histories at least 3.5 years Avg. 45,000 songs
  54. Results were insights about daily and other rhythms: One participant stopped listening to music during the week when she started working. A few noted how artists were rediscovered after going to a concert or them releasing a new album. And gaps and noise could also be seen and identified thanks to photos: Three participants found time periods empty where they were on vacation. • Cycles • (Re-)discovery of artists • Gaps & noise People found:
  55. But much more interesting than these case studies was the feedback we received in the wild: We made LastHistory available for download and it was quickly picked up by blogs like Life Hacker or Newsweek’s website, which gave us a lot of attention. “LastHistory … creates a spectacular visualization of your audio timeline.” “If you're a music junkie who loves the statistics that Last.fm gives you, you'll love how LastHistory extends that even further.” newsweek.com lifehacker.com
  56. In the end, about 5.000 people downloaded the application. We included a link to an online questionnaire that was shown after users had spent some time with the application and at the time of writing the paper, 243 people had filled out that form which was a valuable source of information for us. 5,000 downloads 243 filled-out questionnaires
  57. Our users were predominantly male (95.1% to 4.9%) and most of them (56%) had a background in academia or technology. Age range was 16 to 67 years, with an average of 27.2 years. Gender: 95.1% male, 4.9% female Background: 56% academia/technology Avg. age: 27.2 years
  58. Regarding the available data, almost all users had their own listening histories available (97.1%), but only about one third (37%) photos and 18.5% calendars. Listening history: 97.1% Photos: 37% Calendars: 18.5% Data available:
  59. This, of course, led to a dominance of the analysis mode over the personal mode: 75.7% found it useful, while only 51.8% said that about the personal mode. But when you only ask people who had Analysis mode was useful: 75.7% Personal mode was useful: 51.8%
  60. Photos and calendars available they reach about the same rating. Analysis mode was useful: 75.7% Personal mode was useful (people with photos + calendars): 75.8%
  61. Insights that people gained also represented that: While some said about it: ‘I like this mode the best, it should be the default mode’, or ‘clicking on a photo gallery and listening to what I was listening to at the time was very powerful’ others complained about the missing data. “I like this mode the best, it should be the default mode!” “Clicking on a photo gallery and listening to what I was listening to at the time was very powerful.” About the Personal Mode:
  62. The analysis mode was more open for all users, and insights were mostly based on time and musical taste. One user found ‘I rarely listen to music between the hours of 9-11 a.m., even on weekends’ and another one noted his commuting pattern. A third one said she found ‘those ruts where you get stuck in listening to one particular song.’ Finally, users were also aware of the imperfect data. “I rarely listen to music between the hours of 9- 11 a.m., even on weekends.” “I noted the … commuting pattern.” About the Analysis Mode: “Those ruts where you get stuck in listening to one particular song.” “I listened to music for 4 straight days!”
  63. But the most important finding for us was: People actually wanted something like LastHistory! The positive feedback just blew us away and we were very happy that people liked it. And even though the application wasn’t completely trivial, around 75% of our users found it easy to learn. It worked!
  64. And with that, I want to talk a little bit about what we learned and where to go from here. Beyond music 6.
  65. First, of course, our greatest fear as Infovis researchers is that our work is labeled as ‘pretty pictures’. But if you can build a pretty visualization without sacrificing functionality or making reading it more difficult why not do it? Users always prefer an aesthetically pleasing application to a dull one. And yes, even expert users. Thinking about users … If you have to do it anyway, why not make it pretty?
  66. One new aspect that this very personal data type let us do was integrate a non-tangible data type into the visualization: the user’s memory. Depending on the use case, this can be a powerful source for understanding and learning. Providing suitable memory triggers is essential here.
  67. Finally, lifelogging won’t go away and it will probably only get worse. Especially as all media consumption becomes digital (even books!) tracking every little aspect of reading, watching and listening will become commonplace. The age of lifelogging is upon us.
  68. The concepts that we used in LastHistory would of course also work for these other media and we could basically add all the data streams we could think of to the timeline. But I think what’s more in this regard is that people (1) actually want something like that and (2) that information visualization can help them for understanding their behavior and reminiscing. That means that their data is no longer only useful to companies and governments: These visualizations can give the data back to their creators. And we as visualization researchers and practitioners are the people to build those tools.
  69. Thank you! > DOMINIKUS BAUR FREDERIK SEIFFERT MICHAEL SEDLMAIR SEBASTIAN BORING UNIVERSITY OF MUNICH, GERMANY Download LastHistory (open source) at: http://www.frederikseiffert.de/lasthistory dominikus.baur@ifi.lmu.de twitter: @dominikus
  70. Image credits: Dear Diary: http://www.flickr.com/photos/hippie/2475835909/ Nike Plus: http://www.flickr.com/photos/irisheyes/327069125/ Last.fm Logo: http://www.flickr.com/photos/dekuwa/3383948292/sizes/o/in/photostream/ Audio cassette: http://www.flickr.com/photos/steve-maw/4234388137/ Notebook scribbles: http://www.flickr.com/photos/cherryboppy/4812211497/ Messy Pudding Kid: http://www.flickr.com/photos/ripizzo/2310929170/ Empty computer: http://www.flickr.com/photos/nycgraeme/2287826242/sizes/l/ Grammophone: http://www.flickr.com/photos/jenik/2861494379/sizes/l/ Shopping Cart: http://www.flickr.com/photos/spijker/3273982099/sizes/l/ Computer cat: http://www.flickr.com/photos/kevinsteele/1507196484/sizes/l/ Photo Girl: http://www.flickr.com/photos/blythe_d/1451273161/sizes/o/ Scientists: http://www.flickr.com/photos/marsdd/2986989396/sizes/o/ Reading girl: http://www.flickr.com/photos/12392252@N03/2482835894/ Old photos: http://www.flickr.com/photos/soaringhorse/419617753/ Gallery reception: http://www.flickr.com/photos/bricolage108/319671818/ Light at the end of the tunnel: http://www.flickr.com/photos/mercedesdayanara/881056652/ Papers: [1] Taowei Wang et al.: Aligning Temporal Data by Sentinel Events: Discovering Patterns in Electronic Health Records, CHI 2008 [2] Viegas et al.: Digital Artifacts for remembering and storytelling: Posthistory and social network fragments, HICSS-37, 2004 [3] Viegas et al.: Visualizing email content: portraying relationships from conversational histories, CHI 2006 [4] Byron et al.: Stacked graphs–geometry & aesthetics, InfoVis 2008 [5] Baur et al.: Pulling Strings from a Tangle: Visualizing a Personal Music Listening History, IUI 2009

×