Slides to the talk I gave at the IEEE InfoVis conference on 29/10/10.
You can download LastHistory here:
http://www.frederikseiffert.de/lasthistory/
Here's the abstract of the paper:
The choices we take when listening to music are expressions of our personal taste and character. Storing and accessing our listening histories is trivial due to services like Last.fm, but learning from them and understanding them is not. Existing solutions operate at a very abstract level and only produce statistics. By applying techniques from information visualization to this problem, we were able to provide average people with a detailed and powerful tool for accessing their own musical past. LastHistory is an interactive visualization for displaying music listening histories, along with contextual information from personal photos and calendar entries. Its two main user tasks are (1) analysis, with an emphasis on temporal patterns and hypotheses related to musical genre and sequences, and (2) reminiscing, where listening histories and context represent part of one's past. In this design study paper we give an overview of the field of music listening histories and explain their unique characteristics as a type of personal data. We then describe the design rationale, data and view transformations of LastHistory and present the results from both a lab- and a large-scale online study. We also put listening histories in contrast to other lifelogging data. The resonant and enthusiastic feedback that we received from average users shows a need for making their personal data accessible. We hope to stimulate such developments through this research.
slides presented at a three-hour local AI music course in Taiwan in Oct 2021; part 1: a brief introduction to music information retrieval (+analysis, +generation)
Yi-Hsuan Yang is an Associate Research Fellow with Academia Sinica. He received his Ph.D. degree in Communication Engineering from National Taiwan University in 2010, and became an Assistant Research Fellow in Academia Sinica in 2011. He is also an Adjunct Associate Professor with the National Tsing Hua University, Taiwan. His research interests include music information retrieval, machine learning and affective computing. Dr. Yang was a recipient of the 2011 IEEE Signal Processing Society (SPS) Young Author Best Paper Award, the 2012 ACM Multimedia Grand Challenge First Prize, and the 2014 Ta-You Wu Memorial Research Award of the Ministry of Science and Technology, Taiwan. He is an author of the book Music Emotion Recognition (CRC Press 2011) and a tutorial speaker on music affect recognition in the International Society for Music Information Retrieval Conference (ISMIR 2012). In 2014, he served as a Technical Program Co-chair of ISMIR, and a Guest Editor of the IEEE Transactions on Affective Computing and the ACM Transactions on Intelligent Systems and Technology.
slides presented at a three-hour local AI music course in Taiwan in Oct 2021; part 1: a brief introduction to music information retrieval (+analysis, +generation)
Yi-Hsuan Yang is an Associate Research Fellow with Academia Sinica. He received his Ph.D. degree in Communication Engineering from National Taiwan University in 2010, and became an Assistant Research Fellow in Academia Sinica in 2011. He is also an Adjunct Associate Professor with the National Tsing Hua University, Taiwan. His research interests include music information retrieval, machine learning and affective computing. Dr. Yang was a recipient of the 2011 IEEE Signal Processing Society (SPS) Young Author Best Paper Award, the 2012 ACM Multimedia Grand Challenge First Prize, and the 2014 Ta-You Wu Memorial Research Award of the Ministry of Science and Technology, Taiwan. He is an author of the book Music Emotion Recognition (CRC Press 2011) and a tutorial speaker on music affect recognition in the International Society for Music Information Retrieval Conference (ISMIR 2012). In 2014, he served as a Technical Program Co-chair of ISMIR, and a Guest Editor of the IEEE Transactions on Affective Computing and the ACM Transactions on Intelligent Systems and Technology.
Some of my slides from the AES 122 Vienna Convention, workshop on "Music and the Web" (May 6th, 2007). This presentation was dealing with the Music Ontology, and some of the Linked Data concepts.
Audio Signal Identification and Search Approach for Minimizing the Search Tim...aciijournal
Audio or music fingerprints can be utilize to implement an economical music identification system on a
million-song library, however the system needs great deal of memory to carry the fingerprints and indexes.
Therefore, for a large-scale audio library, memory
imposes a restriction on the speed of music
identifications. So, we propose an efficient music
identification system which used a kind of space-saving
audio fingerprints. For saving space, original finger representations are sub-sample and only one quarters
of the original data is reserved. In this approach,
memory demand is far reduced and therefore the search
speed is criticalincreasing whereas the lustiness and dependability are well preserved. Mapping
audio information to time and frequency domain for
the classification, retrieval or identification tasks
presents four principal challenges. The dimension o
f the input should be considerably reduced;
the ensuing options should be strong to possible distortions of the input; the feature should be informative
for the task at hand simple. We propose distortion
free system which fulfils all four of these requirements.
Extensive study has been done to compare our system
with the already existing ones, and the results sh
ow
that our system requires less memory, provides fast
results and achieves comparable accuracy for a large-
scale database.
Audio Signal Identification and Search Approach for Minimizing the Search Tim...aciijournal
Audio or music fingerprints can be utilize to implement an economical music identification system on a
million-song library, however the system needs great deal of memory to carry the fingerprints and indexes.
Therefore, for a large-scale audio library, memory imposes a restriction on the speed of music
identifications. So, we propose an efficient music identification system which used a kind of space-saving
audio fingerprints. For saving space, original finger representations are sub-sample and only one quarters
of the original data is reserved. In this approach, memory demand is far reduced and therefore the search
speed is criticalincreasing whereas the lustiness and dependability ar well preserved. Mapping
audio information to time and frequency domain for the classification, retrieval or identification tasks
presents four principal challenges. The dimension of the input should be considerably reduced;
the ensuing options should be strong to possible distortions of the input; the feature should be informative
for the task at hand simple. We propose distortion free system which fulfils all four of these requirements.
Extensive study has been done to compare our system with the already existing ones, and the results show
that our system requires less memory, provides fast results and achieves comparable accuracy for a largescale database.
KEYWORDS
AUDIO SIGNAL IDENTIFICATION AND SEARCH APPROACH FOR MINIMIZING THE SEARCH TIM...aciijournal
Audio or music fingerprints can be utilize to implement an economical music identification system on a
million-song library, however the system needs great deal of memory to carry the fingerprints and indexes.
Therefore, for a large-scale audio library, memory imposes a restriction on the speed of music
identifications. So, we propose an efficient music identification system which used a kind of space-saving
audio fingerprints. For saving space, original finger representations are sub-sample and only one quarters
of the original data is reserved. In this approach, memory demand is far reduced and therefore the search
speed is criticalincreasing whereas the lustiness and dependability ar well preserved. Mapping
audio information to time and frequency domain for the classification, retrieval or identification tasks
presents four principal challenges. The dimension of the input should be considerably reduced;
the ensuing options should be strong to possible distortions of the input; the feature should be informative
for the task at hand simple. We propose distortion free system which fulfils all four of these requirements.
Extensive study has been done to compare our system with the already existing ones, and the results show
that our system requires less memory, provides fast results and achieves comparable accuracy for a largescale database.
Audio Signal Identification and Search Approach for Minimizing the Search Tim...aciijournal
Audio or music fingerprints can be utilize to implement an economical music identification system on a
million-song library, however the system needs great deal of memory to carry the fingerprints and indexes.
Therefore, for a large-scale audio library, memory imposes a restriction on the speed of music
identifications. So, we propose an efficient music identification system which used a kind of space-saving
audio fingerprints. For saving space, original finger representations are sub-sample and only one quarters
of the original data is reserved. In this approach, memory demand is far reduced and therefore the search
speed is criticalincreasing whereas the lustiness and dependability ar well preserved. Mapping
audio information to time and frequency domain for the classification, retrieval or identification tasks
presents four principal challenges. The dimension of the input should be considerably reduced;
the ensuing options should be strong to possible distortions of the input; the feature should be informative
for the task at hand simple. We propose distortion free system which fulfils all four of these requirements.
Extensive study has been done to compare our system with the already existing ones, and the results show
that our system requires less memory, provides fast results and achieves comparable accuracy for a largescale database.
Applications of AI and NLP to advance Music Recommendations on Voice AssistantsAI Publications
This paper builds on the popular use case of music requests on voice assistants like Siri, Google Assistant, Alexa, and others and explores the different AI and NLP techniques. The paper particularly focuses on how each of these techniques can be applied in the context of musical recommendations and experiences on voice assistants. It enumerates specific problems in the space of music recommendations and illustrates how specific techniques like multi-armed bandits can be applied.
UPDATED VERSION (2011): http://www.slideshare.net/plamere/music-recommendation-and-discovery
As the world of online music grows, music 2.0 recommendation systems become an increasingly important way for music listeners to discover new music.
Commercial recommenders such as Last.fm and Pandora have enjoyed commercial and critical success. But how well do these systems really work? How good are the recommendations? How far into The Long Tail do these recommenders reach?
In this tutorial we look at the current stateof theart in music recommendation. We examine current commercial and research systems, focusing on the advantages and the disadvantages of the various recommendation strategies. We look at some of the challenges in building music recommenders and we explore some of the ways that MIR techniques can be used to improve future recommenders.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
Some of my slides from the AES 122 Vienna Convention, workshop on "Music and the Web" (May 6th, 2007). This presentation was dealing with the Music Ontology, and some of the Linked Data concepts.
Audio Signal Identification and Search Approach for Minimizing the Search Tim...aciijournal
Audio or music fingerprints can be utilize to implement an economical music identification system on a
million-song library, however the system needs great deal of memory to carry the fingerprints and indexes.
Therefore, for a large-scale audio library, memory
imposes a restriction on the speed of music
identifications. So, we propose an efficient music
identification system which used a kind of space-saving
audio fingerprints. For saving space, original finger representations are sub-sample and only one quarters
of the original data is reserved. In this approach,
memory demand is far reduced and therefore the search
speed is criticalincreasing whereas the lustiness and dependability are well preserved. Mapping
audio information to time and frequency domain for
the classification, retrieval or identification tasks
presents four principal challenges. The dimension o
f the input should be considerably reduced;
the ensuing options should be strong to possible distortions of the input; the feature should be informative
for the task at hand simple. We propose distortion
free system which fulfils all four of these requirements.
Extensive study has been done to compare our system
with the already existing ones, and the results sh
ow
that our system requires less memory, provides fast
results and achieves comparable accuracy for a large-
scale database.
Audio Signal Identification and Search Approach for Minimizing the Search Tim...aciijournal
Audio or music fingerprints can be utilize to implement an economical music identification system on a
million-song library, however the system needs great deal of memory to carry the fingerprints and indexes.
Therefore, for a large-scale audio library, memory imposes a restriction on the speed of music
identifications. So, we propose an efficient music identification system which used a kind of space-saving
audio fingerprints. For saving space, original finger representations are sub-sample and only one quarters
of the original data is reserved. In this approach, memory demand is far reduced and therefore the search
speed is criticalincreasing whereas the lustiness and dependability ar well preserved. Mapping
audio information to time and frequency domain for the classification, retrieval or identification tasks
presents four principal challenges. The dimension of the input should be considerably reduced;
the ensuing options should be strong to possible distortions of the input; the feature should be informative
for the task at hand simple. We propose distortion free system which fulfils all four of these requirements.
Extensive study has been done to compare our system with the already existing ones, and the results show
that our system requires less memory, provides fast results and achieves comparable accuracy for a largescale database.
KEYWORDS
AUDIO SIGNAL IDENTIFICATION AND SEARCH APPROACH FOR MINIMIZING THE SEARCH TIM...aciijournal
Audio or music fingerprints can be utilize to implement an economical music identification system on a
million-song library, however the system needs great deal of memory to carry the fingerprints and indexes.
Therefore, for a large-scale audio library, memory imposes a restriction on the speed of music
identifications. So, we propose an efficient music identification system which used a kind of space-saving
audio fingerprints. For saving space, original finger representations are sub-sample and only one quarters
of the original data is reserved. In this approach, memory demand is far reduced and therefore the search
speed is criticalincreasing whereas the lustiness and dependability ar well preserved. Mapping
audio information to time and frequency domain for the classification, retrieval or identification tasks
presents four principal challenges. The dimension of the input should be considerably reduced;
the ensuing options should be strong to possible distortions of the input; the feature should be informative
for the task at hand simple. We propose distortion free system which fulfils all four of these requirements.
Extensive study has been done to compare our system with the already existing ones, and the results show
that our system requires less memory, provides fast results and achieves comparable accuracy for a largescale database.
Audio Signal Identification and Search Approach for Minimizing the Search Tim...aciijournal
Audio or music fingerprints can be utilize to implement an economical music identification system on a
million-song library, however the system needs great deal of memory to carry the fingerprints and indexes.
Therefore, for a large-scale audio library, memory imposes a restriction on the speed of music
identifications. So, we propose an efficient music identification system which used a kind of space-saving
audio fingerprints. For saving space, original finger representations are sub-sample and only one quarters
of the original data is reserved. In this approach, memory demand is far reduced and therefore the search
speed is criticalincreasing whereas the lustiness and dependability ar well preserved. Mapping
audio information to time and frequency domain for the classification, retrieval or identification tasks
presents four principal challenges. The dimension of the input should be considerably reduced;
the ensuing options should be strong to possible distortions of the input; the feature should be informative
for the task at hand simple. We propose distortion free system which fulfils all four of these requirements.
Extensive study has been done to compare our system with the already existing ones, and the results show
that our system requires less memory, provides fast results and achieves comparable accuracy for a largescale database.
Applications of AI and NLP to advance Music Recommendations on Voice AssistantsAI Publications
This paper builds on the popular use case of music requests on voice assistants like Siri, Google Assistant, Alexa, and others and explores the different AI and NLP techniques. The paper particularly focuses on how each of these techniques can be applied in the context of musical recommendations and experiences on voice assistants. It enumerates specific problems in the space of music recommendations and illustrates how specific techniques like multi-armed bandits can be applied.
UPDATED VERSION (2011): http://www.slideshare.net/plamere/music-recommendation-and-discovery
As the world of online music grows, music 2.0 recommendation systems become an increasingly important way for music listeners to discover new music.
Commercial recommenders such as Last.fm and Pandora have enjoyed commercial and critical success. But how well do these systems really work? How good are the recommendations? How far into The Long Tail do these recommenders reach?
In this tutorial we look at the current stateof theart in music recommendation. We examine current commercial and research systems, focusing on the advantages and the disadvantages of the various recommendation strategies. We look at some of the challenges in building music recommenders and we explore some of the ways that MIR techniques can be used to improve future recommenders.
Similar to The Streams of Our Lives - Visualizing Listening Histories in Context (20)
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™UiPathCommunity
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
The Streams of Our Lives - Visualizing Listening Histories in Context
1. Dominikus and I will present our work ‘The Streams of Our Lives: Visualizing Listening Histories in C
as a project I did together with Frederik Seiffert, Michael Sedlmair and Sebastian Boring.
THE STREAMS
OF OUR LIVES
> DOMINIKUS BAUR
FREDERIK SEIFFERT
MICHAEL SEDLMAIR
SEBASTIAN BORING
UNIVERSITY OF
MUNICH (LMU)
GERMANY
VISUALIZING LISTENING
HISTORIES IN CONTEXT
http://www.flickr.com/photos/natita2/2565850315/
2. talk you will hear about the data space of music listening histories, see LastHistory, a tool for visual
mation plus contextual data and finally, hear what we learned about aesthetics and user appreciation
In this talk:
• The music data space
•
• Large-scale study
3. An abundance of online services gives us the
chance to log almost all aspects of our lives:
5. You can use wakoopa to track what software
you use on your computer and how much time
you waste on the internet.
6. And this new trend seems to have no
boundaries. You can track everything. Really…
everything.
7. One of the older services available is Last.fm.
While they initially tracked a person’s music
consumption to provide suitable
recommendations in their webradio, the
resulting listening histories have become a use
case on their own.
8. But once all this data has been collected,
making sense of it is hard, especially as last.fm
only provides chronologically sorted lists.
Fortunately, they also have an API that let all
kinds of statistical and graphical tools appear.
9. Fan-made tools like Last.fm Explorer or
LastGraph but also Last.fm’s own “playground”
tools give users an entertaining but ultimately
superficial overview of their own listening
habits. So, in this case study we present
10. LastHistory, a casual infovis tool for analyzing
and reminiscing in one’s own listening history.
Several thousand people downloaded it and
we received lots of feedback which I’ll be
coming to in just a minute.
11. First, let me talk a little bit about the data
space’s characteristics that we are talking
about here. To be able to provide a suitable
visualization of this information we first have
to be clear about the attributes of the
visualized data.
The space
of music data
1.
12. A term that I’ve mentioned several times now
is listening history. In our understanding an
ideal listening history describes all songs that a
person has listened to, possibly in their
lifetime. What’s important here is that
Listening history (noun)
A complete chronological
collection of musical items
…
13. Each song is a pre-existing piece of music that
has attributes such as artist, title, etc.
...
Each song:
(1) pre-existing piece of music
...
14. And second, each song has been heard by the
owner of the history at least in parts.
…
(2) has been heard at least partially
15. From the perspective of infovis, listening
histories are multivariate time series. We can
of course interpret it as univariate and time-
centric,
Listening history (data type)
Multivariate time series data
16. as each section of time either contains music
or it does not and can then start to extract e.g.,
listening sessions
as each section of time either contains music
or it does not and can then start to extract e.g.,
listening sessions
Time
17. But much more insight can be gained once we
go beyond this binary classification.
Listening SessionListening Session
Time
18. So, the first thing we can do about that is put
the songs into the musical hierarchy of albums,
artists and genres. While this classification is
not perfect and oftentimes the topic of heated
debates, at least it’s widely-known among all
music listeners. One more step to overcome
the downsides of a strict hierarchy is adding
user-generated keywords into the mix…
Listening SessionListening Session
Time
Genre
Sub-Genre
Artists
Albums
Songs
……
19. … that can become a stand-in for any number
of different hierarchies or classifications. Ok, so
once we have mapped the space of ideal music
listening into this neat format we’re good to go
building a visualization on top of it, but,
unfortunately, the real world is messy…
Listening SessionListening Session
Time
Genre
Sub-Genre
Artists
Albums
Songs
Tags
……
20. Last.fm provides the so-called
“audioscrobbler”, a software that’s running in
the background and tracking all music files that
are played on the computer. This procedure
comes with its own limitations as the resulting
listening histories
26. The user might leave the computer while the
music keeps on playing…
27. Or someone else is using the computer while
the audioscrobbler is still running.
28. So much for the data space and its attributes.
Next, we have to think about who our users
are and what they want to do. All lifelogging
applications are first of all about
User
Requirements
2.
29. Stroking your ego. It’s about learning about
yourself, understanding what you did, maybe
finding patterns that you were not aware of
and remembering the past.
30. We defined two larger types of tasks that
should be possible with one’s listening history:
First, the rather impersonal analysis where
Lifelogging modes
Analysis Personal
31. First, the rather impersonal analysis where
users are looking for patterns within the data.
The nice thing about that is that all possible
insights are hidden within the data, which
means that people who haven’t “created” the
listening history are able to understand what’s
going on (or at least see the patterns). Still,
these insights can only be on an abstract level
– we see which songs are repeated are popular
but don’t know why.
32. To dive into the actual, underlying reasons we
have to ask the creators of the data
themselves, as they might be aware what
intentions they had when picking a certain
song. In this personal mode, a user’s memories
form the second, complementary data source.
The only thing is, we somehow have to reach
the memories that the user’s have about a
certain period in their lives and using songs
with timestamps is not the best way to do it.
33. Much better memory triggers are photos or
other actively created items. Therefore, to
make sense of histories in this personal mode,
adding such contextual information can be
really helpful.
34. Ok, so with these two use cases of analysis and
personal mode in mind, let’s look at related
work from this area.
Related
Work
3.
35. Understanding how people listen to music is
the domain of music psychologists and music
sociologists. They have uncovered fascinating
aspects about human behavior in this regard
but of course nothing about how to visualize
that.
.
36. The concept of the timeline is common in
Infovis, and for personal information it has
been repeatedly applied in projects like
LifeLines (for medical data), and PostHistory or
TheMail (for email archives).
LifeLines2 [1] PostHistory [2]
TheMail [3]
37. Two projects visualize listening histories: The
Stacked or Streamgraphs by Byron and
Wattenberg show overviews of prominent
artists in listening histories. In our own former
work, Pulling Strings from a Tangle, we
presented two playful visualizations for this
type of data, but they, too, could only paint an
abstract picture of it.
Strings & Tangle [5]
Stacked Graphs [4]
38. So, when we went about designing our own
tool, LastHistory, we wanted to create
something that allowed gaining an overview of
a listening history, but also explore it in detail.
Finally, we wanted to integrate contextual
information as memory triggers.
LastHistory4.
39. For our design requirements: We had non-
infovis experts as users. So we decided to keep
the interface as “non-threatening” as possible
and make more complex tasks not obligatory.
Second, the tasks that users try to fulfill with
the tool were nice to have, but not vital to
them. So it was important to give them an
immediate benefit and keep from frustrating
them. Finally, we had to work with missing and
imperfect data and users should be aware of
this fact and not blame the visualization.
Design requirements
Non-expert users
Non-vital tasks
Missing and imperfect data
40. With these considerations we arrived at the
final version of LastHistory and I’ll try to give
you a quick rundown of it.
41. The largest part of the application is taken up
with a 2D timeline: all songs are represented
as small circles and mapped horizontally to the
day and vertically to the time of day of their
timestamps. This way, we can easily see daily
rhythms,
43. And here’s another example: A user who gets
up at the same time everyday and listens to
music first thing in the morning. The great
thing about this mapping is that users have an
immediate benefit even without interacting
with the visualization.
44. Each song’s genre is color-coded, so the user
gets an immediate overview over the variety of
songs. We’re of course restricted in the
number of colors we can use to keep them
distinguishable.
classical
jazz
funk
hip-hop
electronic
rock
metal
unknown/other
45. We also separated the whole interaction into
two modes, ‘analysis’ and ‘personal’. The main
difference is that the personal mode also
displays photos and calendar entries from the
user’s computer to provide context and make
it easier to remember what happened at what
time and understanding the listening decisions.
Users can simply switch between the two
modes with the button in the upper left
corner.
Two usage modes:
Analysis Personal
46. Beyond static visualization, users can navigate
within the visualization by panning, triggered
by dragging with the mouse
X Video of Pan
47. One-dimensional zooming by using the
mouse’s zoom wheel or the slider in the lower
right corner allows them to focus on certain
sections of the history.
X Video of Zoom
48. Hovering over a song shows a box with user-
generated keywords from last.fm, but more
prominently: connects this song with all other
instances of it throughout the history. So, users
can easily see when they listened to this one
song.
49. Preceding and succeeding repeated songs are
also highlighted, so sequences such as albums
or other predefined playlists are automatically
highlighted.
50. Finally, in the upper right corner of the
application, there’s a textbox for filtering
where users can enter freeform terms. It’s
possible to enter song or album titles or artist
names to filter all other songs.
51. But the filter box can also be used for temporal
queries by entering dates, or periods of time,
so users can, for example, see all songs that
they listened to in autumn before noon.
52. Having finished implementing LastHistory we
wanted to find out how well our
considerations had worked and if people
would actually find it useful.
Reception5.
53. First, we gave LastHistory to four participants
in a lab study and let them play around with it
for an hour. They analyzed their own listening
history in combination with photos and
calendar entries and told our experimenter
what they were able to find out and what they
liked or disliked about the application.
4 participants in lab study
Histories at least 3.5 years
Avg. 45,000 songs
54. Results were insights about daily and other
rhythms: One participant stopped listening to
music during the week when she started
working. A few noted how artists were
rediscovered after going to a concert or them
releasing a new album. And gaps and noise
could also be seen and identified thanks to
photos: Three participants found time periods
empty where they were on vacation.
• Cycles
• (Re-)discovery of artists
• Gaps & noise
People found:
55. But much more interesting than these case
studies was the feedback we received in the
wild: We made LastHistory available for
download and it was quickly picked up by
blogs like Life Hacker or Newsweek’s website,
which gave us a lot of attention.
“LastHistory … creates a spectacular
visualization of your audio timeline.”
“If you're a music junkie who loves the
statistics that Last.fm gives you, you'll love how
LastHistory extends that even further.”
newsweek.com
lifehacker.com
56. In the end, about 5.000 people downloaded
the application. We included a link to an online
questionnaire that was shown after users had
spent some time with the application and at
the time of writing the paper, 243 people had
filled out that form which was a valuable
source of information for us.
5,000 downloads
243 filled-out questionnaires
57. Our users were predominantly male (95.1% to
4.9%) and most of them (56%) had a
background in academia or technology. Age
range was 16 to 67 years, with an average of
27.2 years.
Gender: 95.1% male, 4.9% female
Background: 56% academia/technology
Avg. age: 27.2 years
58. Regarding the available data, almost all users
had their own listening histories available
(97.1%), but only about one third (37%) photos
and 18.5% calendars.
Listening history: 97.1%
Photos: 37%
Calendars: 18.5%
Data available:
59. This, of course, led to a dominance of the
analysis mode over the personal mode: 75.7%
found it useful, while only 51.8% said that
about the personal mode. But when you only
ask people who had
Analysis mode was useful:
75.7%
Personal mode was useful:
51.8%
60. Photos and calendars available they reach
about the same rating.
Analysis mode was useful:
75.7%
Personal mode was useful
(people with photos + calendars):
75.8%
61. Insights that people gained also represented
that: While some said about it: ‘I like this mode
the best, it should be the default mode’, or
‘clicking on a photo gallery and listening to
what I was listening to at the time was very
powerful’ others complained about the
missing data.
“I like this mode the best, it should be the
default mode!”
“Clicking on a photo gallery and listening to
what I was listening to at the time was very
powerful.”
About the Personal Mode:
62. The analysis mode was more open for all users,
and insights were mostly based on time and
musical taste. One user found ‘I rarely listen to
music between the hours of 9-11 a.m., even on
weekends’ and another one noted his
commuting pattern. A third one said she found
‘those ruts where you get stuck in listening to
one particular song.’ Finally, users were also
aware of the imperfect data.
“I rarely listen to music between the hours of 9-
11 a.m., even on weekends.”
“I noted the … commuting pattern.”
About the Analysis Mode:
“Those ruts where you get stuck in listening to
one particular song.”
“I listened to music for 4 straight days!”
63. But the most important finding for us was:
People actually wanted something like
LastHistory! The positive feedback just blew us
away and we were very happy that people
liked it. And even though the application
wasn’t completely trivial, around 75% of our
users found it easy to learn.
It worked!
64. And with that, I want to talk a little bit about
what we learned and where to go from here.
Beyond
music
6.
65. First, of course, our greatest fear as Infovis
researchers is that our work is labeled as
‘pretty pictures’. But if you can build a pretty
visualization without sacrificing functionality or
making reading it more difficult why not do it?
Users always prefer an aesthetically pleasing
application to a dull one. And yes, even expert
users. Thinking about users …
If you have to do it anyway,
why not make it pretty?
66. One new aspect that this very personal data
type let us do was integrate a non-tangible
data type into the visualization: the user’s
memory. Depending on the use case, this can
be a powerful source for understanding and
learning. Providing suitable memory triggers is
essential here.
67. Finally, lifelogging won’t go away and it will
probably only get worse. Especially as all
media consumption becomes digital (even
books!) tracking every little aspect of reading,
watching and listening will become
commonplace.
The age of lifelogging is upon us.
68. The concepts that we used in LastHistory
would of course also work for these other
media and we could basically add all the data
streams we could think of to the timeline. But I
think what’s more in this regard is that people
(1) actually want something like that and (2)
that information visualization can help them
for understanding their behavior and
reminiscing. That means that their data is no
longer only useful to companies and
governments: These visualizations can give the
data back to their creators. And we as
visualization researchers and practitioners are
the people to build those tools.
69. Thank you!
> DOMINIKUS BAUR
FREDERIK SEIFFERT
MICHAEL SEDLMAIR
SEBASTIAN BORING UNIVERSITY OF MUNICH, GERMANY
Download LastHistory (open source) at:
http://www.frederikseiffert.de/lasthistory
dominikus.baur@ifi.lmu.de
twitter: @dominikus
70. Image credits:
Dear Diary: http://www.flickr.com/photos/hippie/2475835909/
Nike Plus: http://www.flickr.com/photos/irisheyes/327069125/
Last.fm Logo: http://www.flickr.com/photos/dekuwa/3383948292/sizes/o/in/photostream/
Audio cassette: http://www.flickr.com/photos/steve-maw/4234388137/
Notebook scribbles: http://www.flickr.com/photos/cherryboppy/4812211497/
Messy Pudding Kid: http://www.flickr.com/photos/ripizzo/2310929170/
Empty computer: http://www.flickr.com/photos/nycgraeme/2287826242/sizes/l/
Grammophone: http://www.flickr.com/photos/jenik/2861494379/sizes/l/
Shopping Cart: http://www.flickr.com/photos/spijker/3273982099/sizes/l/
Computer cat: http://www.flickr.com/photos/kevinsteele/1507196484/sizes/l/
Photo Girl: http://www.flickr.com/photos/blythe_d/1451273161/sizes/o/
Scientists: http://www.flickr.com/photos/marsdd/2986989396/sizes/o/
Reading girl: http://www.flickr.com/photos/12392252@N03/2482835894/
Old photos: http://www.flickr.com/photos/soaringhorse/419617753/
Gallery reception: http://www.flickr.com/photos/bricolage108/319671818/
Light at the end of the tunnel: http://www.flickr.com/photos/mercedesdayanara/881056652/
Papers:
[1] Taowei Wang et al.: Aligning Temporal Data by Sentinel Events: Discovering Patterns in Electronic
Health Records, CHI 2008
[2] Viegas et al.: Digital Artifacts for remembering and storytelling: Posthistory and social network
fragments, HICSS-37, 2004
[3] Viegas et al.: Visualizing email content: portraying relationships from conversational histories,
CHI 2006
[4] Byron et al.: Stacked graphs–geometry & aesthetics, InfoVis 2008
[5] Baur et al.: Pulling Strings from a Tangle: Visualizing a Personal Music Listening History, IUI 2009
Editor's Notes
Hi, I’m Dominikus and I will present our work ‘The Streams of Our Lives: Visualizing Listening Histories in Context’. This was a project I did together with Frederik Seiffert, Michael Sedlmair and Sebastian Boring.
In this talk you will hear about the data space of music listening histories, see LastHistory, a tool for visualizing this information plus contextual data and finally, hear what we learned about aesthetics and user appreciation and what you can take away from our large-scale online study.
An abundance of online services gives us the chance to log almost all aspects of our lives:
Nike Plus captures your running behavior and helps with exercising.
You can use wakoopa to track what software you use on your computer and how much time you waste on the internet.
And this new trend seems to have no boundaries. You can track everything. Really… everything.
One of the older services available is Last.fm. While they initially tracked a person’s music consumption to provide suitable recommendations in their webradio, the resulting listening histories have become a use case on their own.
But once all this data has been collected, making sense of it is hard, especially as last.fm only provides chronologically sorted lists. Fortunately, they also have an API that let all kinds of statistical and graphical tools appear.
Fan-made tools like Last.fm Explorer or LastGraph but also Last.fm’s own “playground” tools give users an entertaining but ultimately superficial overview of their own listening habits. So, in this case study we present
LastHistory, a casual infovis tool for analyzing and reminiscing in one’s own listening history. Several thousand people downloaded it and we received lots of feedback which I’ll be coming to in just a minute.
First, let me talk a little bit about the data space’s characteristics that we are talking about here. To be able to provide a suitable visualization of this information we first have to be clear about the attributes of the visualized data.
A term that I’ve mentioned several times now is listening history. In our understanding an ideal listening history describes all songs that a person has listened to, possibly in their lifetime. What’s important here is that
Each song is a pre-existing piece of music that has attributes such as artist, title, etc.
And second, each song has been heard by the owner of the history at least in parts.
From the perspective of infovis, listening histories are multivariate time series. We can of course interpret it as univariate and time-centric,
as each section of time either contains music or it does not and can then start to extract e.g., listening sessions
But much more insight can be gained once we go beyond this binary classification.
So, the first thing we can do about that is put the songs into the musical hierarchy of albums, artists and genres. While this classification is not perfect and oftentimes the topic of heated debates, at least it’s widely-known among all music listeners. One more step to overcome the downsides of a strict hierarchy is adding user-generated keywords into the mix…
… that can become a stand-in for any number of different hierarchies or classifications. Ok, so once we have mapped the space of ideal music listening into this neat format we’re good to go building a visualization on top of it, but, unfortunately, the real world is messy…
Last.fm provides the so-called “audioscrobbler”, a software that’s running in the background and tracking all music files that are played on the computer. This procedure comes with its own limitations as the resulting listening histories
Are both incomplete and noisy.
Gaps in a listening history can come from various places…
One common source is that the listener is using non-supported hardware for listening
Another that music comes from other sources like when shopping or being at a friend’s place.
Noise, i.e., too many songs are tracked is also quite common…
The user might leave the computer while the music keeps on playing…
Or someone else is using the computer while the audioscrobbler is still running.
So much for the data space and its attributes. Next, we have to think about who our users are and what they want to do. All lifelogging applications are first of all about
Stroking your ego. It’s about learning about yourself, understanding what you did, maybe finding patterns that you were not aware of and remembering the past.
We defined two larger types of tasks that should be possible with one’s listening history: First, the rather impersonal analysis where
First, the rather impersonal analysis where users are looking for patterns within the data. The nice thing about that is that all possible insights are hidden within the data, which means that people who haven’t “created” the listening history are able to understand what’s going on (or at least see the patterns). Still, these insights can only be on an abstract level – we see which songs are repeated are popular but don’t know why.
To dive into the actual, underlying reasons we have to ask the creators of the data themselves, as they might be aware what intentions they had when picking a certain song. In this personal mode, a user’s memories form the second, complementary data source. The only thing is, we somehow have to reach the memories that the user’s have about a certain period in their lives and using songs with timestamps is not the best way to do it.
Much better memory triggers are photos or other actively created items. Therefore, to make sense of histories in this personal mode, adding such contextual information can be really helpful.
Ok, so with these two use cases of analysis and personal mode in mind, let’s look at related work from this area.
Understanding how people listen to music is the domain of music psychologists and music sociologists. They have uncovered fascinating aspects about human behavior in this regard but of course nothing about how to visualize that.
The concept of the timeline is common in Infovis, and for personal information it has been repeatedly applied in projects like LifeLines (for medical data), and PostHistory or TheMail (for email archives).
Two projects visualize listening histories: The Stacked or Streamgraphs by Byron and Wattenberg show overviews of prominent artists in listening histories. In our own former work, Pulling Strings from a Tangle, we presented two playful visualizations for this type of data, but they, too, could only paint an abstract picture of it.
So, when we went about designing our own tool, LastHistory, we wanted to create something that allowed gaining an overview of a listening history, but also explore it in detail. Finally, we wanted to integrate contextual information as memory triggers.
For our design requirements: We had non-infovis experts as users. So we decided to keep the interface as “non-threatening” as possible and make more complex tasks not obligatory.
Second, the tasks that users try to fulfill with the tool were nice to have, but not vital to them. So it was important to give them an immediate benefit and keep from frustrating them. Finally, we had to work with missing and imperfect data and users should be aware of this fact and not blame the visualization.
With these considerations we arrived at the final version of LastHistory and I’ll try to give you a quick rundown of it.
The largest part of the application is taken up with a 2D timeline: all songs are represented as small circles and mapped horizontally to the day and vertically to the time of day of their timestamps. This way, we can easily see daily rhythms,
Like at what time this user went to bed.
And here’s another example: A user who gets up at the same time everyday and listens to music first thing in the morning. The great thing about this mapping is that users have an immediate benefit even without interacting with the visualization.
Each song’s genre is color-coded, so the user gets an immediate overview over the variety of songs. We’re of course restricted in the number of colors we can use to keep them distinguishable.
We also separated the whole interaction into two modes, ‘analysis’ and ‘personal’. The main difference is that the personal mode also displays photos and calendar entries from the user’s computer to provide context and make it easier to remember what happened at what time and understanding the listening decisions. Users can simply switch between the two modes with the button in the upper left corner.
Beyond static visualization, users can navigate within the visualization by panning, triggered by dragging with the mouse
One-dimensional zooming by using the mouse’s zoom wheel or the slider in the lower right corner allows them to focus on certain sections of the history.
Hovering over a song shows a box with user-generated keywords from last.fm, but more prominently: connects this song with all other instances of it throughout the history. So, users can easily see when they listened to this one song.
Preceding and succeeding repeated songs are also highlighted, so sequences such as albums or other predefined playlists are automatically highlighted.
Finally, in the upper right corner of the application, there’s a textbox for filtering where users can enter freeform terms. It’s possible to enter song or album titles or artist names to filter all other songs.
But the filter box can also be used for temporal queries by entering dates, or periods of time, so users can, for example, see all songs that they listened to in autumn before noon.
Having finished implementing LastHistory we wanted to find out how well our considerations had worked and if people would actually find it useful.
First, we gave LastHistory to four participants in a lab study and let them play around with it for an hour. They analyzed their own listening history in combination with photos and calendar entries and told our experimenter what they were able to find out and what they liked or disliked about the application.
Results were insights about daily and other rhythms: One participant stopped listening to music during the week when she started working. A few noted how artists were rediscovered after going to a concert or them releasing a new album. And gaps and noise could also be seen and identified thanks to photos: Three participants found time periods empty where they were on vacation.
But much more interesting than these case studies was the feedback we received in the wild: We made LastHistory available for download and it was quickly picked up by blogs like Life Hacker or Newsweek’s website, which gave us a lot of attention.
In the end, about 5.000 people downloaded the application. We included a link to an online questionnaire that was shown after users had spent some time with the application and at the time of writing the paper, 243 people had filled out that form which was a valuable source of information for us.
Our users were predominantly male (95.1% to 4.9%) and most of them (56%) had a background in academia or technology. Age range was 16 to 67 years, with an average of 27.2 years.
Regarding the available data, almost all users had their own listening histories available (97.1%), but only about one third (37%) photos and 18.5% calendars.
This, of course, led to a dominance of the analysis mode over the personal mode: 75.7% found it useful, while only 51.8% said that about the personal mode. But when you only ask people who had
Photos and calendars available they reach about the same rating.
Insights that people gained also represented that: While some said about it: ‘I like this mode the best, it should be the default mode’, or ‘clicking on a photo gallery and listening to what I was listening to at the time was very powerful’ others complained about the missing data.
The analysis mode was more open for all users, and insights were mostly based on time and musical taste. One user found ‘I rarely listen to music between the hours of 9-11 a.m., even on weekends’ and another one noted his commuting pattern. A third one said she found ‘those ruts where you get stuck in listening to one particular song.’ Finally, users were also aware of the imperfect data.
But the most important finding for us was: People actually wanted something like LastHistory! The positive feedback just blew us away and we were very happy that people liked it. And even though the application wasn’t completely trivial, around 75% of our users found it easy to learn.
And with that, I want to talk a little bit about what we learned and where to go from here.
First, of course, our greatest fear as Infovis researchers is that our work is labeled as ‘pretty pictures’. But if you can build a pretty visualization without sacrificing functionality or making reading it more difficult why not do it? Users always prefer an aesthetically pleasing application to a dull one. And yes, even expert users. Thinking about users …
One new aspect that this very personal data type let us do was integrate a non-tangible data type into the visualization: the user’s memory. Depending on the use case, this can be a powerful source for understanding and learning. Providing suitable memory triggers is essential here.
Finally, lifelogging won’t go away and it will probably only get worse. Especially as all media consumption becomes digital (even books!) tracking every little aspect of reading, watching and listening will become commonplace.
The concepts that we used in LastHistory would of course also work for these other media and we could basically add all the data streams we could think of to the timeline. But I think what’s more in this regard is that people (1) actually want something like that and (2) that information visualization can help them for understanding their behavior and reminiscing. That means that their data is no longer only useful to companies and governments: These visualizations can give the data back to their creators. And we as visualization researchers and practitioners are the people to build those tools.