SlideShare a Scribd company logo
1 of 266
Download to read offline
UsingVisualizations for
Music Discovery
ISMIR 2009
Justin Donaldson
Paul Lamere
UsingVisualizations for
Music Discovery
ISMIR 2009
Justin Donaldson
Paul Lamere
Or ...
85 ways to visualize your music
2
Speakers
3
Paul Lamere is the Director of Developer Community at
The Echo Nest, a research-focused music intelligence startup
that provides music information services to developers and
partners through a data mining and machine listening
platform. Paul is especially interested in hybrid music
recommenders and using visualizations to aid music
discovery. Paul also authors 'Music Machinery' a blog
focusing on music discovery and recommendation.
Justin Donaldson is a PhD candidate at Indiana University
School of Informatics, as well as a regular research intern at
Strands, Inc. Justin is interested with the analyses and
visualizations of social sources of data, such as those that are
generated from playlists, blogs, and bookmarks.
Outline of the talk
• Why visualize
• Some history
• Core concepts
• Survey of MusicVisualizations
• Tools / Resources
• Conclusions
4
Music Discovery Challenge
• Millions of songs
• We need tools
• Recommendation
• Playlisting
•Visualization
5
WhyVisualize?
Visualizations can tell a story
6
WhyVisualize?
7
Visualizations can engage our pattern-matching brain
WhyVisualize?
• Anscombe’s “Quartet”
• Each distribution has equivalent summary statistics
• Means, correlations, variance: Sometimes one number summaries
are deceiving.
property value
x mean 9
x var 10
y mean 7.5
y var 3.75
x y cor. 0.816
linear reg. y = 3 + .5x
http://en.wikipedia.org/wiki/Anscombe%27s_quartet
8
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Semantic Pattern Mappings
9
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Semantic Pattern Mappings
10
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Basic Popout Channels
11
Color
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Basic Popout Channels
12
Size
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Basic Popout Channels
13
Shape
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Basic Popout Channels
14
Motion
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Basic Popout Channels
14
Motion
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Basic Popout Channels
14
Motion
Using Visualizations for Music Discovery - ISMIR 2009
Colin Ware -Visual Thinking for Design
Basic Popout Channels
15
Spatial Grouping
Using Visualizations for Music Discovery - ISMIR 2009
Flickr photo by Stéfan
Overloading the channels
16
Using Visualizations for Music Discovery - ISMIR 2009
Visualization Name
Visualization ofVisualizations
17
Using Visualizations for Music Discovery - ISMIR 2009
Tree Map
Graph Time Series
Scatter Hybrid
Connections Interactive
18
Types of visualizations
Using Visualizations for Music Discovery - ISMIR 2009
Objects to be discovered
19
human machine
Artists 23 37
Albums 0 6
Tracks 3 17
Samples 3 0
Playlists 0 0
Ordered
Tracks
0 9
Concerts 0 0
Blogs 0 0
Music
Videos
0 0
human machine
Genre 18 14
Users 0 4
Radio 0 2
DJs 0 1
Games 0 0
Commu
nities
0 0
Venues 0 0
Taste
Trends
2 2
Labels 1 0
26 Human rendered 55 Machine rendered
visualizations
Using Visualizations for Music Discovery - ISMIR 2009
Features Used
20
Content Beat, tempo, timbre, harmonic content, lyrics
Cultural web and playlist co-occurrence, purchase/play history, social
tags, genre, text aura, expert opinion, appearance, popularity
Relational member of, collaborated on, supporting musician for,
performed as, parent, sibling, married, same label, produced
Mood happy, sad, angry, relaxed, autumnal
Location city, country, lat/long,
Time the hits from the 60s, 70s 80s and today, recent play history
Misc IQ,Temperature, Name morphology
Using Visualizations for Music Discovery - ISMIR 2009
A scene from High Fidelity
More features
21
Using Visualizations for Music Discovery - ISMIR 2009
A scene from High Fidelity
More features
21
Human rendered
visualizations
22
Human rendered visualizations
Some common techniques
• The Flow
• The Map
• The Tree
• The Graph
23
• History
• Influences
• Similarity
• Popularity
24
Human rendered visualizations
Some common techniques
• History
• Influences
• Similarity
• Popularity
24
Human rendered visualizations
Some common techniques
• History
• Influences
• Similarity
• Popularity
24
Human rendered visualizations
Some common techniques
Genealogy of Pop and Rock Music
25
Genealogy of Pop and Rock Music - Reebee Garofalo
Genealogy of Pop and Rock Music
25
Genealogy of Pop and Rock Music - Reebee Garofalo
Genealogy of Pop and Rock Music
25
Genealogy of Pop and Rock Music - Reebee Garofalo
Ishkur’s Guide to Electronic Dance Music
26
Ishkur’s Guide to Electronic Dance Music
26
Music History as a London Tube Map
27
Dorian Lynskey - The Guardian
The Guardian | Friday February 3 2006 9
The Guardian | Friday February 3 2006
verstory
oing underground
d we chart the branches and connections of 100 years of music using the London Underground map? Dorian Lynskey explains how a box of coloured crayons and a lot of swearing helped
like a deeply implausible
to plot the history of 20th
music on the London
ound map devised by
eck in 1933. Artist Simon
n transformed the tube
nstellation of famous
1992 work The Great Bear,
have to make them all link
all, a tall order to find a
s also a comedian. But for
ork every interchange had
n the context of musical
nlikely prospect.
ut with a packet of coloured
sheets of A4 taped to-
big box of doubt, but the
racter of each line quickly
certain genre. Pop inter-
erything else, so that had to
Line; classical music for the
upies its own sphere,
t perfect for the Docklands
y. There were a couple of
ut by the end of one after-
signed genres to almost all
thrashed out most of the
ctions. The key stations
nt to the most eclectic
ecessarily the most impor-
les may be more signifi-
k but even their most de-
st admit that they never
.
m thus in place, the next
s were devoted to writing
ibbling them out (sorry,
and Lynyrd Skynyrd), ago-
ertain omissions, asking
ic critic Tom Service for in-
with the DLR, and swear-
e bit. Amazingly, there
mitous blind alleys. It just
ake sense.
r as possible to be objec-
nds I cannot stand are in
ome that I love dearly
ollowed chronology wher-
of the line allowed it. Each
epresents a sub-genre: rock
to grunge and psychedelia
es South-West London;
rges, north of Camden, into
d New York rap. If I was re-
e band name echoed the
on name: Highbury & Is-
me Sly & the Family Stone.
f course, will find flaws.
ences are so labyrinthine
ple equation will be imper-
or example, does pop stop
n? How can you draw a de-
tween soul and funk? These
that have plagued record
tors for decades and they’re
be solved here. But I think
hoices are justifiable given
ns of the form.
ple will quibble with omis-
hame, for example, that
e constantly runs in tan-
her the District or Metro-
thus leaving no room for
such as Kylie Minogue and
Boys. I should also point
eep my head from explod-
the remit to western, pre-
nglo-American music.
e those changes necessi-
don Underground’s under-
nsitivity to explosive refer-
derci, Massive Attack. For
they also took exception to
er Ol’ Dirty Bastard.
not some definitive history
an experiment to see if one
work can be overlaid on a
ifferent one. The elegance
Harry Beck’s design – its
of bustling intersections,
butaries, long, slanting tan-
upt dead ends, all sucked
turned wine bottle of the
eems to spark other con-
appeal to the brain’s innate
terning and structure. Plus
y piece of music journalism
coloured crayons should
u like it.
you think of the map at
an.co.uk/arts
at
umshop.co.uk
ks to Chris Townsend at
r London and Andrew
don Underground.
Music History as a London Tube Map
27
Dorian Lynskey - The Guardian
The Guardian | Friday February 3 2006 9
The Guardian | Friday February 3 2006
verstory
oing underground
d we chart the branches and connections of 100 years of music using the London Underground map? Dorian Lynskey explains how a box of coloured crayons and a lot of swearing helped
like a deeply implausible
to plot the history of 20th
music on the London
ound map devised by
eck in 1933. Artist Simon
n transformed the tube
nstellation of famous
1992 work The Great Bear,
have to make them all link
all, a tall order to find a
s also a comedian. But for
ork every interchange had
n the context of musical
nlikely prospect.
ut with a packet of coloured
sheets of A4 taped to-
big box of doubt, but the
racter of each line quickly
certain genre. Pop inter-
erything else, so that had to
Line; classical music for the
upies its own sphere,
t perfect for the Docklands
y. There were a couple of
ut by the end of one after-
signed genres to almost all
thrashed out most of the
ctions. The key stations
nt to the most eclectic
ecessarily the most impor-
les may be more signifi-
k but even their most de-
st admit that they never
.
m thus in place, the next
s were devoted to writing
ibbling them out (sorry,
and Lynyrd Skynyrd), ago-
ertain omissions, asking
ic critic Tom Service for in-
with the DLR, and swear-
e bit. Amazingly, there
mitous blind alleys. It just
ake sense.
r as possible to be objec-
nds I cannot stand are in
ome that I love dearly
ollowed chronology wher-
of the line allowed it. Each
epresents a sub-genre: rock
to grunge and psychedelia
es South-West London;
rges, north of Camden, into
d New York rap. If I was re-
e band name echoed the
on name: Highbury & Is-
me Sly & the Family Stone.
f course, will find flaws.
ences are so labyrinthine
ple equation will be imper-
or example, does pop stop
n? How can you draw a de-
tween soul and funk? These
that have plagued record
tors for decades and they’re
be solved here. But I think
hoices are justifiable given
ns of the form.
ple will quibble with omis-
hame, for example, that
e constantly runs in tan-
her the District or Metro-
thus leaving no room for
such as Kylie Minogue and
Boys. I should also point
eep my head from explod-
the remit to western, pre-
nglo-American music.
e those changes necessi-
don Underground’s under-
nsitivity to explosive refer-
derci, Massive Attack. For
they also took exception to
er Ol’ Dirty Bastard.
not some definitive history
an experiment to see if one
work can be overlaid on a
ifferent one. The elegance
Harry Beck’s design – its
of bustling intersections,
butaries, long, slanting tan-
upt dead ends, all sucked
turned wine bottle of the
eems to spark other con-
appeal to the brain’s innate
terning and structure. Plus
y piece of music journalism
coloured crayons should
u like it.
you think of the map at
an.co.uk/arts
at
umshop.co.uk
ks to Chris Townsend at
r London and Andrew
don Underground.
Visualizations as a graph
28
Visualizations as a graph
28
Visualizations as a graph
28
Visualizations as a graph
28
Visualizations as a graph
28
Visualizations as a graph
29
www.flickr.com/photos/angrypirate/364960689/
Visualizations as a graph
29
www.flickr.com/photos/angrypirate/364960689/
Visualizations as a graph
29
www.flickr.com/photos/angrypirate/364960689/
Visualizations as a graph
29
www.flickr.com/photos/angrypirate/364960689/
Visualizations as a graph
29
www.flickr.com/photos/angrypirate/364960689/
Using Visualizations for Music Discovery - ISMIR 2009
Fabian Schneidmadel - http://www.firutin.de/blog/
Visual Music Guiden
Images (cc) Fabian Schneidmadel
30
Using Visualizations for Music Discovery - ISMIR 2009
Fabian Schneidmadel - http://www.firutin.de/blog/
Visual Music Guiden
Images (cc) Fabian Schneidmadel
30
Visualizations as a tree
31
Visualizations as a tree
31
Visualizations as a tree
31
Visualizations as a tree
31
Handmade Genre Maps and Trees
32
Handmade Genre Maps and Trees
32
Handmade Genre Maps and Trees
32
Handmade Genre Maps and Trees
32
Handmade Genre Maps and Trees
32
Handmade Genre Maps and Trees
32
Handmade Genre Maps and Trees
32
Handmade Genre Maps and Trees
32
Handmade Genre Maps and Trees
32
Other types of visualizations
33
Other types of visualizations
33
Other types of visualizations
33
Other types of visualizations
33
Other types of visualizations
33
Other types of visualizations
34
Other types of visualizations
34
Artist as a function of temperature
35
Using Visualizations for Music Discovery - ISMIR 2009
Stephan Bauman -Senior Researcher - German Research Center for AI
ISMIR Tutorial 2005
36
Using Visualizations for Music Discovery - ISMIR 2009
Stephan Bauman -Senior Researcher - German Research Center for AI
ISMIR Tutorial 2005
36
Using Visualizations for Music Discovery - ISMIR 2009
Stephan Bauman -Senior Researcher - German Research Center for AI
ISMIR Tutorial 2005
36
Using Visualizations for Music Discovery - ISMIR 2009
Stephan Bauman -Senior Researcher - German Research Center for AI
ISMIR Tutorial 2005
36
Using Visualizations for Music Discovery - ISMIR 2009
Stephan Bauman -Senior Researcher - German Research Center for AI
ISMIR Tutorial 2005
36
Core Concepts
37
How do computers
represent music?
• Score - As MIDI data
• Signal - As digital wave
form
• Association - Through
associations with tags,
people, or other songs
38
(a brief note on)
Symbolic score data
• Typically provides an explicit compositional structure,
and sometimes even explicit performance directions.
• “Easy” to analyze/extract meaningful musicological
features (time signature, key, etc.)
• Scores are typically not present for most music, so
other forms of representation are becoming popular.
• We won’t cover score based representations of
music today.
39
Acoustic Data
• Easy (and even
occasionally legal) to
find digital wave form
representations of
music
• New techniques,
methods, and even
services for music
signal analysis.
http://www.echonest.com/ http://www.magnatune.com/ 40
Association Data
• Relatively new form of
music analysis
• New methods/services
as well
• What connections/
associations/relationships
does music share with
tags, people, playlists,
other music?
http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/matchbox-20-song-lyrics-tag-cloud
http://sixdegrees.hu/last.fm/
http://www.flickr.com/photos/scwn/153098444/
http://www.last.fm
41
Representing
“Lots of Songs”
• Many different types of music-
related information.
• Problems of sparsity, noise, data
set size.
• We need a common appropriate
format of representation and
method of comparison.
• We would like to work with
matrices.
Song Feature1 Feature2
“Love me
do”
0.4 0.9
“Hound
Dog”
3.2 1.4
“Paint it
Black”
3.4 1.5
42
Distances and Similarity
• To form a two/three dimensional
visualization, we will need to express a
distance or dissimilarity* between the
songs in vector form.
• Distance must be symmetric in order
to represent a metric space.
• Distance must be “non-degenerate”
(distance from song A to song A == 0)
Song
“Love
me do”
“Hound
Dog”
“Paint it
Black”
“Love
me do”
n/a 3.2 3.4
“Hound
Dog”
3.2 n/a 1.5
“Paint it
Black”
3.4 1.5 n/a
song
song
B to A
A to B
=
43
“Big Picture”
Visualization
• A couple of “simple” visualizations help to
assess variance, etc. across the relevant
dimensions.
• Boxplots, matrix plots, image plots, etc.
• Selectively normalizing data at this point to
have more control, and removing obvious
outlier (points/dimensions).
44
Multidimensional Scaling
• Reducing the number
of dimensions of the
music vector.
• Similar to “rotating”
the data to find large
dimensions of
variation/
dissimilarity in the
data.
Athens Barcelona Brussels Calais Cherbourg
Athens 0 3313 2963 3175 3339
Barcelona 3313 0 1318 1326 1294
Brussels 2963 1318 0 204 583
Calais 3175 1326 204 0 460
Cherbourg 3339 1294 583 460 0
Cologne 2762 1498 206 409 785
http://www.mathpsyc.uni-bonn.de/doc/delbeke/delbeke.htm 45
DEMO
46
Demo:Acoustic FeatureVectors
• Working with Magnatagatune dataset
• Combination of:
• Magnatune creative commons
licensed songs (clips)
• Tagatune project data
• Echo Nest feature data
• Freely available
http://tagatune.org/Magnatagatune.html
47
Demo: FeatureVector
Echonest data includes:
• loudness
• tempo
• segments with:
• pitch information
• timbre information
http://tagatune.org/Magnatagatune.html
48
Demo: FeatureVector
Demo motivation
• Genre classification has become a prominent trend in the IR
community. Explore the data pertaining to such systems.
• Current approaches can identify “textural” properties of
acoustic sound, but can lack cultural or structural properties of
music.West & Lamere note that:
“...a common example might be the confusion of a classical lute
timbre, with that of an acoustic guitar string that might be found
in folk, pop, or rock music...”
K. West and P. Lamere, “A model-based approach to constructing music similarity functions,”
EURASIP Journal on Advances in Signal Processing 2007 (2007).
49
Demo: FeatureVector
Demo motivation
• Magnatagatune happens to have
a large amount of Classical
music, and Classical Lute music
to boot.
• We’ll try to visualize this part of
the magnatagatune data set,
along with folk music.
50
Tools
51
Tools: FeatureVector
• Download Data
• Use R to process, analyze, and visualize data
• Free! (Libre!)
http://cran.r-project.org/
http://www.cyclismo.org/tutorial/R/
http://cran.r-project.org/doc/manuals/R-intro.html 52
Demo: FeatureVectors
53
Demo: FeatureVector
Basic Echo Nest Features
• (overall) loudness
• tempo
• pitch means (12) *
• pitch stdev (12) *
• timbre means (12) *
• timber stdev (12) *
* weighted by segment onset loudness
54
Boxplots
Boxplots show the five-
number summaries of a
distribution:
• sample maximum
• upper quartile
• median
• lower quartile
• sample minimum
• (outliers)
55
Boxplots
Boxplots show the five-
number summaries of a
distribution:
• sample maximum
• upper quartile
• median
• lower quartile
• sample minimum
• (outliers)
56
Demo: FeatureVector
Raw Feature Boxplots...
• Are these all normal distributions?
• Are all of these weighted evenly?
• Scaling becomes important... why?
57
Demo: FeatureVector
Looking at scaled summary statistics
These are the ‘expected’ values for any
song, so they’re ‘normal’ for a song.
We want to enhance relationships
between songs that differ from the norm.
58
Demo: FeatureVector
Weighted Feature Boxplots...
• Standard normalized data. All means are 0, all standard
deviations are 1.
• We can find ‘interesting’ relationships between songs better this
way. Distances can be understood in terms of standard
deviations from the mean.
• However... 24 pitch and 24 timbre features, vs. just 1 tempo... 59
Demo: FeatureVector
Principle Component
Analysis (PCA) of
timbre and pitch
information reveal a
high degree of
covariance.
60
Demo: FeatureVector
Plotting the PCA
loadings show the
‘structure’ of how
the pitch profiles co-
vary across the
music.
Blue = “Classical
Guitar”
(almost entirely lute
music)
Red = “Folk”
Green= “Both”
(flips and rotations)
61
Demo: FeatureVector
Performing a Factor analysis
shows evidence of
“musicological” factors.
(alternating positive/negative
loadings for both means and
variances, indicating certain
“keys” are followed for songs)
Should pitch profile distances
be figured another way?
Loadings:
Factor1 Factor2
pitch_mean_1 -0.117 0.625
pitch_mean_2 0.576 -0.270
pitch_mean_3 -0.585
pitch_mean_4 0.677 0.126
pitch_mean_5 -0.103 -0.487
pitch_mean_6 0.473
pitch_mean_7 0.332 -0.504
pitch_mean_8 -0.512 0.217
pitch_mean_9 0.527 -0.124
pitch_mean_10 -0.520 -0.274
pitch_mean_11 0.475 0.452
pitch_mean_12 0.216 -0.359
pitch_var_1 -0.318 0.689
pitch_var_2 0.638 -0.377
pitch_var_3 -0.661 0.100
pitch_var_4 0.695 0.289
pitch_var_5 -0.169 -0.536
pitch_var_6 0.658
pitch_var_7 0.363 -0.550
pitch_var_8 -0.660 0.315
pitch_var_9 0.669 -0.104
pitch_var_10 -0.620 -0.213
pitch_var_11 0.385 0.593
pitch_var_12 0.124 -0.399
62
Demo: FeatureVector
Timbre features
clearly split the
different tags we
were concerned
with.
Blue = “Classical
Guitar”
Red = “Folk”
Green= “Both”
63
Demo: FeatureVector
Timbral factors are a lot more
straightforward. Certain songs
have a lot of timbral variance.
Other songs simply have
strong overall timbral means.
Loadings:
Factor1 Factor2
timbre_mean_1 0.987
timbre_mean_2 0.489
timbre_mean_3 0.151 0.728
timbre_mean_4 -0.139 0.349
timbre_mean_5 -0.570
timbre_mean_6
timbre_mean_7 -0.126 0.248
timbre_mean_8 0.175 -0.556
timbre_mean_9 0.122 0.589
timbre_mean_10 0.257 -0.461
timbre_mean_11 -0.363 -0.238
timbre_mean_12
timbre_var_1 0.582 0.614
timbre_var_2 0.681 -0.286
timbre_var_3 0.822 -0.114
timbre_var_4 0.680 -0.387
timbre_var_5 0.779
timbre_var_6 0.241 -0.690
timbre_var_7 0.810
timbre_var_8 0.712
timbre_var_9 0.792
timbre_var_10 0.591 -0.358
timbre_var_11 0.667 -0.140
timbre_var_12 0.736
64
Demo:Association Data
65
Demo:Association Data
• Magnatagatune has ‘tag’ data for songs
• Tags only exist if two people used the same tag for the
same song
• 188 total tags
clip_id no.voice singer duet plucking hard.rock world bongos harpsichord female.singing classical
1 2 0 0 0 0 0 0 0 0 0 0
2 6 0 0 0 0 0 0 0 1 0 0
3 10 0 0 0 1 0 0 0 0 0 0
4 11 0 0 0 0 0 0 0 0 0 0
5 12 0 1 0 0 0 0 1 0 0 1
6 14 0 0 0 0 0 0 0 0 0 0
66
Demo:Association Data
• Tags are distributed by a power-law
• It’s not as meaningful to share a “guitar” tag. A
“soprano” tag is far more informative.
• We need to form a comparison of tags based on
the weighted relevance of the terms.
guitar classical slow techno strings drums electronic rock
4852 4272 3547 2954 2729 2598 2519 2371
fast piano ambient beat violin vocal synth female
2306 2056 1956 1906 1826 1729 1717 1474
indian opera male singing vocals no.vocals harpsichord loud
1395 1296 1279 1211 1184 1158 1093 1086
67
Demo:Association Data
• TF/IDF (term frequency/document frequency)
• n(i,j) = number of times a term i appears in a
‘document’ j.
• n(k,j) = total number of terms in ‘document’ j.
• |D| = total number of ‘documents’
• {d:...} = total documents containing i
• documents = tags for song
http://en.wikipedia.org/wiki/Tf%E2%80%93idf
68
Demo:Association Data
• Forming a similarity/dissimilarity between
weighted tags commonly involves finding the
angle between two vectors.
• Cosine dissimilarity is one (of many) ways of
expressing a distance between vectors.
• A, B = vectors of attributes
• ||A||, ||B|| = magnitudes of vectors
http://en.wikipedia.org/wiki/Cosine_similarity
69
Demo:Association Data
Cosine similarity of tags
also (not surprisingly)
clearly separates the
two tags
70
Demo: Hybrid
71
Demo: FeatureVector
Timbre features
clearly split the
different tags we
were concerned
with.
Blue = “Classical
Guitar”
Red = “Folk”
Green= “Both”
72
Conclusions (demo)
• Visualizations can help understand “big picture” relationships among
songs, and support that the relationships are meaningful. (left plot)
• Visualizations can help understand “mismatches” in modeling/clustering/
classification approaches. (x2)
• Visualizations can help detect noise (x1) and outliers. (x3, x4)
• Visualizations can help find a way to improve the underlying model.
Opportunity: Hybrid Maps
Mixing information from both
timbre & term spaces has a great
opportunity for forming a ‘better’
representation of genre
+ =
74
“Reading” Conventional
Dimensionality Reductions
• Large dissimilarity is preserved
over small dissimilarity.
• Try to understand the plot in
terms of its two biggest
orthogonal distances. If those
don’t make sense, then the plot
is suspect.
• Next, look for local structure,
clustering, etc.
75
Other Techniques
76
Other Techniques
• Locally Linear Embedding : Only consider
‘close’ points in distance matrix. Good at
recovering from “kinks”,“folds”, and
“spirals” in data (chroma)
• Isomap : Establish geodesic
“neighborhood” distances using Dijkstra’s
algorithm, and solve with mds.
• T-SNE : Convert neighbor distances to
conditional probabilities. Perform
gradient descent on Kullback-Leibler
Divergence. Good at recovering clusters
at different scales.
77
Other Techniques
• Many advanced techniques sacrifice “large scale” dissimilarities in order to
preserve/enhance “local” features.
• Sometimes large scale information is useful (to determine if scales are wrong,
or if there are significant outliers)
• Interpretation of advanced techniques is much more complicated. (What are
the loadings, factors, of features etc.?)
• It can be harder to use advanced dimensionality reduction techniques to
inform classifier design.
(observations)
78
Using Visualizations for Music Discovery - ISMIR 2009
The Goal - build an artist graph suitable for music exploration
Building an artist map
79
Problem:The artist space is very complex
Using Visualizations for Music Discovery - ISMIR 2009
The Data
80
• Crawled The Echo Nest API for:
• Artist names
• Artist hotness
• Artist familiarity
• 15 most similar artists
• 70,000 artists
• 250,000 artist-artist links
• Available at musicviz.googlepages.com
Making a good graph
• Minimize edge crossings, bends and curves
• Maximize angular resolution, symmetry
81
Good Bad
Bad
Good
Cognitive measurements of graph aesthetics, Ware, Purchase, Colpoys, McGill
Using Visualizations for Music Discovery - ISMIR 2009
Graphing the Beatles Neighborhood
digraph {
"The Beatles" -> "Electric Light Orchestra";
"The Beatles" -> "The Rutles";
"The Beatles" -> "The Dave Clark Five";
"The Beatles" -> "The Tremeloes";
"The Beatles" -> "Manfred Mann";
"The Beatles" -> "The Lovin' Spoonful";
"The Beatles" -> "The Rolling Stones";
"The Beatles" -> "Creedence Clearwater Revival";
"The Beatles" -> "Bee Gees";
"The Beatles" -> "the Hollies";
"The Beatles" -> "Badfinger";
"The Beatles" -> "Paul McCartney";
"The Beatles" -> "Julian Lennon";
"The Beatles" -> "George Harrison";
"The Beatles" -> "John Lennon";
}
82
http://www.graphviz.org/
Using Visualizations for Music Discovery - ISMIR 2009
Already interconnections start to overwhelm
Artist map - level 1 out
83
Using Visualizations for Music Discovery - ISMIR 2009
Artist map - level 1 in/out
84
Using Visualizations for Music Discovery - ISMIR 2009
Artist map - level 1 in/out
84
Using Visualizations for Music Discovery - ISMIR 2009
Artist map - level 1 in/out
84
Using Visualizations for Music Discovery - ISMIR 2009
Artist map - level 1 in/out
84
Using Visualizations for Music Discovery - ISMIR 2009
Artist map - level 1 in/out
84
Using Visualizations for Music Discovery - ISMIR 2009
Artist map - level 1 in/out
84
Using Visualizations for Music Discovery - ISMIR 2009
Artist map - level 1 in/out
84
Using Visualizations for Music Discovery - ISMIR 2009
116 artists and 665 edges.We are totally overwhelmed.
Artist map - level 2
85
Using Visualizations for Music Discovery - ISMIR 2009
116 artists and 665 edges.We are totally overwhelmed.
Artist map - level 2
85
Using Visualizations for Music Discovery - ISMIR 2009
Eliminate the edges and use spring force embedding to position the artists
Simplifying the graph
86
Using Visualizations for Music Discovery - ISMIR 2009
Eliminate the edges and use spring force embedding to position the artists
Simplifying the graph
86
Using Visualizations for Music Discovery - ISMIR 2009
Attach artists to only one most similar artist
Reducing the edges
87
Using Visualizations for Music Discovery - ISMIR 2009
Attach artists to only one most similar artist
Reducing the edges
87
Using Visualizations for Music Discovery - ISMIR 2009
Attach artists to only one most similar artist
Reducing the edges
87
Using Visualizations for Music Discovery - ISMIR 2009
Attach artists to only one most similar artist
Reducing the edges
87
Using Visualizations for Music Discovery - ISMIR 2009
Attach artists to the most similar artist that has greater familiarity
Putting ELO II in its place
88
Using Visualizations for Music Discovery - ISMIR 2009
Attach artists to the most similar artist that has greater familiarity
Putting ELO II in its place
88
Using Visualizations for Music Discovery - ISMIR 2009
Attach artists to the most similar artist that has greater familiarity
Putting ELO II in its place
88
Using Visualizations for Music Discovery - ISMIR 2009
Build a minimum spanning tree from the graph
Making the graph connected
89
Using Visualizations for Music Discovery - ISMIR 2009
Build a minimum spanning tree from the graph
Making the graph connected
89
Using Visualizations for Music Discovery - ISMIR 2009
Build a minimum spanning tree from the graph
Making the graph connected
89
Using Visualizations for Music Discovery - ISMIR 2009
Prefer connections to bands of similar familiarity
Fixing the swinging blue jeans problem
90
Using Visualizations for Music Discovery - ISMIR 2009
Miles Davis
Other near-neighbor graphs
91
Using Visualizations for Music Discovery - ISMIR 2009
Miles Davis
Other near-neighbor graphs
91
Using Visualizations for Music Discovery - ISMIR 2009
A browsing network for music discovery
2K artists
92
Using Visualizations for Music Discovery - ISMIR 2009
A browsing network for music discovery
2K artists
92
Using Visualizations for Music Discovery - ISMIR 2009
The Artist Graph
Making the graph interactive
93
Using Visualizations for Music Discovery - ISMIR 2009
Danny Holten, Jarke J. van Wijk
Force-Directed Edge Bundling for GraphVisualization
94
Survey of Music
Visualizations
95
Using Visualizations for Music Discovery - ISMIR 2009
Graphs
96
Using Visualizations for Music Discovery - ISMIR 2009
http://www.dimvision.com/musicmap/
musicmap
97
Using Visualizations for Music Discovery - ISMIR 2009
MusicPlasma.com
Music Plasma
98
Using Visualizations for Music Discovery - ISMIR 2009
http://audiomap.tuneglue.net/
tuneglue
99
Using Visualizations for Music Discovery - ISMIR 2009
http://lastfm.dontdrinkandroot.net/
Last.fm artist map
100
Using Visualizations for Music Discovery - ISMIR 2009
http://lastfm.dontdrinkandroot.net/
Last.fm artist map
100
Using Visualizations for Music Discovery - ISMIR 2009
http://rama.inescporto.pt/
RAMA - Relational Artist Maps
101
Late-Breaking Demo Session
Using Visualizations for Music Discovery - ISMIR 2009
http://www.music-map.com/
gnod music-map
102
Using Visualizations for Music Discovery - ISMIR 2009
http://www.musicmesh.net/
musicmesh
103
musicovery
104
http://musicovery.com/
Using Visualizations for Music Discovery - ISMIR 2009
http://www.repeatingbeats.com/thesis.php - Steve Lloyd - QMUL
SoundBite for SongBird
105
Using Visualizations for Music Discovery - ISMIR 2009
http://www.repeatingbeats.com/thesis.php - Steve Lloyd - QMUL
SoundBite for SongBird
105
Using Visualizations for Music Discovery - ISMIR 2009
http://amaznode.fladdict.net/
Amaznode
106
Using Visualizations for Music Discovery - ISMIR 2009
http://torrens.files.wordpress.com/2009/06/ismir.pdf
Torrens, Patrick Hertzog, Josep-Lluís Arcos
Visualizing and Exploring Personal Libraries
107
Using Visualizations for Music Discovery - ISMIR 2009
http://torrens.files.wordpress.com/2009/06/ismir.pdf
Torrens, Patrick Hertzog, Josep-Lluís Arcos
Visualizing and Exploring Personal Libraries
107
Using Visualizations for Music Discovery - ISMIR 2009
http://torrens.files.wordpress.com/2009/06/ismir.pdf
Torrens, Patrick Hertzog, Josep-Lluís Arcos
Visualizing and Exploring Personal Libraries
107
Using Visualizations for Music Discovery - ISMIR 2009
http://musicexplorerfx.citytechinc.com/
Music Explorer FX
108
Using Visualizations for Music Discovery - ISMIR 2009
Last FM tasteOgraph
http://geniol.publishpath.com/last-fm-tasteograph
Social Music Discovery
109
Using Visualizations for Music Discovery - ISMIR 2009
http://www.flokoon.com/#lastfm
Floko-On
110
Using Visualizations for Music Discovery - ISMIR 2009
http://arcus-associates.com/orpheus/
Project Orpheus
111
Using Visualizations for Music Discovery - ISMIR 2009
http://www.sfu.ca/~jdyim/musicianMap/
Ji-DongYim Chris Shaw Lyn Bartram
Musician Map: visualizing music collaborations over time
112
Using Visualizations for Music Discovery - ISMIR 2009
http://www.identitee.com/visualizer.php
lyric connections
113
music tees and tee shirts on i/denti/tee
Using Visualizations for Music Discovery - ISMIR 2009
http://journal.webscience.org/110/
Interacting with linked data about music
114
http://journal.webscience.org/110/
Using Visualizations for Music Discovery - ISMIR 2009
http://journal.webscience.org/110/
Interacting with linked data about music
115
http://journal.webscience.org/110/
Using Visualizations for Music Discovery - ISMIR 2009
http://www.stanford.edu/~dgleich/demos/worldofmusic/WorldOfMusic.html
David Gleich, Matt Rasmussen, Leonid Zhukov, and Kevin Lang
z
The World Of Music
116
Using Visualizations for Music Discovery - ISMIR 2009
http://www.stanford.edu/~dgleich/demos/worldofmusic/WorldOfMusic.html
David Gleich, Matt Rasmussen, Leonid Zhukov, and Kevin Lang
z
The World Of Music
116
Using Visualizations for Music Discovery - ISMIR 2009
http://sixdegrees.hu/last.fm/index.html
Last.fm Artist Map
117
Using Visualizations for Music Discovery - ISMIR 2009
http://sixdegrees.hu/last.fm/index.html
Last.fm Artist Map
117
Visualizations of the connections amongst
geographically located iTunes libraries
118
http://caleblarsen.com/projects/visualization-of-the-inherent-connections-/#0
Visualizations of the connections amongst
geographically located iTunes libraries
118
http://caleblarsen.com/projects/visualization-of-the-inherent-connections-/#0
Using Visualizations for Music Discovery - ISMIR 2009
Maps
119
Using Visualizations for Music Discovery - ISMIR 2009
http://playground.last.fm/iom - Islands of Music - Elias Pampalk
Maps
120
Using Visualizations for Music Discovery - ISMIR 2009
http://margrid.sness.net/
marGrid
121
Using Visualizations for Music Discovery - ISMIR 2009
http://www.ifs.tuwien.ac.at/mir/playsom.html
Vienna University of Technology
PlaySOM - Intuitive access to Music Archives
122
Using Visualizations for Music Discovery - ISMIR 2009
http://fm4.orf.at/soundpark Martin Gasser,Arthur Flexer
FM4 Soundpark
123
Using Visualizations for Music Discovery - ISMIR 2009
http://www.cp.jku.at/projects/nepTune/
nepTune
124
Using Visualizations for Music Discovery - ISMIR 2009
http://www.cp.jku.at/projects/nepTune/
nepTune
124
Using Visualizations for Music Discovery - ISMIR 2009
http://musicminer.sourceforge.net/
Databionic Music Miner
125
Using Visualizations for Music Discovery - ISMIR 2009
http://musicminer.sourceforge.net/
Databionic Music Miner
125
Using Visualizations for Music Discovery - ISMIR 2009
Presented at ISMIR 2009 by Dominik Lübbers
soniXplorer
126
Using Visualizations for Music Discovery - ISMIR 2009
Presented at ISMIR 2009 by Dominik Lübbers
soniXplorer
126
Using Visualizations for Music Discovery - ISMIR 2009
Presented at ISMIR 2009 by Dominik Lübbers
soniXplorer
126
Using Visualizations for Music Discovery - ISMIR 2009
Presented at ISMIR 2009 by Dominik Lübbers
soniXplorer
126
Using Visualizations for Music Discovery - ISMIR 2009
http://www.mhashup.com/
mHashup
127
Using Visualizations for Music Discovery - ISMIR 2009
http://www.mhashup.com/
mHashup
127
Using Visualizations for Music Discovery - ISMIR 2009
ISMIR 2008 - Oscar Celma
GeoMuzik
128
Using Visualizations for Music Discovery - ISMIR 2009
http://www.cs.kuleuven.be/~sten/lastonamfm
Last on AM/FM
129
Using Visualizations for Music Discovery - ISMIR 2009
http://playground.last.fm/demo/tagstube
Last.fm tube tags
130
Using Visualizations for Music Discovery - ISMIR 2009
http://playground.last.fm/demo/tagstube
Last.fm tube tags
130
Using Visualizations for Music Discovery - ISMIR 2009
Scatter plots
131
Using Visualizations for Music Discovery - ISMIR 2009
Justin Donaldson
ZMDSVisualization
132
Using Visualizations for Music Discovery - ISMIR 2009
Visualization Name
133
Using Visualizations for Music Discovery - ISMIR 2009
Visualization Name
133
Using Visualizations for Music Discovery - ISMIR 2009
Visual Playlist Generation on the Artist Map - Van Gulick,Vignoli
Using map for playlist navigation
134
Using Visualizations for Music Discovery - ISMIR 2009
http://mpac.ee.ntu.edu.tw/~yihsuan/pub/MM08_MrEmo.pdf
Mr. Emo: Music Retrieval in the Emotional Plane
135
Using Visualizations for Music Discovery - ISMIR 2009
http://mpac.ee.ntu.edu.tw/~yihsuan/pub/MM08_MrEmo.pdf
Mr. Emo: Music Retrieval in the Emotional Plane
135
Using Visualizations for Music Discovery - ISMIR 2009
http://www.burstlabs.com/
Burst Labs
136
Using Visualizations for Music Discovery - ISMIR 2009
http://mediacast.sun.com/share/plamere/sitm_final.pdf - Lamere, Eck
Search Inside the Music
137
Using Visualizations for Music Discovery - ISMIR 2009
The Mufin Player
3D Track similarity
138
Using Visualizations for Music Discovery - ISMIR 2009
The Mufin Player
3D Track similarity
138
Using Visualizations for Music Discovery - ISMIR 2009
http://thesis.flyingpudding.com/ - Anita Lillie
Music Box
139
Using Visualizations for Music Discovery - ISMIR 2009
http://thesis.flyingpudding.com/ - Anita Lillie
Music Box
139
Using Visualizations for Music Discovery - ISMIR 2009
http://www.flyingpudding.com/projects/viz_music/
Pictures of Songs
140
Using Visualizations for Music Discovery - ISMIR 2009
http://www.cs.kuleuven.be/~jorisk/thrillerMovie.mov
Visualizing Emotion in Lyrics
141
Using Visualizations for Music Discovery - ISMIR 2009
Sampling Connections
142
Using Visualizations for Music Discovery - ISMIR 2009
http://jessekriss.com/projects/samplinghistory/
The History of Sampling
143
Using Visualizations for Music Discovery - ISMIR 2009
http://jessekriss.com/projects/samplinghistory/
The History of Sampling
143
Using Visualizations for Music Discovery - ISMIR 2009
Graham Coleman - http://www.iua.upf.edu/~gcoleman/pubs/mused07.pdf
Mused: Navigating the personal sample library
144
Using Visualizations for Music Discovery - ISMIR 2009
Graham Coleman - http://www.iua.upf.edu/~gcoleman/pubs/mused07.pdf
Mused: Navigating the personal sample library
144
Using Visualizations for Music Discovery - ISMIR 2009 145
Time Series
Using Visualizations for Music Discovery - ISMIR 2009
http://www.diametunim.com/muse/
last.fm spiral
146
Using Visualizations for Music Discovery - ISMIR 2009 147
http://www.leebyron.com/what/lastfm/ - Lee Byron
Visualizing Listening History
Using Visualizations for Music Discovery - ISMIR 2009 147
http://www.leebyron.com/what/lastfm/ - Lee Byron
Visualizing Listening History
Using Visualizations for Music Discovery - ISMIR 2009
http://www.bbc.co.uk/radio/labs/radiowaves/
Radio Waves
148
Using Visualizations for Music Discovery - ISMIR 2009
http://www.bbc.co.uk/radio/labs/radiowaves/
Radio Waves
148
Using Visualizations for Music Discovery - ISMIR 2009
http://bowr.de/blog/?p=49
Dominikus Baur University of Munich
Pulling strings from a tangle
149
Using Visualizations for Music Discovery - ISMIR 2009
http://bowr.de/blog/?p=49
Dominikus Baur University of Munich
Pulling strings from a tangle
149
Using Visualizations for Music Discovery - ISMIR 2009
Trees
150
Genre Map derived from Last.fm tags
Lamere
Using Visualizations for Music Discovery - ISMIR 2009
Trees
151
Visualizing and exploring personal music libraries
Torrens, Hertzog,Arcos
Using Visualizations for Music Discovery - ISMIR 2009
Visualizations : Dave’s Music Library Contents
Treemap of a personal music collection
152
Using Visualizations for Music Discovery - ISMIR 2009
Mirrored Radio dial - by thomwatson
Interactive
153
Using Visualizations for Music Discovery - ISMIR 2009
SongExplorer: Exploring Large Musical Databases Using a Tabletop Interface
Carles F. Julià - Music Technology Group - Universitat Pompeu Fabra
SongExplorer
154
Using Visualizations for Music Discovery - ISMIR 2009
SongExplorer: Exploring Large Musical Databases Using a Tabletop Interface
Carles F. Julià - Music Technology Group - Universitat Pompeu Fabra
SongExplorer
154
Using Visualizations for Music Discovery - ISMIR 2009
http://fidgt.com/visualize
Fidgt:Visualize
155
Using Visualizations for Music Discovery - ISMIR 2009
http://fidgt.com/visualize
Fidgt:Visualize
155
Using Visualizations for Music Discovery - ISMIR 2009
http://waks.nl/nl/musicalnodes
Lisa Dalhuijsen, Lieven vanVelthoven -LIACS, Leiden University
MusicNodes
156
Using Visualizations for Music Discovery - ISMIR 2009
http://waks.nl/nl/musicalnodes
Lisa Dalhuijsen, Lieven vanVelthoven -LIACS, Leiden University
MusicNodes
156
Using Visualizations for Music Discovery - ISMIR 2009
social.zune.net
Zune MixView
157
Using Visualizations for Music Discovery - ISMIR 2009
social.zune.net
Zune MixView
157
Using Visualizations for Music Discovery - ISMIR 2009
http://staff.aist.go.jp/m.goto/Musicream/ Goto and Masataka Goto.
Musicream
158
Using Visualizations for Music Discovery - ISMIR 2009
http://staff.aist.go.jp/m.goto/Musicream/ Goto and Masataka Goto.
Musicream
158
Using Visualizations for Music Discovery - ISMIR 2009
Elias Pampalk & Masataka Goto, "MusicRainbow:A New User Interface to Discover Artists Using
Audio-based Similarity and Web-based Labeling
MusicRainbow
159
Using Visualizations for Music Discovery - ISMIR 2009
Elias Pampalk & Masataka Goto,
"MusicSun:A New Approach to Artist Recommendation"
MusicSun
160
Using Visualizations for Music Discovery - ISMIR 2009
EN Music Explorer
161
Using Visualizations for Music Discovery - ISMIR 2009
EN Music Explorer
161
Using Visualizations for Music Discovery - ISMIR 2009
Here be Dragons
162
Using Visualizations for Music Discovery - ISMIR 2009
Here be Dragons
162
Using Visualizations for Music Discovery - ISMIR 2009
Here be Dragons
162
Visualization Proponents
(TheYale Club)
F.J.Anscombe
(Anscombe’s Quartet)
John Tukey
(stem/leaf plots)
Edward Tufte
(Sparklines)
http://en.wikipedia.org/wiki/F.J._Anscombe
http://en.wikipedia.org/wiki/John_Tukey
http://en.wikipedia.org/wiki/Leland_Wilkinson
http://en.wikipedia.org/wiki/Edward_Tufte
0 | 0000111111222338
2 | 07
4 | 5
6 | 8
8 | 4
10 | 5
12 |
14 |
16 | 0
Leland Wilkinsen
(Cluster heat maps)
163
InteractiveVisualization Proponents
Ben Shneiderman
(Treemaps, NSD
diagrams)
Steve Deunes
(& entire NYT
Graphics
Department)
Ben Fry & Casey
Reas
(Processing)
164
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
Resources
165
Using Visualizations for Music Discovery - ISMIR 2009
VisualizingMusic.com
Visualizing Music
166
musicviz.googlepages.com/home
Other Tools
167
Other Analytic* Tools
• Matlab
• Dimensionality Reduction Toolbox
• (missing good network viz/analysis option)
• Extensive Interactive Pub-quality plotting environment
• Pajek (standalone NetworkVisualization/Analysis Platform)
• IVC Software Framework (research platform for analysis and viz).
• R
• Excellent network viz/analysis package “network”
• PCA/MDS (base library), ISOMap {vegan}
(no LLE,T-SNE yet)
• Pub quality plotting environments (default, ggplot2, iplots)
http://ict.ewi.tudelft.nl/~lvandermaaten/Matlab_Toolbox_for_Dimensionality_Reduction.html
http://cran.r-project.org/web/packages/network/index.html
http://cran.r-project.org/web/packages/vegan/index.html
http://vlado.fmf.uni-lj.si/pub/networks/pajek/
http://iv.slis.indiana.edu/sw/index.html
*for publications
168
Other “Generic”Viz Tools
• Prefuse (java/as3)
• Processing*
• Gephi
• TouchGraph
• Gnuplot
• matplotlib (python)
http://www.prefuse.org/
http://processing.org/
http://gephi.org/
http://www.touchgraph.com/navigator.html
http://www.gnuplot.info/
http://matplotlib.sourceforge.net/
169
Conclusions
• Music visualizations reflect ways of
understanding music.
• Music understanding draws from
many different types of information
(musicological, acoustic, social,
bibliographic, etc.)
• Music visualizations can help to
express this understanding in a
way that can help aid navigation
and discovery.
Questions?
171

More Related Content

What's hot

ケプストラム正則化NTFによるステレオチャネル楽曲音源分離
ケプストラム正則化NTFによるステレオチャネル楽曲音源分離ケプストラム正則化NTFによるステレオチャネル楽曲音源分離
ケプストラム正則化NTFによるステレオチャネル楽曲音源分離NU_I_TODALAB
 
スペクトログラム無矛盾性に基づく独立低ランク行列分析
スペクトログラム無矛盾性に基づく独立低ランク行列分析スペクトログラム無矛盾性に基づく独立低ランク行列分析
スペクトログラム無矛盾性に基づく独立低ランク行列分析Kitamura Laboratory
 
付け爪型センサを用いたケプストラム分析及びMUSIC法に基づく心拍推定
付け爪型センサを用いたケプストラム分析及びMUSIC法に基づく心拍推定付け爪型センサを用いたケプストラム分析及びMUSIC法に基づく心拍推定
付け爪型センサを用いたケプストラム分析及びMUSIC法に基づく心拍推定Kitamura Laboratory
 
時間領域低ランクスペクトログラム近似法に基づくマスキング音声の欠損成分復元
時間領域低ランクスペクトログラム近似法に基づくマスキング音声の欠損成分復元時間領域低ランクスペクトログラム近似法に基づくマスキング音声の欠損成分復元
時間領域低ランクスペクトログラム近似法に基づくマスキング音声の欠損成分復元NU_I_TODALAB
 
Assesment of hearing
Assesment of hearingAssesment of hearing
Assesment of hearingRam Raju
 
独立成分分析に基づく信号源分離精度の予測
独立成分分析に基づく信号源分離精度の予測独立成分分析に基づく信号源分離精度の予測
独立成分分析に基づく信号源分離精度の予測Kitamura Laboratory
 
Hearing aid anatomy
Hearing aid anatomyHearing aid anatomy
Hearing aid anatomyDerek Stiles
 
The purpose of music
The purpose of musicThe purpose of music
The purpose of musicFrank Calberg
 
バイアス付きMMSE-STSA推定器の理論解析およびミュージカルノイズフリー雑音抑圧への拡張
バイアス付きMMSE-STSA推定器の理論解析およびミュージカルノイズフリー雑音抑圧への拡張バイアス付きMMSE-STSA推定器の理論解析およびミュージカルノイズフリー雑音抑圧への拡張
バイアス付きMMSE-STSA推定器の理論解析およびミュージカルノイズフリー雑音抑圧への拡張奈良先端大 情報科学研究科
 

What's hot (10)

ケプストラム正則化NTFによるステレオチャネル楽曲音源分離
ケプストラム正則化NTFによるステレオチャネル楽曲音源分離ケプストラム正則化NTFによるステレオチャネル楽曲音源分離
ケプストラム正則化NTFによるステレオチャネル楽曲音源分離
 
スペクトログラム無矛盾性に基づく独立低ランク行列分析
スペクトログラム無矛盾性に基づく独立低ランク行列分析スペクトログラム無矛盾性に基づく独立低ランク行列分析
スペクトログラム無矛盾性に基づく独立低ランク行列分析
 
付け爪型センサを用いたケプストラム分析及びMUSIC法に基づく心拍推定
付け爪型センサを用いたケプストラム分析及びMUSIC法に基づく心拍推定付け爪型センサを用いたケプストラム分析及びMUSIC法に基づく心拍推定
付け爪型センサを用いたケプストラム分析及びMUSIC法に基づく心拍推定
 
時間領域低ランクスペクトログラム近似法に基づくマスキング音声の欠損成分復元
時間領域低ランクスペクトログラム近似法に基づくマスキング音声の欠損成分復元時間領域低ランクスペクトログラム近似法に基づくマスキング音声の欠損成分復元
時間領域低ランクスペクトログラム近似法に基づくマスキング音声の欠損成分復元
 
Assesment of hearing
Assesment of hearingAssesment of hearing
Assesment of hearing
 
独立成分分析に基づく信号源分離精度の予測
独立成分分析に基づく信号源分離精度の予測独立成分分析に基づく信号源分離精度の予測
独立成分分析に基づく信号源分離精度の予測
 
Hearing aid anatomy
Hearing aid anatomyHearing aid anatomy
Hearing aid anatomy
 
The purpose of music
The purpose of musicThe purpose of music
The purpose of music
 
バイアス付きMMSE-STSA推定器の理論解析およびミュージカルノイズフリー雑音抑圧への拡張
バイアス付きMMSE-STSA推定器の理論解析およびミュージカルノイズフリー雑音抑圧への拡張バイアス付きMMSE-STSA推定器の理論解析およびミュージカルノイズフリー雑音抑圧への拡張
バイアス付きMMSE-STSA推定器の理論解析およびミュージカルノイズフリー雑音抑圧への拡張
 
Early Hearing Detection: Collaboration and Outcomes
Early Hearing Detection: Collaboration and OutcomesEarly Hearing Detection: Collaboration and Outcomes
Early Hearing Detection: Collaboration and Outcomes
 

More from Paul Lamere

How We Listen to Music - SXSW 2015
How We Listen to Music - SXSW 2015How We Listen to Music - SXSW 2015
How We Listen to Music - SXSW 2015Paul Lamere
 
Sxsw 2015 - How we listen to music
Sxsw 2015 - How we listen to musicSxsw 2015 - How we listen to music
Sxsw 2015 - How we listen to musicPaul Lamere
 
Beyond the Play Button - The future of listening
Beyond the Play Button - The future of listeningBeyond the Play Button - The future of listening
Beyond the Play Button - The future of listeningPaul Lamere
 
Echo nest-api-boston-2012
Echo nest-api-boston-2012Echo nest-api-boston-2012
Echo nest-api-boston-2012Paul Lamere
 
I've got 10 million songs in my pocket. Now what?
I've got 10 million songs in my pocket. Now what? I've got 10 million songs in my pocket. Now what?
I've got 10 million songs in my pocket. Now what? Paul Lamere
 
Data Mining Music
Data Mining MusicData Mining Music
Data Mining MusicPaul Lamere
 
Finding Music With Pictures: Using Visualization for Discovery
Finding Music With Pictures: Using Visualization for DiscoveryFinding Music With Pictures: Using Visualization for Discovery
Finding Music With Pictures: Using Visualization for DiscoveryPaul Lamere
 
The Echo Nest workshop for Boston Music Hack Day
The Echo Nest workshop for Boston Music Hack DayThe Echo Nest workshop for Boston Music Hack Day
The Echo Nest workshop for Boston Music Hack DayPaul Lamere
 
Help! My iPod thinks I'm emo.
Help! My iPod thinks I'm emo.Help! My iPod thinks I'm emo.
Help! My iPod thinks I'm emo.Paul Lamere
 
Social Tags and Music Information Retrieval (Part I)
Social Tags and Music Information Retrieval (Part I)Social Tags and Music Information Retrieval (Part I)
Social Tags and Music Information Retrieval (Part I)Paul Lamere
 
Social Tags and Music Information Retrieval (Part II)
Social Tags and Music Information Retrieval (Part II)Social Tags and Music Information Retrieval (Part II)
Social Tags and Music Information Retrieval (Part II)Paul Lamere
 

More from Paul Lamere (11)

How We Listen to Music - SXSW 2015
How We Listen to Music - SXSW 2015How We Listen to Music - SXSW 2015
How We Listen to Music - SXSW 2015
 
Sxsw 2015 - How we listen to music
Sxsw 2015 - How we listen to musicSxsw 2015 - How we listen to music
Sxsw 2015 - How we listen to music
 
Beyond the Play Button - The future of listening
Beyond the Play Button - The future of listeningBeyond the Play Button - The future of listening
Beyond the Play Button - The future of listening
 
Echo nest-api-boston-2012
Echo nest-api-boston-2012Echo nest-api-boston-2012
Echo nest-api-boston-2012
 
I've got 10 million songs in my pocket. Now what?
I've got 10 million songs in my pocket. Now what? I've got 10 million songs in my pocket. Now what?
I've got 10 million songs in my pocket. Now what?
 
Data Mining Music
Data Mining MusicData Mining Music
Data Mining Music
 
Finding Music With Pictures: Using Visualization for Discovery
Finding Music With Pictures: Using Visualization for DiscoveryFinding Music With Pictures: Using Visualization for Discovery
Finding Music With Pictures: Using Visualization for Discovery
 
The Echo Nest workshop for Boston Music Hack Day
The Echo Nest workshop for Boston Music Hack DayThe Echo Nest workshop for Boston Music Hack Day
The Echo Nest workshop for Boston Music Hack Day
 
Help! My iPod thinks I'm emo.
Help! My iPod thinks I'm emo.Help! My iPod thinks I'm emo.
Help! My iPod thinks I'm emo.
 
Social Tags and Music Information Retrieval (Part I)
Social Tags and Music Information Retrieval (Part I)Social Tags and Music Information Retrieval (Part I)
Social Tags and Music Information Retrieval (Part I)
 
Social Tags and Music Information Retrieval (Part II)
Social Tags and Music Information Retrieval (Part II)Social Tags and Music Information Retrieval (Part II)
Social Tags and Music Information Retrieval (Part II)
 

Using Visualizations for Music Discovery

  • 1. UsingVisualizations for Music Discovery ISMIR 2009 Justin Donaldson Paul Lamere
  • 2. UsingVisualizations for Music Discovery ISMIR 2009 Justin Donaldson Paul Lamere Or ...
  • 3. 85 ways to visualize your music 2
  • 4. Speakers 3 Paul Lamere is the Director of Developer Community at The Echo Nest, a research-focused music intelligence startup that provides music information services to developers and partners through a data mining and machine listening platform. Paul is especially interested in hybrid music recommenders and using visualizations to aid music discovery. Paul also authors 'Music Machinery' a blog focusing on music discovery and recommendation. Justin Donaldson is a PhD candidate at Indiana University School of Informatics, as well as a regular research intern at Strands, Inc. Justin is interested with the analyses and visualizations of social sources of data, such as those that are generated from playlists, blogs, and bookmarks.
  • 5. Outline of the talk • Why visualize • Some history • Core concepts • Survey of MusicVisualizations • Tools / Resources • Conclusions 4
  • 6. Music Discovery Challenge • Millions of songs • We need tools • Recommendation • Playlisting •Visualization 5
  • 8. WhyVisualize? 7 Visualizations can engage our pattern-matching brain
  • 9. WhyVisualize? • Anscombe’s “Quartet” • Each distribution has equivalent summary statistics • Means, correlations, variance: Sometimes one number summaries are deceiving. property value x mean 9 x var 10 y mean 7.5 y var 3.75 x y cor. 0.816 linear reg. y = 3 + .5x http://en.wikipedia.org/wiki/Anscombe%27s_quartet 8
  • 10. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Semantic Pattern Mappings 9
  • 11. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Semantic Pattern Mappings 10
  • 12. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Basic Popout Channels 11 Color
  • 13. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Basic Popout Channels 12 Size
  • 14. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Basic Popout Channels 13 Shape
  • 15. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Basic Popout Channels 14 Motion
  • 16. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Basic Popout Channels 14 Motion
  • 17. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Basic Popout Channels 14 Motion
  • 18. Using Visualizations for Music Discovery - ISMIR 2009 Colin Ware -Visual Thinking for Design Basic Popout Channels 15 Spatial Grouping
  • 19. Using Visualizations for Music Discovery - ISMIR 2009 Flickr photo by Stéfan Overloading the channels 16
  • 20. Using Visualizations for Music Discovery - ISMIR 2009 Visualization Name Visualization ofVisualizations 17
  • 21. Using Visualizations for Music Discovery - ISMIR 2009 Tree Map Graph Time Series Scatter Hybrid Connections Interactive 18 Types of visualizations
  • 22. Using Visualizations for Music Discovery - ISMIR 2009 Objects to be discovered 19 human machine Artists 23 37 Albums 0 6 Tracks 3 17 Samples 3 0 Playlists 0 0 Ordered Tracks 0 9 Concerts 0 0 Blogs 0 0 Music Videos 0 0 human machine Genre 18 14 Users 0 4 Radio 0 2 DJs 0 1 Games 0 0 Commu nities 0 0 Venues 0 0 Taste Trends 2 2 Labels 1 0 26 Human rendered 55 Machine rendered visualizations
  • 23. Using Visualizations for Music Discovery - ISMIR 2009 Features Used 20 Content Beat, tempo, timbre, harmonic content, lyrics Cultural web and playlist co-occurrence, purchase/play history, social tags, genre, text aura, expert opinion, appearance, popularity Relational member of, collaborated on, supporting musician for, performed as, parent, sibling, married, same label, produced Mood happy, sad, angry, relaxed, autumnal Location city, country, lat/long, Time the hits from the 60s, 70s 80s and today, recent play history Misc IQ,Temperature, Name morphology
  • 24. Using Visualizations for Music Discovery - ISMIR 2009 A scene from High Fidelity More features 21
  • 25. Using Visualizations for Music Discovery - ISMIR 2009 A scene from High Fidelity More features 21
  • 27. Human rendered visualizations Some common techniques • The Flow • The Map • The Tree • The Graph 23
  • 28. • History • Influences • Similarity • Popularity 24 Human rendered visualizations Some common techniques
  • 29. • History • Influences • Similarity • Popularity 24 Human rendered visualizations Some common techniques
  • 30. • History • Influences • Similarity • Popularity 24 Human rendered visualizations Some common techniques
  • 31. Genealogy of Pop and Rock Music 25 Genealogy of Pop and Rock Music - Reebee Garofalo
  • 32. Genealogy of Pop and Rock Music 25 Genealogy of Pop and Rock Music - Reebee Garofalo
  • 33. Genealogy of Pop and Rock Music 25 Genealogy of Pop and Rock Music - Reebee Garofalo
  • 34. Ishkur’s Guide to Electronic Dance Music 26
  • 35. Ishkur’s Guide to Electronic Dance Music 26
  • 36. Music History as a London Tube Map 27 Dorian Lynskey - The Guardian The Guardian | Friday February 3 2006 9 The Guardian | Friday February 3 2006 verstory oing underground d we chart the branches and connections of 100 years of music using the London Underground map? Dorian Lynskey explains how a box of coloured crayons and a lot of swearing helped like a deeply implausible to plot the history of 20th music on the London ound map devised by eck in 1933. Artist Simon n transformed the tube nstellation of famous 1992 work The Great Bear, have to make them all link all, a tall order to find a s also a comedian. But for ork every interchange had n the context of musical nlikely prospect. ut with a packet of coloured sheets of A4 taped to- big box of doubt, but the racter of each line quickly certain genre. Pop inter- erything else, so that had to Line; classical music for the upies its own sphere, t perfect for the Docklands y. There were a couple of ut by the end of one after- signed genres to almost all thrashed out most of the ctions. The key stations nt to the most eclectic ecessarily the most impor- les may be more signifi- k but even their most de- st admit that they never . m thus in place, the next s were devoted to writing ibbling them out (sorry, and Lynyrd Skynyrd), ago- ertain omissions, asking ic critic Tom Service for in- with the DLR, and swear- e bit. Amazingly, there mitous blind alleys. It just ake sense. r as possible to be objec- nds I cannot stand are in ome that I love dearly ollowed chronology wher- of the line allowed it. Each epresents a sub-genre: rock to grunge and psychedelia es South-West London; rges, north of Camden, into d New York rap. If I was re- e band name echoed the on name: Highbury & Is- me Sly & the Family Stone. f course, will find flaws. ences are so labyrinthine ple equation will be imper- or example, does pop stop n? How can you draw a de- tween soul and funk? These that have plagued record tors for decades and they’re be solved here. But I think hoices are justifiable given ns of the form. ple will quibble with omis- hame, for example, that e constantly runs in tan- her the District or Metro- thus leaving no room for such as Kylie Minogue and Boys. I should also point eep my head from explod- the remit to western, pre- nglo-American music. e those changes necessi- don Underground’s under- nsitivity to explosive refer- derci, Massive Attack. For they also took exception to er Ol’ Dirty Bastard. not some definitive history an experiment to see if one work can be overlaid on a ifferent one. The elegance Harry Beck’s design – its of bustling intersections, butaries, long, slanting tan- upt dead ends, all sucked turned wine bottle of the eems to spark other con- appeal to the brain’s innate terning and structure. Plus y piece of music journalism coloured crayons should u like it. you think of the map at an.co.uk/arts at umshop.co.uk ks to Chris Townsend at r London and Andrew don Underground.
  • 37. Music History as a London Tube Map 27 Dorian Lynskey - The Guardian The Guardian | Friday February 3 2006 9 The Guardian | Friday February 3 2006 verstory oing underground d we chart the branches and connections of 100 years of music using the London Underground map? Dorian Lynskey explains how a box of coloured crayons and a lot of swearing helped like a deeply implausible to plot the history of 20th music on the London ound map devised by eck in 1933. Artist Simon n transformed the tube nstellation of famous 1992 work The Great Bear, have to make them all link all, a tall order to find a s also a comedian. But for ork every interchange had n the context of musical nlikely prospect. ut with a packet of coloured sheets of A4 taped to- big box of doubt, but the racter of each line quickly certain genre. Pop inter- erything else, so that had to Line; classical music for the upies its own sphere, t perfect for the Docklands y. There were a couple of ut by the end of one after- signed genres to almost all thrashed out most of the ctions. The key stations nt to the most eclectic ecessarily the most impor- les may be more signifi- k but even their most de- st admit that they never . m thus in place, the next s were devoted to writing ibbling them out (sorry, and Lynyrd Skynyrd), ago- ertain omissions, asking ic critic Tom Service for in- with the DLR, and swear- e bit. Amazingly, there mitous blind alleys. It just ake sense. r as possible to be objec- nds I cannot stand are in ome that I love dearly ollowed chronology wher- of the line allowed it. Each epresents a sub-genre: rock to grunge and psychedelia es South-West London; rges, north of Camden, into d New York rap. If I was re- e band name echoed the on name: Highbury & Is- me Sly & the Family Stone. f course, will find flaws. ences are so labyrinthine ple equation will be imper- or example, does pop stop n? How can you draw a de- tween soul and funk? These that have plagued record tors for decades and they’re be solved here. But I think hoices are justifiable given ns of the form. ple will quibble with omis- hame, for example, that e constantly runs in tan- her the District or Metro- thus leaving no room for such as Kylie Minogue and Boys. I should also point eep my head from explod- the remit to western, pre- nglo-American music. e those changes necessi- don Underground’s under- nsitivity to explosive refer- derci, Massive Attack. For they also took exception to er Ol’ Dirty Bastard. not some definitive history an experiment to see if one work can be overlaid on a ifferent one. The elegance Harry Beck’s design – its of bustling intersections, butaries, long, slanting tan- upt dead ends, all sucked turned wine bottle of the eems to spark other con- appeal to the brain’s innate terning and structure. Plus y piece of music journalism coloured crayons should u like it. you think of the map at an.co.uk/arts at umshop.co.uk ks to Chris Townsend at r London and Andrew don Underground.
  • 43. Visualizations as a graph 29 www.flickr.com/photos/angrypirate/364960689/
  • 44. Visualizations as a graph 29 www.flickr.com/photos/angrypirate/364960689/
  • 45. Visualizations as a graph 29 www.flickr.com/photos/angrypirate/364960689/
  • 46. Visualizations as a graph 29 www.flickr.com/photos/angrypirate/364960689/
  • 47. Visualizations as a graph 29 www.flickr.com/photos/angrypirate/364960689/
  • 48. Using Visualizations for Music Discovery - ISMIR 2009 Fabian Schneidmadel - http://www.firutin.de/blog/ Visual Music Guiden Images (cc) Fabian Schneidmadel 30
  • 49. Using Visualizations for Music Discovery - ISMIR 2009 Fabian Schneidmadel - http://www.firutin.de/blog/ Visual Music Guiden Images (cc) Fabian Schneidmadel 30
  • 54. Handmade Genre Maps and Trees 32
  • 55. Handmade Genre Maps and Trees 32
  • 56. Handmade Genre Maps and Trees 32
  • 57. Handmade Genre Maps and Trees 32
  • 58. Handmade Genre Maps and Trees 32
  • 59. Handmade Genre Maps and Trees 32
  • 60. Handmade Genre Maps and Trees 32
  • 61. Handmade Genre Maps and Trees 32
  • 62. Handmade Genre Maps and Trees 32
  • 63. Other types of visualizations 33
  • 64. Other types of visualizations 33
  • 65. Other types of visualizations 33
  • 66. Other types of visualizations 33
  • 67. Other types of visualizations 33
  • 68. Other types of visualizations 34
  • 69. Other types of visualizations 34
  • 70. Artist as a function of temperature 35
  • 71. Using Visualizations for Music Discovery - ISMIR 2009 Stephan Bauman -Senior Researcher - German Research Center for AI ISMIR Tutorial 2005 36
  • 72. Using Visualizations for Music Discovery - ISMIR 2009 Stephan Bauman -Senior Researcher - German Research Center for AI ISMIR Tutorial 2005 36
  • 73. Using Visualizations for Music Discovery - ISMIR 2009 Stephan Bauman -Senior Researcher - German Research Center for AI ISMIR Tutorial 2005 36
  • 74. Using Visualizations for Music Discovery - ISMIR 2009 Stephan Bauman -Senior Researcher - German Research Center for AI ISMIR Tutorial 2005 36
  • 75. Using Visualizations for Music Discovery - ISMIR 2009 Stephan Bauman -Senior Researcher - German Research Center for AI ISMIR Tutorial 2005 36
  • 77. How do computers represent music? • Score - As MIDI data • Signal - As digital wave form • Association - Through associations with tags, people, or other songs 38
  • 78. (a brief note on) Symbolic score data • Typically provides an explicit compositional structure, and sometimes even explicit performance directions. • “Easy” to analyze/extract meaningful musicological features (time signature, key, etc.) • Scores are typically not present for most music, so other forms of representation are becoming popular. • We won’t cover score based representations of music today. 39
  • 79. Acoustic Data • Easy (and even occasionally legal) to find digital wave form representations of music • New techniques, methods, and even services for music signal analysis. http://www.echonest.com/ http://www.magnatune.com/ 40
  • 80. Association Data • Relatively new form of music analysis • New methods/services as well • What connections/ associations/relationships does music share with tags, people, playlists, other music? http://manyeyes.alphaworks.ibm.com/manyeyes/visualizations/matchbox-20-song-lyrics-tag-cloud http://sixdegrees.hu/last.fm/ http://www.flickr.com/photos/scwn/153098444/ http://www.last.fm 41
  • 81. Representing “Lots of Songs” • Many different types of music- related information. • Problems of sparsity, noise, data set size. • We need a common appropriate format of representation and method of comparison. • We would like to work with matrices. Song Feature1 Feature2 “Love me do” 0.4 0.9 “Hound Dog” 3.2 1.4 “Paint it Black” 3.4 1.5 42
  • 82. Distances and Similarity • To form a two/three dimensional visualization, we will need to express a distance or dissimilarity* between the songs in vector form. • Distance must be symmetric in order to represent a metric space. • Distance must be “non-degenerate” (distance from song A to song A == 0) Song “Love me do” “Hound Dog” “Paint it Black” “Love me do” n/a 3.2 3.4 “Hound Dog” 3.2 n/a 1.5 “Paint it Black” 3.4 1.5 n/a song song B to A A to B = 43
  • 83. “Big Picture” Visualization • A couple of “simple” visualizations help to assess variance, etc. across the relevant dimensions. • Boxplots, matrix plots, image plots, etc. • Selectively normalizing data at this point to have more control, and removing obvious outlier (points/dimensions). 44
  • 84. Multidimensional Scaling • Reducing the number of dimensions of the music vector. • Similar to “rotating” the data to find large dimensions of variation/ dissimilarity in the data. Athens Barcelona Brussels Calais Cherbourg Athens 0 3313 2963 3175 3339 Barcelona 3313 0 1318 1326 1294 Brussels 2963 1318 0 204 583 Calais 3175 1326 204 0 460 Cherbourg 3339 1294 583 460 0 Cologne 2762 1498 206 409 785 http://www.mathpsyc.uni-bonn.de/doc/delbeke/delbeke.htm 45
  • 86. Demo:Acoustic FeatureVectors • Working with Magnatagatune dataset • Combination of: • Magnatune creative commons licensed songs (clips) • Tagatune project data • Echo Nest feature data • Freely available http://tagatune.org/Magnatagatune.html 47
  • 87. Demo: FeatureVector Echonest data includes: • loudness • tempo • segments with: • pitch information • timbre information http://tagatune.org/Magnatagatune.html 48
  • 88. Demo: FeatureVector Demo motivation • Genre classification has become a prominent trend in the IR community. Explore the data pertaining to such systems. • Current approaches can identify “textural” properties of acoustic sound, but can lack cultural or structural properties of music.West & Lamere note that: “...a common example might be the confusion of a classical lute timbre, with that of an acoustic guitar string that might be found in folk, pop, or rock music...” K. West and P. Lamere, “A model-based approach to constructing music similarity functions,” EURASIP Journal on Advances in Signal Processing 2007 (2007). 49
  • 89. Demo: FeatureVector Demo motivation • Magnatagatune happens to have a large amount of Classical music, and Classical Lute music to boot. • We’ll try to visualize this part of the magnatagatune data set, along with folk music. 50
  • 91. Tools: FeatureVector • Download Data • Use R to process, analyze, and visualize data • Free! (Libre!) http://cran.r-project.org/ http://www.cyclismo.org/tutorial/R/ http://cran.r-project.org/doc/manuals/R-intro.html 52
  • 93. Demo: FeatureVector Basic Echo Nest Features • (overall) loudness • tempo • pitch means (12) * • pitch stdev (12) * • timbre means (12) * • timber stdev (12) * * weighted by segment onset loudness 54
  • 94. Boxplots Boxplots show the five- number summaries of a distribution: • sample maximum • upper quartile • median • lower quartile • sample minimum • (outliers) 55
  • 95. Boxplots Boxplots show the five- number summaries of a distribution: • sample maximum • upper quartile • median • lower quartile • sample minimum • (outliers) 56
  • 96. Demo: FeatureVector Raw Feature Boxplots... • Are these all normal distributions? • Are all of these weighted evenly? • Scaling becomes important... why? 57
  • 97. Demo: FeatureVector Looking at scaled summary statistics These are the ‘expected’ values for any song, so they’re ‘normal’ for a song. We want to enhance relationships between songs that differ from the norm. 58
  • 98. Demo: FeatureVector Weighted Feature Boxplots... • Standard normalized data. All means are 0, all standard deviations are 1. • We can find ‘interesting’ relationships between songs better this way. Distances can be understood in terms of standard deviations from the mean. • However... 24 pitch and 24 timbre features, vs. just 1 tempo... 59
  • 99. Demo: FeatureVector Principle Component Analysis (PCA) of timbre and pitch information reveal a high degree of covariance. 60
  • 100. Demo: FeatureVector Plotting the PCA loadings show the ‘structure’ of how the pitch profiles co- vary across the music. Blue = “Classical Guitar” (almost entirely lute music) Red = “Folk” Green= “Both” (flips and rotations) 61
  • 101. Demo: FeatureVector Performing a Factor analysis shows evidence of “musicological” factors. (alternating positive/negative loadings for both means and variances, indicating certain “keys” are followed for songs) Should pitch profile distances be figured another way? Loadings: Factor1 Factor2 pitch_mean_1 -0.117 0.625 pitch_mean_2 0.576 -0.270 pitch_mean_3 -0.585 pitch_mean_4 0.677 0.126 pitch_mean_5 -0.103 -0.487 pitch_mean_6 0.473 pitch_mean_7 0.332 -0.504 pitch_mean_8 -0.512 0.217 pitch_mean_9 0.527 -0.124 pitch_mean_10 -0.520 -0.274 pitch_mean_11 0.475 0.452 pitch_mean_12 0.216 -0.359 pitch_var_1 -0.318 0.689 pitch_var_2 0.638 -0.377 pitch_var_3 -0.661 0.100 pitch_var_4 0.695 0.289 pitch_var_5 -0.169 -0.536 pitch_var_6 0.658 pitch_var_7 0.363 -0.550 pitch_var_8 -0.660 0.315 pitch_var_9 0.669 -0.104 pitch_var_10 -0.620 -0.213 pitch_var_11 0.385 0.593 pitch_var_12 0.124 -0.399 62
  • 102. Demo: FeatureVector Timbre features clearly split the different tags we were concerned with. Blue = “Classical Guitar” Red = “Folk” Green= “Both” 63
  • 103. Demo: FeatureVector Timbral factors are a lot more straightforward. Certain songs have a lot of timbral variance. Other songs simply have strong overall timbral means. Loadings: Factor1 Factor2 timbre_mean_1 0.987 timbre_mean_2 0.489 timbre_mean_3 0.151 0.728 timbre_mean_4 -0.139 0.349 timbre_mean_5 -0.570 timbre_mean_6 timbre_mean_7 -0.126 0.248 timbre_mean_8 0.175 -0.556 timbre_mean_9 0.122 0.589 timbre_mean_10 0.257 -0.461 timbre_mean_11 -0.363 -0.238 timbre_mean_12 timbre_var_1 0.582 0.614 timbre_var_2 0.681 -0.286 timbre_var_3 0.822 -0.114 timbre_var_4 0.680 -0.387 timbre_var_5 0.779 timbre_var_6 0.241 -0.690 timbre_var_7 0.810 timbre_var_8 0.712 timbre_var_9 0.792 timbre_var_10 0.591 -0.358 timbre_var_11 0.667 -0.140 timbre_var_12 0.736 64
  • 105. Demo:Association Data • Magnatagatune has ‘tag’ data for songs • Tags only exist if two people used the same tag for the same song • 188 total tags clip_id no.voice singer duet plucking hard.rock world bongos harpsichord female.singing classical 1 2 0 0 0 0 0 0 0 0 0 0 2 6 0 0 0 0 0 0 0 1 0 0 3 10 0 0 0 1 0 0 0 0 0 0 4 11 0 0 0 0 0 0 0 0 0 0 5 12 0 1 0 0 0 0 1 0 0 1 6 14 0 0 0 0 0 0 0 0 0 0 66
  • 106. Demo:Association Data • Tags are distributed by a power-law • It’s not as meaningful to share a “guitar” tag. A “soprano” tag is far more informative. • We need to form a comparison of tags based on the weighted relevance of the terms. guitar classical slow techno strings drums electronic rock 4852 4272 3547 2954 2729 2598 2519 2371 fast piano ambient beat violin vocal synth female 2306 2056 1956 1906 1826 1729 1717 1474 indian opera male singing vocals no.vocals harpsichord loud 1395 1296 1279 1211 1184 1158 1093 1086 67
  • 107. Demo:Association Data • TF/IDF (term frequency/document frequency) • n(i,j) = number of times a term i appears in a ‘document’ j. • n(k,j) = total number of terms in ‘document’ j. • |D| = total number of ‘documents’ • {d:...} = total documents containing i • documents = tags for song http://en.wikipedia.org/wiki/Tf%E2%80%93idf 68
  • 108. Demo:Association Data • Forming a similarity/dissimilarity between weighted tags commonly involves finding the angle between two vectors. • Cosine dissimilarity is one (of many) ways of expressing a distance between vectors. • A, B = vectors of attributes • ||A||, ||B|| = magnitudes of vectors http://en.wikipedia.org/wiki/Cosine_similarity 69
  • 109. Demo:Association Data Cosine similarity of tags also (not surprisingly) clearly separates the two tags 70
  • 111. Demo: FeatureVector Timbre features clearly split the different tags we were concerned with. Blue = “Classical Guitar” Red = “Folk” Green= “Both” 72
  • 112. Conclusions (demo) • Visualizations can help understand “big picture” relationships among songs, and support that the relationships are meaningful. (left plot) • Visualizations can help understand “mismatches” in modeling/clustering/ classification approaches. (x2) • Visualizations can help detect noise (x1) and outliers. (x3, x4) • Visualizations can help find a way to improve the underlying model.
  • 113. Opportunity: Hybrid Maps Mixing information from both timbre & term spaces has a great opportunity for forming a ‘better’ representation of genre + = 74
  • 114. “Reading” Conventional Dimensionality Reductions • Large dissimilarity is preserved over small dissimilarity. • Try to understand the plot in terms of its two biggest orthogonal distances. If those don’t make sense, then the plot is suspect. • Next, look for local structure, clustering, etc. 75
  • 116. Other Techniques • Locally Linear Embedding : Only consider ‘close’ points in distance matrix. Good at recovering from “kinks”,“folds”, and “spirals” in data (chroma) • Isomap : Establish geodesic “neighborhood” distances using Dijkstra’s algorithm, and solve with mds. • T-SNE : Convert neighbor distances to conditional probabilities. Perform gradient descent on Kullback-Leibler Divergence. Good at recovering clusters at different scales. 77
  • 117. Other Techniques • Many advanced techniques sacrifice “large scale” dissimilarities in order to preserve/enhance “local” features. • Sometimes large scale information is useful (to determine if scales are wrong, or if there are significant outliers) • Interpretation of advanced techniques is much more complicated. (What are the loadings, factors, of features etc.?) • It can be harder to use advanced dimensionality reduction techniques to inform classifier design. (observations) 78
  • 118. Using Visualizations for Music Discovery - ISMIR 2009 The Goal - build an artist graph suitable for music exploration Building an artist map 79 Problem:The artist space is very complex
  • 119. Using Visualizations for Music Discovery - ISMIR 2009 The Data 80 • Crawled The Echo Nest API for: • Artist names • Artist hotness • Artist familiarity • 15 most similar artists • 70,000 artists • 250,000 artist-artist links • Available at musicviz.googlepages.com
  • 120. Making a good graph • Minimize edge crossings, bends and curves • Maximize angular resolution, symmetry 81 Good Bad Bad Good Cognitive measurements of graph aesthetics, Ware, Purchase, Colpoys, McGill
  • 121. Using Visualizations for Music Discovery - ISMIR 2009 Graphing the Beatles Neighborhood digraph { "The Beatles" -> "Electric Light Orchestra"; "The Beatles" -> "The Rutles"; "The Beatles" -> "The Dave Clark Five"; "The Beatles" -> "The Tremeloes"; "The Beatles" -> "Manfred Mann"; "The Beatles" -> "The Lovin' Spoonful"; "The Beatles" -> "The Rolling Stones"; "The Beatles" -> "Creedence Clearwater Revival"; "The Beatles" -> "Bee Gees"; "The Beatles" -> "the Hollies"; "The Beatles" -> "Badfinger"; "The Beatles" -> "Paul McCartney"; "The Beatles" -> "Julian Lennon"; "The Beatles" -> "George Harrison"; "The Beatles" -> "John Lennon"; } 82 http://www.graphviz.org/
  • 122. Using Visualizations for Music Discovery - ISMIR 2009 Already interconnections start to overwhelm Artist map - level 1 out 83
  • 123. Using Visualizations for Music Discovery - ISMIR 2009 Artist map - level 1 in/out 84
  • 124. Using Visualizations for Music Discovery - ISMIR 2009 Artist map - level 1 in/out 84
  • 125. Using Visualizations for Music Discovery - ISMIR 2009 Artist map - level 1 in/out 84
  • 126. Using Visualizations for Music Discovery - ISMIR 2009 Artist map - level 1 in/out 84
  • 127. Using Visualizations for Music Discovery - ISMIR 2009 Artist map - level 1 in/out 84
  • 128. Using Visualizations for Music Discovery - ISMIR 2009 Artist map - level 1 in/out 84
  • 129. Using Visualizations for Music Discovery - ISMIR 2009 Artist map - level 1 in/out 84
  • 130. Using Visualizations for Music Discovery - ISMIR 2009 116 artists and 665 edges.We are totally overwhelmed. Artist map - level 2 85
  • 131. Using Visualizations for Music Discovery - ISMIR 2009 116 artists and 665 edges.We are totally overwhelmed. Artist map - level 2 85
  • 132. Using Visualizations for Music Discovery - ISMIR 2009 Eliminate the edges and use spring force embedding to position the artists Simplifying the graph 86
  • 133. Using Visualizations for Music Discovery - ISMIR 2009 Eliminate the edges and use spring force embedding to position the artists Simplifying the graph 86
  • 134. Using Visualizations for Music Discovery - ISMIR 2009 Attach artists to only one most similar artist Reducing the edges 87
  • 135. Using Visualizations for Music Discovery - ISMIR 2009 Attach artists to only one most similar artist Reducing the edges 87
  • 136. Using Visualizations for Music Discovery - ISMIR 2009 Attach artists to only one most similar artist Reducing the edges 87
  • 137. Using Visualizations for Music Discovery - ISMIR 2009 Attach artists to only one most similar artist Reducing the edges 87
  • 138. Using Visualizations for Music Discovery - ISMIR 2009 Attach artists to the most similar artist that has greater familiarity Putting ELO II in its place 88
  • 139. Using Visualizations for Music Discovery - ISMIR 2009 Attach artists to the most similar artist that has greater familiarity Putting ELO II in its place 88
  • 140. Using Visualizations for Music Discovery - ISMIR 2009 Attach artists to the most similar artist that has greater familiarity Putting ELO II in its place 88
  • 141. Using Visualizations for Music Discovery - ISMIR 2009 Build a minimum spanning tree from the graph Making the graph connected 89
  • 142. Using Visualizations for Music Discovery - ISMIR 2009 Build a minimum spanning tree from the graph Making the graph connected 89
  • 143. Using Visualizations for Music Discovery - ISMIR 2009 Build a minimum spanning tree from the graph Making the graph connected 89
  • 144. Using Visualizations for Music Discovery - ISMIR 2009 Prefer connections to bands of similar familiarity Fixing the swinging blue jeans problem 90
  • 145. Using Visualizations for Music Discovery - ISMIR 2009 Miles Davis Other near-neighbor graphs 91
  • 146. Using Visualizations for Music Discovery - ISMIR 2009 Miles Davis Other near-neighbor graphs 91
  • 147. Using Visualizations for Music Discovery - ISMIR 2009 A browsing network for music discovery 2K artists 92
  • 148. Using Visualizations for Music Discovery - ISMIR 2009 A browsing network for music discovery 2K artists 92
  • 149. Using Visualizations for Music Discovery - ISMIR 2009 The Artist Graph Making the graph interactive 93
  • 150. Using Visualizations for Music Discovery - ISMIR 2009 Danny Holten, Jarke J. van Wijk Force-Directed Edge Bundling for GraphVisualization 94
  • 152. Using Visualizations for Music Discovery - ISMIR 2009 Graphs 96
  • 153. Using Visualizations for Music Discovery - ISMIR 2009 http://www.dimvision.com/musicmap/ musicmap 97
  • 154. Using Visualizations for Music Discovery - ISMIR 2009 MusicPlasma.com Music Plasma 98
  • 155. Using Visualizations for Music Discovery - ISMIR 2009 http://audiomap.tuneglue.net/ tuneglue 99
  • 156. Using Visualizations for Music Discovery - ISMIR 2009 http://lastfm.dontdrinkandroot.net/ Last.fm artist map 100
  • 157. Using Visualizations for Music Discovery - ISMIR 2009 http://lastfm.dontdrinkandroot.net/ Last.fm artist map 100
  • 158. Using Visualizations for Music Discovery - ISMIR 2009 http://rama.inescporto.pt/ RAMA - Relational Artist Maps 101 Late-Breaking Demo Session
  • 159. Using Visualizations for Music Discovery - ISMIR 2009 http://www.music-map.com/ gnod music-map 102
  • 160. Using Visualizations for Music Discovery - ISMIR 2009 http://www.musicmesh.net/ musicmesh 103
  • 162. Using Visualizations for Music Discovery - ISMIR 2009 http://www.repeatingbeats.com/thesis.php - Steve Lloyd - QMUL SoundBite for SongBird 105
  • 163. Using Visualizations for Music Discovery - ISMIR 2009 http://www.repeatingbeats.com/thesis.php - Steve Lloyd - QMUL SoundBite for SongBird 105
  • 164. Using Visualizations for Music Discovery - ISMIR 2009 http://amaznode.fladdict.net/ Amaznode 106
  • 165. Using Visualizations for Music Discovery - ISMIR 2009 http://torrens.files.wordpress.com/2009/06/ismir.pdf Torrens, Patrick Hertzog, Josep-Lluís Arcos Visualizing and Exploring Personal Libraries 107
  • 166. Using Visualizations for Music Discovery - ISMIR 2009 http://torrens.files.wordpress.com/2009/06/ismir.pdf Torrens, Patrick Hertzog, Josep-Lluís Arcos Visualizing and Exploring Personal Libraries 107
  • 167. Using Visualizations for Music Discovery - ISMIR 2009 http://torrens.files.wordpress.com/2009/06/ismir.pdf Torrens, Patrick Hertzog, Josep-Lluís Arcos Visualizing and Exploring Personal Libraries 107
  • 168. Using Visualizations for Music Discovery - ISMIR 2009 http://musicexplorerfx.citytechinc.com/ Music Explorer FX 108
  • 169. Using Visualizations for Music Discovery - ISMIR 2009 Last FM tasteOgraph http://geniol.publishpath.com/last-fm-tasteograph Social Music Discovery 109
  • 170. Using Visualizations for Music Discovery - ISMIR 2009 http://www.flokoon.com/#lastfm Floko-On 110
  • 171. Using Visualizations for Music Discovery - ISMIR 2009 http://arcus-associates.com/orpheus/ Project Orpheus 111
  • 172. Using Visualizations for Music Discovery - ISMIR 2009 http://www.sfu.ca/~jdyim/musicianMap/ Ji-DongYim Chris Shaw Lyn Bartram Musician Map: visualizing music collaborations over time 112
  • 173. Using Visualizations for Music Discovery - ISMIR 2009 http://www.identitee.com/visualizer.php lyric connections 113 music tees and tee shirts on i/denti/tee
  • 174. Using Visualizations for Music Discovery - ISMIR 2009 http://journal.webscience.org/110/ Interacting with linked data about music 114 http://journal.webscience.org/110/
  • 175. Using Visualizations for Music Discovery - ISMIR 2009 http://journal.webscience.org/110/ Interacting with linked data about music 115 http://journal.webscience.org/110/
  • 176. Using Visualizations for Music Discovery - ISMIR 2009 http://www.stanford.edu/~dgleich/demos/worldofmusic/WorldOfMusic.html David Gleich, Matt Rasmussen, Leonid Zhukov, and Kevin Lang z The World Of Music 116
  • 177. Using Visualizations for Music Discovery - ISMIR 2009 http://www.stanford.edu/~dgleich/demos/worldofmusic/WorldOfMusic.html David Gleich, Matt Rasmussen, Leonid Zhukov, and Kevin Lang z The World Of Music 116
  • 178. Using Visualizations for Music Discovery - ISMIR 2009 http://sixdegrees.hu/last.fm/index.html Last.fm Artist Map 117
  • 179. Using Visualizations for Music Discovery - ISMIR 2009 http://sixdegrees.hu/last.fm/index.html Last.fm Artist Map 117
  • 180. Visualizations of the connections amongst geographically located iTunes libraries 118 http://caleblarsen.com/projects/visualization-of-the-inherent-connections-/#0
  • 181. Visualizations of the connections amongst geographically located iTunes libraries 118 http://caleblarsen.com/projects/visualization-of-the-inherent-connections-/#0
  • 182. Using Visualizations for Music Discovery - ISMIR 2009 Maps 119
  • 183. Using Visualizations for Music Discovery - ISMIR 2009 http://playground.last.fm/iom - Islands of Music - Elias Pampalk Maps 120
  • 184. Using Visualizations for Music Discovery - ISMIR 2009 http://margrid.sness.net/ marGrid 121
  • 185. Using Visualizations for Music Discovery - ISMIR 2009 http://www.ifs.tuwien.ac.at/mir/playsom.html Vienna University of Technology PlaySOM - Intuitive access to Music Archives 122
  • 186. Using Visualizations for Music Discovery - ISMIR 2009 http://fm4.orf.at/soundpark Martin Gasser,Arthur Flexer FM4 Soundpark 123
  • 187. Using Visualizations for Music Discovery - ISMIR 2009 http://www.cp.jku.at/projects/nepTune/ nepTune 124
  • 188. Using Visualizations for Music Discovery - ISMIR 2009 http://www.cp.jku.at/projects/nepTune/ nepTune 124
  • 189. Using Visualizations for Music Discovery - ISMIR 2009 http://musicminer.sourceforge.net/ Databionic Music Miner 125
  • 190. Using Visualizations for Music Discovery - ISMIR 2009 http://musicminer.sourceforge.net/ Databionic Music Miner 125
  • 191. Using Visualizations for Music Discovery - ISMIR 2009 Presented at ISMIR 2009 by Dominik Lübbers soniXplorer 126
  • 192. Using Visualizations for Music Discovery - ISMIR 2009 Presented at ISMIR 2009 by Dominik Lübbers soniXplorer 126
  • 193. Using Visualizations for Music Discovery - ISMIR 2009 Presented at ISMIR 2009 by Dominik Lübbers soniXplorer 126
  • 194. Using Visualizations for Music Discovery - ISMIR 2009 Presented at ISMIR 2009 by Dominik Lübbers soniXplorer 126
  • 195. Using Visualizations for Music Discovery - ISMIR 2009 http://www.mhashup.com/ mHashup 127
  • 196. Using Visualizations for Music Discovery - ISMIR 2009 http://www.mhashup.com/ mHashup 127
  • 197. Using Visualizations for Music Discovery - ISMIR 2009 ISMIR 2008 - Oscar Celma GeoMuzik 128
  • 198. Using Visualizations for Music Discovery - ISMIR 2009 http://www.cs.kuleuven.be/~sten/lastonamfm Last on AM/FM 129
  • 199. Using Visualizations for Music Discovery - ISMIR 2009 http://playground.last.fm/demo/tagstube Last.fm tube tags 130
  • 200. Using Visualizations for Music Discovery - ISMIR 2009 http://playground.last.fm/demo/tagstube Last.fm tube tags 130
  • 201. Using Visualizations for Music Discovery - ISMIR 2009 Scatter plots 131
  • 202. Using Visualizations for Music Discovery - ISMIR 2009 Justin Donaldson ZMDSVisualization 132
  • 203. Using Visualizations for Music Discovery - ISMIR 2009 Visualization Name 133
  • 204. Using Visualizations for Music Discovery - ISMIR 2009 Visualization Name 133
  • 205. Using Visualizations for Music Discovery - ISMIR 2009 Visual Playlist Generation on the Artist Map - Van Gulick,Vignoli Using map for playlist navigation 134
  • 206. Using Visualizations for Music Discovery - ISMIR 2009 http://mpac.ee.ntu.edu.tw/~yihsuan/pub/MM08_MrEmo.pdf Mr. Emo: Music Retrieval in the Emotional Plane 135
  • 207. Using Visualizations for Music Discovery - ISMIR 2009 http://mpac.ee.ntu.edu.tw/~yihsuan/pub/MM08_MrEmo.pdf Mr. Emo: Music Retrieval in the Emotional Plane 135
  • 208. Using Visualizations for Music Discovery - ISMIR 2009 http://www.burstlabs.com/ Burst Labs 136
  • 209. Using Visualizations for Music Discovery - ISMIR 2009 http://mediacast.sun.com/share/plamere/sitm_final.pdf - Lamere, Eck Search Inside the Music 137
  • 210. Using Visualizations for Music Discovery - ISMIR 2009 The Mufin Player 3D Track similarity 138
  • 211. Using Visualizations for Music Discovery - ISMIR 2009 The Mufin Player 3D Track similarity 138
  • 212. Using Visualizations for Music Discovery - ISMIR 2009 http://thesis.flyingpudding.com/ - Anita Lillie Music Box 139
  • 213. Using Visualizations for Music Discovery - ISMIR 2009 http://thesis.flyingpudding.com/ - Anita Lillie Music Box 139
  • 214. Using Visualizations for Music Discovery - ISMIR 2009 http://www.flyingpudding.com/projects/viz_music/ Pictures of Songs 140
  • 215. Using Visualizations for Music Discovery - ISMIR 2009 http://www.cs.kuleuven.be/~jorisk/thrillerMovie.mov Visualizing Emotion in Lyrics 141
  • 216. Using Visualizations for Music Discovery - ISMIR 2009 Sampling Connections 142
  • 217. Using Visualizations for Music Discovery - ISMIR 2009 http://jessekriss.com/projects/samplinghistory/ The History of Sampling 143
  • 218. Using Visualizations for Music Discovery - ISMIR 2009 http://jessekriss.com/projects/samplinghistory/ The History of Sampling 143
  • 219. Using Visualizations for Music Discovery - ISMIR 2009 Graham Coleman - http://www.iua.upf.edu/~gcoleman/pubs/mused07.pdf Mused: Navigating the personal sample library 144
  • 220. Using Visualizations for Music Discovery - ISMIR 2009 Graham Coleman - http://www.iua.upf.edu/~gcoleman/pubs/mused07.pdf Mused: Navigating the personal sample library 144
  • 221. Using Visualizations for Music Discovery - ISMIR 2009 145 Time Series
  • 222. Using Visualizations for Music Discovery - ISMIR 2009 http://www.diametunim.com/muse/ last.fm spiral 146
  • 223. Using Visualizations for Music Discovery - ISMIR 2009 147 http://www.leebyron.com/what/lastfm/ - Lee Byron Visualizing Listening History
  • 224. Using Visualizations for Music Discovery - ISMIR 2009 147 http://www.leebyron.com/what/lastfm/ - Lee Byron Visualizing Listening History
  • 225. Using Visualizations for Music Discovery - ISMIR 2009 http://www.bbc.co.uk/radio/labs/radiowaves/ Radio Waves 148
  • 226. Using Visualizations for Music Discovery - ISMIR 2009 http://www.bbc.co.uk/radio/labs/radiowaves/ Radio Waves 148
  • 227. Using Visualizations for Music Discovery - ISMIR 2009 http://bowr.de/blog/?p=49 Dominikus Baur University of Munich Pulling strings from a tangle 149
  • 228. Using Visualizations for Music Discovery - ISMIR 2009 http://bowr.de/blog/?p=49 Dominikus Baur University of Munich Pulling strings from a tangle 149
  • 229. Using Visualizations for Music Discovery - ISMIR 2009 Trees 150 Genre Map derived from Last.fm tags Lamere
  • 230. Using Visualizations for Music Discovery - ISMIR 2009 Trees 151 Visualizing and exploring personal music libraries Torrens, Hertzog,Arcos
  • 231. Using Visualizations for Music Discovery - ISMIR 2009 Visualizations : Dave’s Music Library Contents Treemap of a personal music collection 152
  • 232. Using Visualizations for Music Discovery - ISMIR 2009 Mirrored Radio dial - by thomwatson Interactive 153
  • 233. Using Visualizations for Music Discovery - ISMIR 2009 SongExplorer: Exploring Large Musical Databases Using a Tabletop Interface Carles F. Julià - Music Technology Group - Universitat Pompeu Fabra SongExplorer 154
  • 234. Using Visualizations for Music Discovery - ISMIR 2009 SongExplorer: Exploring Large Musical Databases Using a Tabletop Interface Carles F. Julià - Music Technology Group - Universitat Pompeu Fabra SongExplorer 154
  • 235. Using Visualizations for Music Discovery - ISMIR 2009 http://fidgt.com/visualize Fidgt:Visualize 155
  • 236. Using Visualizations for Music Discovery - ISMIR 2009 http://fidgt.com/visualize Fidgt:Visualize 155
  • 237. Using Visualizations for Music Discovery - ISMIR 2009 http://waks.nl/nl/musicalnodes Lisa Dalhuijsen, Lieven vanVelthoven -LIACS, Leiden University MusicNodes 156
  • 238. Using Visualizations for Music Discovery - ISMIR 2009 http://waks.nl/nl/musicalnodes Lisa Dalhuijsen, Lieven vanVelthoven -LIACS, Leiden University MusicNodes 156
  • 239. Using Visualizations for Music Discovery - ISMIR 2009 social.zune.net Zune MixView 157
  • 240. Using Visualizations for Music Discovery - ISMIR 2009 social.zune.net Zune MixView 157
  • 241. Using Visualizations for Music Discovery - ISMIR 2009 http://staff.aist.go.jp/m.goto/Musicream/ Goto and Masataka Goto. Musicream 158
  • 242. Using Visualizations for Music Discovery - ISMIR 2009 http://staff.aist.go.jp/m.goto/Musicream/ Goto and Masataka Goto. Musicream 158
  • 243. Using Visualizations for Music Discovery - ISMIR 2009 Elias Pampalk & Masataka Goto, "MusicRainbow:A New User Interface to Discover Artists Using Audio-based Similarity and Web-based Labeling MusicRainbow 159
  • 244. Using Visualizations for Music Discovery - ISMIR 2009 Elias Pampalk & Masataka Goto, "MusicSun:A New Approach to Artist Recommendation" MusicSun 160
  • 245. Using Visualizations for Music Discovery - ISMIR 2009 EN Music Explorer 161
  • 246. Using Visualizations for Music Discovery - ISMIR 2009 EN Music Explorer 161
  • 247. Using Visualizations for Music Discovery - ISMIR 2009 Here be Dragons 162
  • 248. Using Visualizations for Music Discovery - ISMIR 2009 Here be Dragons 162
  • 249. Using Visualizations for Music Discovery - ISMIR 2009 Here be Dragons 162
  • 250. Visualization Proponents (TheYale Club) F.J.Anscombe (Anscombe’s Quartet) John Tukey (stem/leaf plots) Edward Tufte (Sparklines) http://en.wikipedia.org/wiki/F.J._Anscombe http://en.wikipedia.org/wiki/John_Tukey http://en.wikipedia.org/wiki/Leland_Wilkinson http://en.wikipedia.org/wiki/Edward_Tufte 0 | 0000111111222338 2 | 07 4 | 5 6 | 8 8 | 4 10 | 5 12 | 14 | 16 | 0 Leland Wilkinsen (Cluster heat maps) 163
  • 251. InteractiveVisualization Proponents Ben Shneiderman (Treemaps, NSD diagrams) Steve Deunes (& entire NYT Graphics Department) Ben Fry & Casey Reas (Processing) 164
  • 252. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 253. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 254. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 255. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 256. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 257. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 258. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 259. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 260. Using Visualizations for Music Discovery - ISMIR 2009 Resources 165
  • 261. Using Visualizations for Music Discovery - ISMIR 2009 VisualizingMusic.com Visualizing Music 166 musicviz.googlepages.com/home
  • 263. Other Analytic* Tools • Matlab • Dimensionality Reduction Toolbox • (missing good network viz/analysis option) • Extensive Interactive Pub-quality plotting environment • Pajek (standalone NetworkVisualization/Analysis Platform) • IVC Software Framework (research platform for analysis and viz). • R • Excellent network viz/analysis package “network” • PCA/MDS (base library), ISOMap {vegan} (no LLE,T-SNE yet) • Pub quality plotting environments (default, ggplot2, iplots) http://ict.ewi.tudelft.nl/~lvandermaaten/Matlab_Toolbox_for_Dimensionality_Reduction.html http://cran.r-project.org/web/packages/network/index.html http://cran.r-project.org/web/packages/vegan/index.html http://vlado.fmf.uni-lj.si/pub/networks/pajek/ http://iv.slis.indiana.edu/sw/index.html *for publications 168
  • 264. Other “Generic”Viz Tools • Prefuse (java/as3) • Processing* • Gephi • TouchGraph • Gnuplot • matplotlib (python) http://www.prefuse.org/ http://processing.org/ http://gephi.org/ http://www.touchgraph.com/navigator.html http://www.gnuplot.info/ http://matplotlib.sourceforge.net/ 169
  • 265. Conclusions • Music visualizations reflect ways of understanding music. • Music understanding draws from many different types of information (musicological, acoustic, social, bibliographic, etc.) • Music visualizations can help to express this understanding in a way that can help aid navigation and discovery.