Social Tags and Music Information Retrieval (Part I)

Social Tags and Music Information Retrieval Part I
ISMIR 2008
Paul Lamere – Sun Microsystems Inc.
Elias Pampalk – Last.fm

Speaker – Paul Lamere

Speaker – Elias Pampalk

Audience Poll
Have you ever tagged something?
Have you ever tagged music?
Have you ever heard of Last.fm?
Have you ever used tags to help find something?
Do you think social tags are just a web 2.0 fad?
Are you using social tags in your research?
Are you a tag skeptic?

What are social tags? Word Clouds created with Wordle.net

Social Tags - Delicious.com

Social Tags - Flickr.com

Social Tags - LibraryThing

What is Deerhoof? Flickr photo by graciepoo

What is Deerhoof?

The Fear – It's just Graffiti
Folksonomies are notoriously imprecise, full of tags that are "often ambiguous , overly personalized and inexact " resulting in an " uncontrolled and chaotic set of tagging terms that do not support searching as effectively as more controlled vocabularies do"
Guy & Tonkin, 2006
... if users can continuously add tags to items, at some point it is likely that the whole system will become unusable . A folksonomic system threatens to undermine its own usefulness.
Elaine Peterson, 2006
Flickr Photo by funkandjazz Abbey E.Thompson. Playing Tag: An Analysis of Vocabulary Patterns and Relationships Within a Popular Music Folksonomy

Outline
What are social tags?
Why do people tag?
Issues with social tags
Other sources of tags
Search, Discovery & Recommendation
Data & Tools
Future Research
Conclusion
Discussion

Tags
Short, free text labels applied to content
No structure, no vocabulary limits
Typically applied by the generator or the consumer of the item being tagged
Social Tags
The aggregation of individual sets of tags
Sometimes called a Folksonomy - A user-created bottom-up categorical structure development with an emergent thesaurus

What are social tags? Taxonomy vs. Folksonomy
Taxonomy
Strong labeling
Structured vocabulary
Fixed vocabulary
Folksonomy
Weak labeling
Unstructured vocabulary
Free-for-all vocabulary
D. Turnbull, L. Barrington, G. Lanckriet. Five Approaches to Collecting Tags for Music. ISMIR 2008.

What are social tags? Some characteristics of social tags J. Furner, M. Smith, and M. Winget. Collaborative indexing of cultural resources: Some outstanding issues.

What are social tags? Social Tagging System Attributes
Tagging Rights
owner, group, anyone
Tagging Support
blind tagging, suggestive tagging
Aggregation
bag model, set model
Type of object being tagged
artists, tracks, albums, labels, playlists, clips

What are social tags? What can we learn from social tags?
Insights to user behavior and language usage
how different is rap from hip-hop ?
Grouping of items based on tags or taggers
cheese on toast, hooves and paws
Finding social groups with shared interests
people that tag the same items or use the same tags
Generating user profiles from tagging behavior
tag clouds based on tags applied by the user
Representation of taste and/or expertise
Information on how items change over time
track changes to item tags over time
K. Weller. Folksonomies and Ontologies. Two New Players in Indexing and Knowledge Representation.

What are social tags? Example: MusicBrainz
Some MusicBrainz Tag Stats
Artist tags: 37,875
Album tags: 21,423
Track tags: 10,818
Label tags: 1,250

What are social tags? Example: Amazon

What are social tags? Example: MyStrands

What are social tags? Example: imeem

What are social tags? Example: Freesound

What are social tags? Last.fm – Case Study
History
Tagging stats
Integration
Tags and their popularity
More stats

July 2005: Felix Miller (CEO) on internal wiki:
Why use tags:
To build playlists by tagging tracks
Categorise your profile and the global music catalogue
Get recommendations based on tags
Summarize your profile
August 2005: launched
Inspired by delicious and flickr
At the same time audioscrobbler merged with Last.fm
What are social tags? History of tagging at Last.fm

Tags: > 50M
Tracks > 50%
Artists > 40%
Albums < 5%
Labels < 1%
Unique tags: > 1.2M
Unique tags applied to more than 10 items: > 130k
Items tagged: > 3.8M
Tags per month: > 2.5M
Taggers per month: > 300k
What are social tags? Last.fm tagging stats

What are social tags? Integration into Last.fm experience
Where are tags visible?
Music landing page
Tag pages
Tag radio
Artist, album, track, label summaries
Tag clouds
User library, ...

How can users tag something?
Desktop software (Last.fm Player & Scrobbler)
Web site
Add to library dialog
Add tag button on item pages (artist, album, track)
Every listed item (charts etc) has multi-function button
Flash player

Tags are visible almost everywhere
It is very easy to tag something

What are social tags? Tag Popularity: Examples
rock >190k >2M
seen live >75k >1M
female vocalists >60k > 500k
chillout >40k > 250k
guitar >20k >90k
happy >9k >40k
icelandic >8k >20k
brazil >4k >15k
inspirational 1.2k >5k
hammond organ >200 >700
berimbau <50 < 90
people # times applied

What are social tags? Tag Popularity: More Examples
Favorites > 18k > 200k
officially sh*t > 2k > 12k
i am a party girl here is my soundtrack > 350 > 5.9k
tv themes > 250 > 680
songs that i sing along to but i always forget the words so i say duh duh while trying to sound like i do know the words and no one is falling for it but they keep quiet because they are embarrassed for me > 60 > 300
gtasa tracks < 30 > 700
artists with disco in their name 2 49
wherein i compile a comprehensive list of songs about zombies 7 36
people # times applied

What are social tags? Tag Categories beyond Genre by Babs_05 (Last.fm user)
personal meaning
playlists with unusual names
regional
decade
group-specific
descriptive
http://www.last.fm/group/Track-Tag+Bitches/forum/24779/_/319957/1#f4621887
tempo
mood
instruments
films
music to...
album labels
based on radio shows or nightclubs

[Paul Lamere, JNMR 2008] What are social tags? Tagging Frequencies by Category

[Paul Lamere, JNMR 2008] [Klaas Bosteels et al., ISMIR 2008] Multi-tag search queries 51% 7% 4% 2% 5% 26% Time 3% Other 2% What are social tags? Tagging vs. Search Frequencies

What are social tags? Obscure Tags
if you fall in love with me you should know these songs by heart
sure go ahead and depress the hell outta me what do I care
that one song
and finally laughing like a maniac saying I am the ruler of this stupid little world called me
I woke up this morning and the sun was gone
I used to suck blood with gothic dogs but now I wear bermuda shorts and [...]
I was never a Swedish teenager with swedish teen angst so now I will attempt at having it

What are social tags? Number of words per tag [Levy & Sandler, JNMR 2008]

What are social tags? Popular tag radio stations
Pop, rock, jazz, chillout, 80s, indie, dance, hip-hop, alternative, house, pop rock, reggae, electronic , oldies, metal, blues, schlager, ambient, trance, classical
Measured on a day in July 2008. Almost all of the most popular radio stations are genres! Surprise?

What are social tags? Most frequently applied tags
Rock, seen live , alternative, indie, electronic, pop, metal , female vocalists , alternative rock, classic, rock, punk, jazz, indie rock, electronic, singer-songwriter, folk, hip-hop, ambient, dance, experimental
Metal fans are some of the best taggers ever.

[Paul Lamere, JNMR 2008] Artist rank (wrt number of tags applied, 1-1000) Number of tags applied What are social tags? Distribution of tags

What are social tags? Age vs. Tagging Behaviour
Under 19 Average age of users (years)
lame lame lame lame 18.2
tokio hotel 18.5
xd 18.7
<3 18.7
tr00 satanic black metal 18.8

Over 35 Average age of users (years)
zydeco 37.3
soukous 36.6
cajun 35.9
zimbabwe 35.3
jazz saxophone 35.1
What are social tags? Age vs. Tagging Behaviour

What are social tags? Age vs. Tagging Behaviour Preliminary results! http://mir-research.blogspot.com/2008/06/lastfms-api-python-and-tagging_29.html

Outline
Why do people tag?
Data & Tools
Future Research
Conclusion
Discussion

Why do people tag?
Pew Study:
28% of Internet users have tagged or categorized online content
Everyday 7% of internet users tag or categorize online content
Why are people tagging?
Tagging is personal but benefits everyone
Tagging is easy
Tagging is fun
D. Weinberger. How tagging changes peoples relationship to information and each other. Pew Internet & American Life Project . (2007)

Why do people tag? Traditional categorization is hard R. Sinha. Tagging – From Personal to Social http://rashmisinha.com/2006/03/12/my-slides-for-tagging-20-panel-at-sxsw/

Why do people tag? Tagging is simpler R. Sinha. Tagging – From Personal to Social http://rashmisinha.com/2006/03/12/my-slides-for-tagging-20-panel-at-sxsw/

Why do people tag? Memory and Context

Why do people tag? Task Organization

Why do people tag? Opinion Expression / Social Signaling

Why do people tag? Social Contribution
The defectivebydesign tag is used by anti-DRM advocates to mark items that use DRM
http://www.defectivebydesign.org/

Why do people tag? Play and Competition Squared Circle by shutterBRI by plamere

Last.fm: Why do people tag?
Tag radio
global: collaborative playlist
personal: playlist
Example taggers
Last.fm groups focusing on tags
Bookmarking, organizing, sharing knowledge, communicating, ...

drsaunde : male, 33, Canada
12,599 unique tags, up to 1,540 times each
Why do people tag? Last.fm taggers

spacefish : female, 44, Denmark
782 unique tags, up to 3,336 times each

fionapinkstars: female, 22, UK (staff)
43 unique tags, up to 67 times

martind: UK (staff)
tags are a channel for narrative and social interaction personal tags allow you to draw a circle around stuff you care about Why do people tag? Last.fm taggers

E1i45: male, 30, UK (staff)

Average Tagger: male, 24, USA
applied around 50 tags

hm_hallo: female, 25, Germany

Why do people tag? Last.fm groups focusing on tags
Group: The Special Interest Tag Radio Collective >100 members
“ We have the radio you want! :-D If not, join and help us grow musically... YAY!”
Tag radio stations listed:
Cheese on Toast, The Ladies Room, Tambourine, Xylophone & Vibraphone, Forests and Woodland, Insects and Arachnids, They Said Shoes!, Under the Influence..., Awesome Guitar Jams, ...

Group: Subscribers and their tag radio stations
>100 members
This group is to bring together subscribers who have taken the time to tag and share the music they like. No specific genre or participation needed. This is just a way to find and listen to...

Group: Genre-free tags!
>100 members
"That's right; there's more to tags than silly genres."
lesser known yet streamable artists
cover track
first album
i am a spy here is my soundtrack

Group: Thursday Night Party Hat Party
>70 members
“OK, I've started tagging weird country music, in anticipation of Frippgamesh's party on August 21. Right now, I'm the only one who's done any tagging, but I was able to tag 11 artists without working up a sweat, starting with the band Rebel Meets Rebel that I had afflicted aniaki with in a recent double-dare.”

Group: WoopWoop
>30 members
"Listening to a track you're desperate to share with the world? Want to listen to music that others feel the same about?"

More Groups:
Pure Track-Taggers
Track-Tag Bitches
Tag Top
Tags are a Joke
We use tags that have never been used before
...

Why do people tag?
Motivations
Simplicity
Memory and context
Task organization
Opinion expression & social signaling
Social contribution
Play and competition
Last.fm specific
Collaborative/personal playlists (tag radio)
Tag groups

Outline
Why do people tag?
Data & Tools
Future Research
Conclusion
Discussion

Issues with social tags Cold start and obscure content
Tagging something requires:
Knowing it exists (new releases, cold start)
Caring enough to bother (obscure content, long tail)
Problem scope
Many artists, albums, tracks affected
Solution: alternative sources of tags
Autotagging, tag games, ...

Radiohead
Issues with social tags Cold start and obscure content Tagged by 24k listeners on Last.fm

Jersey Budd
Issues with social tags Cold start and obscure content

Weak labeling
E.g. “male vocalist” is sometimes considered a default
Problem scope:
sparsely tagged items are affected most
Popularity & noise
The Beatles:
tagged emo 68 times on Last.fm
These Silhouettes:
tagged emo 11 times on Last.fm
Solution: normalization
Issues with social tags Precision and Recall

Kids having fun
Barney, Paris Hilton, ...
Can be very offensive
Labels/artists trying to “help”
Problem scope:
Limited to very few artists
Issues with social tags Abuse

Issues with social tags Abuse example: Paris Hilton

Do taggers like what they are tagging?
Correlations between taggers
Bot-like patterns
Tag Karma
How “useful” are contributions of a tagger?
E.g. if artist X is tagged metal, but everyone listening to metal radio skips that artist...
Issues with social tags Abuse detection

Examples
that one song
lazy eye
Problem scope
Sparsely tagged items affected most
Issues with social tags Noise

Examples
Polysemy: love
music I love
music about love
Synonym: female vocalists
Problem scope
Sparsely tagged items
Issues with social tags Ambiguity and Synonyms

89277 female vocalists
15874 female
7150 female vocalist
2716 female vocals
2180 female voices
1424 female artists
955 female vocal
698 female singers
http://blogs.sun.com/plamere/entry/determining_synonyms_from_tags Issues with social tags Synonyms Example

Last.fm community is not an unbiased sample of music listeners worldwide
On average a relatively young audience
Tech-savvy
Tastemakers
Eclectic tastes, interested in discovering new music
Non-mainstream
Some regions are underrepresented (e.g. Africa)
Some styles of music are represented stronger than others. E.g. classical music vs. indie/alternative
Issues with social tags Tagger bias

Last.fm community is not an unbiased sample of music listeners worldwide
Electronic music
Nielsen: 3% of total album sales in US
Last.fm: 107k taggers
Country music
Nielsen: 11% of total album sales in US
Last.fm: 23k taggers
Issues with social tags Tagger bias in numbers [Nielsen Report 2007]

Issues with Social Tags
Not enough tags
Cold start
Obscure content (long tail)
Weak labeling
Synonyms and ambiguous tags
Noise
Abuse
Tagger bias
Some styles of music are covered better than others
Not everyone is contributing equally

Outline
Why do people tag?
Data & Tools
Future Research
Conclusion
Discussion

Other sources of tags Games (with a purpose)
Idea:
Humans are best at solving some types of problems
Humans like to play games
Build a game that is
fun to play
Solves a hard problem
ESP Game – Image Labeling Game
Users try to guess what labels partner will apply to an image
Collected 1.2 million image labels in 4 months
With 5K active gamers could label all 425 million in 1 month
L. von Ahn and L. Dabbish. Labeling images with a computer game . SIGCHI 2004.

Other sources of tags Games – Tag a Tune http://www.gwap.com/gwap/gamesPreview/tagatune/

Statistics for 3 months of game play
Total tags: ~260,000
Total songs: ~20,000
High confidence tags: ~50,000
Games played: ~23,000
Unique players: ~6,900
Tags per player per minute: ~4
Other sources of tags Games – Tag a Tune

Other sources of tags Games – Major Miner http://majorminer.com/

Unique asynchronous scoring
Statistics
530 Users
2,345 clips tagged
73,000 user, clip, tag triples
12,000 verified tags
900 unique, verified tags
12,000
3-4 tags per minute per user
5-10 minutes of playtime per user
Data used for the MIREX tagging task
http://www.music-ir.org/mirex/2008/index.php/Audio_Tag_Classification Other sources of tags Games – Major Miner

http://majorminer.com/search/ Other sources of tags Games – Major Miner

Other sources of tags Games – The Listen Game http://www.listengame.org/

Statistics from 2 week pilot study
Listen250 dataset
26,000 annotations
1,558 High confidence positive associations
1,630 High confidence negative associations
250 songs
120 words
440 unique players
775 new words suggested (via freestyle round)
~ 5 Tags per user per minute
Dataset available upon request from Doug Turnbull
Other sources of tags Games – The Listen Game

Other sources of tags Games – MoodSwings Y. E. Kim, E. Schmidt, and L. Emelle. MoodSwings: A collaborative game for music mood label collection. ISMIR 2008.
Collects valence-arousal labels
1 week pilot
100 Users
1K Songs
50K labels

Other sources of tags Games – Herd It "Herd It connects players in real-time and they listen to the same music clip. While they listen, various fun minigames quiz the players on opinions and trivia about the music. Players earn points based on the agreement of their answers with the group consensus." - Luke Barrington

Other sources of tags Games – Moose6 – A BBC Prototype

Pros:
Can tag at the phrase/clip level
Potentially can tag into the long tail
Potentially very high tagging rates
Issues:
Challenge to make addictive/fun game
Tags may be more superficial than social tags
Tags may be applied by non-fans
Lowest common denominator labels
Potential for 'gaming' the system
Don't collect many types of useful labels:
context ( jogging ), opinion ( favorite ), task organization ( check out ), social signaling ( seen live )
Other sources of tags Games (with a purpose)

Other sources of tags Experts/Surveys
Hire experts to hand label content
Advantages
Consistent labels
Strong labeling
Fixed, structured vocabulary
Disadvantages
Small pre-determined vocabulary
Very difficult to construct taxonomy
Human-labor intensive (expensive)
Doesn't scale to the long tail
Limited access to data

Other sources of tags Experts/Surveys – All Music Guide

Pop Tracks: ~13 million
Pop Albums: ~1.4 million
Classical Compositions: ~300 thousand
Biographies: ~84 thousand
Styles: 917
Themes: 86
Moods: 184

Music labeled by paid professional musicians
Fixed vocabulary: ~500 weighted attributes
Structured vocabulary
Other sources of tags Experts/Surveys – Pandora

66 undergrads – paid $10 per hour to label music
1,700 annotations
500 'western popular' songs
135 musically-relevant concepts
Six semantic categories:
emotion – aggressive, weird, thrilling, mellow
genre - alternative, bebop, swing, rock, world
Instrument – guitar, organ, piano, harmonica
usage – at a party, at work, driving, exercising, sleeping
vocals – breathy, rapping, monotone, emotional
song – catchy, fast, danceable, texture acoustic
http://cosmal.ucsd.edu/cal Other sources of tags Experts/Surveys – CAL-500

Other sources of tags Web mining
Internet: the largest digital archive of music
Music blogs, reviews, forum discussions, official artist pages, MySpace, ...
Examples: (Google estimated number of results)
Eminem 46M pages
“The Beatles” 29M pages
“Wolfgang Amadeus Mozart” 7M pages
Unstructured data
Text information retrieval

Whitman & Lawrence (ICMC 2002)
Google: “<artist name>” +music +review
Retrieve text of top ranked pages
Text information retrieval techniques
Issues
Artist names are not unique identifiers
“The War”, “The The”, “Chicago”, “Pink” (not “P!nk”)
Solution: add information to query (e.g. album names)
Cold-start (needs at least one web-page)
Quality of “tags”
Include other artist names, seemingly random noise
Solution: use dictionary
Other sources of tags Web mining – Using Google et al.

ABBA
100 pop
56 song
50 musical
46 swedish
43 uk
41 group
36 band
35 dance
31 world
29 disco
Pampalk et al. (ECDL 2005) http://www.ofai.at/~elias.pampalk/wa/ Other sources of tags Web mining – Examples

Britney Spears
100 pop
99 song
52 girl
36 concert
33 dance
24 uk
20 sexy
18 jive
18 world
15 cool

Metallica
100 metal
43 band
38 concert
22 heavy metal
20 world
20 song
18 dance
15 hard
11 black metal
9 guitar

Ludwig van Beethoven
100 piano
67 symphony
53 classical
22 orchestra
20 movement
18 trio
18 solo
15 variations
15 musical
12 violin

[Knees et al., ACM MM 2006]
Other sources of tags Using Web Mined Tags: nepTune

[Pampalk & Goto, ISMIR 2006]
Other sources of tags Using Web Mined Tags: MusicRainbow

[Pampalk & Goto, ISMIR 2007]
Other sources of tags Using Web Mined Tags: MusicSun

http://ahmadkhabirmurobitun.blogspot.com	28
http://www.slideshare.net	17
http://strumbox.com	2

Social Tags and Music Information Retrieval (Part I)

by Paul Lamere on Sep 12, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 47

Statistics

Social Tags and Music Information Retrieval (Part I) — Presentation Transcript