Social Tags and Music Information Retrieval (Part I)

Social Tags and Music Information Retrieval Part I

ISMIR 2008

Paul Lamere – Sun Microsystems Inc.

Elias Pampalk – Last.fm

Speaker – Paul Lamere

Speaker – Elias Pampalk

Audience Poll

Have you ever tagged something?

Have you ever tagged music?

Have you ever heard of Last.fm?

Have you ever used tags to help find something?

Do you think social tags are just a web 2.0 fad?

Are you using social tags in your research?

Are you a tag skeptic?

What are social tags? Word Clouds created with Wordle.net

Social Tags - Delicious.com

Social Tags - Flickr.com

Social Tags - LibraryThing

What is Deerhoof? Flickr photo by graciepoo

What is Deerhoof?

The Fear – It's just Graffiti

Folksonomies are notoriously imprecise, full of tags that are "often ambiguous , overly personalized and inexact " resulting in an " uncontrolled and chaotic set of tagging terms that do not support searching as effectively as more controlled vocabularies do"

Guy & Tonkin, 2006

... if users can continuously add tags to items, at some point it is likely that the whole system will become unusable . A folksonomic system threatens to undermine its own usefulness.

Elaine Peterson, 2006

Flickr Photo by funkandjazz Abbey E.Thompson. Playing Tag: An Analysis of Vocabulary Patterns and Relationships Within a Popular Music Folksonomy

Outline

What are social tags?

Why do people tag?

Issues with social tags

Other sources of tags

Search, Discovery & Recommendation

Data & Tools

Future Research

Conclusion

Discussion

Outline

Why do people tag?

Data & Tools

Future Research

Conclusion

Discussion

Tags

Short, free text labels applied to content

No structure, no vocabulary limits

Typically applied by the generator or the consumer of the item being tagged

Social Tags

The aggregation of individual sets of tags

Sometimes called a Folksonomy - A user-created bottom-up categorical structure development with an emergent thesaurus

What are social tags? Taxonomy vs. Folksonomy

Taxonomy

Strong labeling

Structured vocabulary

Fixed vocabulary

Folksonomy

Weak labeling

Unstructured vocabulary

Free-for-all vocabulary

D. Turnbull, L. Barrington, G. Lanckriet. Five Approaches to Collecting Tags for Music. ISMIR 2008.

What are social tags? Some characteristics of social tags J. Furner, M. Smith, and M. Winget. Collaborative indexing of cultural resources: Some outstanding issues.

What are social tags? Social Tagging System Attributes

Tagging Rights

owner, group, anyone

Tagging Support

blind tagging, suggestive tagging

Aggregation

bag model, set model

Type of object being tagged

artists, tracks, albums, labels, playlists, clips

What are social tags? What can we learn from social tags?

Insights to user behavior and language usage

how different is rap from hip-hop ?

Grouping of items based on tags or taggers

cheese on toast, hooves and paws

Finding social groups with shared interests

people that tag the same items or use the same tags

Generating user profiles from tagging behavior

tag clouds based on tags applied by the user

Representation of taste and/or expertise

Information on how items change over time

track changes to item tags over time

K. Weller. Folksonomies and Ontologies. Two New Players in Indexing and Knowledge Representation.

What are social tags? Example: MusicBrainz

Some MusicBrainz Tag Stats

Artist tags: 37,875

Album tags: 21,423

Track tags: 10,818

Label tags: 1,250

What are social tags? Example: Amazon

What are social tags? Example: MyStrands

What are social tags? Example: imeem

What are social tags? Example: Freesound

What are social tags? Last.fm – Case Study

History

Tagging stats

Integration

Tags and their popularity

More stats

July 2005: Felix Miller (CEO) on internal wiki:

Why use tags:

To build playlists by tagging tracks

Categorise your profile and the global music catalogue

Get recommendations based on tags

Summarize your profile

August 2005: launched

Inspired by delicious and flickr

At the same time audioscrobbler merged with Last.fm

What are social tags? History of tagging at Last.fm

Tags: > 50M

Tracks > 50%

Artists > 40%

Albums < 5%

Labels < 1%

Unique tags: > 1.2M

Unique tags applied to more than 10 items: > 130k

Items tagged: > 3.8M

Tags per month: > 2.5M

Taggers per month: > 300k

What are social tags? Last.fm tagging stats

What are social tags? Integration into Last.fm experience

Where are tags visible?

Music landing page

Tag pages

Tag radio

Artist, album, track, label summaries

Tag clouds

User library, ...

How can users tag something?

Desktop software (Last.fm Player & Scrobbler)

Web site

Add to library dialog

Add tag button on item pages (artist, album, track)

Every listed item (charts etc) has multi-function button

Flash player

Tags are visible almost everywhere

It is very easy to tag something

What are social tags? Tag Popularity: Examples

rock >190k >2M

seen live >75k >1M

female vocalists >60k > 500k

chillout >40k > 250k

guitar >20k >90k

happy >9k >40k

icelandic >8k >20k

brazil >4k >15k

inspirational 1.2k >5k

hammond organ >200 >700

berimbau <50 < 90

people # times applied

What are social tags? Tag Popularity: More Examples

Favorites > 18k > 200k

officially sh*t > 2k > 12k

i am a party girl here is my soundtrack > 350 > 5.9k

tv themes > 250 > 680

songs that i sing along to but i always forget the words so i say duh duh while trying to sound like i do know the words and no one is falling for it but they keep quiet because they are embarrassed for me > 60 > 300

gtasa tracks < 30 > 700

artists with disco in their name 2 49

wherein i compile a comprehensive list of songs about zombies 7 36

people # times applied

What are social tags? Tag Categories beyond Genre by Babs_05 (Last.fm user)

personal meaning

playlists with unusual names

regional

decade

group-specific

descriptive

http://www.last.fm/group/Track-Tag+Bitches/forum/24779/_/319957/1#f4621887

tempo

mood

instruments

films

music to...

album labels

based on radio shows or nightclubs

[Paul Lamere, JNMR 2008] What are social tags? Tagging Frequencies by Category

[Paul Lamere, JNMR 2008] [Klaas Bosteels et al., ISMIR 2008] Multi-tag search queries 51% 7% 4% 2% 5% 26% Time 3% Other 2% What are social tags? Tagging vs. Search Frequencies

What are social tags? Obscure Tags

if you fall in love with me you should know these songs by heart

sure go ahead and depress the hell outta me what do I care

that one song

and finally laughing like a maniac saying I am the ruler of this stupid little world called me

I woke up this morning and the sun was gone

I used to suck blood with gothic dogs but now I wear bermuda shorts and [...]

I was never a Swedish teenager with swedish teen angst so now I will attempt at having it

What are social tags? Number of words per tag [Levy & Sandler, JNMR 2008]

What are social tags? Popular tag radio stations

Pop, rock, jazz, chillout, 80s, indie, dance, hip-hop, alternative, house, pop rock, reggae, electronic , oldies, metal, blues, schlager, ambient, trance, classical

Measured on a day in July 2008. Almost all of the most popular radio stations are genres! Surprise?

What are social tags? Most frequently applied tags

Rock, seen live , alternative, indie, electronic, pop, metal , female vocalists , alternative rock, classic, rock, punk, jazz, indie rock, electronic, singer-songwriter, folk, hip-hop, ambient, dance, experimental

Metal fans are some of the best taggers ever.

[Paul Lamere, JNMR 2008] Artist rank (wrt number of tags applied, 1-1000) Number of tags applied What are social tags? Distribution of tags

What are social tags? Age vs. Tagging Behaviour

Under 19 Average age of users (years)

lame lame lame lame 18.2

tokio hotel 18.5

xd 18.7

<3 18.7

tr00 satanic black metal 18.8

Over 35 Average age of users (years)

zydeco 37.3

soukous 36.6

cajun 35.9

zimbabwe 35.3

jazz saxophone 35.1

What are social tags? Age vs. Tagging Behaviour

What are social tags? Age vs. Tagging Behaviour Preliminary results! http://mir-research.blogspot.com/2008/06/lastfms-api-python-and-tagging_29.html

Outline

Why do people tag?

Data & Tools

Future Research

Conclusion

Discussion

Why do people tag?

Pew Study:

28% of Internet users have tagged or categorized online content

Everyday 7% of internet users tag or categorize online content

Why are people tagging?

Tagging is personal but benefits everyone

Tagging is easy

Tagging is fun

D. Weinberger. How tagging changes peoples relationship to information and each other. Pew Internet & American Life Project . (2007)

Why do people tag? Traditional categorization is hard R. Sinha. Tagging – From Personal to Social http://rashmisinha.com/2006/03/12/my-slides-for-tagging-20-panel-at-sxsw/

Why do people tag? Tagging is simpler R. Sinha. Tagging – From Personal to Social http://rashmisinha.com/2006/03/12/my-slides-for-tagging-20-panel-at-sxsw/

Why do people tag? Memory and Context

Why do people tag? Task Organization

Why do people tag? Opinion Expression / Social Signaling

Why do people tag? Social Contribution

The defectivebydesign tag is used by anti-DRM advocates to mark items that use DRM

http://www.defectivebydesign.org/

Why do people tag? Play and Competition Squared Circle by shutterBRI by plamere

Last.fm: Why do people tag?

Tag radio

global: collaborative playlist

personal: playlist

Example taggers

Last.fm groups focusing on tags

Bookmarking, organizing, sharing knowledge, communicating, ...

drsaunde : male, 33, Canada

12,599 unique tags, up to 1,540 times each

Why do people tag? Last.fm taggers

spacefish : female, 44, Denmark

782 unique tags, up to 3,336 times each

fionapinkstars: female, 22, UK (staff)

43 unique tags, up to 67 times

martind: UK (staff)

tags are a channel for narrative and social interaction personal tags allow you to draw a circle around stuff you care about Why do people tag? Last.fm taggers

E1i45: male, 30, UK (staff)

Average Tagger: male, 24, USA

applied around 50 tags

hm_hallo: female, 25, Germany

Why do people tag? Last.fm groups focusing on tags

Group: The Special Interest Tag Radio Collective >100 members

“ We have the radio you want! :-D If not, join and help us grow musically... YAY!”

Tag radio stations listed:

Cheese on Toast, The Ladies Room, Tambourine, Xylophone & Vibraphone, Forests and Woodland, Insects and Arachnids, They Said Shoes!, Under the Influence..., Awesome Guitar Jams, ...

Group: Subscribers and their tag radio stations

>100 members

This group is to bring together subscribers who have taken the time to tag and share the music they like. No specific genre or participation needed. This is just a way to find and listen to...

Group: Genre-free tags!

>100 members

"That's right; there's more to tags than silly genres."

lesser known yet streamable artists

cover track

first album

i am a spy here is my soundtrack

Group: Thursday Night Party Hat Party

>70 members

“OK, I've started tagging weird country music, in anticipation of Frippgamesh's party on August 21. Right now, I'm the only one who's done any tagging, but I was able to tag 11 artists without working up a sweat, starting with the band Rebel Meets Rebel that I had afflicted aniaki with in a recent double-dare.”

Group: WoopWoop

>30 members

"Listening to a track you're desperate to share with the world? Want to listen to music that others feel the same about?"

More Groups:

Pure Track-Taggers

Track-Tag Bitches

Tag Top

Tags are a Joke

We use tags that have never been used before

...

Why do people tag?

Motivations

Simplicity

Memory and context

Task organization

Opinion expression & social signaling

Social contribution

Play and competition

Last.fm specific

Collaborative/personal playlists (tag radio)

Tag groups

Outline

Why do people tag?

Data & Tools

Future Research

Conclusion

Discussion

Issues with social tags Cold start and obscure content

Tagging something requires:

Knowing it exists (new releases, cold start)

Caring enough to bother (obscure content, long tail)

Problem scope

Many artists, albums, tracks affected

Solution: alternative sources of tags

Autotagging, tag games, ...

Radiohead

Issues with social tags Cold start and obscure content Tagged by 24k listeners on Last.fm

Jersey Budd

Issues with social tags Cold start and obscure content

Weak labeling

E.g. “male vocalist” is sometimes considered a default

Problem scope:

sparsely tagged items are affected most

Popularity & noise

The Beatles:

tagged emo 68 times on Last.fm

These Silhouettes:

tagged emo 11 times on Last.fm

Solution: normalization

Issues with social tags Precision and Recall

Kids having fun

Barney, Paris Hilton, ...

Can be very offensive

Labels/artists trying to “help”

Problem scope:

Limited to very few artists

Issues with social tags Abuse

Issues with social tags Abuse example: Paris Hilton

Do taggers like what they are tagging?

Correlations between taggers

Bot-like patterns

Tag Karma

How “useful” are contributions of a tagger?

E.g. if artist X is tagged metal, but everyone listening to metal radio skips that artist...

Issues with social tags Abuse detection

Examples

that one song

lazy eye

Problem scope

Sparsely tagged items affected most

Issues with social tags Noise

Examples

Polysemy: love

music I love

music about love

Synonym: female vocalists

Problem scope

Sparsely tagged items

Issues with social tags Ambiguity and Synonyms

89277 female vocalists

15874 female

7150 female vocalist

2716 female vocals

2180 female voices

1424 female artists

955 female vocal

698 female singers

http://blogs.sun.com/plamere/entry/determining_synonyms_from_tags Issues with social tags Synonyms Example

Last.fm community is not an unbiased sample of music listeners worldwide

On average a relatively young audience

Tech-savvy

Tastemakers

Eclectic tastes, interested in discovering new music

Non-mainstream

Some regions are underrepresented (e.g. Africa)

Some styles of music are represented stronger than others. E.g. classical music vs. indie/alternative

Issues with social tags Tagger bias

Last.fm community is not an unbiased sample of music listeners worldwide

Electronic music

Nielsen: 3% of total album sales in US

Last.fm: 107k taggers

Country music

Nielsen: 11% of total album sales in US

Last.fm: 23k taggers

Issues with social tags Tagger bias in numbers [Nielsen Report 2007]

Issues with Social Tags

Not enough tags

Cold start

Obscure content (long tail)

Weak labeling

Synonyms and ambiguous tags

Noise

Abuse

Tagger bias

Some styles of music are covered better than others

Not everyone is contributing equally

Outline

Why do people tag?

Data & Tools

Future Research

Conclusion

Discussion

Other sources of tags Games (with a purpose)

Idea:

Humans are best at solving some types of problems

Humans like to play games

Build a game that is

fun to play

Solves a hard problem

ESP Game – Image Labeling Game

Users try to guess what labels partner will apply to an image

Collected 1.2 million image labels in 4 months

With 5K active gamers could label all 425 million in 1 month

L. von Ahn and L. Dabbish. Labeling images with a computer game . SIGCHI 2004.

Other sources of tags Games – Tag a Tune http://www.gwap.com/gwap/gamesPreview/tagatune/

Statistics for 3 months of game play

Total tags: ~260,000

Total songs: ~20,000

High confidence tags: ~50,000

Games played: ~23,000

Unique players: ~6,900

Tags per player per minute: ~4

Other sources of tags Games – Tag a Tune

Other sources of tags Games – Major Miner http://majorminer.com/

Unique asynchronous scoring

Statistics

530 Users

2,345 clips tagged

73,000 user, clip, tag triples

12,000 verified tags

900 unique, verified tags

12,000

3-4 tags per minute per user

5-10 minutes of playtime per user

Data used for the MIREX tagging task

http://www.music-ir.org/mirex/2008/index.php/Audio_Tag_Classification Other sources of tags Games – Major Miner

http://majorminer.com/search/ Other sources of tags Games – Major Miner

Other sources of tags Games – The Listen Game http://www.listengame.org/

Statistics from 2 week pilot study

Listen250 dataset

26,000 annotations

1,558 High confidence positive associations

1,630 High confidence negative associations

250 songs

120 words

440 unique players

775 new words suggested (via freestyle round)

~ 5 Tags per user per minute

Dataset available upon request from Doug Turnbull

Other sources of tags Games – The Listen Game

Other sources of tags Games – MoodSwings Y. E. Kim, E. Schmidt, and L. Emelle. MoodSwings: A collaborative game for music mood label collection. ISMIR 2008.

Collects valence-arousal labels

1 week pilot

100 Users

1K Songs

50K labels

Other sources of tags Games – Herd It "Herd It connects players in real-time and they listen to the same music clip. While they listen, various fun minigames quiz the players on opinions and trivia about the music. Players earn points based on the agreement of their answers with the group consensus." - Luke Barrington

Other sources of tags Games – Moose6 – A BBC Prototype

Pros:

Can tag at the phrase/clip level

Potentially can tag into the long tail

Potentially very high tagging rates

Issues:

Challenge to make addictive/fun game

Tags may be more superficial than social tags

Tags may be applied by non-fans

Lowest common denominator labels

Potential for 'gaming' the system

Don't collect many types of useful labels:

context ( jogging ), opinion ( favorite ), task organization ( check out ), social signaling ( seen live )

Other sources of tags Games (with a purpose)

Other sources of tags Experts/Surveys

Hire experts to hand label content

Advantages

Consistent labels

Strong labeling

Fixed, structured vocabulary

Disadvantages

Small pre-determined vocabulary

Very difficult to construct taxonomy

Human-labor intensive (expensive)

Doesn't scale to the long tail

Limited access to data

Other sources of tags Experts/Surveys – All Music Guide

Pop Tracks: ~13 million

Pop Albums: ~1.4 million

Classical Compositions: ~300 thousand

Biographies: ~84 thousand

Styles: 917

Themes: 86

Moods: 184

Other sources of tags Experts/Surveys – All Music Guide

Music labeled by paid professional musicians

Fixed vocabulary: ~500 weighted attributes

Structured vocabulary

Other sources of tags Experts/Surveys – Pandora

66 undergrads – paid $10 per hour to label music

1,700 annotations

500 'western popular' songs

135 musically-relevant concepts

Six semantic categories:

emotion – aggressive, weird, thrilling, mellow

genre - alternative, bebop, swing, rock, world

Instrument – guitar, organ, piano, harmonica

usage – at a party, at work, driving, exercising, sleeping

vocals – breathy, rapping, monotone, emotional

song – catchy, fast, danceable, texture acoustic

http://cosmal.ucsd.edu/cal Other sources of tags Experts/Surveys – CAL-500

Other sources of tags Web mining

Internet: the largest digital archive of music

Music blogs, reviews, forum discussions, official artist pages, MySpace, ...

Examples: (Google estimated number of results)

Eminem 46M pages

“The Beatles” 29M pages

“Wolfgang Amadeus Mozart” 7M pages

Unstructured data

Text information retrieval

Whitman & Lawrence (ICMC 2002)

Google: “<artist name>” +music +review

Retrieve text of top ranked pages

Text information retrieval techniques

Issues

Artist names are not unique identifiers

“The War”, “The The”, “Chicago”, “Pink” (not “P!nk”)

Solution: add information to query (e.g. album names)

Cold-start (needs at least one web-page)

Quality of “tags”

Include other artist names, seemingly random noise

Solution: use dictionary

Other sources of tags Web mining – Using Google et al.

ABBA

100 pop

56 song

50 musical

46 swedish

43 uk

41 group

36 band

35 dance

31 world

29 disco

Pampalk et al. (ECDL 2005) http://www.ofai.at/~elias.pampalk/wa/ Other sources of tags Web mining – Examples

Britney Spears

100 pop

99 song

52 girl

36 concert

33 dance

24 uk

20 sexy

18 jive

18 world

15 cool

Metallica

100 metal

43 band

38 concert

22 heavy metal

20 world

20 song

18 dance

15 hard

11 black metal

9 guitar

Ludwig van Beethoven

100 piano

67 symphony

53 classical

22 orchestra

20 movement

18 trio

18 solo

15 variations

15 musical

12 violin

[Knees et al., ACM MM 2006]

Other sources of tags Using Web Mined Tags: nepTune

[Pampalk & Goto, ISMIR 2006]

Other sources of tags Using Web Mined Tags: MusicRainbow

[Pampalk & Goto, ISMIR 2007]

Other sources of tags Using Web Mined Tags: MusicSun

Social Tags and Music Information Retrieval (Part I)

0 comments

Notes on slide 1

8 Favorites