The Echo Nest at Music and Bits, October 21 2009

Thursday, October 22, 2009

I am losing my voice. I am sorry. I am normally louder than this. I also added text to the pictures. Thursday, October 22, 2009

A Short (Personal) History of Computers Listening to Music 1999-2009 Thursday, October 22, 2009

I was a musician for a while. Electronic music. “Intelligent dance music” (worst genre name ever) Thursday, October 22, 2009

“Fish / Cut bait” Handheld-music (1998-2001) I made my own software to make music Thursday, October 22, 2009

“Fish / Cut bait” Handheld-music (1998-2001) Did it make me a better musician? Deﬁnitely not. Thursday, October 22, 2009

It was 1999. Lots of stuff was happening. Thursday, October 22, 2009

I learned about music from reading web sites. Forums, mailing lists. Thursday, October 22, 2009

You could now download a song faster than real time. I ﬁgured things would change quick. Thursday, October 22, 2009

So I went to grad school. I studied information retrieval, language processing. Thursday, October 22, 2009

Columbia University, NYC MIT Media Lab ﬁnishing my dissertation Thursday, October 22, 2009

People were starting to apply IR techniques to music. Audio ﬁles are treated like text. FFT frames became words Songs became “documents” Thursday, October 22, 2009

There’s a problem with that. Just because you can convert an mp3 to #s doesn’t mean you understand it. Thursday, October 22, 2009

“Music IR” was born. The applications are varied, but most have nothing to do with music. Thursday, October 22, 2009

Retrieving Music by Rhythmic Similarity 1.5 110 bpm 112 bpm 1 squared Euclidean distance 114 bpm 120 bpm 122 bpm 116 bpm 124 bpm 0.5 126 bpm 128 bpm 130 bpm 0 130 128 126 124 122 120 118 116 114 112 110 Tempo (bpm) Figure 5. Euclidean Distance vs. Tempo Thursday, October 22, 2009

The worst offender: “Genre Identiﬁcation” Countless PhDs on this useless task. Trying to teach a computer a marketing construct. Thursday, October 22, 2009

Show of hands: Is Bjork “electronic, pop, jazz”? Thursday, October 22, 2009

At MIT I convinced someone to buy lots of computers Thursday, October 22, 2009

And tried to ﬁgure out how to get music into music analysis Thursday, October 22, 2009

Simple things like detecting holiday music is very hard. Thursday, October 22, 2009

I decided if I could get a computer to make holiday music, We could claim we understand it. Thursday, October 22, 2009

This is automatically generated holiday music Music Acquisition (2001-) based on listening to 1,000 Christmas songs Thursday, October 22, 2009

It should be a funny joke that you can run statistics of millions of things and “understand it.” Thursday, October 22, 2009

I built Eigenradio in 2003 to show people What computers hear when they hear music Thursday, October 22, 2009

There’s obviously so much more to music than the audio signal and that other stuff is probably more important Thursday, October 22, 2009

My brother makes music with sine waves and nothing else and gets a 9.7 on Pitchfork. This is fascinating! Thursday, October 22, 2009

My brother makes music with sine waves and nothing else and gets a 9.7 on Pitchfork. This is fascinating! Were the sine waves that good? Thursday, October 22, 2009

Review Regression Thursday, October 22, 2009

It turns out if you understand language and audio at the same time you start learning a lot more. Thursday, October 22, 2009

Here we predict ratings on All Music Guide and Pitchfork By listening to the audio and reading about the artist. Thursday, October 22, 2009

Audio alone was terrible Text alone was better than audio Both together were the best. Thursday, October 22, 2009

100 100 12 10 80 80 Pitchfork Ratings Pitchfork Ratings 10 8 60 8 60 6 40 6 40 4 4 20 20 2 2 2 4 6 8 2 4 6 8 AMG Ratings Randomly selected AMG Ratings 100 .147 120 .127 6 Audio−derived Ratings Audio−derived Ratings 8 80 6 [.080] 100 80 60 [.082] 5 4 60 3 4 40 40 2 2 20 20 1 2 4 6 8 20 40 60 80 100 AMG Ratings Pitchfork Ratings Thursday, October 22, 2009

I became interested in more ridiculous questions: “Can we ﬁnd the saddest song in the world?” Thursday, October 22, 2009

So I started a company in 2005 with my co-founder Tristan, also at the Lab. Thursday, October 22, 2009

Tristan is a DSP “machine listening” expert and I handled the text side Thursday, October 22, 2009

MAGIC Thursday, October 22, 2009

Why does the Echo Nest exist? Thursday, October 22, 2009

The best music experience is still very manual. I am still reading about music, not using a recommender. Thursday, October 22, 2009

& the act of listening to music is easier than ever Thursday, October 22, 2009

But data is hard. Most designers make very bad decisions because their tools are inefﬁcient. Thursday, October 22, 2009

Collaborative ﬁltering (X who did Y also did Z) is so easy to make; but it’s also so terrible. Thursday, October 22, 2009

Collaborative ﬁltering (X who did Y also did Z) is so easy to make; but it’s also so terrible. The SQL join is destroying music. Thursday, October 22, 2009

In 2005 we modeled the worst case scenario: In which collaborative ﬁltering was the only way for an artist to get noticed. The popular ones would eat the unknown ones alive. 3 sets of 3 artists each remained. Thursday, October 22, 2009

Set A Set B Set C Britney Spears Alice in Chains Chris Isaak Backstreet Boys Korn Bob Dylan Cristina Aguilera Faith no More Crowded House Thursday, October 22, 2009

So the Echo Nest gives everyone great data. They can decide on their own how to show it. Thursday, October 22, 2009

The Echo Nest 2005 Somerville, MA USA 2 people 2 computers Lots of ideas 1m documents 10,000 artists 100,000 songs 0 public facing sites Thursday, October 22, 2009

The Echo Nest 2009 Somerville, MA USA 20 people 200 computers Lots of products 5bn documents 1,000,000 artists many millions of songs 0 public facing sites Thursday, October 22, 2009

What We Do Thursday, October 22, 2009

“Know everything about music and listeners.” Thursday, October 22, 2009

“Know everything about music and listeners.” “Give (and sell) great data to everyone.” Thursday, October 22, 2009

“Know everything about music and listeners.” “Give (and sell) great data to everyone.” “Do it automatically with no bias, on everything.” Thursday, October 22, 2009

Customers Crawling Code NLP DSP Machine Learning Thursday, October 22, 2009

Artist Data Song Data Listener Data • Tag Clouds • Similar Songs • demographics • Similar Artists • Tempo - age, gender, location • Analytics • Key • psychographics Familiarity Mode - preferences, lifestyle • • • music preference • Hotttnesss • Time Signature Blogs Beats • listening patterns • • News Downbeats • tastemaker proﬁling • • - writers, bloggers • Reviews • Segments • Audio • Timbre • Video • Pitch • Proﬁle Sites • Loudness • Misspellings • Sections • Aliases Thursday, October 22, 2009

We have a lot of data and we have a lot of products. We sell mostly to social networks, labels; video games; PR ﬁrms; musicians Thursday, October 22, 2009

Artist metrics Acoustic Similarity analysis Feeds Remix Metadata Search / Tags Predictive Recommendation analytics Thursday, October 22, 2009

The reason we are special is 2 things: Scale and Platform Thursday, October 22, 2009

Our scale is limitless. We have hundreds of computers We always do our computation on everything. We can learn about new music very quickly. Thursday, October 22, 2009

Scale All Music Guide Pandora The Echo Nest known artists 280,000 80,000 1,000,000 years to get there 18 8 1 time to understand 1 week 1 day <1 minute one album cost to understand $400 $40 $0.001 one album Thursday, October 22, 2009

Our platform is huge. We have thousands of “free” developers using our API Our customers use the same platform So do we. Thursday, October 22, 2009

Platform Thursday, October 22, 2009

We sell two main products: Fanalytics is a predictive analytics toolset for artists The Knowledge is a dynamic metadata service (recommendation, feeds, data) for web sites Thursday, October 22, 2009

Fanalytics lets artists and labels get a view into the world of online music We recommend blogs for artists We show predicted analytics on activity Thursday, October 22, 2009

Predictive analytics Artist metrics Thursday, October 22, 2009

We also maintain a popular open source remixing community and code base so people can make awesome free mashups, remixes, web sites using our tech Not much of a business but we love it. Thursday, October 22, 2009

Remix Thursday, October 22, 2009

“DonkDJ.com” was made using Remix It automatically “donks” (ask someone what this means) any song you upload Thursday, October 22, 2009

Morecowbell.dj adds cowbell to any song This Is My Jam was a pre-Muxtape (by one day) mixtape sharing site that only let you use 30s samples and made a total mess of the output. Like I said, not much of a business. Thursday, October 22, 2009

We also have artists using Remix -- our data is now powering some next generation electronic music Thursday, October 22, 2009

I’ve always wanted to hear Michael Jackson trying to sing Amerie’s “One Thing” automatically by comparing timbre, pitch and loudness distances. -B.L. Thursday, October 22, 2009

James Brown... FOREVER. Thursday, October 22, 2009

Remix also works on video Thursday, October 22, 2009

Let’s hear Daft Punk’s “Revolution 909” played by a ﬁght scene from Undefeatable! -Y.A. Thursday, October 22, 2009

Our analysis data powers a lot of visualizers and video games (rhythm games on your own MP3s) Thursday, October 22, 2009

Acoustic analysis Thursday, October 22, 2009

The Knowledge is a much better music data service Customers can subscribe to constantly-updated similarity, metadata, feeds, recommendations, etc Thursday, October 22, 2009

Our similarity and recommendation data is some of the best, because we use so many sources and we know about all artists even if they are tiny Thursday, October 22, 2009

Similarity Feeds Thursday, October 22, 2009

Since our similarity is based on so many features: popularity, audio analysis, text analysis, structured metadata, inﬂuences, ... Thursday, October 22, 2009

Since our similarity is based on so many features: popularity, audio analysis, text analysis, structured metadata, inﬂuences, ... We provide our customers with the knobs and let them decide what is important for the task. Thursday, October 22, 2009

Since our similarity is based on so many features: popularity, audio analysis, text analysis, structured metadata, inﬂuences, ... We provide our customers with the knobs and let them decide what is important for the task. We do not give a “single answer.” There is no single answer. Thursday, October 22, 2009

Similarity Thursday, October 22, 2009

We can build paths between artists on any vector Thursday, October 22, 2009

Acoustic Similarity analysis Search / Tags Thursday, October 22, 2009

Our future: Thursday, October 22, 2009

1. Listener analytics Thursday, October 22, 2009

We’ve been running large scale data mining on millions of listeners to help with analytics, for example a gender predictor based on your music taste Thursday, October 22, 2009

Here’s the basis vectors; strongest correlators of gender: Thursday, October 22, 2009

Male Female Pet Shop Boys Eternal Fort Minor Metro Station Justice Gackt Mike Oldﬁeld Paolo Nutini U2 London after Midnight Thursday, October 22, 2009

2. More musicians to use our remix tools Thursday, October 22, 2009

(I’ve noticed the better you are with computers, the worse your music is. This may just be me) Thursday, October 22, 2009

100% 75% Music goodness 50% 25% 0% nothing not much a little somewhat pretty good expert dork prime Computers know-how Thursday, October 22, 2009

3. Search anything APIs Thursday, October 22, 2009

We will soon make all of our acoustic data available for searching and browsing (right now it has to be your content): “Find me a drum hit in this collection that sounds like the break in ‘Single Ladies’” Thursday, October 22, 2009

Combined with Remix this will allow anyone to compose music that uses all music in the world Thursday, October 22, 2009

>> from echonest import search >> segments = search.query(“voice”, soundsLike=”bjork”, pitch=”F#”) >> len(segments) 65706 >> new_song = random.shuffle(segments).write(“bjork2009.mp3”) Thursday, October 22, 2009

To wrap up: 1. Don’t trust computers Thursday, October 22, 2009

To wrap up: 1. Don’t trust computers 2. But trust us, really Thursday, October 22, 2009

To wrap up: 1. Don’t trust computers 2. But trust us, really 3. Sorry I can’t speak very well Thursday, October 22, 2009

http://musicmachinery.com	180
http://notes.variogr.am	176
http://www.hanrss.com	1
http://www.slideshare.net	1
http://thomas.stenumgaard.com	1

The Echo Nest at Music and Bits, October 21 2009

by Brian Whitman on Oct 26, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 359

Statistics

The Echo Nest at Music and Bits, October 21 2009 — Presentation Transcript