The Echo Nest at Music and Bits, October 21 2009
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

The Echo Nest at Music and Bits, October 21 2009

on

  • 7,478 views

Brian Whitman talks about The Echo Nest's "machine listening" platform and the pitfalls and promise of music data.

Brian Whitman talks about The Echo Nest's "machine listening" platform and the pitfalls and promise of music data.

Statistics

Views

Total Views
7,478
Views on SlideShare
6,879
Embed Views
599

Actions

Likes
10
Downloads
63
Comments
0

6 Embeds 599

http://musicmachinery.com 339
http://notes.variogr.am 229
http://wiki.bcmoney-mobiletv.com 28
http://www.hanrss.com 1
http://www.slideshare.net 1
http://thomas.stenumgaard.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The Echo Nest at Music and Bits, October 21 2009 Presentation Transcript

  • 1. Thursday, October 22, 2009
  • 2. I am losing my voice. I am sorry. I am normally louder than this. I also added text to the pictures. Thursday, October 22, 2009
  • 3. A Short (Personal) History of Computers Listening to Music 1999-2009 Thursday, October 22, 2009
  • 4. I was a musician for a while. Electronic music. “Intelligent dance music” (worst genre name ever) Thursday, October 22, 2009
  • 5. Thursday, October 22, 2009
  • 6. “Fish / Cut bait” Handheld-music (1998-2001) I made my own software to make music Thursday, October 22, 2009
  • 7. “Fish / Cut bait” Handheld-music (1998-2001) Did it make me a better musician? Definitely not. Thursday, October 22, 2009
  • 8. It was 1999. Lots of stuff was happening. Thursday, October 22, 2009
  • 9. I learned about music from reading web sites. Forums, mailing lists. Thursday, October 22, 2009
  • 10. Thursday, October 22, 2009
  • 11. You could now download a song faster than real time. I figured things would change quick. Thursday, October 22, 2009
  • 12. So I went to grad school. I studied information retrieval, language processing. Thursday, October 22, 2009
  • 13. Columbia University, NYC MIT Media Lab finishing my dissertation Thursday, October 22, 2009
  • 14. People were starting to apply IR techniques to music. Audio files are treated like text. FFT frames became words Songs became “documents” Thursday, October 22, 2009
  • 15. Thursday, October 22, 2009
  • 16. There’s a problem with that. Just because you can convert an mp3 to #s doesn’t mean you understand it. Thursday, October 22, 2009
  • 17. “Music IR” was born. The applications are varied, but most have nothing to do with music. Thursday, October 22, 2009
  • 18. Retrieving Music by Rhythmic Similarity 1.5 110 bpm 112 bpm 1 squared Euclidean distance 114 bpm 120 bpm 122 bpm 116 bpm 124 bpm 0.5 126 bpm 128 bpm 130 bpm 0 130 128 126 124 122 120 118 116 114 112 110 Tempo (bpm) Figure 5. Euclidean Distance vs. Tempo Thursday, October 22, 2009
  • 19. The worst offender: “Genre Identification” Countless PhDs on this useless task. Trying to teach a computer a marketing construct. Thursday, October 22, 2009
  • 20. Show of hands: Is Bjork “electronic, pop, jazz”? Thursday, October 22, 2009
  • 21. At MIT I convinced someone to buy lots of computers Thursday, October 22, 2009
  • 22. Thursday, October 22, 2009
  • 23. And tried to figure out how to get music into music analysis Thursday, October 22, 2009
  • 24. Simple things like detecting holiday music is very hard. Thursday, October 22, 2009
  • 25. I decided if I could get a computer to make holiday music, We could claim we understand it. Thursday, October 22, 2009
  • 26. This is automatically generated holiday music Music Acquisition (2001-) based on listening to 1,000 Christmas songs Thursday, October 22, 2009
  • 27. It should be a funny joke that you can run statistics of millions of things and “understand it.” Thursday, October 22, 2009
  • 28. Thursday, October 22, 2009
  • 29. I built Eigenradio in 2003 to show people What computers hear when they hear music Thursday, October 22, 2009
  • 30. Thursday, October 22, 2009
  • 31. There’s obviously so much more to music than the audio signal and that other stuff is probably more important Thursday, October 22, 2009
  • 32. My brother makes music with sine waves and nothing else and gets a 9.7 on Pitchfork. This is fascinating! Thursday, October 22, 2009
  • 33. My brother makes music with sine waves and nothing else and gets a 9.7 on Pitchfork. This is fascinating! Were the sine waves that good? Thursday, October 22, 2009
  • 34. Review Regression Thursday, October 22, 2009
  • 35. It turns out if you understand language and audio at the same time you start learning a lot more. Thursday, October 22, 2009
  • 36. Here we predict ratings on All Music Guide and Pitchfork By listening to the audio and reading about the artist. Thursday, October 22, 2009
  • 37. Audio alone was terrible Text alone was better than audio Both together were the best. Thursday, October 22, 2009
  • 38. 100 100 12 10 80 80 Pitchfork Ratings Pitchfork Ratings 10 8 60 8 60 6 40 6 40 4 4 20 20 2 2 2 4 6 8 2 4 6 8 AMG Ratings Randomly selected AMG Ratings 100 .147 120 .127 6 Audio−derived Ratings Audio−derived Ratings 8 80 6 [.080] 100 80 60 [.082] 5 4 60 3 4 40 40 2 2 20 20 1 2 4 6 8 20 40 60 80 100 AMG Ratings Pitchfork Ratings Thursday, October 22, 2009
  • 39. I became interested in more ridiculous questions: “Can we find the saddest song in the world?” Thursday, October 22, 2009
  • 40. Thursday, October 22, 2009
  • 41. So I started a company in 2005 with my co-founder Tristan, also at the Lab. Thursday, October 22, 2009
  • 42. Tristan is a DSP “machine listening” expert and I handled the text side Thursday, October 22, 2009
  • 43. MAGIC Thursday, October 22, 2009
  • 44. Why does the Echo Nest exist? Thursday, October 22, 2009
  • 45. The best music experience is still very manual. I am still reading about music, not using a recommender. Thursday, October 22, 2009
  • 46. Thursday, October 22, 2009
  • 47. Thursday, October 22, 2009
  • 48. & the act of listening to music is easier than ever Thursday, October 22, 2009
  • 49. Thursday, October 22, 2009
  • 50. But data is hard. Most designers make very bad decisions because their tools are inefficient. Thursday, October 22, 2009
  • 51. Collaborative filtering (X who did Y also did Z) is so easy to make; but it’s also so terrible. Thursday, October 22, 2009
  • 52. Collaborative filtering (X who did Y also did Z) is so easy to make; but it’s also so terrible. The SQL join is destroying music. Thursday, October 22, 2009
  • 53. Thursday, October 22, 2009
  • 54. Thursday, October 22, 2009
  • 55. Thursday, October 22, 2009
  • 56. Thursday, October 22, 2009
  • 57. In 2005 we modeled the worst case scenario: In which collaborative filtering was the only way for an artist to get noticed. The popular ones would eat the unknown ones alive. 3 sets of 3 artists each remained. Thursday, October 22, 2009
  • 58. Set A Set B Set C Britney Spears Alice in Chains Chris Isaak Backstreet Boys Korn Bob Dylan Cristina Aguilera Faith no More Crowded House Thursday, October 22, 2009
  • 59. So the Echo Nest gives everyone great data. They can decide on their own how to show it. Thursday, October 22, 2009
  • 60. The Echo Nest 2005 Somerville, MA USA 2 people 2 computers Lots of ideas 1m documents 10,000 artists 100,000 songs 0 public facing sites Thursday, October 22, 2009
  • 61. The Echo Nest 2009 Somerville, MA USA 20 people 200 computers Lots of products 5bn documents 1,000,000 artists many millions of songs 0 public facing sites Thursday, October 22, 2009
  • 62. What We Do Thursday, October 22, 2009
  • 63. “Know everything about music and listeners.” Thursday, October 22, 2009
  • 64. “Know everything about music and listeners.” “Give (and sell) great data to everyone.” Thursday, October 22, 2009
  • 65. “Know everything about music and listeners.” “Give (and sell) great data to everyone.” “Do it automatically with no bias, on everything.” Thursday, October 22, 2009
  • 66. Customers Crawling Code NLP DSP Machine Learning Thursday, October 22, 2009
  • 67. Artist Data Song Data Listener Data • Tag Clouds • Similar Songs • demographics • Similar Artists • Tempo - age, gender, location • Analytics • Key • psychographics Familiarity Mode - preferences, lifestyle • • • music preference • Hotttnesss • Time Signature Blogs Beats • listening patterns • • News Downbeats • tastemaker profiling • • - writers, bloggers • Reviews • Segments • Audio • Timbre • Video • Pitch • Profile Sites • Loudness • Misspellings • Sections • Aliases Thursday, October 22, 2009
  • 68. We have a lot of data and we have a lot of products. We sell mostly to social networks, labels; video games; PR firms; musicians Thursday, October 22, 2009
  • 69. Artist metrics Acoustic Similarity analysis Feeds Remix Metadata Search / Tags Predictive Recommendation analytics Thursday, October 22, 2009
  • 70. The reason we are special is 2 things: Scale and Platform Thursday, October 22, 2009
  • 71. Our scale is limitless. We have hundreds of computers We always do our computation on everything. We can learn about new music very quickly. Thursday, October 22, 2009
  • 72. Scale All Music Guide Pandora The Echo Nest known artists 280,000 80,000 1,000,000 years to get there 18 8 1 time to understand 1 week 1 day <1 minute one album cost to understand $400 $40 $0.001 one album Thursday, October 22, 2009
  • 73. Our platform is huge. We have thousands of “free” developers using our API Our customers use the same platform So do we. Thursday, October 22, 2009
  • 74. Platform Thursday, October 22, 2009
  • 75. We sell two main products: Fanalytics is a predictive analytics toolset for artists The Knowledge is a dynamic metadata service (recommendation, feeds, data) for web sites Thursday, October 22, 2009
  • 76. Fanalytics lets artists and labels get a view into the world of online music We recommend blogs for artists We show predicted analytics on activity Thursday, October 22, 2009
  • 77. Predictive analytics Artist metrics Thursday, October 22, 2009
  • 78. We also maintain a popular open source remixing community and code base so people can make awesome free mashups, remixes, web sites using our tech Not much of a business but we love it. Thursday, October 22, 2009
  • 79. Remix Thursday, October 22, 2009
  • 80. “DonkDJ.com” was made using Remix It automatically “donks” (ask someone what this means) any song you upload Thursday, October 22, 2009
  • 81. Thursday, October 22, 2009
  • 82. Morecowbell.dj adds cowbell to any song This Is My Jam was a pre-Muxtape (by one day) mixtape sharing site that only let you use 30s samples and made a total mess of the output. Like I said, not much of a business. Thursday, October 22, 2009
  • 83. Thursday, October 22, 2009
  • 84. Thursday, October 22, 2009
  • 85. We also have artists using Remix -- our data is now powering some next generation electronic music Thursday, October 22, 2009
  • 86. I’ve always wanted to hear Michael Jackson trying to sing Amerie’s “One Thing” automatically by comparing timbre, pitch and loudness distances. -B.L. Thursday, October 22, 2009
  • 87. James Brown... FOREVER. Thursday, October 22, 2009
  • 88. Remix also works on video Thursday, October 22, 2009
  • 89. Let’s hear Daft Punk’s “Revolution 909” played by a fight scene from Undefeatable! -Y.A. Thursday, October 22, 2009
  • 90. Our analysis data powers a lot of visualizers and video games (rhythm games on your own MP3s) Thursday, October 22, 2009
  • 91. Acoustic analysis Thursday, October 22, 2009
  • 92. The Knowledge is a much better music data service Customers can subscribe to constantly-updated similarity, metadata, feeds, recommendations, etc Thursday, October 22, 2009
  • 93. Our similarity and recommendation data is some of the best, because we use so many sources and we know about all artists even if they are tiny Thursday, October 22, 2009
  • 94. Similarity Feeds Thursday, October 22, 2009
  • 95. Since our similarity is based on so many features: popularity, audio analysis, text analysis, structured metadata, influences, ... Thursday, October 22, 2009
  • 96. Since our similarity is based on so many features: popularity, audio analysis, text analysis, structured metadata, influences, ... We provide our customers with the knobs and let them decide what is important for the task. Thursday, October 22, 2009
  • 97. Since our similarity is based on so many features: popularity, audio analysis, text analysis, structured metadata, influences, ... We provide our customers with the knobs and let them decide what is important for the task. We do not give a “single answer.” There is no single answer. Thursday, October 22, 2009
  • 98. Similarity Thursday, October 22, 2009
  • 99. We can build paths between artists on any vector Thursday, October 22, 2009
  • 100. Acoustic Similarity analysis Search / Tags Thursday, October 22, 2009
  • 101. Our future: Thursday, October 22, 2009
  • 102. 1. Listener analytics Thursday, October 22, 2009
  • 103. We’ve been running large scale data mining on millions of listeners to help with analytics, for example a gender predictor based on your music taste Thursday, October 22, 2009
  • 104. Here’s the basis vectors; strongest correlators of gender: Thursday, October 22, 2009
  • 105. Male Female Pet Shop Boys Eternal Fort Minor Metro Station Justice Gackt Mike Oldfield Paolo Nutini U2 London after Midnight Thursday, October 22, 2009
  • 106. 2. More musicians to use our remix tools Thursday, October 22, 2009
  • 107. (I’ve noticed the better you are with computers, the worse your music is. This may just be me) Thursday, October 22, 2009
  • 108. 100% 75% Music goodness 50% 25% 0% nothing not much a little somewhat pretty good expert dork prime Computers know-how Thursday, October 22, 2009
  • 109. 3. Search anything APIs Thursday, October 22, 2009
  • 110. We will soon make all of our acoustic data available for searching and browsing (right now it has to be your content): “Find me a drum hit in this collection that sounds like the break in ‘Single Ladies’” Thursday, October 22, 2009
  • 111. Thursday, October 22, 2009
  • 112. Combined with Remix this will allow anyone to compose music that uses all music in the world Thursday, October 22, 2009
  • 113. >> from echonest import search >> segments = search.query(“voice”, soundsLike=”bjork”, pitch=”F#”) >> len(segments) 65706 >> new_song = random.shuffle(segments).write(“bjork2009.mp3”) Thursday, October 22, 2009
  • 114. To wrap up: 1. Don’t trust computers Thursday, October 22, 2009
  • 115. To wrap up: 1. Don’t trust computers 2. But trust us, really Thursday, October 22, 2009
  • 116. To wrap up: 1. Don’t trust computers 2. But trust us, really 3. Sorry I can’t speak very well Thursday, October 22, 2009
  • 117. Thursday, October 22, 2009