Machine learning for Music
•
7 likes•2,943 views
This document discusses machine learning techniques for music information retrieval. It provides an overview of music recommendation systems like Spotify's shuffle mode and Pandora's music genome project. Key music information retrieval tasks are identified like genre recognition, mood detection, and audio similarity. Machine learning architectures for music information retrieval are examined including feature extraction from audio, classification with neural networks, and deep learning techniques like convolutional neural networks and autoencoders.
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (12)
Similar to Machine learning for Music
Similar to Machine learning for Music (13)
Recently uploaded
Recently uploaded (20)
Machine learning for Music
- 1. Machine Learning for Music Faculty of Mathematics and Informatics, SU Petko Nikolov April 8, 2015
- 2. About Me Machine Learning Music Information Retrieval Machine Learning / Automated Data Science
- 3. What’s Music Information Retrieval? Musicology Computer Science Signal Processing Machine Learning MIR
- 8. Spotify’s Shuffle Mode ● Not really random ● Certainly some processing ● Probably some MIR behind
- 9. Pandora’s Music Genome Project ● started in 2000 ● 800 000 manually annotated tracks by music experts ● 450 attributes to describe music ● 25 minutes per track to label
- 10. MIREX Music Information Retrieval Evaluation eXchange annual competition featuring more than 20 tasks state-of-the-art algorithms compete against each other
- 12. fingerprinting cover song detection genre recognition instrument recognition mood detection transcription playlist generation beat tracking key detection pitch tracking vocal detection recommendation audio similarity source separation
- 13. genre recognition instrument recognition mood detection vocal detection audio similarity
- 18. Segmentation and Preprocessing Feature Extraction Machine Learning classical piano romantic Bethoven by Daniel Barenboim 2 4 MIR Architecture Audio
- 19. Segmentation and Preprocessing classical piano romantic Bethoven Deep Learning by Daniel Barenboim 2 4 MIR Architecture Audio
- 20. Audio signal
- 21. Audio signal human hearing: 20 Hz to 20 KHz
- 22. Segmentation
- 28. Segmentation Frame 52 ms f1 f2 f3 f4
- 29. Segmentation Frame 52 ms f1 f2 f3 f4 fn
- 30. Spectrum - on frame level Discrete Fourier Transform (DFT) time frequency
- 32. Spectral Centroid where is the ‘center of mass’ of the spectrum
- 33. Spectral Slope fit linear regression and get the slope coef.
- 34. Spectral Slope fit linear regression and get the slope coef.
- 35. Spectral Slope fit linear regression and get the slope coef.
- 36. Spectral Slope fit linear regression and get the slope coef.
- 37. Spectral Correlation is the cosine distance between the frequency vectors of two consecutive frames Variation is (1.0 - correlation) respectively. Spectral Correlation / Variation
- 38. Feature extraction - Result f11 f12 f13 f14 f15 ……… f1m f21 f22 f23 f24 f25 ……… f2m centroid correlation Frames
- 39. Feature extraction - Result f11 f12 f13 f14 f15 ……… f1m f21 f22 f23 f24 f25 ……… f2m centroid correlation Frames frames number vary across audio recordings
- 41. Gaussian Mixture Model frame feature vector
- 42. Gaussian Mixture Model Multivariate Gaussian Distribution
- 45. Gaussian Mixture Model - per track
- 46. Gaussian Mixture Model - per track
- 47. Gaussian Mixture Model - per track
- 48. Gaussian Mixture Model - per track [𝛍1, 𝛍2, 𝛍3, 𝛍4]
- 49. Classification - Example Neural Net aik wk Feature vector Input Hidden Output Likelihood of Rock? Layers:
- 50. Classification - Example Neural Net aik wk Feature vector Input Hidden Output Likelihood of Rock? Layers:
- 51. Classification - Example Neural Net aik wk Feature vector Input Hidden Output Likelihood of Rock? Layers:
- 52. What’s Deep Learning? (defn deep-learning? [neural-net] (hidden-layer? neural-net)) we are trying to learn new high-level representation having many more hidden layers input is as raw as possible
- 53. Mel-spectrum
- 57. Deep Neural Network Backpropagation gradient fades quickly
- 58. Deep Belief Network Input (Mel spectrum) Output Hidden Layer 3 Hidden Layer 2 Hidden Layer 1Restricted Boltzmann Machine RBM RBM RBM Rock Jazz Punk Electronic
- 59. Deep Belief Network Input (Mel spectrum) Hidden Layer 1Restricted Boltzmann Machine
- 60. Deep Belief Network Input (Mel spectrum) Hidden Layer 1Restricted Boltzmann Machine
- 61. Deep Belief Network Input (Mel spectrum) Output Hidden Layer 3 Hidden Layer 2 Hidden Layer 1Restricted Boltzmann Machine RBM RBM RBM Rock Jazz Punk Electronic
- 62. Deep Auto Encoders Mel spectrum Mel spectrumOutput Input
- 63. Deep Auto Encoders Mel spectrum Mel spectrumOutput Input Used for denoising
- 64. Tools essentia - audio retrieval algorithms theano - CPU/GPU symbolic optimization scikit-learn - machine learning in Python