Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Watson DevCon 2016 - Watson Beat: Making Music Cognitive

572 views

Published on

Presentation given by Janani Mukundan and Richard Daskas at Watson DevCon 2016.
As AI advances, scientists are pushing the boundaries of computational creativity. IBMers from Research are teaching Watson to compose original music based on human emotion. Using machine learning algorithms, the Researchers are teaching artificial neural networks to understand music theory, structure (pitch, time signature, key signature), and emotional intent to co-create music with a human partner. "Watson Beat" generates new and unique musical song combinations based on the beat played by the individual and the emotional tone the human chooses. Here’s how it works.

Published in: Technology
  • Be the first to comment

Watson DevCon 2016 - Watson Beat: Making Music Cognitive

  1. 1. IBM confidential ©IBM Corporation Watson Beat: New Cognitive Era
  2. 2. 2©IBM Corporation FutureCurrent
  3. 3. IBM Confidential | Do Not Distribute © 2015 IBM Corporation ▪ Why Music? ▪ Why unsupervised learning? ▪ Why Watson Beat? ▪ Representing music based on parameters is not very intuitive for most people ▪ For example: I like songs with 120 bpm, following B# major etc. ▪ Watson Beat provides a new way to query and compose new music based on reference tracks ▪ Ease of use: no requirement of prior knowledge, music theory etc. ▪ Provide input track(s) to our model, it gives you a new output track(s) Motivation: 3
  4. 4. IBM Confidential | Do Not Distribute © 2015 IBM Corporation ▪ Demo of potential application ▪ I am an amateur video game developer, and I need music for my game, and I like how “Game of Thrones” sounds ▪ Feed it through Watson Beat and get 100s of variations on same music ▪ Ability to steer compositions based on intent — slow, sad, fast, happy, vibrant Learning Music: Basic Idea Learned “Game Of Thrones” Input Track(s) Perturb model using creativity genes (known influences) Extract musical characteristics (pitch, rhythm, dynamics etc.) Reconstruct track by iteratively learning input musical characteristics with added perturbation Output Track(s) 4 Original “Game Of Thrones”
  5. 5. IBM Confidential | Do Not Distribute © 2015 IBM Corporation ▪ RBM: Stochastic Neural Net with ▪ one layer of visible units ▪ one layer of hidden units ▪ Learning RBMs: Contrastive Divergence ▪ DBN: Stack multiple Restricted Boltzmann Machines (RBM) Deep Belief Networks (DBN): 5 x: Visible layer h: Hidden Layer W: weights Input Vector Perturbed Input Vector unsupervised learning of NNs (RBMs, Auto encoder etc.) Output Vector RBM
  6. 6. IBM Confidential | Do Not Distribute © 2015 IBM Corporation ▪ Watson Beat Pandora station ▪ Suggest recreated version of songs that you like, you’ve been listening to etc. ▪ Producers, composers, music engineers create music based on intent (slow, fast, happy vibrant) ▪ Ability for retail stores, small businesses to create their own music based on original tracks ▪ Loop Pedal Mixing: pedal -> DJ Watson mixer -> amp ▪ https://www.youtube.com/watch?v=qX2eJsj9MiQ Applications: Cloud based cognitive music service 6
  7. 7. IBM Confidential | Do Not Distribute © 2015 IBM Corporation Backup Slides 7
  8. 8. IBM Confidential | Do Not Distribute © 2015 IBM Corporation Training RBMs 8
  9. 9. IBM Confidential | Do Not Distribute © 2015 IBM Corporation Recreate original Music using RBMs C# E B Time1/16 1/16 h: Hidden Layer (Holds extracted features of visible layer) x: Visible layer (Holds pitch information) 9 C E# B Time1/16 1/16 x~: Learned visible layer (Holds learned pitch information) p(h|x) p(x|h) Demo: Recreated “Mary had a little lamb”
  10. 10. Example 1: Create new music by adding perturbation 10 h: Hidden Layer (Holds extracted features of visible Demo: Learned “Mary” (less perturbation) Demo: Learned “Mary” (more perturbation) x: Visible layer (Holds initial pitch information) h1: Hidden Layer (Holds extracted features of visible C# rand B Time x: Visible layer (Holds perturbed pitch information) x~: Learned visible layer (Holds learned pitch information) p(h|x) p(x|h) E rand E# rand` B Time A rand` p1(h|x) C# E B Time Demo: Original “Mary had a little lamb”
  11. 11. 11 h: Hidden Layer (Holds extracted features of visible Demo: Spooky String Quartet x: Visible layer (Holds initial pitch information) h1: Hidden Layer (Holds extracted features of visible C# Oct B Time x: Visible layer (Holds perturbed pitch information) x~: Learned visible layer (Holds learned pitch information) p(h|x) p(x|h) E Oct E# Oct` B Time A Oct` p1(h|x) C# E B Time Demo: Original “String Quartet” Example 2: Create new music by steering learning based on emotional intent
  12. 12. Example 3: Create new music by learning from two songs and adding perturbation C# minor B Time Visible layer (Holds pitch + bias information) Hidden Layer (Holds extracted features of visible layer) minor E Song A D# Song B Weights 12 Demo: Learned “Willie Nelson” and ”Miley Cyrus”
  13. 13. 13 Original Adele Learned Adele “Vibrant version” Learned Adele “Mellow version” Example 4: Create new music by steering learning based on emotional intent
  14. 14. 14 x1: Visible layer for RBM1 x2: Visible layer for RBM2 h1: Hidden Layer for RBM 1 W1: weights for RBM1 x3: Visible Layer for RBM3 h2: hidden layer for RBM2 h3: hidden layer for RBM3 W2: weights for RBM2 W3: weights for RBM3

×