Janani Mukundan and Richard Daskas poster on RL for Rhythm generation in Watson Beat

1. https://soundcloud.com/jmukund/sets/watsonbeat-ml4creativity-2017
Watson Beat
Janani Mukundan • jmukund@us.ibm.com
Richard Daskas • daskas@us.ibm.com
Watson Beat, is a cognitive system that composes
original music based on its knowledge of music theory.
One of the underlying technologies used in Watson Beat
is reinforcement learning agent that can learn chords
and melodies.
Watson Beat also employs a deep belief network
(stacked RBMs) that can learn melodies.
Reinforcement learning (RL) studies how autonomous
agents situated in probabilistic environments learn to
maximize a long term goal.
Watson Beat: An unsupervised music
composition engine that fits into the
framework of an RL system.
• The composer is the RL agent.
• The environment can include
relevant state attributes like
(a) the current chord being played,
(b) the current scale being used,
(c) the tempo etc.
• The composer senses the state of the
system, chooses the next note to be
played, and is assigned a numerical
immediate reward.
• The goal of the composer is to
maximize an objective function
(value function) like
(a) generate a simple four bar chord
progression,
(b) generate a syncopated eight bar
melody etc.
Watson Beat RLAgentIntroduction
Background
• The basic structure of an RL system consists of a
stochastic agent and its environment.
• The RL agent interacts with its environment in
discrete time steps, senses its current state, performs
an action, is rewarded for the action, and moves to
another state.
• The objective of the RL agent is to maximize its
long-term cumulative reward by interacting with its
environment and learning an optimal policy that
maps states to actions.
RL Based Algorithms For Chord Generation And Melody Composition
Results
Conclusion
We introduce Watson Beat – a cognitive engine that composes music using the
principles of reinforcement learning and music theory. We detail two case studies
(a) chord generation, and (b) melody composition, based on these principles.
The ratio of chord tones to non-chord tones seen for each episode when training the
RL based melody generator described when the complexity of the melody is set to
(a) simple, and (b) semi-complex
IBM Research, Austin