chord detection

1. Introduction to Audio Content Analysis
module 7.6: chord detection
alexander lerch

2. overview chords chord detection HMMs summary
introduction
overview
corresponding textbook section
section 7.6
lecture content
• musical chords and harmony
• baseline chord detection
• Hidden Markov Models (HMMs) and the Viterbi algorithm
learning objectives
• name basic chords and describe the concept of chord inversions
• discuss commonalities and differences between chord & key detection
• discuss the usefulness of HMMs for chord detection
• explain the Viterbi algorithm with an example
module 7.6: chord detection 1 / 19

3. overview chords chord detection HMMs summary
introduction
overview
corresponding textbook section
section 7.6
lecture content
• musical chords and harmony
• baseline chord detection
• Hidden Markov Models (HMMs) and the Viterbi algorithm
learning objectives
• name basic chords and describe the concept of chord inversions
• discuss commonalities and differences between chord & key detection
• discuss the usefulness of HMMs for chord detection
• explain the Viterbi algorithm with an example
module 7.6: chord detection 1 / 19

4. overview chords chord detection HMMs summary
musical pitch
chords
simultaneous use of several pitches ⇒ chords
usually constructed of (major/minor) thirds
note:
• chord type independent of pitch doubling, pitch order
• same label for keys and chords
module 7.6: chord detection 2 / 19

5. overview chords chord detection HMMs summary
musical pitch
chords
simultaneous use of several pitches ⇒ chords
usually constructed of (major/minor) thirds
note:
• chord type independent of pitch doubling, pitch order
• same label for keys and chords
module 7.6: chord detection 2 / 19

6. overview chords chord detection HMMs summary
musical pitch
chord inversion
most common: root note is lowest note
otherwise: chord inversion
module 7.6: chord detection 3 / 19

7. overview chords chord detection HMMs summary
musical pitch
harmony
key and tonal context define chord’s harmonic function
examples:
• tonic:
chord on 1st scale degree (tonal center)
• dominant:
chord on 5th scale degree (often moves to tonic)
• subdominant:
chord on 4th scale degree
• . . .
module 7.6: chord detection 4 / 19

8. overview chords chord detection HMMs summary
chord detection
introduction: key vs. chord detection
commonalities
• chords are octave independent ⇒ pitch chroma sufficient
• process flow: pitch chroma extraction + classification
differences
• time frame for pitch chroma calculation
• templates
• number of templates/chords
• many results per song (time series)
module 7.6: chord detection 5 / 19

9. overview chords chord detection HMMs summary
chord detection
introduction: key vs. chord detection
commonalities
• chords are octave independent ⇒ pitch chroma sufficient
• process flow: pitch chroma extraction + classification
differences
• time frame for pitch chroma calculation
• templates
• number of templates/chords
• many results per song (time series)
module 7.6: chord detection 5 / 19

10. overview chords chord detection HMMs summary
chord detection
introduction: overview
Pitch Representation
Extraction
Similarity
Computation
Chord Template
Definition
Time Sequence
Modeling
Chord Sequence
Chord Probabilities
module 7.6: chord detection 6 / 19

11. overview chords chord detection HMMs summary
pitch chroma
introduction
pitch class distribution: 12-dimensional vector
map all pitch class bands in all octaves to one
module 7.6: chord detection 7 / 19
matlab
source:
plotPitchChromaGrouping.m

12. overview chords chord detection HMMs summary
pitch chroma
introduction
pitch class distribution: 12-dimensional vector
map all pitch class bands in all octaves to one
module 7.6: chord detection 7 / 19
matlab
source:
plotPitchChroma.m

13. overview chords chord detection HMMs summary
pitch chroma
introduction
pitch class distribution: 12-dimensional vector
map all pitch class bands in all octaves to one
pitch chroma properties
no octave information
• no differentiation between prime and octave
• no info on inversion
robust, timbre-independent representation
module 7.6: chord detection 7 / 19

14. overview chords chord detection HMMs summary
chord detection
chord template
compare extracted pitch
chroma with template
• simplest possible template
and distance: linear
transformation
example — C major:
Γ(0, j) = [1/3, 0, 0, 0, 1/3, 0, 0, 1/3, 0, 0, 0, 0]
⇒ instantaneous chord
likelihood:
ψ(c, n) =
11
X
j=0
Γ(c, j) · ν(j, n)
module 7.6: chord detection 8 / 19
matlab
source:
plotChordTemplates.m

15. overview chords chord detection HMMs summary
chord detection
chord progression 1/2
apply musical knowledge to increase the result’s robustness and accuracy:
different probabilities for different chord progressions (similar to key modulations),
e.g.
• cadences: I-IV-V-I
• sequences: circle progression
⇒ model for chord progression probabilities
1 analytical model based on music theory
• circle of fifths (?!)
• key profile correlation (?!)
2 empirical model based on data
• annotate audio
• symbolic score
module 7.6: chord detection 9 / 19

16. overview chords chord detection HMMs summary
chord detection
chord progression 1/2
apply musical knowledge to increase the result’s robustness and accuracy:
different probabilities for different chord progressions (similar to key modulations),
e.g.
• cadences: I-IV-V-I
• sequences: circle progression
⇒ model for chord progression probabilities
1 analytical model based on music theory
• circle of fifths (?!)
• key profile correlation (?!)
2 empirical model based on data
• annotate audio
• symbolic score
module 7.6: chord detection 9 / 19

17. overview chords chord detection HMMs summary
chord detection
chord progression 1/2
apply musical knowledge to increase the result’s robustness and accuracy:
different probabilities for different chord progressions (similar to key modulations),
e.g.
• cadences: I-IV-V-I
• sequences: circle progression
⇒ model for chord progression probabilities
1 analytical model based on music theory
• circle of fifths (?!)
• key profile correlation (?!)
2 empirical model based on data
• annotate audio
• symbolic score
module 7.6: chord detection 9 / 19

18. overview chords chord detection HMMs summary
chord detection
chord progression 2/2
what properties do chord progression probabilities depend on
module 7.6: chord detection 10 / 19

19. overview chords chord detection HMMs summary
chord detection
chord progression 2/2
what properties do chord progression probabilities depend on
musical key
larger musical context (model order)
style
tempo/length??
module 7.6: chord detection 10 / 19

20. overview chords chord detection HMMs summary
chord detection
markov chain
two possible states E, A
transition probabilities to other state(s) and to self
sum of transition probabilities equals 1
module 7.6: chord detection 11 / 19
from:
https://commons.wikimedia.org/wiki/File:Markovkate_01.svg

21. overview chords chord detection HMMs summary
chord detection
hidden markov model: variables
states:
unknown/hidden
transition probability:
probability of transitioning from one state to the other
observations:
measureable time series
emission probability:
probability of an observation given a state
start probability:
probability of the initial state
module 7.6: chord detection 12 / 19

22. overview chords chord detection HMMs summary
chord detection
hidden markov model: variables
X: states
y: possible observations
a: state transition probabilities
b: emission probabilities
module 7.6: chord detection 13 / 19
from
https://en.wikipedia.org/wiki/File:HiddenMarkovModel.svg

23. overview chords chord detection HMMs summary
chord detection
hidden markov model: example (WP) 1/2
scenario
• doctor diagnoses fever by how patients feel
• patient may feel normal, dizzy, or cold
• patient visits multiple days in a row
what are the states and observations in this case
module 7.6: chord detection 14 / 19

24. overview chords chord detection HMMs summary
chord detection
hidden markov model: example (WP) 1/2
scenario
• doctor diagnoses fever by how patients feel
• patient may feel normal, dizzy, or cold
• patient visits multiple days in a row
what are the states and observations in this case
states
• healthy
• fever
observations:
• normal
• cold
• dizzy
module 7.6: chord detection 14 / 19

25. overview chords chord detection HMMs summary
chord detection
hidden markov model: example (WP) 2/2
start probabilities (initial state assumption)
• healthy: 0.6
• fever: 0.4
emission probabilities (prob of obs given state)
• healthy: normal 0.5, cold 0.4, dizzy 0.1
• fever: : normal 0.1, cold 0.3, dizzy 0.6
transition probabilities
• healthy: healthy 0.7, fever 0.3
• fever: : healthy 0.4, fever 0.6
module 7.6: chord detection 15 / 19

26. overview chords chord detection HMMs summary
chord detection
hidden markov model: example (WP) 2/2
dizzy normal
cold
p
1
=
0
.
6
p
2
=
0
.
4
start probabilities
a12=0.3
a21=0.4
a
1
1
=0.7
a
2
2
=
0
.
6
b
1
2
=
0
.
4 b13=0.5
b
1
1
=
0
.
1
b21=0.6
b
2
2
=
0
.
3
b
2
3
=
0
.
1
transition probabilities
emission probabilities
n states
k observations
module 7.6: chord detection 15 / 19

27. overview chords chord detection HMMs summary
chord detection
hidden markov model: example (WP) 2/2
three observations:
day 1 normal → day 2 cold → day 3 dizzy
module 7.6: chord detection 16 / 19

28. overview chords chord detection HMMs summary
chord detection
hidden markov model: example (WP) 2/2
three observations:
day 1 normal → day 2 cold → day 3 dizzy
module 7.6: chord detection 16 / 19

29. overview chords chord detection HMMs summary
chord detection
hidden markov model: example (WP) 2/2
three observations:
day 1 normal → day 2 cold → day 3 dizzy
module 7.6: chord detection 16 / 19

30. overview chords chord detection HMMs summary
chord detection
hidden markov model: example (WP) 2/2
three observations:
day 1 normal → day 2 cold → day 3 dizzy
module 7.6: chord detection 16 / 19

31. overview chords chord detection HMMs summary
chord detection
HMMs for chord detection
states → chords
observations → pitch chroma
emission probability → trained with pitch chroma
transition probability → trained from dataset
start probability → chord statistics (style dependent?)
module 7.6: chord detection 17 / 19

32. overview chords chord detection HMMs summary
chord detection
chord detection example
module 7.6: chord detection 18 / 19
matlab
source:
plotChordDetection.m

33. overview chords chord detection HMMs summary
summary
lecture content
chords
• combination of three or more pitches
• usually stacked thirds
• can be inverted
chord detection
• processing steps
▶ pitch chroma extraction
▶ template matching
▶ chord transition model
Viterbi algorithm
• find globally optimal path through state space
• estimate state sequence with
▶ emission probabilities
▶ transition probabilities
module 7.6: chord detection 19 / 19