intensity and loudness features

1. Introduction to Audio Content Analysis
module 8.0: intensity
alexander lerch

2. overview intro perception music features summary
introduction
overview
corresponding textbook section
chapter 8
lecture content
• quick overview: human perception of loudness
• intensity related features
learning objectives
• discuss level and loudness
• list and describe typical intensity related low level features
module 8.0: intensity 1 / 15

3. overview intro perception music features summary
introduction
overview
corresponding textbook section
chapter 8
lecture content
• quick overview: human perception of loudness
• intensity related features
learning objectives
• discuss level and loudness
• list and describe typical intensity related low level features
module 8.0: intensity 1 / 15

4. overview intro perception music features summary
intensity, magnitude & loudness
introduction
intensity-related descriptors commonly used
• waveform view
• level monitoring (PPM, VU,. . . )
related terms: magnitude • intensity • envelope • level • volume • velocity • loudness
module 8.0: intensity 2 / 15

5. overview intro perception music features summary
intensity, magnitude & loudness
introduction
intensity-related descriptors commonly used
• waveform view
• level monitoring (PPM, VU,. . . )
related terms: magnitude • intensity • envelope • level • volume • velocity • loudness
module 8.0: intensity 2 / 15

6. overview intro perception music features summary
intensity, magnitude & loudness
human perception 1/2
perception has non-linear relation to magnitude/RMS:
model: logarithmic relation
vdB(n) = 20 · log10
v(n)
v0
• v0: reference constant (0 dB point)
▶ digital: v0 = 1 ⇒ dBFS
▶ sound pressure v0 = 20 · 10−6
⇒ dBSPL
• scaling factor: 1 dB ≈ JNDL for sound pressure level
module 8.0: intensity 3 / 15

7. overview intro perception music features summary
intensity, magnitude loudness
side note: level computation
if v(n) = 0 ⇒: computation of log10(0)
work-arounds
a add constant ϵ
vdB(n) = 20·log10(v(n)+ϵ)
b add if statement
vtrunc(n) =
v(n), if v(n) ≥ ϵ
ϵ, otherwise
module 8.0: intensity 4 / 15

8. overview intro perception music features summary
intensity, magnitude loudness
side note: level computation
if v(n) = 0 ⇒: computation of log10(0)
work-arounds
a add constant ϵ
vdB(n) = 20·log10(v(n)+ϵ)
b add if statement
vtrunc(n) =
v(n), if v(n) ≥ ϵ
ϵ, otherwise
module 8.0: intensity 4 / 15
matlab
source:
plotLogEpsilon.m

9. overview intro perception music features summary
intensity, magnitude loudness
side note: level computation
if v(n) = 0 ⇒: computation of log10(0)
work-arounds
a add constant ϵ
vdB(n) = 20·log10(v(n)+ϵ)
b add if statement
vtrunc(n) =
v(n), if v(n) ≥ ϵ
ϵ, otherwise
module 8.0: intensity 4 / 15

10. overview intro perception music features summary
intensity, magnitude loudness
human perception 2/2
decibel scale is not loudness scale:
• equal-sized steps on the decibel scale not perceived as equal-sized loudness steps
perceptual phenomenon loudness depends on
• frequency
• cochlear resolution
• masking effects
module 8.0: intensity 5 / 15

11. overview intro perception music features summary
intensity, magnitude loudness
human perception 2/2
decibel scale is not loudness scale:
• equal-sized steps on the decibel scale not perceived as equal-sized loudness steps
perceptual phenomenon loudness depends on
• frequency
• cochlear resolution
• masking effects
module 8.0: intensity 5 / 15

12. overview intro perception music features summary
intensity, magnitude loudness
human perception 2/2
module 8.0: intensity 5 / 15
matlab
source:
plotLoudnessWeighting.m

13. overview intro perception music features summary
intensity, magnitude loudness
dynamics in music
score:
• only several rough dynamic steps,e.g.:
pp, p, mf, f, ff
• comparably vague instructions on volume modifications, e.g.:
crescendo, decrescendo, sf
• dynamics influenced by
▶ instrumentation
▶ timbre
▶ number of voices
▶ context and musical tension
MIDI:
• 128 velocity steps
• no standardized relation to magnitude, power, . . .
module 8.0: intensity 6 / 15

14. overview intro perception music features summary
intensity, magnitude loudness
dynamics in music
score:
• only several rough dynamic steps,e.g.:
pp, p, mf, f, ff
• comparably vague instructions on volume modifications, e.g.:
crescendo, decrescendo, sf
• dynamics influenced by
▶ instrumentation
▶ timbre
▶ number of voices
▶ context and musical tension
MIDI:
• 128 velocity steps
• no standardized relation to magnitude, power, . . .
module 8.0: intensity 6 / 15

15. overview intro perception music features summary
intensity, magnitude loudness
features: root mean square 1/2
vRMS(n) =
v
u
u
u
t
1
K
ie(n)
X
i=is(n)
x(i)2
module 8.0: intensity 7 / 15

16. overview intro perception music features summary
intensity, magnitude loudness
features: root mean square 1/2
vRMS(n) =
v
u
u
u
t
1
K
ie(n)
X
i=is(n)
x(i)2
value of this feature for the hypothetical prototype signals
• silence
• sinusoidal (Amplitude A)
module 8.0: intensity 7 / 15

17. overview intro perception music features summary
intensity, magnitude loudness
features: root mean square 1/2
vRMS(n) =
v
u
u
u
t
1
K
ie(n)
X
i=is(n)
x(i)2
module 8.0: intensity 7 / 15
matlab
source:
matlab/plotFeatures.m

18. overview intro perception music features summary
intensity, magnitude loudness
features: root mean square 2/2
common variants (sample processing only):
reduce computational complexity
v2
RMS(n) =
x(ie(n))2
− x(is(n − 1))2
ie(n) − is(n) + 1
+ v2
RMS(n − 1)
vRMS(n) =
q
v2
RMS(n)
single pole approximation
vtmp(i) = α · vtmp(i − 1) + (1 − α) · x(i)2
v∗
RMS(i) =
p
vtmp(i)
module 8.0: intensity 8 / 15

19. overview intro perception music features summary
intensity, magnitude loudness
features: root mean square 2/2
common variants (sample processing only):
reduce computational complexity
v2
RMS(n) =
x(ie(n))2
− x(is(n − 1))2
ie(n) − is(n) + 1
+ v2
RMS(n − 1)
vRMS(n) =
q
v2
RMS(n)
single pole approximation
vtmp(i) = α · vtmp(i − 1) + (1 − α) · x(i)2
v∗
RMS(i) =
p
vtmp(i)
module 8.0: intensity 8 / 15

20. overview intro perception music features summary
intensity, magnitude loudness
features: weighted root mean square
x(i)
H(z) RMS
v(n)
H(z):
A, B, C weighting
RLB (BS.1770)
. . .
module 8.0: intensity 9 / 15

21. overview intro perception music features summary
intensity, magnitude loudness
features: weighted root mean square
x(i)
H(z) RMS
v(n)
H(z):
A, B, C weighting
RLB (BS.1770)
. . .
module 8.0: intensity 9 / 15
matlab
source:
plotLoudnessWeighting.m

22. overview intro perception music features summary
intensity, magnitude loudness
features: peak envelope (max)
vPeak(n) = max
is(n)≤i≤ie(n)
|x(i)|
module 8.0: intensity 10 / 15

23. overview intro perception music features summary
intensity, magnitude loudness
features: peak envelope (max)
vPeak(n) = max
is(n)≤i≤ie(n)
|x(i)|
module 8.0: intensity 10 / 15
matlab
source:
matlab/plotFeatures.m

24. overview intro perception music features summary
intensity, magnitude loudness
features: peak envelope (PPM) 1/2
⊕ ⊗ ⊕ ⊕
z−1
⊗
x(i) |x(i)|
−
αAT
λ
−
vPPM(i)
module 8.0: intensity 11 / 15

25. overview intro perception music features summary
intensity, magnitude loudness
features: peak envelope (PPM) 1/2
⊕ ⊗ ⊕ ⊕
z−1
⊗
x(i) |x(i)|
−
αAT
λ
−
vPPM(i)
release state (|x(i)| vPPM(i − 1) ⇒ λ = αRT)
module 8.0: intensity 11 / 15

26. overview intro perception music features summary
intensity, magnitude loudness
features: peak envelope (PPM) 1/2
⊕ ⊗ ⊕ ⊕
z−1
⊗
x(i) |x(i)|
−
αAT
λ
−
vPPM(i)
release state (|x(i)| vPPM(i − 1) ⇒ λ = αRT)
vPPM(i) = vPPM(i − 1) − αRT · vPPM(i − 1)
= (1 − αRT) · vPPM(i − 1)
module 8.0: intensity 11 / 15

27. overview intro perception music features summary
intensity, magnitude loudness
features: peak envelope (PPM) 1/2
⊕ ⊗ ⊕ ⊕
z−1
⊗
x(i) |x(i)|
−
αAT
λ
−
vPPM(i)
attack state (|x(i)| ≥ vPPM(i − 1) ⇒ λ = 0)
module 8.0: intensity 11 / 15

28. overview intro perception music features summary
intensity, magnitude loudness
features: peak envelope (PPM) 1/2
⊕ ⊗ ⊕ ⊕
z−1
⊗
x(i) |x(i)|
−
αAT
λ
−
vPPM(i)
attack state (|x(i)| ≥ vPPM(i − 1) ⇒ λ = 0)
vPPM(i) = αAT · |x(i)| − vPPM(i − 1)
+ vPPM(i − 1)
= αAT · |x(i)| + (1 − αAT) · vPPM(i − 1)
module 8.0: intensity 11 / 15

29. overview intro perception music features summary
intensity, magnitude loudness
features: peak envelope (PPM) 2/2
gold: max per block
blue: PPM
module 8.0: intensity 12 / 15
matlab
source:
matlab/plotFeatures.m

30. overview intro perception music features summary
intensity, magnitude loudness
features: zwicker loudness
Stimulus Outer Ear
Transfer
Function
Excitation
Patterns
Specific
Loudness
Overall
Loudness
vLoud
module 8.0: intensity 13 / 15

31. overview intro perception music features summary
intensity, magnitude loudness
features: zwicker loudness
Stimulus Outer Ear
Transfer
Function
Excitation
Patterns
Specific
Loudness
Overall
Loudness
vLoud
outer ear transfer function1
1
D. Hammershøi and H. Møller, “Methods for Binaural Recording and Reproduction,” Acta Acustica united with Acustica, vol. 88, no. 3,
pp. 303–311, May 2002.
module 8.0: intensity 13 / 15

32. overview intro perception music features summary
intensity, magnitude loudness
features: zwicker loudness
Stimulus Outer Ear
Transfer
Function
Excitation
Patterns
Specific
Loudness
Overall
Loudness
vLoud
excitation patterns1
1
M. Schleske, Vibrato of the musician, [Online]. Available:
http://www.schleske.de/en/our-research/handbook-violinacoustics/vibrato-of-the-musician.html (visited on 07/29/2015).
module 8.0: intensity 13 / 15

33. overview intro perception music features summary
intensity, magnitude loudness
features: zwicker loudness
Stimulus Outer Ear
Transfer
Function
Excitation
Patterns
Specific
Loudness
Overall
Loudness
vLoud
specific loudness1
1
U. of Salford, Customised metrics, [Online]. Available:
https://www.salford.ac.uk/computing-science-engineering/research/acoustics/psychoacoustics/sound-quality-making-products-
sound-better/sound-quality-testing/customised-metrics (visited on 07/29/2015).
module 8.0: intensity 13 / 15

34. overview intro perception music features summary
intensity, magnitude loudness
features: zwicker loudness
Stimulus Outer Ear
Transfer
Function
Excitation
Patterns
Specific
Loudness
Overall
Loudness
vLoud
overall loudness
vloud =
X
∀i
zi
module 8.0: intensity 13 / 15

35. overview intro perception music features summary
intensity, magnitude loudness
derived features
number or ratio of pauses
dynamic range
other statistical features from (RMS) histogram
. . .
module 8.0: intensity 14 / 15

36. overview intro perception music features summary
summary
lecture content
loudness perception
• nonlinear relation to magnitude or power
• depends also on frequency, level, and signal (masking)
typical features
• derived from envelope (peak, RMS, weighted RMS)
• derived from histogram (range, mode)
module 8.0: intensity 15 / 15