Event-Driven Architecture Masterclass: Challenges in Stream Processing
03-01-ACA-Input-Signals.pdf
1. Introduction to Audio Content Analysis
module 3.1: input representation — signals
alexander lerch
2. overview intro periodic signals random signals signal description summary
introduction
overview
corresponding textbook section
section 3.1
lecture content
• deterministic & periodic signals
• Fourier Series
• random signals
• statistical signal description
• digital signals
learning objectives
• name basic signal categories
• discuss the nature of periodic signals with respect to harmonics
• give a short description of meaning and use of the Fourier Series
• list common descriptors for properties of a random signal
module 3.1: input representation — signals 1 / 18
3. overview intro periodic signals random signals signal description summary
introduction
overview
corresponding textbook section
section 3.1
lecture content
• deterministic & periodic signals
• Fourier Series
• random signals
• statistical signal description
• digital signals
learning objectives
• name basic signal categories
• discuss the nature of periodic signals with respect to harmonics
• give a short description of meaning and use of the Fourier Series
• list common descriptors for properties of a random signal
module 3.1: input representation — signals 1 / 18
4. overview intro periodic signals random signals signal description summary
audio signals
signal categories
deterministic signals:
predictable: future shape of the signal can be known (example: sinusoidal)
random signals:
unpredictable: no knowledge can help to predict what is coming next (example:
white noise)
“real-world” audio signals can be modeled as time-variant combination of
(quasi-)periodic parts
(quasi-)random parts
module 3.1: input representation — signals 2 / 18
5. overview intro periodic signals random signals signal description summary
audio signals
signal categories
deterministic signals:
predictable: future shape of the signal can be known (example: sinusoidal)
random signals:
unpredictable: no knowledge can help to predict what is coming next (example:
white noise)
“real-world” audio signals can be modeled as time-variant combination of
(quasi-)periodic parts
(quasi-)random parts
module 3.1: input representation — signals 2 / 18
6. overview intro periodic signals random signals signal description summary
audio signals
signal categories
deterministic signals:
predictable: future shape of the signal can be known (example: sinusoidal)
random signals:
unpredictable: no knowledge can help to predict what is coming next (example:
white noise)
“real-world” audio signals can be modeled as time-variant combination of
(quasi-)periodic parts
(quasi-)random parts
module 3.1: input representation — signals 2 / 18
9. overview intro periodic signals random signals signal description summary
audio signals
periodic signals 2/5
periodic signal ⇒ representation in Fourier series1
x(t) =
∞
X
k=−∞
akejω0kt
ω0 = 2π · f0
kω0: integer multiples of the lowest frequency
ejω0kt = cos(ω0kt) + j sin(ω0kt)
ak: Fourier coefficients — amplitude of each component
ak =
1
T0
T0/2
Z
−T0/2
x(t)e−jω0kt
dt
1
Jean-Baptiste Joseph Fourier, 1768–1830
module 3.1: input representation — signals 4 / 18
10. overview intro periodic signals random signals signal description summary
audio signals
periodic signals 2/5
periodic signal ⇒ representation in Fourier series1
x(t) =
∞
X
k=−∞
akejω0kt
ω0 = 2π · f0
kω0: integer multiples of the lowest frequency
ejω0kt = cos(ω0kt) + j sin(ω0kt)
ak: Fourier coefficients — amplitude of each component
ak =
1
T0
T0/2
Z
−T0/2
x(t)e−jω0kt
dt
1
Jean-Baptiste Joseph Fourier, 1768–1830
module 3.1: input representation — signals 4 / 18
11. overview intro periodic signals random signals signal description summary
audio signals
periodic signals 2/5
periodic signal ⇒ representation in Fourier series1
x(t) =
∞
X
k=−∞
akejω0kt
ω0 = 2π · f0
kω0: integer multiples of the lowest frequency
ejω0kt = cos(ω0kt) + j sin(ω0kt)
ak: Fourier coefficients — amplitude of each component
ak =
1
T0
T0/2
Z
−T0/2
x(t)e−jω0kt
dt
1
Jean-Baptiste Joseph Fourier, 1768–1830
module 3.1: input representation — signals 4 / 18
12. overview intro periodic signals random signals signal description summary
audio signals
periodic signals 2/5
periodic signal ⇒ representation in Fourier series1
x(t) =
∞
X
k=−∞
akejω0kt
ω0 = 2π · f0
kω0: integer multiples of the lowest frequency
ejω0kt = cos(ω0kt) + j sin(ω0kt)
ak: Fourier coefficients — amplitude of each component
ak =
1
T0
T0/2
Z
−T0/2
x(t)e−jω0kt
dt
1
Jean-Baptiste Joseph Fourier, 1768–1830
module 3.1: input representation — signals 4 / 18
13. overview intro periodic signals random signals signal description summary
audio signals
periodic signals 2/5
periodic signal ⇒ representation in Fourier series1
x(t) =
∞
X
k=−∞
akejω0kt
ω0 = 2π · f0
kω0: integer multiples of the lowest frequency
ejω0kt = cos(ω0kt) + j sin(ω0kt)
ak: Fourier coefficients — amplitude of each component
ak =
1
T0
T0/2
Z
−T0/2
x(t)e−jω0kt
dt
1
Jean-Baptiste Joseph Fourier, 1768–1830
module 3.1: input representation — signals 4 / 18
14. overview intro periodic signals random signals signal description summary
audio signals
periodic signals 3/5
Fourier series
every periodic signal can be represented in a Fourier series
a periodic signal contains only frequencies at integer multiples of the
fundamental frequency f0
Fourier series can only be applied to periodic signals
Fourier series is analytically elegant but only of limited practical use as the
fundamental period has to be known
module 3.1: input representation — signals 5 / 18
15. overview intro periodic signals random signals signal description summary
audio signals
periodic signals 3/5
Fourier series
every periodic signal can be represented in a Fourier series
a periodic signal contains only frequencies at integer multiples of the
fundamental frequency f0
Fourier series can only be applied to periodic signals
Fourier series is analytically elegant but only of limited practical use as the
fundamental period has to be known
module 3.1: input representation — signals 5 / 18
16. overview intro periodic signals random signals signal description summary
audio signals
periodic signals 4/5
reconstruction of periodic signals with limited number of sinusoidals:
x̂(t) =
K
X
k=−K
akejω0kt
module 3.1: input representation — signals 6 / 18
matlab
source:
plotFourierSeries.m
28. overview intro periodic signals random signals signal description summary
audio signals
random process 1/2
random process: ensemble of random series
module 3.1: input representation — signals 8 / 18
matlab
source:
plotRandomProcess.m
29. overview intro periodic signals random signals signal description summary
audio signals
random process 2/2
random process
ensemble of random series
each series represents a sample of the process
the following value is indetermined, regardless of any amount of knowledge
special case: stationarity
statistical properties such as the mean are time invariant
example: white noise
module 3.1: input representation — signals 9 / 18
30. overview intro periodic signals random signals signal description summary
statistical signal description
probability density function
PDF px (x)
x-axis: possible (amplitude) values
y-axis: probability
px (x) ≥ 0, and
∞
Z
−∞
px (x) dx = 1
RFD—Relative Frequency Distribution (sample of PDF)
histogram of (amplitude) values
module 3.1: input representation — signals 10 / 18
31. overview intro periodic signals random signals signal description summary
statistical signal description
probability density function
PDF px (x)
x-axis: possible (amplitude) values
y-axis: probability
px (x) ≥ 0, and
∞
Z
−∞
px (x) dx = 1
RFD—Relative Frequency Distribution (sample of PDF)
histogram of (amplitude) values
module 3.1: input representation — signals 10 / 18
32. overview intro periodic signals random signals signal description summary
statistical signal description
probability density function
PDF px (x)
x-axis: possible (amplitude) values
y-axis: probability
px (x) ≥ 0, and
∞
Z
−∞
px (x) dx = 1
RFD—Relative Frequency Distribution (sample of PDF)
histogram of (amplitude) values
module 3.1: input representation — signals 10 / 18
33. overview intro periodic signals random signals signal description summary
statistical signal description
PDF examples
What is the PDF of the following prototype signals:
module 3.1: input representation — signals 11 / 18
34. overview intro periodic signals random signals signal description summary
statistical signal description
PDF examples
What is the PDF of the following prototype signals:
square wave
sawtooth wave
sine wave
white noise (uniform, gaussian)
DC
module 3.1: input representation — signals 11 / 18
35. overview intro periodic signals random signals signal description summary
statistical signal description
PDF examples
What is the PDF of the following prototype signals:
module 3.1: input representation — signals 11 / 18
matlab
source:
plotPdfExamples.m
36. overview intro periodic signals random signals signal description summary
statistical signal description
RFD: real world signals
module 3.1: input representation — signals 12 / 18
matlab
source:
plotRfd.m
37. overview intro periodic signals random signals signal description summary
statistical signal description
arithmetic mean
from time series x:
µx (n) =
1
K
ie(n)
X
i=is(n)
x(i)
from distribution px :
µx (n) =
∞
X
x=−∞
x · px (x)
module 3.1: input representation — signals 13 / 18
matlab
source:
plotArithmeticMean.m
38. overview intro periodic signals random signals signal description summary
statistical signal description
geometric & harmonic mean
geometric mean
Mgv = N
v
u
u
t
N−1
Y
0
v(n)
= exp
1
N
N−1
X
0
log v(n)
!
.
harmonic mean
Mhv =
N
N−1
P
0
1/v(n)
.
module 3.1: input representation — signals 14 / 18
39. overview intro periodic signals random signals signal description summary
statistical signal description
geometric harmonic mean
geometric mean
Mgv = N
v
u
u
t
N−1
Y
0
v(n)
= exp
1
N
N−1
X
0
log v(n)
!
.
harmonic mean
Mhv =
N
N−1
P
0
1/v(n)
.
module 3.1: input representation — signals 14 / 18
40. overview intro periodic signals random signals signal description summary
statistical signal description
variance standard deviation
measure of spread of the signal around its mean
variance
• from signal block:
σ2
x (n) =
1
K
ie(n)
X
i=is(n)
x(i) − µx (n)
2
• from distribution:
σ2
x (n) =
∞
X
x=−∞
x − µx
2
· px (x)
standard deviation
σx (n) =
q
σ2
x (n)
module 3.1: input representation — signals 15 / 18
matlab
source:
plotStandardDeviation.m
41. overview intro periodic signals random signals signal description summary
statistical signal description
variance standard deviation
measure of spread of the signal around its mean
variance
• from signal block:
σ2
x (n) =
1
K
ie(n)
X
i=is(n)
x(i) − µx (n)
2
• from distribution:
σ2
x (n) =
∞
X
x=−∞
x − µx
2
· px (x)
standard deviation
σx (n) =
q
σ2
x (n)
module 3.1: input representation — signals 15 / 18
matlab
source:
plotStandardDeviation.m
42. overview intro periodic signals random signals signal description summary
statistical signal description
variance standard deviation
measure of spread of the signal around its mean
variance
• from signal block:
σ2
x (n) =
1
K
ie(n)
X
i=is(n)
x(i) − µx (n)
2
• from distribution:
σ2
x (n) =
∞
X
x=−∞
x − µx
2
· px (x)
standard deviation
σx (n) =
q
σ2
x (n)
module 3.1: input representation — signals 15 / 18
matlab
source:
plotStandardDeviation.m
43. overview intro periodic signals random signals signal description summary
statistical signal description
quantiles quantile ranges
dividing the PDF into (equal sized) subsets
QX(cp) = argmin FX(x) ≤ cp
with FX(x) =
x
Z
−∞
px(y) dy
module 3.1: input representation — signals 16 / 18
matlab
source:
plotPdfQuantiles.m
44. overview intro periodic signals random signals signal description summary
statistical signal description
quantile examples
median
QX(0.5) = argmin FX(x) ≤ 0.5
quartiles: QX(0.25), QX(0.5), and QXx(0.75)
quantile range, e.g.
∆QX(0.9) = QX(0.95) − QX(0.05)
module 3.1: input representation — signals 17 / 18
45. overview intro periodic signals random signals signal description summary
summary
lecture content
signals can be categorized into deterministic and random signals
• deterministic signal can be described in a mathematical function
• random processes can only be described by their general properties
periodic signals
• periodic signals are probably the most music-related deterministic signal
• any periodic (pitched) signal is a sum of weighted sinusoidals
• frequencies only at the fundamental frequency and integer multiples
random signals
• noise, unpredictable
real-world signals
• can be seen as a time-varying mixture of these two signal categories
statistical features
• summarize technical signal characteristics in few numerical values
• may be used on a time domain, frequency domain, or feature domain signal
module 3.1: input representation — signals 18 / 18