1. Experimental analysis of optimal window length
for independent low-rank matrix analysis
Daichi Kitamura
Nobutaka Ono
Hiroshi Saruwatari
25th European Signal Processing Conference (EUSIPCO) 2017
SS14: Multivariate Analysis for Audio Signal Source Enhancement
August 30, 14:30-16:10
The University of Tokyo, Japan
National Institute of Informatics, Japan
The University of Tokyo, Japan

2. Contents
• Background
– Blind source separation (BSS) for audio signals
– Motivation: fundamental limitation in frequency-domain BSS
• Methods
– Frequency-domain independent component analysis (FDICA)
– Independent vector analysis (IVA)
– Independent low-rank matrix analysis (ILRMA)
• Experimental analysis
– Optimal window length
• Music signals and speech signals
• Ideal case and more practical case
• Conclusion
2

3. Contents
• Background
– Blind source separation (BSS) for audio signals
– Motivation: fundamental limitation in frequency-domain BSS
• Methods
– Frequency-domain independent component analysis (FDICA)
– Independent vector analysis (IVA)
– Independent low-rank matrix analysis (ILRMA)
• Experimental analysis
– Optimal window length
• Music signals and speech signals
• Ideal case and more practical case
• Conclusion
3

4. • Blind source separation (BSS) for audio signals
– separates original audio sources
– does not require prior information of recording conditions
• locations of mics and sources, room geometry, timbres, etc.
– can be available for many audio app.
• Consider only “determined” situation
Background
4
Recording mixture Separated guitar
BSS
Sources Observed Estimated
Mixing system Demixing system
# of mics
# of sources

5. • Basic theories and their evolution
History of BSS for audio signals
5
1994
1998
2013
1999
2012
Age
Many permutation
solvers for FDICA
Apply NMF to many tasks
Generative models in NMF
Many extensions of NMF
Independent component analysis (ICA)
Nonnegative matrix factorization (NMF)
Frequency-domain ICA (FDICA)
Itakura–Saito NMF (ISNMF)
Independent vector analysis (IVA)
Multichannel NMF
Independent low-rank matrix analysis (ILRMA)
*Depicting only popular methods
2016
2009
2006
2011 Auxiliary-function-based IVA (AuxIVA)
Time-varying Gaussian IVA

6. • Basic theories and their evolution
History of BSS for audio signals
6
1994
1998
2013
1999
2012
Age
Many permutation
solvers for FDICA
Apply NMF to many tasks
Generative models in NMF
Many extensions of NMF
Independent component analysis (ICA)
Nonnegative matrix factorization (NMF)
Frequency-domain ICA (FDICA)
Itakura–Saito NMF (ISNMF)
Independent vector analysis (IVA)
Multichannel NMF
Independent low-rank matrix analysis (ILRMA)
*Depicting only popular methods
2016
2009
2006
2011 Auxiliary-function-based IVA (AuxIVA)
Time-varying Gaussian IVA

7. Motivation: fundamental limitation of BSS
• Mixing assumption in frequency-domain BSS
– “Linear time-invariant mixture” or “rank-1 spatial model”
– Valid only when
• Too long window also causes another problem
– Number of time frames (samples) decreases
• Trade-off between short and long window [S. Araki+, 2003]
– FDICA suffers from the trade-off
– What about for BSS methods
with structural source model?
• IVA and ILRMA 7
: frequency binsObserved
multichannel signal
Source signalsFrequency-wise mixing matrix
: time frames
Statistical bias will increase and estimation becomes unstable
window length used in STFT length of room reverberation
Performance
Window length
Optimal length

8. Contents
• Background
– Blind source separation (BSS) for audio signals
– Motivation: fundamental limitation in frequency-domain BSS
• Methods
– Frequency-domain independent component analysis (FDICA)
– Independent vector analysis (IVA)
– Independent low-rank matrix analysis (ILRMA)
• Experimental analysis
– Optimal window length
• Music signals and speech signals
• Ideal case and more practical case
• Conclusion
8

9. • Frequency-domain ICA (FDICA) [P. Smaragdis, 1998]
• Independent vector analysis (IVA) [A. Hiroe, 2006], [T. Kim, 2006]
BSS methods: FDICA and IVA
9
Observed
Update separation filter so that the estimated
signals obey non-Gaussian distribution we assumed
Estimated
Demixing
matrix
Current
empirical dist.
Non-Gaussian
source dist.
STFT
Frequency
Time
Frequency
Time
Observed Estimated
Current
empirical dist.
STFT
Frequency
Time
Frequency
Time
Non-Gaussian
spherical
source dist.
Scalar r.v.s
Vector
(multivariate) r.v.s
Update separation filter so that the estimated
signals obey non-Gaussian distribution we assumed
Mixture is close to Gaussian
signal because of CLT
Source obeys non-
Gaussian dist.
Mutually
independent
Demixing
matrix Mutually
independent

10. • Spherical Laplace distribution in IVA
• Zero-mean complex Gaussian distribution with TF-
varying variance (Itakura-Saito NMF)[C. Févotte+, 2009]
10
Frequency-uniform scale
Extension of source distribution in IVA
Zero-mean complex
Gaussian in each TF bin Low-rank decomposition
with NMF
Spherical Laplace (bivariate)
Frequency vector
(I-dimensional)
Time-frequency-varying variance
Time-frequency matrix
(IJ-dimensional)
Extended to a more flexible model

11. • Power spectrogram corresponds to variances in TF
plane
Generative source model in ISNMF
11
Frequencybin
Time frame
: Power spectrogram
Small value of power
Large value of power
Complex Gaussian distribution with TF-varying variance
If we marginalize in terms of time or frequency, the distribution
becomes non-Gaussian even though each TF grid is defined in
Gaussian distribution
Grayscale shows the
value of variance

12. BSS methods: ILRMA
• Independent low-rank matrix analysis (ILRMA) [D. Kitamura+,2016]
– Unification of IVA and ISNMF
– Source model in ILRMA
12
Frequency
Basis
Basis
Time
Number of bases can be set to arbitrary value
Frequency
Time
Observed Estimated
Low-rank decomposition
Time
Frequency
Frequency
Time
Update demixing matrix so that estimated signals
have low-rank structure in time-frequency domain
STFT
Demixing
matrix

13. Comparison of source models
13
FDICA source model
Non-Gaussian scalar variable
IVA source model
Non-Gaussian vector variable
with higher-order correlation
ILRMA source model
Non-Gaussian matrix variable
with low-rank time-frequency
structure
Rank of TF matrix
of mixture
Rank of TF matrix
of each source

14. Contents
• Background
– Blind source separation (BSS) for audio signals
– Motivation: fundamental limitation in frequency-domain BSS
• Methods
– Frequency-domain independent component analysis (FDICA)
– Independent vector analysis (IVA)
– Independent low-rank matrix analysis (ILRMA)
• Experimental analysis
– Optimal window length
• Music signals and speech signals
• Ideal case and more practical case
• Conclusion
14

15. Experimental analysis
• Window length in STFT
– If window length is too short
• Mixing assumption does not hold anymore
– If window length is too long
• Estimation becomes unstable (# of time frames decreases)
15
Frequency
Time
…
DFT
DFT
DFT
Spectrogram
…
Window length (= DFT length)
Shift length
Window function
Waveform
• Our expectation
– Full time-frequency modeling of sources in ILRMA may improve the
robustness to a decrease in the number of time frames

16. Experimental analysis
• Dataset: 4 music and 4 speech from SiSEC [S. Araki+, 2012]
• Mixing: convolution with RIR in RWCP [S. Nakamura+, 2000]
16
Signal Data name Source (1/2) Length [s]
Music bearlin-roads acoustic_guit_main/vocals 14.6
Music another_dreamer-the_ones_we_love guitar/vocals 25.6
Music fort_minor-remember_the_name violins_synth/vocals 24.6
Music ultimate_nz_tour guitar/synth 18.6
Speech dev1_female4 src_1/src_2 10.0
Speech dev1_female4 src_3/src_4 10.0
Speech dev1_male4 src_1/src_2 10.0
Speech dev1_male4 src_3/src_4 10.0
2 m
Source 1
5.66cm
50 50
2 m
5.66cm
60 60
Impulse response E2A
(reverberation time: T60 = 300 ms)
Impulse response JR2
(reverberation time: T60 = 470 ms)
Source 2 Source 1 Source 2

17. Experimental analysis
• Compared methods
– FDICA+IPS (ideal permutation solver)
• Align permutation of estimated components using the reference
(oracle) source spectrogram (upper limit performance of FDICA)
– FDICA+DOA (DOA-based permutation solver) [S. Kurita+, 2000]
• Align permutation of estimated components using DOA after FDICA
– IVA [N. Ono, 2011]
• using auxiliary function method (a.k.a. MM algorithm) in optimization
– ILRMA [D. Kitamura+, 2016]
• with several numbers of bases
• Other conditions
– Window function: Hamming window
– Window length: 32 ~ 2048 ms
– Shift length: Always quarter of window length
17

18. Comparison using ideal initialization: condition
• Set initial value of demixing matrix to oracle:
– This initial value provides the best separation performance
under the assumption
• Set initial value of source model as oracle
(only for ILRMA):
18
Power spectrogram of th source
FDICA+DOA & IVA: spatial oracle initialization
FDICA+IPS & ILRMA: spatial and spectral oracle initialization

19. Comparison using ideal initialization: results
19
Music
T60 =0.30 s
Music
T60 =0.47 s
Speech
T60 =0.30 s
Speech
T60 =0.47 s

20. Comparison using random initialization: condition
• Set initial value of demixing matrix to identity
matrix
• Set initial value of source model to uniform
random value between [0,1] (only for ILRMA)
20
FDICA+DOA, IVA, & ILRMA: fully blind method
FDICA+IPS: using oracle spectrogram

21. Comparison using random initialization: results
21
Music
T60 =0.30 s
Music
T60 =0.47 s
Speech
T60 =0.30 s
Speech
T60 =0.47 s

22. Conclusion
• In the case of ILRMA with oracle initialization, the
robustness to long windows (fewer time frames) can
be improved
– optimal window length is longer than that in FDICA or IVA
– thanks to employing not only the independence between
sources but also a full modeling of time-frequency structure
for the estimation of the demixing matrix
• In a practical situation (fully blind case),
– optimal window length is similar to that in FDICA or IVA
– difficulty of the blind estimation of a precise spectral model
in ILRMA
22
Thank you for your attention!