Heart rate estimation of car driver using radar sensors and blind source separation
•Download as PPTX, PDF•
0 likes•75 views
Keito Murata, Daichi Kitamura, Ryo Saito, and Daichi Ueki, "Heart rate estimation of car driver using radar sensors and blind source separation," Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2022), pp. 1157–1164, Chiang Mai, Thailand, November 2022.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Heart rate estimation of car driver using radar sensors and blind source separation
Similar to Heart rate estimation of car driver using radar sensors and blind source separation (20)
More from Kitamura Laboratory
More from Kitamura Laboratory (20)
Recently uploaded
Recently uploaded (20)
Heart rate estimation of car driver using radar sensors and blind source separation
- 1. Heart Rate Estimation of Car Driving Using Radar Sensors and Blind Source Separation Keito Murata, Daichi Kitamura, Ryo Saito, Daichi Ueki 14th Asia Pacific Signal and Information Processing Association Annual Summit and Conference Session: SS14: Emerging signal Processing Technology for Medical Applications/ Biomedical Signal Processing and Systems Time: Wed., 9. Nov, 17:20-17:40 (UTC +7) † † National Institute of Technology, Kagawa College, Japan Murata Manufacturing Co., Ltd. †
- 2. • Background – Some situations are fatal while driving • Sleeping • Sudden seizure such as myocardial infarction • Lose consciousness – Needs the system always monitors the driver’s heart rate • Research summary and purpose – Using a contactless millimeter-wave radar sensor • Less burden for driver • No need to wear • Hygienic – Measuring displacement of the driver’s body surface using the radar sensor – We analyze components of heartbeat from the observed signal Radar sensor Research summary 2
- 3. • Problem – Displacement of breathing and driver’s movement except heart rate is measured simultaneously – Relative position of radar sensor and body surface is unknown • Our approach – Separate heart rate signal from noise signal (breathing, driver’s movement, vibration of car body) with blind source separation (BSS) – Analyzes heart rate from separated heartbeat signal – Evaluates by degree of agreement with heart rate obtained by medical electrode pad Research summary 3 Contact-type electrocardiograph (ECG) sensor
- 4. • Measurement simulating car vibration while driving – Vibrates the vibration plate with sin wave in vertical direction – Measures a microdisplacement of the driver’s body surface with radar sensor embedded in the back of the seat • Condition of vibration plate Measurement system in this research 4 ECG sensor Vibration direction Radar sensor Amp. [mm] Time [s] 0 60 360 420 Vibration time section Frequency：1.2 [Hz] Vibration plate
- 5. • Observed signal of 4 channels directional beam – Measures a 4points near the displacement of driver’s body surface – Sampling frequency：40 [Hz] 5 Observed signal of radar sensor Vibration time of vibration plate Displacement of driver’s body surface [ m m] Time [s] Ch. 1 Ch. 2 Ch. 3 Ch. 4
- 6. • Time-frequency structure (spectrogram) Comparison of radar sensor and ECG sensor 6 Time [s] Time [s] Frequency [Hz] Frequency [Hz] Radar sensor (Ch. 3) ECG sensor
- 7. • Time-frequency structure (spectrogram) Comparison of radar sensor and ECG sensor 6 Time [s] Time [s] Frequency [Hz] Frequency [Hz] Radar sensor (Ch. 3) ECG sensor
- 8. • Apply three processes below – ①Preprocess, ②BSS, ③Heart rate estimation algorithm Hear rate estimation algorithm ③ Process flow of proposed method 8 Observed signal Estimated heartbeat signal Estimated heart rate Heart rate [bpm] Time [s] BSS ② Preprocess ①
- 9. ② Hear rate estimation algorithm ③ Process flow of proposed method 9 Observed signal Estimated heartbeat signal Estimated heart rate Heart rate [bpm] Time [s] BSS Preprocess ① Preprocess ①
- 10. 10 • Component of body movement caused by breathing – Remove breathing by filtering before applying BSS • Specification of digital filter – High-pass FIR filter • Cut-off frequency: 1.5 Hz • Tap length：170 Frequency [Hz] Amp. [dB] ①Preprocess Before applying filter After applying filter Amplitude responses Frequency [Hz] Frequency [Hz] Time [s] Time [s]
- 11. Hear rate estimation algorithm ③ Process flow of proposed method 11 Observed signal Estimated heartbeat signal Estimated heart rate Heart rate [bpm] Time [s] Preprocess ① BSS ②
- 12. • Assumptions – Relative locations of body surface and radar sensor is unknown (mixing system is unknown) – Exploit BSS • Methods of BSS used in this research – Independent vector analysis: IVA [Kim+, 2006] – Independent low-rank matrix analysis: ILRMA [Kitamura+, 2016] – ILRMA based on Student- distribution: -ILRMA [Mogami+, 2017] ②BSS 12 Mixing system Demixing system
- 13. • Independent component analysis: ICA [Comon, 1994] – estimates demixing matrix under the conditions that mixing matrix is unknown – optimizes to maximize independence between original sources • Maximize non-Gaussian between sources ②BSS: ICA 13 Approaches to Gaussian distribution by mixture (central limit theorem) Maximize non-Gaussianity to estimate the sources (ICA)
- 14. • ICA for time-frequency domain (spectrogram) – assume the frequency-wise mixing and demixing matrices – BSS for convolutive mixtures can be achieved ：Frequency index ：Time index ②BSS: BSS in time-frequency domain 14 Mixing system Demixing system Mixing matrix Source signal Observed signal Demixing matrix Observed signal Separated signal Freq. Freq. Freq. Freq. Time Time Time Time
- 15. • Independent vector Analysis (IVA) [Kim+, 2006] – Assumption 1: Maximize independence between each signal (same as ICA) – Assumption 2: Co-occurrence of all the frequency components in the same source – Observed signal obtained by radar sensor • Clear harmonic structure exits in heartbeat and vibration component • Fundamental frequency and its harmonic components are simultaneously activated ②BSS: Summary of IVA 15 Observed signal Separated signal Frequency-wise demixing matrix Estimate satisfied with two assumptions Two assumptions of IVA hold Time [s] Frequency [Hz]
- 16. • Independent low-rank matrix analysis (ILRMA) [Kitamura+, 2016] – Assumption 1: Maximize independence between each signal source (same as ICA) – Assumption 2: Time-frequency structure of each source tends to be low-rank • Low-rank: Spectrogram includes similar spectra many times – Observed signal obtained by radar sensor • Component of heartbeat and vibration plate have almost the same spectrum for a long time ②BSS: Summary of ILRMA 16 Observed signal Separated signal Frequency wise demixing matrix Two assumptions of ILRMA hold Estimate satisfied with two assumptions Frequency [Hz] Time [s]
- 17. • -ILRMA: Generalized method of ILRMA – ILRMA assumes the complex Gaussian distribution as a source generative model – -ILRMA generalizes the source model by using the complex Student’s t distribution – More robust and precise BSS can be achieved by a heavy-tale property • Complex Student’s distribution ②BSS: Summary of -ILRMA 17 : DoF parameter : Scale parameter Equivalent to Gaussian dist. heavy tale dist. Induces robust low-rank modeling
- 18. ②BSS: conditions 18 • Parameter of STFT • Parameter setting of IVA, ILRMA, and -ILRMA Parameter IVA ILRMA -ILRMA Initial value of demixing matrix ww Identity matrix Number of iteration of Iterative optimization algorithm 100 times Initial value of and ww ー Uniform random number (from 0 to 1) Number of rank for each signal source ー 3 DoF parameter nu ー ー 5 Window length of STFT 1.6 s Shift length of STFT 0.1 s Window function Hamming window
- 19. ②BSS: Comparison of IVA, ILRMA, and -ILRMA 19 IVA ILRMA -ILRMA
- 20. Process flow of proposed method 20 Observed signal Estimated heartbeat signal Estimated heart rate Heart rate [bpm] Time [s] BSS ② Preprocess ① ③ Hear rate estimation algorithm ③
- 21. • Convert separated heartbeat spectrogram to the time-domain signal • In general, we apply R-R interval (RRI) estimation to obtain heart rates – RRI estimation does not work due to the residual noise • We apply another heart rate estimation algorithm below ③Heart rate estimation algorithm 21 Emphasize heartbeat signal Band-pass filter Detect spectral peaks Inverse STFT Freq. [Hz] Time [s] Estimated heart rate Hear rate [bpm] Time [s] R wave RRI
- 22. ③Results of heart rate estimation 22 IVA’s Estimated heart rate ILRMA’s Estimated heart rate -ILRMA’s Estimated heart rate Heart rate of ECG sensor Estimated heart rate of separated signal : : Vibration plate working
- 23. Conclusion • Problems – Estimate the driver’s heart rate from observed signal obtained by contactless sensor • breathing and driver’s movement noise are measured simultaneously except heart rate • Measurement – Measure microdisplacement of driver’s body surface with radar sensor • Applications – Apply BSS (IVA, ILRMA, and -ILRMA) • The location of the radar sensor, source of heartbeat, body movement, and breathing are unknown • Estimated heart rate obtained by -ILRMA shows the highest separation accuracy 23 Thank you for your attention.
Editor's Notes
- Hi, everyone, I’m Keito Murata from National Institute of Technology, Kagawa College, Japan. I’m gonna talk about Heart Rate Estimation of Car Driving Using Radar Sensors and Blind Source Separation.
- First, I’m gonna talk about the research background. there are some fatal situations while we driving. For example / sleeping, sudden seizure, lose of consciousness, and so on. Therefore, a system / that constantly monitors driver's heartbeat / is necessary. In this research, / we develop a system / using a millimeter-wave radar sensor, / which is embedded in the driver's seat. This radar sensor can measure / the displacement of the driver's body surface / without contact. And / the displacement includes the movement / derived from the heartbeat. So, we can predict the heartbeat / by analyzing the radar sensor signal(ﾉｰ). Of course, / other noises are measured at the same time, / so / only the heartbeat component / needs to be extracted. まず本研究の背景についてお話しします． 自動車の運転中に運転者の睡眠，心筋梗塞等の突発的な発作や体調の悪化による意識喪失等の死亡事故が少なからず起きています． そのことから，運転者の心拍を常にモニタリングするようなシステムが必要になります． 例えば，運転者が接触型の心拍センサを着用する等の方法がありますが，装着に手間がかかることや，そもそも接触型のセンサは蒸れやかぶれなどの衛生面の心配もあります． そこで本研究では，ミリ波レーダセンサと呼ばれるものを（図を指しながら）運転者のシートに埋め込むようなシステムを考えます． このレーダセンサによって ”非接触のまま” 運転者のからだの表面の変位を計測できます． このからだの表面の変位には実は心拍由来の動きが含まれていますので，レーダセンサの信号を解析することで心拍が予測できる可能性があります． 当然，その他のノイズも同時に測定されるので心拍成分だけを抽出する必要があります．
- As we mentioned earlier, / in addition to heartbeat components, / vibration components are also measured at the same time. Furthermore, it is difficult to know the relative position of radar sensor and body(ﾊﾞﾃﾞｨ) surface in advance / because of differences in driver’s physique. To solve this problem, we apply blind source separation (BSS), as shown in this figure, to observed signal obtained by radar sensor. And we analyze a heart rate / from heartbeat signal / separated by BSS. Then, / we compare the estimated heart rate / and a value obtained by ECG sensor, / as shown in this figure, / to confirm(ｹﾝﾌｩｰﾑ) the heart rate estimation accuracy. 先程申し上げた通り，計測時には心拍由来の体表面変位以外にも車体の振動，運転者の体動，呼吸による体動等の成分が混入する問題があります． さらに，運転者の体格の違いなどもありますのでレーダセンサと体の相対的な位置関係というのは事前に知ることが難しいです． これらの問題の解決方法として，（図を指しながら）図に示しますように，レーダセンサで得られる観測信号に対してブラインド信号源分離（BSS）を適用することで心拍信号とノイズの分離を行い，分離された心拍信号から心拍数を解析します． 解析した心拍がどの程度の精度になるかについては，この図のように運転者の胸部に装着した医療用の電極パッドで得られる心拍数との一致度合いを確認します．
- In this research, / we use a system / as shown in this figure. Vibration plate in this system can simulate a car vibration while driving. And the radar sensor is embedded in the back of the seat. And we can measure microdisplacement of the driver’s body surface. As shown in this figure, / the radar sensor has four directional channels / and can simultaneously measure microdisplacements / at four points near the driver’s back. We measured for 420 seconds, / and the vibration plate vibrates in vertical direction from 60 seconds to 360 seconds. 本研究では実際の運転中の車体の振動を模擬するために，外部から振動を与えることができる”振動台”を用意し，その上に自動車のシートを載せます． 振動台は上下方向に加振することができるシステムです． また，レーダセンサはシートの背部に埋め込まれています． このレーダセンサは図に示すように4チャネルの指向性を持っており，運転者の背中の近傍四点の変位を同時に測定できます． 当然，被験者は測定中に無意識の体の動きや呼吸による体動が生じます． この模擬実験では，合計420秒間の測定行います． その中で，60秒から360秒の5分間だけ振動台は上下に振動します． この時の振幅は10mmで振動周波数1.2Hzのsin波で加振しています．
- This graph shows / observed signals / obtained by the radar sensor / as we describe earlier. The horizontal axis represents time / and the vertical axis represents body surface displacement. And vibration plate vibrates in this section. We obtain slightly different displacements of the body surface because the radar sensor has four channels directivity(ﾃﾞｨﾚｸﾃｨﾋﾞﾃｨ). Time-domain signals only shows amplitude, so we analyze in time-frequency domain. 実際に測定した結果，レーダセンサから得られる観測信号はこのグラフのようになりました． サンプリング周波数は40Hzで測定されています． 先程申し上げた通りレーダセンサは4チャネルの指向性がありますので，観測信号も微妙に異なる体表面の変位4チャネルになります． （時間波形を適宜指しながら）これらの波形の横軸は時間，縦軸は体表面変位量マイクロメートルを表しています． 全体は420秒で60秒から360秒の（指しながら）この区間に振動台が上下に振動しています． 近傍の体表面４点なのでこれらの波形はどれも似たような波形になっているのが分かります． 時間波形では振幅ぐらいしかわからないので，これらの波形を時間周波数解析して見ていきます．
- We compare the radar sensor signal and ECG sensor signal in time-frequency domain. The horizontal axis / represents time / and the vertical axis represents / frequency. Then, / we call this graph / spectrogram. The left graph shows / radar sensor signal of channel 3 / and right graph shows / ECG sensor signal / which is reference value in this research. In the spectrogram of ECG sensor, we can confirm the fundamental frequency and Its harmonic components of the heartbeat. These components are appeared even in the radar sensor although their powers are very low. Also, we can see strong components around 0.3 Hz. This is the movement caused by “breathing” of driver. In addition, there are strong and straight horizontal lines are overlapped with the heartbeat components. They are the movement noise caused by the vibration plate, which simulates the driving noise of a car. So, we need to extract the heartbeat components from the observed signal, where breathing and vibration are mixed. 左側のグラフが，3チャネル目の波形を時間周波数解析した結果です． 横軸が時間，縦軸が周波数を表しており，これをスペクトログラムと呼びます． （接触型心拍系のスペクトログラムを指しながら）また，右側のグラフは参照値となる，”胸に取り付けた接触型心拍計”の信号のスペクトログラムです． 接触型なのでレーダセンサよりもはるかにくっきり心拍の成分が確認できます． 1Hz付近に見える筋が，心拍の基本波成分（指しながら）であり，上の方にはその整数倍の高調波成分がたくさんあらわれています． で，個の心拍成分がレーダセンサの方で見えるかどうかなんですが，（指しながら）ここらへんに高調波成分は確認できます． しかし，0.3Hzぐらいには非常に強い成分が現れており，これは呼吸による体動です． さらに，振動台が動く期間では振動台の成分が強いエネルギーで重なってきます．この横筋に見えるものがすべてそうです． このように，呼吸や振動が混入する信号から（アニメーション全部出す）この心拍成分だけをうまく抽出する必要があります．
- We compare the radar sensor signal and ECG sensor signal in time-frequency domain. The horizontal axis / represents time / and the vertical axis represents / frequency. Then, / we call this graph / spectrogram. The left graph shows / radar sensor signal of channel 3 / and right graph shows / ECG sensor signal / which is reference value in this research. In the spectrogram of ECG sensor, we can confirm the fundamental frequency and Its harmonic components of the heartbeat. These components are appeared even in the radar sensor although their powers are very low. Also, we can see strong components around 0.3 Hz. This is the movement caused by “breathing” of driver. In addition, there are strong and straight horizontal lines are overlapped with the heartbeat components. They are the movement noise caused by the vibration plate, which simulates the driving noise of a car. So, we need to extract the heartbeat components from the observed signal, where breathing and vibration are mixed. 左側のグラフが，3チャネル目の波形を時間周波数解析した結果です． 横軸が時間，縦軸が周波数を表しており，これをスペクトログラムと呼びます． （接触型心拍系のスペクトログラムを指しながら）また，右側のグラフは参照値となる，”胸に取り付けた接触型心拍計”の信号のスペクトログラムです． 接触型なのでレーダセンサよりもはるかにくっきり心拍の成分が確認できます． 1Hz付近に見える筋が，心拍の基本波成分（指しながら）であり，上の方にはその整数倍の高調波成分がたくさんあらわれています． で，個の心拍成分がレーダセンサの方で見えるかどうかなんですが，（指しながら）ここらへんに高調波成分は確認できます． しかし，0.3Hzぐらいには非常に強い成分が現れており，これは呼吸による体動です． さらに，振動台が動く期間では振動台の成分が強いエネルギーで重なってきます．この横筋に見えるものがすべてそうです． このように，呼吸や振動が混入する信号から（アニメーション全部出す）この心拍成分だけをうまく抽出する必要があります．
- We explain a process flow of the proposed method as shown in this figure. First, we apply preprocessing to the observed signal. Second, we apply BSS to the preprocessed signal to remove noise. Finally, we apply a heart rate estimation algorithm to separated signal to confirm a heart rate estimation accuracy. And we compare estimated heart rates calculated from the BSS output and the ECG sensor.
- First, we explain the preprocess.
- As we mentioned earlier, the observed signal contains a very strong breathing components. Therefor, we apply high-pass filter to remove breathing component before applying BSS. We set cutoff frequency to 1.5 Hz and tap length to 170. This graph shows before and after the filter was applied. We confirm components below 1.5 Hz are removed by applying filter. In the following section, we talk about experimental results / with this high-pass filter. 先程述べた通り，観測信号には呼吸による体動成分が顕著に見られます． そのため，BSSを適用する前に，呼吸の体動成分をハイパスフィルタで除去します． カットオフ周波数は1.5Hzに設定し，フィルタの次数は170次のFIRフィルタとしました． 振幅特性はこのようになっています． 呼吸カットフィルタ適用前と適用後を比較します． フィルタを適用したことで1.5Hz以下の呼吸の体動成分を含めたその他の成分がカットされていることが分かります． 以後，フィルタ適用後の信号にBSSを適用した実験結果についてお話していきます．
- Second, we explain blind source separation BSS.
- In the mixing assumption in this research, we don’t know relative positions between a body surface and the radar sensor. So, we cannot know a mixing system A in advance, where mixing system shows how the source signals are mixed, namely, the heartbeat, vibration nosie, and so on. Blind source separation is a technique that estimates source signals under the condition that the mixing system is unknown. In other words, we can estimate demixing matrix W without knowing the mixing system A, where W is an inverse system of the mixing matrix A. In an audio signal processing field, IVA and ILRMA are proposed as a powerful BSS method. In this research, we apply IVA and ILRMA to the observed signals obtained by hte radar sensor. In addition to these methods, we also apply t-ILRMA which is explained later. In the next slide, we explain about most basic BSS ICA. 本研究で想定するシステムの観測条件として，体表面とレーダセンサの位置関係は分かりません． つまり図に示すように振動や体動といった各信号源とレーダセンサとの混合系Aは事前に知ることはできません． このように，信号源の混ざり方が分からない状態でも，混ざる前の各信号源を推定できる技術としてブラインド信号源分離BSSがあります． すなわち，混合系Aの逆システムである分離系Wを，混合系Aを知らないままに推定する技術です． 必要となる仮定は，各信号源が互いに独立である，というものになります． BSSは，独立成分分析ICAから発展してきた技術であり，音響信号処理分野では独立ベクトル分析IVAや独立低ランク行列分析ILRMAと呼ばれる，より高精度な手法が提案されています． 本研究では，レーダセンサから得られる信号に対して，このIVAとILRMAを適用し心拍信号の推定にどの程度効果があるかを検討します． 次のスライドからICA，IVA，ILRMAの詳しい説明をしていきます．
- ICA assumes that / multiple(ﾑｰﾃｨﾎﾟｰ) signal sources / are mixed by an unknown mixing matrix A, / and their mixture is observed, / where the numbers of sensors / and sources are the same. If the matrix W / is the inverse of A, / we can separate the sources / by multiplying W by the observed signal. / So, / we find such the demixing matrix. From the central limit theorem, / the mixture signal approaches to the Gaussian distribution. Therefore, / by maximizing non-Gaussianity of the estimated signals, / we can obtain the separated sources. ICA tries to find such demixing matrix even in a blind situation, namely, the mixing matrix A is unknown. Such a procedure is equivalent to maximizing the statistical(ｽﾀﾃｨｽﾃｨｶﾙ) independence between each estimated signal. まず，最も基本的なBSSであるICAについて説明します． （下の図を指しながら）ICAでは複数の信号源が未知の混合行列Aによって混合され，信号源の数と同じ数のセンサで観測されることを仮定します． もし，行列WがAの逆行列であれば，観測信号にWをかけることで一度混ざった信号源を再び分離することができますので，そのようなWを見つけ必要があります． 複数の信号源が混ざるという現象は，中心極限定理より，統計的性質がガウス分布に近づいていくことに対応します． そのため，Wで分離した後の信号がガウス分布から遠ざかるようにWを最適化することで，混合行列Aが未知の場合でも分離行列Wを推定することができます． このような手続きは，（右端の信号を指しながら）各推定信号間の統計的独立性を最大化しているのと等価です． 従って，混ざる前の信号源は互いに独立であることが必要です．
- This ICA is often applied in the time-frequency domain to achieve more complex BSS. The mixing system can be modeled as a frequency-wise mixing matrix A like this figure. Then, we observe the mixed spectrogram. We can represent this mixing flow as this equation, where i and j are the indices of frequency and time // and sij and xij are the frequency-wise vectors in the depth direction of these figures. Therefor, the demixing matrix is also defined as Wi in each frequency and we estimate the Wi by using frequency-wise ICA. If we estimate all Wi, we can get the separated spectrogram by using this equation. Then, we can get the separated signal by converting the time-frequency domain into the time-domain. IVA and ILRMA are the powerful BSS methods applied in the time-frequency domain. From the next slides, we explain about IVA, ILRMA, and t-ILRMA, which are used in this research. 先程説明したICAは時間領域でのBSSでしたが，時間周波数領域でICAを行うことで，より複雑なBSSが可能となります． （真ん中上図を指しながら）今，各信号源の時間周波数表現であるスペクトログラムを考えます． 信号源が混ざるという過程は，周波数毎に異なる（混合行列指しながら）混合行列Aiが掛け算されているとモデル化し，（上右端の図指しながら）混ざった後の観測信号のスペクトログラムが得られます． これを式で描くと（上式を指しながら）このようになります． ここでiは周波数インデクス，jは時間インデクスであり，sijやxijはこれらの図の奥行方向の時間周波数毎のベクトルです． 従って，分離行列も周波数毎にWiとして定義され，これを全て周波数毎のICAで推定する事になります． Wiがすべて推定できれば（下式指しながら）この式で分離信号のスペクトログラム（右端図）が得られます． 最後に，このスペクトログラムを時間波形に戻せば分離された信号源の推定が得られます． このように時間周波数領域で行うBSSとして有名なものにIVAとILRMAの二種類が提案されています．
- First, / we explain Independent Vector Analysis / IVA. IVA has two assumptions. First assumption is maximizing independence between each signal as with ICA. Second assumption is co-occurrence of all the frequency components in the same source. Namely, in each source, all the frequency components simultaneously have the strong powers / as shown in this figure. Let’s see the spectrogram observed in this research(指しながら). This is the spectrogram of the observed signal obtained by the radar sensor. There is a clear harmonic structure in the heartbeat and vibration components. And their fundamental components and its harmonic components are simultaneously activated. Thus, we can say that the two assumptions of IVA hold. まず，独立ベクトル分析IVAについて説明します． IVAでは先程の時間周波数領域ICAで用いている各信号源間の独立性の最大化という仮定に加えて，（図を指しながら）各信号源の全周波数成分の強弱が時間的に同期するという新しい仮定を追加して，周波数毎の分離行列Wiをより高精度に推定します． つまり，この図のように，全周波数のエネルギーが同時に強くなったり弱くなったりする成分を一つの信号源にまとめるようにBSSが行われます． （図を指しながら）レーダセンサから得られる観測信号では心拍成分及び振動成分に明確な調波構造が存在しており，これらの基本周波数とその整数倍の周波数の成分が同時に強くなったり弱くなったりしていることが分かります． そのため，IVAの新しい仮定がある程度成り立つことが言え，より高精度な分離が期待できます．
- Second, / we explain independent low-rank matrix analysis / ILRMA. ILRMA has two assumptions. First assumption is maximizing independence between each signal as with ICA and IVA. Second assumption is a time-frequency structure of each source tends to be low-rank. Here, low-rank means that spectrogram includes similar spectra many times. In ILRMA, low-rankness is encouraged by modeling as a product of two vectors T and V. Again, let’s see the spectrogram of the observed signal. The component of heartbeat and vibration plate have the same spectrum for a long time. Namely, we can say that this time-frequency structure is low-rank. Therefor, the assumptions in ILRMA hold. 次に，独立低ランク行列分析ILRMAについて説明します． ILRMAではIVAと同様に各信号源間の独立性の最大化という仮定に加えて，（図を指しながら）各信号源の時間周波数構造が低ランクという新しい仮定を追加して，周波数毎の分離行列Wiをより高精度に推定します．ここで，低ランクになる，とは限られた数のスペクトルパターンが繰り返し出現する性質のことです． 例えば，音響信号であればドラムのスネアの音が何度も登場するように，同じ音色が繰り返し出現すれば時間周波数構造は低ランクになります． ILRMAの中では，分離された信号源の時間周波数構造を，TとVという別の二つの行列の積でモデル化することで低ランク性を促しています． （図を指しながら）レーダセンサから得られる観測信号では，心拍成分及び振動台成分はほぼ同じスペクトルが長時間生じていることが分かります． つまり，ILRMAの低ランク性仮定はほぼ成り立つことが予想され，より高精度な分離が期待できます．
- Third, we explain t-ILRMA. T-ILRMA is a generalized method of ILRMA. In ILRMA, we assume the complex Gaussian(ｺﾞｰｽｨｱﾝ) distribution as a source generative model. However, in t-ILRMA, we assume the Student’s t-distribution instead of the Gaussian distribution. The student’s t-distribution is given by this equation, where nu and sigma represent DoF and scale parameters, respectively. If we set nu to 1, 2, and 5, it becomes a heavy-tale distribution. And If we set nu to infinity, the distribution coincides with the Gaussian distribution. This heavy-tale property of the generative model tends to provide more robust and precise results. こちらは，本研究で用いる3つ目のBSSであるt-ILRMAです． t-ILRMAとはILRMAの生成モデルである複素Gauss分布を，それを含む複素Student t分布へ一般化した手法です． 複素Student t分布は複素Gauss分布よりも裾野の重い分布で，音響信号処理の分野においてピーキーなスペクトログラムの多い音楽信号に対しても柔軟にフィットしより高性能で頑健な音源分離を可能にすることが知られています． 複素Student t分布の確率密度関数はこの式のように与えられ，nuが形状母数，sigmaが尺度母数です． nuを無限大とすればGauss分布になります．
- We will now describe the results of applying the three methods explained above IVA, ILRMA, and t-ILRMA to the observed signals. First, let us describe the experimental conditions. The STFT parameters are shown here, and the parameters for each of the three methods are shown here. それでは，観測信号にILRMAを適用した結果について説明していきます． STFTの窓長は1.6秒，シフト長は0.1秒，窓関数はハミング窓を用いています． ILRMAの最適化変数である分離行列Wiの初期値は，全周波数で単位行列とし，反復最適化アルゴリズムの反復回数は100回と設定しました． また，ILRMAの低ランクモデリングに用いる最適化変数であるT及びVの初期値は0から1の一様乱数とし，各信号源のランク数は3としています．
- We compare the separated signals of IVA, ILRMA, and t-ILRMA． It can be seen that the vibration table component is mostly eliminated in all the separation results. In IVA result, we can see that the vibration noise caused by the vibration plate is remains around 3.6Hz. In ILRMA results, we can still see the residual components of the vibration plate around 100 seconds, but the other components are well separated. t-ILRMA shows the highest accuracy of the separation compared with the results of IVA and ILRMA. IVA,ILRMA,t-ILRMAの分離信号の比較をします． 左側からIVA，ILRMA，t-ILRMAの分離信号となっています． どの分離結果も振動台成分が概ね除去されていることが分かります． IVAでは3.6Hz付近に振動台の振動成分が残留していることが分かります． ILRMAでは100付近の，3.6, 5Hzに見られる振動台の振動成分の残留が目立ちますが，その他の振動台成分は分離されていることが分かります． t-ILRMAでは，IVAとILRMAと比較して最も分離精度が高いことが分かります．
- Finally, we explain the heart rate estimation algorithm.
- We explain the heart rate estimation algorithm used to estimate the heart rate from the separated signals. The separated signal obtained by BSS is in the time-frequency domain, so we converted back to a time waveform by applying inverse STFT at first. In general biometric signal processing, the heartbeat is estimated by the RRI, which is the interval of R waves in the time waveform. However, in our case, RRI estimation does not work due to the residual noise. So, we apply another heart rate estimation algorithm. First, we emphasize only the heartbeat components in the time domain. Then, band-pass filter is applied. After that, in the time-frequency domain we detect the peak of spectra to obtain the heart rate values. 分離信号から心拍数を推定するために用いる心拍推定アルゴリズムについて説明します． BSSで得られる分離信号は時間周波数領域なので，まずは逆STFTを適用することで時間波形に戻します． 一般的な生体信号処理では，時間波形中のR波の間隔であるRRIで心拍を推定しますが，本研究の心拍推定信号には，ノイズが残留する影響でうまく心拍を推定できないため（図を指しながら）このような心拍推定方法を用います． 逆STFTを適用した分離信号の時間波形に対して，心拍信号の強調をし，バンドパスフィルタを適用したのちにスペクトルのピーク検出を行い推定心拍値を取得しています．
- Here are the results of applying the heart rate estimation algorithm to the separated results obtained by IVA, ILRMA and t-ILRMA. The vertical axis represents heart rate and the horizontal axis represents time. The red graph shows the heart rate estimation results obtained by separated signals and the blue graph shows the heart rate of the ECG sensor used as the reference value. It can be seen that t-ILRMA shows the highest heart rate estimation accuracy among the three methods. The t-ILRMA shows deviations from the reference value in the first 60 to 100 seconds after the vibration is applied, but at other times the values are almost identical to the reference value. こちらがIVA,ILRMAとt-ILRMAを適用した分離結果に心拍推定アルゴリズムを適用した結果です． 縦軸が心拍数を，横軸が時間を表しています． 赤のグラフが参照値としているECGセンサから得られる心拍推定結果，青のグラフが分離信号の心拍推定結果を表しています． また，左側からIVA，ILRMA，t-ILRMAの心拍推定結果となっています． 3つの手法の中でt-ILRMAが最も高い心拍推定精度を示すことが確認できます． t-ILRMAでは，振動が加えられた初めの60秒から100秒付近で参照値から外れた値を示しますが，その他の時間ではほとんど参照値と一致していることが分かります．
- This is a conclusion. That’s all. Thank you for your attention.
- We explain the optimization algorithm of IVA. IVA optimizes the demixing matrix Wi by solving this minimization problem. The first term can be interpreted as maximizing the independence between sources and the second term as emphasizing the total frequency co-occurrence of each source. An efficient optimization algorithm called the iterative projection method, shown here, has been proposed for this optimization problem and is also used in this research. In this calculation, as shown in the figure, the optimization algorithm updates one row of the demixing matrix Wi while keeping the other rows fixed. By iterating this process over and over for all frequency demixing matrices, it is guaranteed that the value of this equation will become smaller and smaller. 収束性言わなくていい このIVAの最適化アルゴリズムについて説明します． （式を指しながら）IVAではこちらの最小化問題を解くことで分離行列Wiの最適化を行います． 第一項では信号源間の独立性の最大化を，第二項では各信号源の全周波数共起性の強調を行っていると解釈できます． この最適化問題には，こちらの式で示される反復射影法と呼ばれる効率的な最適化アルゴリズムが提案されており，本研究でもこれを用います． この計算では，この図のように分離行列Wiのある行を更新するときに，ほかの行は固定しておくという最適化アルゴリズムです． このような処理を全ての周波数の分離行列に対して何度も反復計算していくことで，この式の値が小さくなっていくことが保証されています．
- We explain the optimization algorithm of ILRMA. ILRMA simultaneously optimizes the separation matrix Wi and the other matrices T and V by solving this minimization problem. The first term can be interpreted as maximizing independence, and the second term as maximizing low-rankness of each signal source. Here, D is the Itakura-Saito pseudo-distance. ILRMA also uses an iterative projection method to optimize Wi as well as IVA. T and V can also be optimized by repeating this equations. By iterating this process over and over again, it is guaranteed that the value of this equation will become smaller. This is the explanation of IVA and ILRMA. ILRMAの最適化アルゴリズムについて説明します． （式を指しながら）ILRMAではこちらの最小化問題を解くことで分離行列Wiとほかの行列TとVの同時最適化を行います． 第一項では独立性の最大化を，第二項では各信号源の低ランク性の最大化を行っていると解釈できます． ここでDは板倉斎藤擬距離を表しています． ILRMAでもIVAと同様にWiの最適化には反復射影法を用いています． TとVについてもこちらの式を繰り返すことで最適化を行えます． このような処理を何度も反復計算していくことで，この式の値が小さくなっていくことが保証されています． 以上がIVAとILRMAの説明になります．
- We explain the optimization algorithm of ILRMA. ILRMA simultaneously optimizes the separation matrix Wi and the other matrices T and V by solving this minimization problem. The first term can be interpreted as maximizing independence, and the second term as maximizing low-rankness of each signal source. Here, D is the Itakura-Saito pseudo-distance. ILRMA also uses an iterative projection method to optimize Wi as well as IVA. T and V can also be optimized by repeating this equations. By iterating this process over and over again, it is guaranteed that the value of this equation will become smaller. This is the explanation of IVA and ILRMA. ILRMAの最適化アルゴリズムについて説明します． （式を指しながら）ILRMAではこちらの最小化問題を解くことで分離行列Wiとほかの行列TとVの同時最適化を行います． 第一項では独立性の最大化を，第二項では各信号源の低ランク性の最大化を行っていると解釈できます． ここでDは板倉斎藤擬距離を表しています． ILRMAでもIVAと同様にWiの最適化には反復射影法を用いています． TとVについてもこちらの式を繰り返すことで最適化を行えます． このような処理を何度も反復計算していくことで，この式の値が小さくなっていくことが保証されています． 以上がIVAとILRMAの説明になります．