Prediction of patient respiratory signal using deep learning model: LSTM

Prediction of patient respiratory signal using
deep learning model (LSTM) :
Comparison deep learning model with machine learning model
Wonjoong Cheon1), Sang Hoon Jung2), Moonhee Lee1), Jinhyeop Lee1),
Heechul Park1), Youngyih Han2)*
1) Department of Health Sciences and Technology, SAIHST, Sungkyunkwan University, Seoul, 06351, Korea.
2) Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, 06351, Korea.
1
제 56회 한국의학물리학회 춘계학술대회 (2018.04.14) @ 제주 켄싱턴 리조트 서귀

Contents
• Purpose: Comparative evaluation of respiratory signal prediction accuracy among AI model
• Introduction
• Why is respiratory signal prediction important?
• What is difference between machine learning and deep learning?
• Materials and Methods
• Real-time position management (RPM)
• Decision Tree vs Multilayer perceptron vs Long-short term memory
• Result
• Conclusion & Discussion

Introduction:
• Intra-fraction motion can be caused by the respiratory, skeletal muscular,
cardiac, and gastrointestinal systems.
• Internal target volume (ITV) or gated radiotherapy are common treatment
method of radiation therapy that consider patient’s respiration.
• BUT, Real-Time tumoR Tracking (RTRT) is the state-of-art technique for
moving tumor treatment in the field of radiotherapy. (FIG .2)
• Prediction of the respiratory signal pattern is indispensable for dose
delivery to the tumor in RTRT.
FIG 2. Real-time tumor tracking (RTRT)
FIG 1. Changes in internal organs due to
patient's respiration
https://www.youtube.com/watch?v=LhhhLE_8nhY
FIG 3. Respiratory breathing signal

Introduction: Conventional methods
• To predict the target position associated with the respiratory motion, a number of mathematical models were
developed by other investigators
• Autoregressive integrated moving average (ARIMA) model (Khashei et al 2011, Babu et al 2014)
• ARIMA model basically assumed that the relationship between prior- and future-respiratory signals was linear and
fitted using a linear function.
• Kalman filtering method
• Requires establishing a state equation and an observation equation first. However, initial parameters in these
equations were difficult to determine
• Artificial neural network (AI) (Yan et al 2006a, 2006b, Seregni et al 2013)
• Provided effective prediction for both linear and non-linear respiratory signals (Tsai and Li 2008).
• However, define the hyper-parameters were difficult to determine
So, we conducted a performance evaluation between the AI methods known to be effective prediction performance.

What is difference
between machine learning and deep learning
https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/

Materials and Methods

Materials and Methods: RPM data
• Data acquisition
• The respiratory signal was obtained RPM system
(Varian Medical System, Palo Alto, CA)
• RPM system comprise infrared tracking camera and infrared reflective
(IR) markers.
• The RPM system could record the patient respiratory patterns with a sampling
rate of 30 Hz
• The cube phantom with IR-markers attached is placed on patient abdomen.

:Preprocessing for learning
• Normalization (0, 1), [ Normalized signal ]
• Smoothing (S-G filter), [ Original signal ], [Filtered signal ]

• [Training set ] & [Test set ]

• [Training set ] & [Test set ]
• [Input ] & [Output 0.5: , 0.6: , 0.7: , 0.8: , 0.9: , 1.0: , 0.5 ~ 1.0 sec: ]
input
seq.
input
seq.
input
seq.
input
seq.
input
Seq.
seq.

:The kinds of algorithm and network for prediction of respiratory signal
Decision Tree Multi-layer perceptron Long-Short term memory
𝑖𝑛𝑝𝑢𝑡𝐺𝑎𝑡𝑒(𝑡) = 𝜎(𝑊𝑥𝑖 ∙ 𝑥(𝑡) + 𝑊ℎ𝑖𝑇 ∙ ℎ(𝑡 − 1) + 𝑏𝑖)
𝑓𝑜𝑟𝑔𝑜𝑡𝐺𝑎𝑡𝑒(𝑡) = 𝜎(𝑊𝑥𝑓 ∙ 𝑥(𝑡) + 𝑊ℎ𝑓𝑇 ∙ ℎ(𝑡 − 1) + 𝑏𝑓)
𝑜𝑢𝑡𝑝𝑢𝑡𝐺𝑎𝑡𝑒(𝑡) = 𝜎(𝑊𝑥𝑜 ∙ 𝑥(𝑡) + 𝑊ℎ𝑜𝑇 ∙ ℎ(𝑡 − 1) + 𝑏𝑜)
• LSTM structure
Root node
Parent node
Child node
FC
• LSTM parameters
• The number of neuron in hidden dim : 8
• Hidden layer: 1
• Activation function: tanh
• Batch size: 200
• Epoch: 1000
• LSTM parameters
• The number of neuron in hidden dim : 8
• Hidden layer: 1
• Activation function: tanh
• Batch size: 200
• Epoch: 1000
• MLP Structure
𝑖𝑛𝑝𝑢𝑡𝐺𝑎𝑡𝑒(𝑡) = 𝜎(𝑊𝑥𝑖 ∙ 𝑥 𝑡 + 𝑏𝑖)

Results

Comparative evaluation: one point prediction
[sec] 0.5 0.6 0.7 0.8 0.9 1
Decision Tree
Ave. CORR 0.840 0.806 0.784 0.749 0.721 0.694
Ave. RMSE 0.144 0.157 0.165 0.176 0.184 0.191
Multi-layer
perceptron
Ave. CORR 0.885 0.848 0.821 0.786 0.750 0.717
Ave. RMSE 0.122 0.140 0.151 0.164 0.175 0.185
Long-short
term memory
Ave. CORR 0.914 0.894 0.873 0.838 0.812 0.794
Ave. RMSE 0.108 0.120 0.129 0.145 0.156 0.166

Comparison among models Per patients:
prediction after 0.5 second
-0.7
-0.5
-0.3
-0.1
0.1
0.3
0.5
0.7
0.9
1.1
1
6
11
16
21
26
31
36
41
46
51
56
61
66
71
76
81
86
91
96
101
106
111
116
121
126
131
136
141
146
151
156
161
166
171
176
181
186
191
196
201
206
211
216
221
226
231
236
241
246
251
256
261
266
271
276
281
286
291
296
Prediction of patient repiratory signal (after 500 ms )
Decision Tree (RMSE) Multi-layer perceptron (RMSE) Long-Short term memory (RMSE)
Decision Tree (CORR) Multi-layer perceptron (CORR) Long-short term memory (CORR)
[patient idx]

Comparative evaluation: Sequence prediction
[sec] Sequence (0.5 ~ 1.0) : 30 Hz
Decision Tree
Ave. CORR 0.7729
Ave. RMSE 0.1675
Multi-layer
perceptron
Ave. CORR 0.8314
Ave. RMSE 0.1474
Long-short
term memory
Ave. CORR 0.8823
Ave. RMSE 0.1189

Optimization Long short-term memory:
Point prediction
Irregularpattern
Regularpattern

Sequence prediction

Conclusion

Conclusion & Discussion
• In the condition that the number of parameters of the AI models are the same, both the problem
of predicting Point and the problem of predicting continuous time show the highest CORR and the
lowest RMSE of LSTM.
• In conclusion, it can be seen that the LSTM exhibits excellent performance in the time-series
data prediction
• In this study, we performed the prediction of respiratory data obtained by external surrogator
of patient. Further studies are underway to predict the internal target motion synchronized with
the external surrogate data to improve the accuracy and stability of the RTRT.
• In the prediction study, the time required for the prediction is also important in order to be
applied to the actual clinical situation. In the case of the LSTM, a time delay of 0.02 sec occurs in
the calculation process.

Thank you for your
attention. 

Back-up slide

Comparison among models Per patients:
prediction after 1.0 second
-0.7
-0.5
-0.3
-0.1
0.1
0.3
0.5
0.7
0.9
1.1
1
6
11
16
21
26
31
36
41
46
51
56
61
66
71
76
81
86
91
96
101
106
111
116
121
126
131
136
141
146
151
156
161
166
171
176
181
186
191
196
201
206
211
216
221
226
231
236
241
246
251
256
261
266
271
276
281
286
291
296
Prediction of patient repiratory signal (after 1000 ms )
Decision Tree Multi-layer perceptron Long-Short term memory
[patient idx]

Point prediction

What is Decision Tree (DT)
• text

Conclusion & Discussion
• AI model들의 파라미터 수를 동일하게 맞춘 조건에서 Point를 예측하는
문제와 연속적인 시간을 예측하는 문제에서 모두 LSTM 가장 높은 CORR
과 가장 낮은 RMSE를 보였다.
• 앞, 뒤 신호간의 관계가 성립될 수 있는 시계열 데이터 예측에 LSTM이 뛰
어난 성능을 보이는 것을 알 수 있다.
• 본 연구에서는 환자의 external surrogator로 획득한 호흡데이터의 예측을
수행하였지만, external surrogate 데이터와 동기화된 internal target
motion 데이터를 토대로 internal target motio을 예측해 RTRT의 정확성
및 안정성을 향상 시킬 수 있는 후속연구를 진행중이다.
• LSTM의 경우 연산과정에서 0.02 sec 의 time delay가 발생한다.

What is Multi-Layer Perceptron (MLP)
• text

What is Long-short term memory (LSTM)
• text
FC
Structure of learning frame

Prediction of patient respiratory signal using deep learning model: LSTM

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Similar to Prediction of patient respiratory signal using deep learning model: LSTM

Similar to Prediction of patient respiratory signal using deep learning model: LSTM (20)

More from WonjoongCheon

More from WonjoongCheon (6)

Recently uploaded

Recently uploaded (20)

Prediction of patient respiratory signal using deep learning model: LSTM

Editor's Notes