Abnormal gait recognition plays an important role in diagnosis of musculoskeletal disorders. Suspicious walking behaviours should be detected as early as possible, and possibly in real-time, in order to prevent further deterioration of any part of the body. Analysis tools should provide useful and accurate information via convenient setup procedures. In this work, we conduct a system for recognizing gait abnormalities in real-time where an input is an image frame captured by a single RGB camera at any instance. We view abnormal gait recognition as a time-series problem which requires learning long-term dependencies. Hence, the system is presented with variants of Recurrent Neural Networks (RNNs). The proposed deep neural network model involves extracting 135 human body key points using OpenPose prior to performing recognition task which are quantitatively evaluated based on a simple RNN, Long Short-Term Memory Network (LSTM), and Gated Recurrent Unit Network (GRU). Each deep neural network has a model accuracy about 73.4%, 82.8%, and 81.6%, respectively. According to the confusion matrices of different predictive models, LSTM and GRU provide less confusing predictive results than that of a simple RNN. Therefore, we have discovered that deep neural network based on LSTM is, by far, the suitable model to recognize abnormal gaits due to the high model accuracy with less training and inference time.

1. Abnormal Gait Recognition
in Real-Time using
Recurrent Neural Networks
Thanaporn Jinnovart1
, Xiongcai Cai1
, and Kundjanasith Thonglek2
1
University of New South Wales, Sydney, Australia
2
Nara Institute of Science and Technology, Nara, Japan
1

2. Outline ❖ Introduction
❖ Methodology
❖ Evaluation
❖ Conclusion
2

3. Introduction
3

4. 4
Gait Abnormality
❖ Gait abnormality is a deviation from normal walking (gait)
❖ Watching patient walk is the most important part of the neurological
examination
❖ Normal gait requires many systems
➢ Strength
➢ Sensation
➢ Coordination

5. 5
Diversity of Gait Abnormalities

6. 6
Existing Techniques
Doctor’s Diagnosis Motion Sensor Depth Camera

7. Methodology
7

8. 8
Approach
❖ We would like to apply artificial intelligence technology to recognize the
gait abnormality for increase the model accuracy and decrease the
recognize time
❖ We apply the recurrent neural network which is the time series
forecasting technique to recognize the gait from the streaming video by
using only one RGB camera

9. 9
Proposed System Architecture

10. Evaluation
11

11. 12
Evaluation Process

12. 13
Confusion Matrices
Recurrent Neural Network
(RNN)
Long Short-Term Memory
(LSTM)
Gated Recurrent Unit
(GRU)

13. 14
Model Accuracy
Method
Normal Gait Abnormal Gait Accuracy
(%)
True False True False
HSTOD 728 272 547 453 63.75
RNN 805 195 889 111 84.70
LSTM 887 113 926 74 90.65
GRU 831 169 903 97 86.70

14. 15
Training time & Inference time
Training Time Inference Time

15. Conclusion
16

16. Conclusion
❖ The proposed method requires only single RGB camera
❖ The proposed method is able to recognize 5 gait abnomalities
➢ Spastic gait abnormality
➢ Scissors gait abnormality
➢ Steppage gait abnormality
➢ Waddling gait abnormality
➢ Propulsive gait abnormality
❖ The proposed method has the recognition accuracy around 82%
17

17. Future Work
❖ We would like to investigate the time-series forecasting
techniques other than recurrent neural networks
❖ The walking postures from other large-scale datasets should be
used to evaluate the generality of our proposed predictive model
for gait abnormalities recognition
❖ Significant features and postures that impact gait abnormalities
will be investigated
18

18. Q & A
Thank you
Email: thonglek.kundjanasith.ti7@is.naist.jp 19