This document describes a system called CogWatch that uses parallel hidden Markov model (HMM) detectors and generative/discriminative models to recognize human activities for rehabilitation of stroke patients. CogWatch instruments objects like a tea set to recognize subgoals of making a cup of tea. It uses sensors on objects and a Kinect camera. Parallel HMM detectors address overlapped subgoals. The HMMs model inter- and intra-user variability in speed and sequences. Output probabilities are modeled using deep neural networks and Gaussian mixture models. Real-time decoding is achieved using a partial traceback algorithm. Experimental results show error reductions of up to 79% for recognizing subgoals using the DNN-HMM system compared to the GMM-HMM

1. Rehabilitation of stroke patients with parallel
GMM- and DNN-HMM based human activity
August 2016
GMM- and DNN-HMM based human activity
recognition using instrumented objects
Maryam Najafian, Roozbeh Nabiei, Prof. Martin Russell, Prof. Allen Wing
m.najafian@utdallas.edu [rxn946, m.j.russell,a.m.wing]@bham.ac.uk
School of Electrical and Computer Engineering, and School of Psychology

2. Outline
Motivation: CogWatch overview
System design: instrumentation and sensors
System description: parallel detector structureSystem description: parallel detector structure
Experimental results and discussion
Summary

3. CogWatch and Stroke Rehabilitation
68% of stroke survivors in the UK suffer from
Apraxia or Action Disorganization Syndrome
Apraxia is impairment of cognitive ability toApraxia is impairment of cognitive ability to
carry out activity of daily livings (ADLs)
– Self feeding (Making cup of tea)
CogWatch aims to enhance the rehabilitation
process of stroke patients suffers Apraxia
https://www.youtube.com/watch?v=MiLUUmPlWkc&index=3&list=PLUVuIyC7hO
z7QXSXm89KKJ2-GnAh-CzAB

4. Sensors Involved in Tea-Making
CogWatch instrumented coaster
– Three-axes accelerometers
– Force sensitive resistors– Force sensitive resistors
– Wireless module
Kinect camera

5. Tea-Making Setup and Sub-goals Involved

6. Parallel Subgoal Detectors

7. Subgoal Detector Diagram

8. Structure of the ‘target sub-goal HMM’ and
‘background HMM’ Models in a Single Detector

9. Different Stages of DNN Based Modelling

10. Real-time Viterbi Decoding (Partial
Traceback Algorithm )

11. Experimental Results
The relative error reduction of 79%, 50%, 42%, 36%, 13% and 13% is achieved for detection of ‘Add sugar’, ‘Pour
kettle’, ‘Add teabag’, ‘Add milk’, ‘Remove teabag’ and ‘Stir’ sub-goals, respectively. Surprisingly, for the ‘Fill kettle’
sub-goal the performance was improved by using the GMM- rather than a DNN-HMM based system. This may be
due to the fact that generative models are stronger in modelling tasks which includes unseen data.

12. Summary
CogWatch system introduced for stroke patients
to improve the rehabilitation
Parallel HMM detector used for action recognition:
addresses overlapped actions
HMM detector: addresses inter- and intra-user
speed and sequence variabilities
HMM output probability distributions were
modeled using DNNs and GMMs: discriminative
rather than generative
Partial trace-back algorithm is used to implement
the real-time Viterbi decoding

13. Thank youThank you