1. MYOELECTRIC PROSTHESES
NOVEL METHODOLOGIES FOR ENHANCING
USABILITY AND CONTROL
Ghulam Rasool
Dissertation Defense
April 29, 2014 1

2. Outline
o Background
o Muscle Synergies Based Task Discrimination (MSD)
 Mathematical Model
 Discrimination Scheme
 Results
o Autoregressive-Generalized Autoregressive Conditional
Heteroscedastic (AR-GARCH) Based Task Discrimination
o Possible Future Extensions and Conclusion
2

3. Myoelectric Control Problem
o Prosthetic devices
 Powered
 Controlled by myoelectric signals
 Leftover musculature in the forearm
 Multifunction capability
o Virtual prosthetic device
3
Robotic
hand
Battery
Amplifiers
EMG electrodes

4. Pattern Classification Systems
o Machine learning and pattern classification schemes [1,2]
 Muscular activations are repeatable
 Training and discrimination
 High discrimination accuracies in off-line
 Acceptable controllability and real-time performance
4

5. Pattern Classification Systems
o Information about system physiology and configuration
 Muscles acting synergistically
 Muscles acting in agonist/antagonist pairs
5
Investigate physiologically relevant mathematical
models and algorithms for real-time task discrimination

6. Contributions
o Development of a physiologically relevant mathematical model
for task discrimination:
 Hypothesis of muscle synergies
 State space representation
 Efficient and robust algorithm
o Verification of the proposed model using experimental data
o Development of a virtual platform (BioPatRec) for upper
extremity myoelectric prosthesis testing.
o Analysis of feature sets for EMG control especially the
autoregressive (AR) feature set.
o Model the EMG signal as an AR-GARCH process and perform
task discrimination using the AR-GARCH model parameters.
6

7. Recent Work
o Muscle synergies as a
predictive framework [24].
 The authors used a subset of
static hand postures of the
American Sign Language (ASL)
to estimate the muscle
synergies.
 Later predicted remaining ASL
postures.
 Not extended for task
discrimination.
7

8. Outline of the Proposed Scheme
Muscle Synergy based task Discrimination (MSD)
8
State-space model
Hypothesis of
muscle synergies
Mathematical
Model
Algorithm
Perform task
discrimination
Extract muscle
synergies
Estimate latent
state

9. Hypothesis of Muscle Synergies
o Commands from higher centers of
the Central Nervous System (CNS) to
the spinal cord.
o Spinal cord circuits modulate these
commands and issue muscle
activation nerve signals.
o Muscle synergies are hypothesized
to exist in the spinal cord.
9

10. 10
⋯
𝑥𝑥1
𝑥𝑥2 ⋯
A B C D E
Activation
coefficients
First Synergy 𝑊𝑊1
Second Synergy 𝑊𝑊2
𝑥𝑥1
𝑥𝑥2
𝑥𝑥1
𝑊𝑊1
𝑥𝑥2
𝑊𝑊2
A B C D E
Hypothesis of Muscle Synergies
Modified from Ref: [7]
Resulting muscle
activation
𝒙𝒙𝟏𝟏
= 𝒙𝒙𝟐𝟐
= 𝟏𝟏
A B C D E

11. Hypothesis of Muscle Synergies
𝑦𝑦1
⋮
𝑦𝑦 𝑚𝑚
=
𝑊𝑊11 … 𝑊𝑊1𝑛𝑛
⋮ ⋮
𝑊𝑊𝑚𝑚𝑚 … 𝑊𝑊𝑚𝑚𝑚𝑚
𝑥𝑥1
⋮
𝑥𝑥 𝑛𝑛
𝑛𝑛 : number of muscle synergies
𝑚𝑚 : number of muscles
𝒚𝒚 = 𝑊𝑊𝒙𝒙
11
Muscle activations
Synergy activation coeffs
Or the neural drive
1st Synergy nth Synergy

12. Hypothesis of Muscle Synergies
Stream of activation coefficients
𝒙𝒙𝟏𝟏
𝟏𝟏
𝒙𝒙𝟏𝟏
𝟐𝟐
,
𝒙𝒙𝟐𝟐
𝟏𝟏
𝒙𝒙𝟐𝟐
𝟐𝟐
⋯
𝒙𝒙𝒌𝒌
𝟏𝟏
𝒙𝒙𝒌𝒌
𝟐𝟐
A B C D E A B C D E A B C D E
Muscle activations at
time 𝑘𝑘
Muscle activations at
time 2
Muscle activations at
time 1
⋯
Muscle Synergy Matrix
12

13. The State-Space Model
o Mathematical interpretation
13
⋯𝒙𝒙1 𝒙𝒙𝑘𝑘−1 𝒙𝒙𝑘𝑘
⋯𝒚𝒚1 𝒚𝒚𝑘𝑘−1 𝒚𝒚𝑘𝑘
System state 𝒙𝒙𝒌𝒌 : synergy activation coefficients
Output 𝒚𝒚𝒌𝒌 : Multichannel EMG signal
System dynamics model
Observation model

14. The State-Space Model
o State-space Model
𝒙𝒙𝑘𝑘+1 = 𝒇𝒇𝑘𝑘(𝒙𝒙𝑘𝑘) + 𝒘𝒘𝑘𝑘,
𝒚𝒚𝑘𝑘 = 𝒉𝒉𝑘𝑘(𝒙𝒙𝑘𝑘) + 𝒗𝒗𝑘𝑘,
𝒙𝒙𝑘𝑘 ∈ ℝ𝑛𝑛𝑥𝑥 : system state, the neural drive, or the synergy coefficients
𝒚𝒚𝑘𝑘 ∈ ℝ𝑛𝑛𝑦𝑦 : system output, the EMG signal,
𝑘𝑘 ∈ ℕ : time index
𝒇𝒇𝑘𝑘: ℝ𝑛𝑛𝑥𝑥 → ℝ𝑛𝑛𝑥𝑥 : state dynamics,
𝒉𝒉𝑘𝑘: ℝ𝑛𝑛𝑥𝑥 → ℝ𝑛𝑛𝑦𝑦 : observation model,
𝑛𝑛𝑥𝑥 and 𝑛𝑛𝑦𝑦 : dimensions of system state and output
𝒘𝒘𝑘𝑘 and 𝒗𝒗𝑘𝑘 : process and measurement noise with known density functions
14

15. The State-Space Model
o System Dynamics : Random walk model
𝒙𝒙𝑘𝑘+1 = 𝒙𝒙𝑘𝑘 + 𝒘𝒘𝑘𝑘
15
o Smooth time-evolution.
o A lack of a priori knowledge.
●
𝑥𝑥1
𝑥𝑥2
𝒙𝒙𝑘𝑘

16. The State-Space Model
o Observation Model: Derived from the hypothesis of
muscle synergies
𝒚𝒚𝑘𝑘 = 𝑊𝑊𝒙𝒙𝑘𝑘 + 𝒗𝒗𝑘𝑘,
where 𝑊𝑊 is the synergy matrix.
o Relates the output 𝒚𝒚𝑘𝑘 to the latent system state 𝒙𝒙𝑘𝑘
through a linear mapping, the synergy matrix 𝑊𝑊.
16

17. MSD Scheme
17
Synergy
extraction
Task
discrimination
Post
processing
𝑰𝑰𝒌𝒌
EMG Data
(𝐾𝐾 Trials,
𝑞𝑞 tasks)
EMG Data
(Single measurement,
unknown task)
Synergy
matrices
Discrimination
decision
State
estimation
Estimated
neural drive
Discrimination
Decision

18. Synergy Extraction
o For 𝐾𝐾 trials, we have
𝒀𝒀 𝑚𝑚×𝐾𝐾 = 𝑾𝑾 𝑚𝑚×𝑛𝑛 × 𝑿𝑿𝑛𝑛×𝐾𝐾
o Blind Source Separation (BSS) problem [8].
o For the case of muscle synergies [9].
 Principal Component Analysis (PCA) [10].
 Factor Analysis (FA) [11].
 Nonnegative Matrix Factorization (NMF) [12].
 Independent Component Analysis (ICA) [13].
 Probabilistic Independent Component Analysis (pICA) [14].
18

19. Synergy Extraction
o pICA Algorithm
 Non-Gaussian Data, statistically independent
 Non-negativity 𝑊𝑊 ≽ 0, and 𝑋𝑋 ≽ 0
≽ implies element-wise inequality
o Consider 𝑞𝑞 tasks
𝑊𝑊𝑖𝑖
, 𝑋𝑋𝑖𝑖
= 𝐩𝐩𝐩𝐩𝐩𝐩 𝐩𝐩 𝑌𝑌𝑖𝑖
𝑖𝑖 = 1, ⋯ , 𝑞𝑞.
o State-space model is now completely specified.
𝒙𝒙𝑘𝑘+1 = 𝒙𝒙𝑘𝑘 + 𝒘𝒘𝑘𝑘
𝒚𝒚𝑘𝑘= 𝑊𝑊𝑖𝑖
𝒙𝒙𝑘𝑘 + 𝒗𝒗𝑘𝑘 𝑖𝑖 = 1, ⋯ , 𝑞𝑞
19

20. State Estimation
o Kalman Filtering
 Kalman filter is the minimum mean-square estimator for
linear systems with additive white Gaussian noise [15].
𝒙𝒙�𝑘𝑘 = 𝐊𝐊𝐊𝐊𝐊𝐊 𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊𝐊 (𝒚𝒚𝑘𝑘)
o Non-negativity constraint [16]
𝒙𝒙�𝑘𝑘 = argmin
𝒙𝒙�
𝒙𝒙�𝑘𝑘 − 𝒙𝒙�𝑘𝑘
𝑇𝑇
(𝒙𝒙�𝑘𝑘 − 𝒙𝒙�𝑘𝑘), 𝒙𝒙�𝑘𝑘 ≽ 0
o We run 𝑞𝑞 Kalman filters in parallel and get {𝑥𝑥�𝑘𝑘
𝑖𝑖
}𝑖𝑖=1
𝑞𝑞
20

21. ⋯𝒙𝒙𝟏𝟏 𝒙𝒙𝒌𝒌−𝟏𝟏
⋯
𝒚𝒚𝟏𝟏 𝒚𝒚𝒌𝒌−𝟏𝟏
System state, the neural drive
System output, the EMG signal
System dynamics model
𝒙𝒙𝒌𝒌
𝒚𝒚𝒌𝒌
Observation
model
Estimated state
𝒒𝒒 Kalman Filters
Prediction
step
Filtering
step
{𝒙𝒙�1}𝑖𝑖=1
𝑞𝑞
𝒒𝒒 Kalman Filters
Prediction
step
Filtering
step⋯
𝒒𝒒 Kalman Filters
Prediction
step
Filtering
step
State estimation
State-space model
Task discrimination
Task discrimination and
post-processing
Task discrimination and
post-processing
Task discrimination and
post-processing
𝑰𝑰1 𝑰𝑰𝑘𝑘−1 𝑰𝑰𝑘𝑘
⋯
State Estimation
21
{𝒙𝒙�𝑘𝑘−1}𝑖𝑖=1
𝑞𝑞
{𝒙𝒙�𝑘𝑘}𝑖𝑖=1
𝑞𝑞

22. Task Discrimination
The pair of a task-specific muscle synergy matrix and its
corresponding coefficients {𝑊𝑊 𝑧𝑧, 𝒙𝒙�𝑘𝑘
𝑧𝑧
}, which will reconstruct
the muscle activations more accurately than all others
corresponds to the task being performed.
22

23. Method
o 12 Able-bodied participants
o Tasks
 Single DOF tasks (6)
 Multi-DOF tasks (6+12)
o Matlab and IBM SPSS
23
TeleMyo DTS DTS analog
module
NI-USB 6009

24. Data Collection
24

25. Results
25
Single-DOFMulti-DOF
Real-timeOff-line
Single-DOF
Analysis strategy
Task discrimination accuracy
1. Number of muscle synergies
2. Analysis window size
3. Computational aspects
4. Similarity measures
5. Comparison with the LDA
6. Robustness analysis
1. Task completion rate
2. Task completion time
3. Path efficiency
Virtual
prosthetic limb
Motion Test [25]
Target Achievement Control Test [17]

26. 0
5
10
15
20
MSD LDA MSD LDA
Single-DOF Tasks Multi-DOF Tasks
Taskdiscriminationerrors(%)
Discrimination schemes
Comparison with the LDA
26
p < .001 p < .001

27. Robustness: Single-DOF Tasks
27
0
1
2
3
4
5
6
7
8
0 2 4 6 8
Discriminationerror(%)
Discrimination error - LDA (%)
LDA MSD
𝑚𝑚 ≈ 45°
𝑦𝑦 = 0.59𝑥𝑥−0.02
𝑚𝑚 ≈ 31°
𝑅𝑅2 = 0.86

28. Robustness: Multi-DOF Tasks
28
0
5
10
15
20
1 6 11 16 21
Discriminationerror(%)
Discrimination error - LDA (%)
LDA MSD
𝑚𝑚 ≈ 45°
𝑦𝑦 = 0.68𝑥𝑥−1.26
𝑚𝑚 ≈ 34°
𝑅𝑅2 = 0.95

29. Real-time Testing (TAC Test)
29Targeted Achievement Control (TAC) Test Ref: [17]
0
0.2
0.4
0.6
0.8
1
0 2 4 6 8 10
Cumulativecompletionrate
Time (sec)
MSD
LDA
0.90±0.19
0.79±0.27
p <. 01
Parameters
o Number of trials = 2
o Repetitions = 3
o Dwell time = 1 sec
o Test time = 15 sec
o Allowed range = 5°

30. AR-GARCH Modeling of the
Myoelectric Signal
30

31. AR Modeling of the Myoelectric Signal
o Effect of choice of feature set.
o AR - common and robust feature set.
o No formal statistical methodology.
o AR modeling residuals exhibit
heteroscedasticity.
o Model AR residuals using the ARCH /
GARCH process.
31
AR
Modeling
Classification
AR coeffs GARCH
Modeling
AR residuals
Classification
AR-GARCH
coeffs
GARCH coeffs
Segmented myoelectric signal
Conventional
scheme
Proposed
scheme

32. AR-GARCH Model
32
Myoelectric
Signal
AR Modeling
GARCH Modeling
Classification
AR residuals
AR coeffs
(m+1)
GARCH coeffs
Class Label
AR-GARCH coeffs
(m+p+q+2)
Ljung-Box Q-Test ascertained correct AR model
Engle’s test established heteroscedasticity
K-S test found that both
processes were generated
from same probability
distribution
Goodness of fit
compared AR and
AR-GARCH models

33. AR-GARCH Model Results
33
0
2
4
6
8
10
12
14
AR(5) AR(9) AR(9)-GARCH(1,1)
Classificationerror(%)
ANN LDA
Extensively used
in the literature
Residuals become
white noise
Proposed
model

34. Future Work
34
•Healthy population
•Amputees
•Robustness under
electrode shift / rotation
•Synergy extraction
algorithms
•Lower extremity
•Multi-DOF
•Nonlinear model
•Constrained Particle
Filter
•Convergence issues
Clinical
Aspects
Algorithmic
Extension
Theoretical
Foundation

35. Conclusion
35

36. Publications
o Journal Publications
 Ghulam Rasool, Kamran Iqbal, Nidhal Bouaynaya and Gannon White, “Real-time Task
Discrimination for Myoelectric Control Employing Task-Specific Muscle Synergies”, under review
in IEEE Transactions on Neural Systems and Rehabilitation Engineering.
 Ghulam Rasool, Nidhal Bouaynaya, Kamran Iqbal and Gannon White, “Surface Myoelectric Signal
Classification Using the AR-GARCH Model”, under review in Biomedical Signal Processing and
Control.
 Gregory S. Taylor, Yupo Chan and Ghulam Rasool, “A Three Dimensional Bin-Packing Model: Exact
Multi-criteria Solution and Computational Complexity” under review in Naval Research Logistics
(NLR).
 Ghulam Rasool, Kamran Iqbal, Gannon A. White, “Myoelectric activity detection during a Sit-to-
Stand movement using threshold methods”, Computers and Mathematics with Applications
(CAMWA), Volume 64, Issue 5, September 2012, pp 1473-1483.
36

37. Publications
o Conference Publications
 Ghulam Rasool, Kamran Iqbal, Nidhal Bouaynaya and Gannon White, “Neural Drive Estimation Using the
Hypothesis of Muscle Synergies and the State-Constrained Kalman Filter”, in The 6th IEEE EMBS Neural
Engineering Conference, San Diego, November, 6-8, 2013.
 Ghulam Rasool, Nidhal Bouaynaya, Kamran Iqbal, “Muscle Activity Detection from the EMG signal based on the
AR-GARCH Method”, in IEEE Statistical Signal Processing Workshop (SSP), Ann Arbor, August 2012.
 Ghulam Rasool, Nidhal Bouaynaya, “Inference of Time-Varying Gene Networks using Constrained and
Smoothed Kalman Filtering, ” in IEEE International Workshop on Genomic Signal Processing and Statistics
(GENSIPS), Washington, DC, December 2012.
 Ghulam Rasool, Nidhal Bouaynaya, Hassan Fathallah-Shaykh and Dan Schonfeld, "Inference of Genetic
Regulatory Networks Using Regularized Likelihood with Covariance Estimation," in IEEE Statistical Signal
Processing Workshop (SSP), Ann Arbor, August 2012.
 Ghulam Rasool and Kamran Iqbal, “Muscle Activity Onset Detection Using Energy Detectors”, 34th Annual
International IEEE EMBS Conference, San Diego, August 28-September 1, 2012.
 Ghulam Rasool, Asif Mahmood Mughal, and Kamran Iqbal “Fuzzy Biomechanical Sit-To-Stand Movement with
Physiological Feedback Latencies”, IEEE International Conference on System, Man and Cybernetics (SMC) 2010,
pp 316-321, Istanbul, Turkey, October, 10-13, 2010.
 Ghulam Rasool, Hamza Farooq and Asif Mahmood Mughal, “Biomechanical Sit-To-Stand Movement with
Physiological Feedback Latencies”, 2010 2nd International Conference on Mechanical and Electronics
Engineering (ICMEE 2010), pp V1-159-V1-163, Kyoto, Japan, August 1-3, 2010.
37

38. References
[1] E. Scheme and K. Englehart, “Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and
challenges for clinical use,” The Journal of Rehabilitation Research and Development, vol. 48, no. 6, p. 643, 2011.
[2] S. Micera, J. Carpaneto, and S. Raspopovic, “Control of hand prostheses using peripheral information,” IEEE Reviews in Biomedical
Engineering, vol. 3, pp. 48–68, 2010.
[3] E. Bizzi, V. Cheung, A. d’Avella, P. Saltiel, and M. Tresch, “Combining modules for movement,” Brain Research Reviews, vol. 57, no. 1, pp.
125–133, Jan. 2008.
[4] J. Roh, W. Z. Rymer, E. J. Perreault, S. B. Yoo, and R. F. Beer, “Alterations in upper limb muscle synergy structure in chronic stroke survivors,”
Journal of Neurophysiology, vol. 109, no. 3, pp. 768–781, Feb. 2013.
[5] A. d’Avella and E. Bizzi, “Shared and specific muscle synergies in natural motor behaviors,” Proceedings of the National Academy of
Sciences of the United States of America, vol. 102, no. 8, pp. 3076–3081, Feb. 2005.
[6] M. Chhabra and R. A. Jacobs, “Properties of synergies arising from a theory of optimal motor behavior,” Neural Computation, vol. 18, no.
10, pp. 2320–2342, Aug. 2006.
[7] M. C. Tresch and A. Jarc, “The case for and against muscle synergies,” Current Opinion in Neurobiology, vol. 19, no. 6, pp. 601–607, Dec.
2009.
[8] A. Cichocki, R. Zdunek, A. H. Phan, and S. Amari, Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data
Analysis and Blind Source Separation. John Wiley & Sons, Jul. 2009.
[9] M. C. Tresch, V. C. K. Cheung, and A. d’Avella, “Matrix factorization algorithms for the identification of muscle synergies: evaluation on
simulated and experimental data sets,” Journal of Neurophysiology, vol. 95, no. 4, pp. 2199–2212, Apr. 2006.
[10] Y. Wang, T. Asaka, and K. Watanabe, “Multi-muscle synergies in elderly individuals: preparation to a step made under the self-paced and
reaction time instructions,” Experimental Brain Research, vol. 226, no. 4, pp. 463–472, May 2013.
[11] G. Cappellini, Y. P. Ivanenko, R. E. Poppele, and F. Lacquaniti, “Motor patterns in human walking and running,” Journal of Neurophysiology,
vol. 95, no. 6, pp. 3426–3437, Jun. 2006.
[12] L. H. Ting and S. A. Chvatal, “Decomposing muscle activity in motor tasks,” in Motor Control: Theories, Experiments, and Applications, F.
Danion and M. Latash, Eds. Oxford University Press, Dec. 2010.
[13] C. B. Hart and S. F. Giszter, “Distinguishing synchronous and time-varying synergies using point process interval statistics: motor primitives
in frog and rat,” Frontiers in Computational Neuroscience, vol. 7, no. 52, May 2013.
38

39. References
[14] P. A. d. F. R. Højen-Sørensen, O. Winther, and L. K. Hansen, “Mean-field approaches to independent component analysis,” Neural
Computation, vol. 14, no. 4, pp. 889–918, 2002.
[15] D. Simon, Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches. Wiley-Interscience, Jun. 2006.
[16] D. Simon, “Kalman filtering with state constraints: a survey of linear and nonlinear algorithms,” IET Control Theory Applications, vol. 4, no.
8, pp. 1303–1318, Aug. 2010.
[17] A. Simon, L. Hargrove, B. Lock, and T. Kuiken, “Target achievement control test: Evaluating real-time myoelectric pattern-recognition control
of multifunctional upper-limb prostheses,” J Rehabil Res Dev, vol. 48, no. 6, pp. 619–628, 2011.
[18] M. Ortiz-Catalan, R. Br°anemark, and B. H°akansson, “BioPatRec: a modular research platform for the control of artificial limbs based on
pattern recognition algorithms,” Source Code for Biology and Medicine, vol. 8, no. 11, Apr. 2013.
[19] Ghulam Rasool and Kamran Iqbal, “Muscle Activity Onset Detection Using Energy Detectors”, 34th Annual International IEEE EMBS
Conference, San Diego, August 28-September 1, 2012.
[20] Ghulam Rasool, Nidhal Bouaynaya, Kamran Iqbal, “Muscle Activity Detection from the EMG signal based on the AR-GARCH Method”, in
IEEE Statistical Signal Processing Workshop (SSP), Ann Arbor, August 2012.
[21] Ghulam Rasool, Kamran Iqbal, Gannon A. White, “Myoelectric activity detection during a Sit-to-Stand movement using threshold methods”,
Computers and Mathematics with Applications, Volume 64, Issue 5, September 2012, Pages 1473-1483, ISSN 0898-1221,
10.1016/j.camwa.2012.03.094.
[22] Ghulam Rasool, Asif Mahmood Mughal, and Kamran Iqbal “Fuzzy Biomechanical Sit-To-Stand Movement with Physiological Feedback
Latencies”, IEEE International Conference on System, Man and Cybernetics (SMC) 2010, pp 316-321, Istanbul, Turkey, October, 10-13, 2010.
[23] Ghulam Rasool, Hamza Farooq and Asif Mahmood Mughal, “Biomechanical Sit-To-Stand Movement with Physiological Feedback Latencies”,
2010 2nd International Conference on Mechanical and Electronics Engineering (ICMEE 2010), pp V1-159-V1-163, Kyoto, Japan, August 1-3,
2010.
[24] A. B. Ajiboye and R. F. Weir, ``Muscle synergies as a predictive framework for the EMG patterns of new hand postures,'' Journal of Neural
Engineering, vol. 6, no. 3, p. 036004, Jun. 2009.
[25] T. A. Kuiken, G. Li, B. A. Lock, R. D. Lipschutz, L. A. Miller, K. A. Stubblefield, and K. Englehart, ``Targeted muscle reinnervation for real-time
myoelectric control of multifunction artificial arms,'' Journal of the American Medical Association, vol. 301, no. 6, pp. 619-628, Feb. 2009.
39