1. CONTROL AND COMMUNICATION FOR
PHYSICALLY DISABLED PEOPLE, BASED
ON VESTIGIAL SIGNALS FROM THE BODY
BY
YVONNE MAY NOLAN
B.E.
A thesis presented to
The National Univerisity of Ireland
in fulfilment of the
requirements for the degree of
PHILOSOPHIAE DOCTOR
in the
DEPARTMENT OF ELECTRONIC AND ELECTRICAL ENGINEERING
FACULTY OF ENGINEERING AND ARCHITECTURE
NATIONAL UNIVERSITY OF IRELAND, DUBLIN
SEPTEMBER 2005
Supervisor of Research:
An tOllamh A.M. de Paor
Head of Department:
Professor T. Brazil
2. Abstract
When people become disabled as a result of a road traffic accident, stroke or
another condition, they may often lose their ability to control their environ-
ment and communicate with others by conventional means. This thesis inves-
tigates methods of harnessing vestigial body signals as channels of control and
communication for people with very severe disabilities, using advanced signal
acquisition and processing techniques. Bioelectrical, acoustic and movement
signals are among the signals investigated.
Some applications are presented that have been developed to assist envi-
ronmental control and communication. These applications rely on a variety
of control signals for operation. Some applications may be controlled by a
simple binary switching action whereas others require user selection from a
wider range of possible options. A mechanical switch or adjustable knob may
be used to interact with these applications but this may not be an option for
people who are very severely disabled.
The remainder of the thesis focuses on alternative methods of enabling user
interaction with these and other applications. If a person who is physically
disabled is able to modify some body signal in such a way that two states can
be distinguished reliably and repeatedly, then this can be used to actuate a
switching action. Reliable detection of more than two states is necessary for
multiple-level switching control. As user’s abilities, requirements and personal
preferences vary greatly, a wide range of body signals have been explored.
Bio-signals investigated include the electrooculogram (EOG), the electromyo-
gram (EMG), the mechanomyogram (MMG) and the conductance of the skin.
The EOG is the electrical signal measurable around the eyes and can be used
to detect eye movements with careful signal processing. The EMG and the
3. MMG are the electrical and mechanical signals observable as a result of mus-
cle contraction. The conductance of the skin varies as a person relaxes or
tenses and with practice it can be consciously controlled. These signals were
all explored as methods of communication and control. Also, investigation of
the underlying physical processes that generate these signals led to the devel-
opment of a number of mathematical models. These models are also presented
here.
Small movements may be harnessed using computer vision techniques. This
has the advantage of being non-contact. Often people who have become dis-
abled will still be capable of making flickers of movement e.g. with a finger
or a toe. While these movements may be too weak to operate a mechanical
switch, if they are repeatable they may be used to provide a switching action
in software through detection with a video camera.
Phoneme recognition is explored as an alternative to speech recognition.
Physically disabled persons who have lost the ability to produce speech may
still be capable of making simple sounds such as single-phoneme utterances.
If these sounds are consistently repeatable then they may be used as the ba-
sis of a communication or control device. Phoneme recognition offers another
advantage over speech recognition in that it may provide a method of con-
trolling a continuously varying parameter through varying the length of the
phoneme or the pitch of a vowel sound. Temporal and spectral features that
characterise different phonemes are explored to enable phoneme distinction.
Phoneme recognition devices developed in both hardware and software are
described.
4. ACKNOWLEGDEMENTS
I would firstly like to thank Harry, my supervisor, for all his support, encour-
agement and advice and for sacrificing his August bank holiday Monday to
help me get this thesis in on time!
Thanks also to all the postgrads who have been in the lab in the NRH with
me over the past three years - Deirdre, Claire, Catherine, Kieran, Ciaran and
Jane. Special thanks to Ted for all his assistance, support and friendship.
Thanks also to Emer for generating some of the graphs for this thesis.
Thanks to my parents for their patience and financial help and to my sisters
Tamara and Jill for keeping the house (relatively) quiet to enable me to get
some work done.
Thanks to all my friends for understanding my disappearance over the past
few months and giving me space to get this thesis finished.
Finally, a big thanks to Conor for being so supportive and patient with me
over the past few months, for giving me a quiet place to work and for helping
me with the pictures for this thesis!
i
5. LIST OF PUBLICATIONS ARISING FROM THIS
THESIS
“An Investigation into Non-Verbal Sound-Based Modes of Human-to-Computer
Communication with Rehabilitation Applications”, Edward Burke, Yvonne
Nolan & Annraoi de Paor, Adjunct Proceedings of 10th International Confer-
ence on Human-Computer Interaction, Crete, June 22-27 2003, pp. 241-2.
“The Mechanomyogram as a Tool of Communication and Control for the Dis-
abled”, Yvonne Nolan & Annraoi de Paor, 26th Annual International Confer-
ence of the IEEE Engineering in Medicine and Biology Society, San Francisco,
CA, September 1-5 2004, pp. 4928-2931.
“An Electrooculogram Based System for Communication and Control Using
Target Position Variation”, Edward Burke, Yvonne Nolan & Annraoi de
Paor, IEEE EMBSS UKRI Postgraduate Conference on Biomedical Engineer-
ing and Medical Physics, Reading, UK, July 18-20 2005, pp. 25-6.
“The human eye position control system in a rehabilitation setting”, Yvonne
Nolan, Edward Burke, Claire Boylan & Annraoi de Paor, International Con-
ference on Trends in Biomedical Engineering, University of Zilina, Slovakia,
September 7-9 2005.
Accepted Paper: “Phoneme Recognition Based Software System for Computer
Interaction by Disabled People”, Yvonne Nolan & Annraoi de Paor, IEEE
EUROCON 2005 - International Conference on “Computers as a Tool”, Uni-
versity of Belgrade, Serbia and Montenegro, November 21-24 2005.
ii
20. Chapter 1
Introduction
This thesis arises from work in the Engineering Research Laboratory in the
National Rehabilitation Hospital (NRH)1
. Typically, the patients in this hos-
pital are people who have become disabled as a result of a stroke, disease
or accident. Advances in medical research are ensuring that more and more
people survive from these disabling conditions. It is important that research
follows that not only keeps these people alive, but also enables a fulfilling and
worthwhile quality of life.
Loss of speech production abilities can be one of the most devastating
elements of severe physical disability. Without the means to communicate by
conventional methods, people may find themselves shut off from the outside
world. Communication with other people is one of the most important actions
that we as humans perform. It is important to be able to converse with loved
ones, and to have a means for expressing our emotions, needs and desires.
Communication with others allows us to build relationships, make requests,
reach our intellectual potential and lead a stimulating and participative life.
The independence of people with severe physical disabilities is also an im-
portant consideration. Results from the 2002 census from the Central Statistics
1
Rochestown Ave., Dun Laoghaire, Co. Dublin, Ireland
1
21. Office [1] indicate that there are 159,000 people in this country who provide
regular unpaid help for a friend or family member with a long-term illness,
health problem or disability. Frequent reliance on family and friends can be
frustrating for the disabled person, both for practical reasons and because it
can compromise a person’s feelings of dignity. As technology advances, it is
important to ensure that systems are developed which can provide disabled
people with the ability to control their living environment, without needing
assistance from others.
1.1 Assistive Technologies
For people who are unable to control their environment and communicate
with others by conventional means, there are various systems available which
provide alternative methods of performing these tasks. The term augmentative
and alternative communication is often used to describe a range of alternative
communication techniques, from the use of gestures, sign language and facial
expressions to the use of alphabet or picture symbol boards [2]. In order to be
able to make use of these systems it is necessary to be able to interact with the
system in some way. Perkins and Stenning [3] state that the main objective
for people who are unable to use a keyboard is to be able to identify a function
or movement over which they have some control and utilise that. This could
be from movement of the head, eyes, chin, arms, hands or feet, for example.
These movements can be converted into such electrical signals as “on” or “off”
switches, or, in the case of those with a little more control, variable voltages.
People with very severe physical disabilities may only be capable of making
very small movements to indicate intent that may be difficult to harness. The
focus of this thesis is on investigating advanced methods of signal acquisition
and signal processing to enable these signals to be captured and used to control
communication and control devices. The principal aims of this thesis may be
outlined as follows.
2
22. • Overview of current methods of providing communication and control
for disabled people.
• Identification of alternative signals from the body which may be har-
nessed from the body for communication and control purposes for people
with very severe disabilities.
• Study of measurement techniques that may be used to acquire these
vestigial signals.
• Investigation of signal processing methods to enable these signals to be
correctly interpreted.
• Development of working systems that demonstrate the capabilities of
these techniques.
• Testing of these techniques and systems with people with severe disabil-
ities.
• Development of some mathematical models that evolved as a result of
studying these body signals.
1.2 Thesis Layout
Some of the causes of paralysis and severe disability are outlined in Chapter 2.
An overview of assistive technology applications that may be of relevance to
people with very severe disabilities is given and the importance of identifying a
switching action is emphasised. An alphabet board based communication tool
was developed as part of this work called the Natterbox. This is also described
in Chapter 2.
The nervous system and the structure of muscle are given in Chapter 3, and
the mechanism of muscle contraction is described. Often people who are dis-
abled will retain some ability to contract certain muscles, but not to a sufficient
3
23. extent to enable a mechanical switch to be used. However, the muscle contrac-
tion may still be harnessed for communication and control purposes through
other means. The electromyogram is the electrical signal observable from the
surface of the skin due to action potentials which occur on contraction. The
electromyogram as a control signal for prosthetics and for communication and
control systems is described. An alternative method of measuring muscle con-
traction for communication and control purposes is proposed. This method
uses the mechanomyogram, which is the mechanical signal observable on the
skin surface due to muscle contraction. A mechanomyogram based system for
communication and control was developed and this is presented here. Some
experiments were also performed with this system to assess its efficacy in con-
trolling an alphabet board. The results of these experiments are reported.
Two more biosignals are investigated in Chapter 4, the electrooculogram
and the electrical conductance of the skin. The electrooculogram is the elec-
trical signal observable around the eyes which can be used to measure eye
movement. An overview of different eye movement measurement techniques
is given and the electrooculogram is described in more detail. Some limita-
tions of the electrooculogram signal as a communication and control signal
are identified and a novel technique is presented that seeks to overcome these
limitations to allow the electrooculogram to be used as a control signal. Study
of movement of the eyes led to development of a mathematical model of the
eye, which is also presented in Chapter 4. This model incorporates the effect
of the muscle spindle on the eye’s torque and predicts saccadic and smooth
pursuit eye movements. The electrical conductance of the skin is also briefly
explored as a control signal. Electrical skin conductance is related to sweat
gland activity on the surface of the skin and may be modulated by tensing
or relaxing, as will be discussed. Resulting from this study, a technique for
measuring the firing rate of the sympathetic nervous system was developed
which uses measurement of the skin conductance as its input.
Visual techniques are discussed in Chapter 5, which use a computer camera
4
24. or another light sensitive device to measure movement. Often people who
have become disabled will retain the ability to make flickers of movement of
a certain body part, for example a finger or a thumb. If these movements are
repeatable then they may be used to indicate intent. A novel algorithm for
describing specific paths of motion is presented. This algorithm is incorporated
into a software program, which detects specific movements and uses them to
generate a switching action. This switching action can then be used to control
any communication and control application operable by one switch.
Acoustic methods of harnessing signals from the body are explored in Chap-
ter 6. For people who have speech production abilities, there is a wide range
of speech recognition technologies available that allow environmental control
using the voice. For those who are unable to speak, there may still be ways
of harnessing acoustic signals from the body. Often people who have lost the
ability to produce speech will still remain capable of producing non-verbal ut-
terances. If these utterances are repeatable then they may be used as the basis
of a communication and control system. A number of acoustic based systems
were developed as part of the work described here and these are presented in
this chapter. A system for controlling a reading machine, an environmental
controller and an alphabet board based communication device are given.
The conclusions drawn from the research presented here are given in Chap-
ter 7. Suggestions are made for future work in the area of communication and
control for disabled people.
5
25. Chapter 2
Assistive Technology
2.1 Introduction
Assistive technology is defined by Lazzaro [4] as any device that enables per-
sons with disabilities to work, study, live, or play independently. Cook and
Hussey [5] describe it as any device or technology that increases or improves
the functional capabilities of individuals with disabilities. Assistive technology
may offer assistance to people with a wide range of disabilities including vi-
sion, hearing, motor, speech and learning impairments. Screen magnifiers and
braille are assistive technologies for blind or partially blind persons. Hearing
aids and subtitled films may be classed as assistive technologies for the deaf.
This thesis focuses on assistive technologies for people who, for one reason or
another, require assistance to communicate with others and to control their
environment. A principal aim of this thesis is to explore ways in which signals
from the body may be harnessed so that people with extremely severe physical
disabilities can interact with control and communication devices.
In this chapter, some of the possible causes of paralysis are first described
in Section 2.2. Section 2.3 reviews some of the available assistive technology
devices that may be of benefit to such people. An application called the
6
26. Natterbox is presented in Section 2.4. This communication application was
developed as part of this work to act as a testing board for switching action
methods described in later chapters.
2.2 Causes of Paralysis
There are many different circumstances that will lead to a person requiring
the use of an assistive device to communicate with others or to control their
environment. Paralysis can result from spinal injury following a road traffic
accident or other trauma. It can be caused by damage to the brain due to a
brain haemorrhage or a tumour. Motor neurone diseases, which cause wasting
of the muscle tissue, may eventually lead to paralysis, and necessitate use of a
communication and control device.
Some of the reasons that may lead to a person becoming severely physically
disabled are discussed in this section although this review is by no means
extensive. A major focus of this thesis is on exploring a range of available
options, so that a suitable assistive technology system may be identified for
each individual user, based on their capabilities and requirements, rather than
offering one single solution that will allow all severely disabled people to use a
control and communication device. Similarly, it is impossible to state here the
exact group of people who might benefit from the methods described in this
thesis. Some of the more common causes of paralysis will now be discussed.
2.2.1 Neurological Damage
Neurological damage, or damage to the brain, can occur due to a number of
different circumstances. One of the most commonly occurring reasons is due
to a stroke. The Irish Health Website [6] estimates that 8500 people in this
country suffer from a stroke annually.
7
27. Stroke is not a disease in itself, but a syndrome of neurological damage
caused by cerebrovascular disease [7]. Although paralysis is the most com-
monly associated aspect of a stroke, the stroke syndrome consists of a number
of different aspects which also include spasticity, contractures, sensory dis-
turbances, psychological impairments, emotional and personality changes and
apraxia (the loss of ability to carry out familiar purposeful movements in the
absence of paralysis [8]).
A stroke occurs when normal blood circulation in the brain is interrupted,
either due to occlusion caused by a blood clot (an ischemic stroke) or through
sudden bursting of blood vessels (a haemorrhagic stroke). Strokes due to
blood clots may be divided into two categories. Cerebral thrombosis occurs
due a clot that develops in situ and cerebral embolism is caused by a clot that
forms elsewhere in the body and travels up to the brain [7]. Paralysis can
result from damage to the frontal lobe and/or damage to the internal capsule
fibres. The frontal lobe of the brain contains the motor area, which connects
to the motor cranial nerve nuclei and the anterior horn cells. The internal
capsule of the brain is the narrow pathway for all motor and sensory fibres
ascending from lower levels to the cortex. Damage to one side of the motor
fibres or the frontal lobe leads to loss of power in the muscles on the side of
the body opposite the lesion [9], a paralysis known as hemiplegia [8].
While paralysis is the main symptom of a stroke relevant here, some other
symptoms caused by damage to the cranial nerves are summarised in Table
2.1. The cranial nerves exist in pairs and damage to one of the nerves may
result in the symptoms listed at the side of the lesion. Note that damage to
the tenth nerve is one of the causes of total or partial loss of speech production
abilities. Speech impairments will be discussed in more detail in Chapter 6.
Following a stroke, some voluntary movement may return within a few
weeks of the incident. This is usually due to a number of causes. Following
cerebral infarction and particularly in the case of a cerebral haemorrhage,
abnormally large amounts of fluid in the surrounding tissue can temporarily
8
28. Table 2.1: Signs and symptoms of cranial damage, adapted from [10], pg. 100
Nerve Name Signs and Symptoms of Damage
V trigeminal Pain and burning on outer and inner aspect of cheek
Loss of sensation over face and cheek
VI abducens Diplopia, external rectus weakness, squint
VII facial Weakness of face
VIII auditory Vertigo, vomiting, nystagmus
Deafness and tinnitus
IX glossopharyngeal Loss of taste
X vagus Dysphagia
Paralysis of vocal cord and palate
disrupt neurological function. As the pressure subsides, the neurons in this
area may regain function. Motor function may also be restored due to central
nervous system reorganisation where other areas of the brain take on the role
of voluntary motor control [7]. This partial return of voluntary movement
following a stroke may be of enormous benefit when considering methods for
enabling stroke victims to interact with control and communication systems.
2.2.2 Spinal Cord Injuries
Spinal cord injuries usually occur as the result of a trauma, which is often
caused by a road traffic accident or a domestic, sporting or work-related injury.
The basic anatomical features of the spine and the innervation of the spinal
cord will first be discussed and the classifications of spinal cord injury will then
be described.
Structure of the Vertebral Column and the Spine
The spinal cord is protected by the vertebral column, a line of bony vertebrae
that runs down the middle of the back. The structure of the vertebral column
9
29. is shown in Figure 2.1. When viewed from the side, the vertebral column
displays five curves - an upper and lower cervical curve, and one each thoracic,
lumbar and sacral [11]. The sacral curve is not shown in Figure 2.1 but it
is located at the very bottom of the vertebral column, from the lumbarsacral
junction to the coccyx. The coccyx is better known as the tailbone, which is
made up of several fused vertebrae at the base of the spine [12]. The spinal
cord terminates before the end of the vertebral column, around the top of the
lumbar vertebrae in adults [13]. The lower tip of the spinal cord is called the
conus medullaris [8]. The area from the conus medullaris to the coccyx is
known as the cauda equina [13].
• The Cervical Spine
The purpose of the cervical spine is mobility. The two curves in the
cervical spine can be divided into upper and lower segments at the second
cervical vertebra. The first cervical vertebra (C1) is called the atlas and
the second cervical vertebra (C2) is called the axis. The upper cervical
muscles move the head and neck and are principally concerned with
positioning of the eyes and the line of vision, hence these muscles are
highly innervated to enable these movements to be made with a fine
degree of precision [11]. The axis provides a pivot about which the atlas
and head rotate. The lower cervical spine (C2-C7) also contribute to
movement of the head and neck.
• The Thoracic Spine and Ribs
An important function of the thoracic spine and rib cage is to protect
the heart, lungs and major vessels from compression. Due to this, the
thoracic area is the least mobile region of the spine. The thoracic ver-
tebrae are numbered T1-T12 and the ribs are numbered R1-12 on each
side. The diaphragm muscle fibres are attached to ribs R7-R12.
10
31. • The Lumbar Spine
The lumbar spine is made up of five vertebrae numbered L1-L5. The fifth
lumbar vertebra (L5) is the largest and its ligaments assist in stabilising
the lumbar spine to the pelvis.
There are 31 pairs of spinal nerves attached to the spinal column. Each
pair is named according to the vertebra to which they are related. The spinal
nerves are shown in Figure 2.2.
Classification of Injury
Injury of the spinal cord may produce damage that results in complete or
incomplete impairment of function. A complete lesion is one where motor and
sensory function are absent below the level of injury. A complete lesion may be
caused by a complete severance of the spinal cord, by nerve fibre breakage due
to stretching of the cord or due to a restriction of blood flow (ischaemia) to the
cord. An incomplete lesion will enable certain degrees of motor and/or sensory
function below the injury [14]. There are recognised patterns of incomplete
spinal cord injuries,which are summarised in Table 2.2.
A spinal cord injury may produce damage to upper motor neurons, lower
motor neurons or both. Upper motor neurons originate in the brain and are lo-
cated within the spinal cord. An upper motor neuron injury will be located at
or above T12. Upper motor neuron injury produces spasticity of limbs below
the level of the lesion and spasticity of bowel and bladder functioning. Lower
motor neurons originate within the spinal cord where they receive nerve im-
pulses from the upper motor neurons. These neurons transmit motor impulses
to specific muscle groups and receive sensory information which is transmitted
back to the upper motor neurons. Lower motor neuron injuries may occur at
the level of the upper neuron but more commonly are identified when occurring
at or below T12. Lower motor neuron injuries produce flaccidity of the legs,
decreased muscle tone, loss of reflexes and atonicity of bladder and bowel [14].
12
33. Table 2.2: Patterns of incomplete spinal cord injuries, from text in [14]
Syndrome Damaged Area Common Cause Characteristics
Central Cord Cervical Region Hyperextension Flaccid arm weakness
injury Good leg function
Brown-Séquard Hemisection of Stab Wound Injured Side
Spinal Cord Loss of Motor Function
Uninjured Side
Loss of temperature
& pain sensation
Anterior Cord Corticospinal Ischaemia Variable loss of motor
& spinothalamic & direct trauma function
tracts Reduced sensitivity to
pain and temperature
Conus medullaris/ Sacral cord or the Flaccid bladder and bowel
cauda equina cauda equina nerves Loss of leg motor function
Spinal cord injuries due to complete lesions are usually classified according
to the level of injury to the spine. Table 2.3 summarises the motor classifica-
tion of spinal cord injury. The word paraplegia describes lower lesion spinal
cord injuries resulting in partial or total loss of the use of the legs. The words
tetraplegia and quadriplegia both describe high level spinal cord injuries, usu-
ally occurring due to injury of the cervical spine. Both terms mean “paralysis
of four limbs” and the injury causes the victim to lose total or partial use of
their arms and legs [15].
The main causes of spinal cord injury may be gauged from figures from the
Duke of Cornwall Spinal Treatment Centre, which are given in [16]. For the
new patient admissions with spinal injuries for the period 1993-1995, 36% are
due to road traffic accidents, 6.5% are due to self harm and criminal assault,
37% are due to domestic and industrial accidents and 20.5% are due to injuries
at sport. Until recently spinal cord injury was recognised as a fatal condition.
14
34. Table 2.3: Motor classification of spinal cord injury, adapted from pg. 63 in [14]
Level Muscles Level Muscles
C4 Deltoids L2 Hip Flexors
C5 Elbow Flexors L3 Knee Extensors
C6 Wrist Extensors L4 Ankle dorsiflexors
C7 Elbow Extensors L5 Long toe extensors
C8 Finger Flexors S1 Ankle Plantar Flexors
T1 Finger Abductors S4-S5 Anal contraction
In the First World War, 90% of patients who suffered a spinal cord injury died
within one year of wounding and only about 1% survived more than 20 years
[16]. The chances of survival from a spinal cord injury began to increase in the
1940s with the introduction of sulfanilamides and antibiotics [14]. Nowadays,
due to better understanding and management of spinal cord injury, the outlook
has greatly improved for people with spinal cord injuries.
There has been a gradual change in the pattern of survival from low-lesion
paraplegia in the 1950s, high-lesion paraplegia in the 1960s and low-lesion
quadriplegia in the 1970s. Finally, in the 1980s, people with spinal cord in-
juries at or above C4, resulting in high-lesion quadriplegia, have been surviving
in significant numbers. It is estimated that each year in the USA, 166 sustain
injury at C1-C3 and 540 people at C4 [14]. As medicine advances, such indi-
viduals will survive in increasing numbers and thus it is important to identify
methods for interaction with communication and control systems for this group
of severely disabled individuals.
The functional ability of tetraplegic patients based on the level of injury
are summarised in Table 2.4. In general, movements of the limbs suffer more
severely than those of the head, neck and truck. Movements of the lower face
also tend to be more severely impaired than those of the upper face [10].
15
35. Table 2.4: Expected functional ability based on level of injury, constructed using
information from [16].
Level of Injury Functional Ability
Complete lesion below C3 Dependent on others for all care
Chin and head movement
Can use breath controlled devices
Complete lesion below C4 Dependent on others for all care
Chin and head movement
Shoulder shrugging possible
Can type/use computer using a mouth stick
Complete lesion below C5 Shoulder movement
Elbow flexion
Complete lesion below C6 Wrist Extension
Complete lesion below C7 Full wrist movement
Some hand function
Complete lesion below C8 All hand muscles except intrinsics preserved
Complete lesion below T1 Complete innervation of arms
16
36. 2.2.3 Diseases of the Nervous System
The words motor neurone disease (MND) and amyotrophic lateral sclerosis
(ALS) are often used interchangeably. However, amyotrophic lateral sclerosis
may be described more accurately as a type of motor neurone disease, and
probably the most well known. Motor neurone diseases affect the motor nerves
in the brain and the spinal cord [17] and the term motor neurone disease may
be used to describe all the diseases of the anterior horn cells and motor system,
including ALS [18].
Motor neurone diseases may be divided into two categories - idiopathic mo-
tor neurone diseases and toxin-related motor neurone diseases. An idiopathic
disease is one of spontaneous origin [8]. The idiopathic motor neurone diseases
include both the familial and juvenile forms of amyotrophic lateral sclerosis.
Also included under this category are progressive bulbar palsy (PBP), pro-
gressive muscular atrophy (PMA), primary lateral sclerosis (PLS), Madras
motor neurone disease and monomelic motor neurone disease [18]. The toxin-
related motor neurone diseases are suspected to be linked to environmental
factors [18]. These include Guamanian ALS (due to a high incidence of ALS
in Guam), lathyrism and Konzo.
The exact figure for the number of people diagnosed with ALS varies, but
it is thought to affect between 1-3 in every 100,000 of the population each year
[17, 18]. There are an estimated 300 people living with amyotrophic lateral
sclerosis at any one time in Ireland [17]. ALS is a progressive fatal disease of the
nervous system and the rate of progression depends on the individual [18]. The
muscles first affected by motor neurone diseases tend to be those in the hands,
feet or mouth and throat. As ALS progresses, the ability to walk, use the
upper limbs and feed orally are progressively reduced. In the terminal stage
of the disease, none of these functions can be independently performed and
respiratory functions become compromised [18]. At this stage of the disease, it
is as important as ever to give the person the best quality of life possible and
17
37. assistive technologies must be considered that can harness the vestigial signals
left to these people. Usually, the motor function of the eye muscles is spared
due to the calcium binding proteins in these nerve cells [18] and this feature
could be used to provide a method of control and communication, as will be
discussed in Chapter 4. Brain computer interface (BCI) technologies are also
often considered at the very latest stages of the disease, these will be briefly
described in Section 2.3.3.
Paralysis can also occur due to demyelinating diseases such as multiple
sclerosis. A demyelinating disease causes impairment of conduction of signals
in nerves as it damages the myelin sheath of neurons. More about the structure
of nerves will be described in Chapter 3. Neurological damage resulting in
paralysis may also occur due to viral infections such as poliomyelitis or polio
[10] or due to bacterial infections such as bacterial meningitis, which affects
the fluid in the spinal cord and the fluid surrounding the brain [19].
2.3 Assistive Technology
Assistive technologies can be of immense benefit to people with severe physical
disabilities such as those described above. As mentioned already, this thesis
focuses mainly on facilitating interaction with two type of assistive technology
applications - control and communication.
Communication applications are usually described in assistive technology
terms as augmentative and alternative communication (AAC) systems [2].
Augmentative and alternative communication systems refer to assistive tech-
nology systems designed for people who have limited or no speech production
abilities. Alternative communication systems usually consist of some sort of
alphabet board or symbolic board [4]. Some alternative communication sys-
tems display text to a computer screen, others output the text to a printer and
some work in conjunction with speech synthesis systems to “speak out” the in-
18
38. tended message. Some are computer operated and some are handheld, such
as the handheld LightWriter1
, a dual display keyboard based communication
aid. Some, such as Voicemate2
, allow the user to record phrases for digitised
playback [4].
Control applications refer to any system that can be operated automatically
using a control signal. For example, a control signal could be used to handle an
environmental control system to operate appliances in the user’s environment,
such as lights, fans or the television. The reading machine described in Chapter
6 is another example of a system that may be operated using a control signal.
Control signals can also be used to operate wheelchairs or electrically powered
prosthetics. The electromyogram muscle signal is often harnessed to replace
muscle function to control prosthetics for amputees, as described in Chapter
3.
2.3.1 Importance of a Switching Action
The simplest control signal is probably the switching action, which is any
action that allows the user to alternate between two possible states, “on” or
“off”. There are numerous systems in use today that may be operated by
pressing a single switch or multiple switches. Such systems are often called
switch-activated input systems [2]. A standard computer keyboard may be
described as a switch based system for interfacing with a computer. The
keyboard usually has around 100 keys or switches and each key press sends
a control signal to the processor which is recognised as a different letter or
symbol by the computer. The combination of two or more key presses may
also be used to increase the number of possible control signals [5].
There are many types of commercially available switches and a comprehen-
sive guide to switches is given in [20]. The standard type of switch is the paddle
1
Lightwriter, Zygo Industries, Inc., P.O. Box 1008, Portland, OR 97202 USA
2
Tash Inc., Unit 1, 91 Station Street, Ajax, Ont. L1S 3H2, Canada.
19
39. type switch. These mechanical switches have movement in one direction and
can be activated by the user by pressing on the switch with any part of the
body. For persons who do not have sufficient strength or ability to operate
these switches there are a number of other types of switches available. These
switches include suck-puff switches, wobble switches, leaf switches and lever
switches [5, 21]. The switch chosen for a particular individual will depend on
the capabilities of the user.
For people who are very severely physically disabled, performing a switch-
ing action using any of these physical switches may not be an option. In
these cases, other methods of harnessing signals from the body to provide a
switching signal must be explored. One of the main objectives in developing
alternative systems for communication and control is to be able to correctly
identify two or more distinct states that a user can voluntarily elicit. If these
states can be reliably distinguished, then transition from one state to another
can be harnessed as a means of effecting a switching action.
2.3.2 Switch Based Systems
Switches are generally used in one of two ways - in a scanning system or in a
coding system. In a coding system, the user taps out a message using some
scheme such as the famous Morse code, using the switch. The Morse code
software functions like a translator, converting Morse code to text in real time
[4]. The coding can either be done using one switch with long switch presses for
the dash and short switch presses for the dots, or using two separate switches to
represent dots and dashes [2]. Morse code based systems have the disadvantage
that the code must first be learnt by the user.
A more popular type of switch-activated input system uses scanning based
selection. These systems are usually based on some variation of the row-
scanning method described by Simpson and Koester [22]. The user is presented
with a screen of options, arranged in rows and columns. The program scans
20
40. through the rows and the user can select a particular row by pressing a switch.
The program then scans through each item on the selected row and the user
can select the desired item by pressing a switch again. Row scanning is often
used in software alphabet boards and can be used to spell out messages [2].
The idea of switch based menu selection has been around for years. The
personal computer became popular in the early 1980s and software based as-
sistive technology systems soon followed. An independent living system known
as the ADAPTER program was developed around 20 years ago by a team in
Lousiana Tech University in the USA [23]. This program uses the row-scanning
method to allow the user to select one of several tasks from a menu. The five
options given are letters, words, codes, phone and environment. The program
is designed to be operated with a mechanical switch and the two examples
mentioned are a push-button switch and a bulb-pressure switch. If the user
selects the letter option on the main menu then they will be presented with a
second sub-menu with rows of letters and numbers which allows messages to be
spelled out. The word option provides quick access to a list of important words
e.g. light, water, bath etc. Selection of the code option allows communication
through Morse code by pressing the switch for long or short periods which is
then converted to text. The phone option displays a pre-programmed list of
names and phone numbers which may be dialled through the computer and
the environment option allows control of appliances in the user’s surroundings.
Another scanning based alphabet board system developed around this time
is described in [21], in which the scanning device is a hardware logic-based mod-
ule that uses LEDs to highlight each character. This device can be connected
to the computer as a substitute for a manually operated keyboard. The sys-
tem uses two switches to scan through the characters and enter the required
character into the computer.
Damper [24] estimates that a communication rate of 6-8 words per minute
is typically achieved using an alphabet board based communication system.
There have been a number of different methods suggested for increasing the
21
41. rate at which the user can select the letters. Perkins and Stenning [3] exper-
imented with the idea of using two or five switches to operate an alphabet
board and also tested the communication rate with different menu layouts.
The two layouts tested had 57 characters - one had letters and each number
once and the second had additional characters related to frequency of use (e.g.
the letter ’E’ appears on the board five times) but no numbers. Simpson and
Koester [22] have proposed a method of increasing text entry rate using an
adaptive row-column scanning algorithm which increases or decreases the scan
delays according to user performance.
Although it is not yet implemented as a switch based text entry system,
the Dasher program by Ward [25] will briefly be described. Rates of 39 words
per minute have been claimed for it when operated using a mouse and 25
words per minute when operated using eye tracking. It is a software based
program which enables a person to spell out words by steering through a
continuously expanding two-dimensional scene containing alphabetical listings
of the letters [26]. A screenshot from this program is shown in Figure 2.3. The
line in the centre of the screen is the cursor. The user is initially presented
with an alphabetical list of letters and the user selects a letter by moving the
cursor inside the area of the letter. As the user approaches a letter the letter
grows in size. Once the letter is selected the user will again find themselves
presented with another list of letters but the relative sizes of all the letters
on the new list is based on the probability of this letter being the desired
letter based on the previous letter. Dasher uses a language model to predict
this, and the model is trainable on example documents in almost any language
[26]. In the example shown in Figure 2.3, the user is spelling out the word
“demonstration” and has already selected “demonstrat”. As the user moves the
cursor closer towards the letter “i”, the letter grows in size until the user is inside
the box. The screenshot also illustrates alternative words that could instead
have been selected such as “demolished”, “demonstrated that”, “demoralise”
and “demonstrative”. A number of different methods for interfacing with the
22
42. Figure 2.3: Dasher program - spelling out the word “demonstration”.
Dasher program are suggested on the Dasher website [27], including a mouse,
a joystick, eye-tracking and head-tracking. Future possible developments of
Dasher are described in [26], and include a suggestion for a modified method
for operation using a single switch. This will allow the user to operate Dasher
using a switch that changes the direction of cursor movement on activation.
2.3.3 Brain Computer Interfaces
Brain computer interfaces (BCI) may offer another method of providing switch-
ing actions in cases of very severe disability. Brain computer interfaces are
usually used in situations of very severe disability where there is no other
method of communication and control possible. These methods allow the user
23
43. to interact with the computer using some measurement of brain activity, such
as function magnetic resonance imaging (fMRI) or the electroencephalogram,
the electrical signal measurable from the surface of the scalp. Correct interpre-
tation of these signals can be used to convey user intention and thus actuate
a switching action. The area of brain computer interfaces for the disabled is
a huge research area and the interested reader is referred to the IEEE review
of the first international BCI technology meeting [28] as a starting point for
more information.
2.4 Communication Device
A software communication device called Natterbox was developed as part of
this study, based on an alphabet board. The code for this program is included
in Appendix I. Although there are many similar communication programs
available commercially, this program was developed for two reasons. Firstly, it
was in response to a request made by one of the occupational therapists in the
hospital, who had been using a previous version of the same program, which
had been developed earlier in our laboratory in the NRH. She was attempting
to use the system with a male patient who had suffered from a brainstem
stroke. The patient had poor visual ability and was also very photosensitive.
This rendered him unable to see the letters of the alphabet board on screen.
She suggested making each of the rows of the alphabet board a different colour,
in accordance with the layout of physical alphabet boards used by occupational
therapists. An auditory facility was then added which speaks out the colours
on each of the different rows as they are highlighted. The patient was able
to learn which letters corresponded to which coloured row and hence could
perform a switching action when the program called out the name of the row
that was desired. The program then calls out each letter in that row in turn,
and the user can again select the desired letter when it is reached, thus enabling
the user to spell out messages.
24
44. The second benefit gained from development of the Natterbox program is
that it served as a useful testing board for different switching mechansims
developed in the work presented here. Since the Natterbox allows the user to
spell out words and sentences simply by performing a single switching action,
it was an invaluable tool in demonstrating translation of different body signals
into communication. The Natterbox program as described here was used by
a number of different patients in the hospital. For each of these patients, a
reliable method of interfacing with the program had to be identified and some
of the techniques used are discussed in this thesis. As the program developed,
various features were added in response to therapist and patient requests. Some
of these will now be briefly outlined.
2.4.1 Technical Details
The Natterbox program was developed with C++ using the Fast Light Tool
Kit3
(FLTK) to develop the graphical user interface. The sound feature was
added using tools from the Simple Directmedia Layer4
(SDL), which is a C++
multimedia library designed to provide access to audio devices. The primary
advantage of using FLTK and SDL is that they are both cross-platform, making
the Natterbox program portable across different operating systems.
2.4.2 The Natterbox Graphical User Inteface
The graphical user interface (GUI) of the Natterbox main menu is shown in
Figure 2.4, demonstrating a message being spelled out. In Figure 2.4(a), the
yellow row is highlighted. The user activates a switch to select this row and the
program begins scanning the letters on that row. In Figure 2.4(b), the symbol
“.” is highlighted. The user again activates a switch to select this symbol.
Figure 2.4(c) shows that the symbol has appeared on the message banner and
3
FLTK Website: http://fltk.org
4
SDL Website: http://www.libsdl.org
25
45. also on the history panel along the right-hand side of the screen.
2.4.3 Switch Interface Box
The switch input required by Natterbox was chosen to be an “F2” keypress.
Thus Natterbox can be used in one of three ways. Firstly it is operable by
simply pressing the physical key on the keyboard. Obviously this is not a very
useful interaction method for people with very severe disabilities. Secondly,
it may be used in conjunction with another program that is monitoring some
signal from the body and will simulate an “F2” keypress when it recognises
intention. Possible methods for harnessing body signals for these purposes
forms much of the remainder of the this thesis.
Thirdly, it may be used with a switch interface box. Any arbitrary two way
switch, such as those mentioned in Section 2.3.1, can be connected to this box.
The switch interface box is connected into the USB port of the computer and
a supplementary software application simulates an “F2” key press on detection
of a switching action. The supplementary program was called USB Switch and
the code is given in Appendix I.
2.4.4 Other Features
Phrases Menu
Due to requests from the occupational therapists in the hospital, the option
of a sub-menu was added to the Natterbox program. This sub-menu provides
quick access to a list of commonly used phrases. This menu may be opened
by selecting the last row in the main menu. The sub-menu screen is shown in
Figure 2.5(a). When the user selects the phrase “Turn on or off fan” it appears
in the message banner back in the main screen. This phrase could be used by
the user to request that the fan is turned off if it is already on, or turned off if
it is on.
26
47. (c)
Figure 2.4: The Natterbox program (a) The program is highlighting the second
(yellow) row. (b) When the user selects the second row the user begins scanning the
letters on this row. The “.” button is currently highlighted. (c) The user selects this
symbol and it appears above on the banner.
28
48. Printing Feature
An option to print the message to paper was added in response to a request
from a patient who wanted a facility for writing letters to her children. This
request was fulfilled by placing an option “Print” at the bottom of the phrases
menu. Selection of this option sends all the text in the history box to an
attached printer. This option could be of immense benefit to users since it
allows the user to prepare lengthy messages in advance.
Cancel Feature
A “cancel” option was added for people who are capable of actuating a sec-
ond switching action. The second switch input cancels the effect of the last
input. Thus if the user has accidently selected a letter they may delete this
letter from the message bar by activating the second switch. If the user has
accidently selected the wrong row and the program is scanning through each
of the items on that row, the user may use the second switch to change back
to row scanning.
Three-Switch Mouse
A three-switch mouse was developed for one of the patients who was in the
hospital who was particularly successful with the Natterbox program. The
patient used a push-button switch placed between his thumb and hand to
operate the program. He also had head movement on both sides so was able to
operate two head switches. The Natterbox program was modified to include a
mouse cursor control system using these three switching actions. The patient
could exit the alphabet board program by selecting an “Exit” option at the
end of the phrases menu. This switches the program into mouse cursor control
mode. The mouse cursor is controlled by the USB Switch program.
The head switches may be used to move the mouse cursor either up and
29
49. (a)
(b)
Figure 2.5: The Natterbox Phrases Menu (a) The program is highlighting the second
phrase “Turn on or off the fan”. (b) When the user selects this phrase it appears on
the banner back in the main menu.
30
50. down, or left and right. Switching between these two directions is performed
using the hand switch. Pressing the hand switch twice in succession actuates
a mouse click.
2.4.5 Possible Future Developments of Natterbox
The addition of a submenu to Natterbox containing numbers and punctuation
marks could be of great benefit. In addition to adding to user dignity by mak-
ing the messages look more presentable, they could also enable emoticons to
be used to add more meaning to messages. Emoticons are being more and
more popular nowadays due to emailing, instant messaging and text messag-
ing. Emoticons (emotion icons) are a method of adding symbols to the end
of messages to represent different facial expressions. These can be used to
communicate more effectively what is meant by the message. For instance,
the simple term “It’s ok” could be interpreted in a number of different ways.
It can be intended straightforwardly and this can be emphasised by placing a
smiley face symbol at the end of the message i.e. “It’s ok :-)”. Conversely, if
the person wishes to impart some sort of satirical tone to the message, they
may express this by adding the sad smiley “It’s ok :-(” or the angry smiley sym-
bol “It’s ok :-@”, depending on intent. These emoticon symbols are becoming
more and more integrated into casual everyday written communications and
could offer an immense benefit to people who are severely disabled and wish to
more effectively convey their emotions when writing messages. The addition
of a speech synthesiser to the complete program to allow the messages to be
spoken out loud is also being considered.
2.5 Conclusions
This chapter has outlined some of the diseases, conditions and circumstances
that may render a person severely physically disabled. A review of assistive
31
51. technology applications has been given and the importance of generating a
switching action has been emphasised. Now that these areas of been discussed,
the aims of this thesis may be more accurately defined. This thesis aims to
investigate alternative methods of harnessing vestigial signals from people who
have been severely paralysed and have very little motor function, such as those
with high-level lesions above C4. These people may be unable to operate a
mechanical switch and thus require a more complex technique to be identified
that will allow a switching action to be actuated. A large part of the remainder
of this thesis focuses on methods of harnessing these vestigial signals to provide
switching actions and other control signals.
32
52. Chapter 3
Muscle Signals
3.1 Introduction
This chapter and Chapter 4 investigate methods of harnessing bio-signals from
the body for control and communication purposes. The exact criteria required
to enable a particular body signal to be described as a bio-signal are not always
well defined. In the broadest sense of the word, a bio-signal may refer to any
signal from the body related to biological function. Under this definition, all of
the signals presented in this thesis would fall under the category of bio-signals,
including the signals obtained through video capture techniques, described
in Chapter 5, and speech signals obtained through audio signal processing
techniques, described in Chapter 6. A more narrow definition of the term bio-
signals is meant here. A bio-signal as discussed in this thesis refers to any signal
that is measurable directly from the surface of the skin. This includes signals
such as biopotentials, which are measured voltages from certain sites on the
body, but also other electrical signals, such as the electrical skin conductance,
and mechanical signals, such as the mechanomyogram.
This chapter discusses two bio-signals which may be used to detect mus-
cle contraction. These are the electrical signal, the electromyogram (EMG),
33
53. and the mechanical signal, the mechanomyogram (MMG). Muscle signal based
switching systems may be an option for people who retain some ability to con-
tract certain muscles but may not be able to operate a mechanical switch.
This may be because the particular muscle that can be contracted is not suit-
able for operating a switch or because the muscle contraction is not strong
enough to operate the switch. This chapter investigates how deliberate muscle
contraction can be used to effect a switching action to operate control and
communication systems.
The anatomy and physiology of the nerves and the nervous system are first
described in Section 3.2.1. Action potentials and the method of information
transfer in the body are described in Section 3.2.2. The anatomy of muscle and
the process of muscle contraction are discussed in Section 3.3. Some different
muscles that may be suitable for use for an EMG-based or MMG-based sys-
tem are identified in Section 3.3.3. The electromyogram as a control signal is
discussed in Section 3.4. Finally the possibility of using the mechanomyogram
as a control signal is explored in Section 3.5.
3.2 The Nervous System
3.2.1 Nerves and the Nervous System
The Nerve Cell
The basic building block of the human body’s nervous system is the nerve cell,
or neuron. The neurons in the body are interconnected to form a network
which is responsible for transmitting information around the body. The spinal
cord, the brain and the sensory organs (such as the eyes and ears) all consist
largely of neurons.
The structure of a neuron is shown in Figure 3.1. The central part of
34
54. Figure 3.1: The Nerve Cell, from pg. 2 in [29]
the neuron is the cell body, or soma, which contains the nucleus. The cell
body has a number of branches leading from its centre, which can either be
dendrites or axons. The dendrites receive information and the axons transmit
information, both in the form of impulses, which will be described in more
detail later. There is generally only one axon per cell. The axon links the
nerve cell with other cells, which can be nerve cells, muscle cells or glandular
cells. In a peripheral nerve, the axon and its supporting tissue make up the
nerve fibre. A bundle of nerve fibres is known as a nerve.
Classification of Nerve Fibres
The peripheral nervous system refers to the neurons that reside outside the
central nervous system (CNS) and consists of the somatic nervous system and
the autonomic nervous system [30]. A nerve fibre may be classified as either an
afferent nerve fibre or an efferent nerve fibre. An afferent nerve fibre transmits
information to the neurons of the CNS and the efferent nerve fibre transmits
35
55. information from the CNS.
Afferent nerve fibres may further be divided into somatic nerve fibres and
visceral nerve fibres. Visceral afferents are nerve fibres from the viscera, which
are the major internal organs of the body. All other afferent nerve fibres in the
body are called somatic afferents. These come from the skeletal muscle, the
joints and the sensory organs such as the eyes and ears, and bring information
to the CNS.
Efferent nerve fibres can be categorised as either motor nerve fibres or
autonomic nerve fibres. Motor efferents control skeletal muscle and autonomic
efferents control the glands, smooth muscle and cardiac muscle. See Figure 3.2
for a summary of nerve fibre classifications.
The visceral afferent nerve fibres and the autonomic efferent nerve fibres
both belong to the autonomic nervous system. The autonomic nervous system
is responsible for controlling such functions as digestion, respiration, perspi-
ration and metabolism which are not normally under voluntary control. The
function of perspiration, controlled by the autonomic nervous system will be
described in more detail in Chapter 4.
Glands
Joints
Skeletal Muscle
Sensory Organs
Skeletal Muscle
Cardiac Muscle
Smooth Muscle
Visceral
Somatic Motor
Autonomic
EfferentsAfferents
Viscera
Central
Nervous
System
Figure 3.2: Classification of Nerve Fibre Types
Supporting Tissue
Neurons are supported by a special type of tissue constructed of glial cells.
These cells perform a similar role to connective tissue in other organs of the
body. In a peripheral nerve, every axon lies within a sheath of cells known as
36
56. Figure 3.3: (A) Myelinated Nerve Fibre (B) Unmyelinated Nerve Fibres, from pg.
8 in [29].
Schwann cells, which are a type of glial cell. The Schwann cell and the axon
together make up the nerve fibre. A nerve fibre may be either a myelinated
nerve fibre or an unmyelinated nerve fibre depending on how the Schwann
cells are positioned around the axon. Myelinated nerve fibres have a higher
conduction velocity than unmyelinated nerve fibres. About two-thirds of the
nerve fibres in the body are unmyelinated fibres, including most of the fibres in
the autonomic nervous system, since these processes generally do not require
a fast reaction time.
In myelinated nerve fibres, the Schwann cell winds around the axon several
times as shown in Figure 3.3. A lipid-protein mixture known as myelin is
laid down in layers between the Schwann cell body, forming a myelin sheath.
This sheath insulates the nerve membrane from the conductive body fluids
surrounding the exterior of the nerve fibre. The myelin sheath is discontinous
along the length of the axon. At regular intervals there are unmyelinated
sections which are called the Nodes of Ranvier. These nodes are essential in
enabling fast conduction in myelinated fibres [29].
As mentioned in Chapter 2, diseases such as multiple sclerosis damage the
myelin sheath of neurons, or dymyelinate the fibres along the cerebrospinal
axis [10]. Paralysis occurs due to impairment of the conduction of signals in
demyelinated nerves.
37
57. 3.2.2 Resting and Action Potentials
The Membrane Potential
A potential difference usually exists between the inside and outside of any cell
membrane, including the neuron. The membrane potential of a cell usually
refers to the potential of the inside of the cell relative to the outside of the cell
i.e. the extracellular fluid surrounding the cell is taken to be at zero potential.
When no external triggers are acting on a cell, the cell is described as being in
its resting state. A human nerve or skeletal muscle cell has a resting potential
of between -55mV and -100mV [29]. This potential difference arises from a
difference in concentration of the ions K+
and Na+
inside and outside the cell.
The selectively permeable cell membrane allows K+
ions to pass through but
blocks Na+
ions. A mechanism known as the ATPase pump pumps only two
K+
ions into the cell for every three Na+
cells pumped out of the cell resulting
in the outside of the cell being more positive than the inside. The origin of
the resting potential is explained in further detail in [29].
The Action Potential
As mentioned already, the function of the nerve cell is to transmit information
throughout the body. A neuron is an excitable cell which may be activated by
a stimulus. The neuron’s dendrites are its stimulus receptors. If the stimulus
is sufficient to cause the cell membrane to be depolarised beyond the gate
threshold potential, then an electrical discharge of the cell will be triggered.
This produces an electrical pulse called the action potential or nerve impulse.
The action potential is a sequence of depolarisation and repolarisation of the
cell membrane generated by a Na+
current into the cell followed by a K+
current out of the cell. The stages of an action potential are shown in Figure
3.4.
38
58. 5
mV
−70
−55
0
30
4
1
2
3
6
Threshold
Resting Potential
Figure 3.4: An Action Potential. This graph shows the change in membrane po-
tential as a function of time when an action potential is elicited by a stimulus. The
time duration varies between fibre types.
• Stage 1 - Activation
When the dendrites receive an “activation stimulus” the Na+
channels
begin to open and the Na+
concentration inside the cell increases, making
the inside of the cell more positive. Once the membrane potential is
raised past a threshold (typically around -50mV), an action potential
occurs.
• Stage 2 - Depolarisation
As more Na+
channels open, more Na+
ions enter the cell and the inside
of the cell membrane rapidly loses its negative charge. This stage is also
known as the rising phase of the action potential. It typically lasts 0.2 -
0.5ms.
• Stage 3 - Overshoot
The inside of the cell eventually becomes positve relative to the outside
of the cell. The positive portion of the action potential is known as the
overshoot.
39
59. • Stage 4 - Repolarisation
The Na+
channels close and the K+
channels open. The cell membrane
begins to repolarise towards the resting potential.
• Stage 5 - Hyperpolarisation
The membrane potential may temporarily become even more negative
than the resting potential. This is to prevent the neuron from responding
to another stimulus during this time, or at least to raise the threshold
for any new stimulus.
• Stage 6
The membrane returns to its resting potential.
Propagation of the Action Potential
An action potential in a cell membrane is triggered by an initial stimulus to
the neuron. That action potential provides the stimulus for a neighbouring
segment of cell membrane and so on until the neuron’s axon is reached. The
action potential then propagates down the axon, or nerve fibre, by successive
stimulation of sections of the axon membrane. Because an action potential is
an all-or-nothing reaction, once the gate threshold is reached, the amplitude
of the action potential will be constant along the path of propagation.
The speed, or conduction velocity, at which the action potential travels
down the nerve fibre depends on a number of factors, including the initial
resting potential of the cell, the nerve fibre diameter and also whether or not
the nerve fibre is myelinated. Myelinated nerve fibres have a faster conduction
velocity as the action potential jumps between the nodes of Ranvier. This
method of conduction is known as saltatory conduction and is described in
more detail in [29].
40
60. Synaptic Transmission
The action potential propagates along the axon until it reaches the axonal
ending. From there, the action potential is transmitted to another cell, which
may be another nerve cell, a glandular cell or a muscle cell. The junction of
the axonal ending with another cell is called a synapse. The action potential is
usually transmitted to the next cell through a chemical process at the synapse.
If the axon ends on a skeletal muscle cell then this is a specialised kind of
synapse known as a neuromuscular end plate. In this case, the action potential
will trigger the muscle to contract. The physical processes that must occur to
enable muscle contraction will be examined in more detail later, but first the
structure of the muscle is described.
3.3 Muscles
3.3.1 Muscle Physiology
There are three types of muscle present in the human body - smooth, skeletal
and cardiac. Smooth muscle is the muscle found in all hollow organs of the
body except the heart, and is generally not under voluntary control. Cardiac
muscle, the only type of muscle which does not experience fatigue, is the muscle
found in the walls of the heart which continuously pumps blood through the
heart. Skeletal muscle is the muscle attached to the skeleton which is the type
of muscle that will be described here. The main function of skeletal muscle
is to generate forces which move the skeletal bones in the body. The basic
structure of a skeletal muscle is shown in Figure 3.5.
Muscle is a long bundle of flesh which is attached to the bones at both ends
by tendons. The muscle is protected by an outer layer of tough tissue called
the epimysium. Inside the epimysium are fasicles or bundles of muscle fibre
cells. The fasicles are surrounded by another layer of connective tissue called
41
63. (a) I−Band A−Band
Actin Z−Disc Myosin
(b)
Figure 3.7: (a) The sarcomere before contraction occurs. The A-band, containing
thick myosin filaments, and the I-band, containing the Z-disc and the thin actin
filaments are shown. (b) On contraction of the muscle, the thin actin filaments slide
between the myosin filaments.
3.3.2 Muscle Contraction
The Motor Unit
Each efferent motor nerve fibre, or α motor neuron as they are also known,
stimulates a number of muscle fibres. The nerve fibre, and the muscle fibres it
innervates, make up the smallest functional unit of muscle contraction known
as the motor unit. Each individual muscle fibre in a motor unit will be stimu-
lated simultaneously by the nerve fibre, so they will each always contract and
relax in synchronisation. The force produced by a muscle can be increased by
increasing either of two parameters:-
(i) The number of active motor units. The motor units are roughly arranged
in parallel along the length of the muscle so by activating more motor
units, more muscle force can be produced. The forces produced by indi-
vidual muscle units sum algebraically to give the total muscle force.
44
64. (ii) The rate at which the nerve fibres activate the muscle fibres, or fire. This
rate is known as the firing frequency. When a single motor unit receives
a single stimulation, the response is a single twitch. The duration of
a single twitch varies depending on whether the muscle fibres are slow-
twitch (Type 1) muscle fibres or fast-twitch (Type 2) muscle fibres. A
motor unit will usually be made up entirely of either fast-twitch muscle
fibres or slow-twitch muscle fibres. The slow motor units have a slower
speed of contraction but will take longer to fatigue. When a muscle
contracts, the slow motor units are recruited first, this principle is known
as the size principle of motor unit recruitment [31]. The duration of a
single twitch in a slow-twitch muscle fibre is about 200ms. The action
potential causing the single twitch is only about 0.5ms in duration so the
twitch goes on for a long time once it has been initiated.
If the length of a single twitch is 200ms and the firing frequency is less
than 5Hz, then the force response will show a series of individual twitches.
As the firing frequency of the motor unit increases, the second stimulus
will begin to stimulate the muscle before the effects of the first stimulus
have subsided. In this cases the forces begin to accumulate. As the fir-
ing frequency increases, the force response becomes larger in magnitude.
For relatively low frequencies (less than 20Hz for slow motor units and
less than 50Hz for fast motor units) there will be some force relaxation
between stimulation pulses. If the muscle force is oscillating, then this
is known as unfused tetanic contraction. At higher firing frequencies the
force will remain constant, this is known as fused tetanic contraction.
Types of Contraction
When a muscle is stimulated by a nerve impulse, it tends to shorten, provided
it can overcome the external resistance imposed on it. Shortening and force
production of muscle is referred to as contraction [31]. A shortening contraction
is called a concentric contraction. In certain instances the muscle is fixed so
45
65. it cannot shorten and the increase in muscle contraction is then measurable
as an increase in the force acting on the muscle. This type of contraction is
known as an isometric contraction. Each muscle has a maximum isometric
force capability which is the maximum amount of force that can be applied to
a muscle which is fixed at a certain length without forcible stretching. If the
muscle is subjected to an external force greater than its maximum isometric
force capability then the muscle is forcibly stretched. This is known as eccentric
contraction. These contractions can be measured in vivo - i.e. while the muscle
is still ‘living’ in the human body. Other muscle contractions are measurable by
severing a muscle at its tendons and placing it in a bath for experiments. These
types of measurements are known as in vitro measurements (literally meaning
in glass). In vitro experiments can be used to measure isotonic or isokinetic
contractions. Isotonic contraction occurs when the muscle is subjected to a
constant load and isokinetic contraction refers to contractions performed at a
constant speed. An in vivo contraction is rarely fully isometric or isotonic.
Molecular Mechanism of Contraction
During an isotonic contraction, it is observed that the width of the A-bands
stays constant but the width of the I-bands becomes narrower. However, the
length of the actin filaments in the I-band are found to stay the same length
during the contraction. The I-band is thus shortened by the actin filaments
sliding in between the myosin filaments. The cross-bridge theory, which was
first postulated by Huxley in 1957 [32], is widely used to describe how the
actin filaments slide between the myosin filaments. When a muscle begins to
contract a cross-bridge is formed between the myosin and actin filaments. The
head of the cross-bridge rotates, which pulls the actin filament between the
myosin filaments. The bridge is then broken and reformed with the next part
of the actin filament and the cycle continues.
As described earlier, a muscle cell is stimulated to contract when it re-
46
66. ceives an action potential. It is thought that the depolarisation of the cell
that occurs during an action potential might cause an increase in the calcium
ion concentration inside the cell. The exterior of the myofibrils consists of a
network of tiny sacks or vesicles, known as the sarcoplasmic reticulum. The
vesicles provide calcium to the Z-discs when the cell is depolarised. The cross-
bridge is formed by a binding of the actin and myosin molecules and requires
calcium ions to split the ATP and release energy for contraction. When the
muscle is in a relaxed state, the sarcomere contains a very low concentration
of calcium ions, so there is no interaction between the actin and myosin and
no ATP splitting. On activation the calcium ion concentration rises and so
cross-bridges are formed between the two sets of filaments, ATP is split and
sliding occurs [30].
3.3.3 Muscle Action in People with Physical Disabilities
Often, even people who have become severely paralysed will retain some level
of ability to contract certain muscles. For example, quadriplegic patients who
have been injured around the C5/C6 level usually retain the ability to move
their head to some extent. In some cases, this movement is sufficient to al-
low the person to communicate intent by operating head-switches, which are
usually affixed to their wheelchair. Unfortunately, although a person may still
be able to activate a muscle voluntarily, often the contractions may be too
weak to operate a conventional mechanical switch. This weakness is caused
largely by a loss of functional input from higher brain centres to the spinal
motor nerves, which leads to partial muscle paralysis and submaximal muscle
activation [33]. In these situations, the contraction must be detected by other
means.
The sternocleidomastoid muscle is one of the muscles which may often
still be under voluntary control in people with high-lesion quadriplegia. This
muscle is one of the muscles which flex the neck. The neck muscles are shown
47
67. Figure 3.8: The Neck Muscles, showing the sternocleidomastoid, from pg. 97 in [11]
in Figure 3.8. The sternocleidomastoid muscle receives motor supply from the
spinal part of the accessory (eleventh cranial nerve). It receives sensory fibres
from the anterior rami of C2 and 3 [11] and thus may still be controlled by
people who still have these nerve fibres intact, which usually includes people
with spinal cord injuries lower than this level. Unilateral contraction of the
sternocleidomastoid laterally flexes the head on the neck, rotating it to the
opposite side, and laterally flexes the cervical spine. Bilateral contraction
draws the head forwards and assists in neck flexion.
Differentiation between muscle contraction and muscle relaxation can be
used to control a single switch system e.g. a communication program. There
are two methods considered for measuring muscle contraction. Muscle con-
traction may be detected non-invasively by measuring either the electrical or
mechanical signal at the surface of the skin. The electrical signal is known as
the electromyogram and the mechanical signal is known as the mechanomyo-
gram. These will now be described in more detail.
48
68. 3.4 Electromyogram
The electromyogram or EMG is an electrical signal that can be used to observe
muscle contraction. It is measured either by using surface electrodes on the
skin (surface EMG) or by invasive needle electrodes which are inserted directly
into the muscle fibre (the invasive, needle or indwelling EMG). As mentioned
already, a muscle fibre contracts when it receives an action potential. The
electromyogram observed is the sum of all the action potentials that occur
around the electrode site. In almost all cases, muscle contraction causes an
increase in the overall amplitude of the EMG. Thus it is possible to determine
when a muscle is contracting by monitoring the EMG amplitude.
The EMG is a stochastic signal with most of its usable energy in the 0-
500Hz frequency spectrum, with its dominant energy in the 50-150Hz range.
The amplitude of the signal varies from 0-10mV (peak-to-peak) or 0-1.5mV
(rms) [34]. An example of an EMG and its frequency spectrum is shown in
Figure 3.9.
3.4.1 EMG Measurement
The EMG may be measured invasively or non-invasively. Clinical electromyo-
graphy almost always uses invasive needle electrodes as it is concerned with
the study of individual muscle fibres [35]. It produces a higher frequency
spectrum than surface electromyography and allows localised measurement of
muscle fibre activity [36]. For simple detection of muscle contraction, it is
usually sufficient to measure the electromyogram non-invasively, using surface
electrodes.
The standard measurement technique for surface electromyography uses
three electrodes. A ground electrode is used to reduce extraneous noise and
interference, and is placed on a neutral part of the body such as the bony part
of the wrist. The two other electrodes are placed over the muscle. These two
49
69. Figure 3.9: EMG and frequency spectrum, from [34], measured from the tibialis
anterior muscle during a constant force isometric contraction at 50% of voluntary
maximum.
electrodes are often termed the pick-up or recording electrode (the negative
electrode) and the reference electrode (the positive electrode) [35]. The signal
from these two electrodes is differentially amplified to cancel the noise, as
shown in Figure 3.10.
The surface electrodes used are usually silver (Ag) or silver-chloride (Ag-
Cl). Saline gel or paste is placed between the electrode and the skin to improve
the electrical contact [37]. Over the past 50 years it has been taught that the
electrode location should be on the motor point of a muscle, at the innervation
zone. According to De Luca [34], this is probably the worst location for detect-
ing an EMG. The motor point is the point where the introduction of electrical
currents causes muscle twitches. Electrodes placed at this point tend to have
a wider frequency spectrum [36] due to the addition and subtraction of action
potentials with minor phase differences. The widely regarded optimum posi-
tion to place the electrodes over the muscle is now on the belly of the muscle,
midway between the motor point and the tendinous insertion, approximately
1cm apart [36]. The electrode position on the muscle is shown in Figure 3.11.
50
