1. School of Engineering and Built Environment
MHW213190
HONOURS PROJECT MODULE
DISSERTATION
Submitted for the Degree of
BSc (Hons) in Audio Technology with Multimedia
2013/2014
Project Title: Music Therapy as a means of Cognitive Rehabilitation
for Stoke Survivors with Memory Issues post Stroke
Author: Samuel McMillan
Supervisor: Patrick Quinn
Except where explicitly stated all work in this report, including the
appendices, is my own
Signed: Date: 10/04/2014
2. “So we don't believe that life is beautiful because we don't recall it but if we get a whiff
of a long-forgotten smell we are suddenly intoxicated and similarly we think we no
longer love the dead because we don't remember them but if by chance we come across
an old glove we burst into tears.”
Marcel Proust
“Where does a thought go when it’s forgotten?”
Sigmund Freud
3. Acknowledgments
This project is dedicated to my family who believe in everything I do.
I am forever indebted to my fiancé for her unconditional support and the hundreds of cups of
tea she brought me during the writing of this dissertation.
Finally huge thanks to my mentor Patrick Quinn for believing in and supporting my project and
work from day one.
4. Abstract
Rehabilitation after a stroke is often challenging due to the damage of complex neural networks
caused by embolisms and cerebral haemorrhaging. Computer based software games present a
unique opportunity for rehabilitation of motor functions after a stroke.
Here the author reviews the possibility of a computer based system which incorporates these
theories along with current work in neurological music therapy as a means for cognitive
rehabilitation of stroke survivors with memory issues post stroke. The project attempts to
solve the issue of designing a system which incorporates these theories whilst considering the
intended user group through the use of user centred design theories. Expert opinions were
sought and user testing was designed as a means of discovering if the system could be
considered beneficial for the intended user group.
Heuristic and user analysis of the system have highlighted the potential benefits of the system
as well as the potential problems of this or similar systems.
6. 2.8 Discussion............................................................................................................................................................. 33
3. Methods.......................................................................................................................................................35
3.1 Introduction........................................................................................................................................35
3.2 Ethical Considerations ....................................................................................................................35
3.3 Subjects.................................................................................................................................................35
3.4 Evaluation............................................................................................................................................36
3.4.1 Heuristic Evaluation................................................................................................................................ 36
3.4.1 Pre Evaluation Questionnaire.............................................................................................................. 36
3.4.2 Post Evaluation Questionnaire............................................................................................................ 37
3.5 Predicted Usability Problems (PUP)..........................................................................................37
3.6 Severity Rating...................................................................................................................................37
3.7 User Testing........................................................................................................................................38
3.7.1 Think Aloud Protocol .............................................................................................................................. 38
3.7.2 Pre Test Questionnaire........................................................................................................................... 38
3.7.3 Post Test Questionnaire......................................................................................................................... 38
3.8 Conclusion ...........................................................................................................................................38
4. Implementation........................................................................................................................................39
4.1 Introduction........................................................................................................................................39
4.2 Heuristic Evaluation ........................................................................................................................39
4.3 Predicted Usability Problems.......................................................................................................41
4.4 Severity Rating...................................................................................................................................41
4.5 User Testing........................................................................................................................................42
4.6 Conclusion ...........................................................................................................................................42
5. Design...........................................................................................................................................................43
5.1 Introduction........................................................................................................................................43
5.2 High Level Design..............................................................................................................................43
5.2.1 Design Goals................................................................................................................................................ 43
5.2.2 Usability Goals............................................................................................................................................ 43
5.2.3 Man Machine Model................................................................................................................................. 44
5.2.4 System Control........................................................................................................................................... 45
5.2.5 System Architecture ................................................................................................................................ 45
5.2.6 Software Architecture............................................................................................................................. 45
5.2.7 Game Play Functionality........................................................................................................................ 45
5.3 Introduction to Interface/Game..................................................................................................46
5.3.1 Memory Game Concept .......................................................................................................................... 46
5.3.2 Memory Game Interface......................................................................................................................... 46
5.3.3 Typical User Scenario.............................................................................................................................. 47
5.3.4 The Grid ........................................................................................................................................................ 48
5.4 Low Level Design & Implementation.........................................................................................49
5.4.1 Eclipse Folder Hierarchy ....................................................................................................................... 49
5.4.2 Eclipse IDE................................................................................................................................................... 50
5.4.3 Eclipse Folder Structure......................................................................................................................... 51
5.4.4 Home Window............................................................................................................................................ 52
5.4.5 Music Selection Window........................................................................................................................ 53
5.4.6 Difficulty Selection Window......................................................................................................... 54
5.4.7 Game Window............................................................................................................................................ 55
5.4.8 Game Window Functionality................................................................................................................ 56
7. 5.5 Design Decisions ...............................................................................................................................57
5.5.1 A Usable & Beneficial Design ............................................................................................................... 57
6. Interface Analysis ....................................................................................................................................58
6.1 Evaluation & Results........................................................................................................................58
7. Project Discussion & Conclusion........................................................................................................59
7.1 Project Critique..................................................................................................................................59
7.2 Future Work........................................................................................................................................59
7.3 Conclusion ...........................................................................................................................................59
8 Appendixes..................................................................................................................................................60
8.1 Appendix A ..........................................................................................................................................61
8.1.1 Home Window Code................................................................................................................................ 61
8.1.2 Music Selection Window Code ............................................................................................................ 63
8.1.3 Level Selection Window Code.............................................................................................................. 65
8.1.4 Main Game Window Code ..................................................................................................................... 67
8.2 Appendix B ..........................................................................................................................................91
8.3 Appendix C........................................................................................................................................ 116
8.3.1 Expert A ..................................................................................................................................... 117
8.3.2 Expert B ..................................................................................................................................... 121
9 Bibliography ............................................................................................................................................ 124
8. Table of Figures
Figure 2- 1 The Successive Stages of Design Purchasing.............................................................................. 22
Figure 2- 2 Double Diamond Design Process Model ........................................................................................ 23
Figure 2- 4 HCD & The Double Diamond Design Process............................................................................... 24
Figure 2- 3 Iterative Design Cycle of HCD............................................................................................................ 24
Figure 2- 5 Quantitative Analysis in a Heuristic Evaluation Experiment................................................ 27
Figure 5- 1 Man Machine Model.............................................................................................................................. 44
Figure 5- 2 System Architecture.............................................................................................................................. 45
Figure 5- 3 Software Architecture.......................................................................................................................... 45
Figure 5- 4 Memory Game Interface...................................................................................................................... 46
Figure 5- 5 Game Grid................................................................................................................................................. 48
Figure 5- 6 Eclipse Folder Hierarchy..................................................................................................................... 49
Figure 5- 7 Eclipse Folder Structure A.................................................................................................................. 50
Figure 5- 8 Eclipse Folder Structure B.................................................................................................................. 50
Figure 5- 9 Eclipse Folder Structure Chart.......................................................................................................... 51
Figure 5- 10 Game Home Window......................................................................................................................... 52
Figure 5- 11 Music Select Window......................................................................................................................... 53
Figure 5- 12 Difficulty Select Window.................................................................................................................. 54
Figure 5- 13 Game Window...................................................................................................................................... 55
Figure 5- 14 Game Window Functionality.......................................................................................................... 56
9. 1 Introduction
‘A stroke is a brain attack. A stroke damages brain cells so they can no longer work properly. As
a result the areas of your body they control are affected’ (Cole, 1979). Every year there are
about 152,000 strokes in the UK. That’s more than one every five minutes. Most people
affected are over the age of 65, but anyone can have a stroke including children and babies. For
many people a stroke happens suddenly and without warning, and often there is little time to
prepare for one. A stroke can affect how you move, feel and think. Although everyone’s
experience is different, some of the effects are more common than others. After a stroke it is
common for short term memory problems to arise. Remembering new information can also be
difficult for many people and the speed at which people use their memory may appear to be
slower than usual (Anon, 2014). It is difficult to say how much recovery is possible, some
experience the most dramatic recovery during their stay in hospital in the weeks after their
stroke whilst many stroke survivors continue to recover over longer periods of time and
sometimes even years (Anon, 2013).
Currently there are a number of different methods of stroke rehabilitation all of which differ
depending on the needs of the individual. Some methods are based on rehabilitation through
some kind of physical motor training. Other methods exist which attempt to aid the
rehabilitation process through cognitive stimulation.
The aim of this project is to research and develop a usable software interface and game which is
beneficial to stroke survivors with memory issues after a stroke. It does this by first looking at
the intended user group and considering the needs of stroke survivor and the different physical
and cognitive impairments of stroke survivors. Research and development of the interface has
been supported through current work in human centred design processes and user centred
design theories in order to ensure the intended users are considered and their needs are met
throughout the project.
Combining the research and development process with the needs of the intended user group,
the double diamond design process, a model used to help identify possible problems and
discover solutions during product design, has been used to aid the development of the interface
and the project. Issues regarding usability, music therapy, memory and cognitive rehabilitation
are addressed through current research in the area of stroke survivors with memory issues post
stroke, music and emotion and user centred design theories.
Subjective and objective data is used to help answer the research question and understand if the
aims of the project have been met.
The report begins by giving an in depth review of the literature considered and a discussion on
the relationship between the literature and the project question and aims. This is followed by
an extensive review of the project methods, design and implementation. The final section of the
report highlights the results and discusses the successes and failures of the project.
10. 1.1 Research Question
For stroke survivors with memory issues post stroke, will the music and memory game
interface provide a usable means of cognitive rehabilitation?
1.2 Project Aims
To design a game which is usable and may benefit stroke survivor with memory issues after a
stroke?
11. 2 Literature Review
2.1 Stroke
2.1.1 Stroke Survivors
Every day your brain needs to process a huge amount of information from the world around
you. Your brain has to organise, interpret and store the information so that you can carry out
your day-to-day activities. A stroke can affect any part of this process, from picking up the
information, to planning how to respond (Helpline, 2012). Problems with memory, thinking or
understanding after a stroke are very common and most people who have had a stroke have
some kind of difficulty. The effects can be very mild or severe. This will depend on how severe
your stroke was and where in your brain it happened (Helpline). Some might experience pain
after a stroke, such as muscle tightness (spasticity) or shoulder pain. Some might also feel pain
and changes in sensation if the ‘pain centre’ in your brain has been damaged (Cole, 1979). Some
cognitive problems get better in time. However, other problems may last longer. No two
strokes are the same and it is very difficult to predict how much recovery you will make
(Helpline, 2012).
2.1.2 Memory
Memory is your ability to take in store and retrieve information. There are three major
processes in memory: encoding, storage and retrieval. In order to form new memories,
information must be changed into a usable form, which occurs through the process known as
encoding. Once information has been successfully encoded, it must be stored in memory for
later use. Much of this stored memory lies outside of our awareness most of the time, except
when we actually need to use it. The retrieval process allows us to bring stored memories into
conscious awareness. (Cherry, 2014). Memory is not one single system. You may remember
people or things you have seen, like what your friend looks like, or where you left your car keys.
This is called visual or spatial memory (Cole, 1979). We can also remember things from
different lengths of time using short term and long term memory. Short-term memory, also
known as active memory, is the information we are currently aware of or thinking about. In
Freudian psychology, this memory would be referred to as the conscious mind. Paying attention
to sensory memories generates the information in short-term memory. Most of the information
stored in active memory will be kept for approximately 20 to 30 seconds. While many of our
short-term memories are quickly forgotten, attending to this information allows it to continue
on the next stage - long-term memory. Long-term memory refers to the continuing storage of
information. In Freudian psychology, long-term memory would be called the preconscious and
unconscious. This information is largely outside of our awareness, but can be called into
working memory to be used when needed. Some of this information is fairly easy to recall,
while other memories are much more difficult to access (Cherry, 2014). Short term memory
recalls information from the recent past whilst long term memory can recall information from
years ago (Cole1979).
12. 2.1.3 Stroke & Memory Impairments
Damage to any perceptual or cognitive process can disrupt learning and recall dependent on
that process. Active learning and recall require alertness and attention: drowsiness, delirium or
confused states, even when mild, impair attention and, secondarily, memory (Westberg et al.,
2007). Memory impairments are common after a stroke. Most people have some problems
with memory after a stroke. Although every stroke is different it is very common for you short
term memory to be affected (Cole1979). There are several important facets of memory
impairment after stroke; every node of the limbic system implicated in memory may be
damaged by stroke but very rarely in isolation and the combination of amnesia with the
associated deficits often illuminates additional aspects of memory functions. Stroke produces
amnesia by damage to critical convergence white matter connections of the limbic system, and
stroke is the only etiology of amnesia that can delineate the entire pathway of memory and
critical convergence points. Stroke also impairs memory, without causing classical amnesia, by
damaging brain regions responsible for cognitive processes, some modality specific and some
more generally strategic, that are essential for normal learning and recall Task Orientated
Training (Lim & Alexander, 2009). Deficits in WM and executive attention are common
problems after acquired brain injury. Stroke-induced deficits in WM and attention are often
severe and result in impairments to vocational performance and social functioning (Westberg et
al., 2007). Also, ‘after a stroke you may have difficulty with your memory, your attention, with
making decisions and with understanding’ (Helpline, 2012).
Working Memory (WM) is also affected after a stroke. WM is the ability to hold and manipulate
information during a short delay, and to be able to make a response based on that internal
representation (Westberg et al., 2007). WM is one of the most important areas of our memory
system and vital if we're to successfully navigate through our world. It can be thought of as the
ability to hold and use a limited amount of information in our heads for a short amount of time.
Working memory can help us overcome a particular problem or perform a task, like mental
arithmetic, using a phone number or following a set of directions. However the amount of
information we can hold is limited and the information itself is very unstable, a sudden
distraction and the information is lost and you have to start again from scratch. Working
memory is essential for learning and development, particularly in childhood (Anon 2006).
13. 2.1.4 Working Memory Training
Research by (Westberg et al., 2007) found that victims who had acquired brain injuries, such as
a stroke can improve their attention by using a software based programme to train working
memory. Participants in the their programme demonstrated strong improvements in all tasks
related to working memory after the training, based on a neuropsychological test battery and a
self- reported rating scale regarding the symptoms of cognitive failure in daily life. Eight of the
nine participants (89 percent) in the treatment group reported a significant reduction in
cognitive failure according to a 25-question assessment. Westberg later commented on the
results of her research on working memory training stating ‘These results are especially
encouraging because there is a high correlation between working memory capacity and the
outcome of physical rehabilitation... This study is an indication of the broad potential of working
memory training. In many ways, we are only beginning to understand the tremendous impact
that this kind off focused training can have on the individual suffering from various cognitive
limitations’ (Anon, 2007). In a recent review of current literature on the subject (Morrison &
Chein, 2011) agreed, although a little more cautiously, with Westberg (2007) on the subject
stating ‘in a variety of areas of cognition (e.g., cognitive control, reading comprehension),
persistence with the use of tightly matched controls demonstrating activation changes in
regions associated with domain-general cognitive performance. Core WM training represents a
favourable approach to achieve broad cognitive enhancement’. Other studies on rehabilitation
of attention have had positive findings: Niemann et al (1990) and Gray et al (1992). Gray et al.
reported that after training, the treatment group showed improvement compared to the control
group on the Paced Auditory Serial Addition Test (PASAT) (Gronwall, 1977), a test requiring
working memory and control of attention. Klingberg (2006) has spent ten over years studying
the neural basis of working memory and developing methods for working memory through
training at the Karolinska Institute in Stockholm Sweden and Stanford University. One of
Klingbergs experiments involved asking participants to remember progressively longer strings
of information. Participants were later required to repeat the strings on request. Klingber was
later able to prove that his treatment group had improved significantly on working memory
tasks.
14. 2.1.5 Mirror Neuron System
Mirror neuron system (MNS) represents one of the most important discoveries of cognitive
neuroscience in the past decade, and it has been found to involve in multiple aspects of brain
functions including action understanding, imitation, language understanding, empathy, action
prediction and speech evolution(Chen & Yuan, 2008). There is increasing evidence that a
mirror neuron system also exists in humans (Small, Buccino, & Solodkin, 2012). The MNS was
first observed in the F5 region of the macaque monkey brain. This neurophysiological discovery
in macaques has inspired several brain imaging experiments in humans. The human brain
imaging experiments have demonstrated neural systems with mirroring properties in human
brain regions anatomically comparable to the monkey’s mirror neuron areas (lacobini &
Mazziotta, 2007).
2.1.6 Mirror Neurons and Goal Directed Actions
Mirror neurons encode the goal of motor commands (e.g., a grasp) rather than the motor
activation (e.g., close vs. open a hand to grasp) needed to achieve the goal (Thill, Caligiore,
Borghi, Ziemke, & Baldassarre, 2013). Fogassi et al (2005) further found that some, but not all,
mirror neurons in PC of monkeys are selective to ultimate (high-level) goals that a given action
contributes to (e.g., “grasp to eat” vs. “grasp to place”).
2.1.7 Mirror Neurons and Stroke Rehabilitation
Buccino et al, (2002) Buccino, et al, (2006) Ertelt et al. (2007) conducted tests with eight stroke
patients suffering moderate to chronic deficits of the upper body. The participants were asked
to observe simple [goal related] actions e.g. adding sugar to a cup, and then lifting the cup.
Patients were then asked to perform the same [goal related] actions. Before, during, and after
the treatment patients underwent a functional evaluation with the Barthel Index, Functional
Independence Measure, Frenchay Arm Test, and Fugl-Meyer test to evaluate the impairment of
the upper limb in everyday activities (Small et al., 2012). The results showed significant
improvement of motor functions in the course of a 4-week treatment, as compared to the stable
pre treatment baseline, and compared with a control group (Buccino et al., 2002; Ertelt et al.,
2007). The improvement lasted for at least 8 weeks after the end of the intervention (Small et
al., 2012). This work demonstrates the benefits of goal related action observation pairing up
with goal related action execution. Previous research has also come to the same conclusion
regarding goal related action observation.
15. 2.1.8 Audiovisual Neurons and Actions
‘About 15% of mirror neurons also respond to the specific sound of an action. These neurons
are called audio-visual mirror neurons’ (Kohler et al., 2002). ‘Audiovisual mirror neurons, code
actions independently of wither these actions are performed, heard or seen’(Gallese &
Rizzolatti, 2013). Experiments conducted by Gallese & Rizzolatti, (2013) concluded that ‘audio-
visual-mirror neurons... discharge not just to the execution or observation of a specific action
but also when the action can only be heard [and that] audiovisual mirror neurons also discharge
during the execution of specific motor actions’. Keysers C, Kohler E, Umilta ` MA, et al (2003)
elaborate on this phenomena stating ‘Another property of [audiovisual] mirror neurons is that
they fire at the sound associated with an action, such as breaking a peanut, tearing paper, and so
on, even though the action is not seen’. ‘Audiovisual mirror neurons could be used, therefore, to
plan/execute actions’ (Gallese & Rizzolatti, 2013). This work is interesting as it highlights a
possible avenue for stimulation of the trimodal neural system through audio associated with
specific goal related actions and the possible pairing of goal related actions with goal related
sounds.
16. 2.2 Music & Emotion
2.2.1 Emotion
Typical emotions arise in real world situations to help humans deal with the fundamental ‘life
tasks’ necessary for survival (Beveridge, Knox, 2009). In psychology, emotion is often defined
as a complex state of feeling that results in physical and psychological changes that influence
thought and behaviour. Emotionality is associated with a range of psychological phenomena
including temperament, personality, mood and motivation (Cherry, 2014). According to Meyers
(2004) human emotion involves physiological arousal, expressive behaviours, and conscious
experience. The James-Lange theory is one of the best-known examples of a physiological
theory of emotion. Independently proposed by psychologist William James and physiologist
Carl Lange, the James-Lange theory of emotion suggests that emotions occur as a result of
physiological reactions to events. According to this theory, you see an external stimulus that
leads to a physiological reaction. Your emotional reaction is dependent upon how you interpret
those physical reactions. For example, if you are walking in the woods and you see a grizzly
bear. You begin to tremble and your heart begins to race. The James-Lange theory proposes
that you will interpret your physical reactions and conclude that you are frightened. To put it
slightly differently, I am trembling, therefore I am afraid (Cherry, 2014).
17. 2.2.2 Music, Emotion & Mental Health
In the human brain, one of the most powerful sources of auditory stimulation is provided by
music (Sacks, 2006). Since the early 1990s, the study of music and its effect on brain function
has increased, providing insights into the neural mechanisms involved in music engagement. A
greater understanding of music neuroscience led to the development of a new approach to
music therapy particularly in rehabilitative clinical practice, neurologic music therapy’ (NMT)
(LaGasse & Thaut, 2013). Musical experience is one of the richest human emotional,
sensorimotor, and cognitive experiences. It involves listening, watching, feeling, moving and
coordinating, remembering, and expecting. It is frequently accompanied by strong emotions
resulting in joy, happiness, bittersweet sadness, or even in over- whelming bodily reactions like
tears in the eyes or shivers down the spine (Altenmuller & Schlaug, 2013). Listening to music is
a complex process for the brain, since it triggers a sequel of cognitive and emotional
components with distinct neural substrates (Peretz and Zatorre, 2005). Active ingredients of
NMT include auditory motor synchronization, an entrainment function whereby the auditory
rhythm is synchronized with movement execution as well as motivational aspects (Thaut,
2008). Music has a well-documented effect on alleviating anxiety, depression and pain in
patients with a somatic illness (Cassileth et al., 2003; Cepeda et al., 2006; Siedliecki and Good,
2006). People typically interact with music and value it for its capacity to evoke and regulate
emotions, provide enjoyment and comfort, and relieve stress (Juslin, Laukka, 2005). During
adulthood, music is strongly linked to emotional and self-conceptual processing, mood, and
memories (North et al,. 2000). By inducing positive affect and heightened arousal, exposure to
pleasant and enjoyable music can also temporarily enhance performance in cognitive domains,
including psychomotor or information processing speed, reasoning, attention and memory, and
creativity in healthy participants (Schellenberg et al., 2007).
Evidence from developmental animal studies also suggests that an enriched sound environment
can enhance auditory cortical functions (Engineer et al., 2004) as well as learning and memory
(Chikahisa et al., 2006; Kim et al., 2006; Angelucci et al., 2007a). Experiments conducted by
Whitall, Waller, Silver, & Macko (2000) which used repetitive bi lateral arm training through
rhythmic audio cuing training (BATRAC) showed ‘BATRAC improved several key measures of
sensorimotor impairments, functional ability (performance time), and functional use in patients
with chronic UE [Upper Extremity] hemi paresis [post sroke]. Improvements were maintained
at 2 months after patients stopped training, suggesting that the motor improvements were
potentially durable’ (Whitall, Waller, Silver, & Macko, 2000). More recently, experiments
conducted by LaGasse & Thaut (2013) which focused on rehabilitation of the upper extremities
in patients post stroke have already shown positive results in people with different speech and
movement problems. Persons with a hemiparetic arm [post stroke] benefited from auditory
rhythmic cueing, demonstrating decreased movement variability, increased speed of movement,
and a smoothing of the movement trajectory. LaGasse & Thaut, (2013) later go on to say ‘a
growing body of evidence has begun to demonstrate that the temporal organization of music
can be effectively utilized for rehabilitation of attention control, memory, psychosocial, and
executive function’.
18. 2.2.3 Music Making & Therapy
The brain as a highly dynamically organized structure can change and adapt as a result of
activities and demands imposed by the environment. Musical activity has proved to be a
powerful stimulus for this kind of brain adaptation, or brain plasticity, as pointed out by Wan
and Schlaug (2010). Making music is a powerful way of engaging multisensory and motor
networks, inducing changes within these networks and linking together distant brain regions.
These multimodal effects of music making together with music’s ability to tap into the emotion
and reward system in the brain can be used to facilitate therapy and rehabilitation of
neurological disorders. Joyful musical behaviours, for example, learning to play a musical
instrument or to sing is characterized by curiosity, stamina, and the ability to strive for
rewarding experiences in future results in incentive goal-directed activities (Altenmuller &
Schlaug, 2013). Additional research into Neurological Music Therapy has further highlighted
the benefits of this type of therapy in a music making type game for people with neurological
damage. Altenmuller & Schlaug, (2013) discuss these benefits suggesting ‘brain plasticity
induced through music making may produce manifold benefits’. Altenmuller & Schlaug (2013)
continue on this point stating ‘This holds not only for changing and/or restoring compromised
sensory motor brain networks but also for influencing neurohormonal status as well as
cognitive and emotional processes in healthy and neurologically diseased/disordered
individuals. Thus various sensory motor, coordinative, or emotional disabilities can be
improved with music supported therapies (MST)’. T Sarkamo et al, M Tarvaniemi, S Laitinen
(2008) also discuss the use of audio stating ‘multimodal stimulation, including auditory, visual
and olfactory stimuli, combined to the enriched motor environment enhanced motor and
cognitive recovery more than the enriched motor environment alone’.
2.2.4 Musical Structure
You are likely to have emotional ups and downs after a stroke. Feelings of depression, anger
and sadness, anxiety, low self esteem and loss of confidence are common (Cole, 1979). ‘Music
with particular structural features (moderate tempo and complexity) may be effective in
reducing pain and anxiety... music with different features such as tempo, complexity and key
signatures, can have a similar therapeutic effect’ (van Wijck et al., 2012). Current work also
suggests there are relationships between variables which relate to preference and key
structural features of music and that ‘it is not sufficient to only consider musical structure (e.g.
tempo) but, additionally, that musical preference of listeners must also be taken into
consideration’ (Van Wijck et al., 2012).
19. 2.3 Synaesthesia
2.3.1 Coloured Hearing Synaesthesia
Another area this project is interested in is synaesthesia. Synaesthesia is defined as the
involuntary and automatic perception of a stimulus in 2 or more sensory modalities (i.e., cross-
modal linkage)(Beeli, Essen, & Jancke, 2008). There are various different types of synaesthesia
e.g Grapheme Colour Synaesthesia and Tone Colour Synaesthesia, also known as Coloured
Hearing Synaesthesia. This project will look at research in Coloured Hearing Synesthesia.
People who suffer from synaesthesia are referred to as synesthetes. Coloured Hearing
Synesthetes experience colours when hearing tones or spoken utterances (Beeli et al., 2008).
Some initial research has also indicated it is a possibility that most people have synaesthesia to
some extent, the difference being, synesthetes are conscious of their synaesthesia.
20. 2.4 Interface Design
2.4.1 The Ergonomic Design Approach
Ergonomics is the study of the interaction between people and machines and the factors that
affect the interaction. Its purpose is to improve the performance of the system by improving
human machine interaction. The implementation of ergonomics in system design should make
the system work better by eliminating aspects of system functioning that are undesirable,
uncontrolled or unaccounted for (Bridger, 2003). An ergonomic approach can be adopted in
virtually any kind of design or purchasing project. Such an approach merely requires the
systematic application of ergonomic principals during each stage of the design (Dul,
Weerdmeester, 2008). An important characteristic of the ergonomic approach is the
involvement of users and other stakeholders in the project, in the earliest possible phase.
The goal of this policy is:
Avoidance of mistakes in design or purchase
Enlargement of acceptation
Development of more ideas
Faster identification of bottlenecks
Giving workers a say in their work and enlarging both autonomy and human well-being
Dul & Weerdmeester (2001) also make the point that it is better to apply ergonomic
requirements as early as possible in a project stating ‘it is better to apply ergonomics from the
outset (prevention) rather than retrospectively (cure)’ [and also that] ‘ergonomic requirements
should therefore be introduced as early as possible in a project, and must play a in every one of
its phases’. In planning the project a logical sequence should be considered as a means of
helping the project flow and to ensure no aspects of the project are overlooked. Dul &
Weerdmeester (2001) propose an iterative process is followed. Iterative design is a process of
designing a product in which the product is tested and evaluated repeatedly at different stages
of design to eliminate usability flaws before the product is designed and launched. In other
words, iterative design is a process of improving and polishing a design over time. An example
of iterative design is Wikipedia, where users can add missing information and correct mistakes
that have been made by former contributors (Anon 2014).
21. 5 Stages of project planning
Dul and Weerdmeester break the project planning down into 5 stages which are described in
more detail below.
1. Initiative
Initiative involves making a detailed survey of the project, formulating the question and
planning the rest of the project. Sub sections of the initiative phase are:
Define who is involved in the project
Make sure that those involved support the project
Agree a code of conduct
Do not raise false expectation on the part of the user
State the limits of the project
Describe the planned course of the project at the outset
2. Problem Identification
Problem identification involves gathering the required data for the project, which usually leads
to a multitude of alternatives and solutions. Sub section of the problem identification section:
Establish at the start how the data will be processed
Select more than one analysis technique
Always start with a survey of existing documents
Make sure that the analysis does not influence the results
3. Selection of solutions
Selection of solution involves selecting from alternatives and further developing the selected
options. Sub sections of select the solutions stage:
Realise the textbook, software and other tools do not provide a complete answer
Allow users to work with a prototype
Remember the indirect user
22. 4. Implementation
The implementation stage is where the solutions are implemented in the project. Sub sections
of the implementation stage:
Select an implementation strategy beforehand
Tests should be realistic
Train all users
Support the implementation by providing good manuals
Give the users a role in organizational challenges
Convince the users of the improvement
5. Evaluation
The final stage is the evaluation of the options and the project. Sub sections of the evaluation
stage:
Keep the same techniques of data collection
Allow teething problems to sort themselves out
Beware of habit information
Prepare your own checklist based on that of others
5 Steps of project planning diagram
Figure 2- 1 The Successive Stages of Design Purchasing
23. 2.4.3 The Double Diamond Design Process Model
The double diamond process is a method used by designers to help identify problems and
solutions whilst designing products. The double diamond design process was developed
through in house research at the design council in 2005 as a simple graphical way of describing
the design process. The design council breaks their interpretation of the design process down
into four distinct phases, Discover, Define, Develop and Deliver. This design process maps the
divergent and convergent stages of the design process. Figure 1.1 below shows the different
modes of thinking used during the process.
Figure 2- 2 Double Diamond Design Process Model
During the first stage the designer, through research begins to think about or diverge on the
possible design problems and fundamental issues before converging and narrowing in on the
real underlying problems of the design. Next design research tools are used to explore possible
answers before converging on a final solution. The design process incorporates many other
areas other than discovering the correct problem and solutions, for this reason the double
diamond process is coupled with other processes of design including the human centred design
process.
24. Idea
Generatio
n
Testing
Observati
on
Prototypi
ng
2.4.2 Human Centred Design Process
Human-centered design (HCD) is the process of ensuring that people’s needs are met, that the
resulting the resulting product is understandable and useable, that it accomplishes the desired
tasks, and that the experience of use is positive and enjoyable. Effective designs need to satisfy
a large number of constraints and concerns including shape and form, cost and efficiency,
reliability and effectiveness, understandability and usability, the pleasure of the appearance, the
pride of ownership, and the joy of actual use. HCD helps us achieve these requirements through
the use of an iterative design cycle. The iterative design cycle of HCD incorporates four main
activities; observation, idea generation, prototyping and testing.
HCD focuses on two things: solving
the right problems and doing so in a
way that meets human needs and
capabilities (Norman 2013). Solving
the right problems is done by first
finding the right problems. Norman
(2013) informs us that after finding
the right problems we then go about
finding the right solutions to those
problems. A benefit of these two
stages of the human centred design
process is that they are easily
incorporated into the double
diamond design process model as
shown in figure 1.2 below
Figure 1.2 shows the different
phases described by the design
council at each of the stages of
the design process. Figure 1.2
also shows how Norman (2013)
two main stages of HCD,
(defining the right problem and
finding the right solution), are
incorporated into the double
diamond design process.
Figure 2- 4 HCD & The Double Diamond Design Process
Observation and idea generation are then combined with the define the problem and define the
solution stages consecutively to provide a complete design process which can be then followed
from the beginning of the project until the end of the project offering a structured method of
research and development. The prototyping and testing stages are then conducted, the results
of which are a reflection on the completeness of the two previous stages.
Figure 2- 3 Iterative Design Cycle of HCD
25. 2.4.2 User Centred Design
User Centred Design is a philosophy based on the needs of the user, with an emphasis on
making things usable and understandable. User centred design principals are also useful in
developing interfaces for human interactions. Norman (1990) describes two main principals of
design, ‘Making Things Visible’ and ‘Providing a good conceptual Model’. Making things visible
is made up of three things; Affordance, which is based on perceived and actual property,
Constraints, which is the limitation on what something can do, and Mapping, the relationship
between two items. Providing a good conceptual model ‘allows us to predict the effects of our
actions’ when we operate a product. Jordan (2001) also describes a number of factors which
should be considered when designing a usable product. Jordan’s Principals of Useable Design
are; ‘Consistency, Compatibility, Consideration of User Resources, Feedback, Error Prevention
and Recovery, User Control, Visual Clarity, Prioritization of Functionality and Information,
Appropriate Transfer of Technology and Explicitness’.
2.4.3 Usability
Usability has become a primary factor in determining the acceptability and consequent success
of computer software (Neilson, 1993; Sneiderman, 1992). Usability is most often described as
the ease of use and acceptability of a system for a particular class of users carrying out specific
tasks in a specific environment. Ease of use can affect the user performance and their
satisfaction, while acceptability affects wither the product is used (Holzinger, 2005). Bevan
(1995) states ‘usability is measured by the extent to which the intended goals of use of the
overall system are achieved ’. Schneiderman and Plaisant (2004) state that setting specific goals
help designers to help ensure a high level of usability may be obtained. These goals should
include; ascertain the user’s needs, ensure proper reliability, promote appropriate
standardization, integration, consistency, and portability and complete projects on schedule
within budget. Sharp, Rogers and Preece (2007) also list the following as important usability
goals; Effectiveness, Efficiency, Safety, Utility, Learability, Memorability.
2.4.4 User Group
When designing computer programmes or information systems it is important to know who the
users of the programme will be. The possibilities and limitation of these persons determine to a
large extent how the user interface should be designed in an optimal way (Dul, Weerdmeester,
2008). Before embarking on a design it is important to be clear about the characteristics of
those who will use the final product (Jordan, 1998). For many products usable design will be
heavily dependent on the user group. Examples of the characteristics could be, say, height,
reach or strength (Jordan, 2008). As well as the user’s physical characteristics, it is also
important to take into account their cognitive characteristics when designing a product. These
include for example, any specialist knowledge that the user may have, attitudes the user may
hold, or any exceptions that the users are likely to have of a product. Again these factors are
likely to vary according to who the target group for the product are (Jordan, 1998).
26. 2.4.5 Cognition
Cognition is what goes through our head when we carry out our everyday activities. It involves
cognitive processes, like thinking, remembering, learning, daydreaming, decision making,
seeing, reading, writing and talking (Sharp, Rogers, Preece, 2007). Norman (1993)
distinguishes between the two general modes: experiential and reflective cognition. Reflective
cognition is a state of mind in which we perceive, act and react to events around us effectively
and effortlessly. It requires reaching a certain level and engagement. Examples include driving
a car, reading a book, having a conversation and playing a video game. Reflective cognition
involves thinking, comparing and decision making. This kind of cognition is what leads to new
ideas and creativity. Examples include designing, learning and writing a book. Norman (1993)
points out that both modes are essential for everyday life, but that each requires different kinds
of technological support.
2.4.6 Cognitive Requirements
Jordan (2001) highlights the importance of cognitive requirements placed upon the user whilst
operating software. Jordan breaks his cognitive requirements down into a number of different
sections including Mental Models, Learning, Experience, Conceptual Model and Metaphors.
Mental Models ‘allow us to make sense of the knowledge we have and to estimate future
actions... [Jordan later goes on to say] Establishing a good mental model assists in learning’
(Jordan 2001). Another technique used to aid the process of designing an interface with lower
level cognitive demands is the use of metaphors. ‘Metaphors are used to make interfaces more
accessible and easier to understand’ (Jordan 2001). Rogers et al (2007) also makes this point
on metaphors stating ‘Metaphors can make learning new systems easier’ and ‘Help the user
understand the underlying conceptual model’. Norman (1990) makes the point that ‘structuring
information helps users store and retrieve information well’. We can recognise material quicker
than we can recall from memory using what Norman calls ‘Knowledge in the World’. Interfaces
which are designed based on knowledge in the world hold users attention better, allowing
functions to be carried out more automatically and reduce the amount of information the user is
asked to remember.
27. 2.5 Interface Design Analysis
2.5.1 Measuring Usability
Usability is a general concept that cannot be measured but is related to several usability
parameters that can be measured. Measurable usability parameters fall into two broad
categories: objective performance measures, which measure how capable the users are at using
the system, and subjective user preference measures, which assess how much the user like the
system (Nielsen, 1993). One way to classify methods for evaluating the usability is from the
kind of results obtained. There are quantitative evaluation methods to test usability and
qualitative evaluation methods, both being complementary methods and not mutually exclusive.
2.5.2 Quantitative and Qualitative Methods
Quantitative and Qualitative analysis methods produce different kinds of results. If set up
properly, quantitative and qualitative analysis method can be complimentary in obtaining the
required data (Gonzalez, Masip, Granollers, & Oliver, 2009). On the one hand, quantitative
usability estimation is typically associated with the calculation of metrics that assess some
factors or dimensions of software quality. On the other hand, qualitative usability estimation is
also normally included, since no quantitative measure can be expressive enough to represent
something as complex as the overall usability of a software problem or a user desire (Gonzalez,
Granollers, & Lores, 2008). Both methods can serve different purposes whilst treating the same
subject as shown in figure 2-5 below
Figure 2- 5 Quantitative Analysis in a Heuristic Evaluation Experiment
28. 2.5.3 Expert Appraisal
An expert appraisal is a non empirical qualitative analysis which uses an expert or experts to
analyse wither a product can be considered as useable (Jordan, 1998). This approach involves
an expert (human factors, system design, software engineering) making an assessment of the
system. This may firstly be an appraisal of the system conformance where the expert compares
the system with established human factors, standards guidelines or principals (Sweeny,
Maguire & Shakel, 1993). Expert reviews can occur early or late in the design phase. The
outcome may be a formal report with problems identified or recommendations for changes
(Schneiderman, Plaisant, 2010). ‘An expert in this context is an investigator who’s education,
professional training and experience make him or her able to make an informed judgment on
usability issues with respect to the product under investigation’(Jordan, 1998). Shcneiderman
and Plaisant (2010) make a similar point stating ‘These methods depend on having experts
whose expertise may be in the application or user interface domain’.
29. 2.5.4 Heuristic Evaluation
There are a variety of expert review methods to choose from including; a Heuristic Evaluation
(HE) and Playability Heuristics. A HE is a cheap and easy method of evaluating a user interface
design. HE is one of the most popular of the usability inspection methods developed by Nielsen
and Molich (1990). Heuristic evaluation refers to a class of techniques evaluator use to examine
an interface for usability issues. A HE is also known as a Usability Inspection Method UIM.
UIM’s such as HE’s can serve as a discovery resource for user testing. HE outlines the usability
problems in user interface design based on usability heuristics. Numerous sets of heuristics can
be applied during HE (Nielsen and Molich, 1990; Nielsen, 1992; Nielsen, 1993; Nielsen, 1994;
Molich and Nielsen, 1990; Nielsen, 1990; Folmer and Bosch, 2004).
In a Heuristic Evaluation the expert reviewer critiques the interface to determine conformance
with a short list of design heuristics like the Eight Golden Rules of Interface Design which are;
strive for consistency, cater to universal usability, offer informative feedback, design dialogs to
yield closure, prevent error, permit easy retrieval actions, support internal locust of control and
reduce short term memory load (Schneiderman, Plaisant, 2010). Another commonly used
heuristic design method which can be used to determine conformance is Nielsen’s ten heuristics
principles. Nielsen’s principals are as follows; visibility of system status, match between system
and the real world, user control and freedom, consistency and standards, error prevention,
recognition rather than recall, flexibility and efficiency of use, aesthetic and minimalistic design,
help users recognise, diagnose and recover from errors and Help and documentation (Nielsen,
1993; Nielsen, 1994). Heuristic can also be useful when looking to discover usability issues in
the design process as this type of evaluation does not require users, who can be hard to get, and
can be used initially to rinse out a number of usability problems (Hvannberg, Law, &
Larusdottir, 2007). In HE sessions, each evaluator inspects the interface alone and generally
only finds a small number of usability problems.
Severity ratings can be collected by sending a questionnaire to the evaluators after all
evaluation sessions, listing the complete set of usability problems that have been discovered,
and asking them to rate the severity of each problem (Kilic Delice & Gungor, 2009). Severity
rating of a usability problem is determined by multiple factors. Nielsen defined the severity of a
usability problem as a combination of the following three factors: 1. frequency with which the
problem occurs: Is it common or rare? 2. The impact of the problem if it occurs: Will it be easy
or difficult for the users to overcome? 3. The persistence of the problem: Is it a one-time
problem that users can overcome once they know about it or will users repeatedly be bothered
by the problem? (Nielsen, 1994).
Several factors, besides the set of usability heuristics can influence the performance of heuristic
evaluations, such as evaluator training, evaluator knowledge of the application domain, task
coverage, problem extraction/description, merging etc (Hvannberg, Law, & Larusdottir, 2007).
Playability heuristics are split into three sections; Game Usability, Mobility Heuristics and Game
Play Heuristics. Game play heuristics are the most difficult to evaluate because familiarity with
all aspects of the game is required.
30. 2.5.5 User Testing
The most revealing method of a usability evaluation is to set up system trials where
representative users are asked to perform a series of tasks. Methods which involve participants
are known as empirical methods (Jordan, 1998). Empirical usability testing can be done by
setting up a controlled experiment. The aim is to gather information about the user’s
performance with the system, their comments as they operate it, their post test reactions and
the evaluator’s observations (Maguire, 2001). A controlled user testing session will typically
involve running test sessions with 8 to 25 users. The main benefit of this approach is that the
system can be tested under conditions which reflect the real world use. While technical
designers and human factors experts may diagnose a large proportion of potential system
problems, experience has shown that working with users will reveal new insights that can affect
the system design (Maguire, 2001). Jordan (1998) also makes a similar point stating “methods
involving participants have an added value - uncovering unexpected usability problems.
Similarly, users may actually be able to cope easily with aspects of a product where, according
to conventional human factors wisdom, they might be expected to struggle”.
There are a number of methods to choose from when performing usability test which involve
the use of participants. Each method has its own unique advantages and disadvantages. Some
methods are more tightly controlled than others. Jordan (1998) gives a detailed description of
recommended empirical methods which includes; private camera conversations, co-discovery,
focus groups, user workshops, think aloud protocols, incident diaries, feature checklists, logging
use, field observation, questionnaires, interviews, valuation method and controlled
experiments.
31. 2.6 Software Design
2.6.1 Java
The Java programming language was orientated by Sun Microsystems in 1995 (Bishop, 2005).
Java is the most widely used object orientated programming language in the world (Anon,
2013). ‘As a primary means for structuring a programme or design, OOP [Object orientated
programming] provides objects. Objects may model real life entities, may function to capture
abstractions of arbitrary complex phenomena, or may represent system artefacts such as stacks
or graphics’ (Eliens, 1995). Bishop (1995) gives a more simple description of Object Orientate
Programming (OOP) stating ‘An OOP is based on one or more simple elements called classes...
The programmer defines the class by providing it with a set of data and with one or more
methods for handling the data’ (Bishop 2005). Methods are short programme segments. ‘The
data and methods are objects which are to be handled in the way set out by the class’ (Bishop
2005). Because software written in Java can be composed of hundreds or thousands of java
classes and interfaces, related classes are placed into packages to help keep things organized.
Anon (2014) describes the package as ‘a name space that organizes a set of classes and
interfaces’. Sets or library’s of packages containing thousands of classes are provided by java to
help programmers complete the tasks most commonly associated with general purpose
programming. These libraries are called Application Programming Interfaces (API’s) (Anon,
2014). API’s allow programmers to focus on the design of programmes rather than the
infrastructure required to make them work (Anon, 2014).
Another benefit of Java is that it is platform independent. This allows java programmes to be
run on different types of computers without having to be programmed differently for each
computer (Skansholm, 2000). Using Java to create a Graphical User Interface is also useful
because ‘Java contains classes to generate graphical user interfaces (GUI’s). GUI programmes
can be written with the help of java, that is, programmes communicating with the user through
windows, menus, buttons etc’ (Skansholm, 2000).
2.6.2 Light Weight Java Gaming Library
Along with the standard libraries provided by java the Lightweight Java Game Library (LWJGL)
is also being used to help create the game. The LWJGL is an open source java software library
for computer game developers. LWJGL exposes high performance cross platform libraries
commonly used in software games and multimedia titles. It exposes OpenGL (Open Graphics
Library), OpenAL (Open Audio Library), OpenCL (Open Computing Library) and allows access to
controllers such as gamepads, steering wheels, and joysticks in a platform neutral way (Anon,
2013). The lightweight Java Game Library is a solution aimed directly at professionals and
amateur Java programmers alike to enable commercial quality games to be written in Java
(Anon, 2013). As well as enabling commercial quality games LWJGL is being used in the project
because ‘it is primarily an enabling technology which allows developers to get at resources that
are simply otherwise unavailable or poorly implemented on the existing Java platform (Anon,
2013). The LWJGL is available under a BSD licence, which means its open source and freely
available at no charge’ (Anon, 2013).
32. 2.6.3 Eclipse
Along with Java and LWJGL the game is being coded inside the Eclipse Integrated Development
Environment. An integrated development environment is a programming environment that has
been packaged as an application programme, typically consisting of a code editor, a compiler, a
debugger, and a graphical user interface builder (Rouse, 2007). IDE’s encompass all the
features a programmer requires and help programmers work faster. IDE’s also include
additional features like automatic code completion, syntax highlighting and graphical user
interface builders. Whilst typing inside an IDE, auto code completion can help by showing a list
of possible options. For example when using a string object a programmer might want to use
one of its methods. As they type, a list of commands they can chose from will appear in a popup
menu (Leahy, 2014). Syntax highlighting assists programmers by highlighting different parts of
code with different colors making code easier to read and debug. Graphical user interface
builders allow users to create interfaces by dragging and dropping swing components onto a
canvas whilst the IDE automatically writes the code required to create the GUI (Leahy, 2014).
‘The Eclipse IDE for Java developers contains what you need to build Java applications.
Considered by many to be the best Java development tool available, the Eclipse IDE for Java
Developers provides superior Java editing with validation, incremental compilation, cross-
referencing, code assist; an XML editor; Mylyn; and much more’ (Anon, 2013).
2.7 Image Design
2.7.1 GIMP
In line with the Interface Design Principals additional research has been carried out on using a
free picture editing software programme called GNU Image Manipulation Programme (GIMP).
GIMP is a ‘raster graphics editor used for image retouching and editing, free-form drawing,
resizing, cropping, photo montages, converting between different image formats and more
specialized tasks’ (Anon, 2014). A number of online tutorials have been studied to ensure a high
level of proficiency in using the programme. This additional work has also been carried out to
help ensure a high level of design may be achieved using professional graphics in the project.
33. 2.8 Discussion
Section 2.1 of the literature review highlights some of the physical and cognitive characteristic
of a stroke survivor post stroke. One of the most important issues of the research is that it
shows that almost everyone suffers from some kind of memory issue after a stroke indicating
the necessity of research and development of interventions to help the people affected. A
number of different issues are highlighted including the problems in memory, thinking and
understanding however in an attempt to help answer the research question this research
focuses more on memory issues. Damage in areas of the brain responsible for cognitive
processing is often sever and affects many functions including vocational performance and
social functioning. Memory training research, work by (Westberg et al, 2007) gives a clear
motive for the design and implementation of a software based system and demonstrates the
benefits of such a system. Work by Klingberg (2006) also identifies the possibilities and
benefits of memory training as well as giving a clear direction as to how memory training might
be carried out successfully. This research then may be helpful in ensuring a game is designed
and features are implemented which are beneficial to stroke survivors and the issues they are
faced with.
Research into the mirror neuron system and synaesthesia is included in this paper as it was part
of the original plans and aims of the project during the early planning stage. The research was
conducted in an attempt to discover a means of cognitive rehabilitation of stroke survivors with
upper extremity issues post stroke before the aims of the project changed to stroke survivors
with memory issues post stroke. The research indicated the possibility of a system which
incorporated feature used to stimulate the audio, visual and motor regions of the brain through
goal directed actions observation and execution. As well as this the research suggested some
interesting features of goal directed sounds which may have been applicable in a software
system to be used as an alternative means of cognitive rehabilitation. The research highlighted
the possibility of an intervention which avoids the issues of pain and muscle spasticity felt by
stroke survivor discussed in section 2.1.1 through stimulation of neural regions via non motor
activities e.g. goal directed action observation. Synaesthesia research was conducted in an
attempt to find a means of reinforcing the audio visual stimulation of the tri modal mirror
neuron system. Synaesthesia research presented unique theory on the mapping and
relationships between sound and light due to the cross modal linkage phenomena described in
section 2.3.1. Theory of the mirror neuron system and synaesthesia combined may then have
been useful in a system which was looking to help stroke survivors with upper extremity issues
through non physical means of rehabilitation. Unfortunately due to previously unrealistic goals
which were out with the scope of the project the aims were changed and research into the
mirror neuron system and synaesthesia has not been considered or implemented in the project.
Section 2.2 highlights the benefits of using music in a system looking to aid the cognitive
rehabilitation process. Through complex musical structures the rich cognitive experience
provided by music can be used to trigger cognitive and emotional components. The benefits of
this include increased cognitive performance, alleviation of anxiety depression and pain.
Increased cognitive abilities such as improvements in speed, reasoning, attention and memory
through music all give a clear motive for use of music in a system which aims to benefit stroke
survivors.
34. Research carried out in section 2.4 was completed in order to ensure the aim of the project to
design a game for stroke survivors was successful. The research highlights a number of ideas
and concepts which are considered by designers throughout user centred design processes. The
research gives clear directions regarding the intended user group, physical and cognitive
characteristics of users and also explains usability in detail. Usability research is also important
because it provides a measurable means of testing, described in section 2.5, to help discover if
the interface is useable by the intended user group which is also part of the project question and
part of the aim of the project. Research into usability highlighted some important
considerations regarding stroke survivors and their needs and requirements.
Regarding memory issues suffered by stroke survivors, usability point out the issues of memory
requirements whilst using interfaces providing important direction things like the amount of
information and details a user should be asked to retain whilst operating the game.
The final section of the research, sections 2.6 and 2.7 provide a means of understanding how a
computer based game may be designed and created using graphics and computer programming
language java. The main issues discovered during this research were the complexity of
computer programming languages due to the different libraries and thousands of lines of code
available to users. Learning java and making decisions on things such as the best or most
appropriate classes and functions for particular task makes the possibility achieving the project
aims particularly difficult.
35. 3. Methods
3.1 Introduction
This section details the methods used during the experimental stage of the project. A detailed
description of each method is given here as well as justification as to why each method has been
added and how this will further the aims of the project.
3.2 Ethical Considerations
The experimental stage of the project involved a heuristic evaluation and a user testing stage.
An application was therefore submitted too, and permission was granted by, the university
ethics committee. Full details of the forms submitted and permissions can be found in
appendixes section D.
3.3 Subjects
Two stroke experts working with Glasgow Caledonian University were recruited to complete
the heuristic evaluation. The user testing was designed to include audio technology and
multimedia students whose recent education on the subject provided them with the means to
give informed judgment and decision on usability issues of the interface.
36. 3.4 Evaluation
3.4.1 Heuristic Evaluation
Having reconsidered ethical issues and the potential problems surrounding permissions and the
use of stroke survivors during the early stages of the project the heuristic evaluation method
seemed appropriate for the project when the project became a design based project which
primarily focuses of researching and developing software. Also the heuristic evaluation
method was chosen because of the unique way in which it allows usability issues to be detected
without the use of any intended users. Another benefit in using a heuristic evaluation as part of
the project was it allowed a number of usability problems to be rinsed out initially before
further research was conducted. This ensured the development stage was completed to a
reasonably high standard before the user testing stage was completed (Hvannberg, Law, &
Larusdottir, 2007).
As explained in section 2.5.4 a heuristic evaluation is a qualitative evaluation method. A
qualitative usability method also seemed a better choice for the experimental section of the
project as no quantitative measure can be expressive enough to represent something as
complex as the overall usability of a software problem or a user desire (Gonzalez, Granollers, &
Lores, 2008). Also a HE is a cheap and easy method of evaluating a user interface design. Using
the HE method then ensured issues regarding funding or finances were avoided during the user
testing stage. As HE’s can serve as a discovery resource for user testing, also described in
section 2.5.4, this method then also afforded the opportunity to introduce a user testing stage as
part of the experimental work which in turn provided more comprehensive results during the
experimental work. HE also presents the tester with a set of techniques which help outlines the
usability problems in user interface design based on usability heuristics which are explained in
more detail in section 4.1.
3.4.1 Pre Evaluation Questionnaire
A pre evaluation questionnaire was added as part of the heuristic evaluation to help gain and
understanding of the area of expertise of each expert taking part in the evaluation. Experts
were asked to give information regarding their current job and also experience in the areas
under investigation as part of the study. The purpose of the pre evaluation questionnaire is to
allow the examiner to gain a better understanding of the experts opinions based on their
particular area of expertise. The questionnaire also aims to discover any additional training or
education which could influence the opinions given as part of the evaluation. The overall aim
here is to ensure opinions used during design based decisions are supported by knowledge,
training or education in the area of study.
37. 3.4.2 Post Evaluation Questionnaire
Performance metrics are important to collect when improving usability but perception matters
just as much. Asking a user to respond to a questionnaire immediately after attempting a task
provides a simple and reliable way of measuring task-performance satisfaction (Sauro, 2010).
For this reason the After Scenario Questionnaire which was designed for computer usability
studies (Lewis, 1991) was used as part of the user testing stage.
3.5 Predicted Usability Problems (PUP)
As part of the heuristic evaluation process a structured problem report format devised by
Cockton & Woolrych (2001) is used during the evaluation. Using a structured problem report
format can improve the reliability of merging predicted usability problems (PUPs) and the
reliability of matching predicted to actual problems (Cockton et al., 2003a), thereby increasing
the overall internal validity of the usability evaluation results (Hvannberg, Law, & Larusdottir,
2007).
3.6 Severity Rating
As described by Nielsen (1994) severity of the usability issues detected by each expert during
the heuristic evaluation are later rated by all experts to help gain a more comprehensive
understanding of the opinions and possible issues. Asking each expert to rate the severity of all
the issues highlighted during the HE helps to determine different levels of severity. This
process ensures separation of more or less serious issues which may then be considered during
the design stage. The severity rating method uses the following terms used to rate the least
serious to most serious design issues; minor, medium or catastrophic. Issues regarding the
selection of appropriate experts and validity of expert opinions are also highlighted during this
process. For example opinions given by one expert during the HE may be reflected and
reinforced by the opinions of all of the other experts during the severity rating procedure
highlighting the validity of said expert’s opinions. This process is then also true and can be used
to help root out or highlight opinions which should be disregarded giving priority to opinions
which can be used to help make better design based decisions.
38. 3.7 User Testing
3.7.1 Think Aloud Protocol
As part of the user testing stage the think aloud protocol was used to help obtain possible design
problems and solutions. The think aloud protocol allows the examiner to prompt the user
participants ‘in order to encourage him or her [the participants] to make helpful verbalizations.
These prompts may simply be of the general type, for example, ‘What are you thinking now?’, or
they may be more specific, perhaps relating to a particular error that has been made’ Jordan
(1998). Information obtained during the think aloud protocol is also important because the
‘think aloud protocols can be an excellent source of prescriptive data, which can lead directly to
design solutions’ Kerr and Jordan (1994).
3.7.2 Pre Test Questionnaire
The pre test questionnaires implemented during the user testing stage were included to help
gain an insight into the technical background of the participants. Gaining an understanding of
the participant’s technical ability was necessary in helping to understand particular issues
found by each participant. Similarly to the severity rating procedure described earlier, the pre
test questionnaire was necessary to help detect and eliminate false problems detected by
participants. Participant will lower level technical skills may find more problems with the
interface design than participant with higher level technical skills. Difference in results which
correlate with difference in technical skills could then be used to help make design decisions
based on the intended user group.
3.7.3 Post Test Questionnaire
Similarly to section 3.4.2, the post test questionnaire was added after the user testing stage to
help measure task performance satisfaction and gain a better understanding of the opinions
given by each participant based on their perception. This is important because gaining an
understanding of the user’s perception of the test can help inform the importance of the
evaluation and the effort of the participants.
3.8 Conclusion
As part of the experimental design process a heuristic evaluation and user testing stage have
been combined to help highlight potential design issue and solutions based on the requirements
of the intended user group. In line with this the PUP structured problem report, severity rating
questionnaire, pre test questionnaire and post test questionnaire are included to help highlight
potential issues with selection of participants involved in the experimental stage of the project.
The experimental stage of the project has been carefully designed using these methods to help
ensure a high level of usability is achieved in line with the project aims.
39. 4. Implementation
4.1 Introduction
This section of the report explains how the methods used in the experimental section of the
project were implemented into the project and how the experimental work was carried out. A
detailed breakdown of the HE, PUP, Severity Rating and User Testing processes used during the
experimental stage is given.
4.2 Heuristic Evaluation
Before the heuristic evaluation process each expert was given a checklist of activities as part of
the heuristic evaluation documentation which were required to be completed as part of the
evaluation. Stage 1 - below show details of the activities check list. Stage 1.1, 1.2 and 1.3 shows
a more detailed breakdown of stage 1 sub stages. The severity rating stage which is shown in
section 1 below is explained in more depth in Stage 2 of the evaluation process. Full details of
the checklist and the complete heuristic evaluation documentation can be found in section B of
the appendixes.
Stage 1 - Checklist of activities they were asked to perform;
1.1 Pre Evaluation Questionnaire
1.2 Evaluation
1.3 Post Evaluation Questionnaire
Stage 2 - Severity Rating (Completed after stage 1 Evaluation results have been gathered).
Stage 1.1 - Pre-Evaluation Questionnaire;
Name
Area of Expertise
Stage 1.2 - Heuristic Evaluation
Introduction to the material including
Guidelines for the procedure
Introduction to the system
Introduction to the reporting format
Evaluation
Stage 1.3 - Post-Evaluation Questionnaire
Report time spend in minutes/hours
Give a list of facilitators and hindrances of the heuristic evaluation method applied
40. Along with the checklist the document was designed to ensure the experts had a clear
understanding of the project as well as their responsibilities having agreed to take part in the
evaluation. The pre evaluation questionnaire was added to help attain a clearer understanding
of the area of expertise of each expert. This information was required to help understand the
answer given by the experts based on their particular area of expertise. In addition to this any
issues not detected by a particular expert but detected by others could then be considered
based on possible inexperience in a given area. The introduction to the game and typical user
scenario were provided as training to help each expert answer heuristics questions successfully
using only the paper prototype. The introduction and description of the aims of a heuristic
evaluation was given to help the experts understand the intentions and goals of the evaluation
and to help ensure the answers given were clearly focused on usability issues. The introduction
to the reporting format which is explained in section 3.4 (Predicted Usability Problems) was
added to ensure the answers given were clear and increase the overall validity of the usability
problems. The post evaluation questionnaire was included to help gain an understanding of the
effort applied by each expert and also the possible problems and hindrances of the evaluation.
Details of the heuristic included in the evaluation section of the document and why can be found
below. The evaluation was comprised of the following heuristics which aim to discover
usability issue in line with the project question and aims:
Consistency
Compatibility
User resources
Feedback
Error prevention and recovery
User control
Visual clarity
Prioritisation of form and information
Explicitness
As well as the above usability goals additional question were included in the questionnaire to
help attain a more comprehensive overview of the possible usability problems. The questions
included were focused on the following heuristics;
Effectiveness
Efficiency
Satisfaction
Safety
Utility
Ease of use
Easy to remember
Additionally questions regarding affordance, constraints, mappings, physical and cognitive
characteristics and the use of a paper prototype were included in the questionnaire to help
obtain a more extensive overview of the opinions of the experts regarding the information
provided as part of the evaluation.
41. 4.3 Predicted Usability Problems
The expert analysing the interface were asked to give information regarding each usability issue
found during the heuristic evaluation stage using the following format.
A numeric Identifier of the problem
A short description of the problem
Likely difficulties for the user
Possible causes of the problem (What is wrong with the design)
The report format was used during Stage 1.2 above as part of the heuristic evaluation section.
4.4 Severity Rating
In this section of the usability testing severity rating questionnaires were devised to be sent to
each of the experts assessing the interface. The questionnaire asked each expert to rate the
severity of all of the usability issues found during the heuristic evaluation using the factor
described below:
Frequency with which the problem occurs: Is it common or rare?
The impact of the problem if it occurs: Will it be easy or difficult for the users to
overcome?
The persistence of the problem: Is it a one-time problem that users can overcome once
they know about it or will users repeatedly be bothered by the problem?
The severity rating questionnaire was comprised of 3 levels. Guidelines on how to use the
severity rating factor were also provided to the experts reviewing the interface design. Below is
a description of the 3 levels used:
Minor – Mild
Includes minor usability problems where users can easily work around the problem. Fixing
them should be given low priority.
Moderate – Medium
Includes medium usability problems where users stumble over the problem, but can quickly
adapt to it. Fixing them should be given medium priority.
Major – Severe
Includes catastrophic usability problems where users are unable to do their work and major
problems where users have difficulty, but are able to find workarounds. Fixing them is
mandatory.
42. 4.5 User Testing
The user testing stage was designed to incorporate the results from the heuristic evaluation. A
paper prototype was also included in the design of the user testing stage. Students from
Glasgow Caledonian University agreed to take part in the user testing stage.
User Testing Stages;
Pre Testing Questionnaire
User Test Tasks
Post Test Questionnaire – After Scenario Questionnaire
http://hcibib.org/perlman/question.cgi
4.6 Conclusion
As described in section 2.5.4 several factors, besides the set of usability heuristics can influence
the performance of heuristic evaluations, such as evaluator training, evaluator knowledge of the
application domain, task coverage, problem extraction/description, merging etc (Hvannberg,
Law, & Larusdottir, 2007). This resulted in a comprehensive document being drawn up as part
of the heuristic evaluation. One of the purposes of the document was training of the methods
explained in 4.2, 4.3 and 4.4. Additional information was also provided regarding the project
background, application domain and the aim of a heuristic evaluation in order to increase
performance levels and results of the evaluation.
Combining the PUP formatted answer with the evaluation was also done in an attempt to
increase validity of the evaluation results by increasing clarity of each problem discovered
during the evaluation. The addition of the severity rating questionnaire was added to further
increased the detail of the results by creating different levels of severity of each problem
discovered.
The user testing stage was implemented to help further clarify usability issues and to help find
potential solutions to any issues which were not specific to the intended user group.
43. 5. Design
5.1 Introduction
This section of the report provides an overview of the software system. The aim of this section
is to explain the functionality and technical details of the system. Initially a high level
description of the system is given at each stage of the design process. This is then followed by
lower level design details. The third section explains how the design ideas were implemented
into the project. The last section gives explanations regarding the implementation of all of the
features in the system and their relationship to the project aims.
5.2 High Level Design
5.2.1 Design Goals
Below is a list of design goals of the project. The design goals are based on the basic
functionality of the system. Information on what the system should do and how it should
behave has been included.
1. Develop a system which is usable and beneficial to stroke survivors with memory issues
2. Develop a usable product
3. Design a system which is easy to use
5.2.2 Usability Goals
1. System should be;
2. Effective to use
3. Efficient to use
4. Safe to use
5. Have good utility
6. Be easy to learn
7. Be easy to remember
In line with the usability goals described in section 5.2.1 the system should be helpful and fun
for the users to use. Additionally features of the system should motivate the user whilst
providing an emotionally fulfilling and cognitively stimulating experience for the user.
44. 5.2.3 Man Machine Model
The man machine model describes the process of using a machine. The model is referred to as a
closed loop feedback system. The man machine model comes from usability engineering. The
system is used to gain an understanding of system operation and highlights some of the
requirements of the user and the system (Dul, Weerdmeester, 2008).
Figure 5- 1 Man Machine Model
Man Processing – User requires ability to cognitively process information
Hand, Feet – User requires ability to control system using body
Controls – System requires physical controls e.g. mouse/keyboard
Operation – User must be able to use controls to operate system
Machine Processing – Machine must be able to process user input
Information Display – System should be able to display information of change created
Feedback – System should be able to feedback change to user via sound, light, etc.
Senses - Hearing detects change in guitar volume
45. 5.2.4 System Control
The software system requires the use of a mouse to control the different functions within the
game. The mouse will give the user the ability to move around the screen and select the
different elements of the system using the right click button.
5.2.5 System Architecture
Figure 5- 2 System Architecture
5.2.6 Software Architecture
Figure 5- 3 Software Architecture
Home Screen
Music Select Screen
Level Select Screen
Main Game Screen
5.2.7 Game Play Functionality
Music Selection Function
Level Selection Function
Complexity Selection Function
Game Start Function
Game Stop Function
Memory Game Function
46. 5.3 Introduction to Interface/Game
In order to help you understand the design implementation explained in section 5.5 this section
has been added to give an overview of the game concept as well as the features of the interface.
5.3.1 Memory Game Concept
The idea behind the music and memory game is simple. The user listens to music which plays at
the correct tempo and intensity as long as the user remembers and selects the correct elements
of a square grid. If the user selects the wrong element of the grid the music tempo and intensity
changes briefly to inform the user of the mistake. As well as this lights on the grid are displayed
to show correct and incorrect selections as the game progresses.
5.3.2 Memory Game Interface
Figure 5.4 below shows a first draft of the game interface and layout ideas.
Figure 5- 4 Memory Game Interface
47. 5.3.3 Typical User Scenario
User in the comfort of their own home opens their laptop and starts the game software
which is controlled using a mouse. The size of the game window is decided by the size of
the user’s laptop.
User first arrives at the game home screen where they are greeted by a short welcome
message whilst the game loads. The user is then prompted to click ‘Next’ to begin
playing the game.
User is prompted to select preferred music on second screen
User is prompted to select level of difficulty on third screen. The user’s choice at this
point affects the number of rows the grid contains horizontally. Easy = 3*6 grid,
Medium = 3*7grid and Hard = 3*8 grid.
User arrives at screen shown in figure 1.7 above
Learn light blinks slowly until user clicks learn which reveals the pattern of lights on the
grid to be remembered by the user.
Next the difficulty lights flash which allow the user to change the complexity of the
pattern. The user then increases or decreases the complexity of the pattern by selecting
1, 2 or 3 from the difficulty section. Less complex patterns are chunked together or
repeat in a manner which is easier for the user to remember. More complex patters are
more randomly placed with no chunking of information.
Finally the play button on the transport bar flashes informing the user to begin the game
when ready. At this point the patter shown to the user disappears, the timeline bar
begins to move from left to right across the grid and the music begins to play.
