The document provides an overview of outcome measures and assessments in occupational therapy and physiotherapy, emphasizing the importance of standardized tests and the various types of data collection methods. It discusses the relationship between assessment, evaluation, and outcome measurement, highlighting the need for evidence-based practice and the critical evaluation of measurement tools for effectiveness, reliability, and validity. Additionally, it outlines goal-setting techniques and the necessity for clear communication and consistency among healthcare professionals in monitoring client progress.

1. OUTCOMES
Outcome measures - what, why, how
A brief overview by Will Wade
(with particular thanks to Gary Derwent and Alison Laver Fawcett’s Book: Principles of Assessment and
Outcome Measurement for Occupational Therapists and Physiotherapists: Theory, Skills and Application)

2. • 1. Introduction - What are you
measuring? • Paediatric tools
• How outcomes fit with • Places to find more
assessment
• 3. Grouping data for service
• Standardised tests - why evaluation
• Standardised tests - type. Validity, • Criterion versus Norm
Reliability & Responsiveness referenced
• Assessing an assessment • Goal setting
• 2.Some measures to be aware of • 4. Improving and developing
• Models • 5. AT & AAC specifically
• MOHO & COPM Specific • 6, Appendix
assessments

3. LETS BE CLEAR
There are no clear answers! There
is no one checkbox fits all solution!
(sorry!)

4. PART 1. WHAT ARE YOU
MEASURING?

5. ARE YOU WANTING TO?
• Analyseperformance of an individual? (e.g. Given one or more
interventions)
• Analyse the performance of your team/service? (e.g. How effective
is the handwriting service?)
• Be careful with terms: Outcomes, Assessment, Audit

6. SOME TERMS
• Assessment is the broad holistic analysis looking at a range of
types of data Evaluation prompts a need for greater specificity
and so narrows the judgement. Outcome measurement
becomes the need for standardized measure
• Outcomes. The visible result. a measurable end result or
consequence of a specified action or essential step
“outcomes should relate closely to the clients social, psychological,
emotional & cultural needs in relation to occupational
performance” (COT, 2003a 25) “The desired outcome could be
improved occupational performance, function or successful
adaption” (Henderson 1991, 13)

7. The relationship between assessment, evaluation and outcome measurement (p13 Fawcett)
Evaluation to
make a judgement
about amount or
value
Outcome
measurement
should be at the
heart of the
assessment process
The Overal assessment
process, encompassing all
data ollection methods
Assessment has numerous purposes:
Descriptive - describing current status
Discriminative - useful as screening tools. Used when no
criterion exists. (Parent reported DCDQ has strong relationships
to therapy carried out MABC & AMPS (Green 2005))
Predictive - predicting a future outcome. E.g. Kitchen Ax to see
if someone safe to be discharged
Evaluative - detect change in function over time. E.g TOMs

8. ASSESSMENT DATA
COLLECTION
• Chia (1996) demonstrated that Paed OTs used a variety of methods
to collect data:
• Interviewing (n 49) 98%
• Standardized (n 40) 80%
• Structured Obs (n 36) 72%
• Non-standardised (n 35) 70%
• Unstructured obs (n 29) 58%

9. OUTCOMES & EBP - THE WHY
• “Now critical appraisal, reflective practice, systematic audit, peer
review, best value review, service evaluation, clinical governance
and a host of other methodologies are accepted parts of the
professionals landscape. The need to deliver evidence-based
practice is well understood and all professionals have to play their
part in the total quality management of service
delivery” (COT, Richards, 2002, p. xvii)

10. WHAT ARE YOU COLLECTING
THE DATA FOR
• Effectiveness: Whether treatments do more good than harm in those
whom they are offered under the usual conditions of care, which may differ
from those in the experimental situation. The measure of ability of a
programme , project or task to produce a desired result that can be
measured.
• Efficiency : Measure of production or productivity relative to input
resources. Operating a programme or project economically. Relates to
resources expended or saved - not effectiveness
• Efficacy: Assessing whether a treatment actually works for those who
receive it under ideal conditions. The degree to which a therapeutic outcome
is achieved in a patient population under rigorously controlled and monitored
circumstances such as RCTs' (Maniadakis & Gray 2004 p27)

11. A REMINDER:
STANDARDISED V’S NOT
• Standardisation: made standard or uniform; to be used without
variation; suggests an invariable way in which a test is to be used
as well as denoting the extent to which the results of the test may
be considered to be both valid and reliable (Hopkins & Smith
1993b, 914) (AOTA)
• If
you use non-standardised or adapted tests its not a crime but
do be aware of the limitations.

12. SOME BENEFITS OF
STANDARDISED TESTS
• Health care policy level - EBP • Larger research - clinic based
demands accountability and quality. research & multicentre trials would
Funding for services is coming be easier as you could combine
increasingly linked to effectiveness small samples
& efficacy.
• Therapist level - improve
• Perception of you as professionals communication (Lewis & Bottomley
improves - and your own 1994)
confidence!
• Client level - receives an improved
• Research theory/Practice gap service in which assessment and
outcome data are based on reliable
valid & sensitive measures

13. REFERENCED TESTS
• Criterion referenced tests examine performance against pre-
defined criteria and produces raw scores that have direct
meaning. Not compared to ability of others. E.g. AMPS. A standard
being “making a jam sandwich”
• Norms referenced tests involve data from a norming study.
Raw scores need to be referenced against the norms to mean
something.

14. VALIDITY
• How well does the test do what it says it does?
• Face validity. What it seems to measure - to a the test-taker, tester,
• Content validity. Does what it says judged on appropriateness of its content - should have
all variables associated with the thing being measured.
• Criterion validity. Effectiveness of a test to measure the performance - measuring
somebody's performance with a independent criterion. Concurrent validity compares it with
variables in other tests and Predictive validity is the accuracy that a test predicts some future event
(e.g. Mortality).
• Construct validity. The ability to perform as hypothesised in some facet. e.g. Those
discharged to an independent living situation should score higher on a self-care ax than those
discharged to a long-term care living situation. Discriminative validity - does it measure between
groups or groups and Factorial validity relates to factor analysis.

15. RELIABILITY
• In retesting of an individual are variations in results due to the intervention - thing being
measured - or some other facet? If therapist b carries out the test the results should be v. similar.
• Look for the reliability coefficient or correlation coefficient. The relationship between two sets of
data being compared. 0 (no correlation) - 1 (perfect). (i.e. Closer to 1 is good)
• Level of significance. How much chance influences results. The lower the level of significance
reported the greater the confidence (i.e. Closer to 0 is good)
• Standard error & confidence interval. How certain are we that the stats are correct and the test
is accurate? Confidence level usually 95% - and states that “You can be 95% confident the true
score lies within the interval range of scores” On a standard bell curve 95% of scores lie within
1.96 s.d of the mean. Rasch analysis can be used.
• Correlations from 0 to +/- .5 are low. +/-.5 to .70 are moderate. +/-.70 to 0.80 are high..

16. TYPES OF RELIABILITY
• Test-retest. Correlation of scores by the same person between two points
• Inter-reliability. Scores between different therapists
• Intra-reliability. Scores between the same therapist over time.
• Rater-severity. How lenient or stringent a therapist is on judging performance
• Parallel form reliability / Alternate form. Correlation of scores between two points but different
test. Used when learning effect may cause issues
• Internal consistency. Degree to which items measure the same construct. Helpful to have related
observations.
• (Responsiveness. The efficiency with which a test detects clinical change. A likelihood ratio
0-1)(NB: not reliability)

17. ASSESSING A TEST
• Clinical
utility. The overall usefulness of an ax in a clinical situation.
Reasonable cost, time, energy & effort, portability, acceptability
• Remember why you are carrying it out. Individual development or
service development? If it is for the latter make sure it does get
used or your team won’t bother with it!
• Lookat & Use the worksheets in Fawcett 2007. Will help you
understand a test’s clinical utility, validity and reliability for your
population

18. Name of test:
Full reference:
Is there a test manual? YES NO
YES NO
Does the test manual describe the test development process?
developed? YES NO
the client group for whom the test has been
Does the test manual describe the purpose of the
test and
YES NO
studiesundertaken establishreliability and validity?
to
Does the test manual provide details ofpsychometric
of the test package? YES NO
Does the test manual describe the materials needed
for test administration or are these included as part
be used for testing? YES NO
Docs the test manual describe thc environment that should
YES NO
instnlctions required to administer the test?
ls there a protocol for test administration that provides all the
Is there guidance on how to score each test item? YES N0
Is therea scoringform for recordingscores? YES NO
Is there guidance forinterpretingscores? YES NO
Ifit is a norm-referencedtest, is the normative sample well described? YES NO NOT APPLICABLE
of scores obtained by the nonnative group? YES NO
Are there norm-tables lrom which you can compare a client’s score with the distribution
/» nf/1_ ciii lfzf Outcome Mea.rure/ne/ztfbr04-cupalio/za/ l1empi.v1.s°and
I T Theory,Skills and Application by Alison Laver Fawcett © 2007

19. http://simmonsatshowcase.wikispaces.com/Universal+Design+for+Learning+-+A+Partnership+Model
PART 2. SOME MEASURES TO
BE AWARE OF

20. MODELS
• MOHO, CMOP
• ICIDH (Intl classification of Impairments, Disabilities and
Handicaps), ICF, NCMRR (National Centre for Medical
Rehabilitation Research)
•A model can help conceptualise your thinking and categorise your
terminology. Helps to articulate your reasoning to others. Within
an MDT can be useful to define your thinking and see where
overlapping areas are/missing areas of data collection.

21. MOHO
• SCOPE - Short child Occ
profile. Overview of
participation. Based on
MOHOST. Simple and Occ
focused.
• COSA - Self-assessment
tool.
• OTPAL - OT Psychosocial Assessment of learning.
6-12 yrs. Observational and descriptive ax tool. • PVQ - Observational
Volition, habituation and enviro fit within
classroom. Volition: sim to PVQ items and
assessment tool examining a
Habituation sim to SCOPE items child’s motivation.
• School Setting Interview - assist intervention
planning. Focused on the student role.

22. COPM
• Measures changes in client’s perceptions of their performance &
their level of satisfaction with their performance of self-care, work
and leisure tasks.
• Originaltest-retest reliability was done in 27 older adults with a
range of impairments (stroke, parkinsons, nof). Reliability was low.
Another study looked at COPD and was high. Another reliability
study looked at stroke. Time though was over 7 days. ? Reliable? In
general yes - but do think about the client group and be aware of
what hasn’t been proven with statistics.

23. PATIENT RECORDED
OUTCOME MEASURES
• “Patients' experience of treatment and care is a
major indicator of quality and there has been a • Population-specific – eg. Child Health and Illness
huge expansion in the development and application Profile-Child Edition/CHIP-CE
of questionnaires, interview schedules and rating
scales that measure states of health and illness from • Dimension-specific - eg: Beck Depression Inventory
the patient’s perspective. Patient-reported outcome
• Generic - eg: SF-36
measures (PROMs) provide a means of gaining an
insight into the way patients perceive their health • Individualised - eg: Patient Generated Index
and the impact that treatments or adjustments to
lifestyle have on their quality of life” • Summary items - eg: UK General Household Survey
questions bout long-standing illness
• Many OT based measures are already PROMs - e.g.
CMOP, SCOPE • Utility measures - eg: EuroQol, EQ-5D
• Disease-specific - eg: Asthma Quality of Life http://phi.uhce.ox.ac.uk/home.php
Questionnaire

24. MORE SPECIFIC TOOLS
• Paediatrictherapy
attracts assessment
tools!
• Occupational Therapy
Assessment Tools: An
Annotated Index 3rd
Edition
• Pediatric
Occupational
Therapy Handbook: A
Guide to Diagnoses
and Evidence-Based
Interventions.

25. http://education.ed.pacificu.edu/cascade/workshop/brainbased.html
PART 3. SERVICE EVALUATION

26. GROUPING YOUR DATA: HOW
DO YOU COLLECT YOUR
DATA CURRENTLY?
• Can you summarise goals and outcomes of these goals easily?
• Are you trying to evaluate a specific part of the process - for
example a group or specific intervention? Is there a measure you
are using already that you can summarise? Is it safe to?!
• Ifyou the answer is no - then your challenge is to find ways to do
this!

27. TOMS &
AUSTOMS
Therapistidentity/code :
Patient/client
Identity: IAMES BOND 007
(Name or Code Number)
EmployingAuthority: ANYWHERE PCT
Locality: PT outpatientjclientclinic
Profession: Speech and Language Therapy, hysiotherapy, ccupational Therapy,
Rehabilitation Nursing, Hearing Therapists
Patient/client/Client
Details
Age at Entg
Date ofBirth : 1959 Carer: SPOUSE (person rated)
dd mm yyyy
Aetiology(Lode l : M 25 AetiologyCode2: R 62.0
• Therapy Outcome Measure’s.
Enderby, John & Petheram 2006.
Impairment Code 1 : M 62.9 TOM Rating Sheet : Musculo-skeletal
Impairment Code 2 : R 52 TOM Rating Sheet : Core Scale used to rate Pain
Ratings
Uses ICF/ICD-10 codes.
Code* Impairment Activity Social Well-being Date Rated
Codel Code Participation Patient/ client
Carer
•
’ '04
Quantitative. Shows data at
°“’ Admission Intermediate
(numerous) and Final
(discharge).
A=Admissionto therapy,First rating:I = Intermediate ratings(when placed at thefirst
entry it denotes previous interventions fromtherapy);F = Final rating.
Number ofContacts: 21 Totaltime : 12 hrs 30 mins Discharge Code: • Easy and quick. Adapted scales
Use R0 if
analysing rating but
case is not discharged
broad ranging. ? Limiting perhaps
Comments:
Please send this formto your keyworker
for checking and dataentry.

28. GOAL SETTING
Goal-setting is the identification of, and agreement on, a
behavioural target which the patient, therapist or team
will work towards over a specified period of time.
(Royal College of Physicians, 2008)
• Holliday et al (2005) found that 30% of 202 respondents in a survey of goal setting
methods used goals as an outcome measure. Wilson (2003) ‘one of the main outcome
measures in our program is the percentage of goals achieved’.
• In one way or another we all set goals, aims and objectives
• Be consistent and careful with your language across the team - make sure you are all on
the same page!
• Make sure SMART goals are being followed
• Make sure they are about the client
• “Standardise” your goal setting across the team. Be aware of the dangers - particularly
when using tools to generalise. E.g. Making them very attainable

29. (MY!) GOAL SETTING LINGO
• Goal: Theoverall, usually one, goal that the client wants to achieve.
Usually long-term. Could be that your service is only playing what
may seem a very minor part of this process
• Aims.(Short-term goals or specific objectives). The individual,
often medium-term aims needed to reach the bigger goal. Aims -
need to be SMART. State the activity the person will perform,
under specified conditions and to a particular degree of success. A
progressive series contribute to the goal.
• Objectives
- the actual on the ground activities that need to be
met to meet each of these aims. Can be overlapping.

30. Prioritised Problem List: VM Skills leading to difficulty with classroom activities, Poor motor control, Unable to form letters
Goal: The overarching end For James to be NB: Indepdence should
result. Its your job to help independent within encapsulate a lot. That means
make this realistic and class without the to do something effectively,
achievable through aim assistance of efficiently and in a timely
setting. "The direction of an learning support staff manner.
action or actions"
For James to be
able to For James to
recognise and For James to correctly form
Aims: The steps to reach have developed letters to enable
manipulate
the goal. "The desired a functional maximum
shapes
outcome or specific handwriting/ occupational
involving
result" Client-centred drawing pencil performance
oblique lines –
essential for grip. within class
optimal (written work).
productivity.
What are the objectives - 1. Using a pencil and 1. To have tried one 1. To correctly form 11
the methods on reaching paper, reproduce two- demonstrated strategy of lower case letters of the
those aims? "Definable lined shapes at 90˚ by the pencil grip with two weeks alphabet with verbal prompts
actions undertaken to end of one week, e.g. of intervention. by the end of the fourth week
achieve a specific result" 2. To accurately 2. To demonstrate one of intervention (c, d, g, o, a, q,
reproduce shapes strategy by the fourth week j, u, e, l, m)
involving simple oblique of intervention with 2. To correctly form a
lines with straws after 4 prompting. different 12 lower case letters
weeks with support, e.g. 3. To demonstrate one of the alphabet with verbal
3. To reproduce three strategy of pencil grip by prompts by the end of the
out of four of the following the end of eight weeks with seventh week of intervention
designs with no support no prompting. (s, x, z, f, p, r, m, n, h, i).
after eight weeks. 3. To correctly form 23 out
of 26 lower case letters of the
alphabet with no verbal
prompts by the end of eight

31. UNIVERSAL SERVICES
• Goalsoften get messy in universal services. What the child wants
may be very far from what you can achieve in your service but do
consider how you can impact into their overall goal - even if it
does seem so far away.
• Make sure they are easy to communicate with a wider group -
plain english. Use standardised measures to back up reasoning.
• Commissioning is moving towards integrated care pathways - not
away. A need to prove the efficacy of your work within it.

32. TOOLS TO COLLECT GOALS
• EKOS - East Kent Outcome System/Sheets (Johnson 1997)
• GAS - Goal Attainment Scaling (Ottenbacher & Cusick 1990)
• Westcotes Individualised Outcome measure (Eames et al 1991)
• (IPPA: Individualised
Priortised Problem Assessment (Wessels,
Persson, Lorentsen, et al 2002) - AT focused)

33. Assessment
Client needs group:
Aetiology & communication
disorder EKOS
Health benefit
Expected outcome
State aim(s) and
• East Kent Outcome Sheets/System
Information for objective(s) of
Managers/ intervention within
Commisoners agreed timescale
• Common across many SLT services
Record baseline
measurements
• “Liked by commissioners”. Simple,
Treatment Plan meaningful easy data collection
Delivery style
No./frequency of contacts method.
Interventions/Small-step
programme
• “a good outcome is one where the
OUTCOME
Have objectives been met? aims of therapy have been
Fully
Mostly
achieved”
Partially
Not

34. EKOS
• Limitationsif not all aspects filled in. Buckles, L 2003 found that
following areas were missing from notes (n=159) timescale of
intervention (18.4%), type of intervention (21%), outcome (5.3%),
reasons for outcome (38.5%), and Health Benefit (38.9%).
• No method of recording client involvement in plans e.g. A
signature to agree, area for client & carer satisfaction.. But you can
modify it.

35. GAS
With GAS you effectively set 5 goals (“states”) for each goal -
defining what will be called “best outcome and worst outcome”
+2
Best expected outcome, Much better than expected
+1
More than expected outcome, Somewhat better than expected
0
Expected outcome
-1
Less than expected outcome, Somewhat worse than expected
-2
Worst expected outcome, Much worse than expected

36. GAS - EXAMPLE
+2
Transfer to toilet with raised toilet seat
independently on home visit.
+1
Transfer to toilet with raised toilet seat twice daily
independently.
0
Transfer to toilet commode with raised toilet seat
supervised twice daily.
-1
Transfer to toilet commode with raised toilet seat with
prompts (50% of time).
-2
Transfer to toilet commode with assistance x 1.

37. GAS - WEIGHTING
• Each goal is weighted for two factors…
• Importance (weighted by client / patient)
• Difficulty (weighted by clinical team)
• Weightings are 0-3
• 0 = not at all important / difficult
• 3 = very important / difficult
• Giving 0 for either weighting effectively rules that goal out of the overall calculation
• If weighting are not required, all weightings are set to 1

38. GAS - SCORING
x = raw score
The ‘raw score’ of each goal
is combined with its
weightings. w = weight (usually importance
x difficulty)
The group of raw scores and
weightings are converted .7 and .3 signify the expected
into a standard normal score correlation of the goal scales
with a mean of 50 and SD of
10 (Known as T-Score)

39. GAS - ISSUES
• Time consuming - effectively 5 goals
• Maths and stats controversial
• Writing an effective goal difficult (Problem for all goal-setting, but magnified for
GAS with 5 statements per goal)
• Many different procedures used
• Different factors altered in each statement
• Where to place baseline ?
• Short or long term goals ?
Derwent G, 2010, Communication Matters Symposium

40. TACO (ET
AL) GOALS
Defines structured approach
to writing a goal. Not
necessarily for GAS
• 1. Defining the goal
• 2. Weighting the goal
• 3. Scaling the goal
• 4. Evaluating the goal
• 5. Scoring
Bovend’Eerdt, T., Botell, R. & Wade, D. (2009). Writing SMART
rehabilitation goals and achieving goal attainment scaling: a
practical guide. Clinical Rehabilitation. 23. 352-361.

41. Defining the goal - useful guidance for any goal system
• 1. Target activity. Be precise. E.g. Walking indoors (not mobilising), cooking a 3-
course meal (not preparing food)
• 2. Specify specific support.
• 2.1 Support by other people. Hands-on, emotional, cognitive
• 2.2 Objects. E.g. Aids -wheelchair, cutlery adaptions, ramps etc
• 2.3 Items in the environment e.g. Sign-posting for orientation, using barriers
to remind someone not to go somewhere, lists to prompt
• 3. Quantify Performance. Performance can be quantified in 3 ways; by the time
taken to achieve a set quantity of the activity, by the quantity of a continuous
activity performed (e.g. Distance) in a set time and/or by the quantity of a
discrete activity occurring in a period of time (e.g. Frequency)
• 4. Specify time period to achieve the desired state

42. COMMISSIONERS
• What goal outcome
based system do they
listen to? It varies.
• GoalWriter -
goalassist.org.uk hopes to
help this by creating a
system to interchange
these outcomes
• Gary Derwent @ RHN

43. TOOLS TO EVALUATE CLIENT
SATISFACTION WITH SERVICES
• CSQ - Client Satisfaction Questionnaire (CSQ-8) (Attkisson CC
1987) not necessarily a measure of a client's perceptions of gain
from treatment or outcome, but does elicit the client's
perspective on the value of services received
• Experience of Service Questionnaire (ESQ) (CAMHS)
• Feedback forms & Self-made questionnaires. Think who your
audience is and what you need to find out. Keep it minimal.
Follow-up.

44. http://www.jmorganmarketing.com/the-social-crm-process/
PART 4. IMPROVING AND
DEVELOPMENT

45. 1. Analyse your currentassessment and
measurementprocesses
't' ' '
. . 2. C
-mpfovemem
plan for future modernisation ___
have everrebre
7. Evaluate whether the change `
has led to the desired
and whether there 3_ rderrrrrvareas your
HVB een alll' assessment and measurement
unforeseen undesirable practice that could benefit from
consequences ofthe change improvement
clear goals for improvement
4. Set .
and think how you could measure
5_|mp|emer-rr
whether this improvement
Changes
has been achieved
5. identify changes required to
achieve the desired improvement
Figure ll.l Process forimprovingyour assessment and measurement process.
• From Fawcett, 2007, p

46. http://www.ogilviedesign.co.uk
PART 5. AAC/AT SPECIFICALLY

47. ASSISTIVE TECHNOLOGY
• "Assistive technology is any item, piece of equipment or product system whether
acquired commercially off the shelf, modified, or customized that is used to increase,
maintain or improve functional capabilities of individuals with disabilities” (Public Law
(PL). 100­407. The technical assistance to the States Act in the US.)
• AT is wide-ranging. Low-tech e.g. Pen, paper, chart, pencil grip, Medium-tech: simple
mechanical operations, High-technology: electronic components/controlled by a
computer.
• What is your view of AT? How does this differ with the outcome measure authors?
• Cook & Hussey define 4 areas: (a) augmentative and alternative communication; (b)
technology that enables mobility; (c) technology that aids manipulation and control of
the environment; and (d) sensory aids.

48. A QUICK NOTE ON
CONCEPTUAL MODELS IN AT
• All the others: ICF, MOHO, CMOP etc.. ICF in particular.
• Human Activity Assistive Technology(HAAT) model . The HAAT is
adapted from the Human Performance Model and is described by
Lenker and Paquet'' as "thoroughly considering person and
environment factors, emphasizing the influence of environment
and culture on task performance."
• The Matching Person Technology (MPT) model

49. MPT
Primary Components
Language Representation Methods Vocabulary Methods of Utterance Generation
Single Meaning Pictures Core SNUG (spontaneous novel utterance
Alphabet-Based Methods Extended generation)
Semantic Compaction Pre-stored sentences
Sec0ndary Components
User Interface Control Interface – Outputs
Selection Methods
Symbols Direct Selection Speech
Navigation Keyboard, head pointing, eye gaze Display
Automaticity Scanning Electronic/Infrared/Radio Frequency
Human Factors Switches Data logging
Physiological (EMG, BCI, etc.)
Morse Code
Tertiary Components
Peripheral and Integrated Features Training and Support Telerehabilitation
Hill & Scherer, 2008; Hill, 2009 in press

50. IMPACT 2 MODEL

51. ASSESSING OUTCOME &
EFFICACY
• Assessing outcome = to demonstrate the efficacy of the application of new
technology, to establish the effectiveness of assistive technology over time or
to steer the development of new assistive technology.
• Efficacy of an AT device = the effect resulting from its use in comparison to
the effect claimed beforehand.
• The nature of this effect may depend on the user(-population), the device and
the use conditions.
• Effectiveness on societal level is often considered in relation to costs; i.e., cost-
effectiveness. The cost-effectiveness of the intervention is established by
weighing the benefits against the costs. (Gelderblom, Witte, 2002)

52. AT IS TRICKY!
AT is more complex than other areas to evaluate Outcome. This complexity originates from:
a) the diversity in contributing variables - impact is dependent on characteristics of
the end user, the context of its use and the type of AT. The functional effect of a wheelchair is
different to a AAC aid
b) outcome being a multidimensional concept - User satisfaction, functional
independence, societal and individual gains, increased social participation, enhanced normative
social roles, the promotion and sustaining of employment and facilitation of activities of daily
living may all add to the outcome of AT.
c) the embedding of an AT device - AT is rarely used in isolation. How is it being used
in different domains of care?
d) the goals to be reached with an assistive device can be diverse and highly
individual

53. OUTCOMES
• Establishing the effect of AT may
• MPT (Predisposition Assessment)
require more than one instrument
depending on the type of question • OT-FACT (Functional Performance)
underlying the assessment of
outcomes. E.g • SCAI (Costs)
• Quebec user evaluation of • Life-H (Social Participation)
satisfaction with assistive technology
(QUEST) • IPPA (Individual Goals)
• The Psychosocial impact of assistive • KWAZO (Quality of service)
devices scale (PIADS)
• EuroQol & PIRS (Quality of Life)
• Family Impact of AT (FIAT)

54. AAC
• Alternative Augmentative Communication
• AAC is a form of AT
• Some difficulties removed by focusing on the specific area of AT - communication.
• But - Communication itself is complex!
• Tools to measure language and cognitive performance - but most norm
referenced. E.g. PPVT-R, WISC-R. Individual Goals make the most sense for
individual/service outcomes.
• Reading homework : Schlosser, 2000

55. http://www.communicationmatrix.org/

56. AAC TOOLS
• COL Determines and defines the levels of
competence using Lights 4 levels of
compentency; Linguistic, Operational, Social,
Strategic.
• Five skill sets within each of these four areas are
identified. Each skill set represents an increment
of increased skill development toward mastery
of a competency area.
• Suggesting intervention and instruction to assist
in the development of communicative
competence using AAC systems
• Measuring progress. Builds on communication
matrix, GAS and good basis of AAC theory
• Kovach, 2009

57. INTERAACT FRAMEWORK
• The framework addresses communication ability levels
(Emergent, Context-Dependent, and Independent) and provides
the opportunity to transition dynamically through communication
ability levels as well as throughout the lifespan.
• Focus on interaction - functional communication
• Communicative competencies “The development of
communicative competence is essential to express needs and
wants, share information with others, and develop social closeness
with family and friends” (Light, Beukleman, Reichle, 2003)

58. F R A M E W O R K
Use this guide to indicate the individual’s stage in life and identify important characteristics about his/her communication. These characteristics can help guide the
individual to the most appropriate set of communication pages in their DynaVox device while maximizing interaction.
E M E R G E N T C O N T E X T - I N D E P E N D E N T
D E P E N D E N T
YOUNG
CHILD May be starting to follow directions within Understands general conversations and Understands communication the same as
2-6 routines and familiar activities. directions as well as same age peers. same-age peers.
May be communicating most successfully using Understands picture symbols that represent Able to talk about a broad range of age-
CHILD
7-13 facial expression, body language, gestures objects and common actions (e.g., run, appropriate topics in exible ways.
and/or behaviors (either socially appropriate paint, eat).
behaviors or challenging behaviors). Combines single words, spelling, phrases and
Starting to understand more abstract picture complete messages together to create novel
TEEN May have a few messages that (s)he symbols (e.g., think, big, hot, few). communication about a variety of subjects as
14 -21 communicates well and/or often using symbols would others of his/her age.
or any methods listed above. Uses symbols and objects spontaneously to
communicate basic needs and wants. Changes the way words and phrases are
YOUNG Frequency and reliability of both understanding combined based on the communication partner
ADULT and expression varies from day-to-day and/or Uses a combination of communication methods
(e.g., pictures, objects, pointing/gestures, speech and situation.
22-50 activity-to-activity.
vocalizations) to express messages. Literacy skills on par with same-age peers.
ADULT Pictures seem to increase both comprehension
and expression. Beginning to combine two or more symbols to Social interaction skills, environments, and
50+ create longer and/or more complex messages. activities are similar to others of his/her age.
Attempts to communicate are most frequent in
motivating situations or favorite activities. Communicates best in routines and regarding Participates in age-appropriate environments
familiar topics. and activities.
If using picture symbols, use one picture at a
time to communicate messages. Bene ts from help to initiate social interaction
Actively participates with communication partner
and/or take additional turns in conversation.
when communication breakdowns occur.
May be beginning to use clear and simple
symbols (including objects, photographs and Bene ts from help to participate in interactions in
picture symbols) in motivating situations and/or new environments and with new people.
favorite activities. May continue to bene t from the help of his/her
Bene ts from help from his/her communication communication partner to narrow down choices,
partner to communicate successfully navigate pages, interpret body language/gestures
(e.g., narrowing choices, page navigation, as these skills develop.
interpreting gestures/body language). Literacy skills developing (e.g. letter names and
May be starting to show interest in social sounds, site words, spelling of simple words).
interactions, especially in speci c situations.
Note: “ Children’s natural actions and behaviors are
the only prerequisites to AAC...Early behaviors
and skills facilitate the gradual development of
more complex communication skills, including
language” (Cress & Marvin, 2003).
We believe that this is true of individuals of all ages. Note: Because these characteristics cover a broad range Note: These individuals interact daily in all environments
As a result, the communication system should of skills, many AAC users fall into this category. in flexible ways at age level.
embrace growth and development.
150117
*Adapted from Patricia Dowden, Ph.D., CCC-SLP, University of Washington, Communicative Independence Model.
InterAACt framework Dynamic AAC Goals
http://www.dynavoxtech.com/training/toolkit/
details.aspx?id=32

59. BLACKSTONE’S SOCIAL
NETWORKS
• Social Networks, created by Sarah Blackstone and Mary-Hunt Berg (2003), provide a unique
way to explore communication partners and environments. A team of individuals, including AAC
users, identify individuals that fit into the following categories: life partners/family members, close
friends, acquaintances, paid workers and strangers.
• Intervention strategies can then be developed based on the Social Networks identified. For
example:
• Many AAC users have few people in the “close friends” circle. In such cases, the team can
work together to provide vocabulary and teach interaction skills that will help this circle to
grow.
• More and more AAC users are active in their communities. To develop or reclaim these
important social roles, individuals need to be able to interact with a variety of people. The
Social Networks program may identify “strangers” such as store clerks, bankers, or other
community workers that an AAC user wants to be able to talk to.

60. http://www.justkiddingcartoons.com/catalog/item/2190410/6718104.htm
APPENDIX

61. THANKS
• Alison
Laver Fawcett, “Principles of Assessment and Outcome
Measurement for Occupational Therapists and Physiotherapists:
Theory, Skills and Application”
• Gary Derwent, Royal Hospital for Neurodisability, Putney
• The ACE Centre, Oxford

62. FURTHER INFORMATION
• Email: will.wade AT nhs.net twitter: willwade
• Citeulike references:
http://citeulike.org/user/willwade/tag/outcomes

63. • A. L. Fawcett (2007). Principles of Assessment and Outcome • Johnson, M and Elias, A (2002) East Kent Outcome System for
Measurement for Occupational Therapists and Physiotherapists: Speech and Language Therapy East Kent Coastal Primary Care
Theory, Skills and Application. Wiley, 1 edn. Trust
• Bovend’Eerdt, T., Botell, R. & Wade, D. (2009). Writing SMART • Johnson, M (1997) Outcome Measurement: towards an
rehabilitation goals and achieving goal attainment scaling: a interdisciplinary approach. British Journal of Therapy and
practical guide. Clinical Rehabilitation. 23. 352-361. Rehabilitation, 4 (9) 472-478
• N. Patricia Bowyer EdD OTR/L BC & S. M. Cahill MAEA OTR/L • Miller, A (2000) Multidisciplinary outcome measurement: is it
(2008). Pediatric Occupational Therapy Handbook: A Guide to possible? British Journal of Therapy and Rehabilitation 7 (8)
Diagnoses and Evidence-Based Interventions. Mosby, 1 edn. 362-365
• Brock, K., Black, S., Cotton, S., Kennedy, G., Wilson S., & • Lowing, K., Bexelius, A., Carlberg, E., (2009) Activity focused
Sutton, E. (2009) Goal achievement in the six months after and goal-directed therapy for children with cerebral palsy – Do
inpatient rehabilitation for stroke. Disability and Rehabilitation. goals make a difference ? Disability and Rehabilitation. 31(22):
31(11), 880-886. 1808-1816.
• P. Enderby, et al. (2006). Therapy Outcome Measures for • McDougall, J., Wright, V. (2009) ICF-CY and Goal Attainment
Rehabilitation Professionals: Speech and Language Therapy, Scaling: Benefits of their combined use for pediatric practice.
Physiotherapy, Occupational Therapy. Wiley, second edn. Disability and Rehabilitation. 31(16): 1362-1372
• Playford, E.D., Siegert, R., Levack, W., Freeman, J. (2009) • C. Unsworth, et al. (2004). ‘Validity of the AusTOM scales: A
Areas of consensus and controversy about goal setting in comparison of the AusTOMs and EuroQol-5D’. Health and
rehabilitation: a conference report. Clinical Rehabilitation. 23, Quality of Life Outcomes 2(1).
334-344.
• Turner-Stokes, L. (2009) Goal attainment scaling (GAS) in
rehabilitation: a practical guide. Clinical Rehabilitation. 23.
•
• Ferguson, A., Worrall, L., Sherratt, S. (2009) The impact of
communication disability on interdisciplinary discussion in
rehabilitation case conferences. Disability and Rehabilitation. 31
(22): 1795-1807.

64. AAC/AT REFERENCES
• R. Andrich, et al. (1998). ‘A model of cost-outcome analysis for • H. Day, et al. (2002). ‘Development of a scale to measure the
assistive technology’. Disability & Rehabilitation 20(1):1–24. psychosocial impact of assistive devices: lessons learned and the
road ahead.’. Disabil Rehabil 24(1-3):31–37.
• Blackstone, S. & Hunt Berg, M. (2003). Social networks: A
communication inventory for individuals with severe • Hill, K., 2004. Augmentative and Alternative Communication and
communication challenges and their communication partners. Language: Evidence-Based Practice and Language Activity
Verona, WI: Attainment Company. Monitoring. Topics in Language Disorders, Vol. 24, No. 1, pp.
18-30.
• L. A. Cushman & M. J. Scherer (1996). ‘Measuring the relationship
of assistive technology use, functional status over time, and • Light, J. (1989). Toward a definition of communicative competence
consumer-therapist perceptions of ATs.’. Assistive technology: the for individuals using augmentative and alternative communication
official journal of RESNA 8(2):103. systems. Augmentative and Alternative Communication , 5 ,
137-144.
• L. Demers, et al. (1999). ‘An international content validation of the
Quebec User Evaluation of Satisfaction with assistive Technology • I. Schraner, et al. (2008). ‘Using the ICF in economic analyses of
(QUEST)’. OTI 6(3):159–175. Assistive Technology systems: Methodological implications of a
user standpoint’. Disability & Rehabilitation 30(12-13):916–926.
• M. J. Fuhrer (2001). ‘Assistive technology outcomes research:
challenges met and yet unmet.’. Am J Phys Med Rehabil 80(7): • R. Wessels, et al. (2002). ‘IPPA: Individually Prioritised Problem
528–535. Assessment.’. Technology & Disability 14(3):141–145.
• G. J. Gelderblom & L. P. de Witte (2002). ‘The Assessment of • R. Wesselsa, et al. (2000). ‘IPPA, a user-centred approach to
Assistive Technology Outcomes, Effects and Costs’. Technology and assess effectiveness of Assistive Technology provision’. Technology
Disability 14(3):91–94. and Disability 13(1):105–115.