What Mokken scaling is and how it can be useful

1. Clinical applications of
Mokken scaling
Roger Watson

2. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
The purpose of measurement in social
research
• The investigation of latent variables/groupings
• Range of methods
– Classical test theory
– Latent class analysis
– Item response theory

3. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Classical test theory
• Internal consistency
• Factor analysis
Problems with classical test theory:
• any combination of scores on any set of items can give the same score on the
latent trait
• therefore, you do not know what a score means

4. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Item response theory
• Rasch analysis
• Partial credit model
• Mokken scaling
Advantages of item response theory:
• only a specific set of items produces a given score on the latent variable
• therefore, you know what the score means

5. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
What a nurse must do questionnaire
1 = ‘never’; 7 = ‘always’
Mean Std. Deviation
A nurse must turn up for work on time 6.09 1.352
A nurse must be able to ask a patient difficult questions 5.34 1.692
A nurse needs to be able to make calculations 4.43 2.157
A nurse must be accountable for his or her actions 4.69 1.921
A nurse must be able to organise others to care for a patient 4.93 1.755
A nurse is expected to behave well 5.65 1.692
A nurse need to write lots of notes 3.72 1.922
A nurse must be able to plan care for a patient 5.04 1.883
A nurse must know how to make a bed 3.77 1.792
A nurse needs to know how to speak to patients 5.35 1.703
A nurse must know how to give injections 4.30 2.027
A nurse need to give medicines to patients 4.12 1.942
A nurse must always look neat and tidy 5.73 1.457
A nurse must be able to assess a patient 5.23 1.787
A nurse must attend regular meetings 4.34 2.280

6. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Structure Matrix
Component
1 2
A nurse is expected to behave well .871 -.084
A nurse must be able to plan care for a patient .841 -.099
A nurse needs to know how to speak to patients .803 -.054
A nurse must be able to ask a patient difficult questions .793 -.054
A nurse must be able to organise others to care for a patient .785 -.082
A nurse must be able to assess a patient .760 -.053
A nurse must always look neat and tidy .704 -.025
A nurse must turn up for work on time .695 -.040
A nurse must be accountable for his or her actions .520 -.123
A nurse must know how to give injections -.082 .801
A nurse need to write lots of notes -.216 .639
A nurse must know how to make a bed -.196 .635
A nurse needs to be able to make calculations .184 .632
A nurse must attend regular meetings .018 .624
A nurse need to give medicines to patients -.083 .599

7. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Descending means
Mean Std. Deviation
A nurse must turn up for work on time 6.09 1.352
A nurse must always look neat and tidy 5.73 1.457
A nurse is expected to behave well 5.65 1.692
A nurse needs to know how to speak to patients 5.35 1.703
A nurse must be able to ask a patient difficult questions 5.34 1.692
A nurse must be able to assess a patient 5.23 1.787
A nurse must be able to plan care for a patient 5.04 1.883
A nurse must be able to organise others to care for a patient 4.93 1.755
A nurse must be accountable for his or her actions 4.69 1.921
A nurse needs to be able to make calculations 4.43 2.157
A nurse must attend regular meetings 4.34 2.280
A nurse must know how to give injections 4.30 2.027
A nurse need to give medicines to patients 4.12 1.942
A nurse must know how to make a bed 3.77 1.792
A nurse need to write lots of notes 3.72 1.922

8. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Guttman scales
Louis Guttman 1916-1987

9. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Concept of a cumulative scale
• Items are ordered reproducibly
• Item are ordered meaningfully
• A score on an item indicates the extent to which the latent trait is present
• The sum of item scores is a measure for order of the latent trait

10. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications

11. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Cumulative scale: example
(Also known as an ‘implicational’ scale)
5) I would have no objections to my son or daughter marrying a
Scottish person
4) At a party I would not hesitate to dance with a Scottish person
3) I would have no objections to having a Scottish person dine in my
house
2) I would not object to having a Scottish family live next door
1) I would not object to sitting next to a Scottish person on a bus
DIFFICULTY

12. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Guttman item responses
P(θ)
θ
1 -
Item i Item j
Items and traits are
related by a dominance
model

13. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Item characteristic curve
P(θ)
θ
1 -
= latent variable

14. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Item characteristic curve
Item
score
Person’s score on the latent trait
3 -

15. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Item characteristic curves
P(θ)
θ
1 -
Item 1 Item 2
• item 2 is more
‘difficult’ than item 1
• it represents more of
the latent variable

16. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Robert Mokken 1929-
Mokken scaling
Mokken suggested a non-parametric item
response theory where characteristic
curves (ICCs) only had to be monotone
and non-intersecting

17. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Mokken Scaling
• Adheres to assumptions of item response theory
• Stochastic version of Guttman scaling
• Hierarchical cumulative scales

18. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
18
Assumptions of IRT
• Unidimensionality
• Local stochastic independence
• Monotone homogeneity
• Double monotonicity (non-intersection) for dichotomous items†
• Invariant item ordering for polytomous items‡
† eg “yes/no”
‡ eg “strongly agree/agree/disagree/..etc”

19. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Criteria for a valid Mokken scale
• Scalability (H>0.3) [a measure of the number of Guttman errors]
• Reliability (Rho>0.7) [single sample test-retest method]
• Probability (p<0.05) [Bonferroni corrected]
• Low violations of: monotone homogeneity [& double monotonicity]
• Invariant item ordering (HT>0.3) [a measure of the ‘distance’ between the items]
0.3 > H/HT < 0.4 = weak scale/IIO
0.4 > H/HT < 0.5 = moderate scale/IIO
0.5 > H/HT = strong scale/IIO

20. EdFED Scale
Supervision
Physical Help
Spillage
Leave food on plate
Refuse to eat
Turn head away
Refuse to open mouth
Spit out food
Leave mouth open
Refuse to swallow food

21. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 0
Never = 0
Sometimes = 1
Often = 2

22. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 1
Never = 0
Sometimes = 1
Often = 2

23. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 2
Never = 0
Sometimes = 1
Often = 2

24. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 3
Never = 0
Sometimes = 1
Often = 2

25. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 4
Never = 0
Sometimes = 1
Often = 2

26. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 5
Never = 0
Sometimes = 1
Often = 2

27. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 6
Never = 0
Sometimes = 1
Often = 2

28. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 7
Never = 0
Sometimes = 1
Often = 2

29. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 8
Never = 0
Sometimes = 1
Often = 2

30. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 9
Never = 0
Sometimes = 1
Often = 2

31. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 10
Never = 0
Sometimes = 1
Often = 2

32. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 11
Never = 0
Sometimes = 1
Often = 2

33. EdFED Scale
Does the patient ever:
Never Sometimes Often
1) Refuse to eat 
2) Refuse to open mouth 
3) Turn head away 
4) Spit out food 
5) Refuse to swallow food 
6) Leave mouth open 
Total score = 12
Never = 0
Sometimes = 1
Often = 2

43. GHQ-30 (Goldberg)
Measures psychological distress
Five dimensions:
•anxiety
•feelings of incompetence
•depression
•difficulty in coping
•social dysfunction.

44. GHQ-9 (Watson et al. 2008)
Feel that life isn’t worth living
Found at times you were unable to do anything because of
your nerves
Been thinking of yourself as a worthless person*
Been losing confidence in yourself*
Been feeling nervous and strung-up all the time
Felt you couldn’t overcome your difficulties*
Been taking things too hard
Felt constantly under strain*
Been (un)able to face up to problems*
Increasing
level of difficulty
* - items included in GHQ-12
HT=0.44
n=6317

45. VoiSS
(The Voice Symptom Scale)
(30 items)

46. VoiSS-17 (Deary et al. 2010)
Do you feel lonely because of your voice problem?
Are you ashamed of your voice problem?
Do people seem irritated by your voice?
Does your voice problem put a strain on your family and friends?
Does your voice make you feel incompetent?
Does your voice problem make you feel stressed and nervous?
Do you have difficulty attracting attention?
Do you feel miserable or depressed because of your voice problem?
Are you embarrassed by your voice problem?
Does your voice ‘give out’ in the middle of speaking?
When talking in company do people fail to hear you?
Do you find the effort of speaking tiring?
Do you have problems talking on the telephone?
Do you have a weak voice?
Does your voice feel creaky and dry?
Do you have difficulty competing against background noise?
Is your voice hoarse?
Increasing
level of difficulty
HT=0.37
n=480

47. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Since 2010
• New graphical techniques to inspect Mokken scaled items
• Confidence intervals of scale coefficients can be calculated
• Sample size can now be estimated

49. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications

50. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications

51. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
51
Polytomous item
response functions
(IRFs) of the Hostility
(HOS) items
HT = 0.67

52. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
52
Polytomous item
response
functions (IRFs)
of the Depression
(DEP) items
HT = 0.47

53. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
53
Polytomous item
response functions
(IRFs) of the
Physical
Functioning (PF)
items
HT = 0.53

54. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
95% CI for H ij should not
include 0
95% CI for Hi should not
include 0.30

55. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications

56. 04/10/2015 © The University of Sheffield / Department of Marketing and Communications
Conclusion
Is Mokken scaling useful in clinical practice?
• Mokken scaling is becoming a very sophisticated psychometric method
• It is useful where relative rather than precise measurement is required
• It brings added meaning to total scale scores
Future developments
• Empirical studies of sample size are required
• Longitudinal studies are required

57. r.watson@hull.ac.uk
0000-0001-8040-7625
@rwatson1955
@jadvnursing
@nursingopen