Clinical Accuracy of Cancer Clinicians
Clinical Accuracy of Cancer Clinicians
Ability of health professionals to identify mood disorders
Ability of health professionals to identify mood disorders
Alex Mitchell www.psycho-oncology.info
Department of Cancer & Molecular Medicine, Leicester Royal Infirmary
Department of Liaison Psychiatry, Leicester General Hospital
LOROS August 2009
LOROS August 2009

1. Background
What methods are used to detect mood disorders?
How often do clinicians look for mood complications?

Methods to Evaluate Depression
Conventional Scales
Short (5-10) Long (10+)

Comment: This is a reminder of the
structure of the HADS scale, this version
adapter for cancer.

Methods to Evaluate Depression
Conventional Scales
Short (5-10) Long (10+)
Ultra-Short (<5)

Methods to Evaluate Depression
Unassisted Clinician Conventional Scales
Untrained Trained Ultra-Short (<5) Short (5-10) Long (10+)
Acceptability ?
Accuracy? Accuracy?
Routine Implementation
vs Comment: schematic overview of
methods to evaluate depression

Comment: Frequency of cancer specialists
n=226 enquiry about depression/distress from
Mitchell et al (2008)

Cancer Staff Psychiatrists
Current Method (n=226)
Other/Uncertain
9% Other/Uncertain
ICD10/DSMIV 2%
0% ICD10/DSMIV
13%
Short QQ
3%
1,2 or 3 Sim ple
QQ
15%
Clinical Skills
Use a QQ Alone
15% 55%
Clinical Skills
Alone
73% 1,2 or 3 Sim ple
QQ
15%
Comment: Current preferred method of eliciting
symptoms of distress/depression

Cancer Staff Psychiatrists
Ideal Method (n=226)
Effective?
Long QQ
8%
Clinical Skills Clinical Skills
Alone Alone
Algorithm 20%
17%
26%
ICD10/DSMIV
24%
ICD10/DSMIV 1,2 or 3 Sim ple
0% 1,2 or 3 Sim ple QQ
QQ 24%
Short QQ 34%
23%
Short QQ
24%
Comment: “Ideal” method of eliciting
symptoms of distress/depression according
to clinician

2. Primary Care - Meta-Analysis
How well do GPs (PCPs) identify depression? (clinical sensitivity)
How well do GPs (PCPs) identify the non-depressed? (clinical specificity)
How important is severity of depression/distress?

Summary
50 371 patients
9 countries
N= 108 studies
N= 41 depression studies
N= 19 depression with specificity
Predictors Examined
Severity
Age
Prevalence
Type of assessment
Duration of assessment

Comment: HSROC Curve plot for all
depression detection studies from primary care

1
Post-test Probability
0.9 Comment: Slide illustrates Bayesian
curve – pre-test post test probability for
every possible prevalence
0.8
0.7
0.6
0.5
0.4
0.3 Baseline Probability
Depression+
0.2
Depression-
0.1
Pre-test Probability
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1
Post-test Probability
0.9 Comment: At a prevalence of 20% GPs
PPV is 40% and NPV 86%
0.8
0.7
0.6
0.5
PPV
0.4
0.3 Baseline Probability
Depression+
0.2
NPV
Depression-
0.1
Pre-test Probability
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Depression vs Distress
Comment: Slide illustrates two HsROC curves, one for depression and one for distress, both from primary care. The following bayesian graph
compares the two more clearly=>

GP Accuracy by Severity
1.00
Post-test Probability
0.90 Comment: Slide illustrates GP diagnosis
of mod-severe depression is more
successful than their diagnosis of
0.80 “distress” or mild depression
0.70
0.60
0.50 Non-Mild Depression+
Non-Mild Depression-
0.40
Baseline Probability
Mild Depression+
0.30
Mild Depression-
Distress+
0.20
Distress-
0.10
Pre-test Probability
0.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

GP Accuracy – Detection of Distress by GHQ Score
McCall et al (2007) Primary Care Psychiatry - Recognition by Severity
90
80
70
Comment: Slide illustrates raw number
60 of people identified by severity on the
GHQ. Although the % detection
increases with severity, the absolute
50 number decreased due to falling
prevalence
40
30
20
10
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

3. Cancer Care - Meta-Analysis
How well do cancer specialists identify depression?
How do doctors compare with nurses?

Testing Clinicians: A Meta-Analysis
Methods (currently unpublished)
12 studies reported in 7 publications.
2 studies examined detection of anxiety,
8 broadly defined depression (includes HADS-T)
3 strictly defined depression and 7 broadly defined distress.
9 studies involved medical staff and 2 studies nursing staff.
Gold standard tools including GHQ60, GHQ12 HADS-T, HADS-D,
Zung and SCID.
The total sample size was 4786 (median 171).

Testing Clinicians: A Meta-Analysis
Results
All cancer professionals
SE =39.5% and SP =77.3%.
Oncologists
SE =38.1% and SP = 78.6%; a fraction correct of 65.4%.
By comparison nurses
SE = 73% and SP = 55.4%; FC = of 60.0%.
When attempting to detect anxiety oncologists managed
SE = 35.7%, SP = 89.0%, FC 81.3%.
Presented at IPOS2009

GPs vs Oncologists vs Nurses
Who is better?
Bayesian analysis

1.00
Post-test Probability
GP+
GP-
0.90 Baseline Probability
Nurse+
Nurse-
0.80 Oncologist+
Oncologists-
0.70
0.60
0.50
0.40
0.30 Comment: Doctors appear to be more
successful at ruling-in or giving a
diagnosis, nurses more successful at
0.20 ruling out
0.10
Pre-test Probability
0.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

4. Cancer Care – Screening Data
What resources are available locally re identifcation
How do nurse specialists identify depression
vs distress
vs anxiety
vs anger
How much difference does a screening tool make?

Testing Clinicians vs DT
114 ratings from clinical nurse specialists (CNS).
81 individuals (71%) scored above a cut-off of 3 (mild distress)
64 patients (56%) scored above a cut-off of 4 (moderate distress)
37 (32.4%) individuals scores above 5 (severe distress)

1.00
Post-test Probability
0.90 Comment: Phase I Data appears to show
less success in detecting severe distress
0.80
0.70
0.60
0.50
0.40
Severe Distress CNS+
0.30
Severe Distress CNS-
Baseline Probability
Mild Distress CNS+
0.20
Mild Distress CNS-
Mod Distress CNS+
Mod Distress CNS-
0.10
Pre-test Probability
0.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.00
Post-test Probability
0.90 Comment: Phase II Data: appears to
show less success for moderate distress
0.80
0.70
0.60
0.50
0.40
Severe Distress CNS+
0.30
Severe Distress CNS-
Baseline Probability
Mild Distress CNS+
0.20
Mild Distress CNS-
Mod Distress CNS+
Mod Distress CNS-
0.10
Pre-test Probability
0.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.00
Post-test Probability
0.90 Comment: Phase II Data: Anger
Clinicians do not accurately identify
anger!
0.80
0.70
0.60
0.50
0.40
Severe Distress CNS+
0.30
Severe Distress CNS-
Baseline Probability
Mild Distress CNS+
0.20
Mild Distress CNS-
Mod Distress CNS+
Mod Distress CNS-
0.10
Pre-test Probability
0.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Comment: Slide illustrates actual gain in
meta-analysis of screening
implementation in primary care

1.00
Post-test Probability
Clinical+
Clinical-
0.90 Baseline Probability
Screen+
Screen-
0.80
0.70
0.60
0.50
0.40
Comment: Slide illustrates Bayesian
0.30 curve comparison from RCT studies of
clinician with and without screening
0.20 This illustrates ACTUAL gain from
screening
0.10
Pre-test Probability
0.00
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

5. Cancer Care – Cumulative Testing
What can enhance detection?

N = 1000
Cancer Population
n = 200 n = 800
Depression No Depression
Se 70%
CNS Assessment Sp 55%
Screen #1 Screen #1
+ve -ve
PPV 28% NPV 88%
TP = 140 TN =440
Possible case FP = 360
Probable Non-Case FN = 60
TN = 440 FP = 360 Se 70% PPV 28%
Yield TP = 140 FN = 60 Sp 55% NPV 88%

N = 1000
Cancer Population
n = 200 n = 800
Depression No Depression
Se 70%
CNS Assessment Sp 55%
Screen #1 Screen #1
+ve -ve
PPV 28% NPV 88%
TP = 140 TN =440
Possible case FP = 360
Probable Non-Case FN = 60
Sp 40%
Oncologist Assessment Sp 80%
Screen #2 Screen #2
+ve +ve
PPV 44% NPV 77%
TP = 56 TN =288
Probable Depression FP = 72
Probable Non-Case FN = 84
TN = 728 FP = 72 Se 28% PPV 44%
Cumulative Yield TP = 56 FN = 144 Sp 91% NPV 83%

Credits & Acknowledgments
Elena Baker-Glenn University of Nottingham
Paul Symonds Leicester Royal Infirmary
Chris Coggan Leicester General Hospital
Burt Park University of Nottingham
Lorraine Granger Leicester Royal Infirmary
Mark Zimmerman Brown University, Rhode Island
Brett Thombs McGill University Canada
James Coyne University of Pennsylvania
Nadia Husain University of Leicester
For more information www.psycho-oncology.info

FURTHER READING:
Screening for Depression in Clinical Practice An
Evidence-Based guide
ISBN 0195380193
Paperback, 416 pages
Nov 2009
Price: £39.99