Top Quality Call Girl Service Kalyanpur 6378878445 Available Call Girls Any Time
Science of Screening Tools
1. Alex Mitchell www.psycho-oncology.info
Department of Cancer & Molecular Medicine, Leicester Royal Infirmary
Department of Liaison Psychiatry, Leicester General Hospital
Portugal 2010Portugal 2010
WORKSHOP Day 1
Science of Screening:
Definitions, analysis, screening tools, case-finding tools,
prevalence, link with physical concerns
WORKSHOP Day 1
Science of Screening:
Definitions, analysis, screening tools, case-finding tools,
prevalence, link with physical concerns
2. Schedule Day 1Schedule Day 1
930-10.00 – Introduction, groups and issues
10.00-11.00 – T1 Basic science of screening
Break
11.30 – 12.30 – Group task #1
Lunch
1.30-2.30 – T2 Symptoms, Burden, Help, Needs in Cancer
Break
3.00 – 4.00 – Evaluation of a screening paper
3. 10 Questions10 Questions
1. How do we understand screening studies
2. Can we design a screening study
3. Which instrument works best
4. Which is the most popular tool
5. How good are clinicians alone
6. Can the DT be improved
7. Is screening effective in clinical practice
8. What are the barriers to successful implementation
9. How can screening be improved
10. Do somatic symptoms interfere with the diagnosis
7. Cancer Death Rates* Among Men, US,1930-2005
*Age-adjusted to the 2000 US standard population.
Source: US Mortality Data 1960-2005, US Mortality Volumes 1930-1959,
National Center for Health Statistics, Centers for Disease Control and Prevention, 2008.
0
20
40
60
80
100
1930
1935
1940
1945
1950
1955
1960
1965
1970
1975
1980
1985
1990
1995
2000
2005
Lung & bronchus
Colon & rectum
Stomach
Rate Per 100,000
Prostate
Pancreas
LiverLeukemia
8. Cancer Death Rates* Among Women, US,1930-2005
*Age-adjusted to the 2000 US standard population.
Source: US Mortality Data 1960-2005, US Mortality Volumes 1930-1959,
National Center for Health Statistics, Centers for Disease Control and Prevention, 2008.
0
20
40
60
80
100
1930
1935
1940
1945
1950
1955
1960
1965
1970
1975
1980
1985
1990
1995
2000
2005
Lung & bronchus
Colon & rectum
Uterus
Stomach
Breast
Ovary
Pancreas
Rate Per 100,000
12. N=23 studies; 50% some treatment 33% minimal treatment N=19 studies; 30% 1 in 1/12; 10% 3 in 3 months
13. T1. Basic Science of ScreeningT1. Basic Science of Screening
Definitions
Graphics
14. Diagnostic Testing……by application (who)Diagnostic Testing……by application (who)
Routine Screening
The systematic application of a test or inquiry, to all individuals
who may have (or who have not sought medical help for that
disorder)
Targeted (High Risk)
The highly selected application of a test or inquiry, to identify
individuals at high risk of a specific disorder by virtue of
known risk factors
Adapted from Department of Health. Annual report of the national screening committee. London:
DoH, 1997.
15. Diagnostic Testing……by aim (why)Diagnostic Testing……by aim (why)
Screening
Rule out those without the disorder with high accuracy (high
NPV)
Case-Finding
Rule in those with the disorder with high accuracy (high PPV)
16. Diagnostic Testing……by method (how)Diagnostic Testing……by method (how)
Screening
A simple tool with high acceptability but good NPV
Case-Finding
An accurate tool with high PPV and NPV
Rating
Simple, patient rated, correl. With QoL and other outcomes
17. Defining Diagnostic Testing…by comparatorDefining Diagnostic Testing…by comparator
Accuracy (aka convergent validity)
The degree of approximation (veracity) to a robust comparator
Validity (aka criterion validity)
The degree of approximation (veracity) to a criterion reference
Precision
The degree of predictability (low SD) in the measure
18. Stage Type Purpose Description
Pre-clinical Development Development of the proposed tool or
test
Here the aim is to develop a screening method that is likely to help in the detection of the
underlying disorder, either in a specific setting or in all setting. Issues of acceptability of the
tool to both patients and staff must be considered in order for implementation to be
successful.
Phase
I_screen
Diagnostic validity Early diagnostic validity testing in a
selected sample and refinement of tool
The aim is to evaluate the early design of the screening method against a known (ideally
accurate) standard known as the criterion reference. In early testing the tool may be
refined, selecting most useful aspects and deleting redundant aspects in order to make the
tool as efficient (brief) as possible whilst retaining its value.
Phase
II_screen
Diagnostic validity Diagnostic validity in a representative
sample
The aim is to assess the refined tool against a criterion (gold standard) in a real world
sample where the comparator subjects may comprise several competing condition which
may otherwise cause difficulty regarding differential diagnosis.
Phase
III_screen
Implementation Screening RCT; clinicians using vs not
using a screening tool
This is an important step in which the tool is evaluated clinically in one group with access
to the new method compared to a second group (ideally selected in a randomized fashion)
who make assessments without the tool.
Phase
IV_screen
Implementation Screening implementation studies using
real-world outcomes
In this last step the screening tool /method is introduced clinically but monitored to discover
the effect on important patient outcomes such as new identifications, new cases treated
and new cases entering remission.
Development of Diagnostic Tests
20. Accuracy 2x2 TableAccuracy 2x2 Table
Depression
PRESENT
Depression
ABSENT
Test +ve True +ve False +ve PPV
Test ‐ve False ‐Ve True ‐Ve NPV
Sensitivity Specificity Prevalence
Reference Standard
Disorder Present
Reference Standard
No Disorder
Test
+ve A B
A/A + B
PPV
Test
-ve C D
D/C + D
NPV
Total A / A + C
Sn
D / B + D
Sp
21. Basic Measures of AccuracyBasic Measures of Accuracy
Sensitivity (Se) a/(a + c) TP / (TP + FN)
A measure of accuracy defined the proportion of patients with disease in whom
the test result is positive: a/(a + c)
Specificity (Sp) d/(b + d) TN / (TN + FP)
A measure of accuracy defined as the proportion of patients without disease in
whom the test result is negative
Positive Predictive Value a/(a+b) TP / (TP + FP)
A measure of rule‐in accuracy defined as the proportion of true positives in
those that screen positive screening result, as follows
Negative Predictive Value c/(c+d) TN / (TN + FN)
A measure of rule‐out accuracy defined as the proportion of true negatives in
those that screen negative screening result, as follows
23. Graphical – Screening principles
Non-Depressed
Depressed
#
of
Individuals
#
of
Individuals
Severity of Depression
24. Graphical – Screening principles
Non-Depressed
Depressed
#
of
Individuals
Cut-Off
#
of
Individuals
Severity of Depression
HighLow
25. Graphical – Screening principles
Non-Depressed
Depressed
#
of
Individuals
Cut-Off
#
of
Individuals
Severity of Depression
HighLow
High Sensitivity >>>>
<<<< high Specificity
26. Graphical – Screening principles
Non-Depressed
Depressed
#
of
Individuals
Cut-Off
#
of
Individuals
Severity of Depression
HighLow
High Sensitivity >>>>
<<<< low Specificity
27. Can This Help establish a syndrome?Can This Help establish a syndrome?
28. Example: A Clear Disease [#1]Example: A Clear Disease [#1]
Disorder
Number
of
Individuals
False +veFalse +ve
True -veTrue -ve
Point of Partial Rarity
Test Result
No Disorder
False -veFalse -ve
True +veTrue +ve
29. Example: A Probable Syndrome [#2]Example: A Probable Syndrome [#2]
Disorder
Number
of
Individuals
False +veFalse +ve False -veFalse -ve
True -veTrue -ve
True +veTrue +ve
MMSE Cognitive Score
No Disorder
30. Example: A Normally Distributed Trait [#3]Example: A Normally Distributed Trait [#3]
Disorder
Number
of
Individuals
False +veFalse +ve False -veFalse -ve
True -veTrue -ve
True +veTrue +ve
MMSE Cognitive Score
No Disorder
32. Hubbert et al (2005) BMC GeriatricsHubbert et al (2005) BMC Geriatrics
MMSE scores for dementia (n=72)
and non-dementia (n=2735)
Huppert et al BMC Geriatrc 2005
34. Thompson et al (2001) n=18,414 HADS-DThompson et al (2001) n=18,414 HADS-D
0
500
1000
1500
2000
2500
3000
Zero
O
ne
Tw
o
Three
Four
Five
Six
Seven
eight
N
ine
Ten
Eleven
Tw
elve
Thirteen
Fourteen
Fifteen
SixteenSeventeen
Eighteen
37. Accuracy in wordsAccuracy in words
Sensitivity
The chance of testing positive among those with the condition
The chance of rejecting the null hypothesis among those that do not satisfy the null hypothesis
Specificity
The chance of testing negative among those without the condition
The chance of accepting the null hypothesis among those that satisfy the null hypothesis
Positive Predictive Value
The chance of having the condition among those that test positive
The chance of not satisfying the null hypothesis among those that reject the null hypothesis
Negative Predictive Value
The chance of not having the condition among those that test negative
The chance of satisfying the null hypothesis among those that accept the null hypothesis
Type I Error or α (alpha) or p-Value or false positive rate
The chance of testing positive among those without the condition
The chance of rejecting the null hypothesis among those that satisfy the null hypothesis
Type II Error or β (beta) or false negative rate
The chance of testing negative among those with the condition
The chance of accepting the null hypothesis among those that do not satisfy the null hypothesis
False Discovery Rate or q-Value
The chance of not having the condition among those that test positive
The chance of satisfying the null hypothesis among those that reject the null hypothesis
False Omission Rate
The chance of having the condition among those that test negative
The chance of not satisfying the null hypothesis among those that accept the null hypothesis
41. Test vs Major DepressionTest vs Major Depression
Depression
PRESENT
Depression
ABSENT
Test +ve 500 1500 2000
Test -ve 500 4500 5000
1000 6000 7000
Sensitivity
50%
PPV 25%
Specificity
75%
NPV 90%
Prevalence 14%
42. Test vs Major + Min DepressionTest vs Major + Min Depression
Depression
PRESENT
Depression
ABSENT
Test +ve 500 1500 2000
Test -ve 500 500 1000
1000 2000 3000
Sensitivity
50%
PPV 25%
Specificity
33%
NPV 50%
Prevalence 33%
45. Added ValueAdded Value
Definition 1:
The additional ability of a test to rule‐in or rule‐out
compared with the baseline rate
PPV minus Prevalence
NPV minus prevalence
Definition 2:
The additional of a test to rule‐in or rule‐out compared
with the unassisted rate
PPV test minus PPV no test (assuming equal prevalence)
LR+ test minus LR+ no test
AUC test minus AUC no test
51. Group Work #1Group Work #1
930-10.00 – Introduction, groups and issues
10.00-11.00 – T1 Basic science of screening
Break
11.30 – 12.30 – Group task #1
Lunch
1.30-2.30 – T2 Symptoms, Burden, Help, Needs in Cancer
Break
3.00 – 4.00 – Evaluation of a screening paper
52. Cancer Mj Depression vs NonMjCancer Mj Depression vs NonMj
Clinicians diagnosis using DSMIV vs SCAN/PSE
50 people with depression
200 without depression
53. Clinicians using DSMIVClinicians using DSMIV
IF: Clinicians diagnosed 50 cases with depression
IF: Their specificity was 95%
Q. What was the sensitivity?
Q. What was the prevalence?
Q. What was the PPV?
Q. What was overall accuracy
54. Test vs Major DepressionTest vs Major Depression
Depression
On SCAN
Depression
ABSENT
Test +ve
(Clinician)
40 10 50
Test -ve 10 190
50 200
Sensitivity
80%
PPV 80%
Specificity
95%
NPV 95%
Prevalence 0.20%
56. Cancer Mj+Mn Depression vs NonCancer Mj+Mn Depression vs Non
Clinicians diagnosis using DSMIV vs SCAN/PSE
50 people with depression
200 without depression => 50 had minor depression
58. Test vs Major DepressionTest vs Major Depression
Depression
On SCAN
Depression
ABSENT
Test +ve
(Clinician)
50 0 50
Test -ve 50 150 200
100 150
Sensitivity
50%
SN-OUT
PPV 100%
Specificity
100%
SP-IN
NPV 40%
Prevalence 66.7%
60. Likelihood RatiosLikelihood Ratios
Likelihood Ratio for Positive Tests
The chance of testing positive among
those with the condition; divided by the
chance of testing positive among those
without the condition
Sensitivity / (1 - Specificity)
[ TP / (TP + FN) ] / [ FP / (FP + TN) ]
= PPV / Prevalence
Likelihood Ratio for Negative Tests
The chance of testing negative among
those with the condition; divided by the
chance of testing negative among those
without the condition
Specificity / (1 – Sensitivity)
[ FN / (FN + TP) ] / [ TN / (TN + FP) ]
= NPV / Prevalence
61. T3. Symptoms, Help, Needs in CancerT3. Symptoms, Help, Needs in Cancer
Clinician Opinion
Patient Opinion
63. 462 (42%)
Meetable Needs
1093 (100%)
Population
388 (84%)
Aware of Need
172 (44%)
Requested Help
80 (47%)
Needs Met
462 needs
17.3%
322 DSMIV
25%
64. T4. How Common is Distress?T4. How Common is Distress?
Clinician Opinion
Patient Opinion
65. Requires depressed mood for
most of the day, for most days
(by subjective account or
observation) for at least 2 years
The symptoms cause clinically
significant distress OR
impairment in social,
occupational, or other
important areas of functioning.
Requires persistently low mood two
(or more) of the following six
symptoms:
(1) poor appetite or overeating
(2) Insomnia or hypersomnia
(3) low energy or fatigue
(4) low self-esteem
(5) poor concentration or difficulty
making decisions
(6) feelings of hopelessness
DSM-IV Dysthymic disorder
Acute: if the disturbance lasts
less than 6 months
Chronic: if the disturbance
lasts for 6 months
These symptoms cause marked
distress that is in excess of
what would be expected from
exposure to the stressor OR
significant impairment in social
or occupational (academic)
functioning
Requires the development of
emotional or behavioral symptoms in
response to an identifiable stressor(s)
occurring within 3 months of the
onset of the stressor(s). Once the
stressor has terminated, the
symptoms do not persist for more
than an additional 6 months.
DSM-IV Adjustment disorder
2 weeksThese symptoms cause
clinically important distress OR
impair work, social or personal
functioning.
Requires two to four out of nine
symptoms with at least at least one
from the first two (depressed mood
and loss of interest).
DSM-IV Minor Depressive Disorder
2 weeksThese symptoms cause
clinically important distress OR
impair work, social or personal
functioning.
Requires five or more out of nine
symptoms with at least at least one
from the first two (depressed mood
and loss of interest).
DSM-IV Major Depressive Disorder
2 weeks unless symptoms are
unusually severe or of rapid
onset).
At least some difficulty in
continuing with ordinary work
and social activities
Requires two of the first three
symptoms (depressed mood, loss of
interest in everyday activities,
reduction in energy) plus at least two
of the remaining seven symptoms
(minimum of four symptoms)
ICD-10 Depressive Episode
DurationClinical SignificanceSymptoms
67. Prevalence of depression in Palliative settings
20 studies involving 2655 individuals
16.9% (95% CI = 13.2% to 21.0%)
13.0% (95% CI = 11.6% to 14.5%) for MDD
p572
Proportion meta-analysis plot [random effects]
0.0 0.2 0.4 0.6
combined 0.17 (0.13, 0.21)
Maguire et al (1999) 0.05 (0.01, 0.14)
Akechi et al (2004) 0.07 (0.04, 0.11)
Kadan-Lottich et al (2005) 0.07 (0.04, 0.11)
Love et al (2004) 0.07 (0.04, 0.11)
Wilson et al (2004) 0.12 (0.05, 0.22)
Chochinov et al (1997) 0.12 (0.08, 0.18)
Wilson et al (2007) 0.13 (0.10, 0.17)
Kelly et al (2004) 0.14 (0.06, 0.26)
Chochinov et al (1994) 0.17 (0.11, 0.24)
Le Fevre et al (1999) 0.18 (0.10, 0.28)
Breitbart et al (2000) 0.18 (0.11, 0.28)
Meyer et al (2003) 0.20 (0.10, 0.35)
Minagawa et al (1996) 0.20 (0.11, 0.34)
Lloyd-Williams et al (2001) 0.22 (0.14, 0.31)
Hopwood et al (1991) 0.25 (0.16, 0.36)
Desai et al (1999) [late] 0.25 (0.10, 0.47)
Payne et al (2007) 0.26 (0.19, 0.33)
Lloyd-Williams et al (2003) 0.27 (0.17, 0.39)
Jen et al (2006) 0.27 (0.19, 0.36)
Lloyd-Williams et al (2007) 0.30 (0.24, 0.36)
proportion (95% confidence interval)
68. Prevalence of depression in Oncology settings
57 studies involving 9195 individuals across 12
countries.
The prevalence of depression was 17.3% (95% CI =
13.8% to 21.2%),
13.0% (95% CI = 11.6% to 14.5%) for MDD
p572
Proportion meta-analysis plot [random effects]
0.0 0.3 0.6 0.9
combined 0.1730 (0.1375, 0.2116)
Colon et al (1991) 0.0100 (0.0003, 0.0545)
Massie and Holland (1987) 0.0147 (0.0063, 0.0287)
Hardman et al (1989) 0.0317 (0.0087, 0.0793)
Derogatis et al (1983) 0.0372 (0.0162, 0.0720)
Lansky et al (1985) 0.0455 (0.0291, 0.0676)
Mehnert et al (2007) 0.0472 (0.0175, 0.1000)
Katz et al (2004) 0.0500 (0.0104, 0.1392)
Singer et al (2008) 0.0519 (0.0300, 0.0830)
Sneeuw et al (1994) 0.0540 (0.0367, 0.0761)
Pasacreta et al (1997) 0.0633 (0.0209, 0.1416)
Lee et al (1992) 0.0660 (0.0356, 0.1102)
Reuter and Hart (2001) 0.0761 (0.0422, 0.1244)
Grassi et al (2009) 0.0826 (0.0385, 0.1510)
Grassi et al (1993) 0.0828 (0.0448, 0.1374)
Walker et al (2007) 0.0831 (0.0568, 0.1165)
Kawase et al (2006) 0.0851 (0.0553, 0.1240)
Coyne et al (2004) 0.0885 (0.0433, 0.1567)
Alexander et al (2010) 0.0900 (0.0542, 0.1385)
Love et al (2002) 0.0957 (0.0650, 0.1346)
Ozalp et al (2008) 0.0971 (0.0576, 0.1510)
Morasso et al (2001) 0.0985 (0.0535, 0.1625)
Costantini et al (1999) 0.0985 (0.0535, 0.1625)
Silberfarb et al (1980) 0.1027 (0.0587, 0.1638)
Desai et al (1999) [early] 0.1111 (0.0371, 0.2405)
Morasso et al (1996) 0.1121 (0.0593, 0.1877)
Prieto et al (2002) 0.1227 (0.0825, 0.1735)
Ibbotson et al (1994) 0.1242 (0.0776, 0.1853)
Payne et al (1999) 0.1290 (0.0363, 0.2983)
Kugaya et al (1998) 0.1328 (0.0793, 0.2041)
Alexander et al (1993) 0.1333 (0.0594, 0.2459)
Gandubert et al (2009) 0.1597 (0.1040, 0.2300)
Razavi et al (1990) 0.1667 (0.1189, 0.2241)
Akizuki et al (2005) 0.1797 (0.1376, 0.2283)
Leopold et al (1998) 0.1887 (0.0944, 0.3197)
Devlen et al (1987) 0.1889 (0.1141, 0.2851)
Berard et al (1998) 0.1900 (0.1184, 0.2807)
Joffe et al (1986) 0.1905 (0.0545, 0.4191)
Berard et al (1998) 0.2100 (0.1349, 0.3029)
Maunsell et al (1992) 0.2146 (0.1605, 0.2772)
Grandi et al (1987) 0.2222 (0.0641, 0.4764)
Evans et al (1986) 0.2289 (0.1438, 0.3342)
Spiegel et al (1984) 0.2292 (0.1495, 0.3261)
Golden et al (1991) 0.2308 (0.1353, 0.3519)
Fallowfield et al (1990) 0.2565 (0.2054, 0.3131)
Hosaka and Aoki (1996) 0.2800 (0.1623, 0.4249)
Kathol et al (1990) 0.2961 (0.2248, 0.3754)
Green et al (1998) 0.3125 (0.2417, 0.3904)
Jenkins et al (1991) 0.3182 (0.1386, 0.5487)
Burgess et al (2005) 0.3317 (0.2672, 0.4012)
Hall et al (1999) 0.3722 (0.3139, 0.4333)
Morton et al (1984) 0.3958 (0.2577, 0.5473)
Baile et al (1992) 0.4000 (0.2570, 0.5567)
Passik et al (2001) 0.4167 (0.2907, 0.5512)
Bukberg et al (1984) 0.4194 (0.2951, 0.5515)
Massie et al (1979) 0.4850 (0.4303, 0.5401)
Ciaramella and Poli (2001) 0.4900 (0.3886, 0.5920)
Levine et al (1978) 0.5600 (0.4572, 0.6592)
Plumb & Holland (1981) 0.7750 (0.6679, 0.8609)
proportion (95% confidence interval)
74. Arrol et al (2005) BMJArrol et al (2005) BMJ
Setting 19 general practitioners in six clinics in New
Zealand. Participants 1025 consecutive patients
receiving no psychotropic drugs.
After screening “is this something you would like help
with?
The help question alone had a sensitivity of 75% and a
specificity of 94%
The general practitioner with PHQ2 diagnosis had a
sensitivity of 79% and a specificity of 94%
75. Arrol (2005) – Mj DepressionArrol (2005) – Mj Depression
47 CIDI cases with Major depression
25 (53%) wanted help
10 (21%) wanted the option of help
12 (25%) did not want help
76. Arrol (2005) – No DepressionArrol (2005) – No Depression
889 CIDI cases with Major depression
27 (3%) wanted help
24 (3%) wanted the option of help
838 (94%) did not want help
77. 2x2 Help Table2x2 Help Table
Clinician thinks:
Help Needed
Clinician thinks:
Help Not Needed
Patient Says:
Help Wanted
=> Intervention => Refuse?
Patient Says:
Help Not Wanted
=> Delay =>Agree discharge
78. MethodologyMethodology
Study I: Baker-Glen, Symonds, Granger “ET Validation”
(a) n=129 chemotherapy attendees
(b) n=86 chemotherapy f/u
Study II: Sampson, Symonds, Granger “ET Extension”
(c) n=250 chemotherapy + late
Study III: Lord, Symonds, Granger “Coping”
(d) n=250
Study IV: Mitchell, Symonds, Steward “SMI RCT”
(e) n=300
79.
80.
81. Help – Who Wants Help?Help – Who Wants Help?
20% said they wanted professional help for psychosocial
issues.
Only 36% of those distressed on the DT wanted help.
82. Help – Do They Need It?Help – Do They Need It?
27% had major depression
62% had major or minor depression
88% had some distress (HADS, PHQ, DT)
83. Are Those Not Wanting Help OK?Are Those Not Wanting Help OK?
41/104 (39%) of decliners had no identifiable condition
=> 61% of those refusing help actually have a potentially
serious psychosocial condition.
84. What Kind of Help is Wanted?What Kind of Help is Wanted?
19% wanted medication (eg antidepressants)
31% want self help guidelines
31% wanted group therapy
56% wanted illness information.
58% complementary therapies
62% face-to-face psychological support
85. Help – Who From?Help – Who From?
Nurse specialists (54%)
Family and friends (21%)
Spiritual advisor (8%)
Psychiatrist (4%).
86. Why Not Needed?Why Not Needed?
“getting help elsewhere” (57%)
“feel well” (41%)
“coping on my own” (31%)
“fear of stigma”, “fear of side effects”, “not likely to be
effective for me”, and “don’t like to talk about
problems” (all less than 10%)
87. 4. Is Help a Predictor?4. Is Help a Predictor?
88. Help as a Predictor of Depression?Help as a Predictor of Depression?
Outcome Predictor Sensitivity Specificity PPV NPV
DSMIV Major
Depression
Help QQ Alone 0.47 0.83 0.27 0.92
DSMIV Mj + Minor
Depression
Help QQ Alone 0.36 0.88 0.39 0.86
DSMIV Mj + Minor
Depression
Help QQ AND PHQ2 0.36 0.99 0.88 0.88
89. Can This Be Used Clinically?Can This Be Used Clinically?