Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to The Ideal World of Grading in Diabetic Retinopathy Screening
Similar to The Ideal World of Grading in Diabetic Retinopathy Screening (20)
More from PHEScreening
More from PHEScreening (20)
Recently uploaded
Recently uploaded (20)
The Ideal World of Grading in Diabetic Retinopathy Screening
- 1. The ideal world of grading Shelley Widdowson National grading lead Public Health England leads the NHS Screening Programmes
- 2. Aims • Discuss the meaning of sensitivity and specificity in diabetic retinopathy grading • Look at why sensitivity and specificity isn’t 100% • Discuss the variance in national referral rates • How good grading can increase screening uptake 2 The world of grading
- 3. 3 The world of grading Ideal world of grading • 100% sensitive Sensitivity measures the proportion of positives that are correctly identified as such (e.g. the percentage of people with referable diabetic retinopathy who are correctly identified as having the condition). • 100% specific Specificity measures the proportion of negatives that are correctly identified as such (e.g. the percentage of people without referable diabetic retinopathy who are correctly identified as not having the condition).
- 4. Real world grading Screening is not 100% sensitive or specific • False negatives: wrongly reported as not having the condition and means patients are not referred appropriately • False positives: wrongly reported as having the condition and means the patient is referred inappropriately 4 The world of grading
- 5. Sensitivity and specificity 5 The world of grading Sensitivity = number of true positives number of true positives + false negatives Specificity = number of true negative number of true negatives + false positives
- 6. TAT sensitivity and specificity 6 The world of grading 1-specificity 1-specificity
- 7. 7 The world of grading How many potential false negatives? Numbers screened Average referral rate (2.8 %) Potential false negatives (based on 85% sensitivity) Potential false negatives (based on 92% sensitivity) Average programme size 30,000 840 148 73 National eligible diabetics (2015- 2016) 2,023,885 56,913 10,043 4,948
- 8. Why aren’t we 100% sensitive and specific? • Referable lesion may be outside the field of view • Lesion may be masked by opacity but yet the images meet the gradability requirements • Grading can be subjective • Graders misinterpret lesions • Training issue • Graders miss lesions • Concentration issue / technique • Wrong use of software • Incorrect selection of features / incorrect outcome 8 The world of grading
- 9. 9 The world of grading Variablenationalreferralrates– is it due to inconsistenciesin nationalgrading?
- 10. Programme 2 10 The world of grading
- 11. How can good grading improve uptake? 11 The world of grading Limiting false referrals Extend screening intervals
- 12. Extend screening intervals • National Screening Committee (NSC) agreed to support extended screening intervals • Health Technology Assessment (HTA) study showed low risk patients (R0M0 both eyes for 2 consecutive years) can be screened 2 yearly • NSC need the assurance that programmes are grading to national standard • NDESP are currently looking at ways to check programme grading • Programmes should start to prepare for this by: • planning regular specific audits of grading to check quality • comparing your data with other screening programmes and explain your referral rates in comparison to the national data GOV.UK > Diabetic eye screening: commission and provide > DES: data and research 12 The world of grading
- 13. Thank you
Editor's Notes
- And this may still be the world in which people (or those being screened) believe that the test they are having done is exactly this. Where as we all know that screening is a sieve and in the words of Dr Muir Gray we ‘Make people an offer to reduce the risk’ and in DES case this is ‘sight loss Maximize detection - Minimize anxiety Original meaning of screening is a sieve but 20th century meaning of screening has become mean that there is no longer the concept of a sieve… Aristotle was a Greek philosopher and scientist Aristotle's famous dictum 'primum non nocere'- the first duty is to do no harm. Consequences of not being 100% sensitive - patient will be recalled for screening in 1 year and in that time they may loose sight Consequences of not being 100% specific – patient will be referred to hospital and told there is nothing wrong. Patient may be anxious and this is an unnecessary referral. Cost to the NHS How has DS with helped? – helps us be more sensitive and specific to maculopathy and therefore patient are only referred when they are closer to requiring treatment.
- Talk about types of errors and some photos to show what we are talking about
- Surrogate markers – can calculate sen and spec with ROG grader – but the ROG grader isn’t always right and the 2D image grading used in screening is a different concept HES outcomes.
- Our specificity to any DR is quite a bit more variable . This is because of cases where se say – is this a dust spot, pigment, end of vessel or artefact
- We don’t actually know what the true sensitivity and specificity to referable disease is in programmes. I have taken the acceptable percentage for sensitivity in the TAT so this s a green flag. So 85% sensitivity for a programme with 30,000 paitent s screened (which is the average) – if there was just 1 grader looking at all the cases then diabetics wouldn’t get an appropriate referral. Tiered grading helps to ensure there are less missed cases 840 = 85% of all true positives Here is the logic. STDR = 85% True positives N not known. True positives that are screened positive (85% of N) is 840. Estimate of N = 840/0.85 = 988. Estimated number missed = 988-840= 148😊 STDR = 92% True positives that are screened positive (92% of N) is 840. Estimate of N = 840/0.92 = 913. Estimated number missed = 913-840= 73😊 The truth is we have no idea how many false negatives there are in DES screening. We have incident reports where patients have presented with sight loss in HES – or have returned to screening the following year with worse disease where the year before there was missed pathology. But we also have cases where the disease has resolved and screening programmes may never look back at the previous images to even notice that there was a missed case the year before. This is the thing about screening for diabetic retinopathy in low risk cases the disease can resolve to a certain level that no longer sits in the referable category. This doesn’t happen in screening for other conditions. Its quite a unique programme. In an ideal world of grading we would have 1 IT system and all the local eye clinics would have access to that system and input the results of the 1st eye clinic visit so that we could really start to understand how well we are doing with specificity…. It’s extremely rare that any clinical test is 100% sensitive. A test with 90% sensitivity will identify 90% of patients who have the disease, but will miss 10% of patients who have the disease.
- We are human after all and grading can be subjective. Talk about types of errors and some photos to show what we are talking about so IRMA versus NVE Even when you add routine and urgent referrals together we still have a significant difference
- We all have these reasons why we aren’t 100% sen and spec – but does it account for this level of variance? The national average for routine referrals is 2,300 cases per 100,000. We can see here there is a 10 fold difference between the programme referring 600 in every 100,000 and the programme referring 6,200 in every 100,000. This appears to be a significant difference and we don’t know why this is happening. This data is published on the PHE website and each programme can now see their grading data mapped against all other screening programmes. If you are programme 2 it might be worth talking to programme 14….
- Look at the orange line – routine referrals – looks like an overall reduction of 50% which will see them more in line with the national average.
- There was a low rate of referable eye disease after 2 years in the low-risk group at 0.3–1.3%, and this steadily increased in the intermediate-risk group (2–9%) and higher-risk groups (13–29%). Limiting false referrals – MDT case studies – review of STDR referred to HES – peer grading
- There was a low rate of referable eye disease after 2 years in the low-risk group at 0.3–1.3%, and this steadily increased in the intermediate-risk group (2–9%) and higher-risk groups (13–29%).