0
Assessment of Radiologists’
Performance with CADe for
Digital Mammography
Elodia B. Cole, MS
Medical University of South C...
Co-Authors
Etta Pisano, MD
Medical University of South Carolina
Department of Radiology and Radiological Science
Zheng Zha...
Background
• The Digital Mammographic Imaging Screening Trial
(DMIST) was conducted between 2001-2003. Study
was compariso...
Study Purpose
• Would CAD have made a difference in the
performance of radiologists during DMIST for
digital mammography h...
Study Aim
• To assess the performance of Radiologists in
interpreting digital mammograms without and
then with CAD for dig...
Methods & Materials
• CAD Systems Tested
– R2 ImageChecker Cenova v1.0 (Hologic)
– iCAD SecondLook v1.4 (iCAD)
• Cases
– D...
Methods & Materials
• Readers
– 15 Radiologists Readers with clinical R2 CAD
experience in the R2 study.
– 14 Radiologists...
Methods & Materials
• The experiment
– All reading sessions took place at UNC. Two dedicated
CAD reading rooms were set up...
Case Characteristics
Machine Type Distribution
0
55
240
5
93
40
162
5
0
50
100
150
200
250
300
Fischer
SenoScan
Fuji CR GE...
Case Characteristics
Cancer Histology Distribution
109 106
41 43
0
20
40
60
80
100
120
iCAD R2
Histology
#ofCases
Invasive...
Case Characteristics
Lesion Size Distribution
17
29
70
34
17
26
66
41
0
20
40
60
80
<5 mm 5 mm - 10
mm
>10 mm missing
Lesi...
Case Characteristics
Lesion Type Distribution
62
49
64
45
0
10
20
30
40
50
60
70
Mass Calcifications
Lesion Type
#ofCases
...
iCAD Results
With CAD Without CAD p-value
mean 95% CI mean 95% CI
AUC 0.7221 (0.67,0.77) 0.7131 (0.66,0.76) 0.0695
Sensiti...
Results R2
With CAD Without CAD p-value
mean 95% CI mean 95% CI
AUC 0.7174 (0.67,0.77) 0.7109 (0.66,0.76) 0.0791
Sensitivi...
1 2 3 4 5 6 7 8 9 10 11 12 13 14
-15-10-5051015
ICAD: Impact of CAD on Reader Accuracy
Reader ID
#ofCases
Accurate w. CAD ...
Impact of R2 on Reader Accuracy
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-15-10-5051015
R2: Impact of CAD on Reader Accuracy
Re...
Conclusions
• It is likely that CAD would not have improved
radiologist performance (sensitivity,
specificity, AUC) had it...
Acknowledgments
• CAD equipment provided by Hologic and
iCAD
• Softcopy review workstations provided by
Hologic and Sectra
Upcoming SlideShare
Loading in...5
×

Assessment of Radiologists' Performance with Computer-Aided ...

246

Published on

Published in: Health & Medicine, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
246
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Assessment of Radiologists' Performance with Computer-Aided ..."

  1. 1. Assessment of Radiologists’ Performance with CADe for Digital Mammography Elodia B. Cole, MS Medical University of South Carolina Department of Radiology and Radiological Sciences ACRIN Fall Meeting 2010
  2. 2. Co-Authors Etta Pisano, MD Medical University of South Carolina Department of Radiology and Radiological Science Zheng Zhang, PhD Brown University Center for Statistical Science Helga Marques, MS Brown University Center for Statistical Science Robert Nishikawa, PhD University of Chicago Department of Radiology Martin Yaffe, PhD Sunnybrook Health Sciences Center Depts of Medical Physics and Medical Imaging R. Edward Hendrick, PhD University of Colorado - Denver Department of Radiology Emily Conant, MD University of Pennsylvania Department of Radiology Constantine Gatsonis, PhD Brown University Center for Statistical Science Laurie Fajardo, MD University of Iowa Department of Radiology Janet Baum, MD Harvard Medical School Department of Radiology
  3. 3. Background • The Digital Mammographic Imaging Screening Trial (DMIST) was conducted between 2001-2003. Study was comparison of two technologies screen-film and digital mammography. • Sensitivity for film-screen and digital mammography in DMIST was 0.41. • While CAD for mammography was available for screen-film mammography during this time it was not available for digital mammography and so CAD was not allowed in DMIST.
  4. 4. Study Purpose • Would CAD have made a difference in the performance of radiologists during DMIST for digital mammography had it been available?
  5. 5. Study Aim • To assess the performance of Radiologists in interpreting digital mammograms without and then with CAD for digital mammograms from DMIST.
  6. 6. Methods & Materials • CAD Systems Tested – R2 ImageChecker Cenova v1.0 (Hologic) – iCAD SecondLook v1.4 (iCAD) • Cases – Digital mammograms (“for processing” and “for presentation” states) acquired from digital mammography systems used in DMIST. 300 cases for each machine type: 150 cancers, 150 non-cancers. • R2 (Hologic – Selenia, GE 2000D, Fischer SenoScan, Fuji CR) • iCAD (Hologic-Selenia, GE 2000D, Fuji CR)
  7. 7. Methods & Materials • Readers – 15 Radiologists Readers with clinical R2 CAD experience in the R2 study. – 14 Radiologists Readers with clinical iCAD CAD experience in the iCAD study. • Image Preparation – DICOM headers for Fuji and Fischer cases were brought to current standards to allow for CAD processing using custom software.
  8. 8. Methods & Materials • The experiment – All reading sessions took place at UNC. Two dedicated CAD reading rooms were set up – one for the R2 readings, one for the iCAD. – Only one reader per CAD system at a time. Each reader session took about two days to complete. – Each reader first reviewed the case without CAD marks and provided BIRADS score for each breast. – The reader then applied CAD to the images by toggle button on mammography review workstation displaying the CAD structured report. – Reader then reviewed case with CAD marks and provided BIRADS score again for each breast.
  9. 9. Case Characteristics Machine Type Distribution 0 55 240 5 93 40 162 5 0 50 100 150 200 250 300 Fischer SenoScan Fuji CR GE 2000D Hologic Selenia Machine Type #ofCases iCAD R2
  10. 10. Case Characteristics Cancer Histology Distribution 109 106 41 43 0 20 40 60 80 100 120 iCAD R2 Histology #ofCases Invasive DCIS
  11. 11. Case Characteristics Lesion Size Distribution 17 29 70 34 17 26 66 41 0 20 40 60 80 <5 mm 5 mm - 10 mm >10 mm missing Lesion Size #ofCases iCAD R2
  12. 12. Case Characteristics Lesion Type Distribution 62 49 64 45 0 10 20 30 40 50 60 70 Mass Calcifications Lesion Type #ofCases iCAD R2
  13. 13. iCAD Results With CAD Without CAD p-value mean 95% CI mean 95% CI AUC 0.7221 (0.67,0.77) 0.7131 (0.66,0.76) 0.0695 Sensitivity 0.5130 (0.43,0.60) 0.4858 (0.40,0.57) 0.0894 Specificity 0.8739 (0.81,0.92) 0.8879 (0.83,0.93) 0.1463
  14. 14. Results R2 With CAD Without CAD p-value mean 95% CI mean 95% CI AUC 0.7174 (0.67,0.77) 0.7109 (0.66,0.76) 0.0791 Sensitivity 0.5302 (0.48,0.58) 0.5101 (0.46,0.56) 0.1831 Specificity 0.8629 (0.82,0.90) 0.8745 (0.83,0.91) 0.2380
  15. 15. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 -15-10-5051015 ICAD: Impact of CAD on Reader Accuracy Reader ID #ofCases Accurate w. CAD & Inaccurate w.o CAD Accurate w.o CAD & Inaccurate w. CAD iCAD Impact on Reader Accuracy
  16. 16. Impact of R2 on Reader Accuracy 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 -15-10-5051015 R2: Impact of CAD on Reader Accuracy Reader ID #ofCases Accurate w. CAD & Inaccurate w.o CAD Accurate w.o CAD & Inaccurate w. CAD
  17. 17. Conclusions • It is likely that CAD would not have improved radiologist performance (sensitivity, specificity, AUC) had it been available for the DMIST study • Radiologists are seldom if ever influenced by CAD marks in making their diagnostic decisions
  18. 18. Acknowledgments • CAD equipment provided by Hologic and iCAD • Softcopy review workstations provided by Hologic and Sectra
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×