2022 Undergraduate Research Symposium: Samaah Saifullah Graduate co-author: Kelsey L. Canada Automated hippocampal subfield segmentation methods have led to exponential growth in the literature. Although automated segmentation is perfectly reliable, deviations from anatomical definitions are common and weaken their validity. To address the lack of guidance for quality control (QC) of automated segmentation methods, we developed a taxonomy of errors and two-step protocol for 1) identification and 2) manual correction. Raters examined segmentations from a customized atlas to identify errors in six categories with severity ratings; only major severity errors are manually corrected in order to minimize introducing human error. The QC procedure was developed and cross-validated with an independent sample on two MRI scanners. Raters had excellent agreement in QC decisions between- and within-raters on all regions, manual corrections introduced less than 3% error and were unrelated to participant demographic. The feasible and reliable QC procedure allows confident interpretation of hippocampal subfield segmentations, and potential applications to other brain regions.

1. Quality Matters: A
Procedure for Valid and
Reliable Hippocampal
Subfield Segmentation
Samaah Saifullah¹, Kelsey L. Canada², Ana M. Daugherty¹,²
¹ Department of Psychology, Wayne State University, Detroit, MI
² Institute of Gerontology, Wayne State University, Detroit, MI

2. Introduction
• Hippocampal subfields: roles in cognitive processes and vulnerability to
disease pathologies
• Automatic segmentation improvements, but still error prone
2
Muddled boundaries Clear boundaries

3. AIM:
Develop and test a Quality Control
(QC) protocol for MRI automatic
segmentation of hippocampal
subfields that is reliable and feasible
for multi-site study

4. Methods
Error Identification
• MRI of hippocampal subfields were automatically segmented via ASHS then analyzed
sequentially by coronal slice
• Error Type Classification:
 Lack of Label (LL)
 Underestimated Boundary (UB)
 Overestimated Label (OL)
 Dropped Pixels (DP)
 Sulcal Cavities (SC)
 Cross-Labeling (CL)
• Identified errors given severity rating from
1 (minor) to 3 (major)
4

5. Methods
Error Identification cont.
• Between rater agreement was assessed in QC decisions for subset n = 10
• Cross-validated in an independent sample
collected at a different site
Error Correction
• Only major (3) errors are corrected to
minimize potential bias introduced by
manual intervention
5

6. Results
• Between Rater Agreement (Error Identification): Is there a level 3 error?
 SS and KC had excellent agreement in all regions: estimated Cohen’s kappa with a quadratic
weighting: K = 0.77-0.89 (cross-validation, K = 0.75-0.95)
• Within-Rater Consistency (Error Correction): How much error does human rater
introduce?
 SS manually corrected all level 3 errors in subset of n = 12 twice, within a 2-week delay
 Identified as having excellent reliability: ICC(3) = 0.98-1.0 (cross-validation: ICC = 0.97-1.0)
 Judged relative to perfect reproducibility of automatic segmentation, we are introducing ≤3%
measurement error when correcting boundaries
6

7. Implications
• Adding QC to automatic segmentation maintains efficiency and ensures confidence in
conclusions drawn from hippocampal subfield segmentation measurements
• Protocol can be implemented for measurements of other brain regions
7