"Fully Automated MRI Bone Segmentation of Glenoid and Humeral Head using Deep Convolutional Neural Networks" - RSNA19
•Download as PPTX, PDF•
0 likes•248 views
Fully Automated MRI Bone Segmentation of Glenoid and Humeral Head using Deep Convolutional Neural Networks
Recommended
More Related Content
What's hot
What's hot (20)
Similar to "Fully Automated MRI Bone Segmentation of Glenoid and Humeral Head using Deep Convolutional Neural Networks" - RSNA19
Similar to "Fully Automated MRI Bone Segmentation of Glenoid and Humeral Head using Deep Convolutional Neural Networks" - RSNA19 (20)
Recently uploaded
Recently uploaded (20)
"Fully Automated MRI Bone Segmentation of Glenoid and Humeral Head using Deep Convolutional Neural Networks" - RSNA19
- 1. FULLY AUTOMATED MRI BONE SEGMENTATION OF THE GLENOID AND HUMERAL HEAD USING DEEP CONVOLUTIONAL NEURAL NETWORKS TATIANE CANTARELLI RODRIGUES, MD CEM M. DENIZ, PHD ERIN ALAIA, MD NATALIA GORELIK, MD JARED DUBLIN, MS JAMES BABB, PHD SOTERIOS GYFTOPOULOS, MD, MSC SSM17-02
- 2. DISCLOSURES NONE OF THE AUTHORS HAVE ANY CONFLICTS OF INTEREST OR FINANCIAL TIES TO DISCLOSE.
- 3. CLINICAL SCENARIO Anterior shoulder instability = common problem US population Bipolar bone injuries = prevalent in ASI INTRODUCTION HILL-SACHS IMAGING IMPORTANT ROLE IN THE EVALUATION ↓ TREATMENT PLANNING GLENOID BONE LOSS
- 4. CREATE 3D MODELS DETECT BONE INJURIES PREDICT RISK OF ENGAGEMENT A SAI ➟I
- 5. 3D MRI RECONSTRUCTIONS * Gyftopoulos et al. Diagnostic Accuracy of MRI in the Measurement of Glenoid Bone Loss. AJR 2012 Have been performed at our institution to evaluate bone injuries in the setting of ASI Accurate as 3D CT in the assessment of normal glenoid anatomy. 3D MR reconstructions of the shoulder can be used to accurately measure glenoid bone loss.*
- 6. MR SCANNER 3D T1-W DIXON ACQUISITION POST-PROCESSING SUBTRACTION 3D LAB SEMI-AUTOMATED SEGMENTATION READING ROOM INTERPRETATION HOW TO CREATE 3D MRI MODELS? CURRENT WORKFLOW 3’30” 1’ 10-20’ IMPACTS WORKFLOW DELAYED RESULTSQUEUE
- 7. Fully-automated shoulder bone segmentation from MRI using AI has not been explored FULLY-AUTOMATED SEGMENTATION DOES NOT REQUIRE A GREAT DEAL OF USER INTERACTION NOT: TIME‐CONSUMING EFFORT‐DEMANDING DL techniques may improve workflow while maintaining accuracy TIME COSTS PRECISION REPRODUCIBILITY HOW TO IMPROVE THE WORKFLOW?
- 8. OBJECTIVE To present and evaluate: Fully automated bone segmentation of the glenohumeral joint with MR imaging using an AI method based on 2D convolutional neural networks
- 9. MATERIALS AND METHODS Supervised Learning • Shoulder MRI • 100 patients • Last 2 years • Different clinical reasons • * including shoulder dislocation • Left/Right • Sex - M/F • Magnetic field -1.5T / 3.0T • * Siemens (Vida, Avanto, Skyra, Verio, Prisma Fit) Transfer Learning • Shoulder MRI • 45 patients • Last 5 years • Shoulder dislocation • Left/Right • Sex - M/F • Magnetic field -1.5T / 3.0T • * Siemens (Sonata, Aera, Skyra, Prisma Fit) Excluded • Poor quality • Incomplete study protocol • Arthrogram • Some clinical conditions: • Surgery • Tumor • Some fractures (humeral shaft) • Infection • Congenital deformities
- 11. PREPARING TRAINING DATA ITK – SNAP http://www.itksnap.org Axial PD FS series Voxel size: 0.4375 x 0.4375 mm Section thickness: 3.3 mm Matrix size: 320 x 320 DICOM (Digital Imaging and Communications in Medicine) NIfTI (Neuroimaging Informatics Technology Initiative)
- 12. Segmentation Mask Fully automated Ground Truth Shoulder MRI 1 64 64 128 128 256 256 512 512 1024 512 256 128 512 256 128 64 64 3 1024 3202 1602 802 402 202 402 802 1602 3202 3202 3202 3202 202 202 * Experiments were performed on a server using an NVIDIA 16GB Tesla P100 GPU card. DEEP LEARNING MODELINPUT OUTPUT TensorFlow software
- 13. Cross-validation technique + random sampling Training set = 75% used to identify a segmentation model to discriminate shoulder bones from background Validation set = 25% used to provide data to test the accuracy of the selected model Trained optimal model Defined by the ensemble of 4 models identified with a 4- fold cross-validation scheme EVALUATING THE PREDICTIVE MODEL TRAINING Each epoch (one full training cycle) takes ˜ 14 min SEGMENTING Total time required to calculate the segmentation masks from one subject: ˜4 sec
- 15. Old Classifier Supervised Learning Model Extract Parameters TRANSFERLEARNING Input: 2D fat-suppressed PD * Large labeled data (100/100) New Classifier Transfer Learning Model Pretrained 2D CNN Model New 2D CNN Model Input: 3D water-only Dixon-based * Small labeled data (6/45)
- 16. TRANSFER LEARNING 2D-CNN PREDICTION 3D water-only Dixon-based POST-PROCESSED* 2D-CNN SEGMENTATION MASK * 3D slicer - https://www.slicer.org 3D MRI RECONSTRUCTIONS FULLY AUTOMATED SEGMENTED
- 17. AI Fully-Automated 3D MRI Bone Models STANDARD Semi-Automated 3D MRI Bone Models Humeral Head Anatomy Glenoid Anatomy Glenoid Bone Loss WW WW DD DD D D d d H H Two MSK radiologists performed measurements on fully automated x semi automated 3D MRI models MEASUREMENTS * Di Giacomo G, de Gasperis N, Scarso P. Bipolar bone defect in the shoulder anterior dislocation. Knee Surg Sports Traumatol Arthrosc 2016;. How often a difference in GBL % FA x SA models would potentially impact patient treatment selection * based on clinical practice at our institution: - < 20% GBL = arthroscopic Bankart repair, - > 20% GBL = bone augmentation surgery, (such as a Latarjet procedure )
- 18. RESULTS
- 19. RESULTS AP = average precision Dice = dice score MEASURE THE ACCURACY OF THE MODEL Region AP (SD) DSC (SD) Precision (SD) Sensitivity (SD) 2DUnet Humerus 0.986 ± 0.018 0.948 ± 0.031 0.951 ± 0.019 0.948 ± 0.056 Glenoid 0.923 ± 0.079 0.861 ± 0.084 0.875 ± 0.053 0.861 ± 0.123 Region AP (SD) 2DUnet Humerus 0.986 ± 0.018 Glenoid 0.923 ± 0.079 Region MSD (mm SD) RMSD (mm SD) HD (mm SD) 2DUnet Humerus 0.507 ± 0.400 1.478 ± 1.127 26.864 ± 23.908 Glenoid 0.788 ± 0.515 1.829 ± 0.855 20.650 ±14.423 HD = Hausdorff distance MSD = mean surface distance RMSQ = residual mean square distance
- 20. COMPARISON: MANUAL X FULLY AUTOMATIC SEGMENTATION BY 2D CNN - U NET 2DCNNGroundTruth Segmentation mask performed by a 2D CNN was closely correlated to a manual segmentation (ground truth) performed by an MSK radiologist
- 21. GROUND TRUTH MANUAL FULLY-AUTOMATED SUPERVISED LEARNING 2D-CNN PREDICTION
- 22. MISCLASSIFICATIONFactors Image artifacts Motion Field Inhomogeneity Inherent heterogeneity of image intensity Bone marrow edema, prominent red marrow Partial volume effects Variability of joint structures Varying image contrast BONE MARROW EDEMA MOTION ARTIFACTS
- 23. GLENOHUMERAL ANATOMY + GLENOID BONE LOSS There was no significant difference between fully automated and semi automated 3D models’ measurements D D d d
- 24. IMPACT ON TREATMENT PLANNING Changes in the type of treatment* would have been rare based on our readers’ measurements There was disagreement for: • Reader 1 = 1 patient • Reader 2 = 2 patients * Di Giacomo G, de Gasperis N, Scarso P. Bipolar bone defect in the shoulder anterior dislocation. Knee Surg Sports Traumatol Arthrosc 2016;24(2):479-488. Arthroscopic Bankart repair Latarjet procedure
- 25. - No external validation - Deep learning methods require a diversity of training datasets – some clinical conditions were excluded - 2D CNN LIMITATIONS
- 26. CONCLUSION We developed a fully-automated way to segment the glenoid and humerus using 2D deep CNN with accuracy and efficiency in time periods short enough for routine use in clinical practice 4 SECONDS AVERAGE PRECISION GLENOID = 0.923 HUMERUS = 0.986 Closely correlated to the manual segmentations of an MSK radiologist No significant difference: fully automated x semi automated 3D models measurements
- 27. Workinprogress 3D CNN 3D bone models Segmentation Region AP (SD) 3DUnet Humerus 0.987 ± 0.019 Glenoid 0.919 ± 0.081 POST-PROCESSED TRANSFER LEARNING 3D-CNN SEGMENTATION MASK PREDICTION BASED ON 3D DIXON
- 28. THANK YOU! TATIANE CANTARELLI RODRIGUES, MD CEM M. DENIZ, PHD ERIN ALAIA, MD NATALIA GORELIK, MD JARED DUBLIN, MS JAMES BABB, PHD SOTERIOS GYFTOPOULOS, MD, MSC tcantarelli@gmail.com @tatcantarelli SSM17-02
Editor's Notes
- Axial 3D T1-weighted FLASH sequence with Dixon-based water-fat separation + subtraction Medical 3D image segmentation is an important image processing step in medical image analysis. Lab – time depends of experience of the performer / Some places there is no 3D LAB > have to do at workstation Delay the read > impact the clinical workflow
- Medical 3D image segmentation is an important image processing step in medical image analysis. Automatic - Quickly - Accurate Segmentation methods with high precision (including high reproducibility) and low bias are a main goal in surgical planning Segmentations can be thought of as probabilities of voxels belonging to particular classes. A key problem in image analysis is understanding the content of the image, i.e. subdividing the image in regions at a pixel level so that each pixel belongs to a specific region > > semantic segmentation and can be conducted either manually or automatically; manual segmentation is exposed to intra- and inter-observer variability
- Electronic database Key words + CPT exam codes Review board approval
- MRI convert - https://lcni.uoregon.edu/downloads/mriconvert ITK – SNAP http://www.itksnap.org
- – U-shaped architecture of the two-dimensional convolutional network (2D CNN) model used for supervised learning. Algorithm model where blue rectangles represent feature maps with the size and the number of feature maps indicated. White boxes represent copied feature maps. The number of feature maps doubles at each pooling. The architecture represented in this model contains 32 feature maps in the first and last layer of the network and 4 layers in the contracting/expanding path. The purpose of the contracting path is to capture the context of the input image in order to be able to do segmentation. The purpose of the expanding path is to enable precise localization combined with contextual information from the contracting path. The color-coded arrows denote different operations in this neural network. Random horizontal flipping of the images was used for data augmentation to generalize on left and right sided images
- 4-fold Crossvalidation Cross-validation is a technique to evaluate predictive models by partitioning the original sample into a training set to train the model, and a test set to evaluate it. In k-fold cross-validation, the original sample is randomly partitioned into k equal size subsamples. Of the k subsamples, a single subsample is retained as the validation data for testing the model, and the remaining k-1 subsamples are used as training data. The cross-validation process is then repeated k times (the folds), with each of the k subsamples used exactly once as the validation data. The k results from the folds can then be averaged (or otherwise combined) to produce a single estimation. The advantage of this method is that all observations are used for both training and validation, and each observation is used for validation exactly once. For classification problems, one typically uses stratified k-fold cross-validation, in which the folds are selected so that each fold contains roughly the same proportions of class labels. In repeated cross-validation, the cross-validation procedure is repeated n times, yielding n random partitions of the original sample. The n results are again averaged (or otherwise combined) to produce a single estimation. OpenML generates train-test splits given the number of folds and repeats, so that different users can evaluate their models with the same splits. Stratification is applied by default for classification problems (unless otherwise specified). The splits are given as part of the task description as an ARFF file with the row id, fold number, repeat number and the class (TRAIN or TEST). The uploaded predictions should be labeled with the fold and repeat number of the test instance, so that the results can be properly evaluated and aggregated. OpenML stores both the per fold/repeat results and the aggregated scores.
- Figure 4 – Diagram of the two-dimensional convolutional network (2D CNN) transfer learning model. The 2D CNN model used in the paper was pretrained through supervised learning on axial 2D fat-suppressed proton-density weighted slices from shoulder MRI to obtain a segmentation mask classifier (yellow rectangle) using a dataset (100 total, 100 labeled). Parameters were extracted from the pretrained model and applied through transfer learning to another dataset (45 total, 6 labeled) of axial 3D water-only Dixon-based sequence axial slices of shoulder MRI to obtain a new segmentation mask classifier (green rectangle). Transfer learning has been used in medical image classification based on CNNs extensively and it has been shown to improve generalization for the task where limited number of samples exists
- By transfer learning we were able to train a small dataset and a different MR sequence
- * based on clinical practice at our institution: a patient with < 20% GBL would undergo arthroscopic Bankart repair, while patients with > 20% would undergo bone augmentation surgery, such as a Latarjet procedure
- Evaluating a standard machine learning model - usually classify our predictions into four categories: true positives, false positives, true negatives, and false negatives. However, for the dense prediction task of image segmentation, it's not immediately clear what counts as a "true positive" and, more generally, how we can evaluate our predictions. The lower values for the glenoid and scapula segmentations in comparison with the humeral segmentations in both studies are likely related to the complex anatomy and morphological variability of the glenoid and scapula
- Automatic segmentation performed by CNN. B. Comparison between manual (white for humerus; grey for glenoid) and automatic segmentation (blue for humerus; red for glenoid). the ground truth GT, acquired with manual segmenta-tion by human experts, and the image being segmented
- Pat 16 2D fully automatic segmentation performed by CNN - Unet
- OUTLIERS Those are the most common errors
- 100 patients not so small to segmentation task For relatively small data sets, internal validation of prediction models by bootstrap techniques may not be sufficient and indicative for the model's performance in future patients. External validation is essential before implementing prediction models in clinical practice. - Training time - considerable amount of time (on the order of days) for a typical training dataset but after created the model > save time