Your SlideShare is downloading. ×
0
A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms Martin Urschler, Stefan...
Motivation Validation very difficult! <ul><li>Lack of direct ground truth </li></ul><ul><li>Lack of gold standard methods ...
Contents <ul><li>Problem Definition </li></ul><ul><li>Related Work & Similar Efforts </li></ul><ul><li>Open Science Nonlin...
The Nonlinear Registration Problem <ul><li>„ Find a  deformable mapping h(x)  aligning moving and fixed image  such that a...
Medical Applications of Nonlinear Image Registration <ul><li>Angiography Studies </li></ul><ul><li>Anatomy & Function   </...
Related Work on Evaluation (Frameworks) There exist some good ideas for evaluation in (Medical) Computer Vision! VALMET Se...
Open Science Evaluation Framework <ul><li>Algorithm  Evaluation  vs. Validation [Hellier et al] </li></ul><ul><ul><li>Spec...
Open Science Evaluation Framework
Synthetic Deformations Used <ul><li>Simple synthetic transformation models </li></ul><ul><ul><li>Regular grid with random ...
Data Sets and Algorithms <ul><li>Public domain data sets from  </li></ul><ul><ul><li>National Library of Medicine Dataset ...
Quantitative Measures <ul><li>Measures on Displacement Fields </li></ul><ul><ul><li>RMS of displacement field differences ...
A Sample Evaluation <ul><li>Purpose: Evaluate intra-modality thoracic CT registration subject to breathing differences </l...
A Sample Evaluation Simulated Data Difference to Original Standard Demons
A Sample Evaluation - Results <ul><li>Quantitative measures </li></ul><ul><ul><li>Large number (see paper) </li></ul></ul>...
Conclusion <ul><li>Open Science Evaluation Framwork presented </li></ul><ul><li>Intra-Modality Sample Evaluation shown </l...
Further Work <ul><li>Common Web Repository (Hosted by Kitware?) </li></ul><ul><li>Upgrade by community effort </li></ul><u...
Thank you for your attention!
 
 
 
Upcoming SlideShare
Loading in...5
×

A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms-8396

1,520

Published on

http://hdl.handle.net/1926/561

Published in: Economy & Finance
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,520
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
33
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms-8396"

  1. 1. A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms Martin Urschler, Stefan Kluckner, Horst Bischof Institute for Computer Graphics and Vision, Graz, University of Technology, Austria
  2. 2. Motivation Validation very difficult! <ul><li>Lack of direct ground truth </li></ul><ul><li>Lack of gold standard methods </li></ul><ul><li>- Highly ill-posed problem </li></ul><ul><li>Large space of possible solutions </li></ul>We claim: Standardizing Evaluation Protocols at least as important as Developing Novel Methods! -> Put framework to discussion: - Open-source community effort - Instantiate framework by showing sample evaluation Presented algorithm at last years MICCAI. Nonlinear Image Registration
  3. 3. Contents <ul><li>Problem Definition </li></ul><ul><li>Related Work & Similar Efforts </li></ul><ul><li>Open Science Nonlinear Registration Evaluation Framework </li></ul><ul><li>Sample Evaluation – Intra-subject Thorax CT </li></ul><ul><li>Conclusion & Outlook </li></ul>
  4. 4. The Nonlinear Registration Problem <ul><li>„ Find a deformable mapping h(x) aligning moving and fixed image such that a defined similarity criterium is minimized.“ </li></ul>h(x) Base Illustration taken from ITK Software Guide. SSD, NCC, Mutual Information, … B-Spline, Thin-Plate Spline, Def Field, … Gradient Descent, BFGS, … Intensity-Based, Feature-Based, …
  5. 5. Medical Applications of Nonlinear Image Registration <ul><li>Angiography Studies </li></ul><ul><li>Anatomy & Function </li></ul><ul><ul><li>Perfusion /Ventilation </li></ul></ul><ul><li>Correction of Motion Artifacts </li></ul><ul><li>Studies of Shape Variation </li></ul><ul><li>Segmentation by Atlas Registration </li></ul><ul><li>Surgical Planning </li></ul><ul><li>… </li></ul>Lung Perfusion for Pulmonary Embolism Detection [Wildberger et al.] Validation Study: Atlas-based Brain Volume Segmentation from MRI Images [Ng02] – taken from ITK Documentation Inter- & Intra-Modality Inter- & Intra-Subject No Ground Truth or Gold Standards Highly Ill-Posed How should we reach clinical acceptance?
  6. 6. Related Work on Evaluation (Frameworks) There exist some good ideas for evaluation in (Medical) Computer Vision! VALMET Segmentation Evaluation [Gerig et al] Community would benefit from open-source implementation! MICCAI 2007 Segmentation Challenge Workshop Multi-View 3D Reconstruction [Seitz et al] Stereo Reconstruction [Scharstein et al] Middlesbury Retrospective Evaluation of Intersubject Brain Registration [Hellier et al] Retrospective Rigid Registration Evaluation Project [West et al] NIREP (Inter-subject Brain-Registration Evaluation) [Christensen et al] Validation of Nonrigid Image Registration Using FEM [Schnabel et al] Segmentations are compared!
  7. 7. Open Science Evaluation Framework <ul><li>Algorithm Evaluation vs. Validation [Hellier et al] </li></ul><ul><ul><li>Specific Problems: Lacking ground truth, ill-posedness, Lack of gold standard </li></ul></ul><ul><ul><li>General Problems: Noise, Partial Volume Effect, Interpolation, Numerics </li></ul></ul><ul><li>Modular Evaluation Framework </li></ul><ul><ul><li>Open-source, open-access, open-data, open protocols </li></ul></ul><ul><li>Building Blocks: </li></ul>Registration Algorithms Public domain data sets Synthetic Deformations Similarity Measures Python Framework as Glue
  8. 8. Open Science Evaluation Framework
  9. 9. Synthetic Deformations Used <ul><li>Simple synthetic transformation models </li></ul><ul><ul><li>Regular grid with random deformations & TPS </li></ul></ul><ul><ul><li>Uniform periodic cosine transformation </li></ul></ul><ul><ul><li>Tailored to breathing difference registration: </li></ul></ul><ul><ul><ul><li>Simulated Breathing Model </li></ul></ul></ul><ul><ul><ul><li>Synthetic Airway Tree Movement from Manual Correspondences </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><li>Increase number of models </li></ul><ul><ul><li>Pool of synthetic transformations defined by pool of algorithms („bronze standard“ Glatard et al – MICCAI 2006) </li></ul></ul><ul><ul><li>Community effort needed </li></ul></ul>
  10. 10. Data Sets and Algorithms <ul><li>Public domain data sets from </li></ul><ul><ul><li>National Library of Medicine Dataset Collection (currently offline) </li></ul></ul><ul><ul><li>MIDAS data collection project </li></ul></ul><ul><ul><li>National Lung Cancer Archives (NCIA) </li></ul></ul><ul><li>ITK Algorithms </li></ul><ul><ul><li>Demons </li></ul></ul><ul><ul><li>Symmetric Demons </li></ul></ul><ul><ul><li>Level Set Motion </li></ul></ul><ul><ul><li>Curvature </li></ul></ul><ul><ul><li>Fast Block Matching (Workshop Contribution) </li></ul></ul><ul><ul><li>Diffeomorphic Demons (Workshop Contribution) </li></ul></ul>
  11. 11. Quantitative Measures <ul><li>Measures on Displacement Fields </li></ul><ul><ul><li>RMS of displacement field differences </li></ul></ul><ul><ul><li>MAD of displacement field differences </li></ul></ul><ul><ul><li>MAX of displacement field differences </li></ul></ul><ul><ul><li>Jacobian determinant of displacement field </li></ul></ul><ul><li>Measures on fixed and warped moving image </li></ul><ul><ul><li>Clamped RMS intensity differences </li></ul></ul><ul><ul><li>MAD intensity differences </li></ul></ul><ul><ul><li>MAX intensity differences </li></ul></ul><ul><ul><li>Normalized Mutual Information </li></ul></ul><ul><ul><li>Edge Overlap </li></ul></ul><ul><li>IMHO only first group says something about registration performance! </li></ul>
  12. 12. A Sample Evaluation <ul><li>Purpose: Evaluate intra-modality thoracic CT registration subject to breathing differences </li></ul><ul><li>2 data sets 256^3 </li></ul><ul><ul><li>NLM </li></ul></ul><ul><ul><li>NormalChestCTNoContrast </li></ul></ul><ul><ul><li>NCIA </li></ul></ul><ul><ul><li>LIDC 30047 </li></ul></ul><ul><li>Single choice of algorithm parameters </li></ul><ul><li>64 bit Opteron with 2.4GHz and 8GB RAM </li></ul>
  13. 13. A Sample Evaluation Simulated Data Difference to Original Standard Demons
  14. 14. A Sample Evaluation - Results <ul><li>Quantitative measures </li></ul><ul><ul><li>Large number (see paper) </li></ul></ul><ul><ul><li>Quantities are single numbers </li></ul></ul><ul><ul><ul><li>Problematic? </li></ul></ul></ul><ul><ul><li>Standard Demons & Diffeomorphic Demons very well </li></ul></ul><ul><ul><li>Algorithm Parameter Studies </li></ul></ul><ul><li>Visual Results (Problem Cases) </li></ul><ul><ul><li>Level Set motion -> artifacts </li></ul></ul><ul><ul><li>Fast Block Matching -> Implementation Issues (Test Framework) </li></ul></ul><ul><ul><li>Symmetric Demons </li></ul></ul>
  15. 15. Conclusion <ul><li>Open Science Evaluation Framwork presented </li></ul><ul><li>Intra-Modality Sample Evaluation shown </li></ul><ul><ul><li>Results should not be seen as final algorithm quality statements </li></ul></ul><ul><ul><ul><li>Framework has to grow… </li></ul></ul></ul><ul><ul><li>Useful for comparing & testing algorithms </li></ul></ul><ul><ul><li>Useful for parameter studies </li></ul></ul><ul><li>A small step towards establishing standardized protocol to gain clinical acceptance… </li></ul>
  16. 16. Further Work <ul><li>Common Web Repository (Hosted by Kitware?) </li></ul><ul><li>Upgrade by community effort </li></ul><ul><ul><li>More algorithms </li></ul></ul><ul><ul><li>More quality measures </li></ul></ul><ul><ul><ul><li>Segmentation based -> Problems for general applicability </li></ul></ul></ul><ul><ul><li>More synthetic deformations, including noise models </li></ul></ul><ul><li>Extension to inter-modality, inter-subject problems </li></ul><ul><li>Cooperate with NIREP? </li></ul><ul><li>How to prevent „evaluation framework tuning“? </li></ul>
  17. 17. Thank you for your attention!
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×