Your SlideShare is downloading. ×
A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms-8396
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms-8396


Published on

Published in: Economy & Finance

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms Martin Urschler, Stefan Kluckner, Horst Bischof Institute for Computer Graphics and Vision, Graz, University of Technology, Austria
  • 2. Motivation Validation very difficult!
    • Lack of direct ground truth
    • Lack of gold standard methods
    • - Highly ill-posed problem
    • Large space of possible solutions
    We claim: Standardizing Evaluation Protocols at least as important as Developing Novel Methods! -> Put framework to discussion: - Open-source community effort - Instantiate framework by showing sample evaluation Presented algorithm at last years MICCAI. Nonlinear Image Registration
  • 3. Contents
    • Problem Definition
    • Related Work & Similar Efforts
    • Open Science Nonlinear Registration Evaluation Framework
    • Sample Evaluation – Intra-subject Thorax CT
    • Conclusion & Outlook
  • 4. The Nonlinear Registration Problem
    • „ Find a deformable mapping h(x) aligning moving and fixed image such that a defined similarity criterium is minimized.“
    h(x) Base Illustration taken from ITK Software Guide. SSD, NCC, Mutual Information, … B-Spline, Thin-Plate Spline, Def Field, … Gradient Descent, BFGS, … Intensity-Based, Feature-Based, …
  • 5. Medical Applications of Nonlinear Image Registration
    • Angiography Studies
    • Anatomy & Function
      • Perfusion /Ventilation
    • Correction of Motion Artifacts
    • Studies of Shape Variation
    • Segmentation by Atlas Registration
    • Surgical Planning
    Lung Perfusion for Pulmonary Embolism Detection [Wildberger et al.] Validation Study: Atlas-based Brain Volume Segmentation from MRI Images [Ng02] – taken from ITK Documentation Inter- & Intra-Modality Inter- & Intra-Subject No Ground Truth or Gold Standards Highly Ill-Posed How should we reach clinical acceptance?
  • 6. Related Work on Evaluation (Frameworks) There exist some good ideas for evaluation in (Medical) Computer Vision! VALMET Segmentation Evaluation [Gerig et al] Community would benefit from open-source implementation! MICCAI 2007 Segmentation Challenge Workshop Multi-View 3D Reconstruction [Seitz et al] Stereo Reconstruction [Scharstein et al] Middlesbury Retrospective Evaluation of Intersubject Brain Registration [Hellier et al] Retrospective Rigid Registration Evaluation Project [West et al] NIREP (Inter-subject Brain-Registration Evaluation) [Christensen et al] Validation of Nonrigid Image Registration Using FEM [Schnabel et al] Segmentations are compared!
  • 7. Open Science Evaluation Framework
    • Algorithm Evaluation vs. Validation [Hellier et al]
      • Specific Problems: Lacking ground truth, ill-posedness, Lack of gold standard
      • General Problems: Noise, Partial Volume Effect, Interpolation, Numerics
    • Modular Evaluation Framework
      • Open-source, open-access, open-data, open protocols
    • Building Blocks:
    Registration Algorithms Public domain data sets Synthetic Deformations Similarity Measures Python Framework as Glue
  • 8. Open Science Evaluation Framework
  • 9. Synthetic Deformations Used
    • Simple synthetic transformation models
      • Regular grid with random deformations & TPS
      • Uniform periodic cosine transformation
      • Tailored to breathing difference registration:
        • Simulated Breathing Model
        • Synthetic Airway Tree Movement from Manual Correspondences
    • Increase number of models
      • Pool of synthetic transformations defined by pool of algorithms („bronze standard“ Glatard et al – MICCAI 2006)
      • Community effort needed
  • 10. Data Sets and Algorithms
    • Public domain data sets from
      • National Library of Medicine Dataset Collection (currently offline)
      • MIDAS data collection project
      • National Lung Cancer Archives (NCIA)
    • ITK Algorithms
      • Demons
      • Symmetric Demons
      • Level Set Motion
      • Curvature
      • Fast Block Matching (Workshop Contribution)
      • Diffeomorphic Demons (Workshop Contribution)
  • 11. Quantitative Measures
    • Measures on Displacement Fields
      • RMS of displacement field differences
      • MAD of displacement field differences
      • MAX of displacement field differences
      • Jacobian determinant of displacement field
    • Measures on fixed and warped moving image
      • Clamped RMS intensity differences
      • MAD intensity differences
      • MAX intensity differences
      • Normalized Mutual Information
      • Edge Overlap
    • IMHO only first group says something about registration performance!
  • 12. A Sample Evaluation
    • Purpose: Evaluate intra-modality thoracic CT registration subject to breathing differences
    • 2 data sets 256^3
      • NLM
      • NormalChestCTNoContrast
      • NCIA
      • LIDC 30047
    • Single choice of algorithm parameters
    • 64 bit Opteron with 2.4GHz and 8GB RAM
  • 13. A Sample Evaluation Simulated Data Difference to Original Standard Demons
  • 14. A Sample Evaluation - Results
    • Quantitative measures
      • Large number (see paper)
      • Quantities are single numbers
        • Problematic?
      • Standard Demons & Diffeomorphic Demons very well
      • Algorithm Parameter Studies
    • Visual Results (Problem Cases)
      • Level Set motion -> artifacts
      • Fast Block Matching -> Implementation Issues (Test Framework)
      • Symmetric Demons
  • 15. Conclusion
    • Open Science Evaluation Framwork presented
    • Intra-Modality Sample Evaluation shown
      • Results should not be seen as final algorithm quality statements
        • Framework has to grow…
      • Useful for comparing & testing algorithms
      • Useful for parameter studies
    • A small step towards establishing standardized protocol to gain clinical acceptance…
  • 16. Further Work
    • Common Web Repository (Hosted by Kitware?)
    • Upgrade by community effort
      • More algorithms
      • More quality measures
        • Segmentation based -> Problems for general applicability
      • More synthetic deformations, including noise models
    • Extension to inter-modality, inter-subject problems
    • Cooperate with NIREP?
    • How to prevent „evaluation framework tuning“?
  • 17. Thank you for your attention!
  • 18.  
  • 19.  
  • 20.