A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms-8396
Upcoming SlideShare
Loading in...5

A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms-8396







Total Views
Views on SlideShare
Embed Views



3 Embeds 25

http://www.insight-journal.org 20
http://insight-journal.org 3
http://www.slideshare.net 2



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms-8396 A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms-8396 Presentation Transcript

  • A Framework for Comparison and Evaluation of Nonlinear Intra-Subject Image Registration Algorithms Martin Urschler, Stefan Kluckner, Horst Bischof Institute for Computer Graphics and Vision, Graz, University of Technology, Austria
  • Motivation Validation very difficult!
    • Lack of direct ground truth
    • Lack of gold standard methods
    • - Highly ill-posed problem
    • Large space of possible solutions
    We claim: Standardizing Evaluation Protocols at least as important as Developing Novel Methods! -> Put framework to discussion: - Open-source community effort - Instantiate framework by showing sample evaluation Presented algorithm at last years MICCAI. Nonlinear Image Registration
  • Contents
    • Problem Definition
    • Related Work & Similar Efforts
    • Open Science Nonlinear Registration Evaluation Framework
    • Sample Evaluation – Intra-subject Thorax CT
    • Conclusion & Outlook
  • The Nonlinear Registration Problem
    • „ Find a deformable mapping h(x) aligning moving and fixed image such that a defined similarity criterium is minimized.“
    h(x) Base Illustration taken from ITK Software Guide. SSD, NCC, Mutual Information, … B-Spline, Thin-Plate Spline, Def Field, … Gradient Descent, BFGS, … Intensity-Based, Feature-Based, …
  • Medical Applications of Nonlinear Image Registration
    • Angiography Studies
    • Anatomy & Function
      • Perfusion /Ventilation
    • Correction of Motion Artifacts
    • Studies of Shape Variation
    • Segmentation by Atlas Registration
    • Surgical Planning
    Lung Perfusion for Pulmonary Embolism Detection [Wildberger et al.] Validation Study: Atlas-based Brain Volume Segmentation from MRI Images [Ng02] – taken from ITK Documentation Inter- & Intra-Modality Inter- & Intra-Subject No Ground Truth or Gold Standards Highly Ill-Posed How should we reach clinical acceptance?
  • Related Work on Evaluation (Frameworks) There exist some good ideas for evaluation in (Medical) Computer Vision! VALMET Segmentation Evaluation [Gerig et al] Community would benefit from open-source implementation! MICCAI 2007 Segmentation Challenge Workshop Multi-View 3D Reconstruction [Seitz et al] Stereo Reconstruction [Scharstein et al] Middlesbury Retrospective Evaluation of Intersubject Brain Registration [Hellier et al] Retrospective Rigid Registration Evaluation Project [West et al] NIREP (Inter-subject Brain-Registration Evaluation) [Christensen et al] Validation of Nonrigid Image Registration Using FEM [Schnabel et al] Segmentations are compared!
  • Open Science Evaluation Framework
    • Algorithm Evaluation vs. Validation [Hellier et al]
      • Specific Problems: Lacking ground truth, ill-posedness, Lack of gold standard
      • General Problems: Noise, Partial Volume Effect, Interpolation, Numerics
    • Modular Evaluation Framework
      • Open-source, open-access, open-data, open protocols
    • Building Blocks:
    Registration Algorithms Public domain data sets Synthetic Deformations Similarity Measures Python Framework as Glue
  • Open Science Evaluation Framework
  • Synthetic Deformations Used
    • Simple synthetic transformation models
      • Regular grid with random deformations & TPS
      • Uniform periodic cosine transformation
      • Tailored to breathing difference registration:
        • Simulated Breathing Model
        • Synthetic Airway Tree Movement from Manual Correspondences
    • Increase number of models
      • Pool of synthetic transformations defined by pool of algorithms („bronze standard“ Glatard et al – MICCAI 2006)
      • Community effort needed
  • Data Sets and Algorithms
    • Public domain data sets from
      • National Library of Medicine Dataset Collection (currently offline)
      • MIDAS data collection project
      • National Lung Cancer Archives (NCIA)
    • ITK Algorithms
      • Demons
      • Symmetric Demons
      • Level Set Motion
      • Curvature
      • Fast Block Matching (Workshop Contribution)
      • Diffeomorphic Demons (Workshop Contribution)
  • Quantitative Measures
    • Measures on Displacement Fields
      • RMS of displacement field differences
      • MAD of displacement field differences
      • MAX of displacement field differences
      • Jacobian determinant of displacement field
    • Measures on fixed and warped moving image
      • Clamped RMS intensity differences
      • MAD intensity differences
      • MAX intensity differences
      • Normalized Mutual Information
      • Edge Overlap
    • IMHO only first group says something about registration performance!
  • A Sample Evaluation
    • Purpose: Evaluate intra-modality thoracic CT registration subject to breathing differences
    • 2 data sets 256^3
      • NLM
      • NormalChestCTNoContrast
      • NCIA
      • LIDC 30047
    • Single choice of algorithm parameters
    • 64 bit Opteron with 2.4GHz and 8GB RAM
  • A Sample Evaluation Simulated Data Difference to Original Standard Demons
  • A Sample Evaluation - Results
    • Quantitative measures
      • Large number (see paper)
      • Quantities are single numbers
        • Problematic?
      • Standard Demons & Diffeomorphic Demons very well
      • Algorithm Parameter Studies
    • Visual Results (Problem Cases)
      • Level Set motion -> artifacts
      • Fast Block Matching -> Implementation Issues (Test Framework)
      • Symmetric Demons
  • Conclusion
    • Open Science Evaluation Framwork presented
    • Intra-Modality Sample Evaluation shown
      • Results should not be seen as final algorithm quality statements
        • Framework has to grow…
      • Useful for comparing & testing algorithms
      • Useful for parameter studies
    • A small step towards establishing standardized protocol to gain clinical acceptance…
  • Further Work
    • Common Web Repository (Hosted by Kitware?)
    • Upgrade by community effort
      • More algorithms
      • More quality measures
        • Segmentation based -> Problems for general applicability
      • More synthetic deformations, including noise models
    • Extension to inter-modality, inter-subject problems
    • Cooperate with NIREP?
    • How to prevent „evaluation framework tuning“?
  • Thank you for your attention!