0
Applications: Using Insight(itk) and The Visualization Toolkit(vtk) Bill Lorensen Jim Miller GE Research Niskayuna, NY Nov...
Agenda <ul><li>Approaches to interfacing vtk and itk </li></ul><ul><li>Case Studies / Demos </li></ul><ul><ul><li>Registra...
Motivation <ul><li>itk has no visualization capabilities </li></ul><ul><li>itk has some interesting segmentation and regis...
vtk/itk Interfaces <ul><li>Connect vtk pipelines to itk pipelines </li></ul><ul><li>Image filter class hierarchies </li></...
Connect pipelines  - vtk to itk <ul><li>vtkImageExport *movingExporter = vtkImageExport::New(); </li></ul><ul><li>movingEx...
Connect Pipelines <ul><li>Implements itk pipeline protocol, invoking vtk callbacks </li></ul><ul><li>PropagateRequestedReg...
vtkITK Class Hierarchies <ul><li>Class Hierarchy </li></ul><ul><li>vtkImageToImageFilter </li></ul><ul><ul><li>vtkITKImage...
vtkITKImageToImageFilter  <ul><li>Changes exception macros to errors/warnings </li></ul><ul><li>Delegates </li></ul><ul><u...
vtkITKImageToImageFilterFF <ul><li>Defines Input/Output typedefs </li></ul><ul><ul><li>typedef float InputImagePixelType; ...
vtkITKDiscreteGaussianImageFilter <ul><li>Instantiates filter </li></ul><ul><li>Delegates filter’s member data set’s/get’s...
vtk Filters Implemented with itk algorithms <ul><li>vtkITKMutualInformationTransform </li></ul><ul><li>Implement InternalU...
Illustrative Examples Registration
Motivation for Registration <ul><li>Intra-modality </li></ul><ul><ul><li>Alzheimer’s </li></ul></ul><ul><ul><ul><li>Longit...
Mutual Information Registration <ul><li>Computes “mutual information” between two datasets, a reference and target </li></...
Longitudinal MRI Study <ul><li>Register multiple volumetric MRI datasets of a patient taken over an extended time </li></u...
Approach <ul><li>Validate the algorithm </li></ul><ul><li>Pick a set of parameters that can be used across all the studies...
The MI Algorithm Parameter Space <ul><li>Learning Rate </li></ul><ul><li>Standard Deviation of Parzen Window estimate </li...
 
 
 
Division of Labor vtk itk vtk Read data Export data Import Data Normalize data Shrink Register Report transform Read data ...
The Pipeline <ul><li>vtkImageReader </li></ul><ul><li>vtkImageExport </li></ul><ul><li>itk::vtkImporter </li></ul><ul><li>...
Oregon Data <ul><li>25 Registrations </li></ul><ul><li>13 Subjects, some with AD </li></ul><ul><li>Qualitative comparison ...
Longitudinal MRI No Registration Checkerboard Source Original image Difference Target Original image
Longitudinal MRI Registration Checkerboard Source Original image Difference Target Original image
Multi Field MRI Data <ul><li>Register 1.5T and 3T to 4T data </li></ul><ul><li>Resampled 1.5T and 3T to correspond to the ...
1.5T vs 4T MRI No Registration Checkerboard Source Original Image Difference Target Original Image
1.5T vs 4T MRI Registration Checkerboard Source Original Image Difference Target Original Image
3D Visualization of the same subject Scanned with different MR field Strengths   4T 3T 1.5T All Registered To 4T
CT Lung Longitudinal Study <ul><li>Register two CT exams of the same patient taken at two different times </li></ul><ul><l...
Lung CT No Registration Checkerboard Source Original Image Difference Target Original Image
Lung CT Registration Checkerboard Source Original Image Difference Target Original Image
 
microPet/Volume CT
microPet/Volume CT
Demo CT/MR fusion
Implementing a vtk Transform with itk <ul><li>vtkITKMutualInformationTransform </li></ul><ul><li>DemoMutual.tcl </li></ul>
Visible Woman Registration Case Study Bill Lorensen, GE Research Peter Ratiu, Brigham and Womens Hospital
Agenda <ul><li>Problem Statement </li></ul><ul><li>Methods </li></ul><ul><li>Results </li></ul><ul><li>Discussion </li></ul>
Problem Statement <ul><li>Register the Visible Human Female Computed Tomography and MRI Data to the cryo-section data </li...
Methods <ul><li>NormalizeImageFilter (itk) </li></ul><ul><li>Mutual Information Registration (itk) </li></ul><ul><ul><li>I...
Methods <ul><li>Multiple registration runs with increasing scales </li></ul><ul><li>Both RGB and CT data scaled to zero me...
CT Results <ul><li>Initial vertical translation of 20 mm </li></ul><ul><li>Checkerboard display of two modalities after ea...
 
 
 
 
 
 
 
 
 
CT Results <ul><li>Generated isosurfaces of skin and bone from CT data </li></ul><ul><li>Transformed according to computed...
 
 
 
 
 
 
 
 
CT Results <ul><li>CT Landmarks transformed by Mutual Information matrix into RGB coordinate system </li></ul><ul><li>Comp...
 
MR Results <ul><li>Initial transform computed using experts anatomical landmarks as correspondences </li></ul><ul><li>Same...
 
 
 
 
 
 
 
 
 
 
MRI Results <ul><li>MRI Landmarks transformed by Mutual Information matrix into RGB coordinate system </li></ul><ul><li>Co...
 
CT Discussion <ul><li>CT to RGB rigid body registration is a good first step for subsequent deformable registration </li><...
MRI Discussion <ul><li>MRI to RGB registration was not effective </li></ul><ul><ul><li>Too much anatomical distortion occu...
Examples Segmentation
Watershed Segmentation <ul><li>I/O and Visualization with vtk </li></ul><ul><li>Image processing and segmentation with itk...
Demo Watershed
Applications: Using Insight(itk) and The Visualization Toolkit(vtk) Bill Lorensen Jim Miller GE Research Niskayuna, NY Nov...
Upcoming SlideShare
Loading in...5
×

ITK Tutorial Presentation Slides-950

3,877

Published on

http://hdl.handle.net/1926/161

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,877
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
103
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "ITK Tutorial Presentation Slides-950"

  1. 1. Applications: Using Insight(itk) and The Visualization Toolkit(vtk) Bill Lorensen Jim Miller GE Research Niskayuna, NY November 8, 2001
  2. 2. Agenda <ul><li>Approaches to interfacing vtk and itk </li></ul><ul><li>Case Studies / Demos </li></ul><ul><ul><li>Registration </li></ul></ul><ul><ul><ul><li>MR/MR </li></ul></ul></ul><ul><ul><ul><li>CT/MR </li></ul></ul></ul><ul><ul><li>Segmentation </li></ul></ul><ul><ul><ul><li>Watershed </li></ul></ul></ul>
  3. 3. Motivation <ul><li>itk has no visualization capabilities </li></ul><ul><li>itk has some interesting segmentation and registration algorithms </li></ul><ul><li>Both itk and vtk have a pipeline architecture </li></ul><ul><li>vtk has a nice wrapping facility to interpreters like tcl and python </li></ul><ul><li>Avoid code duplication and use “best of breed” from each toolkit. </li></ul>
  4. 4. vtk/itk Interfaces <ul><li>Connect vtk pipelines to itk pipelines </li></ul><ul><li>Image filter class hierarchies </li></ul><ul><li>Implement vtk filters with itk algorithms </li></ul><ul><li>Callbacks for filter progress </li></ul>
  5. 5. Connect pipelines - vtk to itk <ul><li>vtkImageExport *movingExporter = vtkImageExport::New(); </li></ul><ul><li>movingExporter->SetInput(movingReader->GetOutput()); </li></ul><ul><li>typedef itk::VTKImageImport<InputType> ImageImportType; </li></ul><ul><li>ImageImportType::Pointer movingImporter = ImageImportType::New(); </li></ul><ul><li>ConnectPipelines(movingExporter, movingImporter); </li></ul>InsightApplications/vtkITK/Common/vtkITKUtility.h
  6. 6. Connect Pipelines <ul><li>Implements itk pipeline protocol, invoking vtk callbacks </li></ul><ul><li>PropagateRequestedRegion() </li></ul><ul><li>UpdateOutputInformation() </li></ul><ul><li>GenerateOutputInformation() </li></ul><ul><li>GenerateData() </li></ul>
  7. 7. vtkITK Class Hierarchies <ul><li>Class Hierarchy </li></ul><ul><li>vtkImageToImageFilter </li></ul><ul><ul><li>vtkITKImageToImageFilter </li></ul></ul><ul><ul><ul><li>vtkITKImageToImageFilterFF </li></ul></ul></ul><ul><ul><ul><li>vtkITKImageToImageFilterUSUS </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><ul><ul><ul><li>vtkITKDiscreteGaussianImageFilter </li></ul></ul></ul></ul>
  8. 8. vtkITKImageToImageFilter <ul><li>Changes exception macros to errors/warnings </li></ul><ul><li>Delegates </li></ul><ul><ul><li>Modified </li></ul></ul><ul><ul><li>DebugOn/Off </li></ul></ul><ul><ul><li>SetInput/Output </li></ul></ul><ul><ul><li>Update </li></ul></ul>
  9. 9. vtkITKImageToImageFilterFF <ul><li>Defines Input/Output typedefs </li></ul><ul><ul><li>typedef float InputImagePixelType; </li></ul></ul><ul><ul><li>typedef float OutputImagePixelType; </li></ul></ul><ul><ul><li>typedef itk::Image<InputImagePixelType, 3> InputImageType; </li></ul></ul><ul><ul><li>typedef itk::Image<OutputImagePixelType, 3> OutputImageType; </li></ul></ul><ul><li>Constructor </li></ul><ul><ul><li>Creates importers/exporters </li></ul></ul><ul><ul><li>Connects vtk and itk pipelines </li></ul></ul>
  10. 10. vtkITKDiscreteGaussianImageFilter <ul><li>Instantiates filter </li></ul><ul><li>Delegates filter’s member data set’s/get’s </li></ul>
  11. 11. vtk Filters Implemented with itk algorithms <ul><li>vtkITKMutualInformationTransform </li></ul><ul><li>Implement InternalUpdate() </li></ul><ul><ul><li>vtkImageExport </li></ul></ul><ul><ul><li>itk::VTKImageImport </li></ul></ul><ul><ul><li>ConnectPipelines() </li></ul></ul><ul><ul><li>Convert vtk transform to quaternion </li></ul></ul><ul><ul><li>itk::RegistrationMethod </li></ul></ul><ul><ul><li>Convert itk quaternion to vtk transform </li></ul></ul>
  12. 12. Illustrative Examples Registration
  13. 13. Motivation for Registration <ul><li>Intra-modality </li></ul><ul><ul><li>Alzheimer’s </li></ul></ul><ul><ul><ul><li>Longitudinal analysis </li></ul></ul></ul><ul><ul><li>CT Lung </li></ul></ul><ul><ul><ul><li>Side-by-side slice reading </li></ul></ul></ul><ul><li>Inter-modality </li></ul><ul><ul><li>PET/MR/CT/US/XRAY Tomo </li></ul></ul><ul><ul><ul><li>Fusion and analysis </li></ul></ul></ul>
  14. 14. Mutual Information Registration <ul><li>Computes “mutual information” between two datasets, a reference and target </li></ul><ul><ul><li>MI(X,Y) = H(X) + H(Y) – H(X,Y) </li></ul></ul><ul><li>Small parameter set </li></ul><ul><li>Developed by Sandy Wells (BWH) and Paul Viola (MIT) in 1995 </li></ul><ul><li>Defacto standard for automatic, intensity based registration </li></ul>
  15. 15. Longitudinal MRI Study <ul><li>Register multiple volumetric MRI datasets of a patient taken over an extended time </li></ul><ul><li>Create a batch processing facility to process dozens of datasets </li></ul><ul><li>Resample the datasets </li></ul>
  16. 16. Approach <ul><li>Validate the algorithm </li></ul><ul><li>Pick a set of parameters that can be used across all the studies </li></ul><ul><li>For each pair of datasets </li></ul><ul><ul><li>Perform registration </li></ul></ul><ul><ul><li>Output a transform </li></ul></ul><ul><li>View the resampled source dataset in context with the target dataset </li></ul>
  17. 17. The MI Algorithm Parameter Space <ul><li>Learning Rate </li></ul><ul><li>Standard Deviation of Parzen Window estimate </li></ul><ul><li>Number of Samples for density estimate </li></ul><ul><li>Number of Iterations </li></ul><ul><li>Image variance </li></ul><ul><li>Translation scale factors </li></ul><ul><li>GE’s Design for Six Sigma (DFSS) Design of Experiments Tool </li></ul>
  18. 21. Division of Labor vtk itk vtk Read data Export data Import Data Normalize data Shrink Register Report transform Read data Reslice Display MIRegistration.cxx MultiCompare.tcl
  19. 22. The Pipeline <ul><li>vtkImageReader </li></ul><ul><li>vtkImageExport </li></ul><ul><li>itk::vtkImporter </li></ul><ul><li>itk::NormalizeImageFilter </li></ul><ul><li>itk::ShrinkImageFilter </li></ul><ul><li>itk::ImageRegistrationMethod </li></ul><ul><li>vtkTransform </li></ul>
  20. 23. Oregon Data <ul><li>25 Registrations </li></ul><ul><li>13 Subjects, some with AD </li></ul><ul><li>Qualitative comparison </li></ul><ul><li>One set of parameters for all studies </li></ul><ul><li>Data Courtesy of Jeffrey Kaye, Oregon Health Sciences Center </li></ul>
  21. 24. Longitudinal MRI No Registration Checkerboard Source Original image Difference Target Original image
  22. 25. Longitudinal MRI Registration Checkerboard Source Original image Difference Target Original image
  23. 26. Multi Field MRI Data <ul><li>Register 1.5T and 3T to 4T data </li></ul><ul><li>Resampled 1.5T and 3T to correspond to the 4T sampling </li></ul><ul><li>Volume rendering of the 3 datasets from the same view </li></ul><ul><li>Demo mri1.bat, MultiCompareAD.tcl </li></ul>InsightApplications/MRIRegistration
  24. 27. 1.5T vs 4T MRI No Registration Checkerboard Source Original Image Difference Target Original Image
  25. 28. 1.5T vs 4T MRI Registration Checkerboard Source Original Image Difference Target Original Image
  26. 29. 3D Visualization of the same subject Scanned with different MR field Strengths 4T 3T 1.5T All Registered To 4T
  27. 30. CT Lung Longitudinal Study <ul><li>Register two CT exams of the same patient taken at two different times </li></ul><ul><li>Side-by-side synchronized view for visual comparison </li></ul>
  28. 31. Lung CT No Registration Checkerboard Source Original Image Difference Target Original Image
  29. 32. Lung CT Registration Checkerboard Source Original Image Difference Target Original Image
  30. 34. microPet/Volume CT
  31. 35. microPet/Volume CT
  32. 36. Demo CT/MR fusion
  33. 37. Implementing a vtk Transform with itk <ul><li>vtkITKMutualInformationTransform </li></ul><ul><li>DemoMutual.tcl </li></ul>
  34. 38. Visible Woman Registration Case Study Bill Lorensen, GE Research Peter Ratiu, Brigham and Womens Hospital
  35. 39. Agenda <ul><li>Problem Statement </li></ul><ul><li>Methods </li></ul><ul><li>Results </li></ul><ul><li>Discussion </li></ul>
  36. 40. Problem Statement <ul><li>Register the Visible Human Female Computed Tomography and MRI Data to the cryo-section data </li></ul><ul><li>Validate the use of the itk Mutual Information Registration algorithm on data of differing modalities </li></ul><ul><li>Assess the performance of the registration using landmarks selected by an anatomical expert </li></ul>
  37. 41. Methods <ul><li>NormalizeImageFilter (itk) </li></ul><ul><li>Mutual Information Registration (itk) </li></ul><ul><ul><li>Implemented as a vtk Transform </li></ul></ul><ul><li>Marching Cubes (vtk) </li></ul><ul><li>Checkerboard Display (vtk) </li></ul>
  38. 42. Methods <ul><li>Multiple registration runs with increasing scales </li></ul><ul><li>Both RGB and CT data scaled to zero mean and unit variance </li></ul><ul><li>1/16, 1/8, ¼, ½, 1 scale </li></ul><ul><li>.01, .01, .004, .004, .001 learning rate </li></ul><ul><li>2000 iterations at each resolution </li></ul>
  39. 43. CT Results <ul><li>Initial vertical translation of 20 mm </li></ul><ul><li>Checkerboard display of two modalities after each resolution run </li></ul>
  40. 53. CT Results <ul><li>Generated isosurfaces of skin and bone from CT data </li></ul><ul><li>Transformed according to computed transformation </li></ul><ul><li>At each triangle vertex, sample the RGB data to color the vertex </li></ul>
  41. 62. CT Results <ul><li>CT Landmarks transformed by Mutual Information matrix into RGB coordinate system </li></ul><ul><li>Compute distance between CT and RGB points at each iteration </li></ul>
  42. 64. MR Results <ul><li>Initial transform computed using experts anatomical landmarks as correspondences </li></ul><ul><li>Same parameter schedule as CT/RGB </li></ul>
  43. 75. MRI Results <ul><li>MRI Landmarks transformed by Mutual Information matrix into RGB coordinate system </li></ul><ul><li>Compute distance between MRI and RGB points at each iteration </li></ul>
  44. 77. CT Discussion <ul><li>CT to RGB rigid body registration is a good first step for subsequent deformable registration </li></ul>
  45. 78. MRI Discussion <ul><li>MRI to RGB registration was not effective </li></ul><ul><ul><li>Too much anatomical distortion occurred between the MR and RGB acquisitions </li></ul></ul>
  46. 79. Examples Segmentation
  47. 80. Watershed Segmentation <ul><li>I/O and Visualization with vtk </li></ul><ul><li>Image processing and segmentation with itk </li></ul><ul><li>User interface with tk </li></ul><ul><li>Prototype in tcl </li></ul>
  48. 81. Demo Watershed
  49. 82. Applications: Using Insight(itk) and The Visualization Toolkit(vtk) Bill Lorensen Jim Miller GE Research Niskayuna, NY November 8, 2001
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×