Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Universitä t Zürich
San Diego Supercomputer Center
Computational Radiology Laboratory
Brigham & Women’s Hospital, Harvard ...
Upcoming SlideShare
Loading in...5
×

majumdar.siampp06.ppt

190

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
190
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "majumdar.siampp06.ppt"

  1. 1. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 1 Dynamic Data Driven Finite Element Modeling of Brain Shape Deformation During Neurosurgery A. Majumdar1 , D. Choi1 , P. Krysl2 , S. K. Warfield3 , N. Archip3 , K. Baldridge1,4 1 San Diego Supercomputer Center & 2 Structural Engineering Dept University of California San Diego 3 Computational Radiology Lab Brigham and Women’s Hospital Harvard Medical School 4 Universitä t Zürich Grants: NSF: ITR 0427183, 0426558; NIH:P41 RR13218, P01 CA67165, LM0078651, I3 grant (IBM)
  2. 2. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 2 Contents of Talk 1. Overview of Image Guided Neurosurgery and Dynamic Data Drive Application System 2. Biomechanical FEM solution 3. Briefly grid scheduling 4. Future : near-continuous DDDAS
  3. 3. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 3 1. Overview of Image Guided Neurosurgery and Dynamic Data Drive Application System
  4. 4. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 4 Neurosurgery Challenge • Challenges : • Remove as much tumor tissue as possible • Minimize the removal of healthy tissue • Avoid the disruption of critical anatomical structures • Know when to stop the resection process • Pre-op MRI compounded by the intra-operative brain shape deformation as a result of the surgical process • Important to quantify and correct for these deformations while surgery is in progress • Real-time constraints – provide images ~once/hour within few mins during surgery lasting ~6 hours
  5. 5. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 5 Intraoperative MRI Scanner at BWH (0.5 T)
  6. 6. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 6 Brain Deformation Before surgeryBefore surgery After surgeryAfter surgery
  7. 7. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 7 Tumor Ventricles Pre-operative ImagePre-operative Image Intra-operative image, after duraIntra-operative image, after dura opened and partial tumor resectionopened and partial tumor resection
  8. 8. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 8 Overall Process Before image guided neurosurgery During image guided neurosurgery Segmentation and Visualization Preoperative Planning of Surgical Trajectory Preoperative Data Acquisition Preoperative data Intraoperative MRI Segmentation Registration Surface matching Solve biomechanical Model for volumetric deformation Visualization Guide surgical process Tetrahedral FE mesh
  9. 9. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 9 Timeline of Image Acquisition and Analysis Time (min) Before surgery During surgery 0 10 20 30 40 Preop processes Intraop MRI Segmentation Registration Surface displacement Biomechanical simulation Visualization Surgical progress Action
  10. 10. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 10 Current DDDAS (Dynamic Data Driven Application System) Pre- and Intra-op 3D MRI (once/hr)Pre- and Intra-op 3D MRI (once/hr) LocalLocal computercomputer at BWHat BWH Crude linear elastic FEMCrude linear elastic FEM solutionsolution Merge pre- and intra-op vizMerge pre- and intra-op viz Intra-opsurgicalIntra-opsurgical decisionandsteerdecisionandsteer Segmentation, Registration,Segmentation, Registration, Surface Matching for BCSurface Matching for BC Once every hour or twoOnce every hour or two for a 6 hour surgeryfor a 6 hour surgery
  11. 11. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 11 Two Research Aspects • Parallel solution of the linear elastic biomechanical model for brain shape deformation during surgery • Grid Architecture – grid scheduling, on demand remote access to multi-teraflop machines, data transfer/sharing
  12. 12. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 12 2. Biomechanical FEM solution
  13. 13. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 13 Brief Concept of Biomechanical Model E d Fud T = +∫ ∫ 1 2 σ ε Ω Ω Ω Ω ε Assuming a linear elastic continuum with no initial stress or strains, theAssuming a linear elastic continuum with no initial stress or strains, the deformation energy of an elastic body submitted to eternally applieddeformation energy of an elastic body submitted to eternally applied forces :forces : F = F(x,y,z) is the vector representing the force applied to the elastic bodyF = F(x,y,z) is the vector representing the force applied to the elastic body u = u(x,y,z) is the displacement vector field we want to computeu = u(x,y,z) is the displacement vector field we want to compute is the strain vector = Lu and the stress vector linked to the strain vectoris the strain vector = Lu and the stress vector linked to the strain vector by the material constitutive equation.by the material constitutive equation. Linear isotropic elastic brain tissue is modeled with two parameters:Linear isotropic elastic brain tissue is modeled with two parameters: Young’s elasticity modulus and Poisson’s ratio.Young’s elasticity modulus and Poisson’s ratio. Introducing FE and some analysis,Introducing FE and some analysis, Ku = -FKu = -F (K is the rigidity matrix)(K is the rigidity matrix) The displacements at the boundary surface nodes are fixed to match those generated by theThe displacements at the boundary surface nodes are fixed to match those generated by the deformable surface model.deformable surface model. σ
  14. 14. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 14 Mesh Model with Brain Segmentation
  15. 15. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 15 Current and New Biomechanical Models • Current linear elastic material model RTBM • Advanced biomechanical model FAMULS (AMR) • Advanced model is based on conforming adaptive refinement method • Inspired by the theory of wavelets this refinement produces globally compatible meshes by construction • Replicate the linear elastic result produced by RTBM using FAMULS
  16. 16. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 16 FEM Mesh : FAMULS & RTBM RTBM (Uniform)RTBM (Uniform)FAMULS (AMR)FAMULS (AMR)
  17. 17. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 17 Deformation Simulation After Cut No – AMR FAMULSNo – AMR FAMULS 3 level AMR FAMULS3 level AMR FAMULS RTBMRTBM
  18. 18. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 18 Petsc setup • PetscMapCreateMPI(PETSC_COMM_WORL D,PETSC_DECIDE,n,&map) ; • MatCreateMPIAIJ(PETSC_COMM_WORLD,.. &K_global) ;
  19. 19. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 19 Domain decomposition • PetscMapGetLocalRange(map,&Istart,&Iend) • • for (each elements) {for (each dof in each nodes is in (lstart, lend)) if it is in the rage { ComputeShape(); ComputeBD(); MatSetValues(K_global,..ADD_VALUES); } }
  20. 20. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 20 Boundary condition • Prescribed forces: VecSetValues(F_global, nodeForces->NIndices, nodeForces- >Indices, nodeForces->Displacements, ADD_VALUES); • Prescribed displacements: (displacements on the surface obtained by active surface algorithm) MatZeroRows(K_global,ISBoundaryNodes,&one); VecSetValues(F_global, bc->NIndices,bc->Indices, bc->Displacements,INSERT_VALUES);
  21. 21. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 21 • KSPCreate(PETSC_COMM_WORLD,&ksp) • KSPSetOperators(ksp,K_global,K_global..) • KSPGetPC(ksp,&pc) • PCSetType(pc,PCBJACOBI) • KSPSetTolerances(ksp,1.e-7..) • KSPSetFromOptions(ksp) • KSPSolve(ksp,F_global,u_displ,&its) Solver setup
  22. 22. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 22 Parallel RTBM Performance (214035 tetrahedral elements) - 10.00 20.00 30.00 40.00 50.00 60.00 1 2 4 8 16 32 # of CPUs ElapsedTime(sec) IBM Power3 IA64 TeraGrid IBM Power4
  23. 23. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 23 Advanced Biomechanical Model • The current solver is based on small strain isotropic elastic principle • New biomechanical model • Inhomogeneous scalable non-linear hyper-elastic or visco-elastic model with AMR • Increase resolution close to the level of MRI voxels i.e. millions of FEM meshes • New high resolution complex model still has to meet the real time constraint of neurosurgery • Requires fast access to remote multi-teraflop systems
  24. 24. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 24
  25. 25. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 25
  26. 26. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 26
  27. 27. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 27 3. Briefly Grid Scheduling
  28. 28. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 28 On-demand Scheduling Experiment on 5 TeraGrid Clusters • The real-time constraint of this application requires that data transfer and simulation altogether take about 10 mins, otherwise these results are not of use to surgeons • Assume simulation and data transfer (both ways) together takes 10 mins and data transfer takes 4 mins • Leaves 6 mins for biomechanical simulation on remote HPC machines • Assume biomechanical model is scalable i.e. better results achieved on higher number of processors • Objective : • Get simulation done in 6 mins • Get maximum number of processors available within 6 mins • Allow 4 mins to wait in the queue; this leaves 2 mins for actual simulation
  29. 29. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 29 Experiment Characteristics • Flooding scheduler approach – experiment 1: • Simultaneously submit 8, 16, 32, 64, 128 procs jobs to multiple clusters - SDSC DataStar, SDSC TG, NCSA TG, ANL TG, PSC TG • When a higher count job starts (at any center) kill all the lower CPU count jobs at all the other centers • Results : out of 1464 job submissions over ~7 days, only 6 failed giving success of 99.59%; 128 CPU jobs ran greater than 50% of time; at least 64 CPU jobs ran more than 80% of time • Next slide gives time varying behavior with 6 hour intervals for this experiment • 4 other experiments were performed by taking out some of the successful clusters as well as taking scheduler cycle time into account on DataStar • As number of clusters were reduced, success rate goes down
  30. 30. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 30
  31. 31. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 31 4. Future : Near-continuous DDDAS
  32. 32. Universitä t Zürich San Diego Supercomputer Center Computational Radiology Laboratory Brigham & Women’s Hospital, Harvard Medical School SIAM PP06 – San FranciscoSIAM PP06 – San Francisco 32 Current DDDAS vs. (future) near-continuous DDDAS • Problem of current DDDAS: • Using current DDDAS procedure, surgeon does not have near-continuous brain deformation info • It takes more than 20 minutes to have whole 3d scan, segmentation, surface matching and FEM solution • Solution is to extend to near continuous DDDAS: • DDDAS approach to provide near-continuous closed loop registration updates using near-continuous 2D MRI slice scans
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×