SP SciComp 6, Berkeley, August 23, 2002
Parallel Medical and Genomics Applications on Power3
and Power4 Machines
Amit Maju...
SP SciComp 6, Berkeley, August 23, 2002
IBM Parallel Machines : compute nodes
• Power2 and Power3 at CAC University of Mic...
SP SciComp 6, Berkeley, August 23, 2002
Application I : Brain Deformation Simulation in Image
Guided Neurosurgery
SP SciComp 6, Berkeley, August 23, 2002
Brain Deformation Simulation in Image Guided
Neurosurgery
• Challenge faced by neu...
SP SciComp 6, Berkeley, August 23, 2002
Algorithm
• Project preoperative image onto intraoperative images
• Allows fusion ...
SP SciComp 6, Berkeley, August 23, 2002
Overall Process
• Before Image Guided Neurosurgery :
• During Image Guided Neurosu...
SP SciComp 6, Berkeley, August 23, 2002
Volumetric Biomechanical Simulation of Brain
Deformation
• During surgery brain sh...
SP SciComp 6, Berkeley, August 23, 2002
Biomechanical Simulation Equations
ndeformatiobrain
cvolumetrifor thesolvetominimi...
SP SciComp 6, Berkeley, August 23, 2002
Biomechanical Simulation Equations
• Mathematical operations, plugging in of inter...
SP SciComp 6, Berkeley, August 23, 2002
Signa SP (GE Medical Systems)
R. Pergolizzi
SP SciComp 6, Berkeley, August 23, 2002
Brain shift (1)
F. Talos
SP SciComp 6, Berkeley, August 23, 2002
Brain shift (2)
F. Talos
SP SciComp 6, Berkeley, August 23, 2002
Linear System Solver
• The PETSc package is used to solve the linear system
• Gene...
SP SciComp 6, Berkeley, August 23, 2002
Performance on Power3 and Power4 Machines
0.10
1.00
10.00
100.00
1 2 4 8 16 32
# o...
SP SciComp 6, Berkeley, August 23, 2002
Timing Table
1 cpu 2 cpu 4 cpu 8 cpu 16 cpu 32 cpu
Tot time p4 30.01 21.11 13.77 9...
SP SciComp 6, Berkeley, August 23, 2002
Observations
• Power4 timings:
• Unexpected timings on Power4 16 and 32 processors...
SP SciComp 6, Berkeley, August 23, 2002
Application II : Monte Carlo SPECT Imaging
SP SciComp 6, Berkeley, August 23, 2002
• Radionuclide therapy
• SPECT imaging in radionuclide therapy
• Monte Carlo simul...
SP SciComp 6, Berkeley, August 23, 2002
Radionuclide therapy
• Cancer cells are sterilized using internally administered
i...
SP SciComp 6, Berkeley, August 23, 2002
I-131 Radionuclide therapy
• I-131 Radioimmunotherapy (RIT) : I-131 labeled
antibo...
SP SciComp 6, Berkeley, August 23, 2002
I-131 RIT for NHL at University of Michigan
• Phase II clinical trial: Out of 76 p...
SP SciComp 6, Berkeley, August 23, 2002
I-131 imaging
• Single photon emission computed tomography
(SPECT) imaging using a...
SP SciComp 6, Berkeley, August 23, 2002
Activity quantification
• For accurate quantification, SPECT data has to be
compen...
SP SciComp 6, Berkeley, August 23, 2002
What is the role of M.C. simulation in SPECT Imaging?
• Monte Carlo is used primar...
SP SciComp 6, Berkeley, August 23, 2002
M.C. simulation of SPECT imaging
• SIMIND Monte Carlo code
• Complete photon trans...
SP SciComp 6, Berkeley, August 23, 2002
SIMIND verification using thorax phantom
• Measurement with
experimental phantom
•...
SP SciComp 6, Berkeley, August 23, 2002
SIMIND verification: thorax phantom
0
2 50
5 00
7 50
1 0 0 0
Intensity
1 5 2 0 2 5...
SP SciComp 6, Berkeley, August 23, 2002
M.C. applications: SPECT quantification accuracy using
voxel-man phantom
• Clinica...
SP SciComp 6, Berkeley, August 23, 2002
Work in progress: Monte Carlo generated patient-
specific recovery coefficients
Pa...
SP SciComp 6, Berkeley, August 23, 2002
A parallel M.C. code for SPECT: motivation
• Fast code for accurate I-131 Monte Ca...
SP SciComp 6, Berkeley, August 23, 2002
SIMIND parallelization
• In the present application the photon histories are
indep...
SP SciComp 6, Berkeley, August 23, 2002
Start
Read data
Reset history counter
Start history
photon transport
More projs?
M...
SP2 timing results of
one SPECT projection of the voxel-man phantom
Small (8.4x10
7
photons/projection)
Medium (8.4x10
8
p...
SP SciComp 6, Berkeley, August 23, 2002
Power3 timing results of
one SPECT projection of the voxel-man phantom
(16.8x107
p...
SP SciComp 6, Berkeley, August 23, 2002
Power4 Timing of a different Monte Carlo
photon transport code
May, 2002 on IBM Sa...
SP SciComp 6, Berkeley, August 23, 2002
More Realistic Simulations
• A 60-projection SPECT simulation of the voxel-man
pha...
SP SciComp 6, Berkeley, August 23, 2002
Effect of Parallel Computing
• Monte Carlo has enabled to evaluate and
improve the...
SP SciComp 6, Berkeley, August 23, 2002
Application III : Parallel Proteomics Application
SP SciComp 6, Berkeley, August 23, 2002
• Sequest is a proteomics application used to analyze mass
spectrometer output and...
SP SciComp 6, Berkeley, August 23, 2002
Chloroquine-resistant
Chloroquine-sensitive
 More individuals on the planet with ...
SP SciComp 6, Berkeley, August 23, 2002
A Proteomics View of the Malaria Parasite Life Cycle
OOne Genome:
~6,000 genes
D...
SP SciComp 6, Berkeley, August 23, 2002
Importance of a malaria vaccine
The battle against malaria hampered by the emergen...
SP SciComp 6, Berkeley, August 23, 2002
SEQUEST®
DTASelect & Contrast
> 1,000 Proteins Identified
Tandem Mass Spectrometer...
SP SciComp 6, Berkeley, August 23, 2002
Single Processor Optimization
• ( Timings on 375 MHz IBM Power3 procs.TF:ThermoFin...
SP SciComp 6, Berkeley, August 23, 2002
Code parallelization
•Parallel version of Sequest has been developed using MPI and...
SP SciComp 6, Berkeley, August 23, 2002
Power 3 Timing and Speedup
# files # proc Time (sec) Speedup
2227 32 2724 – 3031 1...
SP SciComp 6, Berkeley, August 23, 2002
Power3 versus Power4 Speedup on 32 procs
# files Machine Time (sec) Speedup
2227 P...
SP SciComp 6, Berkeley, August 23, 2002
Impact on science
Calculations (anlysis of part of a whole cell lysate of a
merozo...
SP SciComp 6, Berkeley, August 23, 2002
Final Comments
• Genomics community has already extensively used
supercomputing ca...
Upcoming SlideShare
Loading in …5
×

``Parallel Medical and Genomics Applications on Power3 and ...

313 views
216 views

Published on

Published in: Health & Medicine, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
313
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

``Parallel Medical and Genomics Applications on Power3 and ...

  1. 1. SP SciComp 6, Berkeley, August 23, 2002 Parallel Medical and Genomics Applications on Power3 and Power4 Machines Amit Majumdar San Diego Supercomputer Center - UCSD Application I : Brain Deformation Simulation in Image Guided Neurosurgery Simon K. Warfield1 , Florin Talos1,2 , Alida Tei1,3 , Aditya Bharatha1,4 , Arya Nabavi1,2 , Matthieu Ferrante1,5 , Peter McL. Black2 , Ferenc A. Jolesz1 , Ron Kikinis1 , Corey Kemper1 1 Surgical Planing Laboratory and 2 Dept. of Surgery Brigham and Women’s Hospital and Harvard Medical School 3 Massashusetts Institute of Technology 4 University of Toronto Medical School 5 Telecom. Lab., Universite’ Catholique de Louvain, Belgium Application II : Monte Carlo SPECT Imaging Yuni Dewaraja1 , Kenneth Koral1 , Abhijit Bose2 , Michael Ljungberg3 1 Nuclear Medicine, University of Michigan 2 Center for Advanced Computing, University of Michigan 3 Department of Radiation Physics, Univ. of Lund, Sweden Application III : Parallel Proteomics Application John R. Yates1 , Daniel J. Carucci2 ,Giri Chukkapalli3 , Robert Sinkovits3 1 The Scripps Research Institute 2 Naval Medical Research Center, US NAVY 3 SDSC
  2. 2. SP SciComp 6, Berkeley, August 23, 2002 IBM Parallel Machines : compute nodes • Power2 and Power3 at CAC University of Michigan : • 176 160Mhz Power2; 1Gbytes and 256 Mbytes memory; three 110Mbytes/sec HP switches • 3 Power3 nodes, 8 cpu/node, 375 Mhz Power3,8 Gbytes/node; 420 Mbytes/sec Colony switch • Power3 at SDSC : 144 nodes with 8zcpu/node, 375 Mhz Power3; 4 Gbytes/node; 350 Mbytes/sec Colony switch • Power3 at NAVO MSRC : 334 nodes with 4 cpu/node, 375 Mhs • Power4 at TACC University of Texas : 4 Regatta-HPC frames; 16 cpu/node; 1.3 Ghz ; 32 Gbytes/node (can be 64 procs 128 Gbytes memory machine by LL); high-speed dual-plane IBM SP Switch2
  3. 3. SP SciComp 6, Berkeley, August 23, 2002 Application I : Brain Deformation Simulation in Image Guided Neurosurgery
  4. 4. SP SciComp 6, Berkeley, August 23, 2002 Brain Deformation Simulation in Image Guided Neurosurgery • Challenge faced by neurosurgeons • Remove as much as possible tumor tissue while minimizing removal of healthy tissue • Avoid critical anatomical structures • Real Time Brain Mapping • Enhanced visualization of tumor and critical brain structures • Align preoperatively acquired image data with intraoperative images of patient’s brain during surgery • Real time constraints • The code must meet real-time constraints of neurosurgery – provide images within few minutes few times during surgery lasting few hours
  5. 5. SP SciComp 6, Berkeley, August 23, 2002 Algorithm • Project preoperative image onto intraoperative images • Allows fusion of images from multiple imaging modalities and with multiple contrast types • Tracks surfaces of key structures in intraoperatively acquired images – allows projection of preoperative images into the patient’s brain configuration during surgery • A volumetric deformation field is inferred from the surface changes • The field captures nonrigid deformations of the shape of the brain due to brain swelling, cerebrospinal fluid loss, anaesthetic agents and actions of neurosurgeon • Current model uses linear elastic material model to represent brain
  6. 6. SP SciComp 6, Berkeley, August 23, 2002 Overall Process • Before Image Guided Neurosurgery : • During Image Guided Neurosurgery : Segmentation and Visualization Preoperative Planning of Surgical Trajectory Preoperative Data Acquisition Preoperative data Intraoperative MRI Segmentation Registration Surface matching Solve biomechanical Model for volumetric deformation Visualization Surgical process
  7. 7. SP SciComp 6, Berkeley, August 23, 2002 Volumetric Biomechanical Simulation of Brain Deformation • During surgery brain shape changes due to surgical intervention • During surgery surgeon can acquire new volumetric MRI to review current configuration of the entire brain • Volumetric Biomechanical Simulation of Brain Deformation • Match surface from earlier acquisition to the new acquisition • Infer volumetric deformation based upon the surface correspondences • Apply forces to the volumetric model that will produce the same displacement field at the surface as was obtained by the surface matching • Biomechanical model allows the computation of the deformation throughout the volume
  8. 8. SP SciComp 6, Berkeley, August 23, 2002 Biomechanical Simulation Equations ndeformatiobrain cvolumetrifor thesolvetominimizedisequationEnergy meshesahedralwith tetrdoneistiondiscretizaelementFinite operatorlinearaisL;uL propertymaterialngrepresentimatrixelastictheisD D),,,,,( vectorstraintheis;vectorstresstheis computetofieldvectorntdisplacemethe:)z,y,x(uu bodyelasticthetoappliedforces:)z,y,x(FF udTFdT 2 1 E T T xzyzxyzyx =ε ε=τττσσσ=σ εσ = = Ω∫ Ω +Ωε∫ Ω σ=
  9. 9. SP SciComp 6, Berkeley, August 23, 2002 Biomechanical Simulation Equations • Mathematical operations, plugging in of interpolation of nodes in terms of linear functions etc. etc. finally gives : Ku = -F • K is the stiffness matrix. • Displacement at the boundary surface nodes are fixed to match those generated by the active surface model • The force vector F is set equal to the displacement vector for the boundary nodes : F = • Now solving matrix system for unknown displacement produces deformation field for the entire mesh that matches prescribed displacements at the boundary u~
  10. 10. SP SciComp 6, Berkeley, August 23, 2002 Signa SP (GE Medical Systems) R. Pergolizzi
  11. 11. SP SciComp 6, Berkeley, August 23, 2002 Brain shift (1) F. Talos
  12. 12. SP SciComp 6, Berkeley, August 23, 2002 Brain shift (2) F. Talos
  13. 13. SP SciComp 6, Berkeley, August 23, 2002 Linear System Solver • The PETSc package is used to solve the linear system • Generalized Minimal Residual (GMRES) solver with block Jacobi preconditioning • The rows of matrix are divided equally amongst CPUs • Global matrix is assembled in parallel • Each CPU assembles the local matrix for each element in its subdomain • Each CPU has equal # of rows to process • Due to irregular connectivity of the meshes some CPUs may do more work than others
  14. 14. SP SciComp 6, Berkeley, August 23, 2002 Performance on Power3 and Power4 Machines 0.10 1.00 10.00 100.00 1 2 4 8 16 32 # of CPU Time(sec) tot time P4 assem time P4 solve time P4 tot time P3 assem time P3 solve time P3
  15. 15. SP SciComp 6, Berkeley, August 23, 2002 Timing Table 1 cpu 2 cpu 4 cpu 8 cpu 16 cpu 32 cpu Tot time p4 30.01 21.11 13.77 9.52 15.28 12.63 Assem time p4 10.62 5.5 2.5 1.33 0.77 0.57 Solve time p4 8.75 5.9 2.28 1.23 2.08 2.32 Tot time p3 52.2 39.45 30.31 25.43 21.58 18.95 Assem time p3 19.89 11.43 6.19 3.71 2.08 1.24 Solve time p3 16.43 11.71 7.48 5.1 2.83 1.05
  16. 16. SP SciComp 6, Berkeley, August 23, 2002 Observations • Power4 timings: • Unexpected timings on Power4 16 and 32 processors • Scheduler gives exclusive access to nodes and CPUs • Cache , network ? • Power3 timing is consistent (with other machines) • Overall scaling is not good beyond few processors • Serial I/O part contributes to this • Petsc performance : (GMRES with block jacobi precond) • Linear system solver MFLOPS scale well with # of procs on Power3 (have not checked on Power4 yet) • Petsc sparse matrix storage allocation is efficient • # of GMRES iterations increase with # of processors – 41 to 135 iterations on 1 to 16 procs respectively - contributes partially to scaling • Future plan is to investigate scaling further • Investigate viscoelastic modelling ($funding$) • Thanks to Petsc group (Barry Smith) for valuable discussions
  17. 17. SP SciComp 6, Berkeley, August 23, 2002 Application II : Monte Carlo SPECT Imaging
  18. 18. SP SciComp 6, Berkeley, August 23, 2002 • Radionuclide therapy • SPECT imaging in radionuclide therapy • Monte Carlo simulation of SPECT imaging • SIMIND code • Applications • Parallel Monte Carlo code and performance
  19. 19. SP SciComp 6, Berkeley, August 23, 2002 Radionuclide therapy • Cancer cells are sterilized using internally administered ionizing radiation • Some therapeutic isotopes, ex. I-131, produce both beta particles and gamma ray photons • Beta particles kill tumor cells. Beta pathlength span several cells. • Photons used to image radioactivity distribution within patient • Radionuclide therapy has less toxic effect on normal tissue than chemotherapy.
  20. 20. SP SciComp 6, Berkeley, August 23, 2002 I-131 Radionuclide therapy • I-131 Radioimmunotherapy (RIT) : I-131 labeled antibodies selectively target radioactivity to tumor cells while sparing normal tissue. • Shows promise for the treatment of non-Hodgkin’s lymphoma (NHL). NHL is the fifth leading cause of cancer death. Median survival 6-10 years. • I-131 MIBG • Shows promise for the treatment of metastatic neuroblastoma which is a childhood cancer with poor long term survival.
  21. 21. SP SciComp 6, Berkeley, August 23, 2002 I-131 RIT for NHL at University of Michigan • Phase II clinical trial: Out of 76 patients with no previous treatment, 48 achieved a complete response and 26 achieved a partial response. • Patient-specific infusion • Tracer dose: ~ 5 mCi for dosimetry studies to determine therapeutic dose for each patient • Therapeutic dose: 50 -100 mCi one week later
  22. 22. SP SciComp 6, Berkeley, August 23, 2002 I-131 imaging • Single photon emission computed tomography (SPECT) imaging using a rotating gamma camera • Components of the gamma camera • Lead collimator • Detection medium - scintillation crystal • Electronics • Tomographic reconstruction of SPECT data produces a 3-D image of the radioactivity distribution within the patient.
  23. 23. SP SciComp 6, Berkeley, August 23, 2002 Activity quantification • For accurate quantification, SPECT data has to be compensated for • Patient attenuation • Patient scatter • Camera response NaI Crystal Collimator scatter attenuation Patient
  24. 24. SP SciComp 6, Berkeley, August 23, 2002 What is the role of M.C. simulation in SPECT Imaging? • Monte Carlo is used primarily to evaluate compensation methods for scatter, attenuation and camera response and to evaluate the overall accuracy of our clinical activity quantification. • M.C. is ideal for such evaluations because unlike in experiments, the details of photon histories are known.
  25. 25. SP SciComp 6, Berkeley, August 23, 2002 M.C. simulation of SPECT imaging • SIMIND Monte Carlo code • Complete photon transport in phantom and SPECT camera • Complex source distributions: analytical or digital phantoms • Relatively fast • Code has been verified by experiment NaI Crystal Collimator Digital Phantom
  26. 26. SP SciComp 6, Berkeley, August 23, 2002 SIMIND verification using thorax phantom • Measurement with experimental phantom • Simulation with byte- coded digital phantom based on CT images
  27. 27. SP SciComp 6, Berkeley, August 23, 2002 SIMIND verification: thorax phantom 0 2 50 5 00 7 50 1 0 0 0 Intensity 1 5 2 0 2 5 3 0 3 5 4 0 4 5 5 0 Pixel SIMIND Measured
  28. 28. SP SciComp 6, Berkeley, August 23, 2002 M.C. applications: SPECT quantification accuracy using voxel-man phantom • Clinically realistic case • Voxel-man is based on patient CT • Realistic activity distribution in organs and tumor • Quantification error for large, spherical tumors < 3% • Simulation time: 220 hours using 16 SP2 processors (60 projections; 1 billion photon per projection)
  29. 29. SP SciComp 6, Berkeley, August 23, 2002 Work in progress: Monte Carlo generated patient- specific recovery coefficients Patient CT with tumor outline RC= estimated activity true activity Activity = estimated activity x 1/ RC Patient voxel phantom Monte Carlo data Projections Reconstructed image Apply VOI and quantify Measured data Measured projections Reconstructed image Apply VOI and quantify
  30. 30. SP SciComp 6, Berkeley, August 23, 2002 A parallel M.C. code for SPECT: motivation • Fast code for accurate I-131 Monte Carlo simulations • I-131 M.C. simulations are computationally tedious • Physical modeling of collimator • Variance reduction limited • SPECT require a large number of projections • Realistic simulations using high resolution voxel phantoms • When all of the above are included in a simulation, CPU time can be several months using the serial SIMIND code
  31. 31. SP SciComp 6, Berkeley, August 23, 2002 SIMIND parallelization • In the present application the photon histories are independent of each other - “inherently parallel” • Critical to have a good parallel RNG (SPRNG) • Each processor performs entire simulation and reports results to host processor • Code replicated in each of the N processors • Host sums N partial results and calculates the final result • Standard deviation of the combined result is improved by 1/sqrt(N) • Minimal changes to original SIMIND code
  32. 32. SP SciComp 6, Berkeley, August 23, 2002 Start Read data Reset history counter Start history photon transport More projs? MPI_RECEIVE Sum data sets Store image Results from N-1 procs. ? New proj. More histories? End Yes Yes Yes No No No New proj. Start Read data Reset history counter Start history photon transport More projs? MPI_SEND More histories? End Yes Yes No No host processor other processors
  33. 33. SP2 timing results of one SPECT projection of the voxel-man phantom Small (8.4x10 7 photons/projection) Medium (8.4x10 8 photons/projection) N Time (sec) Speed up Efficie ncy Time (sec) Speed up Efficie ncy 1 17131 1.000 1.000 17164 0 1.000 1.000 2 8581 1.997 0.998 85922 1.998 0.999 4 4298 3.986 0.996 42959 3.995 0.999 8 2162 7.923 0.990 21598 7.947 0.993 16 1085 15.785 0.987 10749 15.968 0.998 32 547 31.304 0.978 5408 31.739 0.992
  34. 34. SP SciComp 6, Berkeley, August 23, 2002 Power3 timing results of one SPECT projection of the voxel-man phantom (16.8x107 photons/projection) N Time (sec) Speedup Efficiency 8 25354 1.00 1.00 64 3167 8.00 1.00 512 402 63.06 0.98
  35. 35. SP SciComp 6, Berkeley, August 23, 2002 Power4 Timing of a different Monte Carlo photon transport code May, 2002 on IBM San Mateo center 32 procs Power4 with large page set ( early access to machine may have contributed to results below) #proc Time P4 (sec) Speedup P4 (P3) 1 455 1.00 (1.00) 2 227 2.00 (1.99) 4 114 3.99(3.99) 8 57 7.98 (7.98) 16 29 15.68 (15.64) 29 16 28.43 32 19 23.94 (31.54)
  36. 36. SP SciComp 6, Berkeley, August 23, 2002 More Realistic Simulations • A 60-projection SPECT simulation of the voxel-man phantom simulation of the voxel-phantom. • Realistic values were used for the activity concentration ratios in several organs and tumors (based on typical I-131 RIT patient studies at U. Michigan clinic): kidneys, 81; liver, 26; lungs, 26; spleen 53; blood pool, 48; 100 cc spherical tumor, 100; 50 cc spherical tumor, 100; 20 cc spherical tumor, 100; all other structures, 4. The SPECT matrix size was 64x64x64 with pixel size of 0.8 cm x 0.8 cm x 0.4 cm. Up to 3 orders of scatter was modeled. 1 billion photons were simulated for each projection. • The simulation time on Power3 using 512 processors was 6.5 hours for all 60 projections (time for each projection was 6.5 min).
  37. 37. SP SciComp 6, Berkeley, August 23, 2002 Effect of Parallel Computing • Monte Carlo has enabled to evaluate and improve the quantification of I-131 tumor uptake for dosimetry in NHL patients undergoing RIT at U. Michigan clinic. • May lead to statistically significant dose-response relationships • Speed-up due to the parallel SIMIND code has enabled us to carry out clinically realistic simulations using voxel-phantoms. • In the future we will carry out M.C. based dose calculations • More accurate • Tumor and organ dose distributions
  38. 38. SP SciComp 6, Berkeley, August 23, 2002 Application III : Parallel Proteomics Application
  39. 39. SP SciComp 6, Berkeley, August 23, 2002 • Sequest is a proteomics application used to analyze mass spectrometer output and match to protein database to identify proteins • The Naval Medical Research Center (NMRC) is using Sequest in the research to develop a malaria vaccine based on the expression of proteins in the various stages of the malaria parasite. • Performance of the serial Sequest is currently was a major bottleneck in malaria vaccine project • SDSC computational scientists developed a parallel version of the Sequest code to reduce simulation time significantly
  40. 40. SP SciComp 6, Berkeley, August 23, 2002 Chloroquine-resistant Chloroquine-sensitive  More individuals on the planet with malaria today then ever before in history  300-500 million people become ill with malaria each year  1-3 million children die each year from malaria (200-300 per hour)  Drug resistance is spreading rapidly  There is no licensed vaccine available anywhere in the world  Malaria is a major cause of illness in US troops overseas Facts about Malaria  An efficient vaccine should be achievable:  Immunity can be acquired naturally  Irradiated sporozoites provide > 95% protection  Vaccines targeting single proteins were disappointing  Current strategy: multistage multicomponent vaccine
  41. 41. SP SciComp 6, Berkeley, August 23, 2002 A Proteomics View of the Malaria Parasite Life Cycle OOne Genome: ~6,000 genes DDifferent Proteomes: Distinct Stages Comprehensively Analyze Protein Complements from 4 P. falciparum Cell Types  Identify Stage-specific Targets for Drug and Vaccine Development
  42. 42. SP SciComp 6, Berkeley, August 23, 2002 Importance of a malaria vaccine The battle against malaria hampered by the emergence of drug resistant strains of Plasmodium falciparum, the parasite responsible for majority of malaria infections. Most cases of malaria are concentrated in the world's poorest countries. Malaria vaccine likely to be affordable alternative to expensive drugs.
  43. 43. SP SciComp 6, Berkeley, August 23, 2002 SEQUEST® DTASelect & Contrast > 1,000 Proteins Identified Tandem Mass Spectrometer Peptide MixturePlasmodium falciparum Sporozoites, Trophozoites, Merozoites, Gametocytes Digestion Lysis Proteins High-Throughput Proteomics: MudPIT SCXRP 2D Chromatography 48,000 MS/MS Spectra PySpzS5609 #2438 RT: 66.03 AV: 1 NL: 8.37E6 T: + c d Full ms2 729.75@35.00 [ 190.00-1470.00] 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 RelativeAbundance 545.31 658.36 900.36 1031.40 913.42 1240.53 782.23 896.29 1032.43895.33 546.19 771.24 1028.41 721.31 431.15 801.38 1241.39914.34 427.27 559.13 1258.56317.17 669.39 1033.60 1312.35651.14408.74 1027.22 1142.43 915.53432.40 882.07600.24399.24 986.50 1123.49217.91 1356.10481.13 869.23 1195.44
  44. 44. SP SciComp 6, Berkeley, August 23, 2002 Single Processor Optimization • ( Timings on 375 MHz IBM Power3 procs.TF:ThermoFinnigan;TSRI:The Case Original –O2 Optimize d –O2 Original –O3 Optimized –O3 TF1 86 43 83 45 TF2 140 63 137 63 TF3 209 110 207 111 TF4 388 169 381 173 TSRI 304 145 269 139
  45. 45. SP SciComp 6, Berkeley, August 23, 2002 Code parallelization •Parallel version of Sequest has been developed using MPI and incorporating all single processor optimizations. •Parallelization was done so that all the processors work on a different file simultaneously; files are picked up from a list in round robin distribution by the processors •Tests show that good load balancing is achieved by distributing units of work in a round robin fashion. •Benchmarks show almost linear scaling on thousands of files •Database file (~30 Mb) currently read in once per input file (one test case has ~30,000 input file ; ~1000 procs) •Future plan : once per MPI process (reduces I/O from ~ 1 Tb to 30 Gb); eventually database file will be read once by single node
  46. 46. SP SciComp 6, Berkeley, August 23, 2002 Power 3 Timing and Speedup # files # proc Time (sec) Speedup 2227 32 2724 – 3031 1.0 – 1.0 2227 64 5531 – 5744 2.0 – 1.89
  47. 47. SP SciComp 6, Berkeley, August 23, 2002 Power3 versus Power4 Speedup on 32 procs # files Machine Time (sec) Speedup 2227 Power3 2724 – 3031 1.0 – 1.0 2227 Power4 1869 – 1939 1.45 – 1.50
  48. 48. SP SciComp 6, Berkeley, August 23, 2002 Impact on science Calculations (anlysis of part of a whole cell lysate of a merozoite sample: late-blood stage in the malaria lifecycle) which would have required 30 days on a single processor now require less than an hour on ~1000 processors of IBM SP Power3 NAVO machine 2x speedup due to single processor tuning observed and ~1000x speedup from parallelization observed Sequest is estimated to be in use at 500 laboratories worldwide – this work impacts entire proteomics community
  49. 49. SP SciComp 6, Berkeley, August 23, 2002 Final Comments • Genomics community has already extensively used supercomputing capabilities and will continue to use • Now the proteomics community will use supercomputing more and more • Medical community is starting to use parallel computing in clinical and operation room procedures such as imaging, modeling of physical organs and their functions etc.

×