Multi-infrastructure workflow execution for medical simulation in the Virtual Imaging Platform

VIP
Virtual Imaging Platform

Multi-infrastructure workflow execution
for medical simulation in the

Rafael FERREIRA DA SILVA1, Sorina CAMARASU-POP1, Baptiste GRENIER3
Vanessa HAMAR2, David MANSET3, Johan MONTAGNAT4, Jérôme REVILLARD3
Javier ROJAS BALDERRAMA4, Andrei TSAREGORODTSEV2, Tristan GLATARD1

1 Université de Lyon, CNRS, INSERM, CREATIS
2 Centre de Physique des Particules de Marseille
3 maatG France
4 CNRS/UNS, I3S lab, MODALIS team

Bristol, HealthGrid 2011

VIP ANR-09-COSI-013-02 www.creatis.insa-lyon.fr/vip 1

VIP
Introduction

  Simulation
Computation: 16h

Example: 2D+t ultrasound simulation
[O. Bernard]

Example: Prostate traitement in protontherapy
Computation: 2 months
[L. Grevillot, D. Sarrut]

8.5 CPU
days

US MRI CT PET

VIP
Introduction

  European Grid Infrastructure (EGI)
  High throughput for computation and data transfers
  Challenges
  High latencies
  Low reliability


VIP
Our Goal

  Multi-infrastructure workflow execution
  Grid resources
  Personal clusters (non-grid resources)

  Improve data transfer reliability
  Local reliable storage


VIP
VIP Architecture


VIP
Workflows
  Virtual Imaging Platform (VIP)
  Workflow

Activities

Data sources

Data sinks

Simulated US image
of the heart
  Core Simulation Workflows
  MRI, US, CT and PET


VIP
Workflow Wrapping

  Simulations are described as workflows

  Workflows are interpreted and executed by MOTEUR
  Core engine workflow interpreter
  Data flow evaluation
  Production of computational tasks

  Workflow activities are described using jGASW
  Code wrapper for distributed platforms and local executions
  Grid jobs (bash scripts) are generated from processed data

  Each jGASW descriptor bundles a unique executable

http://modalis.i3s.unice.fr/softwares/moteur/start
http://modalis.i3s.unice.fr/jgasw

VIP
Pilot Jobs
  Pull mode
  Execution environment verification
  Worker node reservation

Moteur + jGASW

User Jobs

Pilot-jobs System

gLite WMS

X
Worker Nodes DIRAC Architecture


VIP
Multi-infrastructure Execution

  Design constraints
  No intervention from the cluster administrator
  No sharing or generic user accounts
  Minimal technical assumptions on the cluster architecture

  Setup
  The agent is launched by the user on his cluster(s) account(s)
  Download of a tar.gz bundle and run setup script
  Support to PBS, BQS and SLURM

  Execution
  Queries WMS for user’s waiting tasks Personal
cluster

  Submits pilots to the local cluster queue VIP cluster bundle

  Embedded data transfer client (VLET)

http://vip.creatis.insa-lyon.fr:9002/projects/dirac-cluster-agent/wiki/Wiki!


VIP
Multi-infrastructure Execution

  Conditions
  EGI
  134-node cluster limited to 67 pilots running concurrently
  3 executions of a PET workflow

  Results

  Conclusion
  Involving a small personal cluster in a simulation has a significant
impact on the execution


VIP
Reliable Data Management

  EGI three-tier data management
  Challenge: data availability between 80-95%
  Storage Elements (SE) – DPM, dCache, STORM, Castor
  Logical File Catalog (LFC) – single index space

  Current Data Management in VIP
  Critical input files are replicated
  Files are cached by pilot jobs
  Output files are stored on site SE
  Jobs error rate: 5-10%

  Local Data Manager
  Failover storage
  Available for users and grid jobs
  Overlay of DPM SE


VIP
Reliable Data Management
  Data Management use case

Input download

Output upload


Impact of the Data Manager
VIP
Virtual Imaging Platform on Job Reliability
  Conditions
  EGI, biomed VO (production infrastructure)
  Ultrasonic Simulation comprised of 128 jobs
  Each job has 5 input files + 1 output file
  Failure rate: 1%

  Results

  Conclusion
  Job data transfers failure rate can be significantly decreased in the
presence of a failover storage


VIP
Web Portal
  Workflow submission and management

  Workflow execution and monitoring


VIP
Web Portal

  Detailed performance information


VIP
Web Portal

  Authentication based on X.509 certificates


VIP
Web Portal

  File transfer operations

http://vip.creatis.insa-lyon.fr


VIP
Platform Usage Statistics

  484 workflows executions (Nov 2010 – Apr 2011)
  10 (real) users
  5 application classes


VIP
Conclusions

  VIP can execute workflows on multi-infrastructures
  Extension of jGASW application description
  Extension of DIRAC to support personal clusters with no administration
intervention and respecting common security rules
  Results show that a small personal cluster can significantly contribute to
a simulation running on EGI

  VIP Data Manager
  A first test shows that job failure rate was decreased from 7.7% to 1.5%

  Try it!
  Requires a certificate registered in the Biomed VO
  http://vip.creatis.insa-lyon.fr


VIP
Future Work

  Workflow Execution
  Scalability and Reliability
  Memory shortage
  Workflow checkpointing

  Provenance of simulated data


VIP
Future Work

  Simulators and Models
  Biological object model sharing for image simulation
  See poster FORESTIER et al. at CBMS 2011
  Semantic catalog of biological models
  3D visualization


VIP
Future Work

  Simulators and Models
  Multi-Modality Simulation Workflow
  See poster MARION et al. at CBMS 2011

SIMRI (MRI) Sindbad (CT)


VIP

Multi-infrastructure workflow execution for medical
simulation in the Virtual Imaging Platform

Questions?

http://www.creatis.insa-lyon.fr/vip!


VIP
References
  Fabrizio Gagliardi, Bob Jones, Fran ̧ois Grey, Marc-Elian Bégin, and Matti Heikkurinen. Building an infrastructure
for scientific grid computing: status and goals of the egee project. Phil. Trans. R. Soc. A 15, 363(1833):1729–
1742, aug 2005.
  T. Li, S. Camarasu-Pop, T. Glatard, T. Grenier, and H. Benoit-Cattin. Optimization of mean-shift scale parameters
on the egee grid. In Studies in health technology and informatics, Proceedings of Healthgrid 2010, volume 159,
pages 203–214, 2010.
  A. Marion, G. Forestier, H. Benoit-Cattin, S. Camarasu-Pop, P. Clarysse, R. Ferreira da Silva, B. Gibaud, T.
Glatard, P. Hugonnard, C. Lartizien, H. Liebgott, J. Tabary, S. Valette, and D. Friboulet. Multi- modality medical
image simulation of biological models with the Virtual Imaging Platform (VIP). In IEEE CBMS 2011, Bristol, UK,
2011. submitted.
  Tristan Glatard, Johan Montagnat, Diane Lingrand, and Xavier Pennec. Flexible and efficient workflow
deployement of data-intensive applications on grids with MOTEUR. International Journal of High Performance
Computing Applications (IJHPCA), 22(3):347–360, August 2008.
  Javier Rojas Balderrama, Johan Montagnat, and Diane Lingrand. jGASW: A Service-Oriented Frame- work
Supporting High Throughput Computing and Non-functional Concerns. In IEEE International Conference on Web
Services, ICWS 2010, Miami (FL), USA, July 2010. IEEE Computer Society.
  A Tsaregorodtsev, N Brook, A Casajus Ramo, Ph Charpentier, J Closier, G Cowan, R Graciani Diaz, E Lanciotti, Z
Mathe, R Nandakumar, S Paterson, V Romanovsky, R Santinelli, M Sapunov, A C Smith, M Seco Miguelez, and
A Zhelezov. DIRAC3 . The New Generation of the LHCb Grid Software. Journal of Physics: Conference Series,
219(6):062029, 2009.
  L. Grevillot, T. Frisson, D Maneval, N. Zahra, J.N. Badel, and D. Sarrut. Simulation of a 6 mv elekta precise linac
photon beam using gate / geant4. Phys Med Biol, 56(4), 2011.


VIP
References
  Silvia Olabarriaga, T. Glatard, and P.T. de Boer. A virtual laboratory for medical image analysis. IEEE T Inf Technol
B, 14(4):979–985, 2010.
  Vladimir V. Korkhov, Jakub T. Moscicki, and Valeria V. Krzhizhanovskaya. Dynamic workload bal- ancing of parallel
applications with user-level scheduling on the grid. Future Generation Computer Systems, 25(1):28 – 34, 2009.
  Ivo D. Dinov, John D. Van Horn, Kamen M. Lozev, Rico Magsipoc, Petros Petrosyan, Zhizhong Liu, Allan
MacKenzie-Graham, Paul Eggert, Parker Douglas S., and Arthur W. Toga. Efficient, Distributed and Interactive
Neuroimaging Data Analysis Using the LONI Pipeline. Frontiers in Neuroinformatics, 3(22):1–10, 2009.
  Thierry Delaitre, Tamas Kiss, Ariel Goyeneche, Gabor Terstyanszky, Stephen Winter, and Péter Kacsuk.
GEMLCA: Running Legacy Code Applications as Grid services. Journal of Grid Computing, 3(1):75– 90, 2005.
  Kyle Chard, Wei Tan, Joshua Boverhof, Ravi Madduri, and Ian Foster. Wrap Scientific Applications as WSRF Grid
Services Using gRAVI. In International Conference on Web Services, ICWS’09, Los Angeles (CA), USA, July
2009.
  Martin Senger, Peter Rice, Alan Bleasby, Tom Oinn, and Mahmut Uludag. Soaplab2: More Reliable Sesame Door
to Bioinformatics Programs. In Bioinformatics Open Source Conference, BOSC’08, Toronto, ON Canada, July
2008.
  Douglas Thain, Todd Tannenbaum, and Miron Livny. Condor and the Grid, pages 299–335. John Wiley & Sons,
Ltd, 2003.
  Vinod Kasam, Jean Salzemann, Marli Botha, Ana Dacosta, Gianluca Degliesposti, Raul Isea, Do- man Kim, Astrid
Maass, Colin Kenyon, Giulio Rastelli, Martin Hofmann-Apitius, and Vincent Breton. Wisdom-ii: Screening against
multiple targets implicated in malaria using computational grid infrastructures. Malaria Journal, 8(1):88, 2009.


VIP
Workflow Wrapping

  Multi-infrastructure code descriptor (jGASW extension)


Multi-infrastructure workflow execution for medical simulation in the Virtual Imaging Platform

Recommended

Recommended

More Related Content

Similar to Multi-infrastructure workflow execution for medical simulation in the Virtual Imaging Platform

Similar to Multi-infrastructure workflow execution for medical simulation in the Virtual Imaging Platform (20)

More from Rafael Ferreira da Silva

More from Rafael Ferreira da Silva (20)

Recently uploaded

Recently uploaded (20)

Multi-infrastructure workflow execution for medical simulation in the Virtual Imaging Platform