The document discusses parallelization and distribution strategies in the LarKC semantic web platform. It describes how LarKC supports coarse-grained parallelization across plug-ins and fine-grained parallelization within plug-ins. LarKC also supports distributed execution of plug-ins on remote systems. This allows it to make use of multiple computing resources and handle large amounts of data. The document provides an example of how parallelization could be applied across different plug-ins in a pipeline.

1. Parallelisation in LarKC

3. “ within a plug-in ” parallelization MPI OpenMP hybrid … “ across plug-ins ” or “ across instances of the same plug-in ” parallelization IBIS/JavaGAT … Grid middleware solutions Parallelization and Distribution strategies in LarKC Scalability at plug-in level Scalability at pipeline level Plug-in scope Platform scope

5. Parallelization and Distribution in the LarKC Platform – Remote execution Implementation in progress Remote Plug-in Manager Query Transformer Plug-in API Remote Plug-in Manager Identifier Plug-in API Remote Plug-in Manager Info. Set Transformer Plug-in API Remote Plug-in Manager Selecter Plug-in API Remote Plug-in Manager Reasoner Plug-in API Stub Plug-in Manager Stub Plug-in Manager Stub Plug-in Manager Stub Plug-in Manager Stub Plug-in Manager Decider Plug-in Registry Pipeline Support System + Support for distributed remote execution

7. High Performance and Distributed Computing support in LarKC (1/2) LarKC supports large-scale HPC and distributed computing environments for executing plug-ins/pipelines Plug-in layer Platform layer Decider Identifier LarKC platform Reasoner LarKC Data Layer Resource layer … Developer extensions LarKC middleware adapters/extensions User environment High-performance and Grid (Cloud) environment Data Storage RDF Store RDF Doc RDF Doc RDF Store High-performance and cluster systems Public Desktop Grid Volunteer resources User desktop machine Cloud resources Native middleware solutions

8. High-performance computing systems (clusters of SMP nodes) Computing environments potentially supported by LarKC Public Desktop Grid (BOINC based) Public Desktop Grid (XtremWeb based) Volunteer resources Public Desktop Grid Local Desktop Grid High Performance and Distributed Computing support in LarKC (2/2) High Performance Computing Grid infrastructure (e.g. EGEE, DEISA, etc.) Cloud computing environments Service Grid (e.g. EDGeS) Implementation in progress

9. footer 04/11/09 end