Catania Science Gateway
Framework
Motivations, architecture, features

Riccardo Rotondo

Tutorial on Science Gateways, Rom...
Outline

 Authentication & Authorisation

 SAGA and its implementations
 Architecture
 Application Server Integration
...
Riccardo Rotondo
Tutorial on Science Gateways, Roma, 03.06.2013
Liferay user database

 Liferay supports several system to store users
data, both local and remote
 Supporting the large...
Authentication

 Authentication is demanded on external IDP
 Communication between Liferay and the IDP
happens thanks to...
Authorisation

 Authorisation is demanded to the LDAP server
 Liferay, through a plugin implemented, request
to Shibbole...
Registration

 In the act of registration user data must be
written on the LDAP connected to Liferay
 A portlet has been...
Integrated Services

JSR 168/268
JSR 168/268

JSR 168/268

JSR 168/268

Catania Science Gateway Framework

GRID

CLOUD

Lo...
Riccardo Rotondo
Tutorial on Science Gateways, Roma, 03.06.2013
A Simple API for Grid Applications (SAGA)
SAGA is an API that provides the basic
functionality required to build distribut...
A Simple API for Grid Applications
(SAGA)


SAGA is composed by:






SAGA Core Libraries: containing the SAGA base
s...
A Simple API for Grid Applications (SAGA)


Several Implementations are available:
 A C++ and a Java implementation deve...
Job Engine
Middleware Independent
JSAGA supports gLite, Globus,
ARC, UNICORE, etc.

Riccardo Rotondo
Tutorial on Science G...
Job Engine - Requirements


The Job Engine has been designed with the following
requirements in mind:
Feature

Descriptio...
Job Engine - Architecture
Worker Threads for
Job Check Status
USERS
TRACKING
DB

WT

WT

WT

WT

WT
Jobs
Submission

WT

W...
Glassfish Integration

 Access to database is not direct but make use
of Glassfish connection pools and hibernate
 JNDI ...
Riccardo Rotondo
Tutorial on Science Gateways, Roma, 03.06.2013
Science Gateway paradigm
 Efforts to grant easy yet secure access to
remote services and related resources brought
to the...
Motivations
Um… isn’t your
computer on fire
?

Image source:

It’s ok, my files
are stored in a
safer place.

Riccardo Rot...
Grid Data Management Challenges
 Make interfaces simple for non expert users
 CLI-based Grid storage interface is not st...
Requirements
 Storage complexity hidden to end users
 Users move files from/to a portal and see it as simple
external st...
Implementations
 Virtual File System requires a database to map
users, virtual resource and real resource
 Object-relati...
References

 Catania Science Gateways url:
http://www.catania-science-gateways.it
 Catania Science Gateway Sourceforge P...
Riccardo Rotondo
Tutorial on Science Gateways, Roma, 04.06.2013
Questions ?

Riccardo Rotondo
Tutorial on Science Gateways, Roma, 03.06.2013
Upcoming SlideShare
Loading in …5
×

Catania Science Gateway Framework

216 views

Published on

Motivations, architectures, feauters

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
216
On SlideShare
0
From Embeds
0
Number of Embeds
12
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Catania Science Gateway Framework

  1. 1. Catania Science Gateway Framework Motivations, architecture, features Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  2. 2. Outline  Authentication & Authorisation  SAGA and its implementations  Architecture  Application Server Integration Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  3. 3. Riccardo Rotondo Tutorial on Science Gateways, Roma, 03.06.2013
  4. 4. Liferay user database  Liferay supports several system to store users data, both local and remote  Supporting the largest number of users in the easiest way  A modular way to distinguish between different services and privileges is need  Science Gateways stores users on an LDAP server Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  5. 5. Authentication  Authentication is demanded on external IDP  Communication between Liferay and the IDP happens thanks to Shibboleth  Shibboleth plugin, installed on Liferay, is responsible to read the token coming from the IDP and to pass it to Liferay Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  6. 6. Authorisation  Authorisation is demanded to the LDAP server  Liferay, through a plugin implemented, request to Shibboleth the mail address(es) an try a match with the ones stored (local, remote) Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  7. 7. Registration  In the act of registration user data must be written on the LDAP connected to Liferay  A portlet has been developed to perform this actions Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  8. 8. Integrated Services JSR 168/268 JSR 168/268 JSR 168/268 JSR 168/268 Catania Science Gateway Framework GRID CLOUD Local Riccardo Rotondo Tutorial on Cluster Science Gateways, Roma, 04.06.2013
  9. 9. Riccardo Rotondo Tutorial on Science Gateways, Roma, 03.06.2013
  10. 10. A Simple API for Grid Applications (SAGA) SAGA is an API that provides the basic functionality required to build distributed applications, tools and frameworks;  It is independent of the details of the underlying infrastructure (e.g., the middleware);  SAGA is an OGF specification: http://www.gridforum.org/documents/GFD.90 .pdf.  Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  11. 11. A Simple API for Grid Applications (SAGA)  SAGA is composed by:    SAGA Core Libraries: containing the SAGA base system, the runtime and the API packages (file management, job management, etc.); SAGA Adaptors: libraries providing access to the underlying grid infrastructure (adaptors are available for Globus, gLite, etc.); SAGA defines a standard We then need an implementation! Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  12. 12. A Simple API for Grid Applications (SAGA)  Several Implementations are available:  A C++ and a Java implementation developed at the Louisiana State University / CCT and Vrije Universiteit Amsterdam (http://saga.cct.lsu.edu);  A Java implementation developed at CCIN2P3 (http://grid.in2p3.fr/jsaga/);  A Python implementation based on those above. Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  13. 13. Job Engine Middleware Independent JSAGA supports gLite, Globus, ARC, UNICORE, etc. Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  14. 14. Job Engine - Requirements  The Job Engine has been designed with the following requirements in mind: Feature Description Status Middleware Independent Capacity to submit job towards resources running different middleware DONE Easiness Create code to run applications on the grid in a very short time DONE Scalability Manage a huge number of parallel job submissions fully exploiting the HW of the machine where the Job Engine is installed DONE Performance Have a good response time DONE Accounting Register every grid operation performed by the users DONE Fault Tolerance Hide middleware failure to the final users DONE Workflow Riccardo Rotondo Providing a way to easily create workflow PARTIALLY Tutorial on Science Gateways, Roma, 04.06.2013
  15. 15. Job Engine - Architecture Worker Threads for Job Check Status USERS TRACKING DB WT WT WT WT WT Jobs Submission WT WT WT WT Resources MONITORING MODULE Jobs Check status/ Get output WT Jobs Queue Worker Threads for Job Submission Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  16. 16. Glassfish Integration  Access to database is not direct but make use of Glassfish connection pools and hibernate  JNDI resource are used as well in order to offer some functionalities working behind the scene of job submission:  Thread pool responsible for job submission  Thread pool responsible for job status updates  Thread pool responsible for retrieving job output Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  17. 17. Riccardo Rotondo Tutorial on Science Gateways, Roma, 03.06.2013
  18. 18. Science Gateway paradigm  Efforts to grant easy yet secure access to remote services and related resources brought to the birth of Science Gateways  Virtual Research Communities access remote resources in a collaboration environment that hides the underlying complexity  SGs help many users to better use the enormous grid computational power  Is large grid&cloud data storage accessible as well in such an easy way? Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  19. 19. Motivations Um… isn’t your computer on fire ? Image source: It’s ok, my files are stored in a safer place. Riccardo Rotondo 19 Tutorial on Science Gateways, Roma, 04.06.2013
  20. 20. Grid Data Management Challenges  Make interfaces simple for non expert users  CLI-based Grid storage interface is not straightforward  Transactions to different e-Infrastructures require different authentication method  Should this transaction involve the Science Gateway directly?  Complexity of current protocols to manage different storage elements  Offer an easy intuitive interface to the end users Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  21. 21. Requirements  Storage complexity hidden to end users  Users move files from/to a portal and see it as simple external storage accessible from a web interface and do not care about grid (or any other) technologies behind  File management smoothly integrated with all the services provided in the SG  Underlining architecture exposes a file-system-like view (i.e., a Virtual File System or VFS) through which users can perform the following actions:  Create, move, delete files/directories with the desired structure  Share files with other users  Set the number of backup copies desired Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  22. 22. Implementations  Virtual File System requires a database to map users, virtual resource and real resource  Object-relation mapping approach  Liferay Service Builder  Database tables are not used only to keep trace of resource (file) but to define referring e-Infrastructures too  Planning to support up to 4 different eInfrastructures: local, remote, grid, cloud (GARRBOX) Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  23. 23. References  Catania Science Gateways url: http://www.catania-science-gateways.it  Catania Science Gateway Sourceforge Project: http://sourceforge.net/projects/ctsciencegtwys/  Gilda Portal (for developers): http://gilda.ct.infn.it/ Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  24. 24. Riccardo Rotondo Tutorial on Science Gateways, Roma, 04.06.2013
  25. 25. Questions ? Riccardo Rotondo Tutorial on Science Gateways, Roma, 03.06.2013

×