This document describes an e-infrastructure and data service that was designed to provide modular access to different computing resources from applications. The service allows reading and writing data to multiple e-infrastructures in a straightforward way. It identifies four types of e-infrastructures - local, remote, grid, and cloud - and allows defining policies and quotas for each. The data service implements common file operations independently of the underlying technologies through storage interfaces. This provides a virtual file system view of data stored across different infrastructure implementations.

1. e-Infrastructure & Data Services
Features and implementations
Catania, 14/03/2014Riccardo Rotondo
riccardo.rotondo@ct.infn.it

2. Outline
 Requirements
 e-Infrastructure Service
 Data Service
 Workflow
 Javadocs
 Implementations
 Demo
2

3. e-Infrastructure & Data Service
 We designed the architecture to provide a Service
accessible from your application.
 Modular attachable resources available to the SG thanks
to the straight coupled service that allows read/write
data from several e-Infrastructures.
3

4. e-Infrastructure Service
 Four different kind of e-Infrastructure identified:
 Local
 Remote
 Grid
 Cloud
 Possibility of defining different Policy and Quota
according e-infrastructures/users availability/needs
 Default and favorite e-Infrastructures selectable
 Disaster recovery and (cross) file replica.
4

5. e-Infrastructure Service
 If you want to copy a file on a file system you plug-in
an external storage to it.
5
HFS
ext4
NFS
FAT

6. e-Infrastructure Service
 On a SG now you are able to plug-in an e-
Infrastructure.
6
Local Storage
Remote Storage
Grid Storage
Cloud Storage

7. Data Services
 Business logic related to file management totally
independent from the underneath technology
 All the common features associated to file provided:
create, copy, delete, move and sharing.
 For each features, we implemented a different Storage
Interface that translates the single commands according
to the technology
 All data are stored in different tables in order to let users
organise files on a virtual file system.
7

8. Data Service
8
Local Storage
Remote Storage
Grid Storage
Cloud Storage
Different
implementations
of the same
storage interface

9. Upload
workflow
9
1.
Sign
in
Database
5. File
Upload
4. e-Infra
found
7. Update
6. Upload on
e-Infras
2. Upload
request
3. e-Infra
Discovery

10. Data Management on various
e-Infrastructures
10
Dec ’13:
release 0.1

11. Service Implementation
 Package name
 Namespace
 Entity
 Interface type
 Finder
 Ordered value
 One-to-many relation
 Many-to-many relation
 Fake entity
Riccardo RotondoTutorial on Science
Gateways, Roma,
service.xml

12. Database created (automatically by service
buider)
12
mysql> show tables like 'eInfrSrv%';
+---------------------------------+
| Tables_in_lportal61 (eInfrSrv%) |
+---------------------------------+
| eInfrSrv_File |
| einfrsrv_files_tags |
| einfrsrv_files_virtualpaths |
| einfrsrv_infrastructure |
| einfrsrv_properties |
| einfrsrv_storageurls |
| einfrsrv_tag |
| einfrsrv_virtualpath |
+---------------------------------+
File
Directory
File url on storage
e-Infrastructures
File tags
Properties (reffered to e-Infra)

13. Javadocs
13
 http://www.catania-science-gateways.it/javadocs/einfrsrv-portlet/index.html

14. Model & Service Implementation
14
Custom model
implementation
Custom service
implementation

15. Main features
15
 e-Infrastructure:
 Define several e-Infrastructures;
 Associate each e-Infra with Policy and Quota
 Storage interface:
 A standard interface defined to support any storage solution
simply implementing the abstract method
 Directory:
 File tree data organisation

16. e-Infrastructure Service
(it.infn.ct.einfrsrv.sb.service.impl.InfrastructureLocalServiceImpl)
16
 Infrastructure
getDefaultInfrastructure(long
companyId, long userId,
java.lang.String portletId,
java.lang.String type)

17. Data Service Storage (it.infn.ct.einfrsrv.data.storage)
17
 java.lang.StringgetStoragePath()
 boolean isURLAvailable(java.lang.String url)
 java.lang.StringputFile(java.lang.String filePath)
 java.lang.StringputFile(java.lang.String source,
java.lang.String destination)
 boolean writeFileToStream(java.lang.String
filePath, java.io.OutputStream out)
 boolean removeFile(java.lang.String filePath)

18. Data Service File
(it.infn.ct.einfrsrv.sb.service.impl.FileLocalServiceImpl)
18
 boolean upload(long companyId, java.lang.String
companyName, long userId, java.lang.String
portletId, Infrastructure infraParam,
java.lang.String fileName, java.lang.String
source, java.lang.String virtualPathParam,
java.lang.String[] tagsParam)
 java.util.List<File> listFiles(long userId,
java.lang.String virtualPathParam)
 boolean writeFileToStream(long userId, long
fileId, java.io.OutputStream out)
 java.net.URL getURL(long userId, long fileId)

19. Data Service Directory
(it.infn.ct.einfrsrv.sb.service.impl.VirtualPathLocalServiceImpl)
19
 VirtualPath makeDir(java.lang.String
parentPath, java.lang.String path,
long userId)

20. Portlet accessing the service
20
 Add to liferay-plugin-package.properties:
 required-deployment-contexts=einfrsrv-portlet

21. Source & Binary
21
 Sourceforge:
http://sourceforge.net/p/ctsciencegtwys/einfsrv/sourc
es/HEAD/tree/
 Rel 0.1:
http://sourceforge.net/p/ctsciencegtwys/einfsrv/sources/H
EAD/tree/tags/release-0.1/
 Binaryhttps://sourceforge.net/projects/ctsciencegtwys/files
/eInfrastructureService/Release/0.1/

22. Demo
22
 Whishing to see e-Infra & Data service in action? Request
access to GARR Science Gateway (https://sgw.garr.it)
 Remember the service is still under development so your
feedback is welcome!

23. Questions ?
23