With diverse and rich offerings from cloud computing providers in the open cloud market, scientists have great opportunities to design and conduct complex applications by utilizing and combining computational resources, software components and data sources in elastic manners. While existing techniques focus mainly on resource elasticity in single cloud infrastructure, scientists expect to design their applications being elastic in multiple dimensions to ensure that they applications can operate on multiple clouds with minimum software engineering effort. In this paper we will focus on providing techniques for scientists to compose elastic high performance applications by utilizing traditional software components, user-provided components and cloud services. We characterize elastic compositions via their resource, quality, cost, available time and usage right elasticity, thus enabling scientists to evaluate and decide how to develop, deploy and control the compositions to match their elastic needs. To illustrate our approach, we will present several real-world application compositions for multi-cloud environments.

1. Elastic High Performance Applications
– a Composition Framework
Tran Vu Pham1, Hong-Linh Truong2, Schahram Dustdar2
1
Faculty of Computer Science and Engineering
Ho Chi Minh City University of Technology
2
Distributed Systems Group, Vienna University of Technology
truong@infosys.tuwien.ac.at
http://www.infosys.tuwien.ac.at/Staff/truong
APSCC 2011, 13 Dec 2011, Jeju, Korean 1

2. Outline
 Motivation
 Elastic components and elastic high
performance applications
 Prototypes and experiments
 Conclusions and future work
APSCC 2011, 13 Dec 2011, Jeju, Korean 2

3. Background
APSCC 2011, 13 Dec 2011, Jeju, Korean 3

4. Motivation (1)
 HPC cloud markets:
 Infrastructure Providers, Software Providers, Service
Vendors, and End-user
 Diverse types of components:
 HPC programs, libraries, operating systems, virtual machine
images, Web services, SaaS, PaaS, and IaaS,
 Different costs and licensing
 Complex application requirements:
 quality, elastic time and money, scale in/out different
cloud environments
APSCC 2011, 13 Dec 2011, Jeju, Korean 4

5. Motivation (2)
 Existing resolving software dependency and compatibility
→ deal wih software dependencies and
compatibilities around a fixed OS
 Workflow composition
→ deal with matching service input/output
 eHPA:
→ conflicting diverse types of components within and
among cloud-based environments dynamically
 Few existing solutions deal with resource elasticity only
→ there are multi-dimensional elasticity
(see “Principles of Elastic Processes – IEEE Internet Computing 15,
5 (September 2011), 66-71”)
APSCC 2011, 13 Dec 2011, Jeju, Korean 5

6. eHPA components and
relationships
APSCC 2011, 13 Dec 2011, Jeju, Korean 6

7. Multiple elasticity dimensions for
eHPA
Resource elasticity: software
(os/library/middleware/servic Non-functional parameters
e) on multiple clouds elasticity: quality, available
time, right of uses
Pricing/Rewarding/Incentive
elasticity: cost
eHPA elasticity See multiple elasticity dimensions at:
http://www.slideshare.net/linhsolar/principles-of-elastic-
processes-on-clouds-and-some-enabling-techniques
 These elasticity metrics are simplified for the sake of brevity
APSCC 2011, 13 Dec 2011, Jeju, Korean 7

8. Requirements and elastic
properties
Functional requirements
Properties End user/Service Vendor Software/Infrastructure Provider
Functions + +
Dependencies +
Conflicts +
Elastic properties
Properties End user/Service Vendor Software/Infrastructure Provider
Resource +
Cost + +
Quality + +
Time + +
Rights of Use + +
APSCC 2011, 13 Dec 2011, Jeju, Korean 8

9. Elastic Component and eHPA
 Elastic component
 Functional description
 Elastic properties
 Elastic High Performance Application (eHPA)
Component properties
and dependencies are
modeled using ontology
APSCC 2011, 13 Dec 2011, Jeju, Korean 9

10. Elastic measurements and
aggregation
 Resource for components
 Internal dependency
 External dependency
 Component cost
 Aggregated cost
 Quality
 Available Time
 Right of Uses
APSCC 2011, 13 Dec 2011, Jeju, Korean 10

11. Composition algorithms (1)
 Requested partitions of
components and
elastic requirements
 eHPA Compostion –
functionality aspect
 Resolve
dependencies,
check conflicts, and
form partitions
 EHPA composition –
elastic requirement
 Check elastic
requirements
APSCC 2011, 13 Dec 2011, Jeju, Korean 11

12. Composition
Algorithms (2)
APSCC 2011, 13 Dec 2011, Jeju, Korean 12

13. Prototype
APSCC 2011, 13 Dec 2011, Jeju, Korean 13

14. Experiments – application
 Star3D
 Solving Euler equations in the cases of 3D flows
 Based on MPI
APSCC 2011, 13 Dec 2011, Jeju, Korean 14

15. Experiments – modeling Star3D
 Elastic properties
 Resources: 32
processes
 Subjective rank:
2-4
 Free of charge
 Unlimited time
 Academic license
APSCC 2011, 13 Dec 2011, Jeju, Korean 15

16. Experiments – possible solutions
 75 components in knowledge based
 Star3D on EC2 with linux
 12 different solutions
 Four groups of solutions, different component external
and internal dependencies
APSCC 2011, 13 Dec 2011, Jeju, Korean 16

17. Experiments – examples of solutions
An eHPA solution using free a
Fortran compiler (solution 8)
An eHPA solution using Portland
Fortran compiler, licensed for
use up to 256 MPI processes
(solution 4).
APSCC 2011, 13 Dec 2011, Jeju, Korean 17

18. Discussion and future work
 Complex HPAs in clouds
 Deal with complex software dependencies and conflicts
 Determine and characterize elastic properties
 Multi-dimensional elasticity metrics: resource, quality, cost, available time and
right of uses
 We propose modeling and composition techniques
 We use simple elastic properties but they can further modeled into
sub-dimensions
 our first step toward multi-dimensional elasticity for HPAs
 Current we do not consider dependencies among these properties
 Our future work
 Integrate with TOSCA (www.open-tosca.org)
 Work on elasticity tradeoff
 Develop runtime packaging and deployment
APSCC 2011, 13 Dec 2011, Jeju, Korean 18

19. Thanks for your attention!
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
Austria
truong@infosys.tuwien.ac.at
http://www.infosys.tuwien.ac.at/staff/truong
APSCC 2011, 13 Dec 2011, Jeju, Korean 19