With diverse and rich offerings from cloud computing
providers in the open cloud market, scientists have
great opportunities to design and conduct complex applications
by utilizing and combining computational resources,
software components and data sources in elastic manners.
While existing techniques focus mainly on resource elasticity
in single cloud infrastructure, scientists expect to design their
applications being elastic in multiple dimensions to ensure
that they applications can operate on multiple clouds with
minimum software engineering effort. In this paper we will
focus on providing techniques for scientists to compose elastic
high performance applications by utilizing traditional software
components, user-provided components and cloud services. We
characterize elastic compositions via their resource, quality,
cost, available time and usage right elasticity, thus enabling
scientists to evaluate and decide how to develop, deploy and
control the compositions to match their elastic needs. To
illustrate our approach, we will present several real-world
application compositions for multi-cloud environments.
Elastic High Performance Applications – A Composition Framework
1. Elastic High Performance Applications
– a Composition Framework
Tran Vu Pham1, Hong-Linh Truong2, Schahram Dustdar2
1
Faculty of Computer Science and Engineering
Ho Chi Minh City University of Technology
2
Distributed Systems Group, Vienna University of Technology
truong@infosys.tuwien.ac.at
http://www.infosys.tuwien.ac.at/Staff/truong
APSCC 2011, 13 Dec 2011, Jeju, Korean 1
2. Outline
Motivation
Elastic components and elastic high
performance applications
Prototypes and experiments
Conclusions and future work
APSCC 2011, 13 Dec 2011, Jeju, Korean 2
4. Motivation (1)
HPC cloud markets:
Infrastructure Providers, Software Providers, Service
Vendors, and End-user
Diverse types of components:
HPC programs, libraries, operating systems, virtual machine
images, Web services, SaaS, PaaS, and IaaS,
Different costs and licensing
Complex application requirements:
quality, elastic time and money, scale in/out different
cloud environments
APSCC 2011, 13 Dec 2011, Jeju, Korean 4
5. Motivation (2)
Existing resolving software dependency and compatibility
→ deal wih software dependencies and
compatibilities around a fixed OS
Workflow composition
→ deal with matching service input/output
eHPA:
→ conflicting diverse types of components within and
among cloud-based environments dynamically
Few existing solutions deal with resource elasticity only
→ there are multi-dimensional elasticity
(see “Principles of Elastic Processes – IEEE Internet Computing 15,
5 (September 2011), 66-71”)
APSCC 2011, 13 Dec 2011, Jeju, Korean 5
7. Multiple elasticity dimensions for
eHPA
Resource elasticity: software
(os/library/middleware/servic Non-functional parameters
e) on multiple clouds elasticity: quality, available
time, right of uses
Pricing/Rewarding/Incentive
elasticity: cost
eHPA elasticity See multiple elasticity dimensions at:
http://www.slideshare.net/linhsolar/principles-of-elastic-
processes-on-clouds-and-some-enabling-techniques
These elasticity metrics are simplified for the sake of brevity
APSCC 2011, 13 Dec 2011, Jeju, Korean 7
8. Requirements and elastic
properties
Functional requirements
Properties End user/Service Vendor Software/Infrastructure Provider
Functions + +
Dependencies +
Conflicts +
Elastic properties
Properties End user/Service Vendor Software/Infrastructure Provider
Resource +
Cost + +
Quality + +
Time + +
Rights of Use + +
APSCC 2011, 13 Dec 2011, Jeju, Korean 8
9. Elastic Component and eHPA
Elastic component
Functional description
Elastic properties
Elastic High Performance Application (eHPA)
Component properties
and dependencies are
modeled using ontology
APSCC 2011, 13 Dec 2011, Jeju, Korean 9
10. Elastic measurements and
aggregation
Resource for components
Internal dependency
External dependency
Component cost
Aggregated cost
Quality
Available Time
Right of Uses
APSCC 2011, 13 Dec 2011, Jeju, Korean 10
11. Composition algorithms (1)
Requested partitions of
components and
elastic requirements
eHPA Compostion –
functionality aspect
Resolve
dependencies,
check conflicts, and
form partitions
EHPA composition –
elastic requirement
Check elastic
requirements
APSCC 2011, 13 Dec 2011, Jeju, Korean 11
14. Experiments – application
Star3D
Solving Euler equations in the cases of 3D flows
Based on MPI
APSCC 2011, 13 Dec 2011, Jeju, Korean 14
15. Experiments – modeling Star3D
Elastic properties
Resources: 32
processes
Subjective rank:
2-4
Free of charge
Unlimited time
Academic license
APSCC 2011, 13 Dec 2011, Jeju, Korean 15
16. Experiments – possible solutions
75 components in knowledge based
Star3D on EC2 with linux
12 different solutions
Four groups of solutions, different component external
and internal dependencies
APSCC 2011, 13 Dec 2011, Jeju, Korean 16
17. Experiments – examples of solutions
An eHPA solution using free a
Fortran compiler (solution 8)
An eHPA solution using Portland
Fortran compiler, licensed for
use up to 256 MPI processes
(solution 4).
APSCC 2011, 13 Dec 2011, Jeju, Korean 17
18. Discussion and future work
Complex HPAs in clouds
Deal with complex software dependencies and conflicts
Determine and characterize elastic properties
Multi-dimensional elasticity metrics: resource, quality, cost, available time and
right of uses
We propose modeling and composition techniques
We use simple elastic properties but they can further modeled into
sub-dimensions
our first step toward multi-dimensional elasticity for HPAs
Current we do not consider dependencies among these properties
Our future work
Integrate with TOSCA (www.open-tosca.org)
Work on elasticity tradeoff
Develop runtime packaging and deployment
APSCC 2011, 13 Dec 2011, Jeju, Korean 18
19. Thanks for your attention!
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
Austria
truong@infosys.tuwien.ac.at
http://www.infosys.tuwien.ac.at/staff/truong
APSCC 2011, 13 Dec 2011, Jeju, Korean 19