Combining software-based and human-based services is crucial for several complex problems that cannot be solved using software-based services alone. In this paper, we present novel methods for modeling and developing hybrid compute units of software-based and human-based services. We discuss high-level programming elements for different types of software- and human-based service units and their relationships. In particular, we focus on novel pro- gramming elements reflecting hybridity, collectiveness and adaptiveness properties, such as elasticity and social connection dependencies, and on-demand and pay-per-use economic properties, such as cost, quality and benefits, for complex problem solving. Based on these programming elements, we present programming constructs and patterns for building complex applications using hybrid ser- vices.

1. Augmenting Complex Problem Solving
with Hybrid Compute Units
Hong-Linh Truong, Hoa Khanh Dam, Aditya Ghose, Schahram
Dustdar
Distributed Systems Group, Vienna University of Technology
University of Wollongong
truong@dsg.tuwien.ac.at
http://dsg.tuwien.ac.at/research/viecom
WESOA@ICSOC 2013, 2.12.2013
1

2. Outline
 Motivation

Programming hybrid, collective adaptive systems in
the cloud
 Hybrid compute units


Fundamental programming elements and constructs
Framework
 Illustrating examples
WESOA@ICSOC 2013,
2.12.2013
2

3. Hybridity and
Diversity of
Computing Models
Human-based
Computing
Things-based
computing
Architecture
Processing
Unit
Machine-based
Computing
S. Dustdar, H. Truong, “Virtualizing Software and Humans
for Elastic Processes in Multiple Clouds – a Service
Management Perspective”, in International Journal of Next
Generation Computing, 2012
Today’s complex problems need
Ad hoc networks
Web of things
hybridity and diversity-aware “collective
SMP
adaptive systems” (HDA-CAS)
Comm.
Grid
WESOA@ICSOC 2013,
2.12.2013
3

4. Motivation -- Dynamic hybridity in a
collective
 Mixture of different types of resources working in
concert in the same collective
 Mixture of different roles performed in the same
collective
 Different functions based on different capabilities
 Mixture of different quality from a single collective
 Functions are performed under different quality
 Resources possible offer different quality wrt their
role
 Mixture of cost/benefit models
 A collective might perform a function with different
costs/qualities
WESOA@ICSOC 2013,
2.12.2013
4

5. Motivation – HAD-CAS
 Challenges/difficulties in programming HDACAS in the cloud
 Human-in-the-loop
 Elasticity of costs, quality/benefits and resources
 Unified framework for humans and software
 Cross-platform with complex underlying APIs
Open problem: there is a lack of suitable programming
frameworks
WESOA@ICSOC 2013,
2.12.2013
5

6. Our goals
 Construct hybrid collective adaptive systems

Consist of service units abstracting software, people
and things

With different service units roles (computation, data,
management)
 Be elastic in multi-cloud environments
 Provide common programming features/APIs that
enable different coordination/composition ways for
CASs
Our approach: hybrid compute units for collective adaptive
systems (hCAS)
WESOA@ICSOC 2013,
2.12.2013
6

7. Unified service unit model for
capturing hybridity in a collective
Consumption,
ownership, provisioning,
price, etc.
Service
model
Elastic
Service
Unit
Function
The functional
capability of the unit
and interface to
access the function
Unit
Dependency
Modeling type of
units (e.g.,
computation, data,
monitor,) and their
dependencies
Elastic
Capability
Capabilities to be elastic
under different
requirements
WESOA@ICSOC 2013,
2.12.2013
7
Software
VolunteersProfessionals
Thing
People
Resources

8. Hybrid compute units for CAS
Hybrid compute unit (HCU): a set of service units
includes software-based services, human-based
services and things-based services that can be
provisioned, deployed and utilized as a collective
on-demand based on different quality, pricing and
incentive models.
This paper focuses on abstracting service units and
relationships to provide fundamental programming elements
for HCU
WESOA@ICSOC 2013,
2.12.2013
8

9. Hybrid compute unit design –
fundamental elements
WESOA@ICSOC 2013,
2.12.2013
9

10. Hybrid compute unit design -Relationships
Relationship Type
HBS
SBS
TBS
HCU
Similarity
Yes
Yes
Yes
Yes
Composition
Yes
Yes
Yes
Yes
Data Dependency
Yes
Yes
Yes
Yes
Control Dependency
Yes
Yes
Yes
Yes
Location Dependency
Yes
Yes
Yes
Yes
Forwarding
Yes
Yes
No
Yes
Delegation
Yes
Yes
No
Yes
Social Relation
Yes
No
No
Yes
Elasticity
Yes
Yes
No
Yes
WESOA@ICSOC 2013,
2.12.2013
10

11. Hybrid compute unit design – highlevel programming constructs
 Current prototype: Fundamental elements can
be implemented as classes in high-level
programming languages


Service units and their properties
Relationship constructs
 But complex runtime is still open


Relationships require complex code libraries for
different templates
Complex integration with existing clouds
WESOA@ICSOC 2013,
2.12.2013
11

12. Hybrid compute unit design – highlevel programming constructs
Relationships
Constructs
Similarity
similarity(U; V; criteria)
?similarity(U; x; criteria)
Data Dependency
datadependency(U;D; [M; ]V )
?datadependency(x;D;M; V ),
?datadependency(U;D;M; x)
?datadependency(U;D; x[c]; V )
Location
locationdependency(U; V; ctx; path)
Brokering
delegate(U; task; V )
forward(U; task; V )
?delegate(U; task; x)
SocialRelation
socialrelation(U; V; ctx; path)
?socialrelation(U; x; distance; ctx;
path).
Elasticity
?elasticity(U; elasticityReq; x)
WESOA@ICSOC 2013,
2.12.2013
12

13. Towards programming framework
for hCAS
HDA-CAS Coordination
and Composition Models
[ICSOC12]
WESOA@ICSOC 2013,
2.12.2013
13

14. Illustrating Example
 Evaluating quality of data (QoD) in
simulations/data analytics processes

Typical components in simulation workflows/data
analytics
preprocessing
solving
postprocessing
data
analysis
 Goal using hCAS to support quality of data
control of the simulations/data analytics

Reasons: reducing wasting time and computational
cost, dealing with tacit quality evaluation, etc.
WESOA@ICSOC 2013,
2.12.2013
14

15. hCAS and simulations
 Only SBS is needed,
 for example, in the QoD evaluation step
before pre-processing
 SBS or HBS is used interchangeably or in
combination
 for example, in the QoD evaluation after preprocessing,
 Only HBS is used
 e.g., in QoD evaluation after solving.
WESOA@ICSOC 2013,
2.12.2013
15

16. hCAS and simulation
workflows/data analytics
hCAS
WESOA@ICSOC 2013,
2.12.2013
16

17. Excerpt -- Programming elasticity
and collectiveness in solving steps
SBS preprocessingUnit =new SBS() ; //create an instance of software unit
File data =new File(fileName);
//...
//create another unit for QoD evaluation
SBS qodEvalUnit = new SBS();
ArrayList paraType = new ArrayList();
paraType.add(File.class.getName());
//the unit must support qodEvaluate function which returns a value in [0,1]
qodEvalUnit.setFunction("qodEvaluate",paraType, Double.class.getName());
//...
//call qod evaluation unit
Double result =(Double)qodEvalUnit.execute("qodEvaluate",params);
…
//call preprocessing activity if QoD is satisfied
if (result > 0.9) {
preProcessedData=preprocessingUnit.execute("preprocessing",params);
}
WESOA@ICSOC 2013,
2.12.2013
17

18. Excerpt -- Programming elasticity
and collectiveness in solving steps
if (qodPreProcessedData < 0.5) {
//initiate a new unit
ICU dataScientist = new ICU();
//create a dropbox place for sharing data
DropboxAPI<WebAuthSession> scuDropbox = null;
// ....
DropboxAPI.DropboxLink link = scuDropbox.share("/hbscloud");
//ask the cloud of HBS to invoke the ICU
VieCOMHBS vieCOMHBS = new VieCOMHBSImpl();
vieCOMHBS.startHBS(dataScientist);
HBSMessage msg = new HBSMessage();
msg.setMsg("pls. use shared dropbox for communication " +link.url);
vieCOMHBS.sendMessageToHBS(dataScientist, msg);
}
WESOA@ICSOC 2013,
2.12.2013
18

19. Excerpt – Programming elasticity
and collectiveness in solving steps
if (qodPreProcessedData < 0.9) {
//specify some static properties of the solver
SBS solverUnit2 = new SBS("solver");
solverUnit2.function.put("DIRTY_DATA", Boolean.valueOf(true));
//specify expected cost and accuracy support
CostModel costModel = new CostModel();
costModel.price = 100; //max in EUR
costModel.usageTime = 1000 * 60 * 60; //1 hour
Quality quality = new Quality();
quality.name = Quality.ACCURACY;
quality.value = 0.95; // minimum value
ArrayList nfps = new ArrayList();
nfps.add(quality); nfps.add(costModel);
//find solvers met quality and cost needs
SBS elasticSolverUnit = (SBS)Relationship.elasticity(solverUnit2,nfps);
Object solverResult2 = elasticSolverUnit.execute("solving", params1);
}
WESOA@ICSOC 2013,
2.12.2013
19

20. Excerpt – Programming location
dependency
//create a new software unit for simulation solvers
SBS solverUnit = new SBS("solver");
//make sure the solver unit and the preprocessing unit are in the same
data center
Relationship.locationDependency(preprocessingUnit,solverUnit,"Amaz
onEC2:Europe");
ArrayList params1 = new ArrayList();
params1.add(preProcessedData);
//execute solver unit with input data from preprocessing unit
Object solverResult= solverUnit.execute("solving",params1);
WESOA@ICSOC 2013,
2.12.2013
20

21. Excerpt – Forwarding and
delegating analysis request
ICU dataScientist = new ICU();
//....
ICU fUnit = new ICU();
Relationship.socialrelation(dataScientist,
fUnit,1,"Linkedin:DataScienceGroup/TUWien");
Relationship.forward(data, fUnit);
//...
SCU studentSCU = new SCU();
//..
Relationship.delegate(data, studentSCU);
WESOA@ICSOC 2013,
2.12.2013
21

22. Conclusions and Future Work
Hybrid compute units for implementing hybridity
and diversity-aware collective adaptive systems
(HDA-CAS)
 Design fundamental programming elements and
constructs
 We are still just at the abstract programming
elements/constructs
 Need to work more on mapping from language
construct abstractions to runtime APIs
 Runtime system working with several underlying
frameworks for supporting different composition and
coordination techniques

WESOA@ICSOC 2013,
2.12.2013
22

23. Many thanks!
truong@dsg.tuwien.ac.at
http://dsg.tuwien.ac.at/research/viecom
WESOA@ICSOC 2013, 2.12.2013
23