Principles of Elastic Processes on Clouds and Some Enabling Techniques


Published on

While in contemporary research the elasticity is mostly interpreted
as the capability of scaling in and out of machine-based computing
resources within a single cloud, we argue that the elasticity can be
performed in multiple dimensions, such as resource, cost, and
quality. Furthermore, the elasticity can also be performed in hybrid
machine- and human-based computing systems. In this talk, first we
will discuss principles of such elastic processes and their research
challenges. Second, we will present some initial results of techniques
that can be used in the development of elastic processes, such as
composable cost evaluation, contract compatibility evaluation, and quality of
data evaluation for workflows using both software and human
capabilities. Finally, we will outline challenges when bringing these
techniques to hybrid systems of software- and human-based computing

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Principles of Elastic Processes on Clouds and Some Enabling Techniques

  1. 1. Presented at Australian Symposium on Services Innovation November 21st and 22nd, 2011, University of Wollongong Principles of Elastic Processes onClouds and Some Enabling Techniques Hong-Linh Truong Distributed Systems Group, Vienna University of Technology Joint work with: Schahram Dustdar, Frank Leymann, Michael Reiter, Tran Vu Pham, Quang Hieu Vu, Yike Guo, Benjamin Satzger, Uwe Breitenbuecher, Dimka KarastoyanovaASIS 11, Wollongong, Nov 22, 2011 1
  2. 2. Outline Principles of elastic processes Some enabling techniques for elastic processes in hybrid systems  Monitoring and analysis of non-functional parametersASSI 11, Wollongong, Nov 22, 2011 2
  3. 3. Data-as-a-service and computation- as-a-service Multiple realtime data sources – data services  Smart environments  Earth monitoring, satellite images  Social media Web information  Personal medical information Data processing algorithms – computing services  Data conversion, data enrichment, data extraction, data mining, domain-specific computational analysis (e.g., CO2 calculation), etc  Algorithms can be wrapped into processing elements/workflow activitiesASSI 11, Wollongong, Nov 22, 2011 3
  4. 4. Data intensive processes Influence factors for the quality of processed results  Data rates, quality of data sources, underlying computing systems, etc. The quality of the output is varying Multiple concurrent consumers  Dynamic requirements for SLA, quality of outputs, etc. Data processing service provider: my process must be elastic!  to optimize resource usage, prices, etc., but must meet consumers requirementsASSI 11, Wollongong, Nov 22, 2011 4
  5. 5. Example of a business/e-science process Influence Quality/cost influence? factors: performance algo2 quality of data Outputs: results, algo4 algo5 accuracy, algo1 performance algo3 algo6 etc. Quality of data? algo5 algo8 =? algo4 Reqs:DaaS algo7 Consumer 1, Consumer 2 Resources, quality and cost? Consumer n Resources in multiple clouds ASSI 11, Wollongong, Nov 22, 2011 5
  6. 6. Elasticity in computing „Elastic computing is the use of computer resources which vary dynamically to meet a variable workload” – „Clustering elasticity is the ease of adding or removing nodes from the distributed data store” – „What elasticity means to cloud users is that they should design their applications to scale their resource requirements up and down whenever possible.“, David Chiu – 11, Wollongong, Nov 22, 2011 6
  7. 7. Elasticity in computing – a broad view in multi-cloud environments  Elastic demands from consumers  Multiple outputs with different price and quality (output elasticity)  Elastic data inputs, e.g., deal with opportunistic data  Elastic pricing and quality models associated resources The service/process itself is elastic in terms of economic theory The service/process structure is elastic in terms of physics ASSI 11, Wollongong, Nov 22, 2011 7
  8. 8. Multiple elasticity dimensionsResource elasticity:software/human-based Non-functional parameters:computing elements, multiple performance, quality of data,clouds service availability, human trust, ... Pricing/Rewarding/Incentive elasticity ElasticityASSI 11, Wollongong, Nov 22, 2011 8
  9. 9. Conceptual architecture of elastic process environmentSchahram Dustdar, Yike Guo, Benjamin Satzger, Hong Linh Truong: Principles of Elastic Processes. IEEE Internet Computing 15(5): 66-71 (2011)ASSI 11, Wollongong, Nov 22, 2011 9
  10. 10. Elastic process – operation principles Operation principles  Monitor, manage and describe dynamic properties  Dynamically refine process functions based on quality  Determine costs based on multiple resource price/reward/incentive models  Provide elasticity across multiple cloud providers An EP can deal with multiple service objectives  N concurrent consumers, access markets of M providers, with K different requirements: K <= NASSI 11, Wollongong, Nov 22, 2011 10
  11. 11. Elastic process – modeling principles Overlaying EPs, functional composition, and dynamic property composition Modeling principles  EPs function as a static property  EPs results based on requirements concerning price/reward/incentive and quality  modeled as a set of constraints influencing resource elasticity  Communication can be based on SOAP/REST  Refinement and composition of resource, price and quality at multiple levels:  activities, fragments, or the whole EPASSI 11, Wollongong, Nov 22, 2011 11
  12. 12. The Vienna Elastic Computing ModelE-science Application Business Application  Service modeling and Elastic Service interface  Alignment techniques Elastic Alignment  Virtualization techniques IT Elastic Process  Monitoring and analysis techniques Elastic Runtime Management  Non-functional parameters characterization IT Elastic Service  Programming models  Governance and service Virtualization evolution Software-based Human-basedComputing Element Computing Element  Heavy crowded in cloud computing and social computing (SE) (HE)  Ready to use but in isolation (single cloud and single type of computing elements)ASSI 11, Wollongong, Nov 22, 2011 12
  13. 13. Clouds of hybrid systems Software- and human-based resource virtualization Middleware Modeling and alignment Evolution Programming languages Refinement and adaptation NFP Characterization, Monitoring and Analysis Pricing/Rewarding/Ince ntive mechanisms Governance and compliance ASSI 11, Wollongong, Nov 22, 2011 13
  14. 14. How to determine the cost model of a complex process?  Coarse- and fine-grainted cost models of clouds at different layers:  Too coarse-grained (networks, storages, machines) or too fine- grained (IO calls)  Application-, software-, data-, human-specific cost/performance models  Cost models for individual parts (workflow, MPI, OpenMP, etc.)  Elastic cost models associated with resources and services in multiple clouds Elastic Process Part Part Part A B COur approach cost/performance cost/performance cost/performance model i model j model k ASSI 11, Wollongong, Nov 22, 2011 14
  15. 15. How to monitor software based computing elements across clouds? Tran Vu Pham, Hong-Linh Truong, Schahram Dustdar "Elastic High Performance Applications - A Composition Framework", The 2011 Asia-Pacific Services Computing Conference (IEEE APSCC 2011), (c) IEEE Computer Society, December 12 - 15, 2011, Jeju, KoreaElastic high performanceapplications on multipleclouds Hong Linh Truong, Schahram Dustdar: Composable cost estimation and monitoring for computational applications in cloud computing environments. Procedia CS 1(1): 2175-2184 (2010) ASSI 11, Wollongong, Nov 22, 2011 15
  16. 16. Are current “software service techniques” suitable for DaaS/data marketplaces Elastic dynamic properties associated with: Data marketplaces and DaaS Service APIs, data APIs and data assets- Quang Hieu Vu, Tran-Vu Pham, Hong-Linh Truong, Schahram Dustdar, Rasool Asal,DEMODS: A Description Model for Data-as-a-Service. AINA 2012. To appear.- Hong-Linh Truong, Schahram Dustdar "On Analyzing and Specifying Concerns for Data as aService", The 2009 Asia-Pacific Services Computing Conference (IEEE APSCC 2009), (c) IEEEComputer Society, December 7-11, 2009, Biopolis, Singapore.- Marco Comerio, Hong-Linh Truong, Flavio De Paoli, Schahram Dustdar, " Evaluating ContractCompatibility for Service Composition in The SeCO2 Framework ", The 9th InternationalConference on Service Oriented Computing (ICSOC 2009), (c) Springer-Verlag, November 24 -27, 2009, Stockholm, Sweden. ASSI 11, Wollongong, Nov 22, 2011 16
  17. 17. How to monitor quality of data in an elastic way?Pull, pass-by-references Pull, pass-by-valuesPush, pass-by-values Hong Linh Truong, Schahram Dustdar: On Evaluating and Publishing Data Concerns for Data as a Service. APSCC 2010: 363-370 ASSI 11, Wollongong, Nov 22, 2011 17
  18. 18. How to deal with subjective and objective evaluation in long running processes? Our approach  Different QoD measurements  Human and software tasks ASSI 11, Wollongong, Nov 22, 2011 18
  19. 19. Experiments: cross-layered monitoring in multiple cloudsOnline analysis Estimation Cloud provider bill ASSI 11, Wollongong, Nov 22, 2011 19 19
  20. 20. Experiments: evaluating and publishing QoD with data resourcesASSI 11, Wollongong, Nov 22, 2011 20
  21. 21. Experiments: evaluating quality of data in workflowsMichael Reiter, Uwe Breitenbuecher, Schahram Dustdar, Dimka Karastoyanova, Frank Leymann, Hong-Linh Truong, A Novel Framework for Monitoring and Analyzing Quality of Data inSimulation Workflows, (c)IEEE Computer Society, The 7th IEEE International Conference on e-Science, 5-8 December, 2011, Stockholm, Sweden ASSI 11, Wollongong, Nov 22, 2011 21
  22. 22. Open questions for enabling techniques in hybrid systems Software, data and human integration on multiple clouds Monitor elastic processes in hybrid systems  Provisioning on-the-fly monitoring tools for complex cost models from different layers  Monitoring human-based computing elements Check contract compatibility and compliance – no static  Evaluating dynamic contract of data, software-based services, and human-based computing elements in the same process Evaluate quality of data in complex workflows  Modeling hybrid processes of human and software for evaluating quality of data in data intensive workflows  Utilizing quality of data in supporting the elasticity of cloud service and infrastructure provisioningASSI 11, Wollongong, Nov 22, 2011 22
  23. 23. Summary Elastic computing is needed in different aspects  Scaling software, services, and people in the same application Not just “resource elasticity”  Resource, price/reward/incentive and quality  Hybrid systems of software-based and human-based computing elements  Multiple clouds Several open research questions for realizing elastic processes of hybrid computing elementsASSI 11, Wollongong, Nov 22, 2011 23
  24. 24. Thanks for your attention! Hong-Linh Truong Distributed Systems Group Vienna University of Technology Austria 11, Wollongong, Nov 22, 2011 24