TUW-ASE-Summer 2014: Engineering human-based services in complex processes
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

TUW-ASE-Summer 2014: Engineering human-based services in complex processes

  • 137 views
Uploaded on

 

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
137
On Slideshare
137
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
6
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Engineering human-based services in complex processes Hong-Linh Truong Distributed Systems Group, Vienna University of Technology truong@dsg.tuwien.ac.at http://dsg.tuwien.ac.at/staff/truong 1ASE Summer 2014 Advanced Services Engineering, Summer 2014 – Lecture 9 Advanced Services Engineering, Summer 2014 – Lecture 9
  • 2. Outline  Human capabilities for complex processes  Human service units  Incorporating human units into complex services  Frameworks  Some issues and solutions ASE Summer 2014 2
  • 3. Human-based services for solving complex problems (1) 3 We need to actively finding and binding human capabilities into the application! We need to actively finding and binding human capabilities into the application! Humans and software for solving complex problems ASE Summer 2014
  • 4. Human-based services for solving complex problems (2) 4 But how to program human-based services and software-based services together? But how to program human-based services and software-based services together? ASE Summer 2014
  • 5. Example: some common tasks in data analytics ASE Summer 2014 5 Human activities in Data Analytics Labeling Annotating Cleansing Evaluating data/content Dectecting patterns Classifying objects Steering analytics
  • 6. Human service units in data analytics -- functions  Evaluating: is the quality of picture good?  Classifying: is it a man‘s or a woman‘s picture?  Decting: any unidentified object in a picture?  Labeling: adding location information of a picture  Cleansing: remove duplicated pictures  Steering: the quality of picture is bad, should we continue to merge it with others? ASE Summer 2014 6 How to model such functions for human units ? E.g., with WSDL or REST? How to model such functions for human units ? E.g., with WSDL or REST?
  • 7. HUMAN SERVICE UNITS ASE Summer 2014 7
  • 8. Human service units ASE Summer 2014 8 Human service unit Human service unit Functions Non- functional parameters Service models Technical interfaces Provisioning mechanisms Interaction models Human acting as a „service unit“Human acting as a „service unit“
  • 9. Human service units 9 Modeling type of units (e.g., computation, data, monitor,) and their dependencies Consumption, ownership, provisioning, price, etc. Elastic Service Unit Service model Unit Dependency Elastic Capability Function The functional capability of the unit and interface to access the function Capabilities to be elastic under different requirements SoftwareSoftware PeoplePeople VolunteersProfessionals ThingThing Resources ASE Summer 2014
  • 10. Forms of human service  Individual Compute Unit  An individual is treated like „a processor“ or “functional unit“. A service can wrap human capabilities to support the communication and coordination of tasks  Social Compute Unit  A set of people and software that are initiated and provisioned as a service for solving tasks  Web services interfaces can be built  Different pricing models and different quality models ASE Summer 2014 10
  • 11. Human service units – provisioning mechanisms (1)  An infrastructure can be introduced for accessing many ICUs in a crowd  Allow people to register their service unit capabilities  Facilitate communication, task bidding, retrieval and result delivery  Act like a marketplace: multiple providers and multiple consumers ASE Summer 2014 11 ICU ICU ICUICU ICU ICU ICU CrowdCrowd
  • 12. Human service units – provisioning mechanisms (2)  An „infrastructure-as-a-service“ for ICUs  Facilitate communication, task retrieval and result delivery  Single ICUaaS provider and multiple consumers ASE Summer 2014 12 ICU ICU ICUICU ICU ICU ICU Cloud provisioning models ICUICU
  • 13. Human service units – provisioning mechanisms (3)  An „infrastructure-as-a-service“ for SCUs  Facilitate communication, task retrieval and result delivery  Single SCUaaS provider and multiple consumers ASE Summer 2014 13 SCU SCU SCUSCU SCU SCU SCU Cloud provisioning models SCUSCU
  • 14. Human service units – technical interfaces (1) ASE Summer 2014 14 People Web Servicce (REST/SOAP) People Web page Analytics Activity Analytics Activity
  • 15. Human service units – technical interfaces (2) ASE Summer 2014 15 Email/SMS/Instant Messaging People Web page Web Servicce (REST/SOAP) Communication Services People Analytics Activity Analytics Activity
  • 16. Human service units – interaction model ASE Summer 2014 16 ICU/SCU Task queue Scheduler ICU/SCU Task queue Scheduler Analytics Activity Analytics Activity ICU/SCU Analytics Activity
  • 17. Human service units -- NfPs ASE Summer 2014 17 Which are important considerations when interpreting NfP for human units? Which are important considerations when interpreting NfP for human units?
  • 18. Incorporating human units into complex processes  How to provision and employ human compute units?  How to select human units?  Where to place human units in data analytics and why?  How to monitor and test human units in data analytics? ASE Summer 2014 18
  • 19. PROVISIONING AND EMPLOYING HUMAN UNITS-- SOME FRAMEWORKS ASE Summer 2014 19
  • 20. Qurk system architecture (1) ASE Summer 2014 20 Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB Endow. 5, 1 (September 2011), 13-24. Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB Endow. 5, 1 (September 2011), 13-24.
  • 21. Qurk system architecture (2) ASE Summer 2014 21 Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB Endow. 5, 1 (September 2011), 13-24. Source: Adam Marcus, Eugene Wu, David Karger, Samuel Madden, and Robert Miller. 2011. Human-powered sorts and joins. Proc. VLDB Endow. 5, 1 (September 2011), 13-24.
  • 22. Jabberwocky approach (1) ASE Summer 2014 22 Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured social computing. UIST 2011: 53-64 Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured social computing. UIST 2011: 53-64
  • 23. Jabberwocky approach (2) ASE Summer 2014 23 Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured social computing. UIST 2011: 53-64 Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured social computing. UIST 2011: 53-64
  • 24. Automan approach ASE Summer 2014 24 Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew McGregor: AutoMan: a platform for integrating human-based and digital computation. OOPSLA 2012: 639-654 Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew McGregor: AutoMan: a platform for integrating human-based and digital computation. OOPSLA 2012: 639-654
  • 25. SW4H approach (1) ASE Summer 2014 25 Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm, David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina: Service-based Integration of Human Users in Workflow-driven Scientific Experiments. In: Proceedings of the 8th IEEE International Conference on eScience (eScience 2012 Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm, David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina: Service-based Integration of Human Users in Workflow-driven Scientific Experiments. In: Proceedings of the 8th IEEE International Conference on eScience (eScience 2012
  • 26. SW4H approach (2)  Similar concepts in collaborative working environments but integrated into workflows  Do not discuss about where and how to select human units ASE Summer 2014 26 Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm, David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina: Service-based Integration of Human Users in Workflow- driven Scientific Experiments. In: Proceedings of the 8th IEEE International Conference on eScience (eScience 2012 Karastoyanova, Dimka; Dentsas, Dimitrios; Schumm, David; Sonntag, Mirko; Sun, Lina; Vukojevic, Karolina: Service-based Integration of Human Users in Workflow- driven Scientific Experiments. In: Proceedings of the 8th IEEE International Conference on eScience (eScience 2012
  • 27. HUMAN SERVICE UNITS IN VIENNA ELASTIC COMPUTING MODEL ASE Summer 2014 27
  • 28. VieCOM -- incorporate humans into a programming paradigm (1) • Abstracting human compute units as program elements • Extending programming languages to support human compute units • Data/control flows via extensible APIs Programming languages • Shared memory (e.g., human –software – human), message passing (human-to-human), artifact-centric, etc., via APIs working atop the compute unit abstraction layer Multiple programming models • Computing capability /profile management: human computing power, reputation and incentive models • Monitoring and enforcing incentives/rewards, quality of results, availability • Communication between human-middleware, among Individual Compute Units (SCU)/Social Compute Units (SCU) for exchanging artifacts and comprehensing l tasks Execution environment 28 http://dsg.tuwien.ac.at/research/viecomASE Summer 2014
  • 29. VieCOM-- incorporate humans into a programming paradigm (2) Volunteers ProfessionalsTeamIndividual Service-based Middleware CommunicationCommunicationMonitoringMonitoring Capability/Profile Management Capability/Profile Management Provisioning/Negotiation/Execution APIProvisioning/Negotiation/Execution API Abstraction of Human-based Compute Units SCU SC U SC U SCU SCU Program languages and programming models Program elements Software Compute Units program human actions and dependencies program incentive condition and rewarding action program result evaluation method Human-to-middleware interfaces: •visualization of collective tasks •embedding of common forms •mobile app 29ASE Summer 2014
  • 30. Cloud of hybrid service units Cloud of HBS: A cloud of HBS includes HBS that can be provisioned, deployed, and utilized on-demand based on different pricing and incentive models. Cloud of HBS: A cloud of HBS includes HBS that can be provisioned, deployed, and utilized on-demand based on different pricing and incentive models. Cloud of hybrid services: A cloud of hybrid services includes SBS and HBS that can be provisioned, deployed and utilized on-demand based on different pricing and incentive models. Cloud of hybrid services: A cloud of hybrid services includes SBS and HBS that can be provisioned, deployed and utilized on-demand based on different pricing and incentive models. Sevilla
  • 31. SCU for independent tasks Application / Workflow SCU Provisoning Middleware Human-Task Requests ICU Clouds ● Task-based crowdsourcing platforms ● Collections of experts on SN ● Enterprise ICU pools Consumer Problems - Complex tasks - Quality control - Flexible quality requirements Problems - Complex tasks - Quality control - Flexible quality requirements SCU ICU Human-Task Requests - ICU: individuals - SCU: a collective of collaborative individuals - ICU: individuals - SCU: a collective of collaborative individuals
  • 32. Elastic SCU provisioning atop ICUs Elastic profile SCU (pre-)runtime/static formation Cloud APIs Muhammad Z.C. Candra, Hong-Linh Truong, and Schahram Dustdar, Provisioning Quality-aware Social Compute Units in the Cloud, ICSOC 2013. Muhammad Z.C. Candra, Hong-Linh Truong, and Schahram Dustdar, Provisioning Quality-aware Social Compute Units in the Cloud, ICSOC 2013. Algorithms  Ant Colony Optimization variants  FCFS  Greedy SCU extension/reduction  Task reassignment based on trust, cost, availability Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute Units, CAISE 2014 Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute Units, CAISE 2014
  • 33. Selecting human units  Do not select at all  Let human units bid the tasks  E.g., in crowdsourcing platforms  Static/fix mapping  E.g., using static information for human-task mapping  Simple selection techniques  Using the requirement of the task to find the suitable human units based on their capabilities  Complex selection techniques  Utilizing complex dependency graphs to find suitable human units ASE Summer 2014 33
  • 34. Selecting SCU based on task graphs 34 SCU Formation Algorithms Business As Usual Corrective Action HBS Constraints Hong Linh Truong, Schahram Dustdar, Kamal Bhattacharya: Programming Hybrid Services in the Cloud. ICSOC 2012: 96-110Hong Linh Truong, Schahram Dustdar, Kamal Bhattacharya: Programming Hybrid Services in the Cloud. ICSOC 2012: 96-110 ASE Summer 2014
  • 35. Utilizing hybrid services for evolving/dependent task graphs Hong-Linh Truong, Schahram Dustdar, Kamal Bhattacharya "Programming Hybrid Services in the Cloud", 10th International Conference on Service-oriented Computing (ICSOC 2012), November 12-16, 2012, Shanghai, China. Best Paper Award. Hong-Linh Truong, Schahram Dustdar, Kamal Bhattacharya "Programming Hybrid Services in the Cloud", 10th International Conference on Service-oriented Computing (ICSOC 2012), November 12-16, 2012, Shanghai, China. Best Paper Award.
  • 36. Placement techniques for human units  Usually at design time the developer/designer decides  Where to put human units  Where some triggers should be put in order to invoke human units if needed  At runtime  Find suitable human units  Invoke human units  Placement of human units  Application-specific  Needs automatic algorithms and supporting tools ASE Summer 2014 36
  • 37. Configuring iSCU  Establish „connectedness“ based on compliance constraints and network topology  Addional cost might occur!  Program SBS and HBS for the iSCU to have a complete working environment.  Different connectedness  E.g., ring-based, star-based, and master-slave topologies
  • 38. Hybrid compute units 38 Hybrid compute unit (HCU): a set of service units includes software-based services, human-based services and things-based services that can be provisioned, deployed and utilized as a collective on-demand based on different quality, pricing and incentive models. Hybrid compute unit (HCU): a set of service units includes software-based services, human-based services and things-based services that can be provisioned, deployed and utilized as a collective on-demand based on different quality, pricing and incentive models. ASE Summer 2014
  • 39. Hybrid compute unit design – fundamental elements 39ASE Summer 2014 Hong-Linh Truong, Hoa Khanh Dam, Aditya Ghose, Schahram Dustdar "Augmenting Complex Problem Solving with Hybrid Compute Units",9th International Workshop on Engineering Service-Oriented Application (WESOA's 2013), In conjunction with ICSOC 2013, Dec 2, 2013, Berlin, Germany, (c)Springer-Verlag Hong-Linh Truong, Hoa Khanh Dam, Aditya Ghose, Schahram Dustdar "Augmenting Complex Problem Solving with Hybrid Compute Units",9th International Workshop on Engineering Service-Oriented Application (WESOA's 2013), In conjunction with ICSOC 2013, Dec 2, 2013, Berlin, Germany, (c)Springer-Verlag
  • 40. Hybrid compute unit design -- Relationships 40 Relationship Type HBS SBS TBS HCU Similarity Yes Yes Yes Yes Composition Yes Yes Yes Yes Data Dependency Yes Yes Yes Yes Control Dependency Yes Yes Yes Yes Location Dependency Yes Yes Yes Yes Forwarding Yes Yes No Yes Delegation Yes Yes No Yes Social Relation Yes No No Yes Elasticity Yes Yes No Yes ASE Summer 2014
  • 41. Towards programming framework for HCU 41 Coordination and Composition Models Coordination and Composition Models [ICSOC12] ASE Summer 2014
  • 42. Illustrating Example  Evaluating quality of data (QoD) in simulations/data analytics processes  Typical components in simulation workflows/data analytics  Goal using hCAS to support quality of data control of the simulations/data analytics  Reasons: reducing wasting time and computational cost, dealing with tacit quality evaluation, etc. 42 pre- processing solving post- processing data analysis ASE Summer 2014
  • 43. HCU and simulations 43  Only SBS is needed,  for example, in the QoD evaluation step before pre-processing  SBS or HBS is used interchangeably or in combination  for example, in the QoD evaluation after pre- processing,  Only HBS is used  e.g., in QoD evaluation after solving. ASE Summer 2014
  • 44. hCAS and simulation workflows/data analytics 44 HCU ASE Summer 2014
  • 45. HOW DO WE EVALUATE HUMAN ACTIVITIES IN DATA ANALYTICS ASE Summer 2014 45
  • 46. Turkalytics  Develop „Interaction Model“ for human activities  Monitor and analyze metrics, e.g., performance and location ASE Summer 2014 46 Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473 http://doi.acm.org/10.1145/1963405.1963473 Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473 http://doi.acm.org/10.1145/1963405.1963473
  • 47. Turkalytics  Track client detail at client side (in browser)  Store tracked information in a log server  Analyze events ASE Summer 2014 47 Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473 http://doi.acm.org/10.1145/1963405.1963473 Paul Heymann and Hector Garcia-Molina. 2011. Turkalytics: analytics for human computation. In Proceedings of the 20th international conference on World wide web (WWW '11). ACM, New York, NY, USA, 477-486. DOI=10.1145/1963405.1963473 http://doi.acm.org/10.1145/1963405.1963473
  • 48. VieCOM  SCU execution model and lifecycle management  Metrics for ICUs and SCUs Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute Units, CAISE 2014 Mirela Riveni, Hong-Linh Truong, and Schahram Dustdar, On the Elasticity of Social Compute Units, CAISE 2014
  • 49. Exercises  Read mentioned papers  Analyze pros and cons of existing frameworks for data analytics  Survey existing algorithms for matching human units to data analytics tasks  Examine requirements for locating places for human units and implement some algorithms  Examine monitoring techniques for cloud of human compute units ASE Summer 2014 49
  • 50. 50 Thanks for your attention Hong-Linh Truong Distributed Systems Group Vienna University of Technology truong@dsg.tuwien.ac.at http://dsg.tuwien.ac.at/staff/truong ASE Summer 2014