Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Advanced Services Engineering,                              WS 2012 Emerging Dynamic Distributed Systems and Challenges fo...
Outline Today‘s Internet Computing Some emerging models – properties and issues    Data provisioning models    Computa...
Today‘s Internet Computing   Internet infrastructure and software connect    contents, things, and people, each has diffe...
Today‘s Internet Computing                      Social             Technologies and                    computing          ...
Discussion time:  WHICH EMERGING FORMS OF  COMPUTING MODELS,  SYSTEMS AND APPLICATIONS  DO YOU SEE?ASE WS 2012         5
Some emerging data provisioning          models (1)              Large (near-       • Satellites and environmental/city se...
Some emerging data provisioning            models (2)   A lot                    A few                    A lot  Things   ...
Examples of large-scale (near-)          realtime dataASE WS 2012        8
Large-scale (near-)realtime data:       properties and issuesSome properties                Some issues Having massive da...
Example of open dataASE WS 2012        10
Open data: properties and issuesSome properties             Some issues Having large, multiple     Fine-grained content ...
Marketable data examplesASE WS 2012        12
Marketable data: properties and       issuesSome properties               Some issues Can be large, multiple       Multi...
Emerging computational      infrastructure/platform provisioning      models Infrastructure-as-a-Service    Machine-as-a...
Examples of Infrastructure-as-a-                       ServiceSource: Hong Linh Truong, Schahram Dustdar: Cloud computing ...
Examples of Platform-as-a-ServiceSource: Hong Linh Truong, Schahram Dustdar: Cloud computing for small research groups in ...
Examples of multiple cloudsSource: Katarzyna Keahey, Mauricio Tsugawa, Andrea Matsunaga, and Jose Fortes. 2009. Sky Comput...
Emerging computational       infrastructure/platform provisioning       models– properties and issuesSome properties      ...
Emerging human computation        models Crowdsourcing platforms    (Anonymous) people computing capabilities exploited ...
Examples of human computation                (1)  Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: ...
Examples of human computation                (2)  Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew Mc...
Examples of human computation                (3)  Source: Muhammad Z.C. Candra, Rostyslav Zabolotnyi, Hong-Linh Truong, an...
Human computation models –       properties and issuesSome properties            Some issues Huge number of people     Q...
Summary of emerging models wrt            advanced service-based systems                             Engineering advanced ...
Discussion time:  WHERE ARE  OPPORTUNITIES?  DO I NEED TO TAKE  OPPORTUNITIES? WHY?ASE WS 2012          25
Recall our motivating example (1)         Infrastructure/Internet of Things          Internet/public cloud                ...
Recall our motivating                                                         Can we combine them                         ...
Recall our motivating example (3)   Source: http://www.undata-api.org/                                                  So...
Discussion time: WHICH OPPORTUNITIES DO YOU SEE?ASE WS 2012         29
Internet-scale service engineering -          - the elasticityASE WS 2012         30
Internet-scale service engineering -        - big/near-real time data impact Which are data concerns that impact the data...
Internet-scale service engineering -              - Steps                                 Single service/platform engineer...
Discussion time: WHAT ARE MISSING?ASE WS 2012         33
Single service/platform engineering          – service unit (1)  The service model and the unit concept can be applied   ...
Single service/platform engineering                – service units (2) Source: Stefan Tai, Philipp Leitner, Schahram Dustd...
Single service/platform engineering                       – service unit provisioning      Provisioning software under se...
Single service/platform engineering        – examples (1) Service engineering with a single  system/platform    Using Ex...
Single service/platform engineering        – examples (2)ASE WS 2012      38
Internet-scale multi-platform                services engineering – required                technologies                  ...
Discussion time WHAT ARE MISSING?ASE WS 2012        40
Exercises Read papers mentioned in slides    Get their main ideas Check services mentioned in examples    Examine capa...
Thanks for              your attention                Hong-Linh Truong                Distributed Systems Group           ...
Upcoming SlideShare
Loading in …5
×

TUW- 184.742 Emerging Dynamic Distributed Systems and Challenges for Advanced Services Engineering

423 views

Published on

This presentation is part of the course "184.742 Advanced Services Engineering" at The Vienna University of Technology, in Winter Semester 2012. Check the course at: http://www.infosys.tuwien.ac.at/teaching/courses/ase/

Published in: Education
  • Be the first to comment

  • Be the first to like this

TUW- 184.742 Emerging Dynamic Distributed Systems and Challenges for Advanced Services Engineering

  1. 1. Advanced Services Engineering, WS 2012 Emerging Dynamic Distributed Systems and Challenges for Advanced Services Engineering Hong-Linh Truong Distributed Systems Group, Vienna University of Technology truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truongASE WS 2012 1
  2. 2. Outline Today‘s Internet Computing Some emerging models – properties and issues  Data provisioning models  Computational infrastructures/frameworks provisioning  Human computation provisioning Internet-scale service engineering Single service/platform engineeringASE WS 2012 2
  3. 3. Today‘s Internet Computing  Internet infrastructure and software connect contents, things, and people, each has different roles (computation, sensing, analytics, etc.) Size does Any * access Economic matter behaviour does matter factors do matter Things Large-scale Unpredictable On-demand, pay- interactions workload as-you-go Big data Scalability ComplexSoftware People generated contract Internet infrastructure and Big quantities to be managed software Hard to control quality ASE WS 2012 3
  4. 4. Today‘s Internet Computing Social Technologies and computing computing models Cloud Service Computing Computing Peer-to- Distributed Peer Computing Computing converge Things Emerging forms of computing introduces models, systems Software People and applicationsASE WS 2012 4
  5. 5. Discussion time: WHICH EMERGING FORMS OF COMPUTING MODELS, SYSTEMS AND APPLICATIONS DO YOU SEE?ASE WS 2012 5
  6. 6. Some emerging data provisioning models (1) Large (near- • Satellites and environmental/city sensor networks (e.g., from specific orgs/countries) ) realtime • Machine-to-machine (e.g., from companies) • Social media (e.g., from people + platform providers) data • Open science and engineering data sets Open data • Open government data Marketable • Statistics and business data • Commercial data in general data Data are assetsASE WS 2012 6
  7. 7. Some emerging data provisioning models (2) A lot A few A lot Things Social Data/Service Platforms Platforms Data Profiling Data and Enrichment Storage APPs Environtments ... Data Data Analytics Query Infrastructures ....ASE WS 2012 7
  8. 8. Examples of large-scale (near-) realtime dataASE WS 2012 8
  9. 9. Large-scale (near-)realtime data: properties and issuesSome properties Some issues Having massive data  Timely analytics Requiring large-scale, big  Performance and (near-) real time scalability processing and storing  Quality control capabilities  Handle of unknown data Enabling predictive and patterns realtime data analytics  Benefit/cost versus quality tradeoffsASE WS 2012 9
  10. 10. Example of open dataASE WS 2012 10
  11. 11. Open data: properties and issuesSome properties Some issues Having large, multiple  Fine-grained content data sources but mainly search static data  Balance between Having good quality processing cost and control in many cases performance Usually providing the data as a whole setASE WS 2012 11
  12. 12. Marketable data examplesASE WS 2012 12
  13. 13. Marketable data: properties and issuesSome properties Some issues Can be large, multiple  Multiple levels of data sources but mainly service/data contracts static data  Compatible with other Having good quality data sources w.r.t. control contract Have strong data contract  Cost w.r.t. up-to-date terms data Some do not offer the whole datasetASE WS 2012 13
  14. 14. Emerging computational infrastructure/platform provisioning models Infrastructure-as-a-Service  Machine-as-a service  Storage as a Service  Database as a Service Platform-as-a-Service  Middleware  Computational frameworks Software Defined NetworkingASE WS 2012 14
  15. 15. Examples of Infrastructure-as-a- ServiceSource: Hong Linh Truong, Schahram Dustdar: Cloud computing for small research groups in computational science and engineering: current status and outlook. Computing 91(1): 75-91 (2011) Amazon S3 Microsoft AruzeAnd more MongoLab OKEANOS ASE WS 2012 15
  16. 16. Examples of Platform-as-a-ServiceSource: Hong Linh Truong, Schahram Dustdar: Cloud computing for small research groups in computational science and engineering: current status and outlook. Computing 91(1): 75-91 (2011) Amazon Elastic MapReduceAnd more StormMQ Globus Online (GO) ASE WS 2012 16
  17. 17. Examples of multiple cloudsSource: Katarzyna Keahey, Mauricio Tsugawa, Andrea Matsunaga, and Jose Fortes. 2009. Sky Computing. IEEE InternetComputing 13, 5 (September 2009), 43-51. DOI=10.1109/MIC.2009.94 http://dx.doi.org/10.1109/MIC.2009.94  aaa SOCloud WS 2011 17
  18. 18. Emerging computational infrastructure/platform provisioning models– properties and issuesSome properties Some issues Rich types of services  On-demand information from multiple providers management from  Better choices in terms of multiple sources functions and costs  APIs complexity Concepts are similar but  Cross-vendor integration diverse APIs  Data locality Strong dependencies/tight ecosystemsASE WS 2012 18
  19. 19. Emerging human computation models Crowdsourcing platforms  (Anonymous) people computing capabilities exploited via task bids Individual Compute Unit  An individual is treated like „a processor“ or “functional unit“. A service can wrap human capabilities to support the communication and coordination of tasks Social Compute Unit  A set of people and software that are initiated and provisioned as a service for solving tasksThe main point: humans are a computing elementASE WS 2012 19
  20. 20. Examples of human computation (1) Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured social computing. UIST 2011: 53-64ASE WS 2012 20
  21. 21. Examples of human computation (2) Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew McGregor: AutoMan: a platform for integrating human-based and digital computation. OOPSLA 2012: 639-654ASE WS 2012 21
  22. 22. Examples of human computation (3) Source: Muhammad Z.C. Candra, Rostyslav Zabolotnyi, Hong-Linh Truong, and Schahram Dustdar, Virtualizing Software and Human for Elastic Hybrid Services, Web Services Handbook, (c)Springer-Verlag, 2012.ASE WS 2012 22
  23. 23. Human computation models – properties and issuesSome properties Some issues Huge number of people  Quality control Capabilities might not  Reliability assurance know in advance  Proactive, on-demand Simple coordination acquisition models  Incentive strategiesASE WS 2012 23
  24. 24. Summary of emerging models wrt advanced service-based systems Engineering advanced service- based systems utilize/consist of Emerging data provisioning models Things Emerging data provisioning models Software PeopleEmerging computational Emerging human Emerging data infrastructure/platform computation provisioning provisioning models models modelsASE WS 2012 24
  25. 25. Discussion time: WHERE ARE OPPORTUNITIES? DO I NEED TO TAKE OPPORTUNITIES? WHY?ASE WS 2012 25
  26. 26. Recall our motivating example (1) Infrastructure/Internet of Things Internet/public cloud Organization-specific boundary boundary Equipment Operation and Maintenance Emergency Management Near Enterprise Civil protection realtime analytics Resource Planning Building Operation Predictive data Optimization analytics Tracking/Log istics Visual Analytics Infrastructure Monitoring ...Cities, e.g. including:10000+ buildings1000000+ sensors Can we combine open government data with building monitoring data? ASE WS 2012 26
  27. 27. Recall our motivating Can we combine them with open government example (2) data?Soilmoistureanalysis forSentinel-1A lot of input data (L0):~2.7 TB per dayA lot of results (L1, L2): e.g., L1 has ~140 MB per day for a grid of 1kmx1km Michael Hornacek,Wolfgang Wagner, Daniel Sabel, Hong-Linh Truong, Paul Snoeij, Thomas Hahmann, Erhard Diedrich, Marcela Doubkova, Potential for High Resolution Systematic Global Surface Soil Moisture Retrieval Via Change Detection Using Sentinel-1, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, April, 2012 ASE WS 2012 27
  28. 28. Recall our motivating example (3) Source: http://www.undata-api.org/ Source: http://www.strikeiron.com/Catalog/StrikeIronServices.aspx Source: http://docs.gnip.com/w/page/23722723/Introduction- to-GnipASE WS 2012 28
  29. 29. Discussion time: WHICH OPPORTUNITIES DO YOU SEE?ASE WS 2012 29
  30. 30. Internet-scale service engineering - - the elasticityASE WS 2012 30
  31. 31. Internet-scale service engineering - - big/near-real time data impact Which are data concerns that impact the data processing? How to use data concerns to optimize data analytics and service provisioning? How to use available data assets for advanced services in an elastic manner? What are the role of human-based servies in dealing with complex data analytics?ASE WS 2012 31
  32. 32. Internet-scale service engineering - - Steps Single service/platform engineeringService units for representing Provisioning of fundamental Engineering with single fundamental things, people service units service units and software Understanding Properties/Concerns Data /Service/Application Monitoring, evaluation and Utilization of data/serviceconcerns; their dependencies provisioning of concerns concerns Large-scale, multi-platform services engineering Identify design units, selection development and Identify the scale,platform/application of existing service Integration, complexity and *city problems units; Optimization ASE WS 2012 32
  33. 33. Discussion time: WHAT ARE MISSING?ASE WS 2012 33
  34. 34. Single service/platform engineering – service unit (1)  The service model and the unit concept can be applied to things, people and software Consumption, ownership, Service provisioning, price, etc. model Service unit „basic component“/“basic function“ modeling Unit Concept and descriptionASE WS 2012 34
  35. 35. Single service/platform engineering – service units (2) Source: Stefan Tai, Philipp Leitner, Schahram Dustdar: Design by Units: Abstractions for Human and Compute Resources for Elastic Systems. IEEE Internet Computing 16(4): 84-88 (2012)ASE WS 2012 35
  36. 36. Single service/platform engineering – service unit provisioning  Provisioning software under services  Provisioning things under services  Provisioning human under services  Crowd platforms of massive numbers of individuals  Individual Compute Unit (ICU)  Social Compute Unit (SCU)1. Mark Turner, David Budgen, and Pearl Brereton. 2003. Turning Software into a Service. Computer 36, 10 (October 2003), 38-44. DOI=10.1109/MC.2003.1236470 http://dx.doi.org/10.1109/MC.2003.12364702. Luigi Atzori, Antonio Iera, and Giacomo Morabito. 2010. The Internet of Things: A survey. Comput. Netw. 54, 15 (October 2010), 2787-2805. DOI=10.1016/j.comnet.2010.05.010 http://dx.doi.org/10.1016/j.comnet.2010.05.0103. Dominique Guinard, Vlad Trifa, Stamatis Karnouskos, Patrik Spiess, Domnic Savio: Interacting with the SOA-Based Internet of Things: Discovery, Query, Selection, and On-Demand Provisioning of Web Services. IEEE T. Services Computing 3(3): 223-235 (2010)4. Schahram Dustdar, Kamal Bhattacharya: The Social Compute Unit. IEEE Internet Computing 15(3): 64-69 (2011)5. Hong-Linh Truong, Schahram Dustdar, Kamal Bhattacharya "Programming Hybrid Services in the Cloud", Springer-Verlag, 10th International Conference on Service-oriented Computing (ICSOC 2012), November 12-16, 2012, Shanghai, China ASE WS 2012 36
  37. 37. Single service/platform engineering – examples (1) Service engineering with a single system/platform  Using Excel to access Azure datamarket places  Using Boto to access data in Amazon S3  Using Hadoop within a cluster to process local data  Using workflows to process data (e.g., Trident/Taverna/ASKALON)  Using StormMQ to store messagesASE WS 2012 37
  38. 38. Single service/platform engineering – examples (2)ASE WS 2012 38
  39. 39. Internet-scale multi-platform services engineering – required technologies Middleware (e.g., Workflows (e.g., StormMQ) Trident) Crowd platforms, Data services (e.g., human-based service Azure, S3) platforms(e.g., Mturks, VieCOM) Internet-scale, Data Multi-platform Services Billing/Monitoring analysis/Computation (e.g., services in cluster Engineering for thecurrencycloud) (e.g., Hadoop) Software, Things and PeopleASE WS 2012 39
  40. 40. Discussion time WHAT ARE MISSING?ASE WS 2012 40
  41. 41. Exercises Read papers mentioned in slides  Get their main ideas Check services mentioned in examples  Examine capabilities of the mentioned services  Including price models and underlying technologies  Examine their size and scale  Examine their ecosystems and dependencies Work on possible categories of single service units that are useful for your work  Some common service units with capabilities and providersASE WS 2012 41
  42. 42. Thanks for your attention Hong-Linh Truong Distributed Systems Group Vienna University of Technology truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truongASE WS 2012 42

×