• Like
  • Save
TUW- 184.742 Emerging Dynamic Distributed Systems and Challenges for Advanced Services Engineering
Upcoming SlideShare
Loading in...5
×
 

TUW- 184.742 Emerging Dynamic Distributed Systems and Challenges for Advanced Services Engineering

on

  • 301 views

This presentation is part of the course "184.742 Advanced Services Engineering" at The Vienna University of Technology, in Winter Semester 2012. Check the course at: ...

This presentation is part of the course "184.742 Advanced Services Engineering" at The Vienna University of Technology, in Winter Semester 2012. Check the course at: http://www.infosys.tuwien.ac.at/teaching/courses/ase/

Statistics

Views

Total Views
301
Views on SlideShare
300
Embed Views
1

Actions

Likes
0
Downloads
4
Comments
0

1 Embed 1

http://www.slashdocs.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    TUW- 184.742 Emerging Dynamic Distributed Systems and Challenges for Advanced Services Engineering TUW- 184.742 Emerging Dynamic Distributed Systems and Challenges for Advanced Services Engineering Presentation Transcript

    • Advanced Services Engineering, WS 2012 Emerging Dynamic Distributed Systems and Challenges for Advanced Services Engineering Hong-Linh Truong Distributed Systems Group, Vienna University of Technology truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truongASE WS 2012 1
    • Outline Today‘s Internet Computing Some emerging models – properties and issues  Data provisioning models  Computational infrastructures/frameworks provisioning  Human computation provisioning Internet-scale service engineering Single service/platform engineeringASE WS 2012 2
    • Today‘s Internet Computing  Internet infrastructure and software connect contents, things, and people, each has different roles (computation, sensing, analytics, etc.) Size does Any * access Economic matter behaviour does matter factors do matter Things Large-scale Unpredictable On-demand, pay- interactions workload as-you-go Big data Scalability ComplexSoftware People generated contract Internet infrastructure and Big quantities to be managed software Hard to control quality ASE WS 2012 3
    • Today‘s Internet Computing Social Technologies and computing computing models Cloud Service Computing Computing Peer-to- Distributed Peer Computing Computing converge Things Emerging forms of computing introduces models, systems Software People and applicationsASE WS 2012 4
    • Discussion time: WHICH EMERGING FORMS OF COMPUTING MODELS, SYSTEMS AND APPLICATIONS DO YOU SEE?ASE WS 2012 5
    • Some emerging data provisioning models (1) Large (near- • Satellites and environmental/city sensor networks (e.g., from specific orgs/countries) ) realtime • Machine-to-machine (e.g., from companies) • Social media (e.g., from people + platform providers) data • Open science and engineering data sets Open data • Open government data Marketable • Statistics and business data • Commercial data in general data Data are assetsASE WS 2012 6
    • Some emerging data provisioning models (2) A lot A few A lot Things Social Data/Service Platforms Platforms Data Profiling Data and Enrichment Storage APPs Environtments ... Data Data Analytics Query Infrastructures ....ASE WS 2012 7
    • Examples of large-scale (near-) realtime dataASE WS 2012 8
    • Large-scale (near-)realtime data: properties and issuesSome properties Some issues Having massive data  Timely analytics Requiring large-scale, big  Performance and (near-) real time scalability processing and storing  Quality control capabilities  Handle of unknown data Enabling predictive and patterns realtime data analytics  Benefit/cost versus quality tradeoffsASE WS 2012 9
    • Example of open dataASE WS 2012 10
    • Open data: properties and issuesSome properties Some issues Having large, multiple  Fine-grained content data sources but mainly search static data  Balance between Having good quality processing cost and control in many cases performance Usually providing the data as a whole setASE WS 2012 11
    • Marketable data examplesASE WS 2012 12
    • Marketable data: properties and issuesSome properties Some issues Can be large, multiple  Multiple levels of data sources but mainly service/data contracts static data  Compatible with other Having good quality data sources w.r.t. control contract Have strong data contract  Cost w.r.t. up-to-date terms data Some do not offer the whole datasetASE WS 2012 13
    • Emerging computational infrastructure/platform provisioning models Infrastructure-as-a-Service  Machine-as-a service  Storage as a Service  Database as a Service Platform-as-a-Service  Middleware  Computational frameworks Software Defined NetworkingASE WS 2012 14
    • Examples of Infrastructure-as-a- ServiceSource: Hong Linh Truong, Schahram Dustdar: Cloud computing for small research groups in computational science and engineering: current status and outlook. Computing 91(1): 75-91 (2011) Amazon S3 Microsoft AruzeAnd more MongoLab OKEANOS ASE WS 2012 15
    • Examples of Platform-as-a-ServiceSource: Hong Linh Truong, Schahram Dustdar: Cloud computing for small research groups in computational science and engineering: current status and outlook. Computing 91(1): 75-91 (2011) Amazon Elastic MapReduceAnd more StormMQ Globus Online (GO) ASE WS 2012 16
    • Examples of multiple cloudsSource: Katarzyna Keahey, Mauricio Tsugawa, Andrea Matsunaga, and Jose Fortes. 2009. Sky Computing. IEEE InternetComputing 13, 5 (September 2009), 43-51. DOI=10.1109/MIC.2009.94 http://dx.doi.org/10.1109/MIC.2009.94  aaa SOCloud WS 2011 17
    • Emerging computational infrastructure/platform provisioning models– properties and issuesSome properties Some issues Rich types of services  On-demand information from multiple providers management from  Better choices in terms of multiple sources functions and costs  APIs complexity Concepts are similar but  Cross-vendor integration diverse APIs  Data locality Strong dependencies/tight ecosystemsASE WS 2012 18
    • Emerging human computation models Crowdsourcing platforms  (Anonymous) people computing capabilities exploited via task bids Individual Compute Unit  An individual is treated like „a processor“ or “functional unit“. A service can wrap human capabilities to support the communication and coordination of tasks Social Compute Unit  A set of people and software that are initiated and provisioned as a service for solving tasksThe main point: humans are a computing elementASE WS 2012 19
    • Examples of human computation (1) Source: Salman Ahmad, Alexis Battle, Zahan Malkani, Sepandar D. Kamvar: The jabberwocky programming environment for structured social computing. UIST 2011: 53-64ASE WS 2012 20
    • Examples of human computation (2) Source: Daniel W. Barowy, Charlie Curtsinger, Emery D. Berger, Andrew McGregor: AutoMan: a platform for integrating human-based and digital computation. OOPSLA 2012: 639-654ASE WS 2012 21
    • Examples of human computation (3) Source: Muhammad Z.C. Candra, Rostyslav Zabolotnyi, Hong-Linh Truong, and Schahram Dustdar, Virtualizing Software and Human for Elastic Hybrid Services, Web Services Handbook, (c)Springer-Verlag, 2012.ASE WS 2012 22
    • Human computation models – properties and issuesSome properties Some issues Huge number of people  Quality control Capabilities might not  Reliability assurance know in advance  Proactive, on-demand Simple coordination acquisition models  Incentive strategiesASE WS 2012 23
    • Summary of emerging models wrt advanced service-based systems Engineering advanced service- based systems utilize/consist of Emerging data provisioning models Things Emerging data provisioning models Software PeopleEmerging computational Emerging human Emerging data infrastructure/platform computation provisioning provisioning models models modelsASE WS 2012 24
    • Discussion time: WHERE ARE OPPORTUNITIES? DO I NEED TO TAKE OPPORTUNITIES? WHY?ASE WS 2012 25
    • Recall our motivating example (1) Infrastructure/Internet of Things Internet/public cloud Organization-specific boundary boundary Equipment Operation and Maintenance Emergency Management Near Enterprise Civil protection realtime analytics Resource Planning Building Operation Predictive data Optimization analytics Tracking/Log istics Visual Analytics Infrastructure Monitoring ...Cities, e.g. including:10000+ buildings1000000+ sensors Can we combine open government data with building monitoring data? ASE WS 2012 26
    • Recall our motivating Can we combine them with open government example (2) data?Soilmoistureanalysis forSentinel-1A lot of input data (L0):~2.7 TB per dayA lot of results (L1, L2): e.g., L1 has ~140 MB per day for a grid of 1kmx1km Michael Hornacek,Wolfgang Wagner, Daniel Sabel, Hong-Linh Truong, Paul Snoeij, Thomas Hahmann, Erhard Diedrich, Marcela Doubkova, Potential for High Resolution Systematic Global Surface Soil Moisture Retrieval Via Change Detection Using Sentinel-1, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, April, 2012 ASE WS 2012 27
    • Recall our motivating example (3) Source: http://www.undata-api.org/ Source: http://www.strikeiron.com/Catalog/StrikeIronServices.aspx Source: http://docs.gnip.com/w/page/23722723/Introduction- to-GnipASE WS 2012 28
    • Discussion time: WHICH OPPORTUNITIES DO YOU SEE?ASE WS 2012 29
    • Internet-scale service engineering - - the elasticityASE WS 2012 30
    • Internet-scale service engineering - - big/near-real time data impact Which are data concerns that impact the data processing? How to use data concerns to optimize data analytics and service provisioning? How to use available data assets for advanced services in an elastic manner? What are the role of human-based servies in dealing with complex data analytics?ASE WS 2012 31
    • Internet-scale service engineering - - Steps Single service/platform engineeringService units for representing Provisioning of fundamental Engineering with single fundamental things, people service units service units and software Understanding Properties/Concerns Data /Service/Application Monitoring, evaluation and Utilization of data/serviceconcerns; their dependencies provisioning of concerns concerns Large-scale, multi-platform services engineering Identify design units, selection development and Identify the scale,platform/application of existing service Integration, complexity and *city problems units; Optimization ASE WS 2012 32
    • Discussion time: WHAT ARE MISSING?ASE WS 2012 33
    • Single service/platform engineering – service unit (1)  The service model and the unit concept can be applied to things, people and software Consumption, ownership, Service provisioning, price, etc. model Service unit „basic component“/“basic function“ modeling Unit Concept and descriptionASE WS 2012 34
    • Single service/platform engineering – service units (2) Source: Stefan Tai, Philipp Leitner, Schahram Dustdar: Design by Units: Abstractions for Human and Compute Resources for Elastic Systems. IEEE Internet Computing 16(4): 84-88 (2012)ASE WS 2012 35
    • Single service/platform engineering – service unit provisioning  Provisioning software under services  Provisioning things under services  Provisioning human under services  Crowd platforms of massive numbers of individuals  Individual Compute Unit (ICU)  Social Compute Unit (SCU)1. Mark Turner, David Budgen, and Pearl Brereton. 2003. Turning Software into a Service. Computer 36, 10 (October 2003), 38-44. DOI=10.1109/MC.2003.1236470 http://dx.doi.org/10.1109/MC.2003.12364702. Luigi Atzori, Antonio Iera, and Giacomo Morabito. 2010. The Internet of Things: A survey. Comput. Netw. 54, 15 (October 2010), 2787-2805. DOI=10.1016/j.comnet.2010.05.010 http://dx.doi.org/10.1016/j.comnet.2010.05.0103. Dominique Guinard, Vlad Trifa, Stamatis Karnouskos, Patrik Spiess, Domnic Savio: Interacting with the SOA-Based Internet of Things: Discovery, Query, Selection, and On-Demand Provisioning of Web Services. IEEE T. Services Computing 3(3): 223-235 (2010)4. Schahram Dustdar, Kamal Bhattacharya: The Social Compute Unit. IEEE Internet Computing 15(3): 64-69 (2011)5. Hong-Linh Truong, Schahram Dustdar, Kamal Bhattacharya "Programming Hybrid Services in the Cloud", Springer-Verlag, 10th International Conference on Service-oriented Computing (ICSOC 2012), November 12-16, 2012, Shanghai, China ASE WS 2012 36
    • Single service/platform engineering – examples (1) Service engineering with a single system/platform  Using Excel to access Azure datamarket places  Using Boto to access data in Amazon S3  Using Hadoop within a cluster to process local data  Using workflows to process data (e.g., Trident/Taverna/ASKALON)  Using StormMQ to store messagesASE WS 2012 37
    • Single service/platform engineering – examples (2)ASE WS 2012 38
    • Internet-scale multi-platform services engineering – required technologies Middleware (e.g., Workflows (e.g., StormMQ) Trident) Crowd platforms, Data services (e.g., human-based service Azure, S3) platforms(e.g., Mturks, VieCOM) Internet-scale, Data Multi-platform Services Billing/Monitoring analysis/Computation (e.g., services in cluster Engineering for thecurrencycloud) (e.g., Hadoop) Software, Things and PeopleASE WS 2012 39
    • Discussion time WHAT ARE MISSING?ASE WS 2012 40
    • Exercises Read papers mentioned in slides  Get their main ideas Check services mentioned in examples  Examine capabilities of the mentioned services  Including price models and underlying technologies  Examine their size and scale  Examine their ecosystems and dependencies Work on possible categories of single service units that are useful for your work  Some common service units with capabilities and providersASE WS 2012 41
    • Thanks for your attention Hong-Linh Truong Distributed Systems Group Vienna University of Technology truong@dsg.tuwien.ac.at http://www.infosys.tuwien.ac.at/staff/truongASE WS 2012 42