Your SlideShare is downloading. ×
Cloud software engineering
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Cloud software engineering

1,561
views

Published on

Introduces the characteristics of clouds and discusses the challenges that these pose for engineering scientific applications on the cloud. Key challenges are programming models, developing PaaS …

Introduces the characteristics of clouds and discusses the challenges that these pose for engineering scientific applications on the cloud. Key challenges are programming models, developing PaaS interfaces for high performance/high throughput computing and developing an SDE for cloud programmng.

Published in: Technology, Business

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,561
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
86
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Challenges for cloud software engineering Ian Sommerville St Andrews University Challenges for Cloud Software Engineering, 2012 Slide 1
  • 2. Why is cloud software engineeringdifferent – or is it? What needs to be done to make cloudsoftware engineering easier for theresearch community? Challenges for Cloud Software Engineering, 2012 Slide 2
  • 3. Public, private, community clouds Challenges for Cloud Software Engineering, 2012 Slide 3
  • 4. There will be a number of public cloudproviders offering different capabilities atdifferent prices at different times Public cloud computing prices willcontinue to fall and it will be the mosteconomical approach Challenges for Cloud Software Engineering, 2012 Slide 4
  • 5. Private (institutional) clouds for scienceresearch are not really viable Community clouds make sense for data-intensive computing if they are locatedclose to the sources of the data Community clouds hosting standardsoftware may be a useful approach –possibly hosted on public clouds Challenges for Cloud Software Engineering, 2012 Slide 5
  • 6. What makes the cloud different (and challenging)? Challenges for Cloud Software Engineering, 2012 Slide 6
  • 7. Scale Thousands of nodes that may be distributed across the world Challenges for Cloud Software Engineering, 2012 Slide 7
  • 8. Elasticity No fixed hardware architecture Scale out and scale in Challenges for Cloud Software Engineering, 2012 Slide 8
  • 9. Pay as you go pricing Variable costs for applications Payment according to resources used Challenges for Cloud Software Engineering, 2012 Slide 9
  • 10. Cloud services Challenges for Cloud Software Engineering, 2012 Slide 10
  • 11. Infrastructure as a service Challenges for Cloud Software Engineering, 2012 Slide 11
  • 12. Instead of buying and running physical equipment (computers, storage, etc.), you ‘rent’ these as required from a cloud provider You ship your programs and data to a remote data centre and run them there rather than locally The familiar Grid computing model Challenges for Cloud Software Engineering, 2012 Slide 12
  • 13. Platform as a service Challenges for Cloud Software Engineering, 2012 Slide 13
  • 14. The cloud provider makes a set of APIs available which you use in your program to interact with the platform Scaleability is transparent. Automatic scale out/scale in as demand changes Primarily geared to writing web-based applications Challenges for Cloud Software Engineering, 2012 Slide 14
  • 15. Software as a service Challenges for Cloud Software Engineering, 2012 Slide 15
  • 16. Software is shared by different users and accessed using a web browser Services are stateless, functional abstractions Multi-tenant databases for efficient operation Accessed via a service interface – SOAP or RESTful interface Challenges for Cloud Software Engineering, 2012 Slide 16
  • 17. Levels of cloud usage Challenges for Cloud Software Engineering, 2012 Slide 17
  • 18. Cloud hosting You use the cloud to provide server capability rather than incurring the capital costs of server purchase Applications need not be ‘cloud aware’ Fixed server infrastructure Can run existing codes in Fortran/C Challenges for Cloud Software Engineering, 2012 Slide 18
  • 19. If server type and platform available, thenshould not require any applicationchanges Almost certainly poorer performancethan local execution. But makes sense for occasional runs/demonstrations etc. Challenges for Cloud Software Engineering, 2012 Slide 19
  • 20. Problems Relatively low level model of interaction. You need to know about lots of stuff (such as certificates, security keys, etc. that have nothing to do with your work) Performance is difficult to predict License issues Costs may be high for high volumes of data transfer in/out of clouds Challenges for Cloud Software Engineering, 2012 Slide 20
  • 21. Software engineering challenge Build a software development environment that radically simplifies hosting scientific applications on a range of clouds Challenges for Cloud Software Engineering, 2012 Slide 21
  • 22. Cloud interfacing You build applications that make use ofthe cloud providers APIs (PaaS) to accesscommon services. ‘High-level’ cloud awareness. The platformtakes care of scaleability on demand. Limited programming language support Challenges for Cloud Software Engineering, 2012 Slide 22
  • 23. Challenge is using existing PaaS APIs for computationally-intensive applications –  Systems are set up to support web-based applications –  Google Apps – 30 second time limit on tasks Significant application change required Several experiments here – mixed conclusions. Challenges for Cloud Software Engineering, 2012 Slide 23
  • 24. Software engineering challenge Investigate how to adapt applications that are computation/data intensive to run within the constraints set by the PaaS interfaces from cloud providers Challenges for Cloud Software Engineering, 2012 Slide 24
  • 25. Cloud software engineering Build ‘cloud aware’ applications which can adapt to the cloud capabilities that are available Challenges for Cloud Software Engineering, 2012 Slide 25
  • 26. Application architecture designed for the cloud Underlying infrastructure management done in the program itself, depending on performance and cost requirements What abstractions do we need for this? Challenges for Cloud Software Engineering, 2012 Slide 26
  • 27. Three grand challenges Challenges for Cloud Software Engineering, 2012 Slide 27
  • 28. Programming models for the cloud Building a PaaS for high performance/ throughput computing Cloud-aware software development environments Challenges for Cloud Software Engineering, 2012 Slide 28
  • 29. Scaling up and scaling out Traditional model to cope with increased computational demand is to scale up Supported by some cloud providers who offer different instance types. Scaling out is the more usual cloud model Challenges for Cloud Software Engineering, 2012 Slide 29
  • 30. Coping with scaling out Redesigning algorithms and code to work with a flexible rather than a fixed number of resources –  Interaction between program and platform to know what resources are being used and what resources are available –  Structuring program into a flexible number of computational components –  Avoiding sequential computations (Amdahl’s Law) Challenges for Cloud Software Engineering, 2012 Slide 30
  • 31. Programming models Are there programming models that can take advantage of elasticity and massive parallelism? –  Invent new models for parallel computation –  Adapt existing problems to the current map/reduce programming model Challenges for Cloud Software Engineering, 2012 Slide 31
  • 32. Map reduce explained http://www.kchodorow.com/blog/2010/03/15/mapreduce-the-fanfiction/ Challenges for Cloud Software Engineering, 2012 Slide 32
  • 33. Map-reduce on a slide -- Split the data into a number of computationally tractable chunks.  in = split (thedata) n = size (in) -- MAP. For each chunk, process in parallel  parfor i = 1 to n do {      out (i) = F (in (i)) } -- REDUCE. Apply a function R to the output that computes the required result -- (of course, A can be a list) A = R (out) Challenges for Cloud Software Engineering, 2012 Slide 33
  • 34. Map-reduce assumptions Input data set can be split into chunks for independent parallel processing The reduce action is a relatively simple, fast computation Clearly, does not fit all types of problem Challenges for Cloud Software Engineering, 2012 Slide 34
  • 35. Can scientific codes be re-engineered to take advantage of the map-reduce model? Elastic + static architecture Use the elastic cloud for data preparation/analysis (MAP) Use a cloud cluster for computation (REDUCE) Challenges for Cloud Software Engineering, 2012 Slide 35
  • 36. Opportunistic computing Taking advantage of low-price/free resources when these are available? Challenges for Cloud Software Engineering, 2012 Slide 36
  • 37. Fixed budget computation Compute till you run out of money Get as much computational capability as your budget will allow Use spot prices Stop and start instances Challenges for Cloud Software Engineering, 2012 Slide 37
  • 38. Budget-driven programming Develop systems that take advantage of variable pricing in different clouds/at different times Potentially significant for computationally-intensive applications Challenges for Cloud Software Engineering, 2012 Slide 38
  • 39. PaaS for Science Develop a set of APIS for data/compute intensive applications to support cloud interaction Long-running applications Cost/demand awareness Performance measurement/tuning Intermittent execution Challenges for Cloud Software Engineering, 2012 Slide 39
  • 40. SDE for the cloud Creating an environment for the development of cloud-aware applications Hiding low level details of IaaS cloud interaction Testing environment Occasional computation Preliminary work – ELVIRA, St Andrews http://cloudresearch.jiscinvolve.org/wp/category/projects/ elvira/ Challenges for Cloud Software Engineering, 2012 Slide 40
  • 41. Software as a service Challenges for Cloud Software Engineering, 2012 Slide 41
  • 42. Effective way for a community to share services Scope for applying external computations to collected data – data mining, visualisation, etc. What are appropriate abstractions? How can large amounts of state be managed? Limiting data transfer between nodes As with all software reuse, the community rather than the project benefits. Needs central funding to pay for reusable element creation Challenges for Cloud Software Engineering, 2012 Slide 42
  • 43. Conclusions Clouds will become the dominant computing environment in future so costs will fall significantly – major driver for developing for the cloud Offers a level of computational resource that cannot really be replicated in an institution Software engineering issues need to be addressed to make the most effective use of the cloud Challenges for Cloud Software Engineering, 2012 Slide 43