New Roles In The Cloud
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

New Roles In The Cloud

on

  • 2,951 views

Presentation on how developer roles change when meeting cloud infrastructure, and how a a "role driven"/template based VM deployment model helps this separation

Presentation on how developer roles change when meeting cloud infrastructure, and how a a "role driven"/template based VM deployment model helps this separation

Statistics

Views

Total Views
2,951
Views on SlideShare
2,909
Embed Views
42

Actions

Likes
5
Downloads
37
Comments
0

4 Embeds 42

http://www.1060.org 26
http://www.slideshare.net 9
http://1060.org 5
http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • 17 March 2010 HP Confidential
  • This is how things are today. Set up for conflict. The big one is developers "ship code that is functional" and ops "run secure services". 17 March 2010 HP Confidential
  • What does all this mean? You don’t need to predict your customer load in advance, though you had better hope your supplier can offer a service to match You don’ t have to wait a few weeks for some order of hardware to get delivered. You can’t buy HA kit: RAID, L7 routers, other nice things, to address availability. You need to design these in You can’t be sure your machines will stay around, that when they come back their names and IP Addresses may change You don’t have someone with a pager in the room who will track down network problems for you 17 March 2010 HP Confidential
  • 17 March 2010 HP Confidential
  • We really need to rethink how to design apps in this world, the old ways don’t. When a VM goes, so does any transient HDD. When a machine gets terminated and re-instantiated, it can have different hostname and address. Nor can that server deal with machines moving around. Which is a pity as the simplest way to deal with app trouble is to reset the VM. No need to worry about what its previous state March 17, 2010 HP Confidential
  • These are where Hadoop contains assumptions that are valid in the physical datacentre, but which don't work in a virtual world. 17 March 2010 HP Confidential
  • What does i t mean for teams March 17, 2010 HP Confidential
  • Here are some of the classic roles of back-end projects. There’s also graphic designers, marketing, content generation, etc. But this is the code side. Everyone’s job is hard. Biz dev: make sure the idea is good, predict demand , get the ops team to work with Arch and Finance to get machines to meet the demand Architecture: design something that works in the machines that ops will bring up Developers: code and test the app, produce something that works 17 March 2010 HP Confidential
  • Even if you design/code/test in a cycle, going live creates problems. Different systems, different networks, etc. Staging is meant to simplify this with a setup that mimics production, but it still has different users . March 17, 2010 HP Confidential
  • Once you stop needing a physical cluster of machines to test on, you can give every developer a virtual cluster which mimics that in production. 17 March 2010 HP Confidential
  • Developers shouldn’t be creating the machine configurations; that’s a job for the architect and ops Biz dev/management may be allowed to bring up machines, but they must be stopped from damaging anything March 17, 2010 HP Confidential
  • Developers shouldn’t be creating the machine configurations; that’s a job for the architect and ops Biz dev/management may be allowed to bring up machines, but they must be stopped from damaging anything March 17, 2010 HP Confidential
  • This for everyone to create machines. You can only create machines in roles you have the right to. This is more than a constrained image, much more of the config is locked down: VM, networking, dynamic options. March 17, 2010 HP Confidential
  • I’ve cheated and added some Hadoop-specificness in the web front end; you can create Hadoop workers and it knows to create the Master first, and passes the master hostname down so that the workers bond properly. This use case needs to be made generic March 17, 2010 HP Confidential
  • This is a fairly weak Web UI but it’s designed to feed into portals. It also happens to test easily. 17 March 2010 HP Confidential
  • 17 March 2010 HP Confidential

New Roles In The Cloud Presentation Transcript

  • 1. NEW ROLES IN THE CLOUD Steve Loughran HP Laboratories March 2010
  • 2.
    • Cloud computing will change
    • how applications are delivered
    • how applications are designed
    • how teams work
  • 3. CLASSIC SERVER-SIDE APPLICATIONS
    • Architect designs the application
    • Developers code and test on local machines
    • Operations buy and configure production machines
    • Developers get blame for things not working
    • Operations get blame for security & availability problems
    • Getting predictions of demand wrong can kill your project
  • 4. CLOUD ELIMINATES
    • Buying hardware based on predicted load
    • 2+ week lead time on new hardware, storage
    • High Availability
    • Homogeneity
    • Static machine names, addresses and capabilities
    • Stable machines
    • A fast private network
    • Someone in the datacentre who cares about you
  • 5. IMPLICATIONS FOR HARDWARE VENDORS
    • (servers, routers, storage, ... )
    • No. of major accounts drops to? 5? 10? Less?
    • Power consumption becomes a key concern
    • Network: power, agile routing, bandwidth, throttling
    • Servers: power, VM-hosting, uniformity, no-extra features
  • 6. AGILE APPLICATIONS
  • 7. WHAT HAS GONE?
    • High Availability through hardware
    • Static hostnames, network addresses,
    • rDNS, multicast IP
    • Maybe: SQL databases
  • 8. APPLICATIONS MUST BE AGILE
    • Directory, database or CM service to configure
    • Applications to rebind on loss of server connectivity
    • Use dynamic DNS services; don’t cache IPAddrs
    • Don’t expect HDD content to last on a single disk
    • Restart VMs on any app failure
    • Nothing is static. Nothing lasts .
  • 9. HADOOP’S ASSUMPTIONS
    • Master nodes don’t move
    • Workers can spin for them
    • Failed workers get blacklisted
    • Single, static hostnames
    • Cache all DNS entries
    • Disks don't move between hosts
    Different strategies are needed
  • 10. AGILE TEAMS
  • 11. CLASSIC TEAM ROLES Business Development Architecture Operations Development
  • 12. THE OLD PROCESS Business Development Architecture Operations Development Design Code Test Staging Live
  • 13. CLOUD-HOSTING BLURS THE ROLES Architecture Development Business Development Design Code Test Live Staging Operations
  • 14. CLOUD ARCHITECT
    • Agile Application
    • Scale up under load
    • Scale down when quiet
    • No known SPOF
    • Deploys with VMs
    Design an agile, HA application from cloud services and VMs
  • 15. CLOUD OPERATIONS
    • Instrumentation
    • Reconfiguration
    • Monitoring
    • Reporting
    • Datamining
    Create the VMs, manage and monitor staging and production
  • 16. ROLES IN CLOUD APIS
    • Different Team roles need different rights and different service interfaces
  • 17. ROLES IN THE CLOUD Role Task Tooling Cloud Architect Design App structure Text editor (?)) Powerpoint (? Cloud Operations Build VMs, set parameters Manage production Text under SCM Web & Command line Developers Request test VMs Web, IDE, build tools Biz Dev Worry about costs and revenue Web & Spreadsheet
  • 18. CLOUDFARMER
  • 19. Role Specification
    • machineRole extends HadoopVMRole {
    • description "A basic machine with SmartFrog installed";
    • vmPrefix "vm";
    • bootAutoVol "vol-0-1-47";
    • min 0;
    • recommendedMin 0;
    • links extends Compound {
    • "HTTP port 80" ["http", 80, "/"];
    • "HTTP port 8080" ["http", 8080, "/"];
    • }
    • }
  • 20. WEBAPP LISTS AVAILABLE ROLES
  • 21. HADOOP CLUSTER: MASTER + WORKER
  • 22. REQUEST COMPLETED
  • 23. LIST HOSTS IN A ROLE Actions: Add a worker, delete all in role “worker”
  • 24. LIST ALL HOSTS
  • 25. VIEW A HOST
  • 26. Benefits over (VM type, disks+)
    • Infrastructure can do some late binding install/deploy
    • Feeds into the UI
    • Web, IDE, build tools, ...
    • “ Role” CPU/network history can aid placement
    • Other templates: network, aggregate clusters
  • 27. Q&A