Cloud Computing

1
UC Berkeley
Cloud Computing:
Past, Present, and Future
Professor Anthony D. Joseph*, UC BerkeleyReliable Adaptive Distributed Systems Lab
RWTH Aachen
22 March 2010
http://abovetheclouds.cs.berkeley.edu/
*Director, Intel Research Berkeley

RAD Lab 5-year Mission
Enable 1 person to develop, deploy, operate next -generation Internet application
Key enabling technology: Statistical machine learning
debugging, monitoring, pwr mgmt, auto-configuration, perfprediction, ...
Highly interdisciplinary faculty & students
PI’s: Patterson/Fox/Katz (systems/networks), Jordan (machine learning), Stoica (networks & P2P), Joseph (security), Shenker (networks), Franklin (DB)
2 postdocs, ~30 PhD students, ~6 undergrads
Grad/Undergrad teaching integrated with research

Course Timeline
Friday
10:00-12:00 History of Cloud Computing: Time-sharing, virtual machines, datacenter architectures, utility computing
12:00-13:30 Lunch
13:30-15:00 Modern Cloud Computing: economics, elasticity, failures
15:00-15:30 Break
15:30-17:00 Cloud Computing Infrastructure: networking, storage, computation models
Monday
10:00-12:00 Cloud Computing research topics: scheduling, multiple datacenters, testbeds

Nexus: A common substrate for cluster computing
Joint work with Benjamin Hindman, Andy Konwinski, Matei Zaharia, Ali Ghodsi,
Scott Shenker, and Ion Stoica

Recall: Hadoop on HDFS
namenode
job submission node
namenode daemon
jobtracker
tasktracker
tasktracker
tasktracker
datanode daemon
datanode daemon
datanode daemon
Linux file system
Linux file system
Linux file system
…
…
…
slave node
slave node
slave node
Adapted from slides by Jimmy Lin, Christophe Bisciglia, Aaron Kimball, & Sierra Michels-Slettvet, Google Distributed Computing Seminar, 2007 (licensed under Creation Commons Attribution 3.0 License)

Problem
Rapid innovation in cluster computing frameworks
No single framework optimal for all applications
Energy efficiency means maximizing cluster utilization
Want to run multiple frameworks in a single cluster

What do we want to run in the cluster?
Pregel
Apache
Hama
Dryad
Pig

Why share the cluster between frameworks?
Better utilization and efficiency (e.g., take advantage of diurnal patterns)
Better data sharing across frameworks and applications

Solution
Nexus is an “operating system” for the cluster over which diverse frameworks can run
Nexus multiplexes resources between frameworks
Frameworks control job execution

Goals
Scalable
Robust (i.e., simple enough to harden)
Flexible enough for a variety of different cluster frameworks
Extensible enough to encourage innovative future frameworks

Question 1: Granularity of Sharing
Option: Coarse-grained sharing
Give framework a (slice of) machine for its entire duration
Data locality compromised if machine held for long time
Hard to account for new frameworks and changing demands -> hurts utilization and interactivity
Hadoop 1
Hadoop 2
Hadoop 3

Question 1: Granularity of Sharing
Nexus: Fine-grained sharing
Support frameworks that use smaller tasks (in time and space) by multiplexing them across all available resources
Hadoop 3
Hadoop 3
Hadoop 2
Hadoop 1
Frameworks can take turns accessing data on each node
Can resize frameworks shares to get utilization & interactivity
Hadoop 1
Hadoop 2
Hadoop 2
Hadoop 1
Hadoop 3
Hadoop 1
Hadoop 2
Hadoop 3
Hadoop 3
Hadoop 2
Hadoop 2
Hadoop 3
Hadoop 2
Hadoop 1
Hadoop 3
Hadoop 1
Hadoop 2

Question 2: Resource Allocation
Option: Global scheduler
Frameworks express needs in a specification language, a global scheduler matches resources to frameworks
Requires encoding a framework’s semantics using the language, which is complex and can lead to ambiguities
Restricts frameworks if specification is unanticipated
Designing a general-purpose global scheduler is hard

Question 2: Resource Allocation
Nexus: Resource offers
Offer free resources to frameworks, let frameworks pick which resources best suit their needs
Keeps Nexus simple and allows us to support future jobs
Distributed decisions might not be optimal

Outline
Nexus Architecture
Resource Allocation
Multi-Resource Fairness
Implementation
Results

Nexus Architecture

Hadoop job
Hadoop job
MPI job
Hadoop v20 scheduler
Hadoop v19 scheduler
MPI
scheduler
Nexus master
Nexus slave
Nexus slave
Nexus slave
MPI
executor
MPI
executor
Hadoop v19 executor
Hadoop v20 executor
Hadoop v19 executor
task
task
task
task
Overview
task

Hadoop job
Hadoop
scheduler
Nexus master
Nexus slave
Nexus slave
MPI executor
task
Resource Offers
MPI job
MPI
scheduler
Pick framework to offer to
Resourceoffer
MPI
executor
task

Resource Offers
MPI job
Hadoop job
MPI
scheduler
Hadoop
scheduler
offer = list of {machine, free_resources}
Example:
[ {node 1, <2 CPUs, 4 GB>},
{node 2, <2 CPUs, 4 GB>} ]
Resource offer
Nexus master
Nexus slave
Nexus slave
MPI executor
MPI
executor
task
task

Hadoop job
Hadoop
scheduler
Nexus master
Nexus slave
Nexus slave
Hadoop
executor
MPI executor
task
Resource Offers
MPI job
MPI
scheduler
Framework-specific scheduling
task
Resourceoffer
Launches & isolates executors
MPI
executor
task

Resource Offer Details
Min and max task sizes to control fragmentation
Filters let framework restrict offers sent to it
By machine list
By quantity of resources
Timeouts can be added to filters
Frameworks can signal when to destroy filters, or when they want more offers

Using Offers for Data Locality
We found that a simple policy called delay scheduling can give very high locality:
Framework waits for offers on nodes that have its data
If waited longer than a certain delay, starts launching non-local tasks

Framework Isolation
Isolation mechanism is pluggable due to the inherent perfomance/isolation tradeoff
Current implementation supports Solaris projects and Linux containers
Both isolate CPU, memory and network bandwidth
Linux developers working on disk IO isolation
Other options: VMs, Solaris zones, policing

Resource Allocation

Allocation Policies
Nexus picks framework to offer resources to, and hence controls how many resources each framework can get (but not which)
Allocation policies are pluggable to suit organization needs, through allocation modules

Example: Hierarchical Fairshare Policy
Cluster Share Policy
Facebook.com
20%
100%
80%
0%
Ads
Spam
User 2
User 1
14%
70%
30%
20%
100%
6%
Job 4
Job 3
Job 2
Job 1
CurrTime
CurrTime
CurrTime

Revocation
Killing tasks to make room for other users
Not the normal case because fine-grained tasks enable quick reallocation of resources
Sometimes necessary:
Long running tasks never relinquishing resources
Buggy job running forever
Greedy user who decides to makes his task long

Revocation Mechanism
Allocation policy defines a safe share for each user
Users will get at least safe share within specified time
Revoke only if a user is below its safe share and is interested in offers
Revoke tasks from users farthest above their safe share
Framework warned before its task is killed

How Do We Run MPI?
Users always told their safe share
Avoid revocation by staying below it
Giving each user a small safe share may not be enough if jobs need many machines
Can run a traditional grid or HPC scheduler as a user with a larger safe share of the cluster, and have MPI jobs queue up on it
E.g. Torque gets 40% of cluster

Example: Torque on Nexus
Facebook.com
Safe share = 40%
40%
20%
40%
Torque
Ads
Spam
User 2
User 1
MPI Job
MPI Job
MPI Job
MPI Job
Job 4
Job 1
Job 2
Job 1

Multi-Resource Fairness

What is Fair?
Goal: define a fair allocation of resources in the cluster between multiple users
Example: suppose we have:
30 CPUs and 30 GB RAM
Two users with equal shares
User 1 needs <1 CPU, 1 GB RAM> per task
User 2 needs <1 CPU, 3 GB RAM> per task
What is a fair allocation?

Definition 1: Asset Fairness
Idea: give weights to resources (e.g. 1 CPU = 1 GB) and equalize value of resources given to each user
Algorithm: when resources are free, offer to whoever has the least value
Result:
U1: 12 tasks: 12 CPUs, 12 GB ($24)
U2: 6 tasks: 6 CPUs, 18 GB ($24)
PROBLEM
User 1 has < 50% of both CPUs and RAM
User 1
User 2
100%
50%
0%
CPU
RAM

Lessons from Definition 1
“You shouldn’t do worse than if you ran a smaller, private cluster equal in size to your share”
Thus, given N users, each user should get ≥ 1/N of his dominating resource (i.e., the resource that he consumes most of)

Def. 2: Dominant Resource Fairness
Idea: give every user an equal share of her dominant resource (i.e., resource it consumes most of)
Algorithm: when resources are free, offer to the user with the smallest dominant share (i.e., fractional share of the her dominant resource)
Result:
U1: 15 tasks: 15 CPUs, 15 GB
U2: 5 tasks: 5 CPUs, 15 GB
User 1
User 2
100%
50%
0%
CPU
RAM

Fairness Properties

Implementation

Implementation Stats
7000 lines of C++
APIs in C, C++, Java, Python, Ruby
Executor isolation using Linux containers and Solaris projects

Frameworks
Ported frameworks:
Hadoop(900 line patch)
MPI (160 line wrapper scripts)
New frameworks:
Spark, Scala framework for iterative jobs (1300 lines)
Apache+haproxy, elastic web server farm (200 lines)

Results

Overhead
Less than 4% seen in practice

Dynamic Resource Sharing

Multiple Hadoops Experiment
Hadoop 1
Hadoop 2
Hadoop 3

Multiple Hadoops Experiment
Hadoop 3
Hadoop 3
Hadoop 2
Hadoop 1
Hadoop 1
Hadoop 1
Hadoop 2
Hadoop 2
Hadoop 1
Hadoop 3
Hadoop 1
Hadoop 2
Hadoop 2
Hadoop 3
Hadoop 3
Hadoop 2
Hadoop 2
Hadoop 3
Hadoop 2
Hadoop 3
Hadoop 1
Hadoop 1
Hadoop 3
Hadoop 2

Results with 16 Hadoops

Web Server Farm Framework

Web Framework Experiment
httperf
HTTP request
HTTP request
HTTP request
Load calculation
Scheduler (haproxy)
Load gen framework
task
resource offer
Nexus master
status update
Nexus slave
Nexus slave
Nexus slave
Web executor
Load gen executor
Web executor
Load gen executor
Load gen
executor
Web executor
task
(Apache)
task
task
(Apache)
task
task
task
task
(Apache)

Web Framework Results

Future Work
Experiment with parallel programming models
Further explore low-latency services on Nexus (web applications, etc)
Shared services (e.g. BigTable, GFS)
Deploy to users and open source

Cloud Computing Testbeds

Open Cirrus™: Seizing the Open Source Cloud Stack OpportunityA joint initiative sponsored by HP, Intel, and Yahoo!
http://opencirrus.org/

Proprietary Cloud Computing stacks
GOOGLE
AMAZON
MICROSOFT
Publicly accessible layer
Applications
Applications
Applications
Application Frameworks
MapReduce, Sawzall, Google App Engine, Protocol Buffers
EMR – Hadoop
.NET Services
Software Infrastructure
VM Management
Job Scheduling
Borg
Storage Management
GFS, BigTable
Monitoring
Borg
VM Management
EC2
Job Scheduling
Storage Management
S3, EBS
Monitoring
Borg
VM Management
Fabric Controller
Job Scheduling
Fabric Controller
Storage Management
SQL Services, blobs, tables, queues
Monitoring
Fabric Controller
Hardware Infrastructure
Borg
Fabric Controller

Applications
Monitoring
Ganglia
Nagios
Zenoss
MON
Moara
Storage
Management
HDFS
KFS
Gluster
Lustre
PVFS
MooseFS
HBase
Hypertable
Pig, Hadoop, MPI, Sprout, Mahout
VM Management
Job Scheduling
Storage Management
Monitoring
Job Scheduling
Maui/Torque
VM Management
Eucalyptus
Enomalism
Tashi
Reservoir
Nimbus,
oVirt
Hardware
Infrastructure
PRS
Emulab
Cobbler
xCat
PRS, Emulab, Cobbler, xCat
Open Cloud Computing stacks
Heavily fragmented
today!

Open Cirrus™ Cloud Computing Testbed
Shared: research, applications, infrastructure (12K cores), data sets
Global services: sign on, monitoring, store. Open src stack (prs, tashi, hadoop)
Sponsored by HP, Intel, and Yahoo! (with additional support from NSF)
9 sites currently, target of around 20 in the next two years.

Open Cirrus Goals
Goals
Foster new systems and services research around cloud computing
Catalyze open-source stack and APIs for the cloud
How are we unique?
Support for systems research and applications research
Federation of heterogeneous datacenters

Open Cirrus Organization
Central Management Office, oversees Open Cirrus
Currently owned by HP
Governance model
Research team
Technical team
New site additions
Support (legal (export, privacy), IT, etc.)
Each site
Runs its own research and technical teams
Contributes individual technologies
Operates some of the global services
E.g.
HP site supports portal and PRS
Intel site developing and supporting Tashi
Yahoo! contributes to Hadoop

Intel BigData Open Cirrus Site
Mobile Rack
8 (1u) nodes
-------------
2 Xeon E5440
(quad-core)
[Harpertown/
Core 2]
16GB DRAM
2 1TB Disk
http://opencirrus.intel-research.net
1 Gb/s
(x8 p2p)
1 Gb/s
(x4)
Switch
24 Gb/s
1 Gb/s
(x8)
1 Gb/s
(x4)
45 Mb/s T3
to Internet
Switch
48 Gb/s
*
Switch
48 Gb/s
1 Gb/s (x2x5 p2p)
1 Gb/s
(x4)
1 Gb/s
(x4)
1 Gb/s
(x4)
1 Gb/s
(x4)
1 Gb/s
(x4)
3U Rack
5 storage nodes
-------------
12 1TB Disks
Switch
48 Gb/s
Switch
48 Gb/s
Switch
48 Gb/s
Switch
48 Gb/s
Switch
48 Gb/s
1 Gb/s
(x4x4 p2p)
1 Gb/s
(x4x4 p2p)
1 Gb/s
(x15 p2p)
1 Gb/s
(x15 p2p)
1 Gb/s
(x15 p2p)
(r1r5)
PDU
w/per-port monitoring
and control
Blade Rack
40 nodes
Blade Rack
40 nodes
1U Rack
15 nodes
2U Rack
15 nodes
2U Rack
15 nodes
20 nodes: 1 Xeon (1-core) [Irwindale/Pent4], 6GB DRAM, 366GB disk (36+300GB)
10 nodes: 2 Xeon 5160 (2-core) [Woodcrest/Core], 4GB RAM, 2 75GB disks
10 nodes: 2 Xeon E5345 (4-core) [Clovertown/Core],8GB DRAM, 2 150GB Disk
2 Xeon E5345
(quad-core)
[Clovertown/
Core]
8GB DRAM
2 150GB Disk
2 Xeon E5420
(quad-core)
[Harpertown/
Core 2]
8GB DRAM
2 1TB Disk
2 Xeon E5440
(quad-core)
[Harpertown/
Core 2]
8GB DRAM
6 1TB Disk
2 Xeon E5520
(quad-core)
[Nehalem-EP/
Core i7]
16GB DRAM
6 1TB Disk
x3
x2
x2
Key:
rXrY=row X rack Y
rXrYcZ=row X rack Y chassis Z
(r2r2c1-4)
(r2r1c1-4)
(r1r1, r1r2)
(r1r3, r1r4, r2r3)
(r3r2, r3r3)

Open Cirrus Sites
Total
1,029
4 PB
12,074
1,746
26.3 TB

Testbed Comparison

Open Cirrus Stack
Compute + network +
storage resources
Management and
control subsystem
Power + cooling
Physical Resource set (Zoni) service
Credit: John Wilkes (HP)

Open Cirrus Stack
Research
Tashi
NFS storage
service
HDFS storage
service
PRS clients, each with theirown “physical data center”
Zoni service

Open Cirrus Stack
Research
Tashi
NFS storage
service
HDFS storage
service
Virtual cluster
Virtual cluster
Virtual clusters (e.g., Tashi)
Zoni service

Open Cirrus Stack
Research
Tashi
NFS storage
service
HDFS storage
service
Virtual cluster
Virtual cluster
Application running
On Hadoop
On Tashi virtual cluster
On a PRS
On real hardware
BigData App
Hadoop
Zoni service

Open Cirrus Stack
Research
Tashi
NFS storage
service
HDFS storage
service
Virtual cluster
Virtual cluster
Experiment/
save/restore
BigData app
Hadoop
Zoni service

Open Cirrus Stack
Research
Tashi
NFS storage
service
HDFS storage
service
Virtual cluster
Virtual cluster
Experiment/
save/restore
BigData App
Hadoop
Platform services
Zoni service

Open Cirrus Stack
Research
Tashi
NFS storage
service
HDFS storage
service
Virtual cluster
Virtual cluster
User services
Experiment/
save/restore
BigData App
Hadoop
Platform services
Zoni service

Open Cirrus Stack
Research
Tashi
NFS storage
service
HDFS storage
service
Virtual cluster
Virtual cluster
User services
Experiment/
save/restore
BigData App
Hadoop
Platform services
Zoni

System Organization
Compute nodes are divided into dynamically-allocated, vlan-isolated PRS subdomains
Apps switch back and forth between virtual and phyiscal
Open
service
research
Apps running in a VM mgmt infrastructure
(e.g., Tashi)
Tashi
development
Production
storage
service
Proprietary
service
research
Open workload monitoring and trace collection

Open Cirrus stack - Zoni
Zoni service goals
Provide mini-datacenters to researchers
Isolate experiments from each other
Stable base for other research
Zoni service approach
Allocate sets of physical co-located nodes, isolated inside VLANs.
Zoni code from HP being merged into Tashi Apache project and extended by Intel
Running on HP site
Being ported to Intel site
Will eventually run on all sites

Open Cirrus Stack - Tashi
An open source Apache Software Foundation project sponsored by Intel (with CMU, Yahoo, HP)
Infrastructure for cloud computing on Big Data
http://incubator.apache.org/projects/tashi
Research focus:
Location-aware co-scheduling of VMs, storage, and power.
Seamless physical/virtual migration.
Joint with Greg Ganger (CMU), Mor Harchol-Balter (CMU), Milan Milenkovic (CTG)

Node
Node
Node
Node
Node
Node
Tashi High-Level Design
Services are instantiated
through virtual machines
Most decisions happen in
the scheduler; manages
compute/storage/power
in concert
Data location
and power
information
is exposed
to scheduler
and services
Scheduler
Virtualization Service
Storage Service
The storage service aggregates the
capacity of the commodity nodes
to house Big Data repositories.
Cluster
Manager
Cluster nodes are assumed
to be commodity machines
CM maintains databases
and routes messages;
decision logic is limited

Location Matters (calculated)

73
Open Cirrus Stack - Hadoop
An open-source Apache Software Foundation project sponsored by Yahoo!
http://wiki.apache.org/hadoop/ProjectDescription
Provides a parallel programming model (MapReduce), a distributed file system, and a parallel database (HDFS)

What kinds of research projects are Open Cirrus sites looking for?
Open Cirrus is seeking research in the following areas (different centers will weight these differently):
Datacenter federation
Datacenter management
Web services
Data-intensive applications and systems
The following kinds of projects are generally not of interest:
Traditional HPC application development
Production applications that just need lots of cycles
Closed source system development

How do users get access to Open Cirrus sites?
Project PIs apply to each site separately.
Contact names, email addresses, and web links for applications to each site will be available on the Open Cirrus Web site (which goes live Q209)
http://opencirrus.org
Each Open Cirrus site decides which users and projects get access to its site.
Developing a global sign on for all sites (Q2 09)
Users will be able to login to each Open Cirrus site for which they are authorized using the same login and password.

Summary and Lessons
Intel is collaborating with HP and Yahoo! to provide a cloud computing testbed for the research community
Using the cloud as an accelerator for interactive streaming/big data apps is an important usage model
Primary goals are to
Foster new systems research around cloud computing
Catalyze open-source reference stack and APIs for the cloud
Access model, Local and global services, Application frameworks
Explore location-aware and power-aware workload scheduling
Develop integrated physical/virtual allocations to combat cluster squatting
Design cloud storage models
GFS-style storage systems not mature, impact of SSDs unknown
Investigate new application framework alternatives to map-reduce/Hadoop

Other Cloud Computing Research Topics: Isolation and DC Energy

Heterogeneity in Virtualized Environments
VM technology isolates CPU and memory, but disk and network are shared
Full bandwidth when no contention
Equal shares when there is contention
2.5x performance difference
EC2 small instances

Isolation Research
Need predictable variance over raw performance
Some resources that people have run into problems with:
Power, disk space, disk I/O rate (drive, bus), memory space (user/kernel), memory bus, cache at all levels (TLB, etc), hyperthreading/etc, CPU rate, interrupts
Network: NIC (Rx/Tx), Switch, cross-datacenter, cross-country
OS resources: File descriptors, ports, sockets

Datacenter Energy
EPA, 8/2007:
1.5% of total U.S. energy consumption
Growing from 60 to 100 Billion kWh in 5 yrs
48% of typical IT budget spent on energy
75 MW new DC deployments in PG&E’s service area – that they know about! (expect another 2x)
Microsoft: $500m new Chicago facility
Three substations with a capacity of 198MW
200+ shipping containers w/ 2,000 servers each
Overall growth of 20,000/month

81
Power/Cooling Issues

First Milestone: DC Energy Conservation
DCs limited by power
For each dollar spent on servers, add $0.48 (2005)/$0.71 (2010) for power/cooling
$26B spent to power and cool servers in 2005 grows to $45B in 2010
Within DC racks, network equipment often the “hottest” components in the hot spot

Thermal Image of Typical Cluster Rack
Rack
Switch
M. K. Patterson, A. Pratt, P. Kumar, “From UPS to Silicon: an end-to-end evaluation of datacenter efficiency”, Intel Corporation

DC Networking and Power
Selectively power down ports/portions of net elements
Enhanced power-awareness in the network stack
Power-aware routing and support for system virtualization
Support for datacenter “slice” power down and restart
Application and power-aware media access/control
Dynamic selection of full/half duplex
Directional asymmetry to save power, e.g., 10Gb/s send, 100Mb/s receive
Power-awareness in applications and protocols
Hard state (proxying), soft state (caching), protocol/data “streamlining” for power as well as b/w reduction
Power implications for topology design
Tradeoffs in redundancy/high-availability vs. power consumption
VLANs support for power-aware system virtualization

Summary
Many areas for research into Cloud Computing!
Datacenter design, languages, scheduling, isolation, energy efficiency (at all levels)
Opportunities to try out research at scale!
Amazon EC2, Open Cirrus, …

Thank you!
adj@eecs.berkeley.edu
http://abovetheclouds.cs.berkeley.edu/
86

http://www.hadoop.co.kr	10180
http://feeds.feedburner.com	7
http://www.hanrss.com	6
http://webcache.googleusercontent.com	5
http://twitter.com	1

Cloud Computing

by butest on Apr 26, 2010

Accessibility

Tags

Upload Details

Usage Rights

5 Embeds 10,199

Statistics

Cloud Computing — Presentation Transcript