Dan Schatzberg, Jonathan Appavoo, Orran Krieger, and Eric Van Hensbergen. Scalable elastic systems architecture. In Proceedings of the ASPLOS Runtime Environment/Systems, Layering, and Virtualized Environments (RESoLVE) Workshop. ASPLOS, March 2011.

1. Scalable Elastic System
Architecture (SESA)
Dan Schatzberg, Boston University
Jonathan Appavoo, Boston University
Orran Krieger,VMware
Eric Van Hensbergen, IBM Research Austin

2. The goal
Perform more computation with fewer
resources

3. Fixed Resources
• Hardware as a fixed resource
• Focus on reducing computation’s need for
hardware resources
• Multiplex hardware resources for different
computations

4. Elastic Resources
• Cloud Computing
• Pay as you go hardware
• Focus on providing hardware to the
computation that requires it

5. Time to scale hardware
Days Minutes
Fixed
Cloud Computing
Hardware

6. Time to scale hardware
Days Minutes
Fixed
Cloud Computing
Hardware
Elastic Applications

7. Time to scale hardware
Days Minutes Milliseconds
Fixed
Cloud Computing
Hardware
?
Elastic Applications

8. Interactive HPC
• Medical imaging application
• interactive
• 1 megapixel image
• quadratic memory consumption - ~14TB

9. Interactive HPC
• Fixed Hardware
• Purchase a cluster

10. Interactive HPC
• Cloud Computing
• Allocate a cluster
• Maintain interactivity
• 650+ EC2 instances - $8000 dollars / 8
hour day

11. Can we do better?

12. Where we’re starting
Treat elasticity as a first-class system
characteristic

13. OUTLINE
1. THE PROBLEM
2. OBSERVATIONS
1. Top-Down Demand
2. Bottom-Up Support
3. Modularity
3. OUR TAKE ON A SOLUTION
4. PROTOTYPE & CHALLENGES

14. Top-Down Demand
System Interface
Software
Hardware

15. Top-Down Demand
System Interface
Hardware

16. Top-Down Demand
System Interface
Hardware

17. Top-Down Demand
System Interface
Hardware

18. Top-Down Demand
System Interface
Hardware

19. Top-Down Demand
System Interface
Hardware

20. Events as Load
• Treat a service request as an event that is
dispatched to resources
• As events occur, load increases
• As events are handled, load decreases
• Each layer being event-driven forces
demand to flow top-down

21. Bottom-Up Support
System Interface
Hardware

22. Bottom-Up Support
System Interface
Hardware
Allocate/Deallocate
Resources

23. Bottom-Up Support
System Interface
Hardware

24. Elastic Interface
• Support elasticity by interfacing via
allocation and deallocation of physical or
logical resources
• Each layer is constructed by being explicit
with respect to resource consumption
• Be explicit with respect to time to meet a
request

25. Modularity
System Interface
Hardware

26. Modularity
System Interface
Hardware

27. Modularity
System Interface
Hardware

28. Modularity
System Interface
Hardware

29. Object model
• Objects can take advantage
• the semantics of their request patterns
• the lifetime of an instance
• the occupancy w.r.t memory, processing
and communication
• We can optimize for elasticity by taking
advantage of modularity in a system

30. OUTLINE
1. THE PROBLEM
2. OBSERVATIONS
3. OUR TAKE ON A SOLUTION : SESA
1. EBB’s : Elastic Building Blocks
2. SEE: Scalable Elastic Executive - A LibOS
3. EPIC: Events as Interrupts
4. PROTOTYPE & CHALLENGES

31. Architecture Overview
Hardware

32. Architecture Overview
FAWN
SSD

33. Architecture Overview
Partitioning
FAWN
SSD

34. Architecture Overview
Kittyhawk
FAWN
SSD

35. Architecture Overview
System Software
Kittyhawk
FAWN
SSD

36. Architecture Overview
HAL
Kittyhawk
FAWN
SSD

37. Architecture Overview
Applications
HAL
Kittyhawk
FAWN
SSD

38. Architecture Overview
?
HAL
Kittyhawk
FAWN
SSD

39. Architecture Overview
?
HAL
Kittyhawk
FAWN
SSD

40. SESA
SE APP/SERVICE
EBB Namespace System Software Layers
Component Layer
SEExecutive
SEE SEE SEE LibOS Layer
SEMachine
SEHAL SEHAL SEHAL Hardware Abstraction Layer
VM/ VM/ VM/
Node Node Node Partitioning Layer
Elastic Partition of Nodes

41. EBB’s
A new Component Model
for expressing and
EBB NameSpace encapsulating fine grain
elasticity.
The Next Generation of
Clustered Objects.

42. Clustered Objects (CO)
dref(ctr)->inc();
val() dec()
de
de
de
de
de
de
de
de
val
val
val
val
val
val
val
val
c
c
c
c
c
c
c
c
C C C C C C C C
inc inc inc inc inc inc inc inc
inc()
Processors Processors
c c c c
Memory cMemory
c c c

43. What did we learn?
• Event-driven architecture for lazy and
dynamic instantiation of resources
• Mechanism to create scalable software

44. Elastic Building Blocks
• Programming Model for
Elastic and Scalable
Components
• Span multiple nodes
• Built in On Demand nature
-- encapsulation of policies
for both allocation and
deallocation of resources

45. SEE
A Distributed Library OS
Model designed to enable
SEExecutive Elastic Software within the
SEE SEE SEE
context of legacy
environments.
Next Generation of Libra

46. Libra
Architecture
Controller Application Application Application vironment for both u
Partition Partition Partition Partition tition is launched fro
hypervisor to create a
App
This script also launc
App tion to access the con
App
App
Gateway
The gateway serv
which is a compact op
App
App
App
General DB systems. Inferno crea
Purpose JVM services such as the u
the network (see Fig
OS Libra Libra Libra pace remotely via the
over a shared-memor
and application parti
Hypervisor tensions to Inferno a
Figure 1. Proposed system architecture. transport is available
Note that nothing
access all resources
nels, hypervisors run other operating systems with few or no mod- allows resources and
ifications [27, 5, 42]. By running an operating system (the con- partition and accesse

47. Libra
X86 Linux Front Ends
Pool of Libra Partitions
$
9p
$
PowerPC Blades: Libra Workers

48. Libra
X86 Linux Front Ends
Pool of Libra Partitions
$ java -cp my.jar
9p
$
PowerPC Blades: Libra Workers

49. Libra
X86 Linux Front Ends
Pool of Libra Partitions
$ java -cp my.jar
9p
$
PowerPC Blades: Libra Workers

50. Libra
X86 Linux Front Ends
Pool of Libra Partitions
$ java -cp my.jar
9p
$ for ((i=0;i<44;i++))
do
java -cp my.jar &
done
PowerPC Blades: Libra Workers

51. Libra
X86 Linux Front Ends
Pool of Libra Partitions
$ java -cp my.jar
9p
$ for ((i=0;i<44;i++))
do
java -cp my.jar &
done
PowerPC Blades: Libra Workers

52. Libra
X86 Linux Front Ends
Pool of Libra Partitions
$ java -cp my.jar
9p
$ for ((i=0;i<44;i++))
do
java -cp my.jar &
done
PowerPC Blades: Libra Workers

53. What did we learn?
• Specialized environment for each
application
• Lightweight system layer implementing
services for performance
• General purpose OS for non-performance
critical services

54. SEE : A LibOS for SESA
• Distributed LibOS that
can elastically span
Per-node EBB manifestations
nodes
Scalable Elastic Executive (SEE)
• Instances cooperate to EBB Infrastructure
FS Name Space
support the allocation Protocol (9p)
and deallocation of locality aware
inter-node
event communication
EBB’s memory
dispatcher protocols and
allocator
primitives
• Enables compatibility
with Front End nodes SEHAL
running via unified 9p
namespace

55. SEMachines and EPICs
SEMachine Hardware Abstraction Layer :
SEHAL SEHAL SEHAL EPIC

56. Programmable
Interrupt Controller
Source
Interrupt
0 1 1 0
Execution

57. Programmable
Interrupt Controller
Source
Interrupt
1 1 1 0
Execution

58. Programmable
Interrupt Controller
Source
Event
1 1 1 0
Action

59. Elastic Programmable
Interrupt Controller
Source
Event
1 0 1 1 1 0 0 0
Action

60. Elastic Programmable
Interrupt Controller
Source
Event
...
1 0 1 1 0 1 1 1
...
Action
...

61. Elastic Programmable
Interrupt Controller
• Programmed by the SEE
• Provides the minimum requirement of
elastic applications - mapping load to
resources
• Portable layer
• Take advantage of network features such as
broadcast and multicast

62. OUTLINE
1. THE PROBLEM
2. OBSERVATIONS
3. OUR TAKE ON A SOLUTION
4. PROTOTYPE & CHALLENGES

63. PROTOTYPE APP
SESA SAGE Traditional HW
Sage* SERVICES
Elastic
SEE: EBB’s + EHAL
OL Matrix
Cache
Advanced HW
Elastic
Matrix
Kittyhawk
Ops

64. Challenges and
Discussion

65. OUTLINE
1. THE PROBLEM
1. Pay as you go computing
2. Insufficient systems support for elasticity
2. OBSERVATIONS
3. OUR TAKE ON A SOLUTION
4. PROTOTYPE & CHALLENGES

66. Pay as you go hardware
Software
Consumer
Provider

67. Pay as you go hardware
Software
Consumer
Request
Provider

68. Pay as you go hardware
Software
Consumer
Provider

69. Elastic Website
Load Balancer
Consumer
Provider

70. Elastic Website
Load Balancer
Consumer
Provider

71. Elastic Website
Load Balancer
Consumer
Provider

72. Other Elastic
Applications
• Analytics
• Batch computation
• Stream processing

73. What’s the problem?
• Allocation/Boot-time
• Programmability

74. Medical Imaging
Application
• Megapixel image
• Quadratic algorithm
• (1 mil pixels * 4 bytes/pixel)^2 ~ 14 TB
• On Amazon EC2 ~ $8000 per day

75. Snowflock
Consumer
Provider

76. Snowflock
Consumer
Provider

77. Snowflock
Consumer
Provider

78. Distributing an Object
Non-Distributed Object Instance
Region List Lock L
Region List R0
Other
R1
Data
Structures
R2

79. L
R0
Root Rep0
L
R0
L
R1
R1 Rep1
L
R2 R0
R2 Rep2

80. Elastic Programmable
Interrupt Controller
Source
Event
1 1 1 0
Action