This document discusses the key challenges in building the PlanetLab platform for distributed systems research. It describes PlanetLab's novel management architecture and usage model, and how its design addressed requirements like supporting long-term experiments, immediate availability, and autonomy while isolating slices and coping with limited resources. The challenges of balancing centralized control with trust, allocating resources fairly between slices, and maintaining stability as the system evolves are also summarized.

1. Experiences Building PlanetLab
Iuliia Proskurnia
EMDC
April 23, 2012

2. Introduction
Background
Slice Management
Resource allocation
Stability
Conclusions
Introduction
Background
Slice Management
Requirements Coverage
Trust Assumption
VM and Resource Pools
Delegation
Management Services
Resource allocation
Requirements Coverage
Workload
Stability
Requirements Coverage
Node Availability
Conclusions
Iuliia Proskurnia EMDC Experiences Building PlanetLab 1

3. Introduction
Background
Slice Management
Resource allocation
Stability
Conclusions
Introduction
Main dimension of the innovation:
I Novel Management Architecture
I Novel Usage Model
I Experience-driven Desigh
I Conflict-driven Design
Iuliia Proskurnia EMDC Experiences Building PlanetLab 2

4. Introduction
Background
Slice Management
Resource allocation
Stability
Conclusions
Introduction
Background
Slice Management
Requirements Coverage
Trust Assumption
VM and Resource Pools
Delegation
Management Services
Resource allocation
Requirements Coverage
Workload
Stability
Requirements Coverage
Node Availability
Conclusions
Iuliia Proskurnia EMDC Experiences Building PlanetLab 3

5. Introduction
Background
Slice Management
Resource allocation
Stability
Conclusions
Requirements
(R1) It must provide a global platform that supports both
short-term experiments and long-running services.
(R2) It must be available immediately, even though no one knows
for sure what it is.
(R3) We must convince sites to host nodes running code written
by unknown researchers from other organizations.
(R4) Sustaining growth depends on support for autonomy and
decentralized control.
(R5) It must scale to support many users with minimal resources.
Iuliia Proskurnia EMDC Experiences Building PlanetLab 4

6. Introduction
Background
Slice Management
Resource allocation
Stability
Conclusions
Initial Design and Challenges
Design
I Distributed Virtualization
I Resource and Security
Isolation Related Systems
I PlanetLab Central(PLC) I ISP
I Hosting Center
Challenges I GRID
I Management Architecture
I Resource Allocation
I Stable system maintenance
Iuliia Proskurnia EMDC Experiences Building PlanetLab 5

7. Introduction
Requirements Coverage
Background
Trust Assumption
Slice Management
VM and Resource Pools
Resource allocation
Delegation
Stability
Management Services
Conclusions
Introduction
Background
Slice Management
Requirements Coverage
Trust Assumption
VM and Resource Pools
Delegation
Management Services
Resource allocation
Requirements Coverage
Workload
Stability
Requirements Coverage
Node Availability
Conclusions
Iuliia Proskurnia EMDC Experiences Building PlanetLab 6

8. Introduction
Requirements Coverage
Background
Trust Assumption
Slice Management
VM and Resource Pools
Resource allocation
Delegation
Stability
Management Services
Conclusions
Challenges
I Minimizing Centralized Component(R4) yet maintaining the
necessary trust assumption(R3);
I Balancing the need for slices to acquire the resources they
need(R1) yet coping with scarce resources(R5);
I Isolating slices from each other(R1) yet allowing some slices
to manage other slices(R2);
Iuliia Proskurnia EMDC Experiences Building PlanetLab 7

9. Introduction
Requirements Coverage
Background
Trust Assumption
Slice Management
VM and Resource Pools
Resource allocation
Delegation
Stability
Management Services
Conclusions
Trust Assumption
Iuliia Proskurnia EMDC Experiences Building PlanetLab 8

10. Introduction
Requirements Coverage
Background
Trust Assumption
Slice Management
VM and Resource Pools
Resource allocation
Delegation
Stability
Management Services
Conclusions
VM and Resource Pools
Require:
I Support long-lived slices(R1) and scarce of resources(R5)
I Decouple slice creation from resource allocation
Node Manager:
I VM - Point-of-presence on a node for a slice
I Resource Pool - Collection of physical and logical resources
possible bound to a VM
Iuliia Proskurnia EMDC Experiences Building PlanetLab 9

11. Introduction
Requirements Coverage
Background
Trust Assumption
Slice Management
VM and Resource Pools
Resource allocation
Delegation
Stability
Management Services
Conclusions
PLC
PLC Structure
I Owners assign all of its resources to PLC for redistribution
I PLC runs a slice creation service on each node (pl conf)
I PLC provide a front-end through which users create slices
I Direct and delegated slice creation
Iuliia Proskurnia EMDC Experiences Building PlanetLab 10

12. Introduction
Requirements Coverage
Background
Trust Assumption
Slice Management
VM and Resource Pools
Resource allocation
Delegation
Stability
Management Services
Conclusions
Management Services
I Slice Creation Service
Resource Pool Assigns to pl conf
I Brokerage Service
Resource Pool Control
I Monitoring Service
States of the local node
I Environment Service
Specify immutability, Installing necessary packages
I Auditing Service
Packet flow log
Iuliia Proskurnia EMDC Experiences Building PlanetLab 11

13. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Workload
Stability
Conclusions
Introduction
Background
Slice Management
Requirements Coverage
Trust Assumption
VM and Resource Pools
Delegation
Management Services
Resource allocation
Requirements Coverage
Workload
Stability
Requirements Coverage
Node Availability
Conclusions
Iuliia Proskurnia EMDC Experiences Building PlanetLab 12

14. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Workload
Stability
Conclusions
Conflicts
I Requirements for isolation
I Reality of limited resources
I Model of Weak Resource Isolation
Iuliia Proskurnia EMDC Experiences Building PlanetLab 13

15. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Workload
Stability
Conclusions
WorkLoad
Iuliia Proskurnia EMDC Experiences Building PlanetLab 14

16. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Workload
Stability
Conclusions
Workload
CPU
Iuliia Proskurnia EMDC Experiences Building PlanetLab 15

17. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Workload
Stability
Conclusions
Workload
Memory
Iuliia Proskurnia EMDC Experiences Building PlanetLab 16

18. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Workload
Stability
Conclusions
Workload
Bandwidth
Pay?
I Total Tra c generated during the month
I Incoming bandwidth
Tra c
I Transmission - 1Mb/s
I Receive - 500Kb/s
Iuliia Proskurnia EMDC Experiences Building PlanetLab 17

19. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Workload
Stability
Conclusions
Workload
Disk
Iuliia Proskurnia EMDC Experiences Building PlanetLab 18

20. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Node Availability
Stability
Conclusions
Introduction
Background
Slice Management
Requirements Coverage
Trust Assumption
VM and Resource Pools
Delegation
Management Services
Resource allocation
Requirements Coverage
Workload
Stability
Requirements Coverage
Node Availability
Conclusions
Iuliia Proskurnia EMDC Experiences Building PlanetLab 19

21. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Node Availability
Stability
Conclusions
Conflicts
Conflicts:
I System on behalf of the user community(R1)
I Evolve the platform to provide long-term sustainability(R2)
Strategies:
I Relatively Stable Environment for Users
I New Releases implemented incrementally
Iuliia Proskurnia EMDC Experiences Building PlanetLab 20

22. Introduction
Background
Slice Management Requirements Coverage
Resource allocation Node Availability
Stability
Conclusions
Node Availability
Iuliia Proskurnia EMDC Experiences Building PlanetLab 21

23. Introduction
Background
Slice Management
Resource allocation
Stability
Conclusions
Introduction
Background
Slice Management
Requirements Coverage
Trust Assumption
VM and Resource Pools
Delegation
Management Services
Resource allocation
Requirements Coverage
Workload
Stability
Requirements Coverage
Node Availability
Conclusions
Iuliia Proskurnia EMDC Experiences Building PlanetLab 22

24. Introduction
Background
Slice Management
Resource allocation
Stability
Conclusions
Conclusions
Conclusions:
I Carefully selected ideas Synthesis
I Real-World usage Finalization
I Large-Scale DS
I Coverage of Conflicting Requirements
Iuliia Proskurnia EMDC Experiences Building PlanetLab 23

25. Experiences Building PlanetLab
Iuliia Proksurnia
EMDC
April 23, 2012