eUDON (Elastic Utility Driven Overlay Network) is a middleware for dynamically scaling the number of instances of a service to ensure a target QoS objective in highly dynamic large-scale infrastructures of non-dedicated servers.

1. Utility Driven Elastic Services
Pablo Chacin, Leandro Navarro
pchacin@ac.upc.edu
Polytechnic University of Catalonia
Computer Architecture Department
Computer Networks and Distributed Systems Group
Barcelona, Spain
DAIS Conference June 6, 2011
() June 6, 2011 1 / 21

2. Agenda
1 Motivation
2 eUDON
3 Experiments
4 Conclusions
() June 6, 2011 2 / 21

3. Motivation
Increased management
complexity
• Emergence of SOA as paradigm
for distributed systems
• Unpredictability of usage
patterns
• Need to adapt to unexpected
situations: failures, flash crowds
• Adoption of large scale
non-dedicated infrastructures Source: Schroth et al. 2007
System developers cannot anticipate management needs at design or even
deployment time.
Handling unexpected situations may require changing algorithms, parameters,
structure.
() June 6, 2011 3 / 21

4. Requirements
A solution to this management problem should have some desirable
properties:
Adaptiveness Support varying workloads and infrastructure changes
App. Independence Offer a generic infrastructure for multiple services
Comprehensiveness Support a broad range of QoS needs
Efficiency Achieve good resource utilization
Endurance Degrade gracefully under overload
Flexibility Accommodate different resource management policies
Manageability Ease of maintain and operate
Non-intrusiveness Require a minimal infrastructure modifications
Reliability Assign requests despite the uncertainly
Resilience Handle continuous activation/deactivation & failures
Robustness Work with incomplete, stale or inconsistent information
Scalability Scale to a very large the number of service instances
() June 6, 2011 4 / 21

5. Self-Adaptation
Self-adaptive systems
Self-adaptations has emerged as an alternative to direct engineering and operation
of system management.
Characteristics
• Aware: of its own state and the environment
• Self-adjusting: capable of changing its behavior, parameters, etc, to cope
with changes in its internal state or the environment
• Automatic: do not need intervention of humans to adapt.
() June 6, 2011 5 / 21

6. Problem Statement
Limitation of existing self-management approaches
• Scale of the system
• Platform/workload not fully under the control of the management component
• Lack of an accurate and up to date global view
• Handle delays and failures during adaptation actions
• Cope with multiple management policies
Objective
”Managing complexity is a key goal of self-adaptive software. If a program must
match the complexity of the environment in its own structure it will be very
complex indeed! Somehow we need to be able to write software that is less
complex than the environment in which it is operating yet operate robustly.”
Laddaga (2000)
() June 6, 2011 6 / 21

7. elastic Utility Driven Overlay Network
eUDON
A middleware for dynamically adapting services deployed on large-scale
infrastructures of non-dedicated servers
Scope
• Membership Management
• Request Routing
• Load Balancing
• Admission Control
• (Limited) Service Placement
• Resource Discovery
Salient Features
• Does not require a performance
model
• Do not require Performance
Isolation
• Implemented by each
service/service class
Limitations
• Service placement over a predefined set of instances
• Monitoring considered, but not currently implemented
() June 6, 2011 7 / 21

8. eUDON Model
() June 6, 2011 8 / 21

9. Overlay Construction and Request Routing
Selector Ranking Routing
Random N/A Round Robin
Age N/A Round Robin
Capacity Greedy
Two Choices
Probabilistic
Routing Overlay
Selector Ranking Routing
Random N/A Random Walk
Age Utility Greedy
Gradient
Search Overlay
() June 6, 2011 9 / 21

10. Admission Control
0
0.2
0.4
0.6
0.8
1
RT0
Utility
Response Time
α = 0.3
α = 0.5
() June 6, 2011 10 / 21

11. Admission Control
0.0
0.2
0.4
0.6
0.8
1.0
Utilization
Total utilization
0.0
0.2
0.4
0.6
0.8
1.0
Utilization
Total utilization
Background load
0
5
10
15
20
Capacity
1.0
UtilityRatio
() June 6, 2011 11 / 21

12. Promotion and Demotion
Heuristic
• Probabilistic adaptation strategy
• Based on current arrival rate
• Needs an estimated of global distribution of arrival rates
• Promote if above 50%
• Demote if below 25%
• Single parameter controls how aggressively adapt
• Don’t require any coordination
0
0.2
0.4
0.6
0.8
1
0 20 40 60 80 100
Arrival rate
k=-3
k=-5
0
0.2
0.4
0.6
0.8
1
0 20 40 60 80 100
Probability
Arrival rate
k=3
k=5
() June 6, 2011 12 / 21

13. Experimental Model
Simulation Model
• Discrete event simulator
• Idealized network that mimics a large cluster
• Each service instance as a M/G/1/k ∗ PS queuing system
• Background load simulated as a random walk
Parameters
• Nodes: 128 . . . 2048
• Exchange set: 1,2,. . . 8
• Neighbor set: 16,32,48
• Update frequency: 1,2,3
• Background load variability
• . . .
Metrics
• Allocated Demand
• Target/Offered QoS Ratio
• Utilization
• Hops
Compared with
Theoretical maximum.
() June 6, 2011 13 / 21

14. Base Scenario
60
70
80
90
100
110
120
130
0 100 200
N´”Nodes
(a) Evolution of the routing overlay size over
time.
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 50 100 150 200
Utilization
1.0
0 50 100 150 200
UtilityRatio
Time (seconds)
(b) Utilization and QoS Ratio.
() June 6, 2011 14 / 21

15. Alternative Load Balancing Heuristics
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Pc 2C RR RR-R
AllocatedDemand
(c) Allocated Demand.
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
1 2 3 4 5 6 7 8 9 10
%Requests
Hops
Pc
2C
RR
RR_R
(d) Distribution of routing hops.
() June 6, 2011 15 / 21

16. Alternative Search Heuristics
0.45
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
1.00
UDON Gradient Random Walk
AllocatedDemand
(e) Allocated Demand
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
1 2 3 4 5 6 7 8 9 10
%Requests
Hops
UDON
Gradient
Random
(f) Routing Hops
() June 6, 2011 16 / 21

17. Peak Load Scenario
0
1000
2000
3000
4000
5000
6000
7000
0 50 100 150 200 250 300
Requests
0.0
20.0
40.0
60.0
80.0
100.0
120.0
0 50 100 150 200 250 300
N´”Nodes
Time (seconds)
(g) Injected load and number of instances
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 50 100 150 200 250 300
Utilization
1.0
0 50 100 150 200 250 300
UtilityRatio
Time (seconds)
(h) Utilization and Utility Ratio.
() June 6, 2011 17 / 21

18. Failure Scenario
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0 50 100 150 200
Utilization
1.0
0 50 100 150 200
UtilityRatio
Time (seconds)
(i) Aggregate utilization and utility ratio.
30.0
40.0
50.0
60.0
70.0
80.0
0 50 100 150 200
Nodes
0.0
1.0
2.0
3.0
4.0
5.0
0 50 100 150 200
Hops
Time (seconds)
(j) Number of instances and Number of
Hops.
() June 6, 2011 18 / 21

19. Conclusions
We addressed the problem of self-adaptation in large scale distributed services.
eUDON exhibits the intended properties.
• Simple yet powerful model
• Non-intrusive
• Easily extensible, adaptable.
• Unifies multiple cases (failures, peak load)
• Scalable to 1000’s of nodes,
• Efficient (95% utilization, 90% allocated demand
Amenable to be included as part of the standard stack of service providers.
We believe this work represents a significant contribution towards the development
of future generation service oriented applications by providing a self-management
solution specifically addressed to this increasingly important category of systems.
() June 6, 2011 19 / 21

20. Future Work
Extend the model to support service composition following the model proposed by
Alrifai et al. (2008) decomposing the utility function into a series of utility
functions which can be evaluated independently for each basic service.
Implement the activation/deactivation mechanism using the same theoretical
approach used to model the market entry decision problem.
Apply the framework to other problems. In particular, the many tasks problem,
like parameter swap and Map Reduce.
() June 6, 2011 20 / 21

21. Thank you ... any questions?
Pablo Chacin
pchacin@ac.upc.edu
http://personals.ac.upc.edu/pchacin
Polytechnic University of Catalonia
Computer Architecture Department
Computer Networks and Distributed Systems Group
Barcelona, Spain
() June 6, 2011 21 / 21