The document discusses optimizing throughput in hierarchical middleware like DIET. It presents DIET and describes its hierarchical architecture. The goal is to determine the optimal deployment of agents and servers across platforms to maximize throughput. It develops models for estimating service times at different levels. It then describes approaches for automatic deployment, including using mixed integer programming or genetic algorithms to evolve populations of solutions.

1. Multiple Services Throughput Optimization in a
Hierarchical Middleware
Eddy Caron1 , Benjamin Depardon2 , Frédéric Desprez1
1 University of Lyon. LIP Laboratory.
UMR CNRS - ENS Lyon INRIA - UCBL 5668.
FRANCE
2 SysFera
CCGrid 2011
May 24th , 2011

2. D IET Problem Model Planning Experiments Conclusion
Introduction
• Solve large problems
• Grids: distributed and large scale environments
• GridRPC approach:
• Clients submit requests to a meta-scheduler (an agent)
• Agent schedules requests and find suitable servers
• Examples: Ninf-G, NetSolve, GridSolve, WebCom-G, D IET
• Several middleware architectures:
• Star graph
• Hierarchy
• Performance depend on the middleware deployment
⇒ Needs for performance modelization and algorithms to choose
the “best” hierarchy
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 2/20

3. D IET Problem Model Planning Experiments Conclusion
Introduction
• Solve large problems
• Grids: distributed and large scale environments
• GridRPC approach:
• Clients submit requests to a meta-scheduler (an agent)
• Agent schedules requests and find suitable servers
• Examples: Ninf-G, NetSolve, GridSolve, WebCom-G, D IET
• Several middleware architectures:
• Star graph
• Hierarchy
• Performance depend on the middleware deployment
⇒ Needs for performance modelization and algorithms to choose
the “best” hierarchy
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 2/20

4. D IET Problem Model Planning Experiments Conclusion
Outline
1 The D IET middleware
2 Problem
3 Hierarchy model
4 Deployment planning
5 Experiments
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 3/20

5. D IET Problem Model Planning Experiments Conclusion
D IET
Distributed Interactive Engineering Toolbox
http://graal.ens-lyon.fr/DIET
• Toolbox for Application Service Provider (ASP)
• Hierarchical architecture: scalability & performance
• GridRPC compliant
• Testbed for theoretical results
• scheduling for heterogeneous platforms
• data (re)distribution and replication
• performance evaluation
• algorithmic for heterogeneous and distributed platforms
• ...
• Used in production for the Decrypthon project
• Now supported by a startup: SysFera
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 4/20

6. D IET Problem Model Planning Experiments Conclusion
Request submission
Client
AGENTS
Servers
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

7. D IET Problem Model Planning Experiments Conclusion
Request submission
Client FindServer()
MA
LA1 LA2
SeD1 SeD2 SeD3 SeD4
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

8. D IET Problem Model Planning Experiments Conclusion
Request submission
Client
MA
FindServer()
LA1 LA2
SeD1 SeD2 SeD3 SeD4
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

9. D IET Problem Model Planning Experiments Conclusion
Request submission
Client
MA
LA1 LA2
FindServer()
SeD1 SeD2 SeD3 SeD4
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

10. D IET Problem Model Planning Experiments Conclusion
Request submission
Client
MA
LA1 LA2
Estimate() SeD1 SeD2 SeD3 SeD4
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

11. D IET Problem Model Planning Experiments Conclusion
Request submission
Client
MA
Aggregate() LA1 LA2
SeD1 SeD2 SeD3 SeD4
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

12. D IET Problem Model Planning Experiments Conclusion
Request submission
Client
Aggregate() MA
LA1 LA2
SeD1 SeD2 SeD3 SeD4
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

13. D IET Problem Model Planning Experiments Conclusion
Request submission
BestServer = SeD1
Client
MA
LA1 LA2
SeD1 SeD2 SeD3 SeD4
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

14. D IET Problem Model Planning Experiments Conclusion
Request submission
Client
MA
RunService()
LA1 LA2
SeD1 SeD2 SeD3 SeD4
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

15. D IET Problem Model Planning Experiments Conclusion
Goal
MA LA1 LA1 ...
Underlying questions
• How many agents?
• How many servers for each type of service?
• What is the shape of the hierarchy?
Objective
Given a platform G = (V , E), and a set of services R, what is the
best attainable throughput, i.e., the number of finished requests
per time unit, in a D IET hierarchy?
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20

16. D IET Problem Model Planning Experiments Conclusion
Goal
SeD1 SeD1 SeD2 SeD2 ...
Underlying questions
• How many agents?
• How many servers for each type of service?
• What is the shape of the hierarchy?
Objective
Given a platform G = (V , E), and a set of services R, what is the
best attainable throughput, i.e., the number of finished requests
per time unit, in a D IET hierarchy?
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20

17. D IET Problem Model Planning Experiments Conclusion
Goal
LA LA
MA
MA LA LA
MA LA
LA SeD2
LA LA LA
SeD1 SeD1 LA LA
LA
SeD1 SeD1 SeD2 SeD2
SeD2
SeD1
Underlying questions
• How many agents?
• How many servers for each type of service?
• What is the shape of the hierarchy?
Objective
Given a platform G = (V , E), and a set of services R, what is the
best attainable throughput, i.e., the number of finished requests
per time unit, in a D IET hierarchy?
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20

18. D IET Problem Model Planning Experiments Conclusion
Goal
LA LA
MA
MA LA LA
MA LA
LA SeD2
LA LA LA
SeD1 SeD1 LA LA
LA
SeD1 SeD1 SeD2 SeD2
SeD2
SeD1
Underlying questions
• How many agents?
• How many servers for each type of service?
• What is the shape of the hierarchy?
Objective
Given a platform G = (V , E), and a set of services R, what is the
best attainable throughput, i.e., the number of finished requests
per time unit, in a D IET hierarchy?
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20

19. D IET Problem Model Planning Experiments Conclusion
More formally. . .
Objective function
Given:
• a platform G = (V , E, W , B),
• a set of services R,
• for each service i ∈ R an objective throughput ρ∗ ,
i
find a hierarchy such that:
ρi ρi
• ∀i, i ∈ R, ρ∗ ρ∗ in steady state,
i i
ρi
• mini∈R ρ∗ is maximized,
i
• and the hierarchy has as few agents as possible.
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 7/20

20. D IET Problem Model Planning Experiments Conclusion
Platform
w1,B1 w2,B2
w6,B6 w3,B3
Bc,c'
w5,B5 w4,B4
• Fully connected platform
• wj : computing power
• Bj,j : bandwidth between any two nodes
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 8/20

21. D IET Problem Model Planning Experiments Conclusion
Servers model
Average computation time
Si being the set of nodes allocated for
servers of type i
server wappi + |Si | .wprei
Tcompi = Server
j∈Si wj mreq mresp
i i
Communications wpre
mreqi i
server
Trecvi j = wapp
Bj,f j i
i
serverj mrespi
Tsendi =
Bj,f j
i
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 9/20

22. D IET Problem Model Planning Experiments Conclusion
Agent model
• Chldij :
Computation children of Aj knowing service
i
j
δi .wreqi + wrespi
j
Chldi
• δij :
agent boolean, does Aj know service
Tcomp j = ρservi .
wj i?
i∈R
Communications Agent
mreq mresp
  i i
j
agentj  δ .mreqi mrespi 
Trecv = ρservi .  i + wreq
Bj,f j Bj,k

i∈R j i
k ∈Chldi
wresp
  i
j
agent  δ .mrespi mreqi  ...
Tsend j = ρservi .  i +
Bj,f j Bj,k

i∈R k ∈Chldi
j
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 10/20

23. D IET Problem Model Planning Experiments Conclusion
Automatic deployment: homogeneous Bj,j
SeD1 SeD1 SeD1 SeD2 SeD2 SeD2 SeD1 SeD2
Bottom-up approach
1. Distribute N nodes to the servers
ρserv ρservi
⇒ ∀i, i ∈ R, ρ∗ i ρ∗ i i
2. With the remaining nodes, try to build an agent hierarchy
• use a mixed integer linear program
• build level by level, until 1 agent is enough
• if not possible, reduce N and goto 1
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20

24. D IET Problem Model Planning Experiments Conclusion
Automatic deployment: homogeneous Bj,j
LA LA LA
SeD1 SeD1 SeD1 SeD2 SeD2 SeD2 SeD1 SeD2
Bottom-up approach
1. Distribute N nodes to the servers
ρserv ρservi
⇒ ∀i, i ∈ R, ρ∗ i ρ∗ i i
2. With the remaining nodes, try to build an agent hierarchy
• use a mixed integer linear program
• build level by level, until 1 agent is enough
• if not possible, reduce N and goto 1
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20

25. D IET Problem Model Planning Experiments Conclusion
Automatic deployment: homogeneous Bj,j
MA
LA LA LA
SeD1 SeD1 SeD1 SeD2 SeD2 SeD2 SeD1 SeD2
Bottom-up approach
1. Distribute N nodes to the servers
ρserv ρservi
⇒ ∀i, i ∈ R, ρ∗ i ρ∗ i i
2. With the remaining nodes, try to build an agent hierarchy
• use a mixed integer linear program
• build level by level, until 1 agent is enough
• if not possible, reduce N and goto 1
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20

26. D IET Problem Model Planning Experiments Conclusion
Automatic deployment: heterogeneous Bj,j
Agent
wreqi
wrespi
...
Server
wprei wprei
wappi wappi
Genetic algorithm
1. Generate a random population of valid solutions
2. Evolve the population
• crossover
• mutation
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20

27. D IET Problem Model Planning Experiments Conclusion
Automatic deployment: heterogeneous Bj,j
1. Random initialization
2. Selection 3. Reproduction
4. Best individual
+
... ... =
...
Genetic algorithm
1. Generate a random population of valid solutions
2. Evolve the population
• crossover
• mutation
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20

28. D IET Problem Model Planning Experiments Conclusion
Automatic deployment: heterogeneous Bj,j
11 11
1 1 3 1 1 3
3 5 8 6 5 9 20 3 5 & 8 6 5 9 20
6 9 8 2 4 15 4 6 9 8 2 4 15 4
10 7 10 7
H1 H2 H'1 H'2
Genetic algorithm
1. Generate a random population of valid solutions
2. Evolve the population
• crossover
• mutation
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20

29. D IET Problem Model Planning Experiments Conclusion
Automatic deployment: heterogeneous Bj,j
Type mutation Parent mutation Pruning
Genetic algorithm
1. Generate a random population of valid solutions
2. Evolve the population
• crossover
• mutation
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20

30. D IET Problem Model Planning Experiments Conclusion
Experimental environment
Platform
Grid’5000
• Clusters:
• Lille: Chti (3784MFlops)
• Rennes: Paradent (4378MFlops)
• Lyon: Sagittaire (3249MFlops) and Capricorne
(2922MFlops)
• 1Gb and 10Gb networks
Services
• dgemm (Atlas library)
• Fibonacci number (naive algorithm)
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 13/20

31. D IET Problem Model Planning Experiments Conclusion
Homogeneous comm. / Heterogeneous comp.
min-first
dgemm 100 Fibonacci 30
2500 2500
Fibonacci throughput (requests / s)
dgemm throughput (requests / s)
2000 2000
1500 dgemm theoretical 1500
Fibonacci theoretical
dgemm experimental
Fibonacci experimental
1000 1000
500 500
0
5
0
5
-1
-1
-2
-2
2-2
3
5
10
15
20
25
1-
5-
Number of nodes (Sagittaire-Capricorne)
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 14/20

32. D IET Problem Model Planning Experiments Conclusion
Homogeneous comm. / Heterogeneous comp.
min-first
dgemm 100 Fibonacci 30
2200 2200
2000 2000
Fibonacci throughput (requests/s)
dgemm throughput (requests/s)
1800 1800
1600 1600
1400 1400
dgemm, min-first
1200 Fibonacci, min-first 1200
dgemm, star graph
Fibonacci, star graph
1000 1000
800 800
600 600
400 400
200 200
0
5
0
5
-1
-1
-2
-2
2-2
3
5
10
15
20
25
1-
5-
Number of nodes (Sagittaire-Capricorne)
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 14/20

33. D IET Problem Model Planning Experiments Conclusion
Homogeneous comm. / Heterogeneous comp.
max-first
dgemm 100 Fibonacci 30
2500 2500
Fibonacci throughput (requests / s)
dgemm throughput (requests / s)
2000 2000
1500 dgemm theoretical 1500
Fibonacci theoretical
dgemm experimental
Fibonacci experimental
1000 1000
500 500
0
5
0
5
-1
-1
-2
-2
2-2
3
5
10
15
20
25
1-
5-
Number of nodes (Sagittaire-Capricorne)
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 15/20

34. D IET Problem Model Planning Experiments Conclusion
Homogeneous comm. / Heterogeneous comp.
max-first
dgemm 100 Fibonacci 30
2200 2200
2000 2000
Fibonacci throughput (requests/s)
dgemm throughput (requests/s)
1800 1800
1600 1600
1400 1400
dgemm, max-first
1200 Fibonacci, max-first 1200
dgemm, star graph
Fibonacci, star graph
1000 1000
800 800
600 600
400 400
200 200
0
5
0
5
-1
-1
-2
-2
2-2
3
5
10
15
20
25
1-
5-
Number of nodes (Sagittaire-Capricorne)
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 15/20

35. D IET Problem Model Planning Experiments Conclusion
Example of produced hierarchy
min-first, 20-20 nodes
MA
LA LA
LA LA LA LA LA LA
D D D D D D D D D D F F F F F F F F F F F
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 16/20

36. D IET Problem Model Planning Experiments Conclusion
GA / min-first / max-first comparison
dgemm
2500
min-first
max-first
Genetic
2000 Genetic mean
Throughput (nb requests/s)
1500
1000
500
0
0
5
0
5
-1
-1
-2
-2
2
3
5
10
15
20
25
1-
2-
5-
Nodes (Sagittaire-Capricorne)
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 17/20

37. D IET Problem Model Planning Experiments Conclusion
GA
dgemm 100 Fibonacci 30
1800 1800
Fibonacci throughput (requests / s)
1600 1600
dgemm throughput (requests / s)
1400 1400
1200 1200
dgemm theoretical
1000 Fibonacci theoretical 1000
dgemm experimental
Fibonacci experimental
800 800
600 600
400 400
200 200
10
13
16
0-
4-
7-
2-1-1
1
3
6
-1
-1
-1
2-
4-
7-
10
13
17
1-
3-
7-
Number of nodes (Sagittaire-Paradent-Chti)
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 18/20

38. D IET Problem Model Planning Experiments Conclusion
Example of produced hierarchy
MA
LA LA F LA
F F LA D D LA F F F
F F D D D
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 19/20

39. D IET Problem Model Planning Experiments Conclusion
Conclusion and future work
Conclusion
• Computation and communication model for hierarchical
middleware
• Two kinds of heuristics:
• Bottom-up heuristic, based on linear programming
• Genetic algorithm
• Experiments closely follow the predictions
Future work
• Heuristics for arbitrary graphs
• Dynamically adapt the hierarchy
• Extend model to multiple hierarchies
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 20/20

40. D IET Problem Model Planning Experiments Conclusion
Conclusion and future work
Conclusion
• Computation and communication model for hierarchical
middleware
• Two kinds of heuristics:
• Bottom-up heuristic, based on linear programming
• Genetic algorithm
• Experiments closely follow the predictions
Future work
• Heuristics for arbitrary graphs
• Dynamically adapt the hierarchy
• Extend model to multiple hierarchies
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 20/20

41. D IET Problem Model Planning Experiments Conclusion
Thank you!
Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 20/20