Multiple Services Throughput Optimization in a Hierarchical Middleware

Multiple Services Throughput Optimization in a Hierarchical Middleware Eddy Caron1 , Benjamin Depardon2 , Frédéric Desprez1 1 University of Lyon. LIP Laboratory. UMR CNRS - ENS Lyon INRIA - UCBL 5668. FRANCE 2 SysFera CCGrid 2011 May 24th , 2011

D IET Problem Model Planning Experiments Conclusion Introduction • Solve large problems • Grids: distributed and large scale environments • GridRPC approach: • Clients submit requests to a meta-scheduler (an agent) • Agent schedules requests and ﬁnd suitable servers • Examples: Ninf-G, NetSolve, GridSolve, WebCom-G, D IET • Several middleware architectures: • Star graph • Hierarchy • Performance depend on the middleware deployment ⇒ Needs for performance modelization and algorithms to choose the “best” hierarchy Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 2/20

D IET Problem Model Planning Experiments Conclusion Outline 1 The D IET middleware 2 Problem 3 Hierarchy model 4 Deployment planning 5 Experiments Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 3/20

D IET Problem Model Planning Experiments Conclusion D IET Distributed Interactive Engineering Toolbox http://graal.ens-lyon.fr/DIET • Toolbox for Application Service Provider (ASP) • Hierarchical architecture: scalability & performance • GridRPC compliant • Testbed for theoretical results • scheduling for heterogeneous platforms • data (re)distribution and replication • performance evaluation • algorithmic for heterogeneous and distributed platforms • ... • Used in production for the Decrypthon project • Now supported by a startup: SysFera Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 4/20

D IET Problem Model Planning Experiments Conclusion Request submission Client AGENTS Servers Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Request submission Client FindServer() MA LA1 LA2 SeD1 SeD2 SeD3 SeD4 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Request submission Client MA FindServer() LA1 LA2 SeD1 SeD2 SeD3 SeD4 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Request submission Client MA LA1 LA2 FindServer() SeD1 SeD2 SeD3 SeD4 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Request submission Client MA LA1 LA2 Estimate() SeD1 SeD2 SeD3 SeD4 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Request submission Client MA Aggregate() LA1 LA2 SeD1 SeD2 SeD3 SeD4 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Request submission Client Aggregate() MA LA1 LA2 SeD1 SeD2 SeD3 SeD4 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Request submission BestServer = SeD1 Client MA LA1 LA2 SeD1 SeD2 SeD3 SeD4 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Request submission Client MA RunService() LA1 LA2 SeD1 SeD2 SeD3 SeD4 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 5/20

D IET Problem Model Planning Experiments Conclusion Goal MA LA1 LA1 ... Underlying questions • How many agents? • How many servers for each type of service? • What is the shape of the hierarchy? Objective Given a platform G = (V , E), and a set of services R, what is the best attainable throughput, i.e., the number of ﬁnished requests per time unit, in a D IET hierarchy? Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20

D IET Problem Model Planning Experiments Conclusion Goal SeD1 SeD1 SeD2 SeD2 ... Underlying questions • How many agents? • How many servers for each type of service? • What is the shape of the hierarchy? Objective Given a platform G = (V , E), and a set of services R, what is the best attainable throughput, i.e., the number of ﬁnished requests per time unit, in a D IET hierarchy? Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20

D IET Problem Model Planning Experiments Conclusion Goal LA LA MA MA LA LA MA LA LA SeD2 LA LA LA SeD1 SeD1 LA LA LA SeD1 SeD1 SeD2 SeD2 SeD2 SeD1 Underlying questions • How many agents? • How many servers for each type of service? • What is the shape of the hierarchy? Objective Given a platform G = (V , E), and a set of services R, what is the best attainable throughput, i.e., the number of ﬁnished requests per time unit, in a D IET hierarchy? Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 6/20

D IET Problem Model Planning Experiments Conclusion More formally. . . Objective function Given: • a platform G = (V , E, W , B), • a set of services R, • for each service i ∈ R an objective throughput ρ∗ , i ﬁnd a hierarchy such that: ρi ρi • ∀i, i ∈ R, ρ∗ ρ∗ in steady state, i i ρi • mini∈R ρ∗ is maximized, i • and the hierarchy has as few agents as possible. Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 7/20

D IET Problem Model Planning Experiments Conclusion Platform w1,B1 w2,B2 w6,B6 w3,B3 Bc,c w5,B5 w4,B4 • Fully connected platform • wj : computing power • Bj,j : bandwidth between any two nodes Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 8/20

D IET Problem Model Planning Experiments Conclusion Servers model Average computation time Si being the set of nodes allocated for servers of type i server wappi + |Si | .wprei Tcompi = Server j∈Si wj mreq mresp i i Communications wpre mreqi i server Trecvi j = wapp Bj,f j i i serverj mrespi Tsendi = Bj,f j i Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 9/20

D IET Problem Model Planning Experiments Conclusion Agent model • Chldij : Computation children of Aj knowing service i j δi .wreqi + wrespi j Chldi • δij : agent boolean, does Aj know service Tcomp j = ρservi . wj i? i∈R Communications Agent mreq mresp   i i j agentj  δ .mreqi mrespi  Trecv = ρservi .  i + wreq Bj,f j Bj,k  i∈R j i k ∈Chldi wresp   i j agent  δ .mrespi mreqi  ... Tsend j = ρservi .  i + Bj,f j Bj,k  i∈R k ∈Chldi j Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 10/20

D IET Problem Model Planning Experiments Conclusion Automatic deployment: homogeneous Bj,j SeD1 SeD1 SeD1 SeD2 SeD2 SeD2 SeD1 SeD2 Bottom-up approach 1. Distribute N nodes to the servers ρserv ρservi ⇒ ∀i, i ∈ R, ρ∗ i ρ∗ i i 2. With the remaining nodes, try to build an agent hierarchy • use a mixed integer linear program • build level by level, until 1 agent is enough • if not possible, reduce N and goto 1 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20

D IET Problem Model Planning Experiments Conclusion Automatic deployment: homogeneous Bj,j LA LA LA SeD1 SeD1 SeD1 SeD2 SeD2 SeD2 SeD1 SeD2 Bottom-up approach 1. Distribute N nodes to the servers ρserv ρservi ⇒ ∀i, i ∈ R, ρ∗ i ρ∗ i i 2. With the remaining nodes, try to build an agent hierarchy • use a mixed integer linear program • build level by level, until 1 agent is enough • if not possible, reduce N and goto 1 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20

D IET Problem Model Planning Experiments Conclusion Automatic deployment: homogeneous Bj,j MA LA LA LA SeD1 SeD1 SeD1 SeD2 SeD2 SeD2 SeD1 SeD2 Bottom-up approach 1. Distribute N nodes to the servers ρserv ρservi ⇒ ∀i, i ∈ R, ρ∗ i ρ∗ i i 2. With the remaining nodes, try to build an agent hierarchy • use a mixed integer linear program • build level by level, until 1 agent is enough • if not possible, reduce N and goto 1 Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 11/20

D IET Problem Model Planning Experiments Conclusion Automatic deployment: heterogeneous Bj,j Agent wreqi wrespi ... Server wprei wprei wappi wappi Genetic algorithm 1. Generate a random population of valid solutions 2. Evolve the population • crossover • mutation Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20

D IET Problem Model Planning Experiments Conclusion Automatic deployment: heterogeneous Bj,j 1. Random initialization 2. Selection 3. Reproduction 4. Best individual + ... ... = ... Genetic algorithm 1. Generate a random population of valid solutions 2. Evolve the population • crossover • mutation Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20

D IET Problem Model Planning Experiments Conclusion Automatic deployment: heterogeneous Bj,j 11 11 1 1 3 1 1 3 3 5 8 6 5 9 20 3 5 & 8 6 5 9 20 6 9 8 2 4 15 4 6 9 8 2 4 15 4 10 7 10 7 H1 H2 H1 H2 Genetic algorithm 1. Generate a random population of valid solutions 2. Evolve the population • crossover • mutation Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20

D IET Problem Model Planning Experiments Conclusion Automatic deployment: heterogeneous Bj,j Type mutation Parent mutation Pruning Genetic algorithm 1. Generate a random population of valid solutions 2. Evolve the population • crossover • mutation Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 12/20

D IET Problem Model Planning Experiments Conclusion Experimental environment Platform Grid’5000 • Clusters: • Lille: Chti (3784MFlops) • Rennes: Paradent (4378MFlops) • Lyon: Sagittaire (3249MFlops) and Capricorne (2922MFlops) • 1Gb and 10Gb networks Services • dgemm (Atlas library) • Fibonacci number (naive algorithm) Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 13/20

D IET Problem Model Planning Experiments Conclusion Homogeneous comm. / Heterogeneous comp. min-ﬁrst dgemm 100 Fibonacci 30 2500 2500 Fibonacci throughput (requests / s) dgemm throughput (requests / s) 2000 2000 1500 dgemm theoretical 1500 Fibonacci theoretical dgemm experimental Fibonacci experimental 1000 1000 500 500 0 5 0 5 -1 -1 -2 -2 2-2 3 5 10 15 20 25 1- 5- Number of nodes (Sagittaire-Capricorne) Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 14/20

D IET Problem Model Planning Experiments Conclusion Homogeneous comm. / Heterogeneous comp. min-ﬁrst dgemm 100 Fibonacci 30 2200 2200 2000 2000 Fibonacci throughput (requests/s) dgemm throughput (requests/s) 1800 1800 1600 1600 1400 1400 dgemm, min-first 1200 Fibonacci, min-first 1200 dgemm, star graph Fibonacci, star graph 1000 1000 800 800 600 600 400 400 200 200 0 5 0 5 -1 -1 -2 -2 2-2 3 5 10 15 20 25 1- 5- Number of nodes (Sagittaire-Capricorne) Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 14/20

D IET Problem Model Planning Experiments Conclusion Homogeneous comm. / Heterogeneous comp. max-ﬁrst dgemm 100 Fibonacci 30 2500 2500 Fibonacci throughput (requests / s) dgemm throughput (requests / s) 2000 2000 1500 dgemm theoretical 1500 Fibonacci theoretical dgemm experimental Fibonacci experimental 1000 1000 500 500 0 5 0 5 -1 -1 -2 -2 2-2 3 5 10 15 20 25 1- 5- Number of nodes (Sagittaire-Capricorne) Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 15/20

D IET Problem Model Planning Experiments Conclusion Homogeneous comm. / Heterogeneous comp. max-ﬁrst dgemm 100 Fibonacci 30 2200 2200 2000 2000 Fibonacci throughput (requests/s) dgemm throughput (requests/s) 1800 1800 1600 1600 1400 1400 dgemm, max-first 1200 Fibonacci, max-first 1200 dgemm, star graph Fibonacci, star graph 1000 1000 800 800 600 600 400 400 200 200 0 5 0 5 -1 -1 -2 -2 2-2 3 5 10 15 20 25 1- 5- Number of nodes (Sagittaire-Capricorne) Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 15/20

D IET Problem Model Planning Experiments Conclusion Example of produced hierarchy min-ﬁrst, 20-20 nodes MA LA LA LA LA LA LA LA LA D D D D D D D D D D F F F F F F F F F F F Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 16/20

D IET Problem Model Planning Experiments Conclusion GA / min-ﬁrst / max-ﬁrst comparison dgemm 2500 min-first max-first Genetic 2000 Genetic mean Throughput (nb requests/s) 1500 1000 500 0 0 5 0 5 -1 -1 -2 -2 2 3 5 10 15 20 25 1- 2- 5- Nodes (Sagittaire-Capricorne) Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 17/20

D IET Problem Model Planning Experiments Conclusion GA dgemm 100 Fibonacci 30 1800 1800 Fibonacci throughput (requests / s) 1600 1600 dgemm throughput (requests / s) 1400 1400 1200 1200 dgemm theoretical 1000 Fibonacci theoretical 1000 dgemm experimental Fibonacci experimental 800 800 600 600 400 400 200 200 10 13 16 0- 4- 7- 2-1-1 1 3 6 -1 -1 -1 2- 4- 7- 10 13 17 1- 3- 7- Number of nodes (Sagittaire-Paradent-Chti) Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 18/20

D IET Problem Model Planning Experiments Conclusion Example of produced hierarchy MA LA LA F LA F F LA D D LA F F F F F D D D Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 19/20

D IET Problem Model Planning Experiments Conclusion Conclusion and future work Conclusion • Computation and communication model for hierarchical middleware • Two kinds of heuristics: • Bottom-up heuristic, based on linear programming • Genetic algorithm • Experiments closely follow the predictions Future work • Heuristics for arbitrary graphs • Dynamically adapt the hierarchy • Extend model to multiple hierarchies Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 20/20

D IET Problem Model Planning Experiments Conclusion Thank you! Frédéric Desprez Multiple Services Throughput Optimization in a Hierarchical Middleware 20/20

Multiple Services Throughput Optimization in a Hierarchical Middleware

by Frederic Desprez on Jun 11, 2011

Accessibility

Upload Details

Usage Rights

1 Embed 2

Statistics

Multiple Services Throughput Optimization in a Hierarchical Middleware — Presentation Transcript