Cake Cutting of CPU Resources among multiple HPC agents on a Cloud

1,557 views

Published on

Slides used by Kausal in his presentation in the third International Conference on Recent Trends in Information Technology (ICRTIT 2013) in MIT, Chennai on 25th July.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,557
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Cake Cutting of CPU Resources among multiple HPC agents on a Cloud

  1. 1. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Cake Cutting of CPU Resources among multiple HPC agents on a Cloud Kausal Malladi, Debargha Ganguly International Institute of Information Technology - Bangalore July 25, 2013 Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 1/26
  2. 2. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Agenda 1 Introduction 2 Existing Algorithms 3 Proposed Algorithm 4 Performance 5 Conclusion 6 References Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 2/26
  3. 3. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Quick Introduction Cloud Computing Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction Resource A resource is any physical or virtual component of limited availability within a computer system Shared resource A shared resource is a piece of information on a computer that can be remotely accessed from another computer, typically via a LAN or an enterprise Intranet Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 3/26
  4. 4. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Quick Introduction Cake Cutting Fair division is the problem of dividing a set of goods between several people, such that each person receives his/her due share. This problem arises in various real-world settings: auctions, divorce settlements, electronic spectrum and frequency allocation, airport traffic management, or exploitation of Earth Observation Satellites High Performance Agents A supercomputer is a computer at the frontline of contemporary processing capacity, particularly speed of calculation Game Theory Game theory is a study of strategic decision making. More formally, it is the study of mathematical models of conflict and cooperation between intelligent rational decision makers Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 4/26
  5. 5. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Quick Introduction Problem Statement Cake Cutting of resources (CPU) among several dynamically adding HPC agents on a Cloud, ensuring that the resources are utilized to the utmost. Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 5/26
  6. 6. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Existing Algorithms Dominant Resource Fairness (DRF) Algorithm, proposed by Ghodsi et. al. Dynamic Dominant Resource Fairness (Dynamic DRF) Algorithm, proposed by Kash et. al. Max-min Algorithm, proposed by Kumar et. al. Assumptions Agents once added don’t leave the system Agents demand resources in fixed proportions Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 6/26
  7. 7. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Dominant Resource Fairness (DRF) Algorithm Sharing Incentive Each user should be better off sharing the cluster, than exclusively using her own partition of the cluster. Consider a cluster with identical nodes and n users. Then a user should not be able to allocate more tasks in a cluster partition consisting of 1/n of all resources Strategy-proofness Users should not be able to benefit by lying about their resource demands. This provides incentive compatibility, as a user cannot improve her allocation by lying Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 7/26
  8. 8. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Dominant Resource Fairness (DRF) Algorithm Envy-freeness A user should not prefer the allocation of another user. This property embodies the notion of fairness Pareto efficiency It should not be possible to increase the allocation of a user without decreasing the allocation of at least another user. This property is important as it leads to maximizing system utilization subject to satisfying the other properties Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 8/26
  9. 9. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Dominant Resource Fairness (DRF) Algorithm Static resource allocation Meets all four properties of a fair allocation policy For every user, computes the share of each resource allocated Maximum among all shares is the dominant share Corresponding resource is called dominant resource Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 9/26
  10. 10. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Dynamic DRF Algorithm Strategyproofness No agent can misreport its demand vector and be strictly better off at any step k, regardless of the reported demands of other agents Dynamic Pareto Optimality At each step the allocation should not be Pareto dominated by any other allocation that only redistributes the collective entitlements of the agents present in the system among those agents Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 10/26
  11. 11. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Dynamic DRF Algorithm Sharing Incentive When an agent arrives it receives an allocation that it likes at least as much as an equal split of the resources. This models a setting where agents have made equal contributions to the system and hence have equal entitlements Dynamic Envy Freeness At any step an agent i envies an agent j only if j arrived before i did and j has not been allocated any resources since i arrived Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 11/26
  12. 12. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Dynamic DRF Algorithm Works for a dynamic setting Satisfies all the four properties and can be implemented in polynomial time HPC agents? Single resource type? Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 12/26
  13. 13. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Max-min Algorithm Maximum of minimum amount of resources that can be allocated to a host Assigns weights to agents Each agent receives allocation proportional to its weight Static setting, proved to satisfy four properties of DRF Algorithm Dynamic setting? Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 13/26
  14. 14. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Features Inspired from Dynamic DRF Algorithm Linear Program (LP) formulation HPC agents Dynamic setting Assumptions Only a certain number of agents can be run on a host Agents follow Game-Theoretic approach in demanding Agents once added won’t leave host Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 14/26
  15. 15. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References The Algorithm Algorithm 1: Modified Dynamic DRF Planner 1 agentCount ← 1; 2 foreach new agent added do 3 if agentCount ≤ maxAgentsthen 4 d[agentCount] ← demand by new agent; 5 proportion ← solve LP(agentCount,d); 6 for i=1 toagentCount do 7 allocation[i] ← proportion[i]∗d[i]/agentCount; 8 i←i+1; 9 end 10 end 11 agentCount ← agentCount+1; 12 end Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 15/26
  16. 16. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References The Algorithm...continued Algorithm 2: solve LP(agentCount,d) Data: N ← total resource units available on host machine Result: proportion [i] 1 Maximize S such that 2 proportion[i][k]≥ S 3 proportion[i][k]≥ proportion[i − 1][k] 4 N k=1 proportion[i][k]*d[k] ≤ i/N Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 16/26
  17. 17. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Results Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 17/26
  18. 18. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Results Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 18/26
  19. 19. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Results Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 19/26
  20. 20. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Results Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 20/26
  21. 21. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Results - Summary Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 21/26
  22. 22. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Winding it up... Proposed Algorithm works in the dynamic setting Works well for Computationally intensive HPC agents Performs better than traditionally implemented algorithms Assumptions are realistic and do not lead to loss of generality Resources utilized to the utmost! Future Work Performance optimization Less number of assumptions Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 22/26
  23. 23. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Acknowledgement Prof. Dr. Shrisha Rao, IIITB Continuous help in terms of showcasing results effectively and suggestions to make the proposed algorithm better Testers Anshul Sharma Pakalapati Srinivas Raju Pratibind Jha Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 23/26
  24. 24. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References References Peter Mell and Tim Grance, “The NIST Definition of Cloud Computing,” 2009 A. D. Procaccia, “Cake cutting: Not just child’s play,” in Communications of the ACM, 2013 C. Vecchiola, S. Pandey and R. Buyya, “High Performance Cloud Computing: A view of Scientific Applications,”in Proceedings of the 2009 10th International Symposium on pervasive Systems, Algorithms and Networks, ISPAN ’09, (Washington, USA), pp 4-16, IEEE Computer Society, 2009 S. Tijs and T. Driessen, “Game Theory and cost allocation Problems,” tech. rep., 1986 Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 24/26
  25. 25. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References References...continued A. Ghodsi, M. Zaharia, B. Hindman, A. Konwinski, S. Shenker and I. Stoica, “Dominant Resource Fairness: Fair Allocation of Multiple Resource Types,” in Proceedings of the 8th USENIX conference on Networked Systems design and implementation, NSDI 11, (Berkeley, CA, USA), pp. 24-24, USENIX Association, 2011 I. Kash, A. D. Procaccia and N. Shah, “No Agent Left Behind: Dynamic Fair Division of Multiple Resources,” in Proceedings of the 11th International Conference on Autonomous Agents and Multi-Agent Systems, AA-MAS, 2013 P. Kumar, A. Verma, “Independent Task Scheduling in Cloud Computing by Improved Genetic Algorithm,” in International Journal of Advanced Research in Computer Science and Software Engineering, vol. 2, issue 5, pp. 111-114, May 2012 Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 25/26
  26. 26. Agenda Introduction Existing Algorithms Proposed Algorithm Performance Conclusion References Thank you! {kausalmalladi, debargha.ganguly}@gmail.com http://www.kausalmalladi.tk Powered by LATEX Kausal Malladi, Debargha Ganguly Paper ID: 158 ICRTIT 2013 26/26

×