Safety-Critical Systems

587 views
492 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
587
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Safety-Critical Systems

  1. 1. Power and Performance Management of Virtualized Computing Environments via Lookahead Control Dara Kusic 1 , Jeffrey O. Kephart 2 , James E. Hanson 2 , Nagarajan Kandasamy 1 , and Guofei Jiang 3 1- Drexel University, Philadelphia, PA 19104 2- IBM T.J. Watson Research Center, Hawthorne, NY 10532 3- NEC Labs America, Princeton, NJ 08540 Presented by Tongping Liu
  2. 2. OUTLINE <ul><li>Motivation and problem statement </li></ul><ul><li>Description of the experimental testbed </li></ul><ul><li>Problem formulation and controller design </li></ul><ul><li>Performance results </li></ul><ul><li>Conclusions </li></ul>
  3. 3. DATA-CENTER ENERGY COSTS <ul><li>Server energy consumption is growing at 9% per year </li></ul>McKinsey & Co. Report: http://uptimeinstitute.org/content/view/168/57 <ul><li>Data centers are projected to surpass the airline industry in CO 2 emissions by 2020 </li></ul>Carbon dioxide emissions as percentage of world total – industries Carbon emissions by countries (Metric tons of CO 2 per year) Data centers Airlines Shipyards Steel plants Data centers Argentina Nether- lands Malaysia
  4. 4. SERVER UTILIZATION IN DATA CENTERS <ul><li>Server utilization averages about 6%, accounting for idle servers </li></ul>Average daily server utilization ( %) 10 20 30 40 50 60 70 80 90 100 Peak daily server utilization (%) 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 90 100 Up to 30% of servers are idle! McKinsey & Co. Report: http://uptimeinstitute.org/content/view/168/57
  5. 5. <ul><li>Performance-isolated platforms, called virtual machines (VMs), allow resources (e.g., CPU, memory) to be shared on a single server </li></ul>VIRTUALIZATION AS THE ANSWER McKinsey & Co. Report: http://uptimeinstitute.org/content/view/168/57 <ul><li>Enables consolidation of online services onto fewer servers </li></ul><ul><li>Increases per-server utilization and mitigates “server sprawl” </li></ul><ul><li>Enables on-demand computing , a provisioning model where resources are dynamically provisioned as per workload demand </li></ul>Technique Efficiency Impact <ul><ul><li>3-5% </li></ul></ul><ul><ul><li>Deploy virtualization for existing and new demand </li></ul></ul><ul><ul><li>25-30% </li></ul></ul><ul><ul><li>Implement free cooling </li></ul></ul><ul><ul><li>0-15% </li></ul></ul><ul><ul><li>Introduce greener and more power efficient servers </li></ul></ul><ul><ul><li>10-20% </li></ul></ul><ul><ul><li>Selectively turn off core components to increase remaining unit efficiency </li></ul></ul>
  6. 6. <ul><li>We address combined power and performance management in a virtualized computing environment </li></ul><ul><ul><li>The problem is posed as one of sequential optimization under uncertainty and solved using limited look-ahead control (LLC) </li></ul></ul><ul><ul><li>The notion of risk is encoded explicitly in the problem formulation </li></ul></ul><ul><li>Summary of main results </li></ul><ul><ul><li>A server cluster managed using LLC saves 26% in power-consumption costs over a 24 hour period when compared to an uncontrolled system </li></ul></ul><ul><ul><li>Power savings are achieved with very few SLA violations (1.6% of the total number of requests) </li></ul></ul>PROBLEM STATEMENT
  7. 7. OUTLINE <ul><li>Motivation and problem statement </li></ul><ul><li>Description of the experimental testbed </li></ul><ul><li>Problem formulation and controller design </li></ul><ul><li>Performance results </li></ul><ul><li>Conclusions </li></ul>
  8. 8. THE EXPERIMENTAL TESTBED <ul><li>The testbed is a two-tier architecture with front-end application servers and back-end databases </li></ul><ul><li>It hosts two online services ( Gold and Silver ) </li></ul><ul><li>Servers are virtualized </li></ul><ul><li>Performance goals </li></ul><ul><ul><li>Minimize power consumption </li></ul></ul><ul><ul><li>Minimize SLA violations </li></ul></ul><ul><li>We target the application and the database tiers </li></ul>
  9. 9. EXPERIMENTAL SYSTEM <ul><li>Six Dell servers (models 2950 and 1950) comprise the experimental testbed </li></ul><ul><li>Virtualization of the CPU and memory is enabled by VMware ESX Server 3.0 </li></ul><ul><li>Virtual machines run SUSE Enterprise Linux Server Edition 10 </li></ul><ul><li>Control directives use the VMware API, Linux shell commands, and IPMI </li></ul><ul><li>Silver application is Trade6 only; Gold application is Trade6 + extra CPU load </li></ul>2.3 GHz 1.6 GHz 1.6 GHz 1.6 GHz 2.3 GHz 2.3 GHz CPU speed 8 8 8 8 2 8 # of CPU cores 8 GB Poseidon 4 GB Demeter 4 GB Chronos 8 GB Bacchus 8 GB Apollo 4 GB Eros Memory Host name
  10. 10. <ul><li>We assume a session-less workload, i.e., incoming requests are independent of each other </li></ul><ul><li>The transaction mixed is fixed to a constant proportion of browse/buy requests </li></ul>CHARACTERISTICS OF THE INCOMING WORKLOAD <ul><li>The workload to the computing system is time varying and shows significant variability over short time periods </li></ul>
  11. 11. APPLICATION ENVIRONMENT <ul><li>Online services are enabled by enterprise applications </li></ul>Application server Database Web Clients Trade Action Trade Servlets Trade Services Trade Server Pages WebSphere Application Server <ul><li>Trade6 is an example </li></ul><ul><ul><li>It is transaction-based stock trading application from IBM </li></ul></ul><ul><ul><li>It can be hosted across one or more servers in a multi-tier architecture </li></ul></ul>DB2 Database
  12. 12. OUTLINE <ul><li>Motivation and problem statement </li></ul><ul><li>Description of the experimental testbed </li></ul><ul><li>Problem formulation and controller design </li></ul><ul><li>Performance results </li></ul><ul><li>Conclusions </li></ul>
  13. 13. <ul><li>The power/performance management problem is posed as a dynamic resource provisioning problem under dynamic operating constraints </li></ul><ul><li>Objectives </li></ul><ul><ul><li>Maximize the profit generated by the system (i.e., minimize SLA violations and the power consumption cost) </li></ul></ul><ul><li>Decisions to be optimized </li></ul><ul><ul><li>Number of servers to turn on or off </li></ul></ul><ul><ul><li>Number of VMs to provision to each service </li></ul></ul><ul><ul><li>The CPU share given to each VM </li></ul></ul><ul><ul><li>Distribute incoming workload to different servers </li></ul></ul>PROBLEM FORMULATION
  14. 14. PROBLEM FORMULATION (Contd.) Refund Reward <ul><li>Dollars generated (Revenue) </li></ul><ul><ul><li>Obtained as per a (nonlinear) reward-refund curve specified by the SLA </li></ul></ul>Reward is defined by SLA for each service class Violation of SLA results in a refund to client
  15. 15. <ul><li>Key characteristics of the control problem </li></ul><ul><ul><li>Some control actions have (long) dead times ; e.g., switching on a server, instantiating VMs, migrating VMs </li></ul></ul><ul><ul><li>Decisions must be optimized over a discrete domain </li></ul></ul><ul><ul><li>Optimization must be performed quickly, given the dynamics of input </li></ul></ul><ul><li>We use a limited look-ahead control ( LLC ) concept </li></ul>PROBLEM FORMULATION (Contd.) <ul><li>Power consumption cost of operating servers </li></ul><ul><li>Switching costs </li></ul><ul><ul><li>Opportunity cost lost due to the unavailability of servers/VMs involved in provisioning decisions </li></ul></ul><ul><ul><li>Transient power consumption costs </li></ul></ul>
  16. 16. THE LLC FRAMEWORK <ul><li>LLC is same as model predictive control, but in a discrete domain and quickly </li></ul><ul><li>Advantages </li></ul><ul><ul><li>Use predictions to improve control performance </li></ul></ul><ul><ul><li>Robust (iterative feedback) even in dynamic operating conditions </li></ul></ul><ul><ul><li>Inherent compensation for dead times </li></ul></ul><ul><ul><li>Multi-objective and non-linear optimization in the discrete domain under explicit constraints </li></ul></ul>
  17. 17. THE LLC FRAMEWORK (Contd.) <ul><li>Obtain an “optimal” sequence of control inputs </li></ul><ul><li>Apply the first control input in the sequence at time k +1; discard the rest </li></ul>x ( k ) k+4 k+3 k+2 <ul><li>Use a system model to estimate future system states over a prediction horizon </li></ul>k+1
  18. 18. WORKLOAD ESTIMATION USING A PREDICTIVE FILTER <ul><li>A Kalman filter is used to estimate the workload over the prediction horizon </li></ul>Prediction error is about 8% Training phase
  19. 19. CONSTRUCTING THE SYSTEM MODEL <ul><li>System model will base on observed state, control input and estimated workload to create new state. </li></ul><ul><li>The behavior of each application is captured using simulation-based learning and stored in an approximation structure (e.g., lookup table, neural network) – OFFLINE mode </li></ul>
  20. 20. CONSTRUCTING THE SYSTEM MODEL <ul><li>Example 1: Given a 3 GHz CPU share and 1 GB of memory, how many requests can a Gold VM handle before incurring SLA violations? </li></ul><ul><li>Average response time is below the limit doesn’t mean that no violations. </li></ul>
  21. 21. CONSTRUCTING THE SYSTEM MODEL <ul><li>Example 2: Given a 6 GHz CPU share and 1 GB of memory, how many requests can a Gold VM handle before incurring SLA violations? </li></ul>
  22. 22. Observation – non-linear behavior <ul><li>3G: 22 requests, 6G: 29 requests, why we can’t achieve 2 speedup if CPU share is 2 times?? </li></ul><ul><li>Memory or IO is not considered???? </li></ul>
  23. 23. CONSTRUCTING THE SYSTEM MODEL (Contd.) <ul><li>Power = current * voltage </li></ul><ul><li>Two observations: </li></ul><ul><ul><li>Power consumption of boot time is larger than idle state. </li></ul></ul><ul><ul><li>Power consumption having VMs is not greatly larger than idle state. </li></ul></ul>
  24. 24. CONSTRUCTING THE SYSTEM MODEL (Contd.) - Power consumptions <ul><li>Power consumption is closely related to CPU usage. </li></ul>
  25. 25. Does increase of CPU utilization increase Computer Power consumption? <ul><li>The more utilization of the CPU, the more signals generated and processed by it. </li></ul><ul><li>Consequently, the more utilization of the CPU, the greater the energy requirement. </li></ul><ul><li>Power = energy x time. </li></ul><ul><li>So we can than conclude that the greater the CPU utilization the greater the power consumption. </li></ul>
  26. 26. Key Observations <ul><li>(1) Idle machine consumes 70% or more power of full utilization </li></ul><ul><ul><li>Conclusion (1): Power down machine to achieve maximum power savings. </li></ul></ul><ul><li>(2) Intensity of the workload at VMs doesnot affect power consumption and cpu utilization. </li></ul><ul><ul><li>Conclusion (2): Only the number of VMs can affect power consumption. </li></ul></ul><ul><li>(3) Power consumed by a server is a function of instantiated VMs on it. </li></ul>
  27. 27. EXPERIMENTAL SYSTEM <ul><li>Six Dell servers (models 2950 and 1950) comprise the experimental testbed </li></ul><ul><li>Virtualization of the CPU and memory is enabled by VMware ESX Server 3.0 </li></ul><ul><li>Virtual machines run SUSE Enterprise Linux Server Edition 10 </li></ul><ul><li>Control directives use the VMware API, Linux shell commands, and IPMI </li></ul><ul><li>Silver application is Trade6 only; Gold application is Trade6 + extra CPU load </li></ul>2.3 GHz 1.6 GHz 1.6 GHz 1.6 GHz 2.3 GHz 2.3 GHz CPU speed 8 8 8 8 8 8 # of CPU cores 8 GB Poseidon 4 GB Demeter 4 GB Chronos 8 GB Bacchus 8 GB Apollo 4 GB Eros Memory Host name
  28. 28. CPU scheduling mode <ul><li>work-conservative mode (WC-mode): in order to keep the server resources well utilized </li></ul><ul><ul><li>Under WC-mode, the shares are merely guarantees, and CPU is idle if and only if there is no runnable work. </li></ul></ul><ul><li>Non-work-conservative: </li></ul><ul><ul><li>With NWC-mode, the shares are caps, i.e., each client owns its fraction of the CPU. </li></ul></ul><ul><ul><li>It means that if one VM is assigned to 3G HZ cpu, this VM cannot use more than this even if the system is 10G HZ and no other VM at all. </li></ul></ul><ul><li>Assumption: </li></ul><ul><ul><li>Esx server is worked on non-work-conservative mode </li></ul></ul><ul><ul><li>Cpu assignment is not larger than maximum limit of hardware. </li></ul></ul>
  29. 29. DEVELOPING THE OPTIMIZER <ul><li>Issue 1: Risk-aware control </li></ul><ul><ul><li>Due to the energy and opportunity costs incurred when switching hosts and VMs on/off, excessive switching caused by workload variability may actually reduce profits </li></ul></ul><ul><ul><li>We need to encode a notion of risk in the cost function </li></ul></ul>Cost that accumulates during the time a server is being turned on but is unavailable to perform any useful service
  30. 30. <ul><li>Environment-input estimates will have prediction errors </li></ul><ul><li>We encode a notion of risk in the optimization problem </li></ul><ul><ul><li>Generate a set of expected next states for lots of the predicted environment inputs </li></ul></ul>RISK-AWARE CONTROL Construct an uncertainty bound for environment input of interest: Averaged past observed error between actual and forecasted arrival rate Estimated environment input
  31. 31. RISK-AWARE CONTROL (Contd.) <ul><li>A utility function encodes risk into the objective function </li></ul>Maximize utility over horizon and client classes Utility model with tunable risk-preference parameter, β <ul><ul><li>β = 0 : risk-neutral </li></ul></ul><ul><ul><li>β > 0 : risk-averse </li></ul></ul><ul><ul><li>β < 0 : risk-seeking </li></ul></ul><ul><ul><li>Tunable risk-preference </li></ul></ul><ul><ul><li>parameter, β </li></ul></ul><ul><ul><li>Uncertainty as variance </li></ul></ul>Apply a mean-square variance model of utility Formulate a utility maximization problem A > 2*Mean
  32. 32. DEVELOPING THE OPTIMIZER (Contd.) <ul><li>We use a control hierarchy to reduce execution-time overhead </li></ul><ul><ul><li>An L0 controller decides the CPU share to assign to VMs </li></ul></ul><ul><ul><li>An L1 controller decides the number of VMs for each service and the number of servers to keep powered on </li></ul></ul><ul><ul><li>The average execution time of the L1 controller is about 10 seconds </li></ul></ul><ul><li>Issue 2: Execution-time overhead of the controller </li></ul><ul><ul><li>“ Curse of dimensionality” - Problem will show an exponential increase in worst-case complexity with more control options and longer prediction horizons </li></ul></ul>
  33. 33. OUTLINE <ul><li>Motivation and problem statement </li></ul><ul><li>Description of the experimental testbed </li></ul><ul><li>Problem formulation and controller design </li></ul><ul><li>Performance results </li></ul><ul><li>Conclusions </li></ul>
  34. 34. EXPERIMENTAL SYSTEM <ul><li>Six Dell servers (models 2950 and 1950) comprise the experimental testbed </li></ul><ul><li>Virtualization of the CPU and memory is enabled by VMware ESX Server 3.0 </li></ul><ul><li>Virtual machines run SUSE Enterprise Linux Server Edition 10 </li></ul><ul><li>Control directives use the VMware API, Linux shell commands, and IPMI </li></ul><ul><li>Silver application is Trade6 only; Gold application is Trade6 + extra CPU load </li></ul>2.3 GHz 1.6 GHz 1.6 GHz 1.6 GHz 2.3 GHz 2.3 GHz CPU speed 8 8 8 8 8 8 # of CPU cores 8 GB Poseidon 4 GB Demeter 4 GB Chronos 8 GB Bacchus 8 GB Apollo 4 GB Eros Memory Host name
  35. 35. EXPERIMENTAL PARAMETERS 3 VMs Initial configuration for Gold service (application tier) L1: 3 steps, L0: 1 step Prediction horizon 2 min. 55 sec. Time delay to power on a host 1 min. 45 sec. Time delay to power on a VM $0.3 Cost per KiloWatt hour L1: 150 sec, L0: 30 sec Control sampling period 3 VMs Initial configuration for Silver Service (application tier) Value Parameter
  36. 36. MAIN RESULTS <ul><li>A risk-neutral controller conserves, on average, 26% more energy than a system without dynamic control with very few SLA violations </li></ul><ul><li>More SLA violations for Silver requests than for Gold requests. </li></ul>32% 45% 17% 17% 18% Energy savings 3.5% 1.1% 1.4% 1.2% 3.2% % of SLA violations (Silver) 1.8% Workload 5 0.4% Workload 3 0.5% Workload 2 2.3% Workload 1 0.2% Workload 4 % of SLA violations (Gold) Workload
  37. 37. RESULTS (Contd.) <ul><li>CPU shares assigned to the Gold and Silver applications over a 24-hour period – L0 layer </li></ul>
  38. 38. RESULTS (Contd.) <ul><li>Number of virtual machines assigned to the Gold and Silver applications over a 24-hour period – L1 layer </li></ul>
  39. 39. EFFECT OF THE RISK PREFERENCE PARAMETER <ul><li>A risk-averse (  = 2 ) controller conserves about the same amount of energy as a risk-neutral (  = 0 ) controller </li></ul>25.3 % 20.8 % Energy savings (risk-neutral control) (  = 0) 25.2 % 20.9 % Energy savings (risk-averse control) (  = 2) Workload 7 Workload 6 Workload
  40. 40. EFFECT OF THE RISK PREFERENCE PARAMETER (Contd.) <ul><li>A risk-averse controller (  = 2 ) maintains a higher QoS (Less violations) than a risk-neutral (  = 0 ) controller by reducing switching activity </li></ul>Best-case risk-averse controller:   34,201 (2.7%) 28,635 (2.3%) SLA violations (risk-neutral control) (  = 0) 25,606 (2.0%) 15,672 (1.7%) SLA violations (risk-averse control) (  = 2) 25% Workload 7 45% Workload 6 % reduction in SLA violations Workload 40 30 Switching activity (risk-neutral control) (  = 0) 30 28 Switching activity (risk-averse control) (  = 2) 25% Workload 7 7% Workload 6 % reduction in switching activity Workload
  41. 41. OPTIMALITY CONSIDERATIONS <ul><li>The controller cannot achieve optimal performance </li></ul><ul><ul><li>Limited by errors in workload predictions </li></ul></ul><ul><ul><li>Limited by constrained control inputs </li></ul></ul><ul><ul><li>Limited by a finite prediction horizon </li></ul></ul><ul><li>To evaluate optimality, profit gains of a risk-neutral and best-case risk-averse controller were compared against an “oracle” controller with perfect knowledge of the future </li></ul>Oracle Risk averse Risk neutral Controller 16.3% 25.2% 25.3% Total Energy Savings 32 14,228 (1.1%) 38 25,606 (2.0%) 40 34,201 (2.7%) Num. times hosts switched Total SLA violations
  42. 42. CONCLUSIONS <ul><li>We have addressed power and performance management in a virtualized computing environment within a LLC framework </li></ul><ul><li>The cost of control and the notion of risk is encoded explicitly in the problem formulation </li></ul><ul><li>A server cluster managed using LLC saves 26% in power-consumption costs over a 24 hour period when compared to an uncontrolled system </li></ul><ul><li>Power savings are achieved with very few SLA violations (1.6% of the total number of requests) </li></ul><ul><li>Our recommendation is a risk-averse controller since it reduces SLA violations and switching activity </li></ul>
  43. 43. Conclusion (1) – Why significant? <ul><li>Why significant? </li></ul><ul><ul><li>Using virtualization, implement a dynamic resource provisioning model </li></ul></ul><ul><ul><li>Integrate power and performance management, reduce energy cost (26%) while causing little SLA( service level agreement) violation (less than 3%) </li></ul></ul>
  44. 44. Conclusion(2) - Alternate approach? <ul><li>Alternate approach? </li></ul>Technique Efficiency Impact <ul><ul><li>3-5% </li></ul></ul><ul><ul><li>Deploy virtualization for existing and new demand </li></ul></ul><ul><ul><li>25-30% </li></ul></ul><ul><ul><li>Implement free cooling </li></ul></ul><ul><ul><li>0-15% </li></ul></ul><ul><ul><li>Introduce greener and more power efficient servers </li></ul></ul><ul><ul><li>10-20% </li></ul></ul><ul><ul><li>Selectively turn off core components to increase remaining unit efficiency </li></ul></ul>
  45. 45. Conclusion (3) – Improvement? <ul><li>Simplify the control logic to reduce the exe. time </li></ul><ul><li>Integrate the memory usage when modify VM configuration </li></ul><ul><li>Provide a mechanism to decide the granularity to create VMs – One 6G VM can handle more requests than that of two 3G VMs. </li></ul>
  46. 46. SCALABILITY <ul><li>Execution time of the controller can be reduced through various techniques </li></ul><ul><ul><li>Approximating control </li></ul></ul><ul><ul><li>Implementing the controller in hardware </li></ul></ul><ul><ul><li>Increasing the number of tiers in the control hierarchy </li></ul></ul><ul><ul><li>Simplifying the iterative search process to “hold” a control input constant over the prediction horizon </li></ul></ul><ul><li>A neural network or regression tree can be trained to learn the decision-making behavior of the optimizer </li></ul>
  47. 47. Scalability problem <ul><li>Scalability is not good, current result is based on 5 hosts only. But there can be dozens or thousands of servers in actual data center. </li></ul><ul><ul><ul><li>5 hosts - < 10 sec </li></ul></ul></ul><ul><ul><ul><li>10 hosts - 2min. 30 sec </li></ul></ul></ul><ul><ul><ul><li>15 hosts – 30 min. </li></ul></ul></ul>
  48. 48. Questions? <ul><li>Thank you! </li></ul>

×