Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Monitor-Based Testing of
Elasti...
© AtlanMod (atlanmod-contact@mines-nantes.fr)
● Cloud Computing Elasticity
● Motivation
● Test Procedure
● Experiments
● C...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
3
● ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
4
● ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
5
● ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
6
● ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
7
● ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
8
● ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
9
● ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Cloud Computing Elasticity
10
●...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
● Elasticity states transition....
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Test Procedure
● Test cloud sys...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Experiments
● Research question...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Experiments
Question 1: system ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Experiments
● First experiment ...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Question 2: test execution duri...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Experiments
● Result of the sec...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Conclusion and Future Work
● Id...
© AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr)
Monitor-Based Testing of
Elasti...
Upcoming SlideShare
Loading in …5
×

Monitor-Based Testing of Elastic Cloud Computing Applications

387 views

Published on

A test procedure that monitors the "elasticity states", and executes the test accordingly.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Monitor-Based Testing of Elastic Cloud Computing Applications

  1. 1. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Monitor-Based Testing of Elastic Cloud Computing Applications Michel Albonico PhD Student - AtlanMod - EMN (Nantes, France) (michel.albonico@inria.fr) Jean-Marie Mottu Gerson Sunyé 1 5thInt.WorkshoponLargeScaleTesting Delft,Netherlands-2016
  2. 2. © AtlanMod (atlanmod-contact@mines-nantes.fr) ● Cloud Computing Elasticity ● Motivation ● Test Procedure ● Experiments ● Conclusion and Future Work Outline 2
  3. 3. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Cloud Computing Elasticity 3 ● Cloud computing elasticity: The ability of a cloud infrastructure/system modifying its resource configuration according to demand.
  4. 4. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Cloud Computing Elasticity 4 ● Thresholds: ○ Scale-out threshold: maximum resource usage, e.g., 80% of CPU usage; ○ Scale-in threshold: minimum resource usage, e.g., 20% of CPU usage; ○ Used to decide when varying a resource. Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold Scale-out Threshold Breaching Scale-in Threshold Breaching Time (s) Resource (Processors) 1 Legend 80% 0.8 20% 0.2
  5. 5. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Cloud Computing Elasticity 5 ● Resource demand varies according to workload variations. ○ Example: ■ number of users increases from 1 to 2, the resource demand doubles. Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold Scale-out Threshold Breaching Scale-in Threshold Breaching Time (s) Resource (Processors) 1 Legend 80% 0.8 20% 0.2
  6. 6. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Cloud Computing Elasticity 6 ● Resource demand varies over time; ● Scale-out threshold breaching. Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold Scale-out Threshold Breaching Scale-in Threshold Breaching Time (s) Resource (Processors) 1 Legend scale-out threshold breaching 80% 0.8 20% 0.2
  7. 7. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Cloud Computing Elasticity 7 ● Resource demand varies over time; ● Scale-out threshold breaching; ● Scale-out reaction time; Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold Scale-out Threshold Breaching Scale-in Threshold Breaching Time (s) Resource (Processors) 1 Legend scale-out reaction time 80% 0.8 20% 0.2
  8. 8. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Cloud Computing Elasticity 8 ● Resource demand varies over time; ● Scale-out threshold breaching; ● Scale-out reaction time; ● Scale-out time, then the thresholds are updated. Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold Scale-out Threshold Breaching Scale-in Threshold Breaching Time (s) Resource (Processors) 2 1 80% 0.8 Legend scale-out time 80% 1.6 20% 0.2 20% 0.4
  9. 9. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Cloud Computing Elasticity 9 ● Scale-in: ○ Scale-in threshold breaching; ○ Scale-in reaction time (resource is no longer available); ■ Thresholds reconfiguration. ○ Scale-in time. Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold Scale-out Threshold Breaching Scale-in Threshold Breaching Time (s) Resource (Processors) 2 1 Legend scale-in time scale-in reaction time 80% 0.8 80% 1.6 20% 0.2 20% 0.4
  10. 10. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Cloud Computing Elasticity 10 ● Elasticity states transition. scale-out threshold breaching scale-in threshold breaching
  11. 11. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) ● Elasticity states transition. ● Related work only test during the ready state; ● Scaling states: ○ Considerable time: in our experiments, scaling-out takes more than 90 seconds (Amazon EC2); ○ Great part of the adaptation tasks: replication data, leader election, etc. Motivation 11
  12. 12. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Test Procedure ● Test cloud systems during all the elasticity states; ● Execute tests dynamically: ○ Associate test cases to a set of elasticity states; ○ Execute the test according to the current elasticity state. ● Test execution: ○ Periodically monitor the resource during the test execution; ■ Current elasticity state. ○ (Re)-execute the associated test cases during the current elasticity state. 12
  13. 13. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Experiments ● Research questions (answered by the experiments): 1. Is it necessary to run the test during different elasticity states? a. Does a cloud system react distinctly depending on the elasticity state? 2. Is it possible to execute the test during different elasticity states and to assign the test verdicts accordingly? 13
  14. 14. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Experiments Question 1: system behavior during different elasticity state. ● First experiment: Measure the performance of a cloud system during different elasticity states. ○ Manually executed; ○ Workload (50% read / 50% write); ○ 2500 operations per second (ops). 14
  15. 15. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Experiments ● First experiment results: ○ 2000 ops: covers all the performance drops; ○ Elasticity states extracted from the log files; RQ1: It is necessary to run the test during different elasticity states. 15 Performance-OperationsperSecond(ops) Minimal Performance Measured Performance R R R R R RSISISO SO SOSI 200 400 1000 1200 800 600 Time (s)
  16. 16. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Question 2: test execution during different elasticity states + test verdicts assignment. ● Second experiment: (same workload) ○ We use our test procedure; ○ We monitor the elasticity states throughout the test execution; ○ Test Case: ■ answered operation >= 2000 ops -> pass ■ otherwise -> fail ○ Same test case associated to every elasticity state. ■ Test case re-executed throughout the cloud system execution. Experiments 16
  17. 17. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Experiments ● Result of the second experiment: ○ Test through different elasticity states; ○ Assign test verdicts to different elasticity states; ■ Proportional to the previous experiment (correct elasticity states). RQ2: It is possible to execute the test according to the elasticity state, and we are able to assign the test verdicts correctly. 17
  18. 18. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Conclusion and Future Work ● Identify all the performance problems; ● Assign the test verdicts to the correct elasticity states (at runtime); ● Address the scaling states, which are not addressed by related work; ● Future work: ○ Write functional test cases; ○ Apply to other study cases; ○ Generate test cases based on elasticity states. 18
  19. 19. © AtlanMod (atlanmod-contact@mines-nantes.fr)© AtlanMod (atlanmod-contact@mines-nantes.fr) Monitor-Based Testing of Elastic Cloud Computing Applications Michel Albonico PhD Student - AtlanMod - EMN (Nantes, France) (michel.albonico@inria.fr) Jean-Marie Mottu Gerson Sunyé 19 5thInt.WorkshoponLargeScaleTesting Delft,Netherlands-2016

×