Preventing Guests from Spinning Around


     How to Deal with Lock-Holder Preemption
                              Thomas...
Spinlock Basics

Spinlocks wait actively as opposed to sleeping locks

Used for short critical sections




 2            ...
Spinlock Wait Times – Kernbench




3           How to Deal with Lock-Holder Preemption
Spinlock Wait Times – Kernbench




4           How to Deal with Lock-Holder Preemption
Spinlocks and Virtualization




5   How to Deal with Lock-Holder Preemption
Spinlocks and Virtualization




6            How to Deal with Lock-Holder Preemption
Spinlocks and Virtualization




7            How to Deal with Lock-Holder Preemption
Spinlocks and Virtualization




8            How to Deal with Lock-Holder Preemption
Spinlocks and Virtualization




9            How to Deal with Lock-Holder Preemption
Spinlocks and Virtualization




10           How to Deal with Lock-Holder Preemption
Is lock-holder preemption problematic?




11    How to Deal with Lock-Holder Preemption
Kernbench in a Guest




12          How to Deal with Lock-Holder Preemption
Kernbench vs. 'while(true)'




13          How to Deal with Lock-Holder Preemption
Time, not Times




14          How to Deal with Lock-Holder Preemption
And in Numbers?




                      guest time                  time spent spinning
                              [s...
What can we do about it?




16   How to Deal with Lock-Holder Preemption
Dealing with lock-holder preemption

LHP avoidance
 No spinlock held in userspace
 Idea: Avoid preempting guest in kerne...
Helping locks: Ingredients

1) Guest kernel: new 'yield' hypercall when waiting
   unusually long
         Modify spinlock...
Threshold: Upper boundary




19         How to Deal with Lock-Holder Preemption
Threshold: Lower boundary




20         How to Deal with Lock-Holder Preemption
Scheduling Strategy

Good choices:
 VCPUs of the same VM to make progress locally
 (Potential) preempted lock-holders
 ...
What about performance?




22   How to Deal with Lock-Holder Preemption
Histogram with 'yield' hypercall




23           How to Deal with Lock-Holder Preemption
Performance




             wall clock guest time time spent spinning
                    [s]         [s]      [s]      [...
Efficiency




25           How to Deal with Lock-Holder Preemption
Efficiency




26           How to Deal with Lock-Holder Preemption
Efficiency




27           How to Deal with Lock-Holder Preemption
Efficiency




28           How to Deal with Lock-Holder Preemption
Efficiency




29           How to Deal with Lock-Holder Preemption
Efficiency




30           How to Deal with Lock-Holder Preemption
Efficiency




31           How to Deal with Lock-Holder Preemption
Efficiency




                      117 sec
                                  ×7.6 % = 3.7 %
                 117 sec 12...
FIFO ticket spinlocks




33   How to Deal with Lock-Holder Preemption
FIFO ticket spinlocks

Next ticket in dispenser: queue tail

„Now serving“ display at counter: queue head




Lock: atomic...
FIFO ticket spinlocks




35           How to Deal with Lock-Holder Preemption
Ticket locks and virtualization




           wall clock guest time                 time spent spinning
                 ...
Ticket locks and virtualization




           wall clock guest time                 time spent spinning
                 ...
Conclusion


       Lock-holder preemption quite serious:
             7.6% guest time wasted

                  Helping l...
Trademark Attribution


AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. in...
Upcoming SlideShare
Loading in...5
×

XS Boston 2008 Guest Spinning

463

Published on

Thomas Friebel: Preventing Guests from Spinning Around - Video

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
463
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

XS Boston 2008 Guest Spinning

  1. 1. Preventing Guests from Spinning Around How to Deal with Lock-Holder Preemption Thomas Friebel June 23, 2008
  2. 2. Spinlock Basics Spinlocks wait actively as opposed to sleeping locks Used for short critical sections 2 How to Deal with Lock-Holder Preemption
  3. 3. Spinlock Wait Times – Kernbench 3 How to Deal with Lock-Holder Preemption
  4. 4. Spinlock Wait Times – Kernbench 4 How to Deal with Lock-Holder Preemption
  5. 5. Spinlocks and Virtualization 5 How to Deal with Lock-Holder Preemption
  6. 6. Spinlocks and Virtualization 6 How to Deal with Lock-Holder Preemption
  7. 7. Spinlocks and Virtualization 7 How to Deal with Lock-Holder Preemption
  8. 8. Spinlocks and Virtualization 8 How to Deal with Lock-Holder Preemption
  9. 9. Spinlocks and Virtualization 9 How to Deal with Lock-Holder Preemption
  10. 10. Spinlocks and Virtualization 10 How to Deal with Lock-Holder Preemption
  11. 11. Is lock-holder preemption problematic? 11 How to Deal with Lock-Holder Preemption
  12. 12. Kernbench in a Guest 12 How to Deal with Lock-Holder Preemption
  13. 13. Kernbench vs. 'while(true)' 13 How to Deal with Lock-Holder Preemption
  14. 14. Time, not Times 14 How to Deal with Lock-Holder Preemption
  15. 15. And in Numbers? guest time time spent spinning [s] [s] [%] single kernbench 109.0 0.2 0.2% kernbench vs while(1) 117.3 9.0 7.6% difference 7.6% 15 How to Deal with Lock-Holder Preemption
  16. 16. What can we do about it? 16 How to Deal with Lock-Holder Preemption
  17. 17. Dealing with lock-holder preemption LHP avoidance  No spinlock held in userspace  Idea: Avoid preempting guest in kernel space  Postpone guest switch to kernel exit  Problem: extraordinary long critical sections, e.g. Apache using sendfile() Helping locks  Instead of busy waiting, switch to preempted lock- holder  Problem: finding the preempted lock-holder 17 How to Deal with Lock-Holder Preemption
  18. 18. Helping locks: Ingredients 1) Guest kernel: new 'yield' hypercall when waiting unusually long Modify spinlock loop ● 2) Reasonable threshold for 'unusually long' Histograms help ● 3) Selecting which VCPU to switch to 18 How to Deal with Lock-Holder Preemption
  19. 19. Threshold: Upper boundary 19 How to Deal with Lock-Holder Preemption
  20. 20. Threshold: Lower boundary 20 How to Deal with Lock-Holder Preemption
  21. 21. Scheduling Strategy Good choices:  VCPUs of the same VM to make progress locally  (Potential) preempted lock-holders  Cache-„near“ VCPUs Neither/nor:  VCPUs in user space Bad choices:  VCPUs which yielded recently 21 How to Deal with Lock-Holder Preemption
  22. 22. What about performance? 22 How to Deal with Lock-Holder Preemption
  23. 23. Histogram with 'yield' hypercall 23 How to Deal with Lock-Holder Preemption
  24. 24. Performance wall clock guest time time spent spinning [s] [s] [s] [%] LHP 34.8 117.3 9.0 7.6% yield 33.5 108.4 0.0 0.0% difference -3.9% -7.6% -7.6% 24 How to Deal with Lock-Holder Preemption
  25. 25. Efficiency 25 How to Deal with Lock-Holder Preemption
  26. 26. Efficiency 26 How to Deal with Lock-Holder Preemption
  27. 27. Efficiency 27 How to Deal with Lock-Holder Preemption
  28. 28. Efficiency 28 How to Deal with Lock-Holder Preemption
  29. 29. Efficiency 29 How to Deal with Lock-Holder Preemption
  30. 30. Efficiency 30 How to Deal with Lock-Holder Preemption
  31. 31. Efficiency 31 How to Deal with Lock-Holder Preemption
  32. 32. Efficiency 117 sec ×7.6 % = 3.7 % 117 sec 126 sec Real result of 3.9% is reasonable ➔ Highly efficient ➔ 32 How to Deal with Lock-Holder Preemption
  33. 33. FIFO ticket spinlocks 33 How to Deal with Lock-Holder Preemption
  34. 34. FIFO ticket spinlocks Next ticket in dispenser: queue tail „Now serving“ display at counter: queue head Lock: atomic( ticket = tail++ ); while ( head != ticket ); Unlock: atomic( head++ ); 34 How to Deal with Lock-Holder Preemption
  35. 35. FIFO ticket spinlocks 35 How to Deal with Lock-Holder Preemption
  36. 36. Ticket locks and virtualization wall clock guest time time spent spinning [s] [s] [s] [%] LHP 2825.1 22434.2 22270.4 99.3% 36 How to Deal with Lock-Holder Preemption
  37. 37. Ticket locks and virtualization wall clock guest time time spent spinning [s] [s] [s] [%] LHP 2825.1 22434.2 22270.4 99.3% yield 34.1 123.6 6.6 5.4% 37 How to Deal with Lock-Holder Preemption
  38. 38. Conclusion Lock-holder preemption quite serious: 7.6% guest time wasted Helping locks: 3.9% system performance improvement! (Amdahl's law explains why) New ticket spinlocks: 30 secs kernbench takes 45 minutes Helping locks help here, too 38 How to Deal with Lock-Holder Preemption
  39. 39. Trademark Attribution AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdictions. Other names used in this presentation are for identification purposes only and may be trademarks of their respective owners. ©2008 Advanced Micro Devices, Inc. All rights reserved. 39 How to Deal with Lock-Holder Preemption
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×