XPDS13: "Unlimited" Event Channels - David Vrabel, Citrix

1,120 views

Published on

Event Channels are Xen's mechanism for paravirtualized interrupts. These were limited to only 4096 which then limits the number of guests that a host may support to around 300 to 500. This presentation will give a brief introduction to event channels, a detailed look at the new, innovative FIFO-based event channel ABI that increases the limit to over 100,000 as well as having several other useful features (e.g., multiple priorties). Some of the key performance measurments of the new ABI will be shown.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,120
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
40
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

XPDS13: "Unlimited" Event Channels - David Vrabel, Citrix

  1. 1. “Unlimited” Event Channels David Vrabel 24 October 2013
  2. 2. What are Event Channels? Paravirtual interrupts Edge triggered Bidirectional Directed at a single VCPU 24 October 2013 “Unlimited” Event Channels 2 / 13
  3. 3. How do Event Channels Work? Domain A 1 Domain B Notify Xen 24 October 2013 Shared Memory 2 4 Set Pending “Unlimited” Event Channels Call Handlers 3 Upcall 3 / 13
  4. 4. Original (2-Level) Design Per-VCPU Shared Memory 64 bits Selector ... 1 Shared Memory 64 bits Pending ... Masked 0 1 0 0 ... ... 0 1 0 ... ... ... 4096 bits 24 October 2013 “Unlimited” Event Channels 4 / 13
  5. 5. Problems Too few No priorities Unfair 24 October 2013 “Unlimited” Event Channels 5 / 13
  6. 6. Additional Requirements Identical ABI between 32-bit and 64-bit guests Reasonable memory usage Easily extensible 24 October 2013 “Unlimited” Event Channels 6 / 13
  7. 7. New (FIFO-based) Design Bit 15 READY 1 0 Bit 31 1 Port 0 1 (Reserved) 30 29 16 0 P M L LINK 1 0 1 6 1 0 1 0 1 0 1 3 2 3 HEAD Priority 0 5 1 ... Per-VCPU shared memory 24 October 2013 6 7 ... 1 15 4 Up to 132,072 words Shared memory “Unlimited” Event Channels 7 / 13
  8. 8. Raising an Event function raise(q, p) 1. Set pending E[p].pending = 1 if not E[p].masked and not E[p].linked E[p].linked = 1 if T[q] != p linked = link(T[q], p) else linked = false if not linked C.head[q] = p T[q] = p 24 October 2013 2. Unmasked and not in list? Add to list 3. Set LINK of tail or Set HEAD in control block 4. Advance tail “Unlimited” Event Channels 8 / 13
  9. 9. Handling an Event function handle_one_event(q) 1. Get local head p = H[q] if p == 0 p = C.head[q] link = unlink(p) H[q] = link if E[p].pending and not E[p].masked handle(p) 2. List was empty? Get new HEAD from control block 3. Clear LINKED and LINK 4. Advance local head 5. Pending and unmasked? Handle IRQ 24 October 2013 “Unlimited” Event Channels 9 / 13
  10. 10. Event Channel Fairness 3000 2-Level FIFO-based Latency (ns) 2500 2000 1500 1000 500 0 75 80 85 90 Event Channel Port 24 October 2013 “Unlimited” Event Channels 10 / 13
  11. 11. Current Status Xen support in 4.4 Linux support in 3.14 24 October 2013 “Unlimited” Event Channels 11 / 13
  12. 12. Further Work Investigate how priorities may be used Add support for limiting number of event channels to other toolstacks (XAPI, libvirt) Add support in other OSes Xen event lock scalability Linux PIRQ setup/teardown refactoring 24 October 2013 “Unlimited” Event Channels 12 / 13
  13. 13. Questions? Design Document http://xenbits.xenproject.org/people/dvrabel/event-channels-F.pdf 24 October 2013 “Unlimited” Event Channels 13 / 13

×