XPDS14: Efficient Interdomain Transmission of Performance Data - John Else, Citrix
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

XPDS14: Efficient Interdomain Transmission of Performance Data - John Else, Citrix

on

  • 304 views

As users demand greater scalability from Citrix XenServer, the transmission of performance data from guests via xenstore is increasingly becoming a bottleneck. Future use of service domains is likely ...

As users demand greater scalability from Citrix XenServer, the transmission of performance data from guests via xenstore is increasingly becoming a bottleneck. Future use of service domains is likely to make this problem worse. A simple, efficient way of transmitting time-varying datasets between userspace components in different domains is required. This talk will propose a lock-free mechanism to allow interdomain reporting of performance data without relying on continuous xenstore usage, and describe how it fits into the XAPI toolstack.

Statistics

Views

Total Views
304
Views on SlideShare
296
Embed Views
8

Actions

Likes
2
Downloads
1
Comments
0

2 Embeds 8

http://www.xenproject.org 5
https://twitter.com 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

XPDS14: Efficient Interdomain Transmission of Performance Data - John Else, Citrix Presentation Transcript

  • 1. Efficient interdomain transmission of performance data John Else john.else@citrix.com
  • 2. XAPI project XenAPI XAPI XCP/XenServer toolstack
  • 3. XAPI project High level OO API around Xen HTTP interface Collects performance data from dom0/guests
  • 4. Performance data Numeric Changes regularly - requires sampling If we sometimes lose a sample - that's OK
  • 5. Interdomain performance data Currently one metric per domU: free memory Transmitted via xenstore Even this has been enough to cause performance problems!
  • 6. Improvements so far Data collection caused laggy API response from XAPI -> XAPI disaggregation: rrdd, networkd and xenopsd
  • 7. Improvements so far High xenstore load reduced bootstorm performance -> Watch xenstore instead of polling
  • 8. What next? Take xenstore out of the data path Create a generic API for: Driver domain monitoring Application monitoring ...
  • 9. rrdd plugins - dom0
  • 10. rrdd plugins - interdomain
  • 11. rrdd plugins - protocol v1 header 10 bytes data length (ASCII) 8 bytes md5sum (ASCII) 32 bytes json data inc. timestamp ? bytes
  • 12. rrdd plugins - protocol v1 DATASOURCES <- header 00000a79 <- data length 6cc0472a94896d245dcea04a37c26474 <- data md5sum { "timestamp": 1407715758, "datasources": { "io_throughput_read_9d0e83ec": { "description": "Data read from the SR, in MiB/s", "owner": "host", "value": "0.55", "value_type": "float", "type": "absolute_to_rate", "units": "MiB/s", "min": "0.00", "max": "inf" } } }
  • 13. rrdd plugins - protocol v1 DATASOURCES <- header 00000a79 <- data length 6cc0472a94896d245dcea04a37c26474 <- data md5sum { "timestamp": 1407715758, "datasources": { "io_throughput_read_9d0e83ec": { "description": "Data read from the SR, in MiB/s", "owner": "host", "value": "0.55", "value_type": "float", "type": "absolute_to_rate", "units": "MiB/s", "min": "0.00", "max": "inf" } } }
  • 14. rrdd plugins - protocol v2 Separate data from metadata Pack as much as possible as binary
  • 15. rrdd plugins - protocol v2 header 10 bytes data crc32 4 bytes metadata crc32 4 bytes datasource count (int32) 4 bytes timestamp (int64) 8 bytes datasource values n * 8 bytes metadata length (int32) 4 bytes json metadata ? bytes
  • 16. rrdd plugins - protocol v2 header 10 bytes data crc32 4 bytes metadata crc32 4 bytes datasource count (int32) 4 bytes timestamp (int64) 8 bytes datasource values n * 8 bytes metadata length (int32) 4 bytes json metadata ? bytes
  • 17. rrdd plugins - protocol v2 DATASOURCES <- header .... <- data CRC .... <- metadata CRC .... <- datasource count ........ <- timestamp ........ <- datasource value .... <- metadata length { "datasources": { "io_throughput_read_9d0e83ec": { "description": "Data read from the SR, in MiB/s", "owner": "host", "value_type": "float", "type": "absolute_to_rate", "units": "MiB/s", "min": "0.00", "max": "inf" } } }
  • 18. rrdd plugins - protocol v2 DATASOURCES <- header .... <- data CRC .... <- metadata CRC .... <- datasource count ........ <- timestamp ........ <- datasource value .... <- metadata length { "datasources": { "io_throughput_read_9d0e83ec": { "description": "Data read from the SR, in MiB/s", "owner": "host", "value_type": "float", "type": "absolute_to_rate", "units": "MiB/s", "min": "0.00", "max": "inf" } } }
  • 19. rrdd plugins - protocol v2 V2 protocol gives about a 10x speedup in the "usual" case, i.e. set of exported datasources doesn't change.
  • 20. Demo
  • 21. Further info Protocol: Frontend: Slides: IRC: freenode #xen-api Email: john.else@citrix.com github.com/xapi-project/rrd-transport github.com/xapi-project/ocaml-rrdd-plugin johnelse.github.io/xen-summit-2014