.
Mission Statement
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://...
.
Mission Statement / Goals
1. a holistic method for performance visualization
in DCs
2. a clean slate + there are no riva...
.
Visualization Primer
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http...
.
Visualization Theory
.
Visualization Tradition...
..
.... is to maximize information density and minimize labeling
• met...
.
Visualization Methods
• Waterpipe Model (the proposal) is explained later
• key point: 2D → 3D may actually make things ...
.
Visualization Examples
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- ht...
.
Viz : Example (1)
• Community QoS visualization from 04
0 6400 12800 19200
Batch size (bytes)
0
800
1600
2400
3200
4000
...
.
Viz : Example (2)
Meter Merger
Analyzer
Analyzer
Analyzer
Buffer 5sTimeout 5s
Sampling
timeout 10s
Buffer 1s
Buffer 10s
...
.
Proposal : The Waterpipe Model
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Cente...
.
Proposal : Waterpipe Model
• Existing (1) : there is already fluid model in traffic simulations
◦ differential equations...
.
Waterpipe : Model 1
• one tap is one VM/PM/etc.
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizati...
.
Waterpipe : Model 2
• same, but more focus on in/out balance of goodput, congestion, etc
M.Zhanikeev -- maratishe@gmail....
.
Implementation
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit...
.
Implementation : Status
• 80% complete
• already have all the components from past research -- see below
• the visual pa...
.
Implementation : Multicore
• heavy use of multicore and realtime processing -- recent paper in 07
Global
Networks
Data C...
.
Implementation : Sockets
• WebSockets -- streaming, native binary support, heavily interactive,
client-side aggregation ...
.
Design : Capture/Processing
• any level of abstraction: PMs, VMs, storage, etc.
• realtime processing and decision-makin...
.
Design : Interactive Display
• aggregation and processing on multicore, highly interactive display
• basically, a player...
.
Wrapup
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/mara...
.
Wrapup
• 80% complete with most important components fully available
• currently testing on a small cloud at Kyutech
• o...
.
That’s all, thank you ...
M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers --...
.
[01] myself+0 (2013)
Multi-Source Stream Aggregation in the Cloud
ADCN Wiley Book
[02] myself+0 (2013)
Experiments with ...
.
[06] J.Steele+1 (2010)
Beautiful Visualization
O'Reilly Media
[07] myself+0 (2014)
A Software Design and Algorithms for ...
.
Extras : Processing Cost
Hashing
Increasing
Per-Unit Cost
Manager
Prefix
Matching
Cores that
do not
match
Process
Stage 1...
.
Extras : Multicore Processing
PF_
RING
PF_
RING
Time
Manager
Shared Memory
Capture
Capture
…
Core 1
Core 2
Core 3
….
Cor...
Towards a Practical Method for Interactive Traffic Visualizations in Data Centers
Upcoming SlideShare
Loading in …5
×

Towards a Practical Method for Interactive Traffic Visualizations in Data Centers

331
-1

Published on

It is not rare when relatively primitive charts or diagrams are referred to as visualizations. When one is monitoring/managing performance at the scale of a data center, it is especially obvious that such visualizations are not very helpful as they fail to capture or present information in a holistic way, thus making it hard to make management decisions. This paper proposes a holistic visualization method which is based on the theory of information visualization, namely (1) metaphor as base, (2) density maximization and (3) proximity packing. The proposed method can concurrently digest multiple streams of packet traffic but can also incorporate streams of alternative monitoring data.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
331
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Towards a Practical Method for Interactive Traffic Visualizations in Data Centers

  1. 1. . Mission Statement M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 2/24 ... 2/24
  2. 2. . Mission Statement / Goals 1. a holistic method for performance visualization in DCs 2. a clean slate + there are no rivals 3. a generic design applicable to various kinds of performance data ◦ traffic in this paper, but ideally anything M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 3/24 ... 3/24
  3. 3. . Visualization Primer M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 4/24 ... 4/24
  4. 4. . Visualization Theory . Visualization Tradition... .. .... is to maximize information density and minimize labeling • metrics are density and proximity -- proximity packing problem • Tufte 05 and others 06 . New (proposed) Formulation... .. .... adds metaphor and enhances the labeling metric • metaphor is a strict requirement -- 3D/3D charts are not metaphors • higher number of metrics 05 E.Tufte+0 "The Visual Display of Quantitative Information" Graphics Press (1983) 06 J.Steele+1 "Beautiful Visualization" O'Reilly Media (2010) M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 5/24 ... 5/24
  5. 5. . Visualization Methods • Waterpipe Model (the proposal) is explained later • key point: 2D → 3D may actually make things worse! Waterpipe Model 2D 3D Metaphor YES NO NO Information Density HIGH LOW BETTER Labeling minimization GOOD BAD BETTER Perceptual enhancement GOOD BAD WORSE M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 6/24 ... 6/24
  6. 6. . Visualization Examples M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 7/24 ... 7/24
  7. 7. . Viz : Example (1) • Community QoS visualization from 04 0 6400 12800 19200 Batch size (bytes) 0 800 1600 2400 3200 4000 OWD(ms)+TXtime(x0.1ms) M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 8/24 ... 8/24
  8. 8. . Viz : Example (2) Meter Merger Analyzer Analyzer Analyzer Buffer 5sTimeout 5s Sampling timeout 10s Buffer 1s Buffer 10s Buffer 30s Lower cluster Upper cluster 47kb 25kb 21kb 12kb 0.6kb 0.3kb Per second Per buffer 10kb 7.4kb 32kb 26kb Different buffer size • also from 04 • Petri Nets are based on the tree metaphor -- simple but powerful 04 myself+0 "A holistic... for measuring end-to-end QoS at data centres" IJCSE (2014) M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 9/24 ... 9/24
  9. 9. . Proposal : The Waterpipe Model M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 10/24 ... 10/24
  10. 10. . Proposal : Waterpipe Model • Existing (1) : there is already fluid model in traffic simulations ◦ differential equations ◦ data unit is aggregate flow rather than packets ◦ you do not actually run such a simulation -- you take a snapshot at a given point in time • Existing (2) : Bucket Model in queueing theory -- a bucket of water • Waterpipe Model is similar, only with a strong visual component -- the metaphor • Data Center is literally viewed as a system of water pipes -- a job for plumbers? • the metaphor is flexible, can support multiple variants M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 11/24 ... 11/24
  11. 11. . Waterpipe : Model 1 • one tap is one VM/PM/etc. M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 12/24 ... 12/24
  12. 12. . Waterpipe : Model 2 • same, but more focus on in/out balance of goodput, congestion, etc M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 13/24 ... 13/24
  13. 13. . Implementation M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 14/24 ... 14/24
  14. 14. . Implementation : Status • 80% complete • already have all the components from past research -- see below • the visual part is new but not hard to design • components: 1. multicore traffic capture and processing 2. client-side web aggregator based on WebSockets and WebWorkers (HTLM5) 3. interactive design -- this part is pending but easy M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 15/24 ... 15/24
  15. 15. . Implementation : Multicore • heavy use of multicore and realtime processing -- recent paper in 07 Global Networks Data Center Internals Gateway Switch Capture Manager CPU CPU CPU CPU CPU CPU … Storage Mirror 07 myself+0 "A Software Design and Algorithms for Multicore Capture..." ACM ASIA-CCS-SFCS (2014) M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 16/24 ... 16/24
  16. 16. . Implementation : Sockets • WebSockets -- streaming, native binary support, heavily interactive, client-side aggregation 01 • recent study shows that 900Mbps can be supported using parallel sockets 02 • again, WebWorkers expoit multicore if present Network Chrome Browser Control …… Media requests Server Buffer Client WebSocket Apache web server WebSocket server WebWorker WebSocket WebWorker Network Server Round Robin Add 01 myself+0 "Multi-Source Stream Aggregation in the Cloud" ADCN Wiley Book (2013) 02 myself+0 "Experiments with application throughput in a browser with full HTML5 support" IEICE ComEx (2013) M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 17/24 ... 17/24
  17. 17. . Design : Capture/Processing • any level of abstraction: PMs, VMs, storage, etc. • realtime processing and decision-making -- towards interactive visualization 07 Web Worker Web Worker CPU Core(s) …. PM PM PM PM Switch To Cloud Split Merge Traffic Mirroring Interactive Display 07 myself+0 "A Software Design and Algorithms for Multicore Capture..." ACM ASIA-CCS-SFCS (2014) M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 18/24 ... 18/24
  18. 18. . Design : Interactive Display • aggregation and processing on multicore, highly interactive display • basically, a player (of performance info) plus interaction • recent study on the most cost effective visualization technologies in browser 03 Web Worker Web Worker Web Worker Web Worker Player 1 2 3 …. ….Web Socket Browser Window (GUI) 03 myself+0 "... the Most Cost Effective Method in Browser-Based Visualizations" IIEEJ (2014) M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 19/24 ... 19/24
  19. 19. . Wrapup M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 20/24 ... 20/24
  20. 20. . Wrapup • 80% complete with most important components fully available • currently testing on a small cloud at Kyutech • only 2 variants shown, but more are possible -- all based on waterpipes • important: not limited to traffic, can work on any performance data ◦ CPU, memory, storage, etc. M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 21/24 ... 21/24
  21. 21. . That’s all, thank you ... M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 22/24 ... 22/24
  22. 22. . [01] myself+0 (2013) Multi-Source Stream Aggregation in the Cloud ADCN Wiley Book [02] myself+0 (2013) Experiments with application throughput in a browser with full HTML5 support IEICE ComEx [03] myself+0 (2014) ... the Most Cost Effective Method in Browser-Based Visualizations IIEEJ [04] myself+0 (2014) A holistic... for measuring end-to-end QoS at data centres IJCSE [05] E.Tufte+0 (1983) The Visual Display of Quantitative Information Graphics Press M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 22/24 ... 22/24
  23. 23. . [06] J.Steele+1 (2010) Beautiful Visualization O'Reilly Media [07] myself+0 (2014) A Software Design and Algorithms for Multicore Capture... ACM ASIA-CCS-SFCS [08] myself+0 (current) 1100+ Examples of Information Visualization http://www.infovis.info/index.php?words=diagrams M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 23/24 ... 23/24
  24. 24. . Extras : Processing Cost Hashing Increasing Per-Unit Cost Manager Prefix Matching Cores that do not match Process Stage 1 Stage 2 Stage 3 M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 23/24 ... 23/24
  25. 25. . Extras : Multicore Processing PF_ RING PF_ RING Time Manager Shared Memory Capture Capture … Core 1 Core 2 Core 3 …. Core X M.Zhanikeev -- maratishe@gmail.com -- ...Interactive Traffic Visualizations in Data Centers -- http://bit.do/marat140523 24/24 ... 24/24
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×