Monitoring and mining network traffic in clouds

16,357 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
16,357
On SlideShare
0
From Embeds
0
Number of Embeds
13,227
Actions
Shares
0
Downloads
62
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Monitoring and mining network traffic in clouds

  1. 1. Monitoring and MiningNetwork Traffic in Clouds Edward J. Yoon <ewardyoon@apache.org>
  2. 2. Contents• Introduction• Analysis for Target Marketing – Overview of Proposed Methodology – Network Usage Trend Analysis – Customer Similarity Analysis – Abnormal Traffic Analysis• Determining the System Expansions
  3. 3. IntroductionWhy do we need monitoring and mining the network traffics?• Business points: – Target marketing – Detecting abnormal traffic• System management points: – Determining the system expansions – Network planning and design
  4. 4. Analysis for Target MarketingTraffic is most important factor of finding ideal target customer:• Network Usage Trend Analysis • who grows faster in particular time period?• Customer Similarity Analysis • who shows a similar pattern of evolution?
  5. 5. Overview of Proposed System - Network Usage Trend Analysis - Customer Similarity Analysis - Abnormal Traffic Analysis - … Analysis Map/Reduce Hama BSP NetFlow Database on Hadoop
  6. 6. Network Usage Trend Analysis• Methodology: Traffic Growth Rate (value at end of period - value at beginning of period) PR  *100 value at beginning of period
  7. 7. Customer Similarity Analysis• Methodology: Vector Space Model – Growth Rate Vector (or, Payload Vector, ...) – Cosine Similarity • The most common similarity metric in NLP
  8. 8. Abnormal Traffic Analysis• Methodology: Entropy, Chi-Square Test, .. – Entropy Analysis • E.g., measure of source IP disorder n H    Pi log2 Pi i 1
  9. 9. Determining the System Expansions• System Expansions Planning• Network Planning and Design• .. and so on.
  10. 10. What’s Next?• VM re-placement based on Resource Usage Pattern.VM1 VM9 … VM2 VM4VM2 VM5 VM7VM3 VM1 VM3 VM2 Replacement t
  11. 11. Thanks 

×