50° Congresso Nazionale AICA

Visualizing Internet-Measurements Data for
Research Purposes:
the NeuViz Data Visualization ...
Introduction
●

Our objective:
●

●

As a Center for Internet & Society we study the net neutrality with a
multidisciplina...
NeuViz Architecture (I)

19th September 2013

Giuseppe Futia – Politecnico di Torino

3
NeuViz Architecture (I - a)
●

The Producers:
●

●

●

PULL static datasets from Neubot and other projects expressed in
di...
NeuViz Architecture (II)

19th September 2013

Giuseppe Futia – Politecnico di Torino

5
NeuViz Architecture (II - a)
●

The Importer Stage:
●

●

●

Written in Python, it is composed of many modules to import d...
NeuViz Architecture (II - b)
●

The Analysis Stage:
●

●

●

●

Written in Python, it is a collection of modules to produc...
Dimensions of Data

19th September 2013

Giuseppe Futia – Politecnico di Torino

8
Number of tests in Canada

19th September 2013

Giuseppe Futia – Politecnico di Torino

9
Median upload difference

19th September 2013

Giuseppe Futia – Politecnico di Torino

10
A case to be investigated...

Empirical Probability Density Function (PDF) of the difference
of the median upload speed of...
Comparison of Speedtest and
BitTorrent performance
In 2013 the median upload difference between Speedtest and
BitTorrent i...
Conclusions and Future Works
●

●

●

NeuViz proved to be a very useful tool to identify cases in which a
protocol seems d...
Thank you
Giuseppe Futia (giuseppe.futia@polito.it)
Nexa Center for Internet & Society (http://nexa.polito.it)
Dept. of Co...
Upcoming SlideShare
Loading in …5
×

Visualizing Internet-Measurements Data for Research Purposes: the NeuViz Data Visualization Tool

16,628 views

Published on

NeuViz, a data processing and visualization architecture for network measurement experiments.

Presented at "50° Congresso Nazionale AICA" - Fisciano (SA), 19th September 2013

Published in: Sports, Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
16,628
On SlideShare
0
From Embeds
0
Number of Embeds
15,973
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Visualizing Internet-Measurements Data for Research Purposes: the NeuViz Data Visualization Tool

  1. 1. 50° Congresso Nazionale AICA Visualizing Internet-Measurements Data for Research Purposes: the NeuViz Data Visualization Tool G. Futia* E. Zimuel+ S. Basso* J.C. De Martin* * Nexa Center for Internet & Society, Politecnico di Torino (DAUIN) + R&D Department, Zend Technologies Inc Fisciano (SA), 19th September 2013 19th September 2013 Giuseppe Futia – Politecnico di Torino 1
  2. 2. Introduction ● Our objective: ● ● As a Center for Internet & Society we study the net neutrality with a multidisciplinary approach What we have: ● Neubot (the Network neutrality bot), a free-software Internet bot that performs active, lightweight network-performance tests: – – BitTorrent (BitTorrent protocol) – ● Speedtest (HTTP protocol) And other tests What we need: ● NeuViz (the Neubot Visualizer): a data processing and visualization architecture for network measurement experiments to detect cases in which a protocol seems discriminated 19th September 2013 Giuseppe Futia – Politecnico di Torino 2
  3. 3. NeuViz Architecture (I) 19th September 2013 Giuseppe Futia – Politecnico di Torino 3
  4. 4. NeuViz Architecture (I - a) ● The Producers: ● ● ● PULL static datasets from Neubot and other projects expressed in different format (CSV or JSON) PUSH (via API) the results of the experiments of network-measurement tools just after the experiments are run The Frontend and the Consumers: ● ● The Web API connects the Frontend and the Backend: the default Consumer is a Web browser that exploits the NeuViz Web interface (developed using the D3.js JavaScript library) Feedback mechanism: we plan to extend the Neubot Master Server to fetch data from NeuViz to provide better suggestions to Neubot instances 19th September 2013 Giuseppe Futia – Politecnico di Torino 4
  5. 5. NeuViz Architecture (II) 19th September 2013 Giuseppe Futia – Politecnico di Torino 5
  6. 6. NeuViz Architecture (II - a) ● The Importer Stage: ● ● ● Written in Python, it is composed of many modules to import data from a specific network-measurement tool and format (also via a Submit API) It converts the input data into a JSON document and it adds geolocation if needed (GeoLite Database) The Raw Database: ● ● It receives heterogeneous data organized in a uniform format by the Importer Stage Implemented using MongoDB: it is not practical to reduce all the input data to the same schema, because each network experiment saves different metrics 19th September 2013 Giuseppe Futia – Politecnico di Torino 6
  7. 7. NeuViz Architecture (II - b) ● The Analysis Stage: ● ● ● ● Written in Python, it is a collection of modules to produce aggregate data for the visualization For both tests it computes the median number of different metrics, for each hour of the day, for each month, along geographical (country, cities) and business dimensions (ISPs) Scalability: we designed the code to exploit Map/Reduce techniques on cloud services The Analysis Database: ● ● Conceptually-separated databases to store data ready to use (or to visualize) Implemented with MongoDB: useful for managing the results of complex analysis that require long time to process 19th September 2013 Giuseppe Futia – Politecnico di Torino 7
  8. 8. Dimensions of Data 19th September 2013 Giuseppe Futia – Politecnico di Torino 8
  9. 9. Number of tests in Canada 19th September 2013 Giuseppe Futia – Politecnico di Torino 9
  10. 10. Median upload difference 19th September 2013 Giuseppe Futia – Politecnico di Torino 10
  11. 11. A case to be investigated... Empirical Probability Density Function (PDF) of the difference of the median upload speed of US and Canada in 2013 19th September 2013 Giuseppe Futia – Politecnico di Torino 11
  12. 12. Comparison of Speedtest and BitTorrent performance In 2013 the median upload difference between Speedtest and BitTorrent in Canada was very often positive, while the same difference was very often negative in the US ● ● ● the BitTorrent protocol seems to be discriminated in Canada, but we need to gather more information we have, in fact, to investigate other possible causes of the observed bias, including the implementation of both tests an advanced Master Server could learn about similar anomalies and ask Neubot instances to conduct detailed experiments 19th September 2013 Giuseppe Futia – Politecnico di Torino 12
  13. 13. Conclusions and Future Works ● ● ● NeuViz proved to be a very useful tool to identify cases in which a protocol seems discriminated, to be investigated with more specific network tests We plan to extend NeuViz to automatically raise warnings and to cooperate with the Neubot Master Server to trigger more-specific network experiments We envision to integrate social networks to have an overview on net neutrality discussions 19th September 2013 Giuseppe Futia – Politecnico di Torino 13
  14. 14. Thank you Giuseppe Futia (giuseppe.futia@polito.it) Nexa Center for Internet & Society (http://nexa.polito.it) Dept. of Computer and Control Engineering (DAUIN) Politecnico di Torino, Italy The NeuViz git repository is available at: http://github.com/neubot/neuviz/ 19th September 2013 Giuseppe Futia – Politecnico di Torino 14

×