The document discusses various methodologies for networking research, including measurement, experimentation, analysis, simulation, and their relationships. It notes that while measurement and experimentation deal with the real world, analysis and simulation use abstract models. Simulation is highlighted as particularly challenging for networking research due to the heterogeneous, huge scale, and constantly changing nature of the Internet.

1. Methodologies for
Networking Research
17 October 2008
CS5229, Semester 1, 2008/09
1

2. Measurementquot;
V. Paxson. quot;End-to-end Internet packet dynamics,”quot;
J. Padhye, V. Firoiu, D. Towesley, and J. Kurose quot;Modeling
TCP Throughput: A Simple Model and its Empirical
Validation,”
17 October 2008
CS5229, Semester 1, 2008/09
2

3. “Reality Check”
Are our assumptions reasonable? Is our
mathematical model a good estimation of the real
world?
17 October 2008
CS5229, Semester 1, 2008/09
3

4. e.g., from Paxson’s studyquot;
1. packet losses are busrtyquot;
2. OTT != RTT/2
17 October 2008
CS5229, Semester 1, 2008/09
4

5. 17 October 2008
CS5229, Semester 1, 2008/09
5

6. 17 October 2008
CS5229, Semester 1, 2008/09
6

7. Experimentationquot;
e.g., V. Jacobson. “Congestion Control and Avoidancequot;
17 October 2008
CS5229, Semester 1, 2008/09
7

8. Deal with implementation
issues
Sometimes unforeseen complexities (e.g. own research
experience in Unreliable TCP)
17 October 2008
CS5229, Semester 1, 2008/09
8

9. Understand the Behavior
of Systems
Some systems are too complex to understand with
“thought experiments” alone.
17 October 2008
CS5229, Semester 1, 2008/09
9

10. 17 October 2008
CS5229, Semester 1, 2008/09
10

11. 17 October 2008
CS5229, Semester 1, 2008/09
11

12. Analysisquot;
D. Chiu and R. Jain, quot;Analysis of the increase and decrease
algorithms for congestion avoidance in computer
networks,”quot;
J. Padhye, V. Firoiu, D. Towesley, and J. Kurose quot;Modeling
TCP Throughput: A Simple Model and its Empirical
Validation,”
17 October 2008
CS5229, Semester 1, 2008/09
12

13. Explore with Complete
Control
We can understand the basic forces that affect the
system. e.g. TCP throughput is inversely propotional
to √p
17 October 2008
CS5229, Semester 1, 2008/09
13

14. Simplify complex systemsquot;
If too simplified, important behavior could be missed
(TCP throughput without timeout)
17 October 2008
CS5229, Semester 1, 2008/09
14

15. Simulationquot;
K. Fall and S. Floyd, quot;Simulation-based comparison of
Tahoe, Reno, and SACK TCP,quot;quot;
S. Floyd, K. Fall, quot;Promoting the Use of End-to-End
Congestion Control in the Internet,”quot;
S. Floyd, V. Jacobson, quot;Random Early Detection
Gateways for Congestion Avoidance,quot;
17 October 2008
CS5229, Semester 1, 2008/09
15

16. Check Correctness of
Analysisquot;
If simulation uses the same assumptions/model as the
analysis, this simply verifies the correctness of the
mathematical derivations.
17 October 2008
CS5229, Semester 1, 2008/09
16

17. Check Correctness of
Analysisquot;
Simulation can relax some assumptions, use more
complex models, etc. to test the limits of analysis.
(Real measurement/experiments still needed to check
the usefulness of analysis results)
17 October 2008
CS5229, Semester 1, 2008/09
17

18. Explore Complex Systemsquot;
Some systems are too difficult/impossible to analyzed
e.g. Internet
17 October 2008
CS5229, Semester 1, 2008/09
18

19. Helps Develop Intuitionquot;
17 October 2008
CS5229, Semester 1, 2008/09
19

20. Measurementquot;
Experimentationquot; } Real World
Analysisquot;
Simulationquot; } Abstract Model
17 October 2008
CS5229, Semester 1, 2008/09
20

21. “Difficulties in Simulating
the Internet” quot;
Sally Floyd, Van Paxsonquot;
ACM/IEEE TON, 9(4) August 2001

22. Why is Internet hard to
simulate?
17 October 2008
CS5229, Semester 1, 2008/09
22

23. 1.!
Internet is diverse
17 October 2008
CS5229, Semester 1, 2008/09
23

24. End-hosts: phones,
desktops, servers, iPod, Wii
17 October 2008
CS5229, Semester 1, 2008/09
24

25. Links: Ethernet, WiFi,
Satellite, Dial-up, 3G
17 October 2008
CS5229, Semester 1, 2008/09
25

26. Transport: TCP variants,
UDP, DCCP
17 October 2008
CS5229, Semester 1, 2008/09
26

27. Applications: games,
videos, web, ftp, bittorrent
17 October 2008
CS5229, Semester 1, 2008/09
27

28. 2.!
Internet is huge
17 October 2008
CS5229, Semester 1, 2008/09
28

29. 570,937,778
Number of Hosts as of July 2008
http://www.isc.org/index.pl?/ops/ds/host-count-history.php
17 October 2008
CS5229, Semester 1, 2008/09
29

30. 3.!
Internet is changing
17 October 2008
CS5229, Semester 1, 2008/09
30

31. http://www.isc.org/ds/
17 October 2008
CS5229, Semester 1, 2008/09
31

32. 17 October 2008
CS5229, Semester 1, 2008/09
32
http://www.dtc.umn.edu/mints/

33. Median File Transfer
Time
Size
March 1998
10.9 kB
December 1998
5.6 kB
December 1999
10.9 kB
June 2000
62 kB
November 2000
10 kB
Measurement at LBNL: Statistical property
of Internet changes as well.

34. Why is Internet hard to
simulate?quot;
1. Heterogeneous quot;
2. Huge quot;
3. Changing
17 October 2008
CS5229, Semester 1, 2008/09
34

35. Suppose you come up
with the greatest
BitTorrent
improvement ever..
17 October 2008
CS5229, Semester 1, 2008/09
35

36. You want to simulate it
to make sure it works
before you release it
(and call the press)
17 October 2008
CS5229, Semester 1, 2008/09
36

37. What Internet topology
should you use in your
simulation?
How end hosts are connected? What are the
properties of the links?
17 October 2008
CS5229, Semester 1, 2008/09
37

38. Topology changes constantlyquot;
Companies keep info secretsquot;
Routes may changequot;
Routes may be asymmetric
17 October 2008
CS5229, Semester 1, 2008/09
38

39. You will need to simulate over
a wide range of connectivity
and link properties
17 October 2008
CS5229, Semester 1, 2008/09
39

40. Suppose you come up
with the greatest TCP
optimization ever..
17 October 2008
CS5229, Semester 1, 2008/09
40

41. You want to know if it
is fair to existing TCP
versions before you
write your SIGCOMM
paper..
17 October 2008
CS5229, Semester 1, 2008/09
41

42. Which TCP version to
use?
17 October 2008
CS5229, Semester 1, 2008/09
42

43. Using “fingerprinting”,
831 different TCP
implementations and
versions are identified.
17 October 2008
CS5229, Semester 1, 2008/09
43

44. Which to use? Which
to ignore?
17 October 2008
CS5229, Semester 1, 2008/09
44

45. What applications to run?
What type of traffic to
generate?quot;
Telnet? FTP? Web? BitTorrent?
Skype?
17 October 2008
CS5229, Semester 1, 2008/09
45

46. How congested should
the network be?
17 October 2008
CS5229, Semester 1, 2008/09
46

47. How congested should
the network be?
17 October 2008
CS5229, Semester 1, 2008/09
47

48. Example from Sally Floyd:
RED vs DropTail
17 October 2008
CS5229, Semester 1, 2008/09
48

49. 17 October 2008
CS5229, Semester 1, 2008/09
49

50. 17 October 2008
CS5229, Semester 1, 2008/09
50

51. Example from Sally Floyd:
Using TFRC for VoIP
17 October 2008
CS5229, Semester 1, 2008/09
51

52. 17 October 2008
CS5229, Semester 1, 2008/09
52

53. 17 October 2008
CS5229, Semester 1, 2008/09
53

54. We can focus our
simulation on dominant
technology/application
today..
17 October 2008
CS5229, Semester 1, 2008/09
54

55. TCP: NewReno SACKSquot;
OS: Windows Linuxquot;
Applications: Web, FTP
17 October 2008
CS5229, Semester 1, 2008/09
55

56. What about tomorrow?
17 October 2008
CS5229, Semester 1, 2008/09
56

57. WiMax? quot;
Sensors? quot;
Virtual World?quot;
DCCP?
17 October 2008
CS5229, Semester 1, 2008/09
57

58. 10 years ago, you came
up with a router
mechanism to improve
TCP Reno.. quot;
No one cares today.
17 October 2008
CS5229, Semester 1, 2008/09
58

59. How to verify the
simulator itself?
17 October 2008
CS5229, Semester 1, 2008/09
59

60. So, how?
17 October 2008
CS5229, Semester 1, 2008/09
60

61. Looking for
Invariants
17 October 2008
CS5229, Semester 1, 2008/09
61

62. 1. Diurnal Patterns
17 October 2008
CS5229, Semester 1, 2008/09
62

63. 17 October 2008
CS5229, Semester 1, 2008/09
63

64. hour #constrained
---- ------------
00 139 2.5% -----------------------------------------------------X
01 144 2.6% ------------------------------------------------------X
02 146 2.6% -------------------------------------------------------X
03 140 2.5% -----------------------------------------------------X
04 119 2.1% ---------------------------------------------X
05 89 1.6% ----------------------------------X
06 69 1.2% --------------------------X
07 55 1.0% ---------------------X
08 45 0.8% -----------------X
09 40 0.7% ---------------X
10 40 0.7% ---------------X
11 42 0.8% ----------------X
12 51 0.9% -------------------X
13 57 1.0% ---------------------X
14 68 1.2% --------------------------X
15 75 1.3% ----------------------------X
16 77 1.4% -----------------------------X
17 92 1.6% -----------------------------------X
18 98 1.8% -------------------------------------X
19 105 1.9% ----------------------------------------X
20 108 1.9% -----------------------------------------X
21 113 2.0% -------------------------------------------X
22 124 2.2% -----------------------------------------------X
23 134 2.4% ---------------------------------------------------X
U Waterloo Data 24 Oct 2007
17 October 2008
CS5229, Semester 1, 2008/09
64

65. 2. Self-Similar Traffic
17 October 2008
CS5229, Semester 1, 2008/09
65

66. The traffic is bursty
regardless of time scale
17 October 2008
CS5229, Semester 1, 2008/09
66

67. Wikipedia
17 October 2008
CS5229, Semester 1, 2008/09
67

68. 3. Poisson Session
Arrival
17 October 2008
CS5229, Semester 1, 2008/09
68

69. Remote logins, starting
FTP, beginning of web
surfing etc.
17 October 2008
CS5229, Semester 1, 2008/09
69

70. (so are dead light bulbs,
spelling mistakes, etc.)
17 October 2008
CS5229, Semester 1, 2008/09
70

71. 4. Log-normal Duration
17 October 2008
CS5229, Semester 1, 2008/09
71

72. 5. Heavy Tail
Distributions
17 October 2008
CS5229, Semester 1, 2008/09
72

73. Self-Similarity in World Wide Web Traffic: Evidence and
Possible Causes, by Mark E. Crovella and Azer Bestavros
17 October 2008
CS5229, Semester 1, 2008/09
73

74. 1. Looking for
Invariants
17 October 2008
CS5229, Semester 1, 2008/09
74

75. 2. Explore
Parameter Space
17 October 2008
CS5229, Semester 1, 2008/09
75

76. Change one parameter,quot;
fix the rest
17 October 2008
CS5229, Semester 1, 2008/09
76

77. Explore a wide range of values
17 October 2008
CS5229, Semester 1, 2008/09
77

78. 3. Use Traces
17 October 2008
CS5229, Semester 1, 2008/09
78

79. e.g. collects traces of web
sessions, video files, VoIP traffic
17 October 2008
CS5229, Semester 1, 2008/09
79

80. Use it to simulate the traffic
source
17 October 2008
CS5229, Semester 1, 2008/09
80

81. But must be careful about traffic
shaping and user/application
adaptation.
17 October 2008
CS5229, Semester 1, 2008/09
81

82. e.g. traces collected during non-
congested time should not be use to
simulate congested networks.
17 October 2008
CS5229, Semester 1, 2008/09
82

83. 4. publish simulator script for
others to verify
17 October 2008
CS5229, Semester 1, 2008/09
83

84. Conclusion
17 October 2008
CS5229, Semester 1, 2008/09
84

85. Simulation is useful but needs to
do it properly
17 October 2008
CS5229, Semester 1, 2008/09
85

86. Be careful about your simulation
model: you want it to be as simple
as possible, but not simpler.
17 October 2008
CS5229, Semester 1, 2008/09
86

87. Be careful about your conclusion:
“A is 13.5% better than B” is
probably useless.
17 October 2008
CS5229, Semester 1, 2008/09
87

88. “A is 13.5% better than B under
these environment”quot;
is better but not general
17 October 2008
CS5229, Semester 1, 2008/09
88

89. Not really for quantitative results,
but more for
17 October 2008
CS5229, Semester 1, 2008/09
89

90. understanding the dynamics,quot;
illustrate a point,quot;
explore unexpected behavior.
17 October 2008
CS5229, Semester 1, 2008/09
90