1. Lecture 10
Transport Protocol
for
Networked Games
29 March 2010
1

2. TCP or UDP ?
2

3. Why use TCP?
• TCP provides reliable, in-order delivery
• TCP goes through most firewalls, UDP
does not
• TCP manages connection for us
3

4. Why not to use TCP?
• TCP incurs higher latency
• Don’t always need reliability and in-order
delivery
• High header overhead
4

5. position = 10
position = 13 X
position = 15
Updated position not delivered to
application until (outdated) lost packet
is received
5

6. A’s position = 10
B’s position = 13 X
C’s position = 15
Some messages need not be delivered in
sequence.
6

7. Gestures from someone far away need
not be received reliably.
7

8. 46%
of ShenZhou Online bandwidth is
occupied by TCP header
8

9. enet.cubik.org
A library that provides
reliability, sequencing,
connection managements
over UDP
9

10. Delivery can be
stream-oriented (like TCP) or
message-oriented (like UDP)
10

11. Supports partial reliability
enet_packet_create (“abc”,
4, ENET_PACKET_FLAG_RELIABLE)
11

12. Retransmission triggered by
timeout-based on RTT
12

13. Data in queue are bundled into
one packet if there is space
13

14. enet.cubik.org
Portable, easy to use, but
still, most firewalls block
UDP traffic
14

15. • Uses TCP: WoW, Lineage I/II,
Guild Wars, Ragnarok Online,
Anarchy Online, Mabinogi
• Uses UDP: EverQuest, SW
Galaxies, City of Heroes, Ultima
Online, Asherons Call, FFXI
15

16. Need to study the use
of TCP for networked
games
16

17. How to provide
reliability over UDP?
17

18. How slow is TCP, really?
Which part of TCP is the
root of slowness?
Can we fix TCP?
18

19. A Quick Review of TCP
19

20. s=100
s=200
s=300
s=400 a=300
a=500
TCP Delayed ACK
20

21. TCP Spec: max 500ms delay
Most implementation: 200ms
21

22. Why delay ACK?
• reduce num of ACKs
• in case receiver wants to send data
within 200ms (in which case it can
piggyback the ACK with data)
• give sender time to buffer more data
for sending (avoid silly window
syndrome)
22

23. s=100
s=200
s=300 a=200
X
s=400
s=500 a=200
a=200
a=200
s=200
TCP Fast Retransmission
23

24. Definition of Dup ACKs
in 4.4BSD and Stevens:
“pure ACK with no data”
24

25. RTO
X
X
2xRTO
TCP Timeout + Exponential Back-off
25

26. RTO
Spurious Retransmission
26

27. RTO estimation
Ei = 7Ei-1/8 + RTT/8
Vi = 3Vi-1/4 + |RTT-Ei-1|/4
RTO = max(Ei + 4Vi, 1s)
27

28. Linux’s RTO estimation
Ei = 7Ei-1/8 + RTT/8
Vi = 3Vi-1/4 + |RTT-Ei-1|/4
Wi = min(Vi, 50ms)
RTO = max(200ms, Ei+Wi)
28

29. delayed ACK
increase RTT
increase RTO
29

30. Congestion Control
30

31. Window Size
Packet Loss
Time
TCP Congestion Control
31

32. Congestion window
resets to 2 after an idle
period (> RTO)
32

33. What does real game
traffic look like?
33

34. low packet rate
small packet size
34

35. “Thin Streams”
35

36. 36

37. Max Application Delay
Max RTT
250ms
Avg RTT
37

38. 38

39. About 4 packets / sec
39

40. Average Payload:
100 Bytes
40

41. Loss Rate 1%
41

42. But some experience 6
retransmissions
42

43. ShenZhou Online
43

44. 44

45. 45

46. 46

47. “Thin Streams”
47

48. Findings 1:
Fast retransmission
rarely triggered
48

49. In ShenZhou Online traces, fail
to trigger fast retransmission
because
insufficient dup ACK (50%)
interrupted by data (50%)
49

50. 50

51. 51

52. Findings 2:
Delay due mostly to
timeout
52

53. 53

54. Findings 3:
Congestion window
reset is frequent
54

55. 55

56. 12% - 18% of packets
faces window reset
56

57. think..
think..
think..
click (tank attack here)
click (missile launch there)
click (charge soldiers)
The last command is delayed as congestion
window = 2
57

58. How to make TCP (or,
transport protocol) go
faster in these games?
58

59. 1. Remove exponential
backoff
59

60. TO
X
X
TO
TCP Timeout
60

61. 2. Make RTO Smaller
61

62. make sure minimum
RTO is not 1s
62

63. spurious retransmission
is not disastrous
63

64. 3. Make Fast Retransmit
Faster
64

65. s=100
s=200
s=300 a=200
X
s=400
s=500 a=200
a=200
a=200
s=200
Retransmit after one duplicate ACK
65

66. 4. Retransmission
Bundling
66

67. X
Retransmit all unacknowledge data in queue
67

68. 5. Redundant Data
Bundling
68

69. X
a=300
a=500
Send any unacknowledged segment in queue
as long as there is space. Lost data gets
recovered in the next transmission before
retransmission.
69

70. 6. Turn off or reduce
Delayed ACKs
70

71. Packet interarrival time
on average > 200ms
(can’t combine two ACKs into one)
71

72. 7. Combine Thin
Streams into Thicker
Stream
72

73. Server
Proxy
73

74. TCP for Games
• remove exponential backoff
• reduce RTO
• make fast retransmit faster
• retransmit agressively
• don’t delay ACK
• combine into thick streams
74

75. Beyond TCP and UDP?
75

76. Ideal Transport Protocol for
Games
• optional reliability
• optional order-of-delivery
• flexibility in organizing messages into
different classes with different
requirements
76

77. SCTP
Stream Control Transport Protocol
77

78. Multi-streaming: multiple
independent streams
78

79. A stream can be either reliable or
non-reliable
79

80. Data from multiple streams can be
bundled into one packets
80

81. Message-oriented (like UDP)
81

82. Message can be flagged for
unordered delivery
82

83. Ideal Transport Protocol for
Games
• optional reliability
• optional order-of-delivery
• flexibility in organizing messages into
different classes with different
requirements
83

84. SCTP for Games?
84

85. States of SCTP
• rumored to be in Vista
• available in FreeBSD 7 and Solaris 10
• lksctp project for Linux
85