Lecture 10
Transport Protocol
        for
Networked Games
     29 March 2010


                     1
TCP or UDP ?



               2
Why use TCP?

• TCP provides reliable, in-order delivery
• TCP goes through most firewalls, UDP
  does not
• TCP manages co...
Why not to use TCP?

• TCP incurs higher latency
• Don’t always need reliability and in-order
  delivery
• High header ove...
position = 10
position = 13          X
position = 15

   Updated position not delivered to
 application until (outdated) l...
A’s position = 10
B’s position = 13        X
C’s position = 15


   Some messages need not be delivered in
               ...
Gestures from someone far away need
      not be received reliably.
                                      7
46%
of ShenZhou Online bandwidth is
     occupied by TCP header




                                  8
enet.cubik.org
 A library that provides
  reliability, sequencing,
connection managements
        over UDP

              ...
Delivery can be
stream-oriented (like TCP) or
message-oriented (like UDP)



                                10
Supports partial reliability

enet_packet_create (“abc”,
4, ENET_PACKET_FLAG_RELIABLE)




                               ...
Retransmission triggered by
  timeout-based on RTT




                              12
Data in queue are bundled into
 one packet if there is space




                                 13
enet.cubik.org

Portable, easy to use, but
still, most firewalls block
        UDP traffic

                             14
• Uses TCP: WoW, Lineage I/II,
 Guild Wars, Ragnarok Online,
 Anarchy Online, Mabinogi

• Uses UDP: EverQuest, SW
 Galaxie...
Need to study the use
of TCP for networked
        games


                        16
How to provide
reliability over UDP?


                        17
How slow is TCP, really?

Which part of TCP is the
root of slowness?

Can we fix TCP?

                           18
A Quick Review of TCP



                        19
s=100
s=200
s=300
s=400                     a=300

                          a=500




        TCP Delayed ACK
           ...
TCP Spec: max 500ms delay
Most implementation: 200ms



                             21
Why delay ACK?
• reduce num of ACKs
• in case receiver wants to send data
  within 200ms (in which case it can
  piggyback...
s=100
s=200
s=300                             a=200
                      X
s=400
s=500                             a=200
...
Definition of Dup ACKs
in 4.4BSD and Stevens:
“pure ACK with no data”


                          24
RTO
            X

                      X

2xRTO




   TCP Timeout + Exponential Back-off
                              ...
RTO




      Spurious Retransmission

                                26
RTO estimation

Ei = 7Ei-1/8 + RTT/8
Vi = 3Vi-1/4 + |RTT-Ei-1|/4
RTO = max(Ei + 4Vi, 1s)

                              27
Linux’s RTO estimation

Ei = 7Ei-1/8 + RTT/8
Vi = 3Vi-1/4 + |RTT-Ei-1|/4
Wi = min(Vi, 50ms)
RTO = max(200ms, Ei+Wi)
      ...
delayed ACK

increase RTT

increase RTO
               29
Congestion Control



                     30
Window Size
                     Packet Loss




                                   Time
       TCP Congestion Control

  ...
Congestion window
resets to 2 after an idle
    period (> RTO)


                            32
What does real game
 traffic look like?


                      33
low packet rate
small packet size


                    34
“Thin Streams”



                 35
36
Max Application Delay




            Max RTT


250ms
                         Avg RTT



                                ...
38
About 4 packets / sec



                        39
Average Payload:
  100 Bytes



                   40
Loss Rate 1%



               41
But some experience 6
    retransmissions


                        42
ShenZhou Online




                  43
44
45
46
“Thin Streams”



                 47
Findings 1:
Fast retransmission
  rarely triggered


                      48
In ShenZhou Online traces, fail
 to trigger fast retransmission
             because
  insufficient dup ACK (50%)
   interr...
50
51
Findings 2:
Delay due mostly to
      timeout


                      52
53
Findings 3:
Congestion window
 reset is frequent


                     54
55
12% - 18% of packets
 faces window reset


                       56
think..
think..
think..
click (tank attack here)
click (missile launch there)
click (charge soldiers)

   The last command...
How to make TCP (or,
transport protocol) go
 faster in these games?


                          58
1. Remove exponential
       backoff


                        59
TO
      X
           X

TO




      TCP Timeout
                    60
2. Make RTO Smaller



                      61
make sure minimum
  RTO is not 1s


                    62
spurious retransmission
   is not disastrous


                          63
3. Make Fast Retransmit
         Faster


                          64
s=100
s=200
s=300                           a=200
                     X
s=400
s=500                           a=200
     ...
4. Retransmission
     Bundling


                    66
X




Retransmit all unacknowledge data in queue
                                             67
5. Redundant Data
     Bundling


                    68
X
                                    a=300

                                    a=500




Send any unacknowledged segment...
6. Turn off or reduce
    Delayed ACKs


                        70
Packet interarrival time
 on average > 200ms
(can’t combine two ACKs into one)




                                    71
7. Combine Thin
Streams into Thicker
      Stream


                       72
Server




Proxy




         73
TCP for Games

• remove exponential backoff
• reduce RTO
• make fast retransmit faster
• retransmit agressively
• don’t de...
Beyond TCP and UDP?



                      75
Ideal Transport Protocol for
           Games

• optional reliability
• optional order-of-delivery
• flexibility in organiz...
SCTP
Stream Control Transport Protocol




                                    77
Multi-streaming: multiple
 independent streams




                            78
A stream can be either reliable or
          non-reliable




                                     79
Data from multiple streams can be
    bundled into one packets




                                    80
Message-oriented (like UDP)




                              81
Message can be flagged for
   unordered delivery




                            82
Ideal Transport Protocol for
           Games

• optional reliability
• optional order-of-delivery
• flexibility in organiz...
SCTP for Games?



                  84
States of SCTP



• rumored to be in Vista
• available in FreeBSD 7 and Solaris 10
• lksctp project for Linux


          ...
Upcoming SlideShare
Loading in …5
×

CS4344 09/10 Lecture 10: Transport Protocol for Networked Games

1,489 views
1,355 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,489
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CS4344 09/10 Lecture 10: Transport Protocol for Networked Games

  1. 1. Lecture 10 Transport Protocol for Networked Games 29 March 2010 1
  2. 2. TCP or UDP ? 2
  3. 3. Why use TCP? • TCP provides reliable, in-order delivery • TCP goes through most firewalls, UDP does not • TCP manages connection for us 3
  4. 4. Why not to use TCP? • TCP incurs higher latency • Don’t always need reliability and in-order delivery • High header overhead 4
  5. 5. position = 10 position = 13 X position = 15 Updated position not delivered to application until (outdated) lost packet is received 5
  6. 6. A’s position = 10 B’s position = 13 X C’s position = 15 Some messages need not be delivered in sequence. 6
  7. 7. Gestures from someone far away need not be received reliably. 7
  8. 8. 46% of ShenZhou Online bandwidth is occupied by TCP header 8
  9. 9. enet.cubik.org A library that provides reliability, sequencing, connection managements over UDP 9
  10. 10. Delivery can be stream-oriented (like TCP) or message-oriented (like UDP) 10
  11. 11. Supports partial reliability enet_packet_create (“abc”, 4, ENET_PACKET_FLAG_RELIABLE) 11
  12. 12. Retransmission triggered by timeout-based on RTT 12
  13. 13. Data in queue are bundled into one packet if there is space 13
  14. 14. enet.cubik.org Portable, easy to use, but still, most firewalls block UDP traffic 14
  15. 15. • Uses TCP: WoW, Lineage I/II, Guild Wars, Ragnarok Online, Anarchy Online, Mabinogi • Uses UDP: EverQuest, SW Galaxies, City of Heroes, Ultima Online, Asherons Call, FFXI 15
  16. 16. Need to study the use of TCP for networked games 16
  17. 17. How to provide reliability over UDP? 17
  18. 18. How slow is TCP, really? Which part of TCP is the root of slowness? Can we fix TCP? 18
  19. 19. A Quick Review of TCP 19
  20. 20. s=100 s=200 s=300 s=400 a=300 a=500 TCP Delayed ACK 20
  21. 21. TCP Spec: max 500ms delay Most implementation: 200ms 21
  22. 22. Why delay ACK? • reduce num of ACKs • in case receiver wants to send data within 200ms (in which case it can piggyback the ACK with data) • give sender time to buffer more data for sending (avoid silly window syndrome) 22
  23. 23. s=100 s=200 s=300 a=200 X s=400 s=500 a=200 a=200 a=200 s=200 TCP Fast Retransmission 23
  24. 24. Definition of Dup ACKs in 4.4BSD and Stevens: “pure ACK with no data” 24
  25. 25. RTO X X 2xRTO TCP Timeout + Exponential Back-off 25
  26. 26. RTO Spurious Retransmission 26
  27. 27. RTO estimation Ei = 7Ei-1/8 + RTT/8 Vi = 3Vi-1/4 + |RTT-Ei-1|/4 RTO = max(Ei + 4Vi, 1s) 27
  28. 28. Linux’s RTO estimation Ei = 7Ei-1/8 + RTT/8 Vi = 3Vi-1/4 + |RTT-Ei-1|/4 Wi = min(Vi, 50ms) RTO = max(200ms, Ei+Wi) 28
  29. 29. delayed ACK increase RTT increase RTO 29
  30. 30. Congestion Control 30
  31. 31. Window Size Packet Loss Time TCP Congestion Control 31
  32. 32. Congestion window resets to 2 after an idle period (> RTO) 32
  33. 33. What does real game traffic look like? 33
  34. 34. low packet rate small packet size 34
  35. 35. “Thin Streams” 35
  36. 36. 36
  37. 37. Max Application Delay Max RTT 250ms Avg RTT 37
  38. 38. 38
  39. 39. About 4 packets / sec 39
  40. 40. Average Payload: 100 Bytes 40
  41. 41. Loss Rate 1% 41
  42. 42. But some experience 6 retransmissions 42
  43. 43. ShenZhou Online 43
  44. 44. 44
  45. 45. 45
  46. 46. 46
  47. 47. “Thin Streams” 47
  48. 48. Findings 1: Fast retransmission rarely triggered 48
  49. 49. In ShenZhou Online traces, fail to trigger fast retransmission because insufficient dup ACK (50%) interrupted by data (50%) 49
  50. 50. 50
  51. 51. 51
  52. 52. Findings 2: Delay due mostly to timeout 52
  53. 53. 53
  54. 54. Findings 3: Congestion window reset is frequent 54
  55. 55. 55
  56. 56. 12% - 18% of packets faces window reset 56
  57. 57. think.. think.. think.. click (tank attack here) click (missile launch there) click (charge soldiers) The last command is delayed as congestion window = 2 57
  58. 58. How to make TCP (or, transport protocol) go faster in these games? 58
  59. 59. 1. Remove exponential backoff 59
  60. 60. TO X X TO TCP Timeout 60
  61. 61. 2. Make RTO Smaller 61
  62. 62. make sure minimum RTO is not 1s 62
  63. 63. spurious retransmission is not disastrous 63
  64. 64. 3. Make Fast Retransmit Faster 64
  65. 65. s=100 s=200 s=300 a=200 X s=400 s=500 a=200 a=200 a=200 s=200 Retransmit after one duplicate ACK 65
  66. 66. 4. Retransmission Bundling 66
  67. 67. X Retransmit all unacknowledge data in queue 67
  68. 68. 5. Redundant Data Bundling 68
  69. 69. X a=300 a=500 Send any unacknowledged segment in queue as long as there is space. Lost data gets recovered in the next transmission before retransmission. 69
  70. 70. 6. Turn off or reduce Delayed ACKs 70
  71. 71. Packet interarrival time on average > 200ms (can’t combine two ACKs into one) 71
  72. 72. 7. Combine Thin Streams into Thicker Stream 72
  73. 73. Server Proxy 73
  74. 74. TCP for Games • remove exponential backoff • reduce RTO • make fast retransmit faster • retransmit agressively • don’t delay ACK • combine into thick streams 74
  75. 75. Beyond TCP and UDP? 75
  76. 76. Ideal Transport Protocol for Games • optional reliability • optional order-of-delivery • flexibility in organizing messages into different classes with different requirements 76
  77. 77. SCTP Stream Control Transport Protocol 77
  78. 78. Multi-streaming: multiple independent streams 78
  79. 79. A stream can be either reliable or non-reliable 79
  80. 80. Data from multiple streams can be bundled into one packets 80
  81. 81. Message-oriented (like UDP) 81
  82. 82. Message can be flagged for unordered delivery 82
  83. 83. Ideal Transport Protocol for Games • optional reliability • optional order-of-delivery • flexibility in organizing messages into different classes with different requirements 83
  84. 84. SCTP for Games? 84
  85. 85. States of SCTP • rumored to be in Vista • available in FreeBSD 7 and Solaris 10 • lksctp project for Linux 85

×