TCP PerformanceFor Mobile Applications     Vladimir Kirillov        @darkproger
Networking Stack
Application SessionTransport NetworkData Link
Application     HTTP Session        TLSTransport       TCP Network         IP              WiFi EdgeData Link             ...
Application     HTTP        NSURLConnection Session        TLS             OpenSSLTransport       TCP           SOCK_STREA...
HTTP        NSURLConnection      ptrace  TLS             OpenSSL          socket                                          ...
capturing iPhone traffic    % udid=$(system_profiler               SPUSBDataType                              | awk /iPhon...
tcpdump -i lo0 -w t.pcap -s0 &nc -l 5000 &echo hello | nc localhost 5000kill %1
# tcpdump -r t.pcap -nnvv -tttt -K tcp port 50002012-11-24 12:23:35.511134   IP6   (hlim 64, next-header TCP (6) payload l...
# tcpdump -r t.pcap -nnvv -tttt -K tcp port 50002012-11-24 12:23:35.511134   IP6   (hlim 64, next-header TCP (6) payload l...
% tcptrace t.pcap17 packets seen, 17 TCP packets tracedelapsed wallclock time: 0:00:00.001344, 12648 pkts/sec analyzedtrac...
% tcptrace -o2 -l t.pcap...      adv wind scale:          4            adv wind scale:          4      req sack:          ...
endpoint       endpoint  SO_RCVBUF   SO_SNDBUF  SO_SNDBUF   SO_RCVBUF
endpoint                   endpoint  SO_RCVBUF   SEG         SO_SNDBUF  SO_SNDBUF         SEG   SO_RCVBUF
2 * LATENCY = RTTendpoint        LATENCY                 endpoint  SO_RCVBUF   SEG                   SO_SNDBUF            ...
• Time from one endpoint to another          • Each connection spans multiple links           • edge           • ethernetL...
• Number of bytes a link can handleBandwidth   • bw = min (bw foreach link)
Bandwidth    Delay   BDP = RTT * BANDWIDTH  Product
client                           server SO_RCVBUF   receiver window   SO_SNDBUF SO_SNDBUF   sender window     SO_RCVBUF
client                                       server SO_RCVBUF   SEG   receiver window   SEG   SO_SNDBUF SO_SNDBUF         ...
client                                          server SO_RCVBUF   SEG   receiver window                   SEG  SEG  SEG S...
TCP    • stateful          • ordered   byte   • reliablestream    • managed
HTTP pairedhas state            TLSno state            TCP            IP
% openssl s_client -      showcerts -connect      google.com:443      CONNECTED(00000003)      depth=1 /C=US/O=Google Inc/...
% openssl s_client -showcerts -      connect ciklum.com:443      CONNECTED(00000003)                                      ...
HTTP       1 RTT      TLS       3 RTTs      TCP       1 RTT       IPIt takes 4 RTTs to serve a HTTPS request
HTTP       1 RTT TLS      3 RTTs TCP      1 RTT IPIt takes 4 RTTs to serve a HTTPS request
TCP Reliability
client                                             server SO_RCVBUF   SEG       receiver window                       SEG ...
AirPort        router        routerclient       Express                                  server SO_RCVBUF    SEG       rec...
AirPort          router         routerclient       Express                                     server SO_RCVBUF    SEG    ...
TCP Congestion Control
client                                        server    SEG     SEG    SEG receiver      SEGSO_RCVBUF SEG SEG SEG SEG     ...
AirPort        router        routerclient       Express                                  server SO_RCVBUF    SEG       rec...
AirPort    router        routerclient                       Express                              server             S   S ...
• Nagle algorithm              while (1)                 write(fd, “5”, 1);    TCP       (telnet syndrom)Artifacts        ...
TCP     •SO_OOBINLINEArtifacts   •TCP URG
API Issues
Async NSURLConnectionAPI Issues   UIScrollView             CFRunLoopAddCommonMode
client                               server             SEG     SEG      SEG       SEG     SEG         SO_RCVBUFSO_RCVBUF ...
More • Additive IncreaseCongestion • Multiplicative Decrease  Control • Slow Start
• setsockopt(2)              • adjust window size              • socket buffer sizes              • TCP_NODELAYiOS / Unix ...
• Responsive UI          • Bad example: InstagramSpeedup   • Bad example:VK           • best UI           • worst reliabil...
• TCP Fast Open           • Linux 3.6           • HAProxy          • TCP/NCSpeedup    • TCP and math (maths)           • h...
• TLS False Start          • TLS NPNSpeedup    • Next Protocol Negotiation          • HTTP Pipelining          • SPDY
kthxbai    @darkprogerhttp://kirillov.im
Владимир Кириллов-TCP-Performance for-Mobile-Applications
Upcoming SlideShare
Loading in …5
×

Владимир Кириллов-TCP-Performance for-Mobile-Applications

434 views
364 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
434
On SlideShare
0
From Embeds
0
Number of Embeds
70
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Владимир Кириллов-TCP-Performance for-Mobile-Applications

  1. 1. TCP PerformanceFor Mobile Applications Vladimir Kirillov @darkproger
  2. 2. Networking Stack
  3. 3. Application SessionTransport NetworkData Link
  4. 4. Application HTTP Session TLSTransport TCP Network IP WiFi EdgeData Link 3G LTE
  5. 5. Application HTTP NSURLConnection Session TLS OpenSSLTransport TCP SOCK_STREAM Network IP kernel WiFi EdgeData Link hardware 3G LTE Level Protocol API / Implementation
  6. 6. HTTP NSURLConnection ptrace TLS OpenSSL socket dtrace API TCP SOCK_STREAM IP kernel bpf(4)WiFi Edge hardware 3G LTEProtocol API / Implementation Introspection
  7. 7. capturing iPhone traffic % udid=$(system_profiler SPUSBDataType | awk /iPhone/{go=1} /Serial/ {if (go) print $3; go=0})276cb9530201bcehelloworldcd55560ed015d00% rvictl -s $udidStarting device 276cb9530201bcehelloworldcd55560ed015d00[SUCCEEDED]% ifconfig rvi0rvi0: flags=3005<UP,DEBUG,LINK0,LINK1> mtu 0
  8. 8. tcpdump -i lo0 -w t.pcap -s0 &nc -l 5000 &echo hello | nc localhost 5000kill %1
  9. 9. # tcpdump -r t.pcap -nnvv -tttt -K tcp port 50002012-11-24 12:23:35.511134 IP6 (hlim 64, next-header TCP (6) payload length: 44) ::1.51734 > ::1.5000: Flags [ S ], seq 453038127, win65535, options [mss 16324,nop,wscale 4,nop,nop,TS val 303407352 ecr 0,sackOK,eol], length 02012-11-24 12:23:35.511175 IP6 (hlim 64, next-header TCP (6) payload length: 20) ::1.5000 > ::1.51734: Flags [ R .], seq 0, ack 453038128,win 0, length 02012-11-24 12:23:35.511226 IP (tos 0x0, ttl 64, id 8400, offset 0, flags [DF], proto TCP (6), length 64) S 127.0.0.1.51735 > 127.0.0.1.5000: Flags [ ], seq 2527137802, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 303407352 ecr0,sackOK,eol], length 02012-11-24 12:23:35.511276 IP (tos 0x0, ttl 64, id 58311, offset 0, flags [DF], proto TCP (6), length 64) 127.0.0.1.5000 > 127.0.0.1.51735: Flags S [ .], seq 494520280, ack 2527137803, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val303407352 ecr 303407352,sackOK,eol], length 02012-11-24 12:23:35.511287 IP (tos 0x0, ttl 64, id 47796, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511298 IP (tos 0x0, ttl 64, id 52186, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511332 IP (tos 0x0, ttl 64, id 31417, offset 0, flags [DF], proto TCP (6), length 58) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [P .], seq 1:7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 62012-11-24 12:23:35.511351 IP (tos 0x0, ttl 64, id 29060, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511354 IP (tos 0x0, ttl 64, id 4019, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 7, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511367 IP (tos 0x0, ttl 64, id 20879, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 17 packets captured2012-11-24 12:23:35.511378 IP (tos 0x0, ttl 64, id 59633, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511388 IP (tos 0x0, ttl 64, id 56794, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [F.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0
  10. 10. # tcpdump -r t.pcap -nnvv -tttt -K tcp port 50002012-11-24 12:23:35.511134 IP6 (hlim 64, next-header TCP (6) payload length: 44) ::1.51734 > ::1.5000: Flags [ S ], seq 453038127, win65535, options [mss 16324,nop,wscale 4,nop,nop,TS val 303407352 ecr 0,sackOK,eol], length 02012-11-24 12:23:35.511175 IP6 (hlim 64, next-header TCP (6) payload length: 20) ::1.5000 > ::1.51734: Flags [ R .], seq 0, ack 453038128,win 0, length 02012-11-24 12:23:35.511226 IP (tos 0x0, ttl 64, id 8400, offset 0, flags [DF], proto TCP (6), length 64) S 127.0.0.1.51735 > 127.0.0.1.5000: Flags [ ], seq 2527137802, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 303407352 ecr0,sackOK,eol], length 02012-11-24 12:23:35.511276 IP (tos 0x0, ttl 64, id 58311, offset 0, flags [DF], proto TCP (6), length 64) 127.0.0.1.5000 > 127.0.0.1.51735: Flags S [ .], seq 494520280, ack 2527137803, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val303407352 ecr 303407352,sackOK,eol], length 02012-11-24 12:23:35.511287 IP (tos 0x0, ttl 64, id 47796, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511298 IP (tos 0x0, ttl 64, id 52186, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511332 IP (tos 0x0, ttl 64, id 31417, offset 0, flags [DF], proto TCP (6), length 58) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [P length 6 .], seq 1:7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352],2012-11-24 12:23:35.511351 IP (tos 0x0, ttl 64, id 29060, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags2012-11-24 12:23:35.511354 IP (tos 0x0, ttl 127.0.0.1.5000 > 127.0.0.1.51735: Flags 64, id 4019, offset 0, flags [DF], proto TCP (6), length 52) ^^ [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 [.], seq 1, ack 7, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511367 IP (tos 0x0, ttl 127.0.0.1.5000 > 127.0.0.1.51735: Flags % stat -f %z t.pcap 64, id 20879, offset 0, flags [DF], proto TCP (6), length 52) [.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 17 packets captured 13062012-11-24 12:23:35.511378 IP (tos 0x0, ttl 64, id 59633, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511388 IP (tos 0x0, ttl 64, id 56794, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [F.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0
  11. 11. % tcptrace t.pcap17 packets seen, 17 TCP packets tracedelapsed wallclock time: 0:00:00.001344, 12648 pkts/sec analyzedtrace file elapsed time: 0:00:00.000305TCP connection info:1: localhost:52132 - localhost:5000 (a2b) 1> 1< (reset)2: localhost:52133 - localhost:5000 (c2d) 8> 7< (complete) (reset)
  12. 12. % tcptrace -o2 -l t.pcap... adv wind scale: 4 adv wind scale: 4 req sack: Y req sack: Y sacks sent: 0 sacks sent: 0 urgent data pkts: 0 pkts urgent data pkts: 0 pkts urgent data bytes: 0 bytes urgent data bytes: 0 bytes mss requested: 16344 bytes mss requested: 16344 bytes max segm size: 6 bytes max segm size: 0 bytes min segm size: 6 bytes min segm size: 0 bytes avg segm size: 5 bytes avg segm size: 0 bytes max win adv:146976 bytes max win adv: 146976 bytes min win adv:146976 bytes min win adv: 146976 bytes zero win adv: 0 times zero win adv: 0 times avg win adv: 146976 bytes avg win adv: 122480 bytes initial window:6 bytes initial window: 0 bytes initial window: 1 pkts initial window: 0 pkts ttl stream length: 6 bytes ttl stream length: 1 bytes missed data: 0 bytes missed data: 1 bytes truncated data: 0 bytes truncated data: 0 bytes truncated packets: 0 pkts truncated packets: 0 pkts data xmit time: 0.000 secs data xmit time: 0.000 secs idletime max: 0.1 ms idletime max: 0.0 ms throughput: 27027 Bps throughput: 0 Bps
  13. 13. endpoint endpoint SO_RCVBUF SO_SNDBUF SO_SNDBUF SO_RCVBUF
  14. 14. endpoint endpoint SO_RCVBUF SEG SO_SNDBUF SO_SNDBUF SEG SO_RCVBUF
  15. 15. 2 * LATENCY = RTTendpoint LATENCY endpoint SO_RCVBUF SEG SO_SNDBUF BANDWIDTH SO_SNDBUF SEG SO_RCVBUF
  16. 16. • Time from one endpoint to another • Each connection spans multiple links • edge • ethernetLatency • fiber • wifi • latency = sum (lat foreach link) • RTT = 2 * latency
  17. 17. • Number of bytes a link can handleBandwidth • bw = min (bw foreach link)
  18. 18. Bandwidth Delay BDP = RTT * BANDWIDTH Product
  19. 19. client server SO_RCVBUF receiver window SO_SNDBUF SO_SNDBUF sender window SO_RCVBUF
  20. 20. client server SO_RCVBUF SEG receiver window SEG SO_SNDBUF SO_SNDBUF sender window SEG SEG SO_RCVBUF
  21. 21. client server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF SO_SNDBUF SEG receiver windowSEG sender window SEG SEG SEG SEG SEG SEG SO_RCVBUF
  22. 22. TCP • stateful • ordered byte • reliablestream • managed
  23. 23. HTTP pairedhas state TLSno state TCP IP
  24. 24. % openssl s_client - showcerts -connect google.com:443 CONNECTED(00000003) depth=1 /C=US/O=Google Inc/CN=Google Internet Authority verify error:num=20:unable to get local issuer certificateTLS verify return:0 --- Certificate chain 0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/ CN=*.google.com i:/C=US/O=Google Inc/CN=Google Internet Authority ... 1 s:/C=US/O=Google Inc/CN=Google Internet Authority i:/C=US/O=Equifax/OU=Equifax Secure Certificate Authority ... SSL handshake has read 2388 bytes and written 316 bytes
  25. 25. % openssl s_client -showcerts - connect ciklum.com:443 CONNECTED(00000003) ciklum.net depth=0 /C=UA/OU=Domain Control Validated/CN=*. verify error:num=20:unable to get local issuer certificate verify return:1 lol :) depth=0 /C=UA/OU=Domain Control Validated/CN=*.ciklum.net verify error:num=27:certificate not trusted verify return:1TLS depth=0 /C=UA/OU=Domain Control Validated/CN=*.ciklum.net verify error:num=21:unable to verify the first certificate verify return:1 --- Certificate chain 0 s:/C=UA/OU=Domain Control Validated/CN=*.ciklum.net i:/O=AlphaSSL/CN=AlphaSSL CA - G2 ... Server certificate subject=/C=UA/OU=Domain Control Validated/CN=*.ciklum.net issuer=/O=AlphaSSL/CN=AlphaSSL CA - G2 SSL handshake has read 1854 bytes and written 328 bytes
  26. 26. HTTP 1 RTT TLS 3 RTTs TCP 1 RTT IPIt takes 4 RTTs to serve a HTTPS request
  27. 27. HTTP 1 RTT TLS 3 RTTs TCP 1 RTT IPIt takes 4 RTTs to serve a HTTPS request
  28. 28. TCP Reliability
  29. 29. client server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K
  30. 30. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K router
  31. 31. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K router retransmit on timeout
  32. 32. TCP Congestion Control
  33. 33. client server SEG SEG SEG receiver SEGSO_RCVBUF SEG SEG SEG SEG SEG windowSEG SEG SO_SNDBUFSO_SNDBUF sender window SO_RCVBUF
  34. 34. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A SO_SNDBUF C receiver window sender window C C SO_RCVBUF K K K overloaded router ^^^ What congestion control is actually designed for
  35. 35. AirPort router routerclient Express server S S S S SO_RCVBUF E E E E SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF G G G G A A A A SO_SNDBUF C sender window receiverSEG C window C SEG C SO_RCVBUF K K K K router ^^^ What actually happens on mobile devices
  36. 36. • Nagle algorithm while (1) write(fd, “5”, 1); TCP (telnet syndrom)Artifacts Delayed ACK http://www.stuartcheshire.org/ papers/NagleDelayedAck/
  37. 37. TCP •SO_OOBINLINEArtifacts •TCP URG
  38. 38. API Issues
  39. 39. Async NSURLConnectionAPI Issues UIScrollView CFRunLoopAddCommonMode
  40. 40. client server SEG SEG SEG SEG SEG SO_RCVBUFSO_RCVBUF SEG SEG SEG SEG SEG receiver window SO_SNDBUF SO_SNDBUF sender window SO_RCVBUF
  41. 41. More • Additive IncreaseCongestion • Multiplicative Decrease Control • Slow Start
  42. 42. • setsockopt(2) • adjust window size • socket buffer sizes • TCP_NODELAYiOS / Unix • etc •getsockopt(2) • monitoring • low-latency responding to socket events • do not let the buffer stay full
  43. 43. • Responsive UI • Bad example: InstagramSpeedup • Bad example:VK • best UI • worst reliability
  44. 44. • TCP Fast Open • Linux 3.6 • HAProxy • TCP/NCSpeedup • TCP and math (maths) • http://dspace.mit.edu/openaccess- disseminate/1721.1/58796 • TCP Westwood+
  45. 45. • TLS False Start • TLS NPNSpeedup • Next Protocol Negotiation • HTTP Pipelining • SPDY
  46. 46. kthxbai @darkprogerhttp://kirillov.im

×