Проблемы использования TCP в мобильных приложениях.  Владимир Кириллов
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Проблемы использования TCP в мобильных приложениях. Владимир Кириллов

  • 744 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
744
On Slideshare
706
From Embeds
38
Number of Embeds
2

Actions

Shares
Downloads
3
Comments
0
Likes
1

Embeds 38

http://dev.by 37
http://www.dev.by 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. TCP PerformanceFor Mobile Applications Vladimir Kirillov @darkproger
  • 2. Networking Stack
  • 3. Application SessionTransport NetworkData Link
  • 4. Application HTTP Session TLSTransport TCP Network IP WiFi EdgeData Link 3G LTE
  • 5. Application HTTP (Http|NS)URLConnection Session TLS OpenSSLTransport TCP SOCK_STREAM Network IP kernel WiFi EdgeData Link hardware 3G LTE Level Protocol API / Implementation
  • 6. gdb HTTP (Http|NS)URLConnection ptrace TLS OpenSSL socket dtrace API TCP SOCK_STREAM bpf(4) IP kernel LSFWiFi Edge hardware 3G LTE Protocol API / Implementation Introspection
  • 7. capturing iPhone traffic% udid=$(system_profiler SPUSBDataType | awk /iPhone/{go=1} /Serial/ {if (go) print $3; go=0})276cb9530201bcehelloworldcd55560ed015d00% rvictl -s $udidStarting device 276cb9530201bcehelloworldcd55560ed015d00[SUCCEEDED]% ifconfig rvi0rvi0: flags=3005<UP,DEBUG,LINK0,LINK1> mtu 0
  • 8. capturing Android traffic# adb connect 192.168.56.100# adb shellshell@android:/ $ suTest propsu allows access thanks toandroVM.su.bypass propertyshell@android:/ # tcpdump -i eth1
  • 9. tcpdump -i lo0 -w t.pcap -s0 &nc -l 5000 &echo hello | nc localhost 5000kill %1
  • 10. # tcpdump -r t.pcap -nnvv -tttt -K tcp port 50002012-11-24 12:23:35.511134 IP6 (hlim 64, next-header TCP (6) payload length: 44) ::1.51734 > ::1.5000: Flags [ S ], seq 453038127, win65535, options [mss 16324,nop,wscale 4,nop,nop,TS val 303407352 ecr 0,sackOK,eol], length 02012-11-24 12:23:35.511175 IP6 (hlim 64, next-header TCP (6) payload length: 20) ::1.5000 > ::1.51734: Flags [ R .], seq 0, ack 453038128,win 0, length 02012-11-24 12:23:35.511226 IP (tos 0x0, ttl 64, id 8400, offset 0, flags [DF], proto TCP (6), length 64) S 127.0.0.1.51735 > 127.0.0.1.5000: Flags [ ], seq 2527137802, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 303407352 ecr0,sackOK,eol], length 02012-11-24 12:23:35.511276 IP (tos 0x0, ttl 64, id 58311, offset 0, flags [DF], proto TCP (6), length 64) 127.0.0.1.5000 > 127.0.0.1.51735: Flags S [ .], seq 494520280, ack 2527137803, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val303407352 ecr 303407352,sackOK,eol], length 02012-11-24 12:23:35.511287 IP (tos 0x0, ttl 64, id 47796, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511298 IP (tos 0x0, ttl 64, id 52186, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511332 IP (tos 0x0, ttl 64, id 31417, offset 0, flags [DF], proto TCP (6), length 58) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [P .], seq 1:7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 62012-11-24 12:23:35.511351 IP (tos 0x0, ttl 64, id 29060, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511354 IP (tos 0x0, ttl 64, id 4019, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 7, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511367 IP (tos 0x0, ttl 64, id 20879, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 17 packets captured2012-11-24 12:23:35.511378 IP (tos 0x0, ttl 64, id 59633, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511388 IP (tos 0x0, ttl 64, id 56794, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [F.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0
  • 11. # tcpdump -r t.pcap -nnvv -tttt -K tcp port 50002012-11-24 12:23:35.511134 IP6 (hlim 64, next-header TCP (6) payload length: 44) ::1.51734 > ::1.5000: Flags [ S ], seq 453038127, win65535, options [mss 16324,nop,wscale 4,nop,nop,TS val 303407352 ecr 0,sackOK,eol], length 02012-11-24 12:23:35.511175 IP6 (hlim 64, next-header TCP (6) payload length: 20) ::1.5000 > ::1.51734: Flags [ R .], seq 0, ack 453038128,win 0, length 02012-11-24 12:23:35.511226 IP (tos 0x0, ttl 64, id 8400, offset 0, flags [DF], proto TCP (6), length 64) S 127.0.0.1.51735 > 127.0.0.1.5000: Flags [ ], seq 2527137802, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 303407352 ecr0,sackOK,eol], length 02012-11-24 12:23:35.511276 IP (tos 0x0, ttl 64, id 58311, offset 0, flags [DF], proto TCP (6), length 64) 127.0.0.1.5000 > 127.0.0.1.51735: Flags S [ .], seq 494520280, ack 2527137803, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val303407352 ecr 303407352,sackOK,eol], length 02012-11-24 12:23:35.511287 IP (tos 0x0, ttl 64, id 47796, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511298 IP (tos 0x0, ttl 64, id 52186, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511332 IP (tos 0x0, ttl 64, id 31417, offset 0, flags [DF], proto TCP (6), length 58) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [P length 6 .], seq 1:7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352],2012-11-24 12:23:35.511351 IP (tos 0x0, ttl 64, id 29060, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags2012-11-24 12:23:35.511354 IP (tos 0x0, ttl 127.0.0.1.5000 > 127.0.0.1.51735: Flags 64, id 4019, offset 0, flags [DF], proto TCP (6), length 52) ^^ [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 [.], seq 1, ack 7, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511367 IP (tos 0x0, ttl 127.0.0.1.5000 > 127.0.0.1.51735: Flags % stat -f %z t.pcap 64, id 20879, offset 0, flags [DF], proto TCP (6), length 52) [.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 17 packets captured 13062012-11-24 12:23:35.511378 IP (tos 0x0, ttl 64, id 59633, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511388 IP (tos 0x0, ttl 64, id 56794, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [F.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0
  • 12. % tcptrace t.pcap17 packets seen, 17 TCP packets tracedelapsed wallclock time: 0:00:00.001344, 12648 pkts/sec analyzedtrace file elapsed time: 0:00:00.000305TCP connection info:1: localhost:52132 - localhost:5000 (a2b) 1> 1< (reset)2: localhost:52133 - localhost:5000 (c2d) 8> 7< (complete) (reset)
  • 13. % tcptrace -o2 -l t.pcap... adv wind scale: 4 adv wind scale: 4 req sack: Y req sack: Y sacks sent: 0 sacks sent: 0 urgent data pkts: 0 pkts urgent data pkts: 0 pkts urgent data bytes: 0 bytes urgent data bytes: 0 bytes mss requested: 16344 bytes mss requested: 16344 bytes max segm size: 6 bytes max segm size: 0 bytes min segm size: 6 bytes min segm size: 0 bytes avg segm size: 5 bytes avg segm size: 0 bytes max win adv:146976 bytes max win adv: 146976 bytes min win adv:146976 bytes min win adv: 146976 bytes zero win adv: 0 times zero win adv: 0 times avg win adv: 146976 bytes avg win adv: 122480 bytes initial window:6 bytes initial window: 0 bytes initial window: 1 pkts initial window: 0 pkts ttl stream length: 6 bytes ttl stream length: 1 bytes missed data: 0 bytes missed data: 1 bytes truncated data: 0 bytes truncated data: 0 bytes truncated packets: 0 pkts truncated packets: 0 pkts data xmit time: 0.000 secs data xmit time: 0.000 secs idletime max: 0.1 ms idletime max: 0.0 ms throughput: 27027 Bps throughput: 0 Bps
  • 14. endpoint endpoint SO_RCVBUF SO_SNDBUF SO_SNDBUF SO_RCVBUF
  • 15. endpoint endpoint SO_RCVBUF SEG SO_SNDBUF SO_SNDBUF SEG SO_RCVBUF
  • 16. 2 * LATENCY = RTTendpoint LATENCY endpoint SO_RCVBUF SEG SO_SNDBUF BANDWIDTH SO_SNDBUF SEG SO_RCVBUF
  • 17. • Time from one endpoint to another • Each connection spans multiple linksLatency • latency = sum (lat foreach link) • RTT = 2 * latency
  • 18. • Number of bytes a link can handleBandwidth • bw = min (bw foreach link)
  • 19. Bandwidth Delay BDP = RTT * BANDWIDTH Product
  • 20. client server SO_RCVBUF receiver window SO_SNDBUF SO_SNDBUF sender window SO_RCVBUF
  • 21. client server SO_RCVBUF SEG receiver window SEG SO_SNDBUF SO_SNDBUF sender window SEG SEG SO_RCVBUF
  • 22. client server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF SO_SNDBUF SEG receiver windowSEG sender window SEG SEG SEG SEG SEG SEG SO_RCVBUF
  • 23. TCP • stateful • ordered byte • reliablestream • managed
  • 24. HTTP paired TLShas state TCPno state IP
  • 25. HTTPTLS SYNTCP 1 RTT SYN,ACK ACKIP
  • 26. "Oh, a SSL certificate warning. Ill read it carefully and understand the possibleTLS implications before proceeding.” -- no User, ever.
  • 27. "Oh, a SSL library. Ill understand carefully its semantics and will not breakTLS authentication.” -- unknown developer.
  • 28. % openssl s_client - showcerts -connectTLS internet.velcom.by:443
  • 29. CONNECTED(00000003)depth=3 Thawte Premium Server CAverify error:num=19:self signed certificate incertificate chainverify return:0
  • 30. Certificate chain 0 s:/C=BY/ST=Minsk/L=Minsk/O=FE Velcom/CN=internet.velcom.by i:/C=US/O=Thawte, Inc./CN=Thawte SSL CA-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----
  • 31. 1 s:/C=US/O=Thawte, Inc./CN=Thawte SSL CA i:/C=US/O=thawte, Inc./OU=Certification ServicesDivision/OU=(c) 2006 thawte, Inc. - For authorized useonly/CN=thawte Primary Root CA-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----
  • 32. 2 s:/C=US/O=thawte, Inc./OU=Certification ServicesDivision/OU=(c) 2006 thawte, Inc. - For authorized useonly/CN=thawte Primary Root CA i:/C=ZA/ST=Western Cape/L=Cape Town/O=ThawteConsulting cc/OU=Certification Services Division/CN=Thawte Premium Server CA/emailAddress=premium-server@thawte.com-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----
  • 33. 3 s:/C=ZA/ST=Western Cape/L=Cape Town/O=ThawteConsulting cc/OU=Certification Services Division/CN=Thawte Premium Server CA/emailAddress=premium-server@thawte.com i:/C=ZA/ST=Western Cape/L=Cape Town/O=ThawteConsulting cc/OU=Certification Services Division/CN=Thawte Premium Server CA/emailAddress=premium-server@thawte.com-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----
  • 34. Server certificatesubject=/C=BY/ST=Minsk/L=Minsk/O=FE Velcom/CN=internet.velcom.byissuer=/C=US/O=Thawte, Inc./CN=Thawte SSL CASSL handshake has read 4736 bytes and written 328 bytes
  • 35. % openssl s_client -showcerts - connect ciklum.com:443 CONNECTED(00000003) depth=0 /C=UA/OU=Domain Control Validated/CN=*. ciklum.net verify error:num=20:unable to get local issuer certificate verify return:1 depth=0 /C=UA/OU=Domain Control Validated/CN=*.ciklum.net verify error:num=27:certificate not trustedTLS verify return:1 depth=0 /C=UA/OU=Domain Control Validated/CN=*.ciklum.net verify error:num=21:unable to verify the first certificate verify return:1 --- Certificate chain 0 s:/C=UA/OU=Domain Control Validated/CN=*.ciklum.net i:/O=AlphaSSL/CN=AlphaSSL CA - G2 ... Server certificate subject=/C=UA/OU=Domain Control Validated/CN=*.ciklum.net issuer=/O=AlphaSSL/CN=AlphaSSL CA - G2 SSL handshake has read 1854 bytes and written 328 bytes
  • 36. HTTP SYN SYN,ACKTLS 2 RTTs ACK, ClientHello ServerHello, CertificateTCP 1 RTT ClientKEX, ChangeCipherSpec ChangeCipherSpec,FinishedIP
  • 37. SYNHTTP 1 RTT SYN,ACK ACK, ClientHelloTLS 2 RTTs ServerHello, Certificate ClientKEX, ChangeCipherSpecTCP 1 RTT ChangeCipherSpec,Finished GET OKIPIt takes 4 RTTs to serve a HTTPS request
  • 38. SYNHTTP 1 RTT SYN,ACK ACK, ClientHelloTLS 2 RTTs ServerHello, Certificate ClientKEX, ChangeCipherSpecTCP 1 RTT ChangeCipherSpec,Finished GET OKIPIt takes 4 RTTs to serve a HTTPS request
  • 39. TCP Reliability
  • 40. client server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K
  • 41. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K router
  • 42. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K router retransmit on timeout (~200ms)
  • 43. TCP Congestion Control
  • 44. client server SEG SEG SEG receiver SEGSO_RCVBUF SEG SEG SEG SEG SEG windowSEG SEG SO_SNDBUFSO_SNDBUF sender window SO_RCVBUF
  • 45. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A SO_SNDBUF C receiver window sender window C C SO_RCVBUF K K K overloaded router ^^^ What congestion control is actually designed for
  • 46. Crappy Wi-Fi AirPort router routerclient Express server S S S S SO_RCVBUF E E E E SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF G G G G A A A A SO_SNDBUF C sender window receiverSEG C window C SEG C SO_RCVBUF K K K K router ^^^ What actually happens on mobile devices
  • 47. • Nagle algorithm while (1) write(fd, “5”, 1); TCP (telnet syndrom)Artifacts Delayed ACK http://www.stuartcheshire.org/ papers/NagleDelayedAck/
  • 48. •SO_OOBINLINE TCP •TCP URGArtifacts •RFC 6093
  • 49. API Issues
  • 50. Async NSURLConnectionAPI Issues UIScrollView CFRunLoopAddCommonMode
  • 51. client server SEG SEG SEG SEG SEG SO_RCVBUFSO_RCVBUF SEG SEG SEG SEG SEG receiver window SO_SNDBUF SO_SNDBUF sender window SO_RCVBUF
  • 52. Congestion • Additive Increase Avoidance Multiplicative Decrease • • Slow Start TCP Reno
  • 53. Android
  • 54. Android
  • 55. # cat /proc/sys/net/ipv4/tcp_slow_start_after_idle1# cat /proc/sys/net/ipv4/tcp_no_metrics_save0# echo 0 > /proc/sys/net/ipv4/tcp_slow_start_after_idle# echo 1 > /proc/sys/net/ipv4/tcp_no_metrics_save
  • 56. # find /proc/sys/net/ipv4 | grep cong | xargs -tn1 catcat /proc/sys/net/ipv4/tcp_allowed_congestion_controlcubic renocat /proc/sys/net/ipv4/tcp_available_congestion_controlcubic renocat /proc/sys/net/ipv4/tcp_congestion_controlcubic
  • 57. # ip route showdefault via 192.168.56.1 dev eth1 initcwnd 10 initrwnd 10
  • 58. • setsockopt(2) • adjust window size • socket buffer sizes • TCP_NODELAY (Nagle)Sockets • etc •getsockopt(2) • monitoring • low-latency responding to socket events • do not let the buffer stay full
  • 59. getsockopt(SOL_TCP, TCP_INFO)ESTAB 0 176 10.1.1.1:22 10.1.1.2:61984 users:(("sshd",18989,3))!mem:(r0,w1168,f2928,t0)ts sack bic wscale:4,5 rto:280rtt:56.25/7.5 ato:40 cwnd:8 ssthresh:7send 1.6Mbps rcv_rtt:50 rcv_space:14480 #include <linux/tcp.h> iproute2
  • 60. Speedup
  • 61. Do not create connections!
  • 62. for _i in $(seq 10); ssh -f thailand cat
  • 63. for _i in $(seq 10); ssh -o ControlMaster yes -f thailand cat
  • 64. • InstagramResponsive • VK UI • best UI • worst reliability
  • 65. • TCP Fast OpenSteroids • Linux 3.6 • HAProxy
  • 66. • TCP/NC • TCP and math (maths)Steroids • http://dspace.mit.edu/openaccess- disseminate/1721.1/58796
  • 67. • TCP Westwood+ (LFN) • TCP Veno (Wi-Fi)Scheduling, • http://www.apan.net/meetings/Algorithms honolulu2004/materials/engineering/ APAN_ppt.pdf •CONF_TCP_CONG_VENO
  • 68. • TLS False Start • TLS NPNSteroids • Next Protocol Negotiation • HTTP Pipelining • SPDY
  • 69. • https://github.com/proger/iproute2 ss -I • https://github.com/proger/captcpResearch • tcptrace • tcpflow • monitoring
  • 70. kthxbai @darkprogerhttp://kirillov.im