Проблемы использования TCP в мобильных приложениях.  Владимир Кириллов
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Проблемы использования TCP в мобильных приложениях. Владимир Кириллов

on

  • 722 views

 

Statistics

Views

Total Views
722
Views on SlideShare
684
Embed Views
38

Actions

Likes
1
Downloads
3
Comments
0

2 Embeds 38

http://dev.by 37
http://www.dev.by 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Проблемы использования TCP в мобильных приложениях. Владимир Кириллов Presentation Transcript

  • 1. TCP PerformanceFor Mobile Applications Vladimir Kirillov @darkproger
  • 2. Networking Stack
  • 3. Application SessionTransport NetworkData Link
  • 4. Application HTTP Session TLSTransport TCP Network IP WiFi EdgeData Link 3G LTE
  • 5. Application HTTP (Http|NS)URLConnection Session TLS OpenSSLTransport TCP SOCK_STREAM Network IP kernel WiFi EdgeData Link hardware 3G LTE Level Protocol API / Implementation
  • 6. gdb HTTP (Http|NS)URLConnection ptrace TLS OpenSSL socket dtrace API TCP SOCK_STREAM bpf(4) IP kernel LSFWiFi Edge hardware 3G LTE Protocol API / Implementation Introspection
  • 7. capturing iPhone traffic% udid=$(system_profiler SPUSBDataType | awk /iPhone/{go=1} /Serial/ {if (go) print $3; go=0})276cb9530201bcehelloworldcd55560ed015d00% rvictl -s $udidStarting device 276cb9530201bcehelloworldcd55560ed015d00[SUCCEEDED]% ifconfig rvi0rvi0: flags=3005<UP,DEBUG,LINK0,LINK1> mtu 0
  • 8. capturing Android traffic# adb connect 192.168.56.100# adb shellshell@android:/ $ suTest propsu allows access thanks toandroVM.su.bypass propertyshell@android:/ # tcpdump -i eth1
  • 9. tcpdump -i lo0 -w t.pcap -s0 &nc -l 5000 &echo hello | nc localhost 5000kill %1
  • 10. # tcpdump -r t.pcap -nnvv -tttt -K tcp port 50002012-11-24 12:23:35.511134 IP6 (hlim 64, next-header TCP (6) payload length: 44) ::1.51734 > ::1.5000: Flags [ S ], seq 453038127, win65535, options [mss 16324,nop,wscale 4,nop,nop,TS val 303407352 ecr 0,sackOK,eol], length 02012-11-24 12:23:35.511175 IP6 (hlim 64, next-header TCP (6) payload length: 20) ::1.5000 > ::1.51734: Flags [ R .], seq 0, ack 453038128,win 0, length 02012-11-24 12:23:35.511226 IP (tos 0x0, ttl 64, id 8400, offset 0, flags [DF], proto TCP (6), length 64) S 127.0.0.1.51735 > 127.0.0.1.5000: Flags [ ], seq 2527137802, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 303407352 ecr0,sackOK,eol], length 02012-11-24 12:23:35.511276 IP (tos 0x0, ttl 64, id 58311, offset 0, flags [DF], proto TCP (6), length 64) 127.0.0.1.5000 > 127.0.0.1.51735: Flags S [ .], seq 494520280, ack 2527137803, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val303407352 ecr 303407352,sackOK,eol], length 02012-11-24 12:23:35.511287 IP (tos 0x0, ttl 64, id 47796, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511298 IP (tos 0x0, ttl 64, id 52186, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511332 IP (tos 0x0, ttl 64, id 31417, offset 0, flags [DF], proto TCP (6), length 58) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [P .], seq 1:7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 62012-11-24 12:23:35.511351 IP (tos 0x0, ttl 64, id 29060, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511354 IP (tos 0x0, ttl 64, id 4019, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 7, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511367 IP (tos 0x0, ttl 64, id 20879, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 17 packets captured2012-11-24 12:23:35.511378 IP (tos 0x0, ttl 64, id 59633, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511388 IP (tos 0x0, ttl 64, id 56794, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [F.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0
  • 11. # tcpdump -r t.pcap -nnvv -tttt -K tcp port 50002012-11-24 12:23:35.511134 IP6 (hlim 64, next-header TCP (6) payload length: 44) ::1.51734 > ::1.5000: Flags [ S ], seq 453038127, win65535, options [mss 16324,nop,wscale 4,nop,nop,TS val 303407352 ecr 0,sackOK,eol], length 02012-11-24 12:23:35.511175 IP6 (hlim 64, next-header TCP (6) payload length: 20) ::1.5000 > ::1.51734: Flags [ R .], seq 0, ack 453038128,win 0, length 02012-11-24 12:23:35.511226 IP (tos 0x0, ttl 64, id 8400, offset 0, flags [DF], proto TCP (6), length 64) S 127.0.0.1.51735 > 127.0.0.1.5000: Flags [ ], seq 2527137802, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val 303407352 ecr0,sackOK,eol], length 02012-11-24 12:23:35.511276 IP (tos 0x0, ttl 64, id 58311, offset 0, flags [DF], proto TCP (6), length 64) 127.0.0.1.5000 > 127.0.0.1.51735: Flags S [ .], seq 494520280, ack 2527137803, win 65535, options [mss 16344,nop,wscale 4,nop,nop,TS val303407352 ecr 303407352,sackOK,eol], length 02012-11-24 12:23:35.511287 IP (tos 0x0, ttl 64, id 47796, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511298 IP (tos 0x0, ttl 64, id 52186, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [.], seq 1, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511332 IP (tos 0x0, ttl 64, id 31417, offset 0, flags [DF], proto TCP (6), length 58) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [P length 6 .], seq 1:7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352],2012-11-24 12:23:35.511351 IP (tos 0x0, ttl 64, id 29060, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags2012-11-24 12:23:35.511354 IP (tos 0x0, ttl 127.0.0.1.5000 > 127.0.0.1.51735: Flags 64, id 4019, offset 0, flags [DF], proto TCP (6), length 52) ^^ [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 [.], seq 1, ack 7, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511367 IP (tos 0x0, ttl 127.0.0.1.5000 > 127.0.0.1.51735: Flags % stat -f %z t.pcap 64, id 20879, offset 0, flags [DF], proto TCP (6), length 52) [.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0 17 packets captured 13062012-11-24 12:23:35.511378 IP (tos 0x0, ttl 64, id 59633, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.51735 > 127.0.0.1.5000: Flags [F.], seq 7, ack 1, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 02012-11-24 12:23:35.511388 IP (tos 0x0, ttl 64, id 56794, offset 0, flags [DF], proto TCP (6), length 52) 127.0.0.1.5000 > 127.0.0.1.51735: Flags [F.], seq 1, ack 8, win 9186, options [nop,nop,TS val 303407352 ecr 303407352], length 0
  • 12. % tcptrace t.pcap17 packets seen, 17 TCP packets tracedelapsed wallclock time: 0:00:00.001344, 12648 pkts/sec analyzedtrace file elapsed time: 0:00:00.000305TCP connection info:1: localhost:52132 - localhost:5000 (a2b) 1> 1< (reset)2: localhost:52133 - localhost:5000 (c2d) 8> 7< (complete) (reset)
  • 13. % tcptrace -o2 -l t.pcap... adv wind scale: 4 adv wind scale: 4 req sack: Y req sack: Y sacks sent: 0 sacks sent: 0 urgent data pkts: 0 pkts urgent data pkts: 0 pkts urgent data bytes: 0 bytes urgent data bytes: 0 bytes mss requested: 16344 bytes mss requested: 16344 bytes max segm size: 6 bytes max segm size: 0 bytes min segm size: 6 bytes min segm size: 0 bytes avg segm size: 5 bytes avg segm size: 0 bytes max win adv:146976 bytes max win adv: 146976 bytes min win adv:146976 bytes min win adv: 146976 bytes zero win adv: 0 times zero win adv: 0 times avg win adv: 146976 bytes avg win adv: 122480 bytes initial window:6 bytes initial window: 0 bytes initial window: 1 pkts initial window: 0 pkts ttl stream length: 6 bytes ttl stream length: 1 bytes missed data: 0 bytes missed data: 1 bytes truncated data: 0 bytes truncated data: 0 bytes truncated packets: 0 pkts truncated packets: 0 pkts data xmit time: 0.000 secs data xmit time: 0.000 secs idletime max: 0.1 ms idletime max: 0.0 ms throughput: 27027 Bps throughput: 0 Bps
  • 14. endpoint endpoint SO_RCVBUF SO_SNDBUF SO_SNDBUF SO_RCVBUF
  • 15. endpoint endpoint SO_RCVBUF SEG SO_SNDBUF SO_SNDBUF SEG SO_RCVBUF
  • 16. 2 * LATENCY = RTTendpoint LATENCY endpoint SO_RCVBUF SEG SO_SNDBUF BANDWIDTH SO_SNDBUF SEG SO_RCVBUF
  • 17. • Time from one endpoint to another • Each connection spans multiple linksLatency • latency = sum (lat foreach link) • RTT = 2 * latency
  • 18. • Number of bytes a link can handleBandwidth • bw = min (bw foreach link)
  • 19. Bandwidth Delay BDP = RTT * BANDWIDTH Product
  • 20. client server SO_RCVBUF receiver window SO_SNDBUF SO_SNDBUF sender window SO_RCVBUF
  • 21. client server SO_RCVBUF SEG receiver window SEG SO_SNDBUF SO_SNDBUF sender window SEG SEG SO_RCVBUF
  • 22. client server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF SO_SNDBUF SEG receiver windowSEG sender window SEG SEG SEG SEG SEG SEG SO_RCVBUF
  • 23. TCP • stateful • ordered byte • reliablestream • managed
  • 24. HTTP paired TLShas state TCPno state IP
  • 25. HTTPTLS SYNTCP 1 RTT SYN,ACK ACKIP
  • 26. "Oh, a SSL certificate warning. Ill read it carefully and understand the possibleTLS implications before proceeding.” -- no User, ever.
  • 27. "Oh, a SSL library. Ill understand carefully its semantics and will not breakTLS authentication.” -- unknown developer.
  • 28. % openssl s_client - showcerts -connectTLS internet.velcom.by:443
  • 29. CONNECTED(00000003)depth=3 Thawte Premium Server CAverify error:num=19:self signed certificate incertificate chainverify return:0
  • 30. Certificate chain 0 s:/C=BY/ST=Minsk/L=Minsk/O=FE Velcom/CN=internet.velcom.by i:/C=US/O=Thawte, Inc./CN=Thawte SSL CA-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----
  • 31. 1 s:/C=US/O=Thawte, Inc./CN=Thawte SSL CA i:/C=US/O=thawte, Inc./OU=Certification ServicesDivision/OU=(c) 2006 thawte, Inc. - For authorized useonly/CN=thawte Primary Root CA-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----
  • 32. 2 s:/C=US/O=thawte, Inc./OU=Certification ServicesDivision/OU=(c) 2006 thawte, Inc. - For authorized useonly/CN=thawte Primary Root CA i:/C=ZA/ST=Western Cape/L=Cape Town/O=ThawteConsulting cc/OU=Certification Services Division/CN=Thawte Premium Server CA/emailAddress=premium-server@thawte.com-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----
  • 33. 3 s:/C=ZA/ST=Western Cape/L=Cape Town/O=ThawteConsulting cc/OU=Certification Services Division/CN=Thawte Premium Server CA/emailAddress=premium-server@thawte.com i:/C=ZA/ST=Western Cape/L=Cape Town/O=ThawteConsulting cc/OU=Certification Services Division/CN=Thawte Premium Server CA/emailAddress=premium-server@thawte.com-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----
  • 34. Server certificatesubject=/C=BY/ST=Minsk/L=Minsk/O=FE Velcom/CN=internet.velcom.byissuer=/C=US/O=Thawte, Inc./CN=Thawte SSL CASSL handshake has read 4736 bytes and written 328 bytes
  • 35. % openssl s_client -showcerts - connect ciklum.com:443 CONNECTED(00000003) depth=0 /C=UA/OU=Domain Control Validated/CN=*. ciklum.net verify error:num=20:unable to get local issuer certificate verify return:1 depth=0 /C=UA/OU=Domain Control Validated/CN=*.ciklum.net verify error:num=27:certificate not trustedTLS verify return:1 depth=0 /C=UA/OU=Domain Control Validated/CN=*.ciklum.net verify error:num=21:unable to verify the first certificate verify return:1 --- Certificate chain 0 s:/C=UA/OU=Domain Control Validated/CN=*.ciklum.net i:/O=AlphaSSL/CN=AlphaSSL CA - G2 ... Server certificate subject=/C=UA/OU=Domain Control Validated/CN=*.ciklum.net issuer=/O=AlphaSSL/CN=AlphaSSL CA - G2 SSL handshake has read 1854 bytes and written 328 bytes
  • 36. HTTP SYN SYN,ACKTLS 2 RTTs ACK, ClientHello ServerHello, CertificateTCP 1 RTT ClientKEX, ChangeCipherSpec ChangeCipherSpec,FinishedIP
  • 37. SYNHTTP 1 RTT SYN,ACK ACK, ClientHelloTLS 2 RTTs ServerHello, Certificate ClientKEX, ChangeCipherSpecTCP 1 RTT ChangeCipherSpec,Finished GET OKIPIt takes 4 RTTs to serve a HTTPS request
  • 38. SYNHTTP 1 RTT SYN,ACK ACK, ClientHelloTLS 2 RTTs ServerHello, Certificate ClientKEX, ChangeCipherSpecTCP 1 RTT ChangeCipherSpec,Finished GET OKIPIt takes 4 RTTs to serve a HTTPS request
  • 39. TCP Reliability
  • 40. client server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K
  • 41. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K router
  • 42. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A A SO_SNDBUF C receiver window sender window C SEG SEG C C SO_RCVBUF K K K K router retransmit on timeout (~200ms)
  • 43. TCP Congestion Control
  • 44. client server SEG SEG SEG receiver SEGSO_RCVBUF SEG SEG SEG SEG SEG windowSEG SEG SO_SNDBUFSO_SNDBUF sender window SO_RCVBUF
  • 45. AirPort router routerclient Express server SO_RCVBUF SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF A A A SO_SNDBUF C receiver window sender window C C SO_RCVBUF K K K overloaded router ^^^ What congestion control is actually designed for
  • 46. Crappy Wi-Fi AirPort router routerclient Express server S S S S SO_RCVBUF E E E E SEG receiver window SEG SEG SEG SEG SEG SO_SNDBUF G G G G A A A A SO_SNDBUF C sender window receiverSEG C window C SEG C SO_RCVBUF K K K K router ^^^ What actually happens on mobile devices
  • 47. • Nagle algorithm while (1) write(fd, “5”, 1); TCP (telnet syndrom)Artifacts Delayed ACK http://www.stuartcheshire.org/ papers/NagleDelayedAck/
  • 48. •SO_OOBINLINE TCP •TCP URGArtifacts •RFC 6093
  • 49. API Issues
  • 50. Async NSURLConnectionAPI Issues UIScrollView CFRunLoopAddCommonMode
  • 51. client server SEG SEG SEG SEG SEG SO_RCVBUFSO_RCVBUF SEG SEG SEG SEG SEG receiver window SO_SNDBUF SO_SNDBUF sender window SO_RCVBUF
  • 52. Congestion • Additive Increase Avoidance Multiplicative Decrease • • Slow Start TCP Reno
  • 53. Android
  • 54. Android
  • 55. # cat /proc/sys/net/ipv4/tcp_slow_start_after_idle1# cat /proc/sys/net/ipv4/tcp_no_metrics_save0# echo 0 > /proc/sys/net/ipv4/tcp_slow_start_after_idle# echo 1 > /proc/sys/net/ipv4/tcp_no_metrics_save
  • 56. # find /proc/sys/net/ipv4 | grep cong | xargs -tn1 catcat /proc/sys/net/ipv4/tcp_allowed_congestion_controlcubic renocat /proc/sys/net/ipv4/tcp_available_congestion_controlcubic renocat /proc/sys/net/ipv4/tcp_congestion_controlcubic
  • 57. # ip route showdefault via 192.168.56.1 dev eth1 initcwnd 10 initrwnd 10
  • 58. • setsockopt(2) • adjust window size • socket buffer sizes • TCP_NODELAY (Nagle)Sockets • etc •getsockopt(2) • monitoring • low-latency responding to socket events • do not let the buffer stay full
  • 59. getsockopt(SOL_TCP, TCP_INFO)ESTAB 0 176 10.1.1.1:22 10.1.1.2:61984 users:(("sshd",18989,3))!mem:(r0,w1168,f2928,t0)ts sack bic wscale:4,5 rto:280rtt:56.25/7.5 ato:40 cwnd:8 ssthresh:7send 1.6Mbps rcv_rtt:50 rcv_space:14480 #include <linux/tcp.h> iproute2
  • 60. Speedup
  • 61. Do not create connections!
  • 62. for _i in $(seq 10); ssh -f thailand cat
  • 63. for _i in $(seq 10); ssh -o ControlMaster yes -f thailand cat
  • 64. • InstagramResponsive • VK UI • best UI • worst reliability
  • 65. • TCP Fast OpenSteroids • Linux 3.6 • HAProxy
  • 66. • TCP/NC • TCP and math (maths)Steroids • http://dspace.mit.edu/openaccess- disseminate/1721.1/58796
  • 67. • TCP Westwood+ (LFN) • TCP Veno (Wi-Fi)Scheduling, • http://www.apan.net/meetings/Algorithms honolulu2004/materials/engineering/ APAN_ppt.pdf •CONF_TCP_CONG_VENO
  • 68. • TLS False Start • TLS NPNSteroids • Next Protocol Negotiation • HTTP Pipelining • SPDY
  • 69. • https://github.com/proger/iproute2 ss -I • https://github.com/proger/captcpResearch • tcptrace • tcpflow • monitoring
  • 70. kthxbai @darkprogerhttp://kirillov.im