Presentation material including summary of "High Performance Browser Networking" by Ilya Grigorik. This book includes very good summary of computer network not only for internet browsing but also multimedia streaming.

3. • Faster Site Lead to:
– Better user engagement: Let user to start to use
– Better user retention: Keep user to use it
– Higher conversion: Quick response
 Performance Bottom line of online businesses
• Critical Network Traffic Performance Components
– Latency:
• The time
• from the source sending a packet
• to the destination receiving it
– Bandwidth:
• Maximum throughput
• of a logical or physical communication path
• Lowest capacity link

5. • Propagation delay
– Time required for a message to travel from the sender to
receiver.
– A function of distance and speed of signal(The speed of light).
• Transmission delay
– Time required to push all the packet's bits into the link
– A function of the packet's length and data rate of the link
• Processing delay
– Time required to process the packet header, check for bit-level
errors, and determine the packet's destination
• Queuing delay
– Time the incoming packets is waiting in the queue until it can
be processed.
• Total delay
– The sum of all listed delays above

6. • Signal latencies in vacuum and fiber
• Perceptible lag
– over 100-200msec
• Sluggish:
– interaction over 300msec
• Mental context switch :
– more than 1 sec.
 Our application needs to respond within 100 ms.
Route Distance
Time, light
in vacuum
Time, light in fiber
Round-trip time
(RTT) in fiber
New York to
San Francisco
4,148 km 14 ms 21 ms 42 ms
New York to
London
5,585 km 19 ms 28 ms 56 ms
New York to
Sydney
15,993 km 53 ms 80 ms 160 ms
Equatorial
Circumference
40,075 km 133.7 ms 200 ms 200 ms

7. • Infamous last-mile problem
– Significant latency introduced in last few miles
– "Measuring Broadband America“ from FCC Feb 2013
• FTTH (Fiber-to-the-home): 18ms
• Cable: 26ms
• DSL: 44ms
• traceroute
– Shows volumes about the topology and performance of your
internet provider

8. • Optical fiber vs Metal Wire
– An optical fiber
• Slightly thicker than a human hair
• acts as a simple "light pipe,"
• designed to transmit light between the two ends of the cable.
– Metal wires
• Higher signal loss,
• Electromagnetic interference
• Higher lifetime maintenance costs.
– Long-distance hops, a fiber-optic link is used
• Bandwidth of Optical fiber
– 171Gbit/s per channel
– Over 400 wavelengths which are multiplexed (WDM)
– Total: over 70 Tbit/s per single fiber link

9. • Technology at the Network Edge
– dial-up, DSL, Cable, a host of wireless tech, FTTH
• Available Bandwidth to the User
– by Akamai servers in Q1 2013
Rank Country Average Mbps
Year-over-year chang
e
- Global 3.1 17%
1 South Korea 14.2 -10%
2 Japan 11.7 6.8%
3 Hong Kong 10.9 16%
4 Switzerland 10.1 24%
5 Netherlands 9.9 12%
…
9 United States 8.6 27%
• High bandwidth is desirable.
Not a guarantee of stable e2e performance.

10. • Higher bandwidth requirement is growing fast.
– Streaming high quality videos.
– The requirement is getting difficult.
• Fiber link utilization is getting higher.
– Maybe, we need to add more fibers.
• Need to Improvise latency
– Speed of light places a hard limit on the minimum
latency.
– Altenatives? Make distance shorter!
 caching, pre-fetching, variety of similar techniques in
subsequent chapters.

12. • TCP/IP: Internet Protocol Suite
– The IP, or Internet Protocol :
• Providing host-to-host routing and addressing
– TCP: Transmission Control Protocol
• Provding abstraction of a reliable network running over an
unreliable channel
– by Vint Cert and Bob Kahn in 1974 paper titled "A
Protocol for Packet Network Intercommunication“
• RFCs : Internet Standards
– Original Proposal/RFC 675 is revised several times.
– 1981 V4 spec was published as separate ones
• RFC 791 - Internet Protocol
• RFC 793 - Transmission Control Protocol

13. • TCP
– May popular application: WWW, email, file transfer and
many others
– TCP provides an effective abstraction of a reliable network
running over an unreliable channel
– Hiding most of the complexity of network communication
from our applications.
• Data loss, In-order delivery, Congestion control and avoidance,
Data integrity, and more
– HTTP does not specify TCP as the only transport protocol.
However, all HTTP traffic on Internet today uses TCP
– Understanding of TCP is very essential for understanding
web experience. Same with streaming as well.

14. • All TCP connection begin with a three-way
handshake

15. • Performance Implication
– New TCP connection have a full roundtrip of latency
before any application data can be transferred.“
– Reuse Connection!!!
• a critical optimization for any application running over TCP.
• TCP Fast Open (TFO)
– allows data transfer within the SYN packet
– could decrease:
• HTTP transaction network latency by 15%,
• whole-page load times by over 10% on average,
• and in some cases by up to 40% in high-latency scenarios.

16. • Congestion collapse:
– John Nagle mentioned in RFC 896
– Assumptions: gateways connect networks of widely
different bandwidth
– Symptoms:
• Roundtrip time exceed the maximum retransmission interval
for any host
• Host begins to introduce more and more copies of the same
datagrams into the net
• Proposed mechanisms
– Flow control,
– Congestion control, and
– Congestion avoidance.

17. • To prevent the sender not to send data which the
receiver cannot process
• TCP connection advertises its own receive window (rwnd)
– the size of the available buffer space to hold the incoming data

18. • How it works?
– If, for any reason, one of the sides is not able to keep up,
then it can advertise a smaller window to the sender.
– If the window reaches zero, then it is treated as a signal
that no more data should be sent until the existing data
in the buffer has been cleared by the application layer.
– each ACK packet carries the latest rwnd value for each
side
• Window Scaling (RFC 1323)
– maximum value (216, or 65,535 bytes)
– RFC 1323: 65,535 bytes to 1 gigabyte
– Major platforms support this.

19. • No mechanism to prevent issue from network
– Why? Each peer don't know available bandwidth at the
beginning of a new connection
– 1. Need a mechanism to estimate it
– 2. To adapt their speeds to the continuously changing
conditions within the network.

20. • Algorithms for adapting congestion
– In 1988, Van Jacobson and Michael J. Karels
– slow-start, congestion avoidance, fast retransmit, and fast
recovery.
• Slow Start
– After handsaking.
– To start, the server initializes a new congestion window
(cwnd) variable per TCP connection
– Sets its initial value to a conservative, system-specified
value (initcwnd on Linux).
– Congestion window size (cwnd)
• Sender-side limit on the amount of data the sender can have in
flight before receiving an acknowledgment (ACK) from the client.
• it will be a private variable maintained by the sender
• the maximum amount of data in flight (not ACKed) is the
minimum of the rwnd and cwnd variables.

21. • cwnd start value
– Firstly, 1 network segment
– maximum of 4 segments in April 1999(RFC 2581)
– 10 segments by RFC 6928 in April 2013.
• cwnd increase
– Exponential Increase. Double cwnd size with every roundtrip

22. • Web Performance with TCP, HTTP
– Web application are often short and busty. (Web with small text
and images)
– Connections terminate before the maximum window size is reached.
– The performance is often limited by the roundtrip time
– Slow-start limits the available bandwidth throughput.
 Reuse TCP connection!
• Slow-Start Restart
– resets the cwnd after it has been idle for a defined period of time
– To catch changed the network conditions during idle.
– To avoid congestion, the window is reset to a "safe" default.
– A significant impact on performance of long-lived TCP connections
– Recommended to disable SSR on the server.
– On Linux platforms, the SSR setting can be checked and disabled
via the following commands:
• $> sysctl net.ipv4.tcp_slow_start_after_idle
• $> sysctl -w net.ipv4.tcp_slow_start_after_idle=0

23. • Roundtrip time: 56 ms
• Client and server bandwidth:
5 Mbps
• Client and server receive
window: 65,535 bytes
• Initial congestion window: 4
segments
(4×1460 bytes≈5.7 KB)
• Server processing time to
generate response: 40 ms
• No packet loss, ACK per
packet, GET request fits into
single segment

24. • Algorithm to help regulate the performance.
– Use packet loss as a feedback. cwnd increases until packet
loss happens.
• cwnd is reset
• Increase and reset cwnd according to give algorithm
– Variants of algorithm
• TCP Tahoe and Reno (original implementations) (AIMD)
• TCP Vegas
• TCP New Reno
• TCP BIC
• TCP CUBIC (default on Linux) or Compound TCP (default on
Windows)
• Proportional Rate Reduction for TCP (RFC 6939)
– Improve the speed of recovery when a packet is lost
– 3-10% reduction of in average latency for connection with packet
loss

25. • Bandwidth-delay product (BDP)
– = Bandwidth * Delay
maximum amount of unacknowledged data in flight.
– How big rwnd and cwnd?
. rwnd/cwnd 16KB and RTT 100ms = 1.31Mbps
. 10Mbps BW, RTT 100ms
122.1KB cwnd, rwnd size.

26. • In-order delivery
– Better to use UDP for the situation

27. • Unchanging Core Principles of TCP
– TCP three-way handshake introduces a full roundtrip
of latency.
– TCP slow-start is applied to every new connection.
– TCP flow and congestion control regulate throughput
of all connections.
– TCP throughput is regulated by current congestion
window size
– In most cases, latency, not bandwidth, is the
bottleneck for TCP

28. • Turning Server Configuration
– “Increasing TCP’s Initial Congestion Window”
• Allows TCP transfers more data in the first roundtrip
• Accelerates the window growth
• For bursty and short-lived connections.
– Disable “Slow-Start Restart”
• Disabling slow-start after idle
• long-lived TCP connections, which transfer data in bursts.
– Enable “Window Scaling (RFC 1323)”
• Increases the maximum receive window size
• Allows high-latency connections to achieve better throughput.
– “TCP Fast Open”
• Data sending in the initial SYN packet in certain situations.
• Requires support both on client and server;
• Investigate if your application can make use of it

29. • Tuning Application Behavior
– No bit is faster than one that is not sent; send fewer bits.
– We can’t make the bits travel faster, but we can move the
bits closer. (CDN)
– TCP connection reuse is critical to improve performance.
• Performance Checklist
– Upgrade server kernel to latest version (Linux: 3.2+).
– Ensure that cwnd size is set to 10.
– Disable slow-start after idle.
– Ensure that window scaling is enabled.
– Eliminate redundant data transfers.
– Compress transferred data.
– Position servers closer to the user to reduce roundtrip
times.
– Reuse established TCP connections whenever possible.

31. • User Datagram Protocol, or UDP, (RFC 768)
– Added to the core network protocol suite in Aug. 1980 by Jon
Postel,
– Referred to as a null protocol,
– The primary feature and appeal of UDP is not in what it
introduces, but rather in all the features it chooses to omit.
– Domain Name System (DNS) uses UDP : given a human-friendly
computer hostname
– Web Real-Time Communication (WebRTC)
• Jointly developed by the IETF and W3C
• Enabling real-time communication based UDP
– voice and video calling and other forms of peer-to-peer (P2P)
communication,
• Datagram
– The term "datagram" is often reserved for packets delivered via
an unreliable service
– UDP acronym, to form "Unreliable Datagram Protocol.”

32. • UDP is a simple, stateless protocol, suitable for
bootstrapping other application protocols on top

33. • No guarantee of message delivery
• No acknowledgments, retransmissions, or timeouts
• No guarantee of order of delivery
• No packet sequence numbers, no reordering, no
head-of-line blocking
• No connection state tracking
• No connection establishment or teardown state
machines
• No congestion control
• No built-in client or network feedback mechanisms

34. • The IP Network Address Translator (NAT)
– 32bit long IPv4 addresses
• maximum of 4.29 billion unique IP addresses.
• IPv4 address depletion problem
– Introduced in mid-1994 (RFC 1631) as interim
solution

35. • TCP
– Well-defined protocol state machine
• A handshake,
• Application data transfer
• A well-defined exchange to close the connection.
– Good to manage NAT entries
• UDP
– UDP does not support connection states
– How to manage routing records?
• Delete when a given timer is expired.
• Needs to introduce bidirectional keepalive packets to
periodically reset the timers

36. • Not reachable to the device behind NAT
– Need to act as both client and server for P2P apps
• VoIP, games, and file sharing
– The client needs to know public IP and shares that as P2P
application data
– NAT needs to keep the NAT entry for the peer to reach the
client

37. • Session Traversal Utilities for NAT (STUN, RFC 5389)
– Protocol Features
• To discover the presence of a NAT
• To obtain the public IP and port for the current connection
– Operation
• Requires STUN server that must reside on the public network.
• Sends Binding request STUN server
• Replies with a response that contains the public IP and port
• keepalive pings keeps the NAT routing entries from timing out.

38. • Traversal Using Relays around NAT (TURN, RFC 5766)
– Protocol Features
• A fallback of STUN. STUN can fail because of firewall, etc.
• Relaying communication between peers. No more P2P.
• Run over UDP and Switch to TCP if all else fails.
– Operation
• Both clients begin their connections by sending an allocate request
to the same TURN server
• Permissions negotiation.
• Both peers sending their data to the TURN server,
• TURN server relays it to the other peer.

39. • Interactive Connectivity Establishment (ICE, RFC 5245)
– Protocol Features
• To Build an effective NAT traversal solution
• Seek to establish the most efficient tunnel between the participants
– Operation
• Direct connection where possible, leveraging STUN negotiation
where needed,
• Finally fallback to TURN if all else fails.

40. • "Unicast UDP Usage Guidelines for Application Designers" RFC 540
– focuses on design guidelines for applications delivered via unicast UDP.
– Here is a short sample of the recommendations:
• Application must tolerate a wide range of Internet path conditions.
• Application should control rate of transmission.
• Application should perform congestion control over all traffic.
• Application should use bandwidth similar to TCP.
• Application should back off retransmission counters following loss.
• Application should not send datagrams that exceed path MTU.
• Application should handle datagram loss, duplication, and reordering.
• Application should be robust to delivery delays up to 2 minutes.
• Application should enable IPv4 UDP checksum, and must enable IPv6
checksum.
• Application may use keepalives when needed (minimum interval 15
seconds).