Tcp2

226 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
226
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • In order to send some data we need just two things: the data we want to send and the address of recepient. DatagramPacket is an auxillary structure for storing these two things together. DatagramSocket is a facility which will send the data.DatagramPacket may contain offset. It stores a reference to array, so any changes in array are reflected. These objects are mutable.
  • There is no way to receive all the incoming data. We need to specify a local address. Only this process will be able to get those data. Method receive() will block until data is received. Packet should be big enough for incoming data to fit. Modified fields are: content of data starting from offset, length, remote address.
  • IPC means inter-process communication
  • IPC means inter-process communication
  • IPC means inter-process communication
  • IPC means inter-process communication
  • IPC means inter-process communication
  • IPC means inter-process communication
  • IPC means inter-process communication
  • IPC means inter-process communication
  • IPC means inter-process communication
  • Stack picture here
  • Stack picture here
  • Stack picture here
  • Stack picture here
  • Stack picture here
  • IPC means inter-process communication
  • Network can break a sequence.
  • Stack picture here
  • IPC means inter-process communication
  • Tcp2

    1. 1. TCP/IPSimple introduction to protocols and programming
    2. 2. Plan• We discuss both protocols and how to program them. You cannot program a protocol if you don’t know how it works.• Programming will be explained from application developer’s point of view. Protocol implementors and tool developers need some specific knowledge• Programming is explained using Java, but principles are applicable to most other languages• Java APIs are redundant, anyway we discuss all of them• Tools and troubleshooting are included
    3. 3. UDP• One of the simplest protocols ever• Java API reflects that simplicity
    4. 4. Code for senderbyte[] data = "Hello!".getBytes();InetSocketAddress remote = new InetSocketAddress("10.50.3.70", 7767);DatagramPacket packet = new DatagramPacket(data, data.length, remote);DatagramSocket socket = new DatagramSocket();socket.send(packet);
    5. 5. Code for receiverInetSocketAddress local = new InetSocketAddress("10.50.3.70", 7767);DatagramSocket socket = new DatagramSocket(local);byte[] buffer = new byte[100];DatagramPacket packet = new DatagramPacket(buffer, buffer.length);socket.receive(packet);String data = new String(packet.getData(), packet.getOffset(), packet.getLength());System.out.println("From " + packet.getSocketAddress() + " received " + data);
    6. 6. Programming model: blocking IPC Receiving OS application application thread is running
    7. 7. Programming model: blocking IPC Receiving OS application receive() is a “system call” from application to OS
    8. 8. Programming model: blocking IPC Receiving OS application receive() is a “system call” from application to OS. It makes OS to block application thread until data are available
    9. 9. Programming model: blocking IPC Sending Receiving application OS OS applicationapplication thread is running Application thread blocked
    10. 10. Programming model: blocking IPC Sending Receiving application OS OS application UDP Application send() is a packet thread blocked system call
    11. 11. Programming model: blocking IPC Sending Receiving application OS OS application UDP Application send() is a thread blocked system call, but packetit doesn’t block application thread Packet is delivered from OS to OS
    12. 12. Programming model: blocking IPC Sending Receiving application OS OS application UDP Incoming packet makes OS to resume packet application thread and return from system callapplication thread is running
    13. 13. Programming model: blocking IPC Sending Receiving application OS OS applicationapplication thread is application running thread is running
    14. 14. Programming model: blocking IPC Sending Receiving application OS OS application UDP receive() is a send() is a packet blocking system call.non-blocking system call It returns when data are received
    15. 15. Party identification• IP address used by underlying IP layer• Four bytes, usually written as dot-separated decimal numbers: “10.50.3.70”• InetAddress• UDP port: integer between 0 and 65535• InetSocketAddress = InetAddress + port
    16. 16. Protocol’s stack Remote IP address, remote port and payload are provided by Application application when it makes a system call Remote IP/port PayloadOS/Application boundary Local IP/port UDP Local IP address and port are bound to socket IP Ethernet
    17. 17. Protocol’s stack Application Remote IP/port Payload Local IP/port UDP•Destination port UDP Payload•Source port•Checksum IP Ethernet
    18. 18. Protocol’s stack Application Remote IP/port Payload Local IP/port UDP•Dest IP address UDP Payload•Source IP address IP address is resolved•Protocol ID (UDP=17) into MAC address IP IP UDP Payload Ethernet
    19. 19. Protocol’s stack Application Payload UDP•Source MAC address UDP Payload•Dest MAC address•EtherType (IP=0x0800) IP IP UDP Payload OS invokes driver to send data through the wire Ethernet Eth IP UDP Payload Eth
    20. 20. Protocol’s stackApplication Application Payload Dispatch to UDP applications, UDP buffering UDP Payload IP packet is IP dispatched to IP UDP layer IP UDP Payload Ethernet packet Ethernet is dispatched to Ethernet IP layer Eth IP UDP Payload Eth
    21. 21. Sniffer• Sniffer is a tool which intercepts packets at link level
    22. 22. Communication failure Sending application OS OS Global failure: no receiver process UDP send() packet completes normally ICMP message: port unreachable
    23. 23. Local address and binding• Sockets which were not bound will choose their address and port automatically• Failure: can’t bind if host:port are occupied• Special address: any local address
    24. 24. Closing sockets• Sockets consume OS resources• Closed automatically when process dies• Nothing is sent when socket is closed• Sockets can’t be re-used after they are closed, attempt to use will result in IOException
    25. 25. Bi-directional communication• Receive-reply example• Receiver replies to the sender• The same socket is re-used by both sides, once for sending, and once for receiving• Receiver can work in a loop. Each sender has only one transaction at time
    26. 26. Multi-party communication• One DatagramSocket can send and receive packets to any address• Receive twice and send twice examples
    27. 27. Buffering• Send 100 messages in a loop
    28. 28. Buffering Application Application OS OS Thread is running
    29. 29. Buffering Application Application OS OS OS buffers data until next receive() operation Thread is running send() UDP returns
    30. 30. Buffering Application Application OS OS Thread is running send() returns UDP OS adds new data to buffer, preserving boundaries
    31. 31. Buffering Application Application OS OS Thread is running send() returns UDP
    32. 32. Buffering Application Application OS OS Thread is running receive() doesn’t block and returns first packet from buffer
    33. 33. Buffering Application Application OS OS receive() returns second packet from buffer
    34. 34. Buffering Application Application OS OS OS buffers data until next receive() operation Thread is running send() UDP returns UDP receive() doesn’t UDP block and returns first packet from buffer receive() returns second packet from buffer
    35. 35. Buffering• If buffer is full, then new data packets are discarded• Happens often for highly-loaded networks• Programmer can set buffer size of particular socket through setReceiveBufferSize(int)• There is an upper limit enforced by OS. You need root access to change it
    36. 36. UDP as a service• Attempt to deliver a packet of data to specified address and port through a heterogenious network (wraps IP service)• Multiplexing (ports)• Buffering
    37. 37. Application protocol• A protocol which uses UDP as transport and provides some actual service• Examples: DNS, SNMP, DHCP, TFTP, RTP, SIP
    38. 38. DNS• Resolution of domain names into IP addresses• Hierarchial database• “Server” is a node which performs a service. “Client” is a node which asks for a service• DNS is an on-demand data translation service. Client provides source data (domain name) and server returns translated data (IP address)
    39. 39. DNS over UDP• Port number is 53• Two messages required to implement a service: request from client, and response from a server. Such two-directional exchange is called a transaction• Message is a one data packet• Transaction support means correllation and failure handling (timeout)• DNS should implement transactions itself (ID is 16-bit length)
    40. 40. RTP• Service: unidirectional streaming of data• “Server” here is a sender, “Client” is a receiver• Client doesn’t ask for streaming to start• Client may not accept packets, server will not know about that• Some data may be lost, this is acceptable
    41. 41. RTP over UDP• RTP should identify streams. Such large communication structure is called “session”• UDP delivers data packets. RTP should split stream into packets and assemble a stream back• RTP should sequence packets• Support for sessions and sequencing is a task for RTP itself, since UDP doesn’t support them.
    42. 42. TCP• Reliable session establishement• Full duplex sessions• Sequencing• Retransmissions• Buffering with overload control• Timeouts• Peer death detection
    43. 43. Connection stages• Asymmetrical session establisheement: one side active, one side passive• Symmetrical session usage• Session closure is a complex thing
    44. 44. Code for accepter (“server”)InetSocketAddress local = new InetSocketAddress("10.50.3.70", 7788);ServerSocket servSock = new ServerSocket();servSock.bind(local);Socket socket = servSock.accept();
    45. 45. Code for connector (“client”)Socket socket = new Socket();InetSocketAddress remote = new InetSocketAddress("10.50.3.70", 7788);socket.connect(remote);
    46. 46. Connection spawning ServerSocket Socket
    47. 47. Connection spawning ServerSocketSocket Socket
    48. 48. Connection spawning ServerSocketSocket Socket
    49. 49. Connection spawning ServerSocketSocket Socket Socket
    50. 50. Connection spawning ServerSocketSocket Socket Socket Socket
    51. 51. Connection spawning ServerSocketSocket Socket Socket Socket
    52. 52. Three-way handshake Receiving OS application application thread is running
    53. 53. Three-way handshake Receiving OS application accept() is a blocking system call
    54. 54. Three-way handshake Sending Receiving application OS OS applicationapplication thread is application running thread is blocked
    55. 55. Three-way handshake Sending Receiving application OS OS application application thread is SYN blocked connect() is a blocking system call
    56. 56. Three-way handshake Sending Receiving application OS OS application application thread is blocked application thread is running SYN, ACK
    57. 57. Three-way handshake Sending Receiving application OS OS application application thread is blocked connect() returns ACK
    58. 58. Three-way handshake Sending Receiving application OS OS application accept() returnsapplication ACK thread is running
    59. 59. Three-way handshake: overview Sending Receiving application OS OS application SYN accept() is a connect() is blocking a blocking system call system call SYN, ACK ACK Packet is delivered from OS to OS
    60. 60. Protocol’s stack Application Payload•Source port•Destination port TCP•Checksum•Flags (SYN, ACK, FIN, RST)•Sequence number TCP Payload•Acknowledged number•Window size IP•Source IP address IP TCP Payload•Dest IP address•Protocol ID (TCP=6) Ethernet Eth IP UDP Payload Eth
    61. 61. Failure: no acceptor process Sending application OS OS Global failure: no receiver process SYN connect() throws IOException ICMP message: port unreachable
    62. 62. Writing dataOutputStream outstream = socket.getOutputStream();outstream.write("I love TCP".getBytes());
    63. 63. Reading dataInputStream instream = socket.getInputStream();byte[] data = new byte[100];int length = instream.read(data);System.out.println("Received " + new String(data, 0, length));
    64. 64. Reading and writing Application OS OS Application Thread Thread is is running running
    65. 65. Reading and writing Application OS OS Application Thread is running PUSH write() is a system call
    66. 66. Reading and writing Application OS OS buffers data OS Application until next read() operation Thread is running PUSH write() returns without waiting for ACK ACK ACK confirms that another side have received data
    67. 67. Reading and writing Application OS OS Application Thread read() doesn’t is block, since running there are data in buffer
    68. 68. Reading and writing Application OS OS Application Thread is running read() blocks, since buffer is empty
    69. 69. Reading and writing Application OS OS Application Thread is blocked by OS in read() PUSH write() operation
    70. 70. Reading and writing Application OS OS Application read() returns with received PUSH write() data ACK
    71. 71. Reading and writing: overview Application OS OS Application PUSH write() read() doesn’t block, since ACK there are data in buffer read() blocks, since buffer is PUSH empty write() ACK
    72. 72. TCP as a stream protocol• Modify code to read and write data in loop• No message borders• When buffer is full
    73. 73. Sequencing and retransmissions Application OS OS Application write() 104 PUSH (seq = 104) sends large 105 array of data 106
    74. 74. Sequencing and retransmissions Application OS OS Application write() PUSH (seq = 104) sends large array of 104 data 105 ACK (seq=104) 106 ACK releases packet from send buffer
    75. 75. Sequencing and retransmissions Application OS OS Application 105 106 PUSH (seq = 105) 104
    76. 76. Sequencing and retransmissions Application OS OS Application 105 106 PUSH (seq = 106) 104
    77. 77. Sequencing and retransmissions Application OS OS Application PUSH (seq = 106) 104 105 106 ACK (seq=106)
    78. 78. Sequencing and retransmissions Application OS OS Application 105 PUSH (seq = 105) 104 106
    79. 79. Sequencing and retransmissions Application OS OS Application PUSH (seq = 105) 104 105 106 ACK (seq=105)
    80. 80. Sequencing and retransmissions Application OS OS Application write() PUSH (seq = 104) sends large array of 104 data ACK (seq=104) PUSH (seq = 105) PUSH (seq = 106) 104 106 ACK (seq=106) PUSH (seq = 105) 104 105 106 ACK (seq=105)
    81. 81. Sequencing and retransmissions• Single packet loss is fixed by retransmissions and is transparent to application• Packets have sequence numbers and they are ordered inside socket buffers• Packet may be sent before previous packet was acknowledged. This increases throughtput and latency
    82. 82. Flow control• TCP has a flow control mechanism which prevents sender from sending packets which receiver can’t buffer• This means that write() may block
    83. 83. Flow control Application Application OS OS PUSH write()
    84. 84. Flow control Application Application OS OS PUSH write() ACK, window size = 0 Data are buffered by receiver, he reports window size=0 to avoid packets which it will have to drop
    85. 85. Flow control Application Application OS OSwrite() storesdata in localbuffer
    86. 86. Flow control Application Application OS OS write() blocks since local buffer is full
    87. 87. Flow control overview Application Application OS OS PUSH write() ACK, window size = 0write() storesdata in local Data are buffered by receiver, hebuffer reports window size=0 to avoid packets which it will have to drop write() blocks since local buffer is full
    88. 88. Failures• TCP doesn’t fix those for you. Instead, they are reported as Exceptions• Recover as you can
    89. 89. Failure: network connectivity loss Application OS PUSH write()
    90. 90. Failure: network connectivity loss Application OS write() PUSH If ACK is not received, then data are retransmitted by OS
    91. 91. Failure: network connectivity loss Application OS write() PUSH Connection timeout occurs
    92. 92. Failure: network connectivity loss Application OS attepmt to read() or write() will result in IOException
    93. 93. Failure: network connectivity loss Application OS PUSH write() PUSH If ACK is not received, then data are retransmitted by OS PUSH attepmt to read() or write() will Connection result in timeout occurs IOException
    94. 94. Failure: network connectivity loss• TCP doesn’t tell to application which data were delivered• You can’t get back your data from local send buffer
    95. 95. Failure: host reboot Application OS OS Application
    96. 96. Failure: host reboot Application OS OS Application PUSH write() ACK
    97. 97. Failure: host reboot Application OS OS Application PUSH write() ACK Crash!!!
    98. 98. Failure: host reboot Application OS OS
    99. 99. Failure: host reboot Application OS write() PUSH returns OS normally
    100. 100. Failure: host reboot Application OS write() PUSH returns OS normally RST
    101. 101. Failure: host reboot Application OS OS write() results inexception
    102. 102. Failure: host reboot overview Application OS OS Application PUSH write() ACK Crash!!! write() PUSH returns OS normally RST write() results inexception
    103. 103. Closing a connection• To free OS resources• It can have some semantics in your protocol
    104. 104. Closing a connection Application OS OS Application read() is blocked
    105. 105. Closing a connection Application OS OS Application FIN close() is a system call Socket is closed. It can’t send or receive any data. Such attempt will result in error
    106. 106. Closing a connection Application OS OS Application FIN read() returns - close() is a 1, which system call means EOF ACK Socket is not yet closed
    107. 107. Closing a connection Application OS OS Application FIN read() returns -1, close() is a which means system call EOF ACK FIN close()
    108. 108. Closing a connection Application OS OS Application FIN read() returns -1, close() is a which means system call EOF ACK FIN close() ACK
    109. 109. Closing a connection• It is not as easy as it seems• Subject to race conditions
    110. 110. Race condition Application OS OS Application
    111. 111. Race condition Application OS OS Application FIN close()
    112. 112. Race condition Application OS OS Application FIN close() ACK
    113. 113. Race condition Application OS OS Application FIN close() ACK write() PUSH
    114. 114. Race condition Application OS OS Application FIN close() ACK write() PUSH returns normally RST OS replies with RST since socket was closed
    115. 115. Race condition Application OS OS Application FIN close() ACK PUSHread() RSTresults inIOException, OS replies with RSTbuffer is since socket was closedburned
    116. 116. Using RST to close a socket• A fast way of closing a socket and burn all remaining data is by sending RST• Used by Windows when process terminates• To apply use: socket.setSoLinger(true, 0);
    117. 117. Shutdown of output stream• A way to send FIN without closing a socket
    118. 118. Shutdown of output stream Application OS OS Application
    119. 119. Shutdown of output stream Application OS OS Application FIN shutdown() is a system call
    120. 120. Shutdown of output stream Application OS OS Application FIN shutdown() ACK
    121. 121. Shutdown of output stream Application OS OS Application read() blocks in OS
    122. 122. Shutdown of output stream Application OS OS Application write() PUSH
    123. 123. Shutdown of output stream Application OS OS Application write() returns PUSH read() normally returns ACK
    124. 124. Shutdown of output stream Application OS OS Applicationread()returns datafrom bufferthen EOF read() blocks in OS
    125. 125. Shutdown of output stream Application OS OS Applicationclose() issafe,because FINreceivebuffer isempty
    126. 126. Shutdown of output stream Application OS OS Application read() FIN returns EOF close() ACK
    127. 127. Shutdown of output stream Application OS OS Application FIN shutdown() ACK write() PUSH read() read() returns EOF ACK read() FIN returns close() EOF ACK
    128. 128. What service TCP does not provide• Messages• Multicast• Encryption
    129. 129. Java IO API: TCP vs UDP• DatagramPacket = • SocketAddress, byte[], l SocketAddress + byte[] ength, offset + offset + length• send(), receive() • read(), write()
    130. 130. Java NIO API• Convenience of ByteBuffers• Blocking and non-blocking modes• Selectors
    131. 131. UDP sender: compare IO and NIObyte[] data = "Hello!".getBytes(); byte[] data = "Hello!".getBytes();InetSocketAddress remote = new InetSocketAddress remote = new InetSocketAddress("10.50.3.70", 7767); InetSocketAddress("10.50.3.70", 7767);DatagramPacket packet = new DatagramPacket(data, ByteBuffer packet = ByteBuffer.wrap(data); data.length, remote);DatagramSocket socket = new DatagramSocket(); DatagramChannel channel = DatagramChannel.open();socket.send(packet); channel.send(packet, remote);
    132. 132. UDP receiver: compare IO and NIOInetSocketAddress local = new InetSocketAddress local = new InetSocketAddress("10.50.3.70", 7767); InetSocketAddress("10.50.3.70", 7767);DatagramSocket socket = new DatagramSocket(local); DatagramChannel channel = DatagramChannel.open(); channel.socket().bind(local);byte[] buffer = new byte[100]; byte[] buffer = new byte[100];DatagramPacket packet = new ByteBuffer packet = ByteBuffer.wrap(buffer); DatagramPacket(buffer, buffer.length);socket.receive(packet); SocketAddress remote = channel.receive(packet); packet.flip();String data = new String data = new String(packet.getData(), packet.getOffset(), packet.g String(buffer.array(), 0, buffer.remaining()); etLength()); System.out.println("From " + remote+ " received " + data);System.out.println("From " + packet.getSocketAddress() + " received " + data);
    133. 133. ByteBuffer• Similar to DatagramPacket without address• Supports writing to it and reading from position limit capacity it• You can “return” data back by moving a position
    134. 134. Using a ByteBuffer position limit Some initial state. Garbage inside
    135. 135. Using a ByteBuffer position limitposition limit clear() Prepared to accept some content. Full capacity.
    136. 136. Using a ByteBufferposition limit position limit receive() New data stored, position moved to point right after the data. Can be invoked in sequence. Attempt to write beyond limit will result in BufferOverflowException
    137. 137. Using a ByteBuffer position limitposition limit flip() Limit fixes data size, position is moved to the beginning. Ready to read these data
    138. 138. Using a ByteBuffer get() copies data from ByteBuffer to provided destination. Position is moved. Can be invoked in sequence. Attempt to read beyond limit will result in BufferUnderflowExceptionposition limit position limit get() remaining()
    139. 139. Using a ByteBuffer Method compact() moves all remaining data to the beginning of buffer. Position is placed right behind those data. Limit is moved to capacity. position limit position limit compact()
    140. 140. Using a ByteBuffer: overview position limitposition limit clear() position limit receive()position limit flip() position limit get() position limit compact()
    141. 141. TCP acceptor: compare IO and NIOInetSocketAddress local = new InetSocketAddress local = new InetSocketAddress("10.50.3.70", 7788); InetSocketAddress("10.50.3.70", 7788);ServerSocket servSock = new ServerSocket(); ServerSocketChannel servSock = ServerSocketChannel.open();servSock.bind(local); servSock.socket.bind(local);Socket socket = servSock.accept(); SocketChannel channel = servSock.accept();
    142. 142. TCP connecor: compare IO and NIOSocket socket = new Socket(); SocketChannel chan = SocketChannel.open();InetSocketAddress remote = new InetSocketAddress remote = new InetSocketAddress("10.50.3.70", 7788); InetSocketAddress("10.50.3.70", 7788);socket.connect(remote); chan.connect(remote);
    143. 143. Advanced UDP• Broadcasting• Multicasting• Connected sockets• Fragmentation
    144. 144. Advanced TCP• OOB-data• Transmission control
    145. 145. The endThank you very much for attending
    146. 146. Wrapping stream into channel
    147. 147. Non-blocking operations
    148. 148. Multiplexing through Selectors
    149. 149. NIO2: Asynchronous sockets
    150. 150. UNUSED SLIDES
    151. 151. Programming model: IPC Sending application Receiving application receive() is a “system call” from application to OS. It makes OS to block application thread send() is a UDP until data are available system call, but packet it doesn’t block application Incoming packet thread makes OS to resume application thread and return from system call
    152. 152. Connection spawning ServerSocketSocket Socket Socket Socket
    153. 153. Buffering Application Application OS OS OS buffers data until next receive() operation Thread is running send() UDP returns UDP receive() doesn’t UDP block and returns first packet from buffer receive() returns second packet from buffer
    154. 154. Blocking IPC Sending Receiving application OS OS application accept() is a blocking system call connect() is SYN a blocking system call SYN, ACK ACK Packet is delivered from OS to OS
    155. 155. Protocol analysis framework• Medium of communication (“wire”)• Parties which communicate (two or more)• Direction of communication• A unit of communication (“message” ?)• Larger communication constructs (transactions, sessions)• Communication failures (global and per- attempt)

    ×