App layer


Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

App layer

  2. 2. A closer look at network structure network edge: applications and hosts network core:  routers  network of networks access networks, physical media: communication links
  3. 3. The network edgerun application programs e.g. Web, email at “edge of network” 3
  4. 4. Network Application Architecture Client Server P2P Hybrid
  5. 5. Client-server architecture server:  always-on host  permanent IP address  server farms for scaling  clients:  communicate with server  may be intermittently connected  may have dynamic IP addresses  do not communicate directly with each other 5
  6. 6. Pure P2P architecture no always-on server arbitrary end systems directly communicate peers are intermittently connected and change IP addresses example: GnutellaHighly scalable but difficult to manage 2: Application Layer 6
  7. 7. Hybrid of client-server and P2PSkype  Internet telephony app  Finding address of remote party: centralized server(s)  Client-client connection is direct (not through server) Instant messaging  Chatting between two users is P2P  Presence detection/location centralized:  User registers its IP address with central server when it comes online  User contacts central server to find IP addresses of buddies 2: Application Layer
  8. 8. Creating a network appWrite programs that application  run on different end systems transport network and communicate over a data link physical network.  e.g., Web: Web server software communicates with browser software little software written for devices in network core application application transport transport network  network core devices do not run network data link data link physical user application code physical  application on end systems allows for rapid app
  9. 9. Processes communicatingProcess: program running within a host. within same host, two processes communicate using inter-process communication (defined by OS). processes in different hosts communicate by exchanging messages Note: applications with P2P architectures have client processes & server processes Client process: process that initiates communication Server process: process that waits to be contacted
  10. 10. Sockets process sends/receives messages to/from its socket socket analogous to door  sending process shoves message out door  sending process relies on transport infrastructure on other side of door which brings message to socket at receiving process Socket is an interface between application layer and transport layer within a host : it is referred as Application Programming Interface (API): Application developer can only control (on transport layer side) (1) choice of transport protocol (TCP or UDP); (2) ability to fix a few parameters such as max buffer and max segment size 10
  11. 11. Sockets host or host or server server controlled by app developer process process socket socketTCP with TCP with buffers, Internet buffers,variables variables controlled by OS 11
  12. 12. SocketsSteps followed by client to establish the connection:1. Create a socket2. Connect the socket to the address of the server3. Send/Receive data4. Close the socketSteps followed by server to establish the connection:1. Create a socket2. Bind the socket to the port number known to all clients3. Listen for the connection request4. Accept connection request5. Send/Receive data
  13. 13. Addressing processes to receive messages, process must have identifier host device has unique32-bit IP address Q: does IP address of host on which process runs suffice for identifying the process?  Answer: NO, many processes can be running on same host identifier includes both IP address and port numbers associated with process on host. Example port numbers:  HTTP server: 80  Mail server: 25
  14. 14. App-layer protocol defines Types of messages exchanged, Public-domain protocols:  defined in RFCs  e.g., request, response  allows for interoperability Message syntax:  e.g., HTTP, SMTP  what fields in messages &  Proprietary protocols: how fields are delineated  e.g., KaZaA Message semantics  meaning of information in fields Rules for when and how processes send & respond to messages
  15. 15. Transport service requirements of common applications Application Data loss Bandwidth Time Sensitive file transfer no loss elastic no e-mail no loss elastic no Web documents no loss elastic noreal-time audio/video loss-tolerant audio: 5kbps-1Mbps yes, 100’s msec video:10kbps-5Mbps stored audio/video loss-tolerant same as above yes, few secs interactive games loss-tolerant few kbps up yes, 100’s msec instant messaging no loss elastic yes and no
  16. 16. Internet transport protocols servicesTCP service: UDP service: connection-oriented: setup  unreliable data transfer required between client and between sending and server processes receiving process reliable transport between  does not provide: sending and receiving process connection setup, flow control: sender won’t reliability, flow control, overwhelm receiver congestion control, congestion control: throttle timing, or bandwidth sender when network guarantee overloaded does not provide: timing, Q: why bother? Why is there minimum bandwidth guarantees a UDP?
  17. 17. Internet apps: application, transport protocols Application Underlying Application layer protocol transport protocol e-mail SMTP [RFC 2821] TCPremote terminal access Telnet [RFC 854] TCP Web HTTP [RFC 2616] TCP file transfer FTP [RFC 959] TCP streaming multimedia proprietary TCP or UDP (e.g. RealNetworks) Internet telephony proprietary (e.g., Vonage,Dialpad) typically UDP
  18. 18. Web and HTTP First some jargon Web page consists of objects Object can be HTML file, JPEG image, Java applet, audio file,… Web page consists of base HTML-file which includes several referenced objects Each object is addressable by a URL Example URL: host name path name
  19. 19. HTTP overviewHTTP: hypertext transfer HT protocol TP req PC running HT ues Web’s application layer TP t Explorer res pon protocol se client/server model u est eq  client: browser that TT Pr pon se Server H res running requests, receives, HT TP Apache Web server “displays” Web objects  server: Web server sends Mac running objects in response to Navigator requests
  20. 20. HTTP overview (continued) HTTP is “stateless”Uses TCP:  server maintains no client initiates TCP connection information about past (creates socket) to server, port client requests 80 aside server accepts TCP connection Protocols that maintain “state” are from client complex!  past history (state) must be maintained HTTP messages (application-  if server/client crashes, their views of layer protocol messages) “state” may be inconsistent, must be exchanged between browser reconciled (HTTP client) and Web server (HTTP server) TCP connection closed
  21. 21. HTTP connectionsNonpersistent HTTP At most one object is sent over a TCP connection. HTTP/1.0 uses nonpersistent HTTPPersistent HTTP Multiple objects can be sent over single TCP connection between client and server. HTTP/1.1 uses persistent connections in default mode
  22. 22. Nonpersistent HTTPSuppose user enters URL (contains text, references to 10 1a. HTTP client initiates TCP jpeg images) connection to HTTP server (process) at on port 80 1b. HTTP server at host waiting for TCP connection at port 80. “accepts” connection, notifying 2. HTTP client sends HTTP client request message (containing URL) into TCP connection 3. HTTP server receives request socket. Message indicates that message, forms response client wants object message containing requested someDepartment/home.index object, and sends message into time its socket
  23. 23. Nonpersistent HTTP (cont.) 4. HTTP server closes TCP 5. HTTP client receives responsetime connection. message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects 6. Steps 1-5 repeated for each of 10 jpeg objects
  24. 24. Non-Persistent HTTP: Response timeDefinition of RTT: time to send a small packet to travel from client to server and back.Response time: initiate TCP one RTT to initiate TCP connection connection RTT one RTT for HTTP request and first request few bytes of HTTP response to return file file transmission time time to RTT transmittotal = 2RTT+transmit time file file received time time 24
  25. 25. Persistent HTTPPersistent HTTP server leaves connection open after sending response subsequent HTTP messages between same client/server sent over open connection Persistent without pipelining: • client issues new request only when previous response has been received • one RTT for each referenced object Persistent with pipelining:  default in HTTP/1.1  client sends requests as soon as it encounters a referenced object  as little as one RTT for all the referenced objects
  26. 26. HTTP message: general formattwo types of HTTP messages: request, response
  27. 27. HTTP request message  HTTP request message:  ASCII (human-readable format) request line (GET, POST, GET /somedir/page.html HTTP/1.1HEAD commands) Host: User-agent: Mozilla/4.0 header Connection: close lines Accept-language:fr Carriage return, (extra carriage return, line feed) line feed indicates end of message
  28. 28. HTTP response message status line (protocol HTTP/1.1 200 OK status code Connection closestatus phrase) Date: Thu, 06 Aug 1998 12:00:15 GMT Server: Apache/1.3.0 (Unix) header Last-Modified: Mon, 22 Jun 1998 …... lines Content-Length: 6821 Content-Type: text/htmldata, e.g., data data data data data ...requestedHTML file
  29. 29. Method types HTTP/1.1HTTP/1.0  GET, POST, HEAD GET  PUT POST  uploads file in entity body to HEAD path specified in URL field  asks server to leave requested  DELETE object out of response  deletes file specified in the URL field
  30. 30. HTTP response status codesIn first line in server->client response message.A few sample codes:200 OK  request succeeded, requested object later in this message301 Moved Permanently  requested object moved, new location specified later in this message (Location:)400 Bad Request  request message not understood by server404 Not Found  requested document not found on this server505 HTTP Version Not Supported
  31. 31. Internet Directory Service: DNS: Domain Name SystemPeople: many identifiers: Domain Name System:  SSN, name, passport #  distributed database implemented in hierarchy of many name servers Internet hosts, routers:  application-layer protocol host,  IP address (32 bit) - used routers, name servers to for addressing datagrams communicate to resolve names (address/name translation)  “name”, e.g.,  note: core Internet function, - used implemented as application- by humans layer protocolQ: map between IP addresses  complexity at network’s “edge” and name ?
  32. 32. DNSDNS services Why not centralize DNS? Hostname to IP address  single point of failure translation  traffic volume Host aliasing  distant centralized database  Canonical and alias  maintenance names Mail server aliasing doesn’t scale! Load distribution  Replicated Web servers: set of IP addresses for one canonical name
  33. 33. Distributed, Hierarchical Database Root DNS Servers com DNS servers org DNS servers edu DNS servers DNS servers DNS serversDNS serversDNS servers DNS serversClient wants IP for; 1st approx: Client queries a root server to find com DNS server Client queries com DNS server to get DNS server Client queries DNS server to get IP address for, 33
  34. 34. DNS: Root name servers contacted by local name server that can not resolve name root name server:  contacts authoritative name server if name mapping not known  gets mapping  returns mapping to local name server a Verisign, Dulles, VA c Cogent, Herndon, VA (also Los Angeles) d U Maryland College Park, MD k RIPE London (also Amsterdam, g US DoD Vienna, VA Frankfurt) h ARL Aberdeen, MD i Autonomica, Stockholm (plus 3 j Verisign, ( 11 locations) other locations) m WIDE Tokyo e NASA Mt View, CA f Internet Software C. Palo Alto, CA (and 17 other locations) 13 root name servers worldwide b USC-ISI Marina del Rey, CA l ICANN Los Angeles, CA 34
  35. 35. TLD and Authoritative Servers Top-level domain (TLD) servers: responsible for com, org, net, edu, etc, and all top-level country domains uk, fr, ca, jp.  Network solutions maintains servers for com TLD  Educause for edu TLD Authoritative DNS servers: organization’s DNS servers, providing authoritative hostname to IP mappings for organization’s servers (e.g., Web and mail).  Can be maintained by organization or service provider
  36. 36. Local Name Server Does not strictly belong to hierarchy Each ISP (residential ISP, company, university) has one.  Also called “default name server” When a host makes a DNS query, query is sent to its local DNS server  Acts as a proxy, forwards query into hierarchy.
  37. 37. Example root DNS serverHost at wants IP address for 2 3 TLD DNS server 4 5 local DNS server 7 6 1 8 authoritative DNS server requesting host 37
  38. 38. Recursive queries root DNS serverrecursive query: puts burden of name resolution on contacted 2 3 name server heavy load? 7 6iterated query: TLD DNS server contacted server replies with name of server to contact local DNS server “I don’t know this name, but 5 4 ask this server” 1 8 authoritative DNS server requesting host
  39. 39. DNS: caching and updating records once (any) name server learns mapping, it caches mapping  cache entries timeout (disappear) after some time  TLD servers typically cached in local name servers  Thus root name servers not often visited update/notify mechanisms under design by IETF  RFC 2136 
  40. 40. DNS recordsDNS: distributed db storing resource records (RR) RR format: (name, value, type, ttl)Type=A Type=CNAME  name is hostname  name is alias name for some  value is IP address “canonical” (the real) name is reallyType=NS name is domain (e.g.  value is canonical name value is hostname of authoritative name server for this domain Type=MX  value is name of mailserver associated with name
  41. 41. DNS protocol, messagesDNS protocol : query and reply messages, both with same message format msg header  identification: 16 bit # for query, reply to query uses same #  flags:  query or reply  recursion desired  recursion available  reply is authoritative 41
  42. 42. DNS protocol, messages Name, type fields for a query RRs in response to query records for authoritative servers additional “helpful”info that may be used
  43. 43. Inserting records into DNS Example: just created startup “Network Utopia” Register name at a registrar (e.g., Network Solutions)  Need to provide registrar with names and IP addresses of your authoritative name server (primary and secondary)  Registrar inserts two RRs into the com TLD server:  (,, NS)  (,, A) Put in authoritative server Type A record for and Type MX record for How do people get the IP address of your Web site?
  44. 44. For students to do Cookies Web Caching
  45. 45. User-server state: cookies Example:Many major Web sites use  Susan access Internet always cookies from same PCFour components:  She visits a specific e- 1) cookie header line of commerce site for first time HTTP response message  When initial HTTP requests arrives at site, site creates a 2) cookie header line in unique ID and creates an entry HTTP request message in backend database for ID  Set-cookie : 1678453 included 3) cookie file kept on in http response message which user’s host, managed by browser saves in special cookie- user’s browser file and sends it with every request. 4) back-end database at Web site
  46. 46. Cookies: keeping “state” (cont.) client server Cookie file usual http request msg en server da try i tab n b usual http response + creates ID as ac e ebay: 8734 k en Set-cookie: 1678 1678 for user d Cookie file usual http request msg amazon: 1678 cookie: 1678 cookie- ss ebay: 8734 specific acce usual http response msg action ssone week later: ce ac Cookie file usual http request msg cookie- amazon: 1678 cookie: 1678 spectific ebay: 8734 usual http response msg action
  47. 47. Cookies (continued)What cookies can bring: authorization aside shopping carts Cookies and privacy:  cookies permit sites to learn a lot recommendations about you user session state (Web e-  you may supply name and e-mail to sites mail) How to keep “state”:  Protocol endpoints: maintain state at sender/receiver over multiple transactions  cookies: http messages carry state
  48. 48. Web caches (proxy server) Goal: satisfy client request without involving origin server user sets browser: Web accesses via cache origin server browser sends all HTTP HT Proxy requests to cache TP req server que st HT ues re client TP TP ons e  object in cache: cache res pon t HT res p se TP returns object HT st ue eq  else cache requests Pr on se TT p H res object from origin HTT P server, then returns client object to client origin server
  49. 49. More about Web caching Cache acts as both client and Why Web caching? server  Reduce response time for Typically cache is installed client request. by ISP (university, company,  Reduce traffic on an residential ISP) institution’s access link.  Internet dense with caches: enables “poor” content providers to effectively deliver content (but so does P2P file sharing)
  50. 50. Caching exampleAssumptions origin average object size = 100,000 bits servers avg. request rate from institution’s public browsers to origin servers = 15/sec Internet delay from institutional router to any origin server and back to router = 2 secConsequences 1.5 Mbps access link utilization on LAN = 15% institutional utilization on access link = 100% network 10 Mbps LAN total delay = Internet delay + access delay + LAN delay = 2 sec + minutes + milliseconds
  51. 51. Caching example (cont)Possible solution origin servers increase bandwidth of access public link to, say, 10 Mbps InternetConsequences utilization on LAN = 15% 10 Mbps utilization on access link = 15% access link Total delay = Internet delay + institutional network access delay + LAN delay 10 Mbps LAN = 2 sec + msecs + msecs often a costly upgrade
  52. 52. Caching example (cont)Install cache suppose hit rate is .4 origin serversConsequence public 40% requests will be satisfied Internet almost immediately 60% requests satisfied by origin server 1.5 Mbps utilization of access link access link reduced to 60%, resulting in institutional negligible delays (say 10 network 10 Mbps LAN msec) total avg delay = Internet delay + access delay + LAN delay = .6*(2.01) secs + . institutional 4*milliseconds < 1.4 secs cache
  53. 53. Conditional GET cache server HTTP request msg Goal: don’t send object if cache If-modified-since: has up-to-date cached version <date> object not cache: specify date of cached modified copy in HTTP request HTTP response HTTP/1.0  If-modified-since: <date> 304 Not Modified server: response contains no object if cached copy is up-to- HTTP request msg date: If-modified-since: HTTP/1.0 304 Not Modified <date> object modified HTTP response HTTP/1.0 200 OK <data>
  54. 54. Electronic Mail outgoingThree major components: message queue user agents user user mailbox mail servers agent simple mail transfer protocol: SMTP mail user server agent SMTPUser Agent mail a.k.a. “mail reader” server user SMTP agent composing, editing, reading mail messages SMTP user e.g., Eudora, Outlook, elm, Netscape mail server agent Messenger outgoing, incoming messages stored user on server agent user agent
  55. 55. Electronic Mail: mail servers userMail Servers agent mail mailbox contains incoming user server agent messages for user SMTP mail message queue of outgoing (to server user be sent) mail messages SMTP agent SMTP protocol between mail SMTP servers to send email messages mail user server agent  client: sending mail server  “server”: receiving mail user agent server user agent
  56. 56. Electronic Mail: SMTP [RFC 2821] uses TCP to reliably transfer email message from client to server, port 25 direct transfer: sending server to receiving server three phases of transfer  handshaking (greeting)  transfer of messages  closure command/response interaction  commands: ASCII text  response: status code and phrase messages must be in 7-bit ASCII
  57. 57. Sample SMTP interactionS: 220 hamburger.eduC: HELO crepes.frS: 250 Hello, pleased to meet youC: MAIL FROM: <>S: 250 Sender okC: RCPT TO: <>S: 250 ... Recipient okC: DATAS: 354 Enter mail, end with "." on a line by itselfC: Do you like ketchup?C: How about pickles?C: .S: 250 Message accepted for deliveryC: QUITS: 221 closing connection
  58. 58. SMTP: final words SMTP uses persistent connections SMTP requires message (header & body) to be in 7-bit ASCII SMTP server uses CRLF.CRLF to determine end of message
  59. 59. Mail message formatSMTP: protocol for exchanging email msgs header blankRFC 822: standard for text line message format: header lines, e.g., body  To:  From:  Subject:  different from SMTP commands! body  the “message”, ASCII characters only
  60. 60. Message format: multimedia extensions MIME(multi purpose internet mail extensions): multimedia mail extension, RFC 2045, 2056 additional lines in msg header declare MIME content type From: MIME version To: Subject: Picture of yummy crepe. method used MIME-Version: 1.0 to encode data Content-Transfer-Encoding: base64 Content-Type: image/jpeg multimedia data type, subtype, base64 encoded data .....parameter declaration ......................... ......base64 encoded data encoded data
  61. 61. Scenario: Alice sends message to Bob1) Alice uses UA to compose message 4) SMTP client sends Alice’s and “to” message over the TCP2) Alice’s UA sends message to her connection mail server; message placed in 5) Bob’s mail server places the message queue message in Bob’s mailbox3) Client side of SMTP opens TCP 6) Bob invokes his user agent to connection with Bob’s mail server read message 1 mail mail server user user server 2 agent agent 3 6 4 5
  62. 62. Mail access protocols SMTP SMTP access user user protocol agent agent sender’s mail receiver’s mail server server SMTP: delivery/storage to receiver’s server Mail access protocol: retrieval from server  POP: Post Office Protocol [RFC 1939]  authorization (agent <-->server) and download  IMAP: Internet Mail Access Protocol [RFC 1730]  more features (more complex)  manipulation of stored msgs on server  HTTP: Hotmail , Yahoo! Mail, etc.
  63. 63. POP3 protocol S: +OK POP3 server readyauthorization phase C: user bob client commands: S: +OK C: pass hungry  user: declare username S: +OK user successfully logged on  pass: password C: list server responses S: 1 498  +OK S: 2 912 S: .  -ERR C: retr 1transaction phase, client: S: <message 1 contents> S: . list: list message numbers C: dele 1 retr: retrieve message by number C: retr 2 S: <message 1 contents> dele: delete S: . quit C: dele 2 C: quit S: +OK POP3 server signing off
  64. 64. POP3 (more) and IMAPMore about POP3 IMAP Previous example uses  Keep all messages in one “download and delete” mode. place: the server Bob cannot re-read e-mail if  Allows user to organize he changes client messages in folders “Download-and-keep”: copies  IMAP keeps user state across of messages on different sessions: clients  names of folders and POP3 is stateless across mappings between sessions message IDs and folder name
  65. 65. FTP: the file transfer protocol2: Application Layer FTP file transfer FTP FTP user client server interface user at host local file remote file system system  transfer file to/from remote host  client/server model  client: side that initiates transfer (either to/from remote)  server: remote host  ftp: RFC 959  ftp server: port 21 65
  66. 66. FTP: separate control, data connections2: Application Layer TCP control connection  FTP client contacts FTP server at port 21 port 21, specifying TCP as transport protocol TCP data connection  Client obtains authorization over FTP port 20 FTP control connection client server  Client browses remote directory Ì Server opens another TCP by sending commands over data connection to transfer control connection. another file.  When server receives file transfer Ì Control connection: “out of command, server opens 2nd TCP band” connection (for file) to client Ì FTP server maintains “state”:  After transferring one file, server current directory, earlier closes data connection. authentication 66
  67. 67. FTP commands, responses2: Application Layer Sample commands: Sample return codes  sent as ASCII text over control  status code and phrase (as in channel HTTP)  USER username  331 Username OK,  PASS password password required  125 data connection  LIST return list of file in already open; transfer current directory starting  RETR filename retrieves  425 Can’t open data (gets) file connection  STOR filename stores (puts)  452 Error writing file file onto remote host 67
  68. 68. P2P file sharing2: Application Layer Example  Alice chooses one of the peers, Bob.  Alice runs P2P client  File is copied from Bob’s application on her notebook computer PC to Alice’s notebook: HTTP  Intermittently connects to  While Alice downloads, Internet; gets new IP address for each other users uploading from connection Alice.  Alice’s peer is both a Web  Asks for “Hey Jude” client and a transient Web  Application displays other server. peers that have copy of Hey Jude. All peers are servers = highly scalable! 68
  69. 69. P2P: centralized directory Bob original “Napster” design centralized directory server 1) when peer connects, it 1 informs central server: peers 1  IP address  content 1 3 2) Alice queries for “Hey 2 1 Jude” 3) Alice requests file from Bob Alice 2: Application Layer 69
  70. 70. P2P: problems with centralized directory2: Application Layer  Single point of failure file transfer is  Performance bottleneck decentralized, but locating content is highly  Copyright infringement centralized 70
  71. 71. Query flooding: Gnutella2: Application Layer  fully distributed overlay network: graph  no central server  edge between peer X and Y  public domain protocol if there’s a TCP connection  many Gnutella clients  all active peers and edges implementing protocol is overlay net  Edge is not a physical link  Given peer will typically be connected with < 10 overlay neighbors 71
  72. 72. Gnutella: protocol File transfer:72 HTTP Query message sent over existing TCP connections Query QueryHit peers forward Query message y Qu u er ery t Q y Hi QueryHit u er sent over Q reverse Query path QueryHit Scalability: Qu er limited scope y flooding 2: Application Layer
  73. 73. Exploiting heterogeneity: KaZaA  Each peer is either a group leader or assigned to a group leader.  TCP connection between peer and its group leader.  TCP connections between some pairs of group leaders.  Group leader tracks the content in all its children. o r d in a r y p e e r g r o u p - le a d e r p e e r n e ig h o r in g r e la tio n s h ip s in o v e r la y n e tw o r k 2: Application Layer 73
  74. 74. KaZaA: Querying74  Each file has a hash and a descriptor  Client sends keyword query to its group leader  Group leader responds with matches:  For each match: metadata, hash, IP address  If group leader forwards query to other group leaders, they respond with matches  Client then selects files for downloading  HTTP requests using hash as identifier sent to peers holding desired file 2: Application Layer
  75. 75. KaZaA tricks75  Limitations on simultaneous uploads  Request queuing  Incentive priorities  Parallel downloading 2: Application Layer
  76. 76. Multimedia Introduction to Audio Audio Compression Streaming Audio Internet Radio Voice over IP Introduction to Video Video Compression Video on Demand The MBone – The Multicast Backbone