Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Distributed computing bsics

413 views

Published on

Distributed Computing Basics

Published in: Education
  • Be the first to comment

Distributed computing bsics

  1. 1. Distributed Systems Deepak John Department Of Computer Applications SJCET-Pala
  2. 2. Distributed System Definitions 2 A system in which hardware or software components located at networked computers communicate and coordinate their actions only by message passing(synchronous or asynchronous ). Consequences Concurrency of components: Components in distributed systems are executed in concurrent processes. Integrity of the system may be violated if concurrent updates are not coordinated. No global clock. Independent failures. A collection of independent computers that appear to the users of the system as a single computer. 2
  3. 3. Centralized vs. Distributed Computing mainframe computer workstation network host network link terminal centralized computing distributed computing 3
  4. 4. Why Distributed Systems 4 The need to share data across remote geographies Functional distribution: computers have different functional capabilities (i.e., sharing of resources with specific functionalities). Load distribution / balancing: assign tasks to processors such that the overall system performance is optimized. Replication of processing power: independent processors working on the same task. Use of heterogeneous components •Less resource-demanding sub-tasks run on less powerful computers More efficient use of resources Physical separation: systems that rely on the fact that computers are physically separated. Economics: collections of microprocessors offer a better price/performance ratio than large mainframes. mainframes: 10 times faster, 1000 times as expensive 4
  5. 5. Administrative/functional issues Scalability The system can be extended by adding more components (i.e., WWW) Resilience to failures If one component fails, others can proceed with work on the task 5
  6. 6. Distributed Systems Examples-The Internet 6 The Internet is a vast interconnected collection of computer networks of many types. Its design enabling a program running anywhere to address messages to programs anywhere else. Allowing its users to make use of many services as: WWW, E-Mail, Web hosting, and File transfer. Small organizations and individual users can to access internet services through Internet Service Providers (ISPs). Independent intranets are linked together by high transmission capacity circuits called backbones. 6
  7. 7. 7 intranet ISP desktop computer: backbone satellite link server: network link: A typical portion of the Internet 7
  8. 8. Distributed Systems Examples-Intranets 8 • An Intranet is a portion of the internet that is administrated separately and its local security policies are enforced by a configured boundary. • Composed of several local area networks (LANs) linked by backbone connections to allow its users to access the provided services. • Connected to the Internet via a router. • Many organization protect their own services from unauthorized use by filtering incoming and outgoing messages using a firewall. 8
  9. 9. 9 the rest of email server Web server Desktop computers File server router/firewall print and other servers other servers print Local area network email server the Internet A typical Intranet 9
  10. 10. Distributed Systems Examples- Mobile and Ubiquitous Computing 10 The portability of many computing devices and the ability to connect to networks in different places makes mobile computing possible. Mobile computing is the performance of computing tasks while the users are on the move and away from their residence intranet but still provided with access to resources via the devices they carry with them. Ubiquitous computing is the harnessing of many small cheap computational devices that are present in user’s physical environments. 10
  11. 11. 11 Laptop Mobile Printer Camera Internet Host intranet Home intranet WAP Wireless LAN phone gateway Host site Portable and handheld devices in a distributed system 11
  12. 12. Resource Sharing 12 Resources in a distributed system are: Encapsulated within computers. Can only accessed from other computers by communication. Managed by a program that offers a communication interface. The term service is used for a distinct part of a computer system which manages a collection of related resources and presents their functionality to users and applications via a well-defined set of operations. Server refers to a running program on a networked computer that accepts request messages from programs, clients, running on other computers to perform a service and responds appropriately by reply messages. A complete interaction between a client and a server is called a remote invocation. 12
  13. 13. The World Wide Web (WWW) 13 WWW is an open client-server architecture implemented on top of the Internet. Users use the Web through available web browsers to retrieve and view documents of many types and interact with unlimited set of services. Web is based on three main standard technological components: The HyperText Markup Language (HTML) specifies the contents and layout of pages. Uniform Resource Locators (URLs) identify documents and other resources accessible via the web. The HyperText Transfer Protocol (HTTP) determines the standard rules for interaction between browsers and web servers to fetch documents and other resources. 13
  14. 14. 14 Internet BrowsersWeb servers www.google.com www.cdk3.net www.w3c.org Protocols Activity.html http://www.w3c.org/Protocols/Activity.html http://www.google.com/search?q=kindberg http://www.cdk3.net/ File system of www.w3c.org Web servers and web browsers 14
  15. 15. World Wide Web Components-HTML 15 HTML is used to specify: The text and images that make up the contents of a web page, How web page components are formatted for presentation to the user, and Links and which resources are associated with them. HTML is produced using a standard text editor or an HTML tool. HTML uses tags to specify content: <IMG SRC = “http://www.cdk3.com/WebExample/Images/earth.jpg”> <P> Welcome to Earth! Visitor may also be interested in taking a look at the <A HREF = “http://www.cdk3.com/WebExample/Moon.html”> Moon </A> </P> ……………………… HTML text is stored in a file accessible by a web server. 15
  16. 16. World Wide Web Components-URLs 16 Browsers examine URLs in order to fetch the corresponding resources from web servers. A URL is typed by the user into the browser or selected from their bookmarks. Also, the browser looks up the corresponding URL when the user clicks on a link or fetches a resource embedded in a web page such an image. Every URL in its full form has two top-level components: scheme : scheme-specific-location URL examples: http://www.cdk3.net/ http://www.google.com/search?q=kindberg. ftp://ftp.downloadIt.com/software/aProg.exe mailto:metwally@ccis.ksu.edu.sa 16
  17. 17. World Wide Web Components-HTTP 17 HTTP defines the ways in which browsers and any other types of client interact with web server. HTTP is a request-reply protocol: The client sends a request message to the server containing the URL of the required resource, Then the server looks up the pathname and if it exists, sends back the file’s contents in a reply message to the client. HTTP allows the client to request one resource at a time. 17
  18. 18. Distributed Systems Challenges 18 A number of challenges are recognized in the design of distributed systems. These challenges have been tackled with varying degrees of success in existing systems. The list below gives some indication of measures that are employed to meet each challenge: Heterogeneity : standards and protocols; middleware; virtual machine; Openness: publication of services; notification of interfaces; Security: firewalls; encryption; Scalability: replication; caching; multiple servers; Failure Handling: failure tolerance; recover/roll-back; redundancy; Concurrency: concurrency control to ensure data consistency; Transparency: middleware; location transparent naming; anonymity 18
  19. 19. Distributed Systems Challenges-Heterogeneity 19 The internet enables users to access services over a variety and difference: Networks. Operating systems. Implementations by different developers. The middleware software layers provide a programming abstraction as well as masking the heterogeneity of the underlying networks, hardware, operating systems and programming languages. Virtual machines approach provides a way of making code executable on any hardware – mobile code. Computer hardware. Programming languages. 19
  20. 20. Distributed Systems Challenges-Openness 20 The openness of distributed systems is determined by the degree to which new resource-sharing services can be added and be available for use by variety of client programs (services publication). The specification and documentation of key software interfaces of the system components must be available(interfaces notification). Systems that are designed to support resource sharing in this way are termed open distributed systems to emphasize the fact that they are extensible. 20
  21. 21. Distributed Systems Challenges-Security 21 Security for information resources has three components: Confidentiality: protection against disclosure to unauthorized individuals. Integrity: protection against alteration or corruption. Availability: protection against interference with the means to access the resources. A firewall can be used to form a barrier around an intranet to protect it from outside users but does not deal with ensuring the appropriate use of resources by users within the intranet. Encryption can be used to provide adequate protection of shared resources and to keep sensitive information secret when is transmitted in messages over the internet. 21
  22. 22. Distributed Systems Challenges-Scalability 22 A system is described as scalable if will remain effective when there is a significant increase in the number of resources and the number of users. The design of scalable dist. systems presents many challenges: Controlling the cost of physical resources. Controlling the performance loss. Preventing software resources running out. Avoiding performance bottlenecks. 22
  23. 23. Distributed Systems Challenges-Failure Handling 23 Failures in a distributed system are partial, therefore the handling of it is particularly difficult. Some failures can be detected but others are difficult or impossible to detect it. Some detected failures can be hidden or made less severe. Recovery from failures involves the design of software so that the state of permanent data can be rolled back after a server has crashed. Services can be made to tolerate failures by the use of redundant components: At least two different routes between any two routers in the internet. Databases and domain name tables must be replicated in several servers. 23
  24. 24. Distributed Systems Challenges-Concurrency 24 Several clients in a distributed system can access a shared resource at the same time Any object that represents a shared resource in a distributed system (a programmer that implement it) must be responsible for ensuring that it operate correctly in a concurrent environment. Operations of objects in a concurrent environment must be synchronized in a way that its data remains consistent. The synchronization of an object operations can be achieved by standard techniques such as semaphores. 24
  25. 25. Distributed Systems Challenges-Transparency 25 Access transparency: enables local and remote resources to be accessed using identical operations. Location transparency: enables resources to be accessed without knowledge of their location. Concurrency transparency: enables several processes to operate concurrently using shared resources without interference between them. Replication transparency: enables multiple instances of resources to be used to increase reliability and performance without knowledge of the replicas by users or application programmers. Failure transparency: enables the concealment of faults, allowing users and application programs to complete their tasks despite the failure of hardware or software components. Mobility transparency: allows the movement of resources and clients within a system without affecting the operation of users or programs. Performance transparency: allows the system to be reconfigured to improve performance as loads vary. Scaling transparency: allows the system and applications to expand in scale without change to the system structure or the application algorithms. Access and location transparency together provide network transparency. 25
  26. 26. System Models 26 Architectural model defines the way in which the components of the system are placed and how they interact with one another and the way in which they are mapped onto the underlying network of computers. Fundamental models: Interaction model deals with communication details among the components and their timing and performance details. Failure model gives specification of faults and defines reliable communication and correct processes. Security model specifies possible threats and defines the concept of secure channels. 26
  27. 27. Architectural Model 27 Concerned with placement of its parts and relationship among them. Example: client-server model, peer-to-peer model Abstracts the functions of the individual components. Defines patterns for distribution of data and workload. Defines patterns of communication among the components. Example: Definition of server process, client process and peer process and protocols for communication among processes; definition client/server model and its variations. 27
  28. 28. Software and hardware service layers in distributed systems 28 Applications, services Computer and network hardware Platform Operating system Middleware 28
  29. 29. Middleware 29 Layer of software whose purpose is to mask the heterogeneity and to provide a convenient programming model for application programmers. Middleware supports such abstractions as remote method invocation, group communications, event notification, replication of shared data, real-time data streaming. Examples: Java RMI, grid software (Globus, Open grid Services), Web services. Platform Implemented independently in each computer. Provide services to the layers above them 29
  30. 30. Clients invoke individual servers 30 Server Client Client invocation result Server invocation result Process: Key: Computer: EX: browser, web client EX: Web server EX: 1. File server, 2. Web crawler Client-Server •Servers provide services •Clients access services 30
  31. 31. Peer to Peer 31 •potentially large number of participants. •communication pattern varies over time 31
  32. 32. A service provided by multiple servers 32 Server Server Server Service Client Client 32
  33. 33. Web proxy server and caches 33 Client Proxy Web server Web server server Client Cache: local copies of remote objects for faster access •Browser cache •Proxy server Additional roles: filtering, firewall 33
  34. 34. Mobile code and web applets 34 a) client request results in the downloading of applet code Web server Client Web serverApplet Applet code Client b) client interacts with the applet EX: Code streaming; mobile code 34
  35. 35. Mobile agents 35 •running program is moved from node to node (with both code and data). Thin Clients •Client performs just I/O, no computation •Basically a display with keyboard •Remote computation, storage 35
  36. 36. Interfaces and Objects 36 At the programming level •Need standard interfaces Access/provide services/objects RPC/RMI 36
  37. 37. Design Requirements for Distributed Architectures 37
  38. 38. Interaction Models 3838
  39. 39. 39 two variants of interaction: synchronous and asynchronous synchronous distributed systems • lower and upper bounds for execution of a step •message transmission in bounded time •clock drift rate is bounded •failures can be detected when bounds are exceeded •accomplished by allocating sufficient resources asynchronous distributed systems •no bounds on process speed, message delay, clock drift rate •failures are harder to detect •performance can't be guaranteed 39
  40. 40. 40 •Event ordering Relative ordering might be more important than exact time logical clocks--ordering events without physical clocks. logical time: Assign a number L to each event, such that L(e1) < L(e2) if e1 happens before e2 40
  41. 41. Failure Model 41 Failure of processes or communication channels 41
  42. 42. Omission and arbitrary failures 42 Class of failure Affects Description Fail-stop Process Process halts and remains halted. Other processes may detect this state. Crash Process Process halts and remains halted. Other processes may not be able to detect this state. Omission Channel A message inserted in an outgoing message buffer never arrives at the other end’s incoming message buffer. Send-omission Process A process completes a send, but the message is not put in its outgoing message buffer. Receive-omission Process A message is put in a process’s incoming message buffer, but that process does not receive it. Arbitrary (Byzantine) Process or channel Process/channel exhibits arbitrary behaviour: it may send/transmit arbitrary messages at arbitrary times, commit omissions; a process may stop or take an incorrect step. 42
  43. 43. Timing failures 43 Class of Failure Affects Description Clock Process Process’s local clock exceeds the bounds on its rate of drift from real time. Performance Process Process exceeds the bounds on the interval between two steps. Performance Channel A message’s transmission takes longer than the stated bound. 43
  44. 44. • masking failures 44 hiding--use another server to respond, converting it into more acceptable--drop the packet if it is corrupted • reliable one-to-one communication validity: eventually delivered integrity: content not corrupted or duplicated 44
  45. 45. Security Model 45 • securing processes and channels against unauthorized access • Protecting objects: authorization (access rights to principals) authentication (identity of parties/principals) 45
  46. 46. 46 • Threat to processes & communication channels 46
  47. 47. 47
  48. 48. • Introduction • The API for the Internet protocols • External data representation and marshalling • Client-Server communication • Group communication • Case study: interprocess communication in UNIX • Summary Interprocess Communication
  49. 49. Distributed systems are built around communication. Because communications are critical to distributed systems, A key form of communications is inter process communications, based on low-level message passing over the network. Inter Process Communication 49
  50. 50. Middleware layer TCP ( UDP) from a programmers point of view TCP : two way stream UDP : datagram, message passing Java interface(Provides both datagram and stream communication primitives/interfaces – building blocks for communication protocols), UNIX Socket Marshalling(process of converting the data or the objects into a byte- stream) and demarshalling data form translation Client-Server communication Request-Reply protocols Java RMI, RPC Introduction 50
  51. 51. Middleware layers Applications Middleware layersRequest reply protocol External data representation Operating System RMI, RPC and events Programming model Interprocess communication
  52. 52. The API for the Internet protocols Characteristics of IPC – message passing using send/receive facilities for sync and addressing in distributed programs Use of sockets as API for UDP and TCP implementation Synchronous Queues at remote sites are established for message placement by clients (sender). The local process (at remote site) dequeues the message on arrival If synchronous, both the sender and receiver must synchronize on each message, i.e., both send and receive invocations are blocking-until 52
  53. 53. Asynchronous communication Send from client is non-blocking and proceeds in parallel with local operations Receive could be non-blocking (requiring a background buffer) AND if blocking, perhaps, remote process needs the message, then the process must wait on it Having both sync/async is advantageous, e.g., one thread of a process can do blocked-receive while other thread of same process perform non-block receive or are active – simplifies synchronization. In general non- blocking-receive is simple but complex to implement due to messages arriving out-of-order in the background buffer 53
  54. 54. Message destinations: Typically: send(IP, port#, buffer) – a many-to-one (many senders to a single receiving port), except multicast, which is many-to-group. Possibility: receiving process can have many ports for different message types Server processes usually publish their service-ports for clients Reliability validity: messages are guaranteed to be delivered despite a reasonable number of packets being dropped or lost Integrity: messages arrive uncorrupted and without duplication Ordering the messages be delivered in sender order 54
  55. 55. Endpoint for communication between processes Both forms of communication (UDP and TCP ) use the socket abstraction Each computer has 216 possible ports available to local processes for receiving messages a process cannot share ports with other processes on the same computer Socket message agreed port any port socketsocket Internet address = 138.37.88.249Internet address = 138.37.94.248 other ports client server 55
  56. 56. App 3App 2App 1 Sockets The application level API to TCP & UDP Allows users to write application “protocol objects” which sit on top of TCP and UDP. Execution of TCP/UDP/IP are in kernel space. Sockets provide the bridge to user space. In Java provides 3 types of sockets DatagramSocket – for UDP ServerSocket – for TCP server Socket – endpoint of a TCP stream IP TCP UDP socket socket socket 56
  57. 57. Java API for IPs For either TCP or UDP, Java provides an InetAddress class, which contains a method: getByName(DNS) for obtaining IP addresses, irrespective of the number of address bits (32 bits for IPv4 or 128 bits for IPv6) by simply passing the DNS hostname. For example, a user Java code invokes: InetAddress aComputer = InetAddress.getByName(“nsfcopire.spsu.edu”); The class encapsulates the details of representing the IP address 57
  58. 58. UDP datagram communication UDP Datagram communication Steps: Client finds an available port for UDP connection Client binds the port to local IP (obtained from InetAddress.getByName(DNS) ) Server finds a designated port, publicizes it to clients, and binds it to local IP Server process issues a receive method and gets the IP and port # of sender (client) along with the message 58
  59. 59. Issues Message size – set to 8KByte for most, general protocol support 216 bytes, possible truncation if receiver buffer is smaller than message size Blocking – send is non-blocking and operation returns if message gets pass the UDP and IP layers; receive is blocking. Timeouts – reasonably large time interval set on receiver sockets to avoid indefinite blocking Receive from any – no specification of sources (senders), typically many-to- one, but one-to-one is possible by a designated send-receive socket (know by both C/S) 59
  60. 60. Failure model omission failure: message be dropped due to checksum error or no buffer space at sender side or receiver side ordering: message be delivered out of sender order application maintains the reliability of UDP communication channel by itself (ACK) 60
  61. 61. DatagramPacket Methods It provides a constructor to make an array of bytes comprising Message content ,Length of message,Internet address ,Local port number getAddress():Returns the IP address of the machine to which this datagram is being sent or from which the datagram was received. getData():Returns the data received or the data to be sent. getLength():Returns the length of the data to be sent or the length of the data received. getPort():Returns the port number on the remote host to which this datagram is being sent or from which the datagram was received. It provides another similar constructor for receiving a message. Java API for UDP datagrams 61
  62. 62. DatagramSocket Methods This class supports sockets for sending and receiving UDP datagram. getLocalAddress():Gets the local address to which the socket is bound. getLocalPort():Returns the port number on the local host to which this socket is bound. getSoTimeout():Retrive setting for SO_TIMEOUT. receive(DatagramPacket):Receives a datagram packet from this socket. send(DatagramPacket):Sends a datagram packet from this socket.
  63. 63. Datagram Sockets Creating a datagram socket DatagramSocket soc = new DatagramSocket(port-no.) Creating a datagram packect DatagramPacket p = new DatagramPacket(byte[] data, int len, InetAddress dest, int dest-port); Sending a packet soc.send( p ) Receiving a packet q = new DatagramPacket(buff, len); soc.receive( q ); 63
  64. 64. UDP client sends a message to the server and gets a reply import java.net.*; import java.io.*; public class UDPClient{ public static void main(String args[]){ // args give message contents and server hostname DatagramSocket aSocket = null; try { aSocket = new DatagramSocket(); byte [] m = args[0].getBytes(); InetAddress aHost = InetAddress.getByName(args[1]); int serverPort = 6789; DatagramPacket request = new DatagramPacket(m, args[0].length(), aHost, serverPort); aSocket.send(request); byte[] buffer = new byte[1000]; DatagramPacket reply = new DatagramPacket(buffer, buffer.length); aSocket.receive(reply); System.out.println("Reply: " + new String(reply.getData())); }catch (SocketException e){System.out.println("Socket: " + e.getMessage()); }catch (IOException e){System.out.println("IO: " + e.getMessage());} }finally {if(aSocket != null) aSocket.close();} } } 64
  65. 65. UDP server repeatedly receives a request and sends it back to the client import java.net.*; import java.io.*; public class UDPServer{ public static void main(String args[]){ DatagramSocket aSocket = null; try{ aSocket = new DatagramSocket(6789); byte[] buffer = new byte[1000]; while(true){ DatagramPacket request = new DatagramPacket(buffer, buffer.length); aSocket.receive(request); DatagramPacket reply = new DatagramPacket(request.getData(), request.getLength(), request.getAddress(), request.getPort()); aSocket.send(reply); } }catch (SocketException e){System.out.println("Socket: " + e.getMessage()); }catch (IOException e) {System.out.println("IO: " + e.getMessage());} }finally {if(aSocket != null) aSocket.close();} } } 65
  66. 66. The API to the TCP provide the abstraction of a stream of bytes to which data may be written and from which data may be read Hidden network characteristics message sizes (how much data to write/read) lost messages (ACK) flow control (match the speed) message duplication and ordering (identifier with each IP Packet) message destinations (establish connection) Once connection is established, no need of address & ports TCP stream communication 66
  67. 67. While connection establishing: client/server Client :Connect request, Server :Accept request, becomes Peers Client: creates stream socket, bind to a port, and makes a connect request to server Server: creates a listening socket, bind to a server port, and waits for clients Use of TCP Many services that run over TCP connections, with reserved port number are: HTTP (Hypertext Transfer Protocol) FTP (File Transfer Protocol) Telnet SMTP (Simple Mail Transfer Protocol)
  68. 68. issues related to stream communication Matching of data items: agree to the contents of the transmitted data Blocking: send blocked until the data is written in the receiver’s buffer, receive blocked until the data in the local buffer becomes available Threads: server create a new thread for every connection failure model integrity and validity have been achieved by checksum, sequence number, timeout and retransmission in TCP protocol connection could be broken due to unknown failures Can’t distinguish between network failure and the destination process failure Can’t tell whether its recent messages have been received or not 68
  69. 69. ServerSocket Intended for use by a server to create a socket at a server port for listening connect request from clients. Has an accept method that is listen for connect requests from clients Socket Used by a pair of processes with a stream connection. Has a constructor used by the client to create its stream socket and send a connect request to the specified remote server. Provides getInputStream and getOutputStream methods for accessing the two input and output streams associated with a socket. read and write methods are used for bytes reading from and writing to InputStream and OutputStream respectively. Java API for TCP Streams 69
  70. 70. TCP byte stream sockets Server socket – waits for connections soc = new ServerSocket( port-no ); Socket newsoc = soc.accept( ); Client socket – connects to server client = new Socket(server-addr, server-port); When connected, get streams InputStream in = client.getInputStream( ); OutputStream out = client.getOutputStream( ) Server side Server socket Client Side client socket 1. Contact server New socket 2. Create new Stream socket 3. Establish stream communication 70
  71. 71. TCP client makes connection to server, sends request and receives reply import java.net.*; import java.io.*; public class TCPClient { public static void main (String args[]) { // arguments supply message and hostname of destination Socket s = null; try{ int serverPort = 7896; s = new Socket(args[1], serverPort); DataInputStream in = new DataInputStream( s.getInputStream()); DataOutputStream out = new DataOutputStream( s.getOutputStream()); out.writeUTF(args[0]); // UTF is a string encoding see Sn 4.3 String data = in.readUTF(); System.out.println("Received: "+ data) ; }catch (UnknownHostException e){ System.out.println("Sock:"+e.getMessage()); }catch (EOFException e){System.out.println("EOF:"+e.getMessage()); }catch (IOException e){System.out.println("IO:"+e.getMessage());} }finally {if(s!=null) try {s.close();}catch (IOException e){System.out.println("close:"+e.getMessage());}} } }
  72. 72. TCP server makes a connection for each client and then echoes the client’s request import java.net.*; import java.io.*; public class TCPServer { public static void main (String args[]) { try{ int serverPort = 7896; ServerSocket listenSocket = new ServerSocket(serverPort); while(true) { Socket clientSocket = listenSocket.accept(); Connection c = new Connection(clientSocket); } } catch(IOException e) {System.out.println("Listen :"+e.getMessage());} } } // this figure continues on the next slide
  73. 73. class Connection extends Thread { DataInputStream in; DataOutputStream out; Socket clientSocket; public Connection (Socket aClientSocket) { try { clientSocket = aClientSocket; in = new DataInputStream( clientSocket.getInputStream()); out =new DataOutputStream( clientSocket.getOutputStream()); this.start(); } catch(IOException e) {System.out.println("Connection:"+e.getMessage());} } public void run(){ try { // an echo server String data = in.readUTF(); out.writeUTF(data); } catch(EOFException e) {System.out.println("EOF:"+e.getMessage()); } catch(IOException e) {System.out.println("IO:"+e.getMessage());} } finally{ try {clientSocket.close();}catch (IOException e){/*close failed*/}} } }
  74. 74. The information stored in running programs is represented as data structures, whereas the information in messages consists of sequences of bytes. External data representation An agreed standard for the representation of data structures and primitive values Marshalling (unmarshalling) The process of taking a collection of data items and assembling them into a form suitable for transmission in a message Usage: for data transmission or storing in files Two alternative approaches CORBA’s(Common Object Request Broker Architecture)common data representation , Java’s object serialization and XML External data representation and marshalling introduction 74
  75. 75. Concerned with an external representation for the structured and primitives types. 15 primitive types Short (16bit), long(32bit), unsigned short, unsigned long, float, char, Constructed types Types that composed by several primitive types Marshaling and unmarshalling operations are generated automatically from the specification of data items transmitted in a message. • The type definitions of data structures and primitive data items are described in CORBA Interface Definition Language (IDL). • CORBA interface compiler generates appropriate marshaling and unmarshalling operations from the type definitions of remote methods arguments and results CORBA’s Common Data Representation (CDR) 75
  76. 76. CORBA CDR for constructed types Type Representation sequence length (unsigned long) follow ed by elements in order string length (unsigned long) follow ed by charactersin order (can also can have wide characters) array array elements in order (no length specified because it is fixed) struct in the order of declaration of the components enumerated unsigned long (the values are specified by the order declared) union type tag followed by th e sele cted member 76
  77. 77. CORBA CDR message Person struct with value: {‘Smith’, ‘London’, 1934} 0–3 4–7 8–11 12–15 16–19 20-23 24–27 5 "Smit" "h___" 6 "Lond" "on__" 1934 index in sequence of bytes 4 bytes notes on representation length of string ‘Smith’ length of string ‘London’ unsigned long 77 The type of a data item is not given with the data representation in message It is assumed that the sender and recipient have common knowledge of the order and types of the data items in a message. •shows a message in CORBA CDR that contains the three fields of a struct whose respective types are string, string, and unsigned long. Struct Person { string name; string place; long year; };
  78. 78. Serialization (deserialization) The activity of flattening an object or a connected set of objects into a serial form that is suitable for storing on the disk or transmitting in a message Include information about the class of each object Handles: references to other objects are serialized as handles Serialization of a specific object is done by creating an instance of the class ObjectOutputStream and invoking its writeObject method with the object name as rgument. ObjectInputStream class is opened on a stream of data to deserialize an object from that stream using the readObject method. Java object serialization 78
  79. 79. •Only class name and object’s data is saved •If that data is an object, it is also saved •Each object is given a serial number •If an object has already been saved then only the serial number is saved •Methods are not saved •Static information not saved
  80. 80. Indication of Java serialized form Serialized values Person 3 1934 8-byte version number int year 5 Smith java.lang.String name: 6 London h0 java.lang.String place: h1 Explanation class name, version number number, type and name of instance variables values of instance variables Public class Person implements Serializable { private String name; private String place; private int year; public Person (String aName, String aPlace, int aYear){ name = aName; place = aPlace; year = aYear; } // followed by methods for accessing the instance variables } Person p = new Person(“Smith”, “London”, 1934); 80
  81. 81. Representation of a remote object reference Internet address port number time object number interface of remote object 32 bits 32 bits 32 bits 32 bits Remote object references are needed when a client invokes an object that is located on a remote server. A remote object reference is passed in the invocation message to specify which object is to be invoked. Remote object references must be unique over space and time. generic format for remote object references is shown 81
  82. 82. XML •Extensible markup language (XML) •User-defined tags (vs. HTML has a fixed set of tags) Ex: Person struct in XML Tag names: person id, name, place, year Element: <name>Smith</name> Attribute: id="123456789” of person Binary data need to be converted to characters (base64) <person id="123456789"> <name>Smith</name> <place>London</place> <year>1934</year> <!-- a comment --> </person >
  83. 83. XML namespace •Name clashes within an application •Namespaces: a set of names for a collection of element types and attributes •xmlns: xml namespace •pers: name of the name space (used as a prefix) • http://www.cdk4.net/person :location of schema <person pers:id="123456789" xmlns:pers = "http://www.cdk4.net/person"> <pers:name> Smith </pers:name> <pers:place> London </pers:place > <pers:year> 1934 </pers:year> </person>
  84. 84. XML schema •Defines elements and attributes •xsd: namespace for xml schema definition <xsd:schema xmlns:xsd = URL of XML schema definitions > <xsd:element name= "person" type ="personType" /> <xsd:complexType name="personType"> <xsd:sequence> <xsd:element name = "name" type="xs:string"/> <xsd:element name = "place" type="xs:string"/> <xsd:element name = "year" type="xs:positiveInteger"/> </xsd:sequence> <xsd:attribute name= "id" type = "xs:positiveInteger"/> </xsd:complexType> </xsd:schema>
  85. 85. Request-reply protocols are designed to support client-server communication with the following characteristics: • Synchronous in normal cases because the client process blocks until the reply arrives from the server. • Acknowledgments are not required since requests are followed by replies. • Flow control is not needed due to small amounts of data transferred • Connection establishment involves only two extra pair of messages. • Usually implemented over UDP datagrames but can also applied over TCP streams. 85 Client –Server Communication Request-reply protocol
  86. 86. 86 Request-reply protocol is based on a triple of communication primitives: doOperation. getRequest. sendReply. Request ServerClient doOperation (wait) (continuation) Reply message getRequest execute method message select object sendReply
  87. 87. doOperation: Used by clients to send a request message for invoking remote operations. Its arguments specify the remote object reference, the method to be invoked and the arguments of that method. Its result is a remote method reply. getRequest: Used by a server process to acquire service request messages. sendReply: Used by a server process to send the reply messages to clients Its arguments are the reply, IP address and port of designated client.
  88. 88. The structure of a request or a reply message as follow: Message type: indicate whether the message is a request or a reply. Message ID: contain a request message identifier generated by the client and the server refer to it into the corresponding reply message to enable the client to check that the reply is the result of the current request. Object reference: the reference of remote object. Method ID: the identifier for the method to be invoked. Arguments of the invoked method messageType requestId objectReference methodId arguments int (0=Request, 1= Reply) int RemoteObjectRef int or Method array of bytes
  89. 89. Failure model Problems: Omission failures. No guarantee of delivery in order. Timeout Use timeout and resend request when timeout expires and reply hasn’t arrived Duplicate request messages: filter out duplicates by requestID Idempotent (unchangable) operations: same result obtained on every invocation. Non-idempotent operations: re-send result stored from previous request, requires maintenance of a history of replies. History: server contains a record of reply messages that have been transmitted to avoid re-execution of operations. 89
  90. 90. RPC exchange protocol Three protocols are used for implementing various types of RPC. In the R protocol, a single request message is sent by the client to the server.The R protocol may be used when there is no value to be returned from the remote method. The RR protocol is useful for most client-server exchanges because it is based on request-reply protocol. RRA protocol is based on the exchange of three messages: request-reply- acknowledge reply. Name Messagessent by Client Server Client R Request RR Request Reply RRA Request Reply Acknowledge reply 90
  91. 91. HTTP is a request-reply protocol for the exchange of network resources between web clients and web servers. HTTP protocol steps are: I. Connection establishment between client and server at the default server port or at a port specified in the URL. II.client sends a request. III.server sends a reply. IV.connection close. HTTP specifies: Messages,Methods,Arguments,Results,Rules for representing data in messages. HTTP: an example of a request – reply protocol 91
  92. 92. HTTP 1.1 uses persistent connections. Persistent connections are connections that remains open over a series of request-reply exchanges between client and server. Marshalling Request and replies are marshalled into messages as ASCII text string Resources are represented as byte sequences and may be compressed MIME (Multipurpose Internet Mail Extensions) to send text, images and so on.
  93. 93. HTTP - methods GET: Requests the resource whose URL is given as argument. HEAD: Similar to GET, but no data returned. POST: Can deal with data supplied with the request. PUT: data supplied in the request is stored with the given URL as its identifier either as modification of the an existing resource or as a new resource. DELETE: Server deletes resource identified by the URL. OPTIONS: Server supplies client with list of methods that it allows. TRACE: Server sends the request back. 93
  94. 94. HTTP request / reply messages GET //www.dcs.qmw.ac.uk/index.html HTTP/ 1.1 URL or pathnamemethod HTTP version headers message body HTTP/1.1 200 OK resource data HTTP version status code reason headers message body Request/Reply: A request message specifies the method name, the URL of resource, the protocol version, some headers and an optional message body (not needed in case of data resource). A reply message specifies the protocol version, a status code and reason, some header and an optional message body. 94
  95. 95. Group Communication • Multicasting: operation to send a single message from one process to each of the members of a predefined group of processes transparently. •useful for: 1. fault tolerance based on replicated services requests multicast to servers, some may fail, the client will be served 2. discovering services multicast to find out who has the services 3. better performance through replicated data multicast updates 4. event notification new items arrived, advertising services
  96. 96. IP Multicast – group communication A multicast group is specified by a class D Internet address Sender is unaware of the identities of the individual recipients Available only via UDP The membership of a group is dynamic It is possible to send datagram to a multicast group without being a member and join or leave the group at any time. IPv4 • Multicast IP routers 1. IP packets can be multicast both on local network and on the wider Internet. 2. Local multicast uses local network such as Ethernet. 3. To limit the distance of propagation of a multicast datagram, the sender can specify the number of routers it is allowed to pass- called the time to live, or TTL for short. 96
  97. 97. • Multicast address allocation 1. Multicast addressing may be permanent or temporary. 2. Permanent groups. 3. Multicast addressing by temporary groups must be created before use and cease to exit when all members have left. Java API to IP multicast 1. The Java API provides a datagram interface to IP multicast through the class MulticastSocket, which is a subset of DatagramSocket with the additional capability of being able to join multicast groups. 2. The class MulticastSocket provides two alternative constructors , allowing socket to be creative to use either a specified local port, or any free local port.
  98. 98. Multicast peer joins a group and sends and receives datagrams import java.net.*; import java.io.*; public class MulticastPeer{ public static void main(String args[]){ // args give message contents & destination multicast group (e.g. "228.5.6.7") MulticastSocket s =null; try { InetAddress group = InetAddress.getByName(args[1]); s = new MulticastSocket(6789); s.joinGroup(group); byte [] m = args[0].getBytes(); DatagramPacket messageOut = new DatagramPacket(m, m.length, group, 6789); s.send(messageOut); 98
  99. 99. // get messages from others in group byte[] buffer = new byte[1000]; for(int i=0; i< 3; i++) { DatagramPacket messageIn = new DatagramPacket(buffer, buffer.length); s.receive(messageIn); System.out.println("Received:" + new String(messageIn.getData())); } s.leaveGroup(group); }catch (SocketException e){System.out.println("Socket: " + e.getMessage()); }catch (IOException e){System.out.println("IO: " + e.getMessage());} }finally {if(s != null) s.close();} } } 99
  100. 100. UNIX socket Datagram communication Datagram Socket Bind Sendto recvfrom Stream communication stream socket , bind Accept Connect Write and read 100 Case Study
  101. 101. Sockets used for datagrams ServerAddress and ClientAddress are socket addresses Sending a message Receiving a message bind(s, ClientAddress) sendto(s, "message", ServerAddress) bind(s, ServerAddress) amount = recvfrom(s, buffer, from) s = socket(AF_INET, SOCK_DGRAM, 0)s = socket(AF_INET, SOCK_DGRAM, 0) 101
  102. 102. Sockets used for streams Requesting a connection Listening and accepting a connection bind(s, ServerAddress); listen(s,5); sNew = accept(s, ClientAddress); n = read(sNew, buffer, amount) s = socket(AF_INET, SOCK_STREAM,0) connect(s, ServerAddress) write(s, "message", length) s = socket(AF_INET, SOCK_STREAM,0) ServerAddress and ClientAddress are socket addresses 102
  103. 103. Introduction Communication between distributed objects Remote procedure call Events and notifications Java RMI case study Distributed objects and remote invocation
  104. 104. Interface Specifies accessible procedures and variables • Interface in distributed system Can’t access variables directly Input argument and output argument Pointers can’t be passed as arguments or returned results RPC’s Service interface specification of the procedures of the server, defining the types of the input and output arguments of each procedure RMI’s Remote interface Specification of the methods of an object that are available for objects in other processes, defining the types of them. may pass objects or remote object references as arguments or returned result Interfaces
  105. 105. An Interface Definition Language, or IDL, is a formal specification that defines a remote module’s: Parameters Arguments Types Ex:java RMI
  106. 106. CORBA IDL example //In file Person.idl struct Person { string name; string place; long year; } ; interface PersonList { readonly attribute string listname; void addPerson(in Person p) ; void getPerson(in string name, out Person p); long number(); }; remote interface remote interface defines methods for RMI CORBA has a struct parameters are in, out or inout
  107. 107. each process contains objects, some of which can receive remote invocations, others only local invocations those that can receive remote invocations are called remote objects objects need to know the remote object reference of an object in another process in order to invoke its methods. the remote interface specifies which methods can be invoked remotely invocation invocation remote invocation remote local local local invocation invocation A B C D E F Communication Between Distributed Objects Distributed Objects and Remote Invocation-RMI
  108. 108. interface remote m1 m2 m3 m4 m5 m6 Data implementation remoteobject { of methods Remote object and remote interface • Remote interface: – specifies the methods of an object available for remote invocation – an interface definition language (or IDL) is used to specify remote interfaces. E.g. CORBA IDL,Java RMI – Java RMI would have a class for Person, but CORBA has a struct
  109. 109. Fault Tolerance Measures Retry Request Message Determines if the requesting system should retransmit a request until a response is received Duplicate Filtering Determines if the same request received multiple times should be filtered down to one request, or if all requests should be re-executed Retransmission of Results Determines if the results from requests should be saved and resent instead of re-executing a request
  110. 110. RMI Invocation Semantics 1. Invocation semantics depend upon implementation of Request- Reply protocol used by R MI 2. Maybe, At-least-once, At-most-once Transparency 1. remote invocations should be made transparent in the sense that syntax of a remote invocation is the same as the syntax of local invocation (access transparency) but programmers should be able to distinguish between remote and local objects by looking at their interfaces, e.g. in Java RMI, remote objects implement the Remote interface Design Issues of RMI
  111. 111. Maybe Invocation Semantics Remote method requests may be executed once or not at all Used when request messages are not resent when a fault occurs Requesting system cannot tell if the server executed the request before the fault occurred. At-Least-Once Invocation Semantics Requesting system receives either a result or an exception Server will re-execute a request for each retry request message received Not appropriate when executing the same method multiple times will alter the result At-Most-Once Invocation Semantics Requesting system either receives a result or an exception If the requesting system receives a result, it is because the server executed the result exactly once Used when duplicate request messages are filtered out
  112. 112. object A object B skeleton Request proxy for B Reply CommunicationRemote Remote referenceCommunication modulemodulereference module module for B’s class & dispatcher remote client server The architecture of remote method invocation
  113. 113. Proxy - makes RMI transparent to client.Marshals requests and unmarshals results. Forwards request. Communication Module: 1. Transmits request and reply messages 2. Concerned with the Message Type, requestId, and the remote reference of the object 3. Specifies the Invocation Semantics (Ex: At-Most-Once) Skeleton - implements methods in remote interface. Unmarshals requests and marshals results. Invokes method in remote object.
  114. 114. RMI software - between application level objects and communication and remote reference modules Remote reference module 1. Translates the remote object reference in a message to a local object message and vice versa 2. Creates new remote object references 3. Maintains the local to remote object references in a Remote Object Table Dispatcher - gets request from communication module and invokes method in skeleton (using methodID in message).
  115. 115. Distributed Objects and Remote Invocation – RPC • Similar to RMI , a client program calls a procedure in another program running in a server process. • may be implemented to have one of the choices of invocation semantics – at-least-once, at-most-once are generally chosen. • Implemented using request-reply protocol . • only addresses procedure calls. • A client that accesses a server includes one stub procedure for each procedure in the service interface. • A client stub procedure is similar to a proxy method of RMI . • A server stub procedure is similar to a skeleton method of RMI
  116. 116. Sun RPC Client-server communication in the SUN NFS (network file system) Also called ONC (Open Network Computing) RPC In other unix OS as well UDP or TCP Interface Definition Language (IDL) initially XDR is for data representation, extended to be IDL less modern than CORBA IDL and Java program numbers instead of interface names procedure numbers instead of procedure names single input parameter (structs) rpcgen: compiler for XDR client stub server main procedure, dispatcher, and server stub XDR marshalling, unmarshaling
  117. 117. const MAX = 1000; typedef int FileIdentifier; typedef int FilePointer; typedef int Length; struct Data { int length; char buffer[MAX]; }; struct writeargs { FileIdentifier f; FilePointer position; Data data; }; struct readargs { FileIdentifier f; FilePointer position; Length length; }; program FILEREADWRITE { version VERSION { void WRITE(writeargs)=1; 1 Data READ(readargs)=2; 2 }=2; } = 9999;
  118. 118. binding (registry) local binder--portmapper server registers its program/version/port numbers with portmapper client contacts the portmapper at a fixed port with program/version numbers to get the server port different instances of the same service can be run on different computers--different ports authentication request and reply have additional fields unix style (uid, gid), shared key for signing, Kerberos
  119. 119. Distributed Objects and Remote Invocation -Events and notifications Distributed event-based systems allow objects at different locations to be notified of events taking place in an object. This is achieved through the publish subscribe paradigm. Notifications are objects that represent events. Events may be of different types. Each event having several attributes and both are notifications used in subscription and notifications.
  120. 120. Dealer’s computer Information provider Dealer External source External source Information provider Dealer Dealer Dealer Notification Notification Notification Notification Notification Notification Notification Notification Dealer’s computer Dealer’s computerDealer’s computer Notification Notification subscribers: dealers in stock exchange
  121. 121. subscriberobserverobject of interest Event service object of interest object of interest observer subscriber subscriber 3. 1. 2. notification notification notification notification [3: observer checks for changes; object of interest is not part of the event service]
  122. 122. Observers An Observer is one or more objects that sits between an object of interest and the subscribers that can filter and massage event notifications before they reach subscribers. Observers can provide: Forwarding: Objects of interest send a single notification to an observer. The observer handles sending bulk notification, letting the object of interest continue processing without delay Filtering: Observes can use logic to only send a handful of notification to subscribers Notification Mailboxes: Observers can store notification until subscribers are ready to consume the messages while the object of interest can continue processing unhindered
  123. 123. possible event notification schemes: 1) Object of interest publishes notifications directly to subscribers 2) Object of interest publishes to an observer. The observer handles publication to subscribers 3) Object of interest is outside the local event service. An observer periodically queries the object of interest to see if an event has happened. The observer will publish notifications to subscribers if it finds an event. The participants are: the object of interest, an event, a notification, a subscriber, an observer object, and a publisher.
  124. 124. Java RMI it is a library that allows a Java program on one machine to call functions (methods) inside an object on a remote machine. A remote object must implement Remote interface. In the example we do the followings: Define the remote interface Implement the server Implement the client Compile the source files Start the Java RMI registry, server, and client Ex:Shared whiteboard It allows a group of users to share a common view of a drawing surface containing graphical objects. The server maintains the current state of a drawingClients can poll the server about the latest shape of a drawing. The server attaches version numbers to new arriving shapes.
  125. 125. Remote Interface Remote interfaces are defined by extending an interface called Remote provided in the java.rmi package. Figure given shows an example of two remote interface called Shape and ShapeList. In this example, GraphicalObject is a class that holds the state of a graphical object. GraphicalObject must implement the Serializable interface. Ordinary and remote objects can appear as input and output arguments.
  126. 126. import java.rmi.*; import java.util.Vector; public interface Shape extends Remote { int getVersion() throws RemoteException; GraphicalObject getAllState() throws RemoteException; } public interface ShapeList extends Remote { Shape newShape(GraphicalObject g) throws RemoteException; Vector allShapes() throws RemoteException; int getVersion() throws RemoteException; }
  127. 127. Parameter and result passing 1. Passing remote objects The result is passed by (object) reference. In line2, the return value of the method newShape is defined as shape - a remote interface. When a remote object reference is received, an RMI can be issued on the object refered to by this reference. 2. Passing non-remote objects The result is passed by value. A new object is created locally, with the state differing from the original object.
  128. 128. Downloading of Classes Classes can be transferred from one Java VM to another. Parameters are passed by value or by reference. If the recipient does not yet possess the class of an object passed by value, its code is downloaded automatically. If the recipient of a remote object reference does not yet possess the class for a proxy, its code is downloaded automatically. RMIregistry The RMIregistry is the binder for Java RMI. The RMIregistry runs on every server that hosts remote objects. It maps local object names of the form //computerName:port/objName to object references. This service is not a global service. Clients need to query a particular host to get reference.
  129. 129. void rebind (String name, Remote obj):This method is used by a server to register the identifier of a remote object by name, void bind (String name, Remote obj):This method can alternatively be used by a server to register a remote object by name, but if the name is already bound to a remote object reference an exception is thrown. void unbind (String name, Remote obj):This method removes a binding. Remote lookup (String name):This method is used by clients to look up a remote object by name String [] list():This method returns an array of Strings containing the names bound in the registry. Naming Class of Java RMI Registry
  130. 130. import java.rmi.*; public class ShapeListServer{ public static void main (String args[]){ System.setSecurityManager (new RMISecurityManager()); try{ ShapeList aShapeList = new ShapeListServant(); 1 Naming.rebind(“Shape List”, aShapeList); 2 System.out.println(“ShapeList server ready”); }catch(Exception e) { System.out.println(“ShapeList server main” + e.getMessage());} } } Figure 6. Java class ShapeListServer with main method
  131. 131. Security Manager implements various security policies for client accesses Main method 1: create instance of ShapeListServant 2: binds name "ShapeList" to newly created instance in RMI Registry. ShapeListServant implements ShapeList Figure 7 gives an outline of the class ShapeListServant. 1: UnicastRemoteObject - objects that live only as long as creating process 2: factory method - client can request creation of a new object
  132. 132. import java.rmi.*; import java.rmi.server.UnicastRemoteObject; import java.util.Vector; public class ShapeListServant extends UnicastRemoteObject implements ShapeList{ private Vector theList; // contains the list of Shapes private int version; public ShapeListServant()throws RemoteException{….} public Shape newShape(GraphicalObject g) throws RemoteException{ version++ Shape s = new ShapeServant(g, version); theList.addElement(s); return s; } public Vector allShapes() throws RemoteException{…} public int getVersion() throws RemoteException{…} }
  133. 133. Client program A simplified client for the ShapeList sever is illustrated in Figure 8. polling loop: • 1: look up remote reference • 2: invoke allShapes() in remote object
  134. 134. import java.rmi.*; import java.rmi.server.*; import java.util.Vector; public class ShapeListClient{ public static void main(String args[]){ System.setSecurityManager(new RMISecurityManager()); ShapeList aShapeList = null; try{ aShapeList = (ShapeList) Naming.lookup(“//bruno.ShapeList”); Vector sList = aShapeList.allShapes(); } catch(RemoteException e) {System.out.println(e.getMessage()); } catch(Exception e) {System.out.println(“Client:”+e.getMessage());} } } Figure 8. Java client of ShapeList
  135. 135. The source files for this example can be compiled as follows: javac Hello.java Server.java Client.java Start the Java RMI registry : rmiregistry Start the Server: java Server Start the Client: java Client
  136. 136. Figure 9. Classes supporting Java RMI the inheritance structure of the classes supporting Java RMI servers.

×