This document provides an overview of the syllabus for the course 18CS3040 Distributed Systems. It introduces the topics that will be covered in each of the 6 modules, including introduction to distributed systems, distributed objects and file systems, name services and global states, distributed transaction and concurrency control, replication and distributed shared memory, and distributed multimedia systems. It also lists the course objectives and outcomes, such as describing distributed system models, distinguishing inter-process communication mechanisms, and evaluating efficient distributed systems.

1. 18CS3040
DISTRIBUTED SYSTEMS
Module – I
Dr.A.Kathirvel, Professor, Computer Science and Engg.
Karunya Institute of Technology and Sciences, CBE

2. 18CS3040 Distributed Systems
Module I
Introduction to Distributed systems - Examples of
distributed systems, resource sharing and the
web, challenges - System model - introduction -
architectural models - fundamental models -
Introduction to inter-process communications -
API for Internet protocol - external data.
Dr.A.Kathirvel, Professor,
DCSE, KITS
kathirvel@karunya.edu

3. Module I
Introduction to Distributed systems -
Examples of distributed systems, resource
sharing and the web, challenges - System
model - introduction - architectural models
- fundamental models - Introduction to
inter-process communications - API for
Internet protocol - external data.
Book Referred: Coulouris, Dollimore, Kindberg and Blair
Distributed Systems:Concepts and Design Edition 5, Addison-Wesley 2012

4. Agenda
Syllabus
Text Book
Why Distributed System?
Applications

5. Course Objectives: Enable the student to
1.describe the concepts of resource sharing, system modeling, inter-process
communication and file systems in distributed systems.
2.apply the skill sets in name services and global states, distributed transaction
and concurrency control model, distributed shared memory and distributed
multimedia system.
3.compare the replication techniques and shared memory schemes in
distributed systems.
Course Outcomes: The student will be able to
1.describe the distributed system models.
2.distinguish between different inter-process communication mechanisms and
their application scenarios.
3.illustrate a transaction and concurrency control scheme for a real time
application.
4.compare the various file system architectures used in distributed systems
5.construct a real time distributed system with suitable IPC, event coordination,
file management, name service, transaction and concurrency control
mechanisms.
6.evaluate an efficient distributed system and its qualities

6. Module I - Introduction:
Introduction to Distributed systems - Examples of distributed systems, resource sharing and the
web, challenges - System model - introduction - architectural models - fundamental models -
Introduction to inter-process communications - API for Internet protocol - external data.
Module II - Distributed objects and file systems:
Introduction - Communication between distributed objects - Remote procedure call - Events and
notifications - case study - Operating system support - introduction - operating system layer -
protection - process and threads - communication and invocation - architecture - Introduction to
DFS - File service architecture - Sun network file system - Andrew file system - Enhancements
and future developments.
Module III - Name services and global states:
Introduction to Name Services- Name services and DNS - Directory services - case study -
Introduction to peer-to-peer systems - Napster and its legacy - Peer-to-peer middleware - Routing
overlays - case study - Introduction to time and global states - Clocks, Events and Process states -
Synchronizing physical clocks - Logical time and logical clocks - Global states - Distributed
debugging.
Module IV - Distributed transaction and concurrency control:
Introduction to coordination and agreement - Distributed mutual exclusion - elections - multicast
communication - consensus and related problems - Introduction to transaction and concurrency
control - Transactions - Nested transaction - Locks - Optimistic concurrency control - Timestamp
ordering - Comparison of methods for concurrency control - Introduction to distributed
transactions - Flat and nested distributed transactions - Atomic commit protocols - Concurrency
control in distributed transactions - Distributed deadlocks - Transaction recovery.

7. Module V - Replication, distributed shared memory: Introduction to Replication
- System model and group communications - Fault tolerant services - Highly
available services - Transactions with replicated data - Distributed shared memory -
design and implementation issues - sequential consistency and Ivy - release
consistency and Munin - other consistency models-
Module 6 - distributed multimedia system
Distributed multimedia systems - characteristics of multimedia data - quality of
service management - resource management - stream adaptation - case study.
Reference Books:
1. George Coulouris, Jean Dollimore, and Tim Kindberg, Distributed Systems
Concepts and Design, Fifth edition, Addison-Wesley, 2011. ISBN:
0132143011.
2. Andrew S. Tanenbaum and Maartenvan Steen, Distributed Systems: Principles
and Paradigms, Second edition, Prentice Hall, 2006. ISBN: 0132392275.
3. William Buchanan, Distributed Systems and Networks, McGraw-Hill,
2001.ISBN: 0077095839.
4. Pradeep K. Sinha, Distributed Operating Systems: Concepts and Design,
Wiley- IEEE Press, 1996. ISBN: 0780311191.

9. Characterization and Design
Chapter 1
9

10. Outline
1. What is a Distributed System
2. Examples of Distributed Systems
3. Common Characteristics
4. Basic Design Issues
5. Summary
10

11. 1. Distributed System Types
Fully
Distributed
Processors
Control
Fully replicated
Not fully replicated
master directory
Local data,
local directory
Master-slave
Autonomous
transaction based
Autonomous
fully cooperative
Homog.
special
purpose
Heterog.
special
purpose
Homog.
general
purpose
Heterog.
general
purpose
11

12. 1. What is a Distributed System?
Definition: A distributed system is one in which
components located at networked computers
communicate and coordinate their actions only by
passing messages. This definition leads to the
following characteristics of distributed systems:
• Concurrency of components
• Lack of a global ‘clock’
• Independent failures of components 12

13. Centralized System Characteristics
One component with non-autonomous parts
Component shared by users all the time
All resources accessible
Software runs in a single process
Single point of control
Single point of failure
13

14. Distributed System Characteristics
Multiple autonomous components
Components are not shared by all users
Resources may not be accessible
Software runs in concurrent processes on different
processors
Multiple points of control
Multiple points of failure
14

15. Applications
Most people know about the Internet (a
computer network) through applications
World Wide Web
Email
Online Social Network
Streaming Audio Video
File Sharing
Instant Messaging
…
15

16. Examples of Distributed Systems
Local Area Network and Intranet
Database Management System
Automatic Teller Machine Network
Internet/World-Wide Web
Mobile and Ubiquitous Computing
16
16

17. Local Area Network
the rest of
email server
Web server
Desktop
computers
File server
router/firewall
print and other servers
other servers
print
Local area
network
email server
the Internet
17
17

18. Database Management System
18
18

19. Automatic Teller Machine Network
19
19

20. Internet
intranet
ISP
desktop computer:
backbone
satellite link
server:
%
network link:
%
%
%
20
20

21. Fig1.6: Growth of the Internet
(computers and web servers)
Date Computers Web servers Percentage
1993, July 1,776,000 130 0.008
1995, July 6,642,000 23,500 0.4
1997, July 19,540,000 1,203,096 6
1999, July 56,218,000 6,598,697 12
2001, July 125,888,197 31,299,592 25
42,298,371
2003, July
2005, July
~200,000,000
353,284,187 67,571,581
21
19
21

22. World-Wide-Web
22
22

23. Web Servers and Web Browsers
Internet
Browsers
Web servers
www.google.com
www.uu.se
www.w3c.org
Protocols
Activity.html
http://www.w3c.org/Protocols/Activity.html
http://www.google.comlsearch?q=lyu
http://www.uu.se/
File system of
www.w3c.org
23
23

24. Mobile and Ubiquitous Computing
Laptop
Mobile
Printer
Camera
Internet
Host intranet Home intranet
GSM/GPRS
Wireless LAN
phone
gateway
Host site
24
24

25. 25
Selected application domains and
associated networked applications
Finance and commerce eCommerce e.g. Amazon and eBay, PayPal, online banking
and trading
The information society Web information and search engines, ebooks, Wikipedia; social
networking: Facebook and MySpace.
Creative industries and
entertainment
online gaming, music and film in the home, user-generated
content, e.g. YouTube, Flickr
Healthcare health informatics, on online patient records, monitoring
patients
Education e-learning, virtual learning environments; distance learning
Transport and logistics GPS in route finding systems, map services: Google Maps,
Google Earth
Science The Grid as an enabling technology for collaboration between
scientists
Environmental
management
sensor technology to monitor earthquakes, floods or tsunamis

26. Common Characteristics
What are we trying to achieve when we construct a
distributed system?
Certain common characteristics can be used to assess
distributed systems
Heterogeneity
Openness
Security
Scalability
Failure Handling
Concurrency
Transparency
26
26

27. 1 Heterogeneity
Variety and differences in
Networks
Computer hardware
Operating systems
Programming languages
Implementations by different developers
Middleware as software layers to provide a programming
abstraction as well as masking the heterogeneity of the
underlying networks, hardware, OS, and programming
languages (e.g., CORBA, COM).
Mobile Code to refer to code that can be sent from one
computer to another and run at the destination (e.g., Java
applets and Java virtual machine). 27
27

28. 2 Openness
Openness is concerned with extensions and
improvements of distributed systems.
Detailed interfaces of components need to be
published.
New components have to be integrated with
existing components.
Differences in data representation of interface
types on different processors (of different vendors)
have to be resolved.
28
28

29. 3 Security
 Confidentiality
 Integrity
 Availability
 In a distributed system, clients send requests to access data managed
by servers, resources in the networks:
 Doctors requesting records from hospitals
 Users purchase products through electronic commerce
 Security is required for:
 Concealing the contents of messages: security and privacy
 Identifying a remote user or other agent correctly (authentication)
 New challenges:
 Denial of service attack
 Security of mobile code
29
29

30. 4 Scalability
 Adaptation of distributed systems to
accommodate more users
respond faster (this is the hard one)
 Usually done by adding more and/or faster processors.
 Components should not need to be changed when scale of a system
increases.
 Design components to be scalable!
 Design Challenges
Controlling the cost pf physical resource
Controlling the performance loss
Preventing software resources running out
Avoiding performance bottlenecks
30
30

31. 5 Failure Handling (Fault Tolerance)
Hardware, software and networks fail!
Distributed systems must maintain availability
even at low levels of hardware/software/network
reliability.
Fault tolerance is achieved by
Detecting Failure
Masking Failures
Tolerating Failures
recovery
redundancy 31
31

32. 6 Concurrency
Components in distributed systems are executed
in concurrent processes.
Components access and update shared resources
(e.g. variables, databases, device drivers).
Integrity of the system may be violated if
concurrent updates are not coordinated.
Lost updates
Inconsistent analysis
32
32

33. 7 Transparency
Distributed systems should be perceived by users
and application programmers as a whole rather
than as a collection of cooperating components.
Transparency has different aspects.
These represent various properties that
distributed systems should have.
33
33

34. 7.1 Access Transparency
Enables local and remote information objects to
be accessed using identical operations.
Example: File system operations in NFS.
Example: Navigation in the Web.
Example: SQL Queries
34
34

35. 7.2 Location Transparency
Enables information objects to be accessed
without knowledge of their location.
Example: File system operations in NFS
Example: Pages in the Web
Example: Tables in distributed databases
35
35

36. Network Transparency
Access Transparency
Location Transparency
36

37. 7.3 Concurrency Transparency
Enables several processes to operate
concurrently using shared information objects
without interference between them.
Example: NFS
Example: Automatic teller machine network
Example: Database management system
37
37

38. 7.4 Replication Transparency
Enables multiple instances of information objects
to be used to increase reliability and performance
without knowledge of the replicas by users or
application programs
Example: Distributed DBMS
Example: Mirroring Web Pages.
38
38

39. 7.5 Failure Transparency
Enables the concealment of faults
Allows users and applications to complete their
tasks despite the failure of other components.
Example: Database Management System
39
39

40. 7.6 Mobility Transparency
Allows the movement of information objects
within a system without affecting the operations of
users or application programs
Example: NFS
Example: Web Pages
40
40

41. 7.7 Performance Transparency
Allows the system to be reconfigured to improve
performance as loads vary.
Example: Distributed make.
41
41

42. 7.8 Scaling Transparency
Allows the system and applications to expand in
scale without change to the system structure or
the application algorithms.
Example: World-Wide-Web
Example: Distributed Database
42
42

43. Transparencies
Access transparency: enables local and remote resources to be accessed using identical
operations.
Location transparency: enables resources to be accessed without knowledge of their
physical or network location (for example, which building or IP address).
Concurrency transparency: enables several processes to operate concurrently using
shared resources without interference between them.
Replication transparency: enables multiple instances of resources to be used to increase
reliability and performance without knowledge of the replicas by users or application
programmers.
Failure transparency: enables the concealment of faults, allowing users and application
programs to complete their tasks despite the failure of hardware or software
components.
Mobility transparency: allows the movement of resources and clients within a system
without affecting the operation of users or programs.
Performance transparency: allows the system to be reconfigured to improve
performance as loads vary.
Scaling transparency: allows the system and applications to expand in scale without
change to the system structure or the application algorithms.
43

44. Fig2.1:Generations of DS
44

45. Fig2.2:Communicating entities and
communication paradigms
45

46. Fig2.3:Clients invoke individual servers
Server
Client
Client
invocation
result
Server
invocation
result
Process:
Key:
Computer:
46

47. Fig2.4a:Peer-to-peer architecture
47

48. Fig2.4b: A service provided by multiple
servers
Server
Server
Server
Service
Client
Client
48

49. Fig2.5:Web proxy server
Client
Proxy
Web
server
Web
server
server
Client
49

50. Fig2.6: Web applets
a) client request results in the downloading of applet code
Web
server
Client
Web
server
Applet
Applet code
Client
b) client interacts with the applet
50

51. Fig2.7:Software and hardware
service layers in distributed Systems
Applications, services
Computer and network hardware
Platform
Operating system
Middleware
51

52. Fig2.8:Two-tier & three-tier architectures
52

53. Fig2.9:AJAX example: soccer score
updates
new Ajax.Request('scores.php? game=Arsenal:Liverpool’,
{onSuccess: updateScore});
function updateScore(request) {
.....
( request contains the state of the Ajax request including the
returned result.
The result is parsed to obtain some text giving the score, which
is used to update the relevant portion of the current page.)
....
}
53

54. Fig2.10:Thin clients and compute
servers
Thin
Client
Application
Process
Network computer or PC
Compute server
network
54

55. Fig2.11:The web service architectural
pattern
55

56. Fig2.12: Categories of middleware
56

57. Fig2.13:Real-time ordering of events
send
receive
send
receive
m1 m2
2
1
3
4
X
Y
Z
Physical
time
A
m3
receive receive
send
receive receive receive
t1 t2 t3
receive
receive
m2
m1
57

58. Fig2.14:Processes and channels
process p process q
Communication channel
send
Outgoing message buffer Incoming message buffer
receive
m
58

59. Fig2.15: Omission and arbitrary failures
Class of failure Affects Description
Fail-stop Process Process halts and remains halted. Other processes may
detect this state.
Crash Process Process halts and remains halted. Other processes may
not be able to detect this state.
Omission Channel A message inserted in an outgoing message buffer never
arrives at the other end’s incoming message buffer.
Send-omission Process A process completes a send,but the message is not put
in its outgoing message buffer.
Receive-omissionProcess A message is put in a process’s incoming message
buffer, but that process does not receive it.
Arbitrary
(Byzantine)
Process or
channel
Process/channel exhibits arbitrary behaviour: it may
send/transmit arbitrary messages at arbitrary times,
commit omissions; a process may stop or take an
incorrect step.
59

60. Fig2.11:Timing failures
Class of FailureAffects Description
Clock Process Process’s local clock exceeds the bounds on its
rate of drift from real time.
Performance Process Process exceeds the bounds on the interval
between two steps.
Performance Channel A message’s transmission takes longer than the
stated bound.
60

61. Fig2.17:Objects and principals
Network
invocation
result
Client
Server
Principal (user) Principal (server)
Object
Access rights
61

62. Fig2.18: The enemy
Communication channel
Copy of m
Process p Process q
m
The enemy
m’
62

63. Fig2.19: Secure channels
Principal A
Secure channel
Process p Process q
Principal B
63

64. Fig3.1:Network performance
km
64

65. Fig3.2: Conceptual layering of
protocol software
Layer n
Layer 2
Layer 1
Message sent Message received
Communication
medium
Sender Recipient
65

66. Fig3.3: Encapsulation as it is applied
in layered protocols
Presentation header
Application-layer message
Session header
Transport header
Network header
66

67. Fig3.4:Protocol layers in the ISO Open
Systems Interconnection (OSI) model
Application
Presentation
Session
Transport
Network
Data link
Physical
Message sent Message received
Sender Recipient
Layers
Communication
medium
67

68. Fig3.5: OSI protocol summary
Layer Description Examples
Application Protocols that are designed to meet the communication requirements of
specific applications, often defining the interface to a service.
HTTP, FTP , SMTP,
CORBA IIOP
Presentation Protocols at this level transmit data in a network representation that is
independent of the representations used in individual computers, which may
differ. Encryption is also performed in this layer, if required.
Secure Sockets
(SSL),CORBA Data
Rep.
Session At this level reliability and adaptation are performed, such as detection of
failures and automatic recovery.
Transport This is the lowest level at which messages (rather than packets) are handled.
Messages are addressed to communication ports attached to processes,
Protocols in this layer may be connection-oriented or connectionless.
TCP, UDP
Network Transfers data packets between computers in a specific network. In a WAN
or an internetwork this involves the generation of a route passing through
routers. In a single LAN no routing is required.
IP, ATM virtual
circuits
Data link Responsible for transmission of packets between nodes that are directly
connected by a physical link. In a WAN transmission is between pairs of
routers or between routers and hosts. In a LAN it is between any pair of hosts.
Ethernet MAC,
ATM cell transfer,
PPP
Physical The circuits and hardware that drive the network. It transmits sequences of
binary data by analogue signalling, using amplitude or frequency modulation
of electrical signals (on cable circuits), light signals (on fibre optic circuits)
or other electromagnetic signals (on radio and microwave circuits).
Ethernet base- band
signalling, ISDN
68

69. Fig3.6:Internetwork layers
Underlying network
Application
Network interface
Transport
Internetwork
Internetwork packets
Network-specific packets
Message
Layers
Internetwork
protocols
Underlying
network
protocols
69

70. Fig3.7:Routing in a wide area network
Hosts
Links
or local
networks
A
D E
B
C
1
2
5
4
3
6
Routers
70

71. Fig3.8:Routing tables for the network
in Fig3.7
Routings from D Routings from E
To Link Cost To Link Cost
A
B
C
D
E
3
3
6
local
6
1
2
2
0
1
A
B
C
D
E
4
4
5
6
local
2
1
1
1
0
Routings from A Routings from B Routings from C
To Link Cost To Link Cost To Link Cost
A
B
C
D
E
local
1
1
3
1
0
1
2
1
2
A
B
C
D
E
1
local
2
1
4
1
0
1
2
1
A
B
C
D
E
2
2
local
5
5
2
1
0
2
1
71

72. Fig3.9:Pseudo-code for RIP routing
algorithm
Send: Each t seconds or when Tl changes, send Tl on each non-faulty outgoing
link.
Receive: Whenever a routing table Tr is received on link n:
for all rows Rr in Tr {
if (Rr.link | n) {
Rr.cost = Rr.cost + 1;
Rr.link = n;
if (Rr.destination is not in Tl) add Rr to Tl;
// add new destination to Tl
else for all rows Rl in Tl {
if (Rr.destination = Rl.destination and
(Rr.cost < Rl.cost or Rl.link = n)) Rl = Rr;
// Rr.cost < Rl.cost : remote node has better route
// Rl.link = n : remote node is more authoritative
}}}
72

73. Fig3.10: Simplified view of part of a
university campus network
file
compute
dialup
hammer
henry
hotpoint
138.37.88.230
138.37.88.162
bruno
138.37.88.249
router/
sickle
138.37.95.241
138.37.95.240/29
138.37.95.249
copper
138.37.88.248
firewall
web
138.37.95.248/29
server
desktop computers 138.37.88.xx
subnet
subnet
Eswitch
138.37.88
server
server
server
138.37.88.251
custard
138.37.94.246
desktop computers
Eswitch
138.37.94
hub
hub
Student subnet
Staff subnet
other
servers
router/
firewall
138.37.94.251
☎
1000 Mbps Ethernet
Eswitch: Ethernet switch
100 Mbps Ethernet
file server/
gateway
printers
Campus
router
Campus
router
138.37.94.xx
73

74. Fig3.11:Tunnelling for IPv6 migration
A B
IPv6 IPv6
IPv6 encapsulated in IPv4 packets
Encapsulators
IPv4 network
74

75. Fig3.12: TCP/IP layers
Messages (UDP) or Streams (TCP)
Application
Transport
Internet
UDP or TCP packets
IP datagrams
Network-specific frames
Message
Layers
Underlying network
Network interface
75

76. Fig3.13:Encapsulation in a message
transmitted via TCP over an Ethernet
Application message
TCP header
IP header
Ethernet header
Ethernet frame
port
TCP
IP
76

77. Fig3.14:The programmer's conceptual
view of a TCP/IP Internet
IP
Application Application
TCP UDP
77

78. Fig3.15: Internet address structure,
showing field sizes in bits
7 24
Class A: 0 Network ID Host ID
14 16
Class B: 1 0 Network ID Host ID
21 8
Class C: 1 1 0 Network ID Host ID
28
Class D (multicast): 1 1 1 0 Multicast address
27
Class E (reserved): 1 1 1 1 unused
0
28
78

79. Fig3.16:Decimal representation of
Internet addresses
octet 1 octet 2 octet 3
Class A: 1 to 127
0 to 255 0 to 255 1 to 254
Class B: 128 to 191
Class C: 192 to 223
224 to 239
Class D (multicast):
Network ID
Network ID
Network ID
Host ID
Host ID
Host ID
Multicast address
0 to 255 0 to 255 1 to 254
0 to 255 0 to 255 0 to 255
0 to 255 0 to 255 0 to 255
0 to 255 0 to 255 1 to 254
240 to 255
Class E (reserved):
1.0.0.0 to
127.255.255.255
128.0.0.0 to
191.255.255.255
192.0.0.0 to
223.255.255.255
224.0.0.0 to
239.255.255.255
240.0.0.0 to
255.255.255.255
Range of addresses
79

80. Fig3.17:IP packet layout
data
IP address of destination
IP address of source
header
up to 64 kilobytes
Figure 3.19 IPv6 header layout
Source
address
(128 bits)
Destination
address
(128 bits)
Version (4 bits) Traffic class (8 bits) Flow label (20 bits)
Payload length (16 bits) Hop limit (8 bits)
Next header (8 bits)
80

81. Fig3.18:A typical NAT-based home
network
81

82. Fig3.20: The MobileIP routing
mechanism
Sender
Home
Mobile host MH
Foreign agent FA
Internet
agent
First IP packet
addressed to MH
Address of FA
returned to sender
First IP packet
tunnelled to FA
Subsequent IP packets
tunnelled to FA
82

83. Fig3.21:Firewall configurations
Internet
Router/
Protected intranet
a) Filtering router
Internet
b) Filtering router and bastion
filter
Internet
R/filter
c) Screened subnet for bastion R/filter Bastion
R/filter Bastion
web/ftp
server
web/ftp
server
web/ftp
server
83

84. Fig3.22: IEEE 802 network standards
IEEE No. Name Title Reference
802.3 Ethernet CSMA/CD Networks (Ethernet) [IEEE 1985a]
802.4 Token Bus Networks [IEEE 1985b]
802.5 Token Ring Networks [IEEE 1985c]
802.6 Metropolitan Area Networks [IEEE 1994]
802.11 WiFi Wireless Local Area Networks [IEEE 1999]
802.15.1 Bluetooth Wireless Personal Area Networks [IEEE 2002]
802.15.4 ZigBee Wireless Sensor Networks [IEEE 2003]
802.16 WiMAX Wireless Metropolitan Area Networks [IEEE 2004a]
84

85. Fig3.23:Ethernet ranges and speeds
10Base5 10BaseT 100BaseT 1000BaseT
Data rate 10 Mbps 10 Mbps 100 Mbps 1000 Mbps
Max. segment lengths:
Twisted wire (UTP) 100 m 100 m 100 m 25 m
Coaxial cable (STP) 500 m 500 m 500 m 25 m
Multi-mode fibre 2000 m 2000 m 500 m 500 m
Mono-mode fibre 25000 m 25000 m 20000 m 2000 m
85

86. Fig3.24: Wireless LAN configuration
LAN
Server
Wireless
LAN
Laptops
Base station/
access point
Palmtop
radio obstruction
A B C
D
E
86

87. Fig3.25: Bluetooth frame structure
SCO packets (e.g. for voice data) have a 240-bit payload containing 80
bits of data triplicated, filling exactly one timeslot.
bits: 72 18 18 18 0 - 2744
Access code Header
copy 1
Header
copy 2
Header
copy 3
Data for transmission
bits: 3 1 1 1 4 8
Destination Flow Ack Seq Type Header checksum
Address within
Piconet
= ACL, SCO,
poll, null
Header
87

88. Designing Distributed Systems
Google Case Study
88

89. Fig21.1:Outline architecture of the
original Google search engine
89

90. Fig21.2: Example Google applications
90

91. Fig21.3:Organization of the Google
physical infrastructure
(To avoid
clutter the
Ethernet
connection
s are
shown
from only
one of the
clusters to
the
external
links)
91

92. Fig21.4:The scalability problem-Google
92

93. Fig21.5:The overall Google systems
architecture
93

94. Fig21.6: Google infrastructure
94

95. Fig21.7: Protocol buffers example
95

96. Fig21.8a:Summary of design choices
related to communication-paradigmspart
96

97. Fig21.8b:Summary of design choices
related to communication paradigms-part2
97

98. Fig21.9:Overall architecture of GFS
98

99. Fig21.10:Chubby API
99

100. Fig21.11:Overall architecture of Chubby
100

101. Fig21.12: Message exchanges in
Paxos (in absence of failures) - step 1
101

102. Fig21.12:Message exchanges in Paxos
(in absence of failures) - step 2
102

103. Fig21.12:Message exchanges in
Paxos (in absence of failures)-step 3
103

104. Fig21.13:The table abstraction in Bigtable
104

105. Fig21.14:Overall architecture of Bigtable
105

106. Fig21.15:The storage architecture-Bigtable
106

107. Fig21.16:The hierarchical indexing
scheme adopted by Bigtable
107

108. Fig21.17:Summary of design choices
related to data storage and coordination

109. Fig21.18:Examples of the use of MapReduce
109

110. Fig21.19:The overall execution of a
MapReduce program
110

111. Fig21.20:The overall execution of a
Sawzall program
111

112. Fig21.21:Summary of design choices
related to distributed computation
112

113. Fig4.1: Middleware layers
INTERPROCESS COMMUNICATION
113

114. Fig4.2:Sockets and ports
message
agreed port
any port socket
socket
Internet address = 138.37.88.249
Internet address = 138.37.94.248
other ports
client server
114

115. UDP client sends a message to the
server and gets a reply
import java.net.*;
import java.io.*;
public class UDPClient{
public static void main(String args[]){
// args give message contents and server hostname
DatagramSocket aSocket = null;
try {
aSocket = new DatagramSocket();
byte [] m = args[0].getBytes();
InetAddress aHost = InetAddress.getByName(args[1]);
int serverPort = 6789;
DatagramPacket request = new DatagramPacket(m, m.length(), aHost, serverPort);
aSocket.send(request);
byte[] buffer = new byte[1000];
DatagramPacket reply = new DatagramPacket(buffer, buffer.length);
aSocket.receive(reply);
System.out.println("Reply: " + new String(reply.getData()));
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e){System.out.println("IO: " + e.getMessage());}
}finally {if(aSocket != null) aSocket.close();}
} }
115

116. UDP server repeatedly receives a
request and sends it back to the client
import java.net.*;
import java.io.*;
public class UDPServer{
public static void main(String args[]){
DatagramSocket aSocket = null;
try{
aSocket = new DatagramSocket(6789);
byte[] buffer = new byte[1000];
while(true){
DatagramPacket request = new DatagramPacket(buffer, buffer.length);
aSocket.receive(request);
DatagramPacket reply = new DatagramPacket(request.getData(),
request.getLength(), request.getAddress(), request.getPort());
aSocket.send(reply);
}
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e) {System.out.println("IO: " + e.getMessage());}
}finally {if(aSocket != null) aSocket.close();}
}}
116

117. TCP client makes connection to server,
sends request and receives reply
import java.net.*;
import java.io.*;
public class TCPClient {
public static void main (String args[]) {
// arguments supply message and hostname of destination
Socket s = null;
try{
int serverPort = 7896;
s = new Socket(args[1], serverPort);
DataInputStream in = new DataInputStream( s.getInputStream());
DataOutputStream out =new DataOutputStream( s.getOutputStream());
out.writeUTF(args[0]); // UTF is a string encoding see Sn 4.3
String data = in.readUTF();
System.out.println("Received: "+ data) ;
}catch (UnknownHostException e){ System.out.println("Sock:"+e.getMessage());
}catch (EOFException e){System.out.println("EOF:"+e.getMessage());
}catch (IOException e){System.out.println("IO:"+e.getMessage());}
}finally {if(s!=null) try {s.close();}catch (IOException
e){System.out.println("close:"+e.getMessage());}}
}}
117

118. TCP server makes a connection for each
client and then echoes the client’s request
import java.net.*;
import java.io.*;
public class TCPServer {
public static void main (String args[]) {
try{ int serverPort = 7896;
ServerSocket listenSocket = new ServerSocket(serverPort);
while(true) {
Socket clientSocket = listenSocket.accept();
Connection c = new Connection(clientSocket); }
} catch(IOException e) {System.out.println("Listen :"+e.getMessage());}
}
}
// this continues on the next slide
118

119. class Connection extends Thread {
DataInputStream in;
DataOutputStream out;
Socket clientSocket;
public Connection (Socket aClientSocket) {
try {
clientSocket = aClientSocket;
in = new DataInputStream( clientSocket.getInputStream());
out =new DataOutputStream( clientSocket.getOutputStream());
this.start();
} catch(IOException e) {System.out.println("Connection:"+e.getMessage());}
}
public void run(){
try { // an echo server
String data = in.readUTF();
out.writeUTF(data);
} catch(EOFException e) {System.out.println("EOF:"+e.getMessage());
} catch(IOException e) {System.out.println("IO:"+e.getMessage());}
} finally{ try {clientSocket.close();}catch (IOException e){/*close failed*/}}
}
}
119

120. CORBA CDR for constructed types
Type Representation
sequence length (unsigned long) followed by elements in order
string length (unsigned long) followed by characters in order (can also
can have wide characters)
array array elements in order (no length specified because it is fixed)
struct in the order of declaration of the comp onents
enumerated unsigned long (the values are specified by the order declared)
union type tag followed by the selected m emb er
120

121. CORBA CDR message
The flattened form represents a Person struct with value: {‘Smith’, ‘London’, 1984}
0–3
4–7
8–11
12–15
16–19
20-23
24–27
5
"Smit"
"h___"
6
"Lond"
"on__"
1984
index in
sequence of bytes 4 bytes
notes
on representation
length of string
‘Smith’
length of string
‘London’
unsigned long
121

122. Indication of Java serialized form
The true serialized form contains additional type markers; h0 and h1 are handles
Serialized values
Person
3
1984
8-byte version number
int year
5 Smith
java.lang.String
name:
6 London
h0
java.lang.String
place:
h1
Explanation
class name, version number
number, type and name of
instance variables
values of instance variables
122

123. XML definition of the Person structure
<person id="123456789">
<name>Smith</name>
<place>London</place>
<year>1984</year>
<!-- a comment -->
</person >
use of namespace in the Person structure
<person pers:id="123456789" xmlns:pers="http://www.cdk5.net/person">
<pers:name> Smith </pers:name>
<pers:place> London </pers:place >
<pers:year> 1984 </pers:year>
</person>
123

124. An XML schema for the Person structure
<xsd:schema xmlns:xsd = URL of XML schema definitions >
<xsd:element name= "person" type ="personType" />
<xsd:complexType name="personType">
<xsd:sequence>
<xsd:element name = "name" type="xs:string"/>
<xsd:element name = "place" type="xs:string"/>
<xsd:element name = "year" type="xs:positiveInteger"/>
</xsd:sequence>
<xsd:attribute name= "id" type = "xs:positiveInteger"/>
</xsd:complexType>
</xsd:schema>
Representation of a remote object reference
Internet address
port number time object number
interface of
remote object
32 bits 32 bits 32 bits 32 bits
124

125. Multicast peer joins a group and sends
and receives datagrams
import java.net.*;
import java.io.*;
public class MulticastPeer{
public static void main(String args[]){
// args give message contents & destination multicast group (e.g. "228.5.6.7")
MulticastSocket s =null;
try {
InetAddress group = InetAddress.getByName(args[1]);
s = new MulticastSocket(6789);
s.joinGroup(group);
byte [] m = args[0].getBytes();
DatagramPacket messageOut =
new DatagramPacket(m, m.length, group, 6789);
s.send(messageOut);
// this figure continued on the next slide
125

126. // get messages from others in group
byte[] buffer = new byte[1000];
for(int i=0; i< 3; i++) {
DatagramPacket messageIn =
new DatagramPacket(buffer, buffer.length);
s.receive(messageIn);
System.out.println("Received:" + new String(messageIn.getData()));
}
s.leaveGroup(group);
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e){System.out.println("IO: " + e.getMessage());}
}finally {if(s != null) s.close();}
}
}
126

127. Types of overlay
table continues on the next slide
127

129. Skype overlay architecture
129

130. Fig:4.17 An overview of point-to-point
communication in MPI
130

131. Selected send operations in MPI

132. Questions?