Inter Process Communication

© 2010-15 SysPlay Workshops <workshop@sysplay.in>
All Rights Reserved.
Inter Process Communication

2© 2010-15 SysPlay Workshops <workshop@sysplay.in>
What to Expect?
W's of Inter Process Communication
Various IPC mechanisms in Linux
Pipes
FIFOs
Message Queues
Shared Memory & Process Semaphores

Why separate IPC mechanisms?
Processes are non-sharing entities
But, we may need communication
Already looked at one mechanism
Signals
But those are very basic
Mostly asynchronous
Connection-less
No data communication
Separate mechanisms are answer to all those

What is IPC?
Inter Process Communication
Mechanism whereby one process communicates
with another process
Communication = Exchange of Data
Directions
Uni-directional
Bi-directional
Multi-directional

Example for IPC
Print the filenames in a directory
'ls' outputs the filenames
'lpr' prints its input
But ls & lpr are separate processes
What are the solutions?
Put into a file → ls > file; lpr < file
Setup an IPC, say a pipe (|) → ls | lpr

Types of IPC mechanisms
Pipe: Allows the flow of data in one direction only
Named Pipe: Pipe with a specific name
Message Queues: Message passing using a queue
Shared Memory & Semaphore
Shared Memory allows the interchange of data through a defined area
of memory
Semaphore is used in solving problems associated with
synchronization and avoiding race conditions during sharing
Mapped Memory: Similar to shared memory, but use file instead
of memory
Why so many?
Based on the requirements, flexibility & benefits

Pipes

W's of a Pipe
Connects 2 ends to allow transfer
One-way Serial Communication Channel
One end: Output (write end)
Another end: Input (read end)
Limited Capacity
Provides automatic Synchronization
Read blocks on no data
Write blocks on full data

Pipe Usage
On Shell
Using |
In Programming / C
Creation
Using popen(), pclose()
Using system call pipe()
Communication as with any other fd
Using system calls read(), write(), ...

Pipe in a Process
Pipe Creation: Kernel sets up two file descriptors
1st for Read: Path for obtaining the data
2nd for Write: Path for data input
Pipe to communicate with itself !!!
May be among threads of a process
Process Kernelwrite
read

Making Sense with Pipes
Info: Child process inherits parent's open file
descriptors
This Info + Pipe Creation = IPC within the family
Final Critical Decision
Remember pipes are half-duplex
So, who communicates with whom?
Parent Process Kernel Child Process
in in
outout

Multi-Process Communication
Usual system calls operating on file descriptors
Send data to the pipe: write()
Retrieve data from the pipe: read()
Though calls like lseek(), don't work
Parent Process Kernel Child Process
in
out
write()
read()

Pipes Extras & Limitations
Two-way Pipes
Open two pipes
Assign the file descriptors, appropriately
pipe() call must PRECEDE a call to fork()
Pipes are only for related processes
That is processes with a related ancestry
What about unrelated processes?
There are special type of pipes called ...

FIFOs

W's of a FIFO
A Pipe with a name in the file system
Also called Named Pipe
Properties identical to those of a Pipe
including the Serial Communication
which is now termed as First-In-First-Out
Except that now processes can be unrelated
Achieved by it being a device special file
which persists irrespective of Process existence
And any process can operate on it

FIFO Usage
On Shell
Creation: mknod, or mkfifo
Communication: cat, echo, ...
In Programming / C
Creation
System Call: mknod(), open(), close()
Library Function: mkfifo(), fopen(), fclose()
Communication
System Calls: read(), write(), …
Library Functions: fread(), fwrite, fputs, ...

Experiment on Shell
mkfifo /tmp/fifo
ls -l /tmp/fifo
cat < /tmp/fifo
<See the text>
Open another shell
cat > /tmp/fifo
<Type some text>
rm /tmp/fifo

Programming Examples
With System Calls
Writer: fifo_write.c
Reader: fifo_read.c
With Library Functions
Writer: fifo_client.c
Reader: fifo_server.c
Use a shell for each of reader & writer

Broken Pipe
Typical error condition with pipes
Read terminates while write is still on
Writer receives the signal SIGPIPE
the next time it tries to write()
Default signal handler for SIGPIPE
Prints "Broken Pipe" and exits
Avoid abnormal exit by a user handler

Is FIFO good enough?
As long as FIFO meets our requirements
May not be helpful for
Out-of-order processing
Non-blocking requirements
Large storage communications
Unknown opening/closing sequences
Answer to that is ...

Message Queues

W's of a Message Queue
Process Communication by means of IPC messages
Message sent goes & stays in an IPC message
queue
until another process reads it
Like FIFOs Unlike FIFOs
Typically follows FIFO Ways to pull certain urgent messages
before they reach the front
Exists independently of both sending &
receiving processes
Always ready - No need to open/close
Data Transfer between unrelated
processes
No synchronization issues between
open & close

Message Queue Usage
Shell Way
Create: ipcmk -Q [ -p <mode> ]
Get Information: ipcs [ -q [ -i id ] ]
Remove: ipcrm [ -Q <msgqkey> | -q <msgqid> ]
Programming Way in C (using sys calls)
Create/Access Message Queue: msgget()
Send Message: msgsnd()
Receive Message: msgrcv()
Other Message Queue Operations: msgctl()
Helper Function: ftok()

Message Queue Operations
Creating / Connecting to Message Queue
int msgget(key_t key, int msg_flg);
Returns
Message Queue id on success
-1 on error (and sets errno)
key: System-wide unique id for a Msg Queue
Process connects to a particular message queue using its
creation key
msg_flg: operation | permissions
IPC_CREATE, IPC_EXCL

What is key?
key_t = long
Could be any number
But must be system-wide unique
Low probability but what if clashes?
Solutions
Use IPC_PRIVATE as the key
Always creates a new unique message queue (irrespective of flags)
But could be shared only with related processes
Use ftok() to generate the key
key_t ftok(const char *path, int id);
path: process readable file (for inode based info)
id: typically some character like 'A', etc

Typical Message Format
struct msgbuf
{
long mtype; /* > 0 */
char mdata[1];
}

Sending a Message
int msgsnd(int msgq_id, const void *msg_ptr,
size_t msg_sz, int msg_flg);
Returns: 0 on success; -1 on error (and sets errno)
msgq_id: Message Queue id returned by msgget()
msg_ptr: Pointer to the message to be sent
msg_sz: Size of the message data in bytes
msg_flg: Optional flag parameters
IPC_NOWAIT → May return -1 with error EAGAIN

Receiving a Message
int msgrcv(int msgq_id, void *msg_ptr, size_t msg_sz, long int msg_type, int
msg_flg);
Returns
Number of bytes placed in the receive buffer
msg_ptr: Pointer to the buffer for receiving the message
msg_sz: Size of the message data in bytes
msg_type: Allows reception priority implementation
== 0: Normal Operation
> 0: First message with msg_type is retrieved
< 0: First message with msg_type <= | msg_type | is retrieved
msg_flg: Controls behaviour when no matching message is found
IPC_NOWAIT, MSG_EXCEPT, MSG_NOERROR

Message Queue Operations ...
Control Operations on the Message Queue like
Setting Message Queue Size
Deleting a Message Queue
Any many more
int msgctl(int msgq_id, int cmd, struct msqid_ds *buf);
cmd
IPC_STAT – Get the message queue information into buf
IPC_SET – Set the various message queue info specified by buf
IPC_RMID – Remove the message queue (buf is ignored)

Caution on
Deleting Message Queues
Invoking _exit() or exec* does not do it
Needs explicit removal
Using IPC_RMID cmd of msgctl()
Otherwise, may violate the system limit
Of total number of message queues

Interesting Message Queue DS
struct msqid_ds
{
struct ipc_perm msg_perm; // Ownership & Permissions
time_t msg_stime; // Time of last sent message
time_t msg_rtime; // Time of last received message
time_t msg_ctime; // Time of last change
msgqnum_t msg_qnum; // Message Count in the Queue
msglen_t msg_qbytes; // Maximum Bytes allowed in the Queue
pid_t msg_lspid; // Last Sending Process Id
pid_t msg_lrpid; // Last Receiving Process Id
...
}

Interesting Message Q DS ...
struct ipc_perm
{
key_t __key; // Key supplied to msgget()
uid_t uid; // Effective UID of owner
gid_t gid; // Effective GID of owner
uid_t cuid; // Effective UID of creator
gid_t cgid; // Effective GID of creator
unsigned short mode; // Permissions
...
};

Let's try an Example
Two Programs on Two Shells
msg_send.c
msg_recv.c
Observe the various APIs
And their various invocations

Standard Question
Would Msg Q meet all our requirements?
Obviously not. But what not?
Random Data “Sharing”
Without Data Copying
With Least Overheads
Immediate Change Reflections
That's where we go to the next topic ...

Shared Memory

W's of Shared Memory
Rawest Communication Mechanism
Piece of Memory which is Shared
Between two or more processes
Fastest form of IPC
As no unnecessary copying of data
And no other overheads – not even synchronization :)
Synchronization is Users responsibility
As desired by him
Process Semaphores is the Mechanism provided

Linux Memory Model
All accesses in Linux are through virtual addresses
Each Process has its “own” Virtual Address Space
Which is split into Virtual Pages
Each Process maintains a mapping table
From its Physical Pages to these Virtual Pages
For access to its actual data
And mappings of multiple processes can point to
the same physical page
Enabling the sharing of “actual” memory

Shared Memory Usage
Shell Way
Create: ipcmk -M <size> [ -p <mode> ]
Get Information: ipcs [ -m [ -i id ] ]
Remove: ipcrm [ -M <shmkey> | -m <shmid> ]
Physical Memory Allocation: shmget()
Mapping / Attaching to Virtual Memory: shmat()
Access through usual (Virtual) Address Dereferencing
Unmapping / Detaching from Virtual Memory: shmdt()
Physical Memory Deallocation & Other Shared Memory
Control Operations: shmctl()

Physical Memory Allocation
Done by only one of the sharing Processes
int shmget(key_t key, size_t size, int shm_flg);
Returns
Shared Memory id on success
key: System-wide unique id for a Shared Memory
Same as in Message Queue
Process gets a particular shared memory id using its creation key
size: Number of bytes to allocate or get for the shared memory
shm_flg: operation | permissions

Shared Memory Attachment
Mapping to Process Virtual Address Space
Done by all the sharing Processes to Access the Shared Memory
void *shmat(int shm_id, const void *shm_addr, int shm_flg);
Returns
Virtual Start Address of the attached Shared Memory on success
MAP_FAILED i.e. (void *)(-1) on error (and sets errno)
shm_id: Shared Memory id returned by shmget()
shm_addr
Requested process virtual start address
NULL enables Kernel to choose one
shm_flg
SHM_RND: round down shm_addr to a multiple of the page size
SHM_RDONLY: shared memory will be only read, not written
Children created by fork() inherit attached shared segments

Shared Memory Detachment
Unmapping from Process Virtual Address Space
Done by all the Processes, which did attach, or inherit
int shmdt(const void *shm_addr);
shm_addr: Process virtual address map returned by
shmat()
Call to _exit() or exec*() automatically detaches
Detachment may deallocate the shared memory
If done by the last process
And already marked to deallocate

Shared Memory Control Ops
Control Operations on the Shared Memory like
Physical Memory Deallocation for Shared Memory
Any others
int shmctl(int shm_id, int cmd, struct shmid_ds *buf);
shm_id: Shared Memory id returned by shmget()
cmd
IPC_STAT – Get the shared memory information into buf
IPC_SET – Set the various shared memory info specified by buf
IPC_RMID – (Mark) Deallocate the physical memory (buf is
ignored)

Caution on
Deallocating Shared Memory
Needs explicit deallocation
Using IPC_RMID cmd of shmctl()
Of total number of shared memory segments

Interesting Shared Memory DS
struct shmid_ds
{
struct ipc_perm shm_perm; // Ownership & Permissions
size_t shm_segsz; // Size of shared memory in bytes
time_t shm_atime; // Time of last attach
time_t shm_dtime; // Time of last detach
time_t shm_ctime; // Time of last change
pid_t shm_cpid; // Creator Process Id
pid_t shm_lpid; // Last Attaching/Detaching Process Id
shmatt_t shm_nattch; // Number of current attaches
...
}

shm_write.c
shm_read.c

Need for Synchronization
We already talked
Shared Memory has no Access Coordination
It has to be handled by us – the User
Moreover, now we observed
The issue with the Example
So, now its time to look into resolving it
That's where we go to the next topic ...

Process Semaphores

W's of Process Semaphores
First, it isn't any form of IPC
But a mechanism for synchronization used by IPCs
Counter used to provide synchronized access
To a shared data object for multiple processes
Variable protected by Kernel
No direct access by the user
Counter
> 0 → Resource is available
== 0 → Resource is busy / in use
< 0 → Should never happen

Typical Semaphore Usage
Semaphore ↔ Shared Resource
Accessing the Shared Resource
Check semaphore value
> 0 (Available) → Decrement by 1
== 0 (Busy) → Sleep until it is > 0 (and repeat check)
Access the Shared Resource
Once done, increment the semaphore value
At this point, sleeping processes are wakened up
For this to work correctly
Check (Test) & Decrement (Set) has to be atomic
A typical processor instruction available in Kernel

Specialized Semaphores
Binary semaphore
Semaphore controlling a single resource “availability”
Possible values
0 → Unavailable
1 → Available
Typically initialized to 0
Mutex
Specialized Binary Semaphore
Used for a “Usable” rather than a “Consumable” Resource
Process using it, only releases it
Giving a notion of ownership
Typically initialized to 1

Process Semaphore Usage
Shell Way
Create: ipcmk -S <nsems> [ -p <mode> ]
Get Information: ipcs [ -s [ -i id ] ]
Remove: ipcrm [ -S <semkey> | -s <semid> ]
Semaphore Allocation/Access: semget()
Semaphore Deallocation & Other Semaphore Control
Operations: semctl()
Semaphore Usage Operations: semop()

Process Semaphore Allocation
Done by all of the sharing Processes
int semget(key_t key, int nsems, int sem_flg);
Returns
Semaphore Array id on success
key: System-wide unique id for a Semaphore Array
Same as in Message Queue & Shared Memory
Process gets access to a particular semaphore array using its creation
key
nsems: Number of semaphores to create or access in the Array
sem_flg: operation | permissions

Semaphore Control Ops
Control Operations on the Semaphore like
Deleting a Semaphore Array
Any many more
int semctl(int sem_id, int sem_num, int cmd, ...);
sem_id: Semaphore Array id returned by semget()
sem_num: Index of semaphore in the array
cmd
IPC_STAT – Get the semaphore information array into optional arg (sem_num is ignored)
IPC_SET – Set the various semaphore array info specified by optional arg (sem_num is
ignored)
IPC_RMID – Remove the semaphore array (sem_num is ignored)
SETVAL – Set sem_num'th semaphore value as specified by optional arg
...

The Optional Argument
union semun
{
int val; // Value for SETVAL
struct semid_ds *buf; // Buffer for IPC_STAT & IPC_SET
unsigned short *array; // Array for GETALL, SETALL
…
}
struct semid_ds
{
struct ipc_perm sem_perm; // Ownership & Permissions
time_t sem_otime; // Time of last semop
time_t sem_ctime; // Time of last change
unsigned short sem_nsems; // No of semaphores in the array
}

Caution on
Deleting Process Semaphores
Needs explicit removal
Using IPC_RMID cmd of semctl()
Of total number of process semaphores

Initializing Process Semaphores
Two ways though both using semctl()
Initializing all in the array
cmd = SETALL
arg.array = Array of values to be set
Initializing one at time
sem_num = Semaphore number to be initialized
cmd = SETVAL
arg.val = Value to be set

Semaphore Usage Ops
Usage Operations on the Semaphore like
Increment, Decrement, …
int semop(int sem_id, struct sembuf *sops, unsigned int nsops);
sem_id
Semaphore Array id returned by semget()
sops
Array of operations to be performed
nsops
Number of elements in sops

Operation Structure Decoded
struct sembuf
{
unsigned short sem_num; // Index of semaphore to be operated on
short sem_op; // Value to be added to the current semaphore value
short sem_flg; // Operation flags
}
Operation Specifics
sem_op > 0 → Value added & Processes waiting on it are signaled
sem_op < 0 and current semaphore value >= | sem_op | → Value added
sem_op < 0 and current semaphore value < | sem_op |
It would block / wait until the semaphore value becomes >= | sem_op |
Operation Flags
IPC_NOWAIT – Prevents the operation from blocking
Returns -1 with EAGAIN in the above blocking case
SEM_UNDO – Kernel automatically undoes the semaphore operation on process exit

sem_one.c
sem_two.c
IPC used: Shared Memory

Mapped Memory

W's of Mapped Memory
Shared Memory variant
Uses a shared disk file instead of shared physical memory
So, got a name in the File System like Named Pipes
Forms an association between a file and a process’s memory
So, the file’s contents are accessed like ordinary memory
How does it work?
It's like allocating a buffer to hold a file’s entire contents
Then, Reading the file into that buffer for access as memory
And, Writing the buffer back to the file, afterwards, if modified
Advantages & Usage
Fast Access to Files: Specially read
Program Loader is one of its popular user

Memory Mapping a File
Done using the mmap() system call
void *mmap(void *addr, size_t len, int prot, int flags, int fd, off_t off);
Returns
Virtual Start Address of the mapped memory on success
MAP_FAILED i.e. (void *)(-1) on error (and sets errno)
addr (must be page aligned)
Requested process virtual start address for the mapping
NULL enables Kernel to choose one
len: Length of the map in bytes
prot: Protection on the mapped address range
PROT_READ, PROT_WRITE, PROT_EXEC (bitwise ORed), PROT_NONE
flags: Additional options (bitwise ORed)
fd: Open file descriptor of the file to be mapped
off: Offset in the file from where to map

Interesting Flags for Mapping
MAP_FIXED: Use the “addr” to map the file or fail
MAP_PRIVATE: No write through to the actual file
MAP_SHARED
Complement of MAP_PRIVATE
Writes are immediately reflected in the underlying file
Useful as an IPC
MAP_ANONYMOUS (Shared memory between related processes)
Mapping not backed by any file. fd & off are ignored
Contents initialized to zero
MAP_LOCKED: Lock the region as by mlock()
MAP_NORESERVE: No swap reservation
Might lead to SIGSEGV on write, if no physical memory is available

Cleaning Up the Act
Once done using the memory map, release it by
using munmap()
int munmap(void *addr, size_t length);
addr: Virtual Start Address returned by mmap()
length: Length of the mapped memory region
Even otherwise the Kernel automatically unmaps
mapped regions on process termination using
_exit() or exec*() calls

mmap1.c
mmap2.c

Map Access Synchronization
Synchronization Issues
Exist with Access of Mapped Memory
Similar to the Shared Memory case
Hence, similarly
User need to take care of it
Process Semaphores may be used

What all have we learnt?
W's of Inter Process Communication
Various IPC mechanisms in Linux
Pipes
FIFOs: Named Pipes
Message Queues
Shared & Mapped Memories
IPC Synchronization using
Process Semaphores

Any Queries?

Inter Process Communication

In this document

More Related Content

What's hot

Viewers also liked

Similar to Inter Process Communication

More from Anil Kumar Pugalia

Recently uploaded

Inter Process Communication