This document discusses processes and threads in distributed systems. It begins by defining key terms like process, thread, and context. It then explains that threads allow blocking calls without blocking the entire process, making them attractive for distributed systems. The document provides examples of how multithreading can improve client and server performance by hiding network latency and enabling simple scaling to multiprocessors. Overall, multithreading is popular for distributed systems because it facilitates organization and parallelism while allowing the use of blocking calls.

1. Parallel and Distributed Computing
Processes
Dr. Danish Mahmood

2. Distributed Systems
(3rd Edition)
Chapter 03: Processes
Version: February 25, 2017
Modified: Oct 2021

3. Chapter 3
1. Distributed Systems
Will study later with grid and
cluster computing paradigms

4. Processes: Threads Introduction to threads
Introduction
Basic idea
To execute a program, an OS creates a number of virtual processors, each
running a different program
To keep track to these virtual processors, the OS has a process table
containing entries to store CPU register values memory maps open files etc –
this forms a processor context.
The process is often called as a program in execution i.e., a program that is
being executed on one of the virtual processor having a processor context.
Problem- OS has to take care that independent processes cannot maliciously
affect the correctness of each other’s behavior- Hardware support is required
to ensure this
4 /47

5. Processes: Threads Introduction to threads
Introduction
Basic idea
Hence, concurrency transparency is required and for that
Each time, a process is created, OS must assign and schedule its
independent space - for swapping between processes, OS must modify
registers of MMU- memory management unit.
Like processes, a thread executes its own piece of code independently to
other threads
5 /47

6. Processes: Threads Introduction to threads
Introduction to threads
Basic idea
Processor: Provides a set of instructions along with the capability of
automatically executing a series of those instructions.
Process: a program in execution that a program that is currently
being executed on an OS’s virtual processor.
Thread: A minimal software processor in whose context a series of
instructions can be executed. Saving a thread context implies
stopping the current execution and saving all the data needed to
continue the execution at a later stage.
6 /47

7. Processes: Threads Introduction to threads
Introduction to threads
Contexts
Processor context: The minimal collection of values stored in the registers
of a processor used for the execution of a series of instructions (e.g.,
stack pointer, addressing registers, program counter).
Thread context: Almost nothing more than a processor context along with
some other info for thread management.
Process context: The minimal collection of values stored in registers and
memory, used for the execution of a thread (i.e., thread context, but now
also at least memory management unit (MMU) register values).
3 /47

8. Processes: Threads Introduction to threads
Context switching- Thread Vs Process
Observations
1
2
3
Threads share the same address space. Thread context switching can be
done entirely independent of the operating system.
Process switching is generally (somewhat) more expensive as it
involves getting the OS in the loop.
Creating and destroying threads is much cheaper than doing so for
processes.
4 /47

9. Processes: Threads Introduction to threads
Why use threads
Some simple reasons
Avoid needless blocking: a single-threaded process willblockwhen doing
I/O; in a multi-threaded process, the operating system can switch the CPU
to another thread in that process.
Exploit parallelism: the threads in a multi-threaded process can be
scheduled to run in parallel on a multiprocessor or multicore processor.
Avoid process switching: structure large applications not as a collection of
processes, but through multiple threads.
Thread usage in nondistributed systems 5 /47

10. Processes: Threads Introduction to threads
Why use threads
Thread usage in nondistributed systems 5 /47
Threads Vs multiple pipeline
Pipeline is the internal structure of the CPU. Its the hardware implementation
of a CPU.
A thread is a stream of instructions that are fed into the CPU. They are the
logical representation of the CPU core.
Different families of x64 CPUs have very different pipelines. But all process
basically the same types of threads.
That is why you can run the same exe on both a intel x64 and an amd x64
processor. But inside the CPUs the two processors are very different. That is
why some processors are faster and others slower, but unless someone
messed up majorly, both programs will give you the same output.

11. Processes: Threads Introduction to threads
Avoid process switching
Avoid expensive context switching
Process A Process B
S1: Switch from user space
to kernel space
S3: Switch from kernel
space to user space
Operating system
S2: Switch context from
process A to process B
Trade-offs
Threads use the same address space: more prone to errors
No support from OS/HW to protect threads using each other’s memory
Thread context switching may be faster than process context switching
Thread usage in nondistributed systems 6 /47

12. Multi
threading

13. Processes: Threads Introduction to threads
Thread Implementation
Threads are often provided in form of thread package
Each package contains operations to create and destroy
along with other operations
Two approaches to implement thread package
construct a thread library in user space
kernel be aware of threads and schedule then
Basic idea
Thread implementation 10 /47

14. Processes: Threads Introduction to threads
Thread Implementation
Its cheap to create and destroy at user address space
• Assigning user address space and freeing user address
space both operations are easy to manage and
implement.
Switching thread context can be done in a few instructions
• Only CPU register values are to be stored and reloaded
with previously stored values.
construct a thread library in user space- Advantages
Thread implementation 10 /47

15. Processes: Threads Introduction to threads
Thread Implementation
Deploying many to one threading model-
Multiple threads are mapped to a single schedulable entity.
Hence, the invocation of a blocking system call will
immediately block the entire process.
Hence in such scenarios, user level threads are of no help
construct a thread library in user space- Disadvantages
Thread implementation 10 /47

16. Processes: Threads Introduction to threads
Thread Implementation
The problems in user level threads can be overcome by
implementing threads in OS kernel- targeting one to one
threading model.
In one to one threading model, each thread independently is
a schedulable entity.
Price to pay is its creating and deletion ad synchronization
and scheduling – all operations requiring separate system
calls.
Kernel level threads
Thread implementation 10 /47

17. Processes: Threads Introduction to threads
Lightweight processes
Hybrid of user and kernel level implementation
Many to many threading model
Implemented in form of lightweight processes (LWP)
There can be many LWP in a process (at kernel level) and also
offers user level thread package offering application that
usual operation of creating and destroying threads
Basic idea
Thread implementation 10 /47

18. Processes: Threads Introduction to threads
Lightweight processes
Basic idea
Introduce a two-level threading approach:lightweight processesthat can
execute user-level threads.
Lightweight process
Thread
Kernel space
LWP executing a thread
Thread state
User space
Thread implementation 10 /47
An alternative approach- hybrid of above two methods- many to many
threading model
This concept has been virtually abandoned – it’s just either user-level or
kernel-level threads.

19. Processes: Threads Threads in distributed systems
Threads in Distributed Systems
Basic idea
• An Important property of thread is that they can provide
convenient means of allowing blocking system calls
without blocking the entire process in which the thread is
running.
• This property makes it attractive to be used in distributed
systems.
• REASON: It makes it much easier to express communication
in the form of maintaining multiple logical connections at
the same time.
• Lets have an example of multi threaded clients and servers.
Multithreaded clients 13 /47

20. Processes: Threads Threads in distributed systems
Using threads at the client side
Multithreaded web client
Hiding network latencies:
Web browser scans an incoming HTML page, and finds thatmore files
need to be fetched.
Each file is fetched by a separate thread, each doing a (blocking) HTTP
request.
As files come in, the browser displays them.
Multiple request-response calls to other machines (RPC)
A client does several calls at the same time, each one by a different
thread.
It then waits until all results have been returned.
Note: if calls are to different servers, we may have a linear speed-up.
Multithreaded clients 12 /47

21. Processes: Threads Threads in distributed systems
Multithreaded clients: does it help?
Thread-level parallelism: TLP
Let ci denote the fraction of time that exactly i threads are being executed
simultaneously.
TLP =
∑N
i=1 i ·ci
1−c0
with N the maximum number of threads that (can) execute at the same time.
Practical measurements
A typical Web browser has a TLP value between 1.5 and 2.5 ⇒ threads are
primarily used forlogically organizingbrowsers.
Multithreaded clients 13 /47

22. Processes: Threads Threads in distributed systems
Using threads at the server side
Improve performance
Starting a thread is cheaper than starting a new process.
Having a single-threaded server prohibits simple scale-up to a
multiprocessor system.
As with clients:hide network latencyby reacting to next request while
previous one is being replied.
Better structure
Most servers have high I/O demands. Using simple,well-understood
blocking calls simplifies the overall structure.
Multithreaded programs tend to be smaller and easier to understanddue
tosimplified flow of control.
Multithreaded servers 14 /47

23. Processes: Threads Threads in distributed systems
Why multithreading is popular: organization
Dispatcher/worker model
Dispatcher thread
Worker thread
Server
Operating system
Request coming in
from the network
Request dispatched
to a worker thread
Overview
Model Characteristics
Multithreading Parallelism, blocking system calls
Single-threaded process No parallelism, blocking system calls
Finite-state machine Parallelism, nonblocking system calls
Multithreaded servers 15 /47

24. Processes: Virtualization Principle of virtualization
Virtualization
Essence
• Threads and processes can be seen as a way to do more
things at the same time, on a single processor.
• Actually it is not happening, we are providing an illusion
by rapidly switching between threads and processes.
• This separation between having a single CPU and able to
pretend parallelism is what we call resource
virtualization.
• (Execution is extended to other resources in
parallel)
16 /47

25. Processes: Virtualization Principle of virtualization
Virtualization
Observation
Virtualization is imp:
Hardware changes faster than software
Ease of portability and code migration
Isolation of failing or attacked components
Principle: mimicking interfaces
Program
Interface A
Hardware/software system A
Program
Interface A
Implementation of
mimicking Aon B
Interface B
Hardware/software system B
16 /47

26. Processes: Virtualization Principle of virtualization
Mimicking interfaces- types of Virtualization
Four types of interfaces at three different levels
1
2
3
Instruction set architecture: the set of machine instructions, with two
subsets:
Privileged instructions: allowed to be executed only by the operating
system.
General instructions: can be executed by any program.
System calls as offered by an operating system.
Library calls, known as an application programming interface (API)
Types of virtualization 17 /47

27. Processes: Virtualization Principle of virtualization
Ways of virtualization
(a) Process VM, (b) Native VMM, (c) Hosted VMM
Application/Libraries
Runtime system
Operating system
Hardware
Application/Libraries
Operating system
Virtual machine monitor
Hardware
Application/Libraries
Operating system
Virtual machine monitor
Operating system
Hardware
(a) (b) (c)
Differences
(a)Separate set of instructions, an interpreter/emulator, running atop an OS.
(b)Low-level instructions, along with bare-bones minimal operating system
(c)Low-level instructions, but delegating most work to a full-fledged OS.
Types of virtualization 18 /47

28. Processes: Virtualization Principle of virtualization
Zooming into VMs: performance
Refining the organization
Virtual machine monitor
Application/Libraries
Hardware
Host operating system
Guest operating system
Privileged
instructions General
instructions
Privileged instruction: if and only if
executed in user mode, it causes atrapto
the operating system (allowed to be
executed only by OS)
General Instructions: The rest of all
Special instructions
Control-sensitive instruction: may affect configuration of a machine (e.g.,
one affecting relocation register or interrupt table).
Behavior-sensitive instruction: effect is partially determined by context
(e.g., an interrupt-enabled flag, but only in system mode).
Types of virtualization 19 /47

29. Processes: Virtualization Principle of virtualization
Condition for virtualization
Necessary condition
For any conventional computer, a virtual machine monitor may be constructed
if the set of sensitive instructions for that computer is a subset of the set of
privileged instructions.
Problem: condition is not always satisfied
There may be sensitive instructions that are executed in user mode without
causing a trap to the operating system.
Solutions
Emulate all instructions
Wrap nonprivileged sensitive instructions to divert control to VMM
Paravirtualization: modify guest OS, either by preventing nonprivileged
sensitive instructions, or making them nonsensitive (i.e., changing the
context).
Types of virtualization 20 /47

30. Processes: Virtualization Application of virtual machines to distributed systems
VMs and cloud computing
Three types of cloud services
Infrastructure-as-a-Service covering the basic infrastructure
Platform-as-a-Service covering system-level services
Software-as-a-Service containing actual applications
IaaS
Instead of renting out a physical machine, a cloud provider will rent out a VM
(or VMM) that may possibly be sharing a physical machine with other
customers ⇒ almost complete isolation between customers (although
performance isolation may not be reached).
21 /47

33. Processes: Code migration Reasons for migrating code
Code migration
44 /47
• Till now we were concerned with distributed systems
in which communication is limited to passing data.
• However, there are situations in which passing
programs, even while they are being executed. As it
simplifies the design of a distributed system
• This distribution of programs/ executable codes is
termed as code migration.

34. Processes: Code migration Reasons for migrating code
Reasons to migrate code
Load distribution
Ensuring that servers in a data center aresufficientlyloaded (e.g., to
prevent waste of energy)
Minimizing communication by ensuring that computations are close to
where the data is (think of mobile computing).
Flexibility: moving code to a client when needed
Differentiate between
code shipping and
code fetching.
Client Server
Service-specific
client-side code
Code repository
1. Client fetches code
2. Client and server
communicate
41 /47

35. Processes: Code migration Reasons for migrating code
Models for code migration
Before execution After execution
Client Server Client Server
CS
code
exec
resource
code
exec*
resource
REV
code
−→ exec
resource
−→
code
exec*
resource
CS: Client-Server REV: Remote evaluation
42 /47

36. Processes: Code migration Reasons for migrating code
Strong and weak mobility
Object components
Code segment: contains the actual code
Data segment: contains the state
Execution state: contains context of thread executing the object’s code
Weak mobility: Move only code and data segment (and reboot execution)
Relatively simple, especially if code is portable
Distinguishcode shipping(push) fromcode fetching(pull)
Strong mobility: Move component, including execution state
Migration: move entire object from one machine to the other
Cloning: start a clone, and set it in the same execution state.
44 /47

37. Processes: Code migration Migration in heterogeneous systems
Migration in heterogeneous systems
Main problem
The target machine may not besuitable to execute the migrated code
The definition of process/thread/processor context ishighly dependent on
local hardware, operating system and runtime system
Only solution: abstract machine implemented on different platforms
Interpreted languages, effectively having their own VM
Virtual machine monitors
45 /47

38. Processes: Code migration Migration in heterogeneous systems
Migrating a virtual machine
Migrating images: three alternatives
1
2
3
Pushing memory pages to the new machine and resending the ones that
are later modified during the migration process.
Stopping the current virtual machine; migrate memory, and start the new
virtual machine.
Letting the new virtual machine pull in new pages as needed: processes
start on the new virtual machine immediately and copy memory pages on
demand.
46 /47

39. Processes: Code migration Migration in heterogeneous systems
Performance of migrating virtual machines
Problem
A complete migration may actually take tens of seconds. We also need to
realize that during the migration, a service will be completely unavailable for
multiple seconds.
Measurements regarding response times during VM migration
Time
Migration
Downtime
Response
time
47 /47

40. Homework
1. Install a VMM on top of your laptop’s operating system. (have a
deep study of https://www.virtualbox.org/manual/ch01.html)
2. What is django framework? How it helps in code/ data migration?
3. What are basic commands/ libraries you will need while working on
code/data migration in django?
4. Write a simple example in python regarding code/data migration
using django / python.
5. Submit at google classroom within next week.