Linux Kernel
+ Linux Kernel Architecture
What is Linux ?
What is the Linux Kernel ?
How does the Kernel effect system
performance ?
+ Kernel
P
P
P P P
P
H
H H H
Kernel
+ Linux Kernel Architecture
Monolithic Kernel (diigo.com/085lp2)
Supports for System Calls (diigo.com/085lq5)
Support for Loadable Modules
Implements Preemptive Multitasking
Virtual Memory / Filesystem
Demand Paging / Loading
etc..
+ Linux Kernel Architecture
Kernel Function Libraries
– System (/proc, generic drivers)
– Networking (TCP, sockets, drivers)
– Storage (VFS, block devices)
– Memory (VM, PM)
– Processing (scheduler, fork, exec)
– Human Interface (HI devices)
+ Linux Kernel Architecture
Kernel Layers
– User space interface
– Virtual subsystem
– L2V bridge / transformer
– Logical subsystem
– Hardware interface
+ Linux Kernel Architecture
www.makelinux.net/kernel_map/
+ Linux Kernel and Processes
Linux Kernel and Processes
The Linux Kernel supports preemptive multitasking
(diigo.com/085lqo) which allows multiple processes
to run individually at the same time while prioritising
these with higher importance
What is a “Context Switch” ?
+ Linux Kernel and Processes
Context Switch
In order to support multi tasking a context switch
is being handled
Context switch stores or restores the CPU state
to where is 'stopped' before the last CS
+ Linux Kernel and Processes
Context switch can be forced by an interrupt
timer in case of a low latency operation
e.g: Disk I/O
Context Switch is also being used in order to
handle Interrupt requests
e.g: Completed read request
+ Linux Kernel and Processes
Context switch is being handled by the OS
Each process stack has a Process Control Block
which holds the relevant meta data for the CS
e.g: Program Counter, Registers Address List
Some CPUs (all x86 variants included) has a
Task State Segment (TSS) which is used to
store the PCB
+ Linux Kernel and Processes
In case a process needs to transite to Kernel
mode (during a system call) a Mode Switch is
being initiated
Price
Context switch is the most resource consuming
task of the OS.
+ Linux Kernel and Processes
To review Context Switch statistics use
– 'sar -w'
– 'vmstat'
Exercise:
 - Review the reciprocal relations between
User / System CPU utilization and CS
+ Linux Kernel and Processes
To review an individual process structure use
– 'pmap'
Exercise
 - Review the structure of several
processes. Can you identify the process
stack ?
+ Linux Kernel and Processes
CPU Scheduler
The Linux Kernel has a power full and high
performance CPU scheduler.
The CUP scheduler is designed to provide the
most appropriate amount of CPU [time]slices
– Scheduling is based on a dynamic
process priority rank
– Priority is changed to make sure no
process is 'hogging' the CUP nor starving
+ Linux Kernel and Processes
CPU Scheduler
– For each CPU the scheduler maintain
two priority ordered arrays for active and
expired tasks. After the active one is
done arrays are swapped
(Exception: diigo.com/085lrk)
4
3
2
1
4
3
2
1
+ Linux Kernel and Processes
CPU Scheduler
– The scheduling policy involves assessing
the priority for each task
– There are two classifications for tasks in
terms of scheduling
• CPU bound
• I/O bound
 These can be either Interactive, Batch or
RT processes
+ Linux Kernel and Processes
CPU Scheduler
– Process pre-emption occurs, when a
process receives a hardware interrupt
– Pre-emptive kernel, can ‘prefer’ a user
mode process over a kernel mode
process (even during an exception
handling), in order to support interactivity
+ Linux Kernel and Processes
CPU Scheduler
– When setting ‘nice’ value to a process, one
set a static portion of the scheduler priority
for that task
– Exercise
 ‘grep a /etc/services >/dev/null’
 Can you ‘tune’ this command group to
run faster ?
+ Linux Kernel and Processes
Multi Threading
Multi Threading is being used in order to have
several processing objects within the same
process
– Each Thread is being scheduled
separately by the Kernel
– Multiple threads share the same Memory
segment
+ Linux Kernel and Processes
Symmetric Multi Process systems can run
threads of the same process simultaneously
Linux implements 1-1 (1 scheduling object per
thread) threading via the Native Posix Thread
Library (NTPL)
* Just like Solaris and FreeBSD
+ Linux Kernel and Processes
To review Threads information use
– 'ps -eLf'
Exercise
– Can you identify multi threaded
processes on your system ?
– Can you find any difference between PID
and Thread ID ?
+ Inter Process Communication
IPC
IPC is an implementation of techniques to
exchange data between different threads
IPC can be used internally or across a computer
network
+ Inter Process Communication
IPC has 4 techniques
– Message passing
– Synchronization
– Shared memory
– Remote procedure call
+ Inter Process Communication
Message passing
Message can be a signal or data packet
Passing can be synchronous or asynchronous
Synchronisation
Used in order to keep several threads
synchronized and able to provide the same /
parallel service(s)
+ Inter Process Communication
Shared memory
Simply a shared form of memory
No control from of OS side
RPC
Method of running a procedure (subroutine) on a
remote process
+ Inter Process Communication
IPC implementations
PIPE – pass data between hierarchy related
threads
FIFO – pass data between non-related threads
Socket – pass large amount of data. Can
implement TCP internally
Shared Memory (SHM) – share defined amount
of storage between threads
+ Inter Process Communication
IPC implementations
Semaphore (SEM) – shared abstract data (like a
class) used to control access on a shared
resource
+ Inter Process Communication
IPC
Use the following commands to review IPC
information
– 'ipcs' – Review SHM, SEM, MSG
– 'ipcrm' – clean up IPC objects
– 'rpcinfo' – Review RPC
– 'popen' – Create a PIPE
– 'mkfifo' – Create a FIFO pipe
 *Try 'man ipc'
+ Inter Process Communication
IPC
Exercise
– Review IPC information on your system
Are there any SHMs ?
– Review RPC Services available on your
system
– Create a named pipe. Try to use it to pass a
message between two processes
(e.g: bash)
Can you name any advantages of using
fifo ?
+ Linux Kernel
Kernel Tune able parameters
Changing the kernel's behaviour when working
with processes, IPC, etc
Tune able parameters can be used via the '/proc'
filesystem or the 'syscall' command
+ Linux Kernel
Kernel Tune able parameters
General parameters
– 'ctrl-alt-del': sets the kernel reaction to
'ctrl-alt-del' key sequence
– 'domainname': sets machine's domain
name
– 'hostname': sets machine's host name
– 'panic': sets timer before restart after panic
– 'pid_max': sets maximum PID number
+ Linux Kernel
Kernel Tune able parameters
IPC parameters
– 'shmax': maximum size of shared memory
segment (bytes)
– 'shmall': total amount of shared memory
pages system wide
– 'shmmni': maximun number of shared
memory segments system wide
+ Linux Kernel
Kernel Tune able parameters
IPC parameters
– 'sem': holds 4 parameters
• maximum semaphores per array
• total maximum semaphores
• maximum operations per semaphore
operation call
• total maximum arrays
+ Linux Kernel
Kernel Tune able parameters
IPC parameters
– 'msgmni': maximun message queues
system wide
– 'msgmax': maximum size of message
(bytes)
– 'msgmnb': maximum size of queue (bytes)

Linux Performance Tunning Kernel

  • 1.
  • 2.
    + Linux KernelArchitecture What is Linux ? What is the Linux Kernel ? How does the Kernel effect system performance ?
  • 3.
    + Kernel P P P PP P H H H H Kernel
  • 4.
    + Linux KernelArchitecture Monolithic Kernel (diigo.com/085lp2) Supports for System Calls (diigo.com/085lq5) Support for Loadable Modules Implements Preemptive Multitasking Virtual Memory / Filesystem Demand Paging / Loading etc..
  • 5.
    + Linux KernelArchitecture Kernel Function Libraries – System (/proc, generic drivers) – Networking (TCP, sockets, drivers) – Storage (VFS, block devices) – Memory (VM, PM) – Processing (scheduler, fork, exec) – Human Interface (HI devices)
  • 6.
    + Linux KernelArchitecture Kernel Layers – User space interface – Virtual subsystem – L2V bridge / transformer – Logical subsystem – Hardware interface
  • 7.
    + Linux KernelArchitecture www.makelinux.net/kernel_map/
  • 8.
    + Linux Kerneland Processes Linux Kernel and Processes The Linux Kernel supports preemptive multitasking (diigo.com/085lqo) which allows multiple processes to run individually at the same time while prioritising these with higher importance What is a “Context Switch” ?
  • 9.
    + Linux Kerneland Processes Context Switch In order to support multi tasking a context switch is being handled Context switch stores or restores the CPU state to where is 'stopped' before the last CS
  • 10.
    + Linux Kerneland Processes Context switch can be forced by an interrupt timer in case of a low latency operation e.g: Disk I/O Context Switch is also being used in order to handle Interrupt requests e.g: Completed read request
  • 11.
    + Linux Kerneland Processes Context switch is being handled by the OS Each process stack has a Process Control Block which holds the relevant meta data for the CS e.g: Program Counter, Registers Address List Some CPUs (all x86 variants included) has a Task State Segment (TSS) which is used to store the PCB
  • 12.
    + Linux Kerneland Processes In case a process needs to transite to Kernel mode (during a system call) a Mode Switch is being initiated Price Context switch is the most resource consuming task of the OS.
  • 13.
    + Linux Kerneland Processes To review Context Switch statistics use – 'sar -w' – 'vmstat' Exercise:  - Review the reciprocal relations between User / System CPU utilization and CS
  • 14.
    + Linux Kerneland Processes To review an individual process structure use – 'pmap' Exercise  - Review the structure of several processes. Can you identify the process stack ?
  • 15.
    + Linux Kerneland Processes CPU Scheduler The Linux Kernel has a power full and high performance CPU scheduler. The CUP scheduler is designed to provide the most appropriate amount of CPU [time]slices – Scheduling is based on a dynamic process priority rank – Priority is changed to make sure no process is 'hogging' the CUP nor starving
  • 16.
    + Linux Kerneland Processes CPU Scheduler – For each CPU the scheduler maintain two priority ordered arrays for active and expired tasks. After the active one is done arrays are swapped (Exception: diigo.com/085lrk) 4 3 2 1 4 3 2 1
  • 17.
    + Linux Kerneland Processes CPU Scheduler – The scheduling policy involves assessing the priority for each task – There are two classifications for tasks in terms of scheduling • CPU bound • I/O bound  These can be either Interactive, Batch or RT processes
  • 18.
    + Linux Kerneland Processes CPU Scheduler – Process pre-emption occurs, when a process receives a hardware interrupt – Pre-emptive kernel, can ‘prefer’ a user mode process over a kernel mode process (even during an exception handling), in order to support interactivity
  • 19.
    + Linux Kerneland Processes CPU Scheduler – When setting ‘nice’ value to a process, one set a static portion of the scheduler priority for that task – Exercise  ‘grep a /etc/services >/dev/null’  Can you ‘tune’ this command group to run faster ?
  • 20.
    + Linux Kerneland Processes Multi Threading Multi Threading is being used in order to have several processing objects within the same process – Each Thread is being scheduled separately by the Kernel – Multiple threads share the same Memory segment
  • 21.
    + Linux Kerneland Processes Symmetric Multi Process systems can run threads of the same process simultaneously Linux implements 1-1 (1 scheduling object per thread) threading via the Native Posix Thread Library (NTPL) * Just like Solaris and FreeBSD
  • 22.
    + Linux Kerneland Processes To review Threads information use – 'ps -eLf' Exercise – Can you identify multi threaded processes on your system ? – Can you find any difference between PID and Thread ID ?
  • 23.
    + Inter ProcessCommunication IPC IPC is an implementation of techniques to exchange data between different threads IPC can be used internally or across a computer network
  • 24.
    + Inter ProcessCommunication IPC has 4 techniques – Message passing – Synchronization – Shared memory – Remote procedure call
  • 25.
    + Inter ProcessCommunication Message passing Message can be a signal or data packet Passing can be synchronous or asynchronous Synchronisation Used in order to keep several threads synchronized and able to provide the same / parallel service(s)
  • 26.
    + Inter ProcessCommunication Shared memory Simply a shared form of memory No control from of OS side RPC Method of running a procedure (subroutine) on a remote process
  • 27.
    + Inter ProcessCommunication IPC implementations PIPE – pass data between hierarchy related threads FIFO – pass data between non-related threads Socket – pass large amount of data. Can implement TCP internally Shared Memory (SHM) – share defined amount of storage between threads
  • 28.
    + Inter ProcessCommunication IPC implementations Semaphore (SEM) – shared abstract data (like a class) used to control access on a shared resource
  • 29.
    + Inter ProcessCommunication IPC Use the following commands to review IPC information – 'ipcs' – Review SHM, SEM, MSG – 'ipcrm' – clean up IPC objects – 'rpcinfo' – Review RPC – 'popen' – Create a PIPE – 'mkfifo' – Create a FIFO pipe  *Try 'man ipc'
  • 30.
    + Inter ProcessCommunication IPC Exercise – Review IPC information on your system Are there any SHMs ? – Review RPC Services available on your system – Create a named pipe. Try to use it to pass a message between two processes (e.g: bash) Can you name any advantages of using fifo ?
  • 31.
    + Linux Kernel KernelTune able parameters Changing the kernel's behaviour when working with processes, IPC, etc Tune able parameters can be used via the '/proc' filesystem or the 'syscall' command
  • 32.
    + Linux Kernel KernelTune able parameters General parameters – 'ctrl-alt-del': sets the kernel reaction to 'ctrl-alt-del' key sequence – 'domainname': sets machine's domain name – 'hostname': sets machine's host name – 'panic': sets timer before restart after panic – 'pid_max': sets maximum PID number
  • 33.
    + Linux Kernel KernelTune able parameters IPC parameters – 'shmax': maximum size of shared memory segment (bytes) – 'shmall': total amount of shared memory pages system wide – 'shmmni': maximun number of shared memory segments system wide
  • 34.
    + Linux Kernel KernelTune able parameters IPC parameters – 'sem': holds 4 parameters • maximum semaphores per array • total maximum semaphores • maximum operations per semaphore operation call • total maximum arrays
  • 35.
    + Linux Kernel KernelTune able parameters IPC parameters – 'msgmni': maximun message queues system wide – 'msgmax': maximum size of message (bytes) – 'msgmnb': maximum size of queue (bytes)