Linux Internals Part - 2

© 2012 SysPlay eLearning Academy for You <https://sysplay.in>
All Rights Reserved.
Linux Internals (Day 2)

2
© 2012 Pradeep Tewani <pradeep@sysplay.in>
What to Expect?
GNU OS & GNU Development Tools
Compiler, Assembler and linker
Writing Large Programs
Writing Makefiles
Linux Process & Process Control
Programs on Process control

GNU Operating System
A Unix-like operating system that is free software – respects your
freedom
GNU is a recursive acronym for “GNU's Not Unix” because
GNU's design is Unix like, but differs by being free software and containing
no Unix Code.
A software collection of application, libraries and developer tools.
And a program to allocate resources and talk to hardware, known as Kernel
Used with the Kernel called Linux
Together known as GNU/Linux Operating System

GNU Tools
Contains a rich set of software packages ranging from
Archiving, Audio, Business Productivity to Health,
Hobbies and so on
Archiving – cpio, gzip, tar...
Audio – EMMS, Gmediaserver
Database – Ferret, Gdbm, Guil dbi..
Games – Acm, Aetherspace
And the list continues...

GNU Development Tools
The GNU development tools are collection of
Compiler, Assembler, Linker, Libraries, Makefile,
debugger and so on.
GCC – GNU Compiler Collection
Binutils – Linker(ld), Assembler (gas) and so on
GNU C Library – libc
Debugger – GDB, DDD

GNU Compiler Collection (GCC)
Most widely used compiler
A full featured ANSI C compiler with support for
C, C++, java and Fortran
Contains the libraries for these languages as
well.
Historically, known as 'GNU C Compiler'

7
W's of Building a Program
A Process to translate a program from a Higher
Level Language to an executable
Four stages for a C Program to become an
executable
Pre-processing
Compiling
Assembling
Linking

8
Stages for Building a Program

9
Pre-Processor
Accepts Source Code as input and is responsible for
Removing Comments
Interpreting special pre-processor directives denoted by #
#include – includes contents of named file
#include <stdio.h>
#define – defines symbolic name or constants
#define MAX_SIZE 100
gcc -E hello_world.c -o hello_world.i
gives hello_world.i

10
Compiler, Assembler and Linker
Compiler
Translates a pre-Processed source code to assembly code
gcc -S hello_world.i -o hello_world.s
Gives hello_world.s
Assembler
Translate assembly code to machine code with unresolved symbols
gcc -c hello_world.s -o hello_world.o
Gives hello_world.o

11
Assembler, Compiler & Linker...
Linker
Translates the object code into the executable
All the called functions named would be replaced by
address of these function.
gcc hello_world.o -o hello_world
Gives hello_world

12
Useful gcc Compiler Options
-c
Suppress the linking process and produce a .o file for each source file listed
gcc -c file1.c file2.c ...
-llibrary: Link with object libraries which are archived.
Example : math library
gcc sqrt.c -o sqrt -lm
-L<path> - Look into the non standard path for libraries
gcc -o bar bar.o -L/home/user/ -lfoo
-I<path> - Look into the non standard path for headers
Gcc -c -I/home/user foo.c
-g : invoke debugging option
-D: define symbols either as identifies or as values

GNU Binutils
The GNU binutils are a collection of binary tools.
Most important
as – GNU assembler
ld – GNU linker
Also, includes
Addr2line, ar, gprof, nlmconv, nm, objcopy, objdump,
ranlib, readelf, size, strings, strip

Getting the Listing with Assembler
Use the '-Wa' option in gcc to pass arguments to the
assembler
The arguments must be separated from each other by
commas.
Eg:
gcc -c -g -Wa,-alh,-L file.c
-alh – emit the listing to standard output with high level and
assembly source
-L - Retain local symbols in symbol table

Exploring GCC Linking Process
gcc -c main.c func.c
Give main.o and func.o
gcc func.o main.o -o main
ldd main
Prints the shared libraries needed by program
Eg for printf,
Code Relocations
Are the entries within a binary that are left to be filled at the link time or
run time.
Use objdump and readelf

nm command
Provides the information on the symbols being used in the an object file or
executable file
Provides
Virtual address of the Symbol
A character to depict the symbol type
Lower case – Symbol is local
Upper case – Symbol is global
Name of the symbol
nm <object file or executable name>
nm -A ./*.o - display the symbol with function which contains it
nm -u <executable> – displays all the unresolved symbols in a executable file

Strings Command
Finds and displays the printable strings in given
executable, binary or object file
strings <filename>

Building Libraries
Collection of object files can be used to form a
library
Use ar command
ar cru libfoo.a foo1.o foo2.o foo3.o
Will create a file libfoo.a from foo1.o foo2.o foo3.o
Ranlib libfoo.a
Generates and adds the symbol table to .a file
Gcc main.o libfoo.a

Writing a Larger Programs
Divide the programs into modules
Separate source file. Main() in main.c and others will
contain the functions
Can Create our own library of functions.
Can be shared amongst many programs
Advantages
The modules will divide into common group of functions
We can link each module separately and link in compiled
modules

Header Files
Each module needs to have a variable definition,
function definitions etc with itself.
Several modules need to share such some
declarations.
Centralize the PreProcessor commands function
prototypes in a file called header with an extension .h.
#include <stdio.h> - Standard Header file
#inlcude “my_head.h”

Sharing the Variables
Pass the variable as Paramters
Passing long variable list to many functions can be laborious.
Large arrays and structures are difficult to store locally -
Memory problems with stack.
External Variable and functions
Defined outside of functions
Potentially available to the whole program
We have global variable – AnotherString declared in main.c and
shared with WriteMyString.

Advantages of Using Several Files
Each programmer works on different file.
An Object orientation style can be used.
Files can contain all functions from a related group.
Code re-usability for well implemented objects or
functions.
When changes are made to a file, only that file needs
to be recompiled to rebuild the program

Makefile & make Utility
make is a utility for automatically building large
applications
Files specifying the instructions for make are
Makefile in the directory which it was invoked
Make is an expert system that tracks which files
have changed since the last time the project was
build and invokes the compiler on only those files
and their dependency

Why make?
A software project which consists of many source codes,
can have complex and long compiler commands. Using
make, it can be reduced.
Programming project sometimes need specialized
compiler options that are so rarely used they are hard to
remember; with make this can be reduced.
Maintaining a consistent development environment.
Automating the build process, because make can be
called easily from a shell script.

Makefile Components
Makefile is a set of rules. Each rule has the form
TARGET: DEPENDENCIES
<TAB> COMMAND
<TAB> COMMAND
The <TABS> are Mandatory
Target
Either a name of the file that is generated by the program or the name
of the action to carry out
Object files and executable files are example of files that are generated
by other programs. Cleaning up the object files is an example of an
action that we might need to carry

Make File Components...
Dependency
A name of the file that is used to create the target
More than one dependencies must be separated out by spaces
Commands
Must be prepended with <TAB>
If the target is the file, commands explains how to create that file
If the target is an action, then the commands describe the action

How it works?
Works on the Timestamps
Dependency is up-to-date, if and only if the target's latest updates
happened after the dependency's latest update.
Dependency is changed, if and only if the target's latest update happened
before the dependency' s latest update
Makefile takes the action only if Dependency is changed.
Starts from the first target & chain proceeds and all the other
targets are ignored.
Type make <target> for specific target
Let's try a simple Makefile

Building a C Program
a.o : a.c
gcc -c a.c -o a.o
clean:
rm -rf *.o

29
What is Process?
Running instance of program
Program in execution.
Executable Program Loaded -> Process
Two instances of same program -> ? Processes

30
Program Vs Process

31
Why we need a process?
Improve the system performance
Multitasking
In turn needs
Time sharing (on same processor)
Scheduling
Priority
And for all these: Process Identifier (PID)

32
Let's view
Shell Local process: ps
Console attached System processes: ps a
All system processes: ps ax
List Many other details: Add l
Observe uid, ppid, priority, nice, status, tty, time
Dynamic process status : top
Try pid.c

33
Linux Process Scheduling
Based on the time-sharing technique
Several processes are allowed to run concurrently
CPU time is divided into “slices”, one for each
runnable process.
Preemption
Based on various task priorities
Specified by its scheduling policies

34
Types of Processes
IO Bound
Spend much of the time waiting on I/O
Runs for only short duration
CPU Bound
Spend much of the time executing code
Tend to run until they are preempted.
Linux Implicitly favours I/O bound processes over CPU
bound processes

35
Process Context Switch

36
Generic Process State Diagram

37
Linux Process States
The possible states are:
TASK_RUNNING (R) – ready or running
TASK_INTERRUPTIBLE (S) – blocked (waiting for an event)
TASK_UNINTERRUPTIBLE (D) – blocked (usually for IO)
TASK_ZOMBIE (Z) – terminated, but not cleaned by its parent
TASK_STOPPED (T) : execution stopped
Mutually exclusive
Additional Information: Foreground (+), Threaded (I)

38
Process Management in Linux
Information associated with each process
Address Space (Code, Variables, stack, heap)
Processor state (PC, Registers,..)
CPU registers
Scheduling info, priority
Memory Management information
Stored in a structure of type 'task_struct'
Maintained by Linux Kernel for each process
Also, called the Process Descriptor/
Process Control Block

39
Basic Process Management
bg – Starts a suspended process in the background
fg = Starts a suspended process in the foreground
jobs – Lists the jobs running
pidof – Find the Process ID of a running process
top – Display the processes that are using the most
cpu resources

40
Process Creation
From Shell
By running a command / Program / Script
By 'exec' ing a Command / Program
By Programming
Using System()
Simple, but inefficient
Using fork and exec family function
Comparatively complex
Greater flexibility, speed and security

41
Using the System Function
Execution Flow
Creates a sub-process running the standard shell
Hands the command to that shell for execution
Used to execute the command from within a program
Subjected to the features and limitations of the system
shell
Try this example (system.c)

42
Using fork()
Creates a child process that is an copy of its parent

43
Distinguishing Parent & Child
Child has a distinct PID from that of parent
The fork provides a different return values to
parent and child.
Return Values
PID of the child in Parent process
0 in the Child process

44
Using Exec function
An exec family of functions replace the current
program by new one
Before exec(),
PID
program
code
Program a.c
program
counter
r e g i s t e r
stack
data
section

45
Using exec function...
program
code
ProgramA.c
PID
program
counter
stack
data
section
r e g i s t e r
heap
Preserved Reset
Overwritten by the
new program
Change program code

46
Exec Family of Functions
execve()
execv() execvp()
execlp()
execle()
execl()

47
Cons of fork & exec
Fork
Forked child process typically executes the copy of
parent process' program
Can't execute the program external to current
executable
Exec
Doesn't creates a new process
Calling program ceases to execute

48
fork complementing exec
New
Copy of
Parent
Initial process
Fork
Original
process
Continues
new_Program
(replacement)
execv(new_program)
fork() returns pid=0 and runs as a
cloned parent until execv is called
Returns a
new PID

49
Copy On Write (COW)
Parent and Child shares the Address space (ro)
Data & other resources are marked COW
If written to, a duplicate is make and each process receives a unique
copy
Consequently, the duplication of resources occurs only when they
are written to
Avoids copy in case of immediate exec
fork()'s only overheads
Duplication of parent's page table
Creation of unique PCB for child

50
Process Termination
Parent and Child process terminate as usual
Success Exit – Normal Success
Error Exit – Normal failure
Fatal Exit – Signaled from Kernel space for a bug
Kill Exit – Signaled by process
But which one of them terminates first?
If it matters, parent can wait for their children using
wait family of system calls.

51
Wait family of system calls
Four different system calls in the wait family
wait() : Block until one of the child process terminates
waitpid() : Wait for a specific child to exit/stop/resume.
wait3() : Along with, return the resource usage info about
exiting/stopping/resuming child process.
Wait4() : wait3() for specific child or parent.
All of these fill a status code
In an integer pointer argument
About how the child process exited
Which can decoded using...

52
wait status macros
WEXITSTATUS : Extracts the child process exit code
WIFEXITED: Determine whether child process exited normally or
died from unhandled signal
WIFSIGNALED : Determines if the child process was signaled
WTERMSIG : Extracts from the process' exit status, the signal by
which it died
Other
WIFSTOPPED, WCOREDUMP, WSTOPSIG, WIFCONTINUED

53
Zombie Process
A process that is terminated but the resources
are not yet cleaned up is called zombie process
Reason
The child process exited before the parent
The parent didn't call the wait
Even if a parent does a wait later, it remains a
Zombie till then.

54
Who cleans up a Zombie?
Typically, again a Parent
By doing a wait on it
What if the parent exits without “wait”?
Does it stay around in the system?
Not really. It gets inherited by init
which then cleans it up right there

55
Orphan Process
If the parent dies before the children
Child become orphan
And are adopted by init
which then does the clean up on their exit

56
What all we have learnt?
GNU OS & GNU Development Tools
Compiler, Assembler and linker
Writing Large Programs
Writing Makefiles
Linux Process & Process Control
Programs on Process control

57
Any Queries?

Linux Internals Part - 2

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Linux Internals Part - 2

Similar to Linux Internals Part - 2 (20)

More from SysPlay eLearning Academy for You

More from SysPlay eLearning Academy for You (15)

Recently uploaded

Recently uploaded (20)

Linux Internals Part - 2