2. Shell Programming & Scripting Languages
Course Objectives
ď˝ This course explains UNIX Operating System in thorough with
its commands, editor, file systems along with applications.
ď˝ Implementation of Shell scripts with Bash shell programming
using functions, Process communication, Administration.
ď˝ Scripting languages extends the functionality of existing
software. This was addressed by the Python language with its
features and programming with object oriented concepts.
ď˝ This course also explains about the designing of integrated web
Applications and Data base interfaces.
3. Course Outcomes
ď˝ Identify the basic commands as well as different utilities of the UNIX
& analyze why to use unix.
ď˝ Understand the commands, functions and Practice with real time
applications
ď˝ Learn about shell and how to write, debug & execute shell script along
with different commands.
ď˝ Explain file handling and directory handling using different system
calls.
ď˝ Discuss the fundamentals of the Process as well as Signal functions &
controlling of process using signals.
ď˝ Learning the Python language programming with its features
ď˝ Design the web applications through python
ď˝ Implementation of object oriented programming through python.
5. UNIT â I Contents
ď˝ Introduction to Unix:- Architecture of Unix, Features of
Unix , Unix utilities â process utilities, disk utilities,
networking commands, text processing utilities and
backup utilities.
ď˝ Introduction to UNIX file system, vi editor, file handling
utilities, security by file permissions.
6. Software
ď˝ Collection of Programs
ď˝ 2 types
S y s t e m
S o f t w a r e
A p p lic a t io n
S o f t w a r e
S o f t w a r e
7. ď˝ Application software
ď˝ Easy-to-use programs designed to perform specific
tasks
ď˝ System software
ď˝ Programs that support the execution and
development of other programs
ď˝ Two major types
ď˝ Operating systems
ď˝ Translation systems (compilers & linkers)
Software
10. ď˝ Controls and manages the computing resources
ď˝ Examples
ď˝ Windows, Unix, MSDOS,
ď˝ Important services that an operating system provides:
ď˝ Security: prevent unauthorized users from accessing
the system
ď˝ Commands to manipulate the file system
ď˝ Input and output on a variety of devices
ď˝ Window management
Operating System
11. Operating System
An Operating System is a system software that provides
interface between the user and computer hardware.
Operating System controls (manages) hardware and
software.
provides support for peripherals such as keyboard,
mouse, screen, disk drives, âŚ
ďŽ software applications use the OS to communicate with
peripherals.
ďŽ The OS typically manages (starts, stops, pauses, etc)
applications.
12. What is UNIX
⢠UNIX is a Multi-user and Multi-tasking operating
system.
⢠That is it is designed to be used by many people at the
same time (multi-user) and for multiple tasks at a time.
⢠Runs on a variety of processors
⢠It provides a number of facilities:
â management of hardware resources
â directory and file system
â loading / execution / suspension of programs
12
13. UNIX versions
Multics
Unics
UNIXV1
UNIXV3
UNIXV4
UNIXV7
1965 1969 1970 1973 1973 1979
BSD
UNIX
AT&T
UN
IX
BSD
4.1
BSD
4.2
BSD
4.4
1981 1983 1993
System
III
System
V
1982 1984
1998 â X/OPEN & IEEE started work to merge two standards
2001 â Single UNIX Specification Version 3 (SUSV3)
Also known as IEEE1003.1:2001
2002 â ISO standardized SUSV3 and IEEE1003.1:2001
Multiple Standards
AT&T â System V Interface Definition
X/OPEN â X/OPEN Portability Guide (XPG)
IEEE â Portable Operating System Interface for Computing Environments (POSIX)
14. UNIX History
ď˝ The UNIX operating system was developed at AT & T Bell
Labs in late 1960s.
ď˝ It originally began as a one man project led by Ken Thompson
of Bell Labs, and has since grown to become the most widely
used operating system.
ď˝ In the time since UNIX was first developed, it has gone through
many different generations and even mutations.
ď˝ Some differ substantially from the original version, like
Berkeley Software Distribution (BSD) or Linux.
15. Why to Use UNIX?
⢠Multi-tasking / multi-user
⢠Networking capability
⢠Distributed Processing
⢠graphical (with command line)
⢠Easy to program
⢠portable (PCs, mainframes, super-computers)
15
16. Features of UNIX Operating System
ď˝ Multi-user & Multi-tasking â UNIX allows multiple users to log
onto the system, and have each run multiple tasks. This is standard
for most modern OS.
ď˝ Large Number of Applications â There are an enormous amount of
applications available for UNIX. They range from commercial
applications such as CAD, Maya, WordPerfect, to many free
applications.
ď˝ Free Applications and Open source Operating System - of all of
the applications available under UNIX, many of them are free. Most
of the development that we do in programming courses is done under
the Linux OS.
ď˝ Less Resource Intensive - UNIX installations tend to be much less
demanding on system resources. In many cases, the old family
computer that can barely run Windows is more than sufficient to run
the latest version of Linux.
ď˝ Internet Development - Much of the backbone of the Internet is run
by UNIX servers. Many of the more general web servers run UNIX
with the Apache web server - another free application.
17. Structure of the UNIX system
Applications
Shell
Kernel (OS)
Hardware
There are many
standard applications:
⢠file system commands
⢠text editors
⢠compilers
⢠text processing
18. UNIX System Structure âŚ..
ď˝ The Kernel - handles memory management, input and output
requests, and program scheduling. The kernel is the heart of OS. It
provides the basic software connection to the hardware.
ď˝ The Shell and Graphical User Interfaces (GUIs) - UNIX shell
provides a âcommand lineâ interface which allows the user to type
in commands. These commands are translated by the shell into
something the kernel can comprehend, and then executed by the
kernel.
ď˝ Primary command-line shells are:
ď˝ Bourne shell, Bash Shell, korn Shell, C Shell
19. ď˝ The Built-in System Utilities - Programs that allow a user to
perform tasks which involve complex actions. Utilities provide
user interface functions that are basic to an operating system.
Examples: Commands to see the contents of a directory, move
& copy files, remove files, etc...
ď˝ Application Software & Utilities â The programs that are
bundled with the OS distribution, or available separately.
These can range from additional or different versions of basic
utilities, to full scale commercial applications.
UNIX System Structure âŚ..
20. Flavors of Unix
There are many versions of Unix:
SCO UNIX
SysV (from AT&T)
ďŽ BSD (from Berkeley)
ďŽ Solaris (Sun)
ďŽ IRIX (SGI)
ďŽ AIX (IBM)
ďŽ LINUX (free software)
21. ď˝ File System is a group of files
ď˝ The relevant information arranged in a logical manner.
ď˝ Different Operating systems have different file systems.
ď˝ In Unix, everything will be treated as a file. Whether it is application or
device or utility or data or directory
ď˝ It is similar to tree data structure.
ď˝ The files will be denoted with absolute or relative path
ď˝ Ex: /home/usr2/file1 ď absolute path
ď˝ Relative path is nothing but pwd (present working directory)
ď˝ cd ram ď relative path
ď˝ To check the block size the command is $cmchk
The UNIX File System
22. The UNIX File System
⢠A simplified UNIX directory/file system:
/ Root
bin
dev
lib
User
home/usr
tmp
. . .. . .
User2User1 User3
22
cat date
etc
23. ⢠/bin: contains executable files for most of the unix
commands.
⢠/dev: contain files that control various input & output
devices.
⢠/etc: contains system administration files, such as
password file.
⢠/lib: contains all the library functions in binary form.
⢠/usr: contains several directories each associated with a
particular user.
⢠/tmp: contain the temporary files created by unix or by
any user.
⢠/etc: contains configuration files of the system.
23
24. ⢠File: It is a container for storing information.
⢠A file is of the following types.
⢠Text file or Ordinary file : contains printable
characters.
Ex: program source code, documents, scripts
⢠Binary file: contains both printable & non printable
characters. Ex: gif, jpeg, executable etc..
⢠Directory file: It maintains some details of the files &
subdirectories that it contains.
24
Files
25. ⢠Every directory entry contains 2 components.
⢠1.file name.
⢠2. Unique identification number for the file or
directory.
25
26. Files
ď˝ The file systems in UNIX maintains with four blocks
ď˝ Boot Block ď Bootstrap loader program
ď˝ Super Block ď state of the file systemâ how large it
is, etc..
ď˝ Inode Table
ď˝ Data Block ď contains actual file contents. An
allocated block can belong to only one file in the file
system.
27. Inode Table
ď˝ The information related to all the files is stored in an Inode Table on
the disk.
ď˝ For each file there is an inode entry in the table.
ď˝ Each entry is made up of 64 bytes and contains relevant details for
that file.
ď˝ The details are:
ď˝ Owner of the file
ď˝ Group to which the owner belongs
ď˝ Type of the file
ď˝ File access permissions
ď˝ Date and time of last access
ď˝ Date and time of last modification
ď˝ Number of links to the file
ď˝ Size of the file
ď˝ Addresses of the blocks where the file is physically present
28. How Does Unix Access Files?
ď˝ A file is identified by Unix by a unique â Inode numberâ
associated with it.
ď˝ Can display the inode number with a file by using the
command ls-i
ď˝ Ex: $ls â i reports
ď˝ reports 12324
ď˝ Here 12324 is inode number.
ď˝ The inode no is nothing but the index into the inode table
where the information about the file is stored.
ď˝ Among all the slots the inode table slot number 12324 contains
information about the file reports
29. Storage of Files
ď˝ Each inode entry in the inode table specifies
completely where the contents of the files are stored
on the disk.
ď˝ These addresses may be numbered 0 through 12.
ď˝ The addresses 0 to 9 point to 1kb blocks on disk.
ď˝ Ex: a file of 3kb may have its entries as shown in the
following diagram.
31. Creating files
ď˝ cat
ď˝ Used to create a file
ď¨ To append the files
ď¨ To display the file content.
ď˝ $cat > test
ď˝ Type the content of the file and press Ctrl d to stop / save
the file.
ď˝ $cat test
ď˝ Displays the content of file test.
ď˝ $cat sample1 sample2 >> newsample
ď˝ newsample file will be created with the contents of sample
and sample2
32. Creating files âŚ
ď˝ If new sample already contains something it would be
overwritten.
ď˝ If you want it should remain intact and content of
sample1 and sample2 should get append to it then use
â append output redirection operatorâ >>
ď˝ Ex: $ cat sample1 sample2 >> newsample
33. Cp (Copying Files)
⢠To create an exact copy of a file you can use the âcpâ command.
Syntax: cp [-option] source destination
Eg: cp file1 file2
Here the contents of file1 is copied to file2. if file2 is not existed
then it will be created.
Eg: cp file1 file2 dir
file1 file2 contents are copied to the directory named as dir.
⢠To copy files from or to different directories is by specifying their
names with the path.
⢠$cp /usr/aa/chapter1 /usr/aa/newbook/chap1
33
File handling utilities
34. Cp turns to interactive when âi option is used &
destination file also exists.
$cp -i file1 file2
overwrite file2 (yes/no)?
Y at this prompt overwrites the file.
34
Copying Files
35. ⢠mv (Moving and Renaming Files)
Used to rename the files / directories.
$ mv test sample
Here test is renamed as sample, provided sample
directory name is not already existing.
⢠Moving a file implies removing it from its current
location and copying it at a new location.
35
mv command
36. Listing Files and Directories
ď˝ $ls ď lists files and directories
ls â a ď hidden file. Displays the hidden files also.
$ ls p* ď lists the files starts with p
$ls ?ain ď lists the files whose names match with ain from
second character onwards
ls [a-m][c-z][4-9]?? ď this will list all 5 character filenames in the
current directory whose first character is in the range a to m,
second character in the range c to z, the third character is in
the range 4 to 9, where as the fourth and fifth character are
any valid characters.
37. ls - l
ď˝ ls âl
ď˝ long listing
ď˝ Out put:
total 22 ď no. of disk blocks occupied
-rwxr-xâx 1 user1 group 02 jan 16 10:15 ecm32
drwxr-xr-x 1 user1 group 04 jan 16 12:23 ecm31
38. File types ( meaning of first character)
File Type Meaning
- Ordinary file
d Directory file
c Character special file
b Block special file
l Symbolic link file
s Semaphore
p Named pipe
M Shared memory file
40. ⢠Used to create links (both soft & hard links).
⢠It creates the alias & increase the link count by one.
⢠$ln file1 file2
⢠Hard link means the link will be established with the
data on the disk
⢠Softlink is nothing but symbolic link that refers to
existed filename. ( it refers to fine name itself.)
40
ln command
43. wc command
ď˝ Counts the number of lines, words and characters in the
specified file or files.
ď˝ -l ď counts and display no. of lines in the file
ď˝ -w ď counts and display no. of words in the file
ď˝ -c ď counts and display no. of bytes in the file.
ď˝ -m ď displays the no. of characters in the file.
ď˝ -L ď displays the length of the longest line in the file.
ď˝ $ wc âlc file1 file2
44. ⢠Rm (Deleting Files)
â To delete or remove a file, you use the ârmâ
command.
â Ex 1: $rm mylisting will delete âmylistingâ.
-f deletes the file forcibly
âi option removes the files interactively.
Ex 2: $rm âi file1
With âr option recursively removes directories.
$rm âr dir1
44
45. ⢠rmdir (remove directories )
⢠Removes empty directories.
⢠$rmdir book
removes directory named book if it is empty.
⢠$rmdir dbs doc dmc
Removes the directories dbs doc dmc.
45
46. ⢠find: Searches the files in a directory hierarchy
⢠It recursively examines a directory tree to look for
matching file and then takes some action on the
selected files.
Syntax:
⢠find path_list selection_criteria action
⢠To locate all files named a. out use
$find / -name a. out âprint
⢠â/â indicates search should start from root directory.
46
47. ⢠To locate all c files in current directory
$find . -name â*.câ âprint
⢠To find all files begin with an uppercase letter use
$find . âname â[A-Z]*â âprint
Find operators:
Find uses 3 operators
!,-a ,-o
47
48. Security by file permissions
Unix follows a 3-tiered file protection system.
There are 3 categories- owner ,group ,others.
ownerâs permission group permission otherâs permission
r w x r - x r - -
Each category contains read ,write ,execute permissions .
rwx->presence of all permissions.
r-x->absence of write permission
r-- -> absence of write ,execute permission
48
49. Security by file permissions
⢠Chmod: change file permission mode
chmod sets a fileâs permission (read, write
and execute) mode for all three categories of users
(owner, group and others)
49
50. Syntax
chmod category operation permission file(s)
The command contains three components:
⢠category of user (owner, group or others)
⢠operation to be performed (assign or remove a
permission)
⢠Permission type (read, write or execute)
50
53. chown
ď˝ This command changes file owner and / or group
ownership for each given file.
ď˝ Syntax: chown [OPTION]... [OWNER][:[GROUP]] FILE
ď˝ Ex1: chown ramu file1
ď˝ Ex2: chown ram /path/to/file1 /path/to/file2 /path/to/file3
or
$ chown ram /path/to/{file1, file2, file3}
54. Disk related CommandsÂ
ď˝ $df: disk filesystem
ď˝ Displays free as well as used disk space for all the file systems on the
computer.
ď˝ i.e. The information of device name, total blocks, total disk space, used
disk space, available disk space and mount points on a file system.
ď˝ Filesystem 1K-blocks Used Available Use% Mounted
on
ď˝ /dev/cciss/c0d0p2 78361192 23185840 51130588 32% /
ď˝ /dev/cciss/c0d0p5 24797380 22273432 1243972 95% /home
ď˝ Ex: $df
ď˝ / (/dev/root): 12970 blocks 27857 i-nodes
ď˝ The root file systems is /dev/root.df reports the number of free disk
blocks and free inodes for this file system.
ď˝ If we want a more detailed information about disk usage we have to use:
ď˝ $df -ivt
55. disk utilities
ulimit
user limit and it contains a value which signifies the
largest file that can be created by the user in the file system.
Ex: $ulimit
2097152 ď the user cannot create a file whose size is
bigger than 2097152 bytes, or 2048 KB.
To reduce this value:
$ulimit 1
Allocates 512 bytes of space. After the user logout the value will
change
57. file commands âŚ
rm ď used to remove a file. If you have write permission
to the file, then only you can remove the file.
mkdir ď creates a directory.
$mkdir ecm
$mkdir âp works/bpb/unix/book
$mkdir dbs doc dmc
Creates  three directories
pwd: present working directory
58. sortÂ
ď˝ used to sort the contents of a file.
ď˝ Can merge multiple sorted files and store the result
in the specified output file.
ď˝ $ sort myfile
ď˝ To sort the contents of several files at one shot:
ď˝ $sort âo result file1 file2 file3
ď˝ To merge already sorted files
ď˝ sort âm file1 file2
Text Processing Utilities
59. Sort options
-b Ignores leading spaces and tabs
-c Checks if files are already sorted. If already sorted then it
do nothing
-d Sorts in directory order
-f Ignore cases
-m Merges files that have already been sorted.
-n Sorts in numeric order
-ofile Sorts output file.
-r Reverses sort
-tc Separates fields with character
-u Unique output
60. cut command
ď˝ It cuts or picks up a given number of character or fields
from the specified file.
ď˝ In a database, to see only some fields( say 2nd
, 7th
fields)
then use cut command.
ď˝ $cut âf 2,7 emp
ď˝ To see from 2nd
field upto 7th
field.
ď˝ $cut âf 2-7 emp
61. head, tail  â Other text processing UtilitiesÂ
To view the beginning or end of the lines of a file use head
and tail commands.
$head -5 myfile
$pg +10 -15 -p âpage no. %dâ âs myfile
Starts displaying of myfile contents, 15 lines at a time from
10th
line onwards. At the end of each page the page no
will be displayed.
62. diff command
ď˝ Compares two files, directories and reports all the
differences between two files.
ď˝ diff file1 file2
64. more example
more +10 -15 -s âd myfile yourfile
ď˝ -s option squeeze multiple blank lines in a file to a single
blank line.
ď˝ -d option changes the normal -- more â prompt
displayed at the end of each page to explanatory â[ hit
space to continue, delete to abort]â
70. nohup command
ď˝ The processes we have executed should not die even
after we log out, for this purpose we have to use nohup
command.
ď˝ nohup stands for no hangups.
ď˝ Ex: $ nohup sort emp.dat > output.emp
ď˝ If we donât mention the output file then the sorted
output will be stored in the file nohup.out
71. Killing a process
ď˝ In case of system hanging, we have to kill the running
process.
ď˝ To kill the process, we must know the process-id using
ps command.
ď˝ Ex: kill 9876
72. Changing Process Priorities
ď˝ The priority of a process is decided by a number
associated with a number. This number is called nice
value of the process.
ď˝ The nice value of a process can range between 0 to 39.
ď˝ 20 is the default value of the process.
ď˝ nice cat employee.dat
ď˝ nice value increases to 30
ď˝ $nice -15 cat employee.dat
73. at command, batch command
ď˝ Used to execute the unix commands at a future date and
time.
ď˝ $at 16:30
ď˝ echo â itâs 4:30 PM! Backup your files and logoutâ
ď˝ ctrl + d
batch command:
The system decides when to execute a sequence of
commands. i.e. whenever it is free, the system will
execute these commands.
74. crontab command
ď˝ More powerful than at command. Carry out a submitted
job everyday for years together.
77. finger options
-i displays the idle time of the system
Ex: $ finger âi
Login tty when idle
Ramu *tty01 Wen Jan27 8 Minutes 14 seconds
* means message is set to -n
79. Compress and Un-Compress files
ď˝ compress -v file_name
gzip filename
ď˝ uncompress file_name.Z
or
gunzip filename
80. Disk utilities
⢠Du: disk usage
⢠Du command estimate the file space usage on the
disk.
⢠It produces a list containing the usage of each
subdirectory of its argument and finally produces a
summary.
⢠$du /home/usr1
80
81. Disk utilities
⢠Df: displays the amount of free space available on the
disk. The output displays for each file system
separately.
⢠$df
⢠Mount:
⢠Used to mount the file systems.
⢠Takes 2 arguments-device name ,mount point.
81
82. Disk Utilities
⢠Mount uses an option to specify the type of file
system.
⢠To mount a file system on the /oracle directory on
Linux system use
$mount ât ext2 /dev/hda3 /oracle
$mount ât iso9660 /dev/cdrom /mnt /cdrom
$mount ât vfat /dev/hda1 /msdos
$mount ât msdos /dev/fd0 /floppy
82
83. Disk utilities
⢠Umount: unmounting file systems
⢠Unmounting is achieved with the umount command.
which requires either file system name or the mount
point as argument.
⢠$umount /oracle
⢠$umount /dev/hda3
⢠Unmounting a file system is not possible if the file is
opened.
83
84. ⢠ulimit: user limit
⢠It contains a value which signifies the largest file that
can be created by the user in the file system.
⢠When used by itself it displays the current setting.
⢠$ulimit
unlimited
User can also set the ulimit value by using
$ulimit 10
84
85. unmask:
When u create files and directories, the default
permissions that are assigned to them depend on the
systemâs default setting. Actually this default is
transformed
By subtracting the user mask from it to remove one or
more permissions. This value is evaluated by umask
without arguments.
$umask
022
85
86. Networking commands
⢠ftp: file transfer protocol
⢠ftp is used to transfer files. It can be used
with host name.
$ftp Saturn
Connected to Saturn
220 Saturn ftp server
Name (Saturn: summit ): Henry
Password: ******
86
87. ⢠To quit ftp use close and then bye or quit.
⢠ftp>close
221 good bye
⢠ftp>bye
⢠Transferring files:
can be of 2 types.
⢠Uploading( put & mput):
⢠To upload the web pages & graphic files to website.
⢠The put command sends a single file to the remote machine.
87
88. ⢠ftp>binary
200 type set to I
⢠ftp>put penguin. gif
To copy multiple files use mput.
⢠ftp>mput t*.sql
Downloading files: get & mget
⢠To download the files from remote machine use get &
mget.
⢠ftp>get ls-lR.gz
⢠ftp>_
88
89. Networking commands
telnet: Remote login
If u have an account on the host in a local network (or
on internet ),u can use this with the host name or the
ip address as argument.
$telnet Saturn
Trying to 192.168.0.1âŚ
Connected to Saturn
89
90. ⢠Login:----
⢠Password:-----
⢠quit telnet by using exit command.
⢠telnet prompt:
When telnet used without Ip address the system displays a telnet>
prompt . U can invoke a login session from here with open.
telnet> open 192.168.0.8
Trying to 192.168.0.8âŚ
Connected to 192.168.0.8
90
91. Networking commands
⢠rlogin: remote login without password
⢠rlogin is the Berkley's implementation of the remote
login facility.
⢠U can log on to ur own identical remote account
without using either the user name or password.
⢠$rlogin Jupiter
⢠Last login :âŚ.
⢠rlogin is terminated with ctrl+d or exit or logout.
91
92. vi Editor
⢠vi is a full screen text editor
⢠It was created by Bill Joy.
⢠Bram Moolenaor improved it and called it vim (vi
improved).
⢠Invoking vi:
⢠$Vi file name
92
93. vi Editor
We notice a tilde (~) on each line following the cursor. A tilde
represents an unused line. If a line does not begin with a tilde and
appears to be blank, there is a space, tab, newline, or some other
nonviewable character present.
94. vi editor
Command Description
vi filename Creates a new file if it already does
not exist, otherwise opens existing
file.
vi -R filename Opens an existing file in read only
mode.
view filename Opens an existing file in read only
mode.
95. Modes of operation
⢠Vi has 3 mode of operation.
1.Command mode: In this mode all the keys pressed by
the user are interpreted as commands. It may perform
some actions like move cursor, save, delete text, quit
vi, etc.
2. Input/Insert mode: used for inserting text.
â start by typing i; finish with Esc
95
96. Modes of operation
⢠Ex mode or last line mode:
⢠Used for giving commands at command line.
⢠The bottom line of vi is called the command
line.
96
97. Basic Cursor Movements
j down one
k up one
h move cursor one place to left
l right one
w move forward one word
b back one word
97
98. Finishing a vi Session
⢠Get to command mode (press ESc)
ZZ save changes to the file and quit
(no RETURN)
:q! quit without saving
(press RETURN)
:wq! Saves the file & quit.
98
99. Inserting Text
⢠Move to insertion point
⢠Switch to input mode: i
⢠Start typing; BACKSPACE or DELETE
for deletion
⢠ESC finish; back in command mode
No RURN
99
100. Deletion
⢠Must be in command mode.
x Delete character that cursor is on.
dd Delete current line.
D Delete from cursor position to
end of line
u Undo last command
100
101. Screen Commands
⢠Ctrl-f To move screen forward
⢠Ctrl-b To move one screen backwards.
⢠Ctrl-u Moves the cursor half a screen forward.
⢠Ctrl- d Moves the cursor half screen backward.
⢠Ctrl-r
or
Ctrl-l To redraw the original screen.
⢠Ctrl-g Displays the status on the status line.
name of file, current line number