System Programming

1. 1
Chapter 2 : UNIX Commands

2. 2
What is Unix?
 A multi-user networked operating system
» “Operating System”
– Handles files, running other programs, input/output
– Looks like DOS…but more powerful
– The internet was designed on it, thus networking is an
intrinsic part of the system
» “Multi-user”
– Every user has different settings and permissions
– Multiple users can be logged in simultaneously

3. 3
Kernel (OS)
Interacts directly with the hardware
through device drivers
Provides sets of services to programs,
insulating these programs from the
underlying hardware
Manages memory, controls access, maintains
file system, handles interrupts, allocates
resources of the computer
Programs interact with the kernel
through system calls

4. 4
Structure of the UNIX system
Applications
Shell
Kernel (OS)
Hardware
There are many
standard applications:
• file system
commands
• text editors
• compilers
• text processing

5. 5
Unix and Users
Most flavors of Unix (there are many)
provide the same set of applications
to support humans (commands and
shells).
Although these user interface programs
are not part of the OS directly, they are
standardized enough that learning your
way around one flavor of Unix is enough.

6. 6
Flavors of Unix
There are many versions of Unix that
are used by lots of people:
 SysV (from AT&T)
 BSD (from Berkeley)
 Solaris (Sun)
 IRIX (SGI)
 AIX (IBM)
 LINUX (free software)

7. 7
Unix vs. Linux
 Age
» Unix: born in 1970 at AT&T/Bell Labs
» Linux: born in 1992 in Helsinki, Finland
 Sun, IBM, HP are the 3 largest vendors of Unix
» These Unix flavors all run on custom hardware
 Linux is FREE!
» Linux was written for Intel/x86, but runs on many platforms

8. 8
UNIX Shell
 A system program that allows a user to execute:
» shell functions (internal commands)
» other programs (external commands)
» shell scripts
 Linux/UNIX has a bunch of them, the most common
are
» tcsh, an expanded version of csh (Bill Joy, Berkley, Sun)
» bash, one of the most popular and rich in functionality
shells, an expansion of sh (AT&T Bell Labs)
» ksh, Korn Shell
» zhs
» ...

9. 9
Command Format
 Format: command name and 0 or more arguments:
% commandname [arg1] ... [argN]
 % sign – represents prompt.
 Arguments can be
» options (switches to the command to indicate a mode of
operation) ; usually prefixed with a hyphen (-) or two (--) in GNU
style
» non-options, or operands, basically the data to work with (actual
data, or a file name)

10. 10
Shell I/O
 Shell is a “power-user” interface, so the user interacts
with the shell by typing in the commands.
 The shell interprets the commands, that may produce
some results, they go back to the user and the control is
given back to the user when a command completes
(in general).
 In the case of external commands, shell executes actual
programs that may call functions of the OS kernel.
 These system commands are often wrapped around a
so-called system calls, to ask the kernel to perform an
operation (usually privileged) on your behalf.

11. 11
Command I/O
 Input to shell:
» Command name and arguments typed by the user
 Input to a command:
» Keyboard, file, or other commands
 Standard input: keyboard.
 Standard output: screen.
 These STDIN and STDOUT are often together referred to as a
terminal.
 Both standard input and standard output can be redirected
from/to a file or other command.
 File redirection:
» < input
» > output
» >> output append

12. 12
Common Commands

13. 13
man
 Manual Pages
 The first command to remember
 Contains info about almost everything :-
» other commands
» system calls
» c/library functions
» other utils, applications, configuration files
 To read about man itself type:
% man man
 NOTE: unfortunately there’s no
% man woman ...

14. 14
which
 Displays a path name of a command.
 Searches a path environmental variable for
the command and displays the absolute path.
 To find which tcsh and bash are actually in
use, type:
% which tcsh
% which bash
 % man which for more details

15. 15
chsh
 Change Login Shell
 Login shell is the shell that interprets commands
after you logged in by default.
 You can change it with chsh (provided that your
system admin allowed you to do so).
 To list all possible shells, depending on
implementation:
% chsh -l
% cat /etc/shells
 % chsh with no arguments will prompt you for the
shell.

16. 16
whereis
 Display all locations of a command (or some other
binary, man page, or a source file).
 Searchers all directories to find commands that
match whereis’ argument
 % whereis tcsh

17. 17
passwd
 Change your login password.
 A very good idea after you got a new one.
 It’s usually a paranoid program asking your
password to have at least 6 chars in the
password, at least two alphabetical and one
numerical characters. Some other restrictions (e.g.
dictionary words or previous password similarity) may
apply.
 Depending on a privilege, one can change user’s
and group passwords as well as real name, login
shell, etc.
 % man passwd

18. 18
date
 Displays dates in various formats
 % date
 % date -u
» in GMT
 % man date

19. 19
cal
 Calendar
» for month
» entire year
 Years range: 1 -
9999
 No year 0
 % cal current month
 % cal 2 2000 Feb 2000, leap
year
 % cal 2 2100 not a leap year
 % cal 2 2400 leap year
 % cal 0 error
 % cal 2002 whole year

20. 20
clear
 Clears the screen
 There’s an alias for it: Ctrl+L
 Example sequence:
» % cal
» % clear
» % cal
» Ctrl+L

21. 21
sleep
 “Sleeping” is doing nothing for some time.
 Usually used for delays in shell scripts.
 % sleep 2 2 seconds pause

22. 22
Command Grouping
 Semicolon: “;”
 Often grouping acts as if it were a single
command, so an output of different commands
can be redirected to a file:
 % (date; cal; date) > out.txt

23. 23
alias
 Defined a new name for a command
 % alias
» with no arguments lists currently active aliases
 % alias newcommand oldcommand
» defines a newcommand
 % alias cl cal 2003
 % cl

24. 24
unalias
 Removes alias
 Requires an argument.
 % unalias cl

25. 25
history
 Display a history of
recently used
commands
 % history
» all commands in the
history
 % history 10
» last 10
 % history -r 10
» reverse order
 % !!
» repeat last command
 % !n
» repeat command n in the
history
 % !-1
» repeat last command =
!!
 % !-2
» repeat second last
command
 % !ca
» repeat last command that
begins with ‘ca’

26. 26
apropos
 Search man pages
for a substring.
 % apropos word
 Equivalent:
 % man -k word
 % apropos date
 % man -k date
 % apropos password

27. 27
exit / logout
 Exit from your login session.
 % exit
 % logout

28. 28
shutdown
 Causes system to shutdown or reboot cleanly.
 May require superuser privileges
 % shutdown -h now - stop
 % shutdown -r now - reboot

29. 29
Directories and Files

30. 30
Directories
 Files are grouped together in other files called directories -
analogous to folders in Windows
 Directory paths are separated by a forward slash: /
» Example: /homes/iws/robh/classes/cse326
 The hierarchical structure of directories (the directory tree)
begins at a special directory called the root, or /
» Absolute paths start at / (root directory)
– Example: /homes/iws/robh/classes/cse326
» Relative paths start in the current directory
– Example: classes/cse326
 Your home directory “~” is where your personal files are
located, and where you start when you log in.
» Example: /homes/iws/robh

31. 31
Directories (cont’d)
 Handy directories to know: ~ Your home directory, .. The parent
directory, . The current directory
 ls
» LiSts the contents of a specified files or directories (or the current
directory if no files are specified)
» Syntax: ls [<args> … ]
» Example: ls backups/
» Has whole bunch of options, e.g
– % ls -a : all files except those starting with a “.”
 pwd
» personal working directory - displays Working Directory
 find
» Looks up a file in a directory tree.
» Syntax: find . -name name
» Example: find . (-name ‘w*’ -or -name ‘W*’ )

32. 32
Directories (cont’d further)
 cd
» Change current Directory to a new one – absolute or relative
» Syntax: cd <directory>
» Examples:
– cd backups/unix-tutorial
– cd ../class-notes
– cd returns to home directory
 mkdir
» MaKe DIRectory - Creates a directory
» Syntax: mkdir <directories>
» Example: mkdir backups class-notes
 rmdir
» ReMove DIRectory, which must be empty first
» Syntax: rmdir <directories>
» Example: rmdir backups class-notes

33. 33
Files
 Unix file types (e.g. “executable files, ” “data files,” “text
files”) are not determined by file extension (e.g.
“foo.exe”, “foo.dat”, “foo.txt”) – unlike in Windows
 Thus, the file-manipulation commands are few and
simple …
 Many use only 2 letters
 rm
» ReMoves a file, without a possibility of “undelete!”
» Syntax: rm <file(s)>
» Example: rm tutorial.txt backups/old.txt

34. 34
Files (cont’d)
 cp
» CoPies a file / directory, preserving the original
» Syntax: cp [options] <sources> <destination>
» Example: cp tutorial.txt tutorial.txt.bak
» Useful option: -i to prevent overwriting existing files and
prompt the user to confirm.
 mv
» MoVes or renames a file / directo, destroying the original
» Syntax: mv <sources> <destination>
» Examples:
– mv tutorial.txt tutorial.txt.bak
– mv tutorial.txt tutorial-slides.ppt backups/

35. 35
cat
 Display and concatenate files.
 % cat
» Will read from STDIN and print to STDOT every line you
enter.
 % cat file1 [file2] ...
» Will concatenate all files in one and print them to STDOUT
 % cat > filename
» Will take whatever you type from STDIN and will put it into
the file filename
 To exit cat or cat > filename type Ctrl+D to
indicate EOF (End of File).

36. 36
more / less
 Pagers to display contents of large files page by
page or scroll line by line up and down.
 Have a lot of viewing options and search capability.
 Interactive. To exit: ‘q’

37. 37
less
 less ("less is more") a bit more smart than the more
command
 to display contents of a file:
» % less filename
 To display line numbers:
» % less -N filename
 To display a prompt:
» % less -P"Press 'q' to quit" filename
 Combine the two:
» % less -NP"Blah-blah-blah" filename
 For more information:
» % man less

38. 38
touch
 By touching a file you either create it if it did not
exists (with 0 length).
 Or you update it’s last modification and access
times (update the time stamps of the files listed on
the command line to the current time)
 There are options to override the default behavior.
 % touch file
 % man touch

39. 39
script
 Writes a log (a typescript) of whatever
happened in the terminal to a file.
 % script [file]
 % script
» all log is saved into a file named typescript
 % script file
» all log is saved into a file named file
 To exit logging, type:
» % exit

40. 40
File attributes
Every file has some attributes:
 Access Times:
 when the file was created
 when the file was last changed
 when the file was last read
 Size
 Owners (user and group)
 Permissions
 Type – directory, link, regular file, etc.
 ACLs – access control lists (not today)

41. 41
File Time Attributes
Time Attributes:
 when the file was last changed ls -l
 sort by modification time ls -lt

42. 42
File Owners
Each file is owned by a user.
You can find out the username of the
file's owner with the -l or -o option to
ls:
$ ls -l foo
-rw-rw---- 1 hollingd grads 13 Jan 10 23:05 foo
permissions
owner group
size
time
name

43. 43
File Permissions
Each file has a set of permissions that control
who can mess with the file.
There are three types of permissions:
 read abbreviated r
 write abbreviated w
 execute abbreviated x
There are 3 sets of permissions:
1. user
2. group
3. other (the world, everybody else)

44. 44
ls -l and permissions
-rwxrwxrwx
User Group Others
Type of file:
- – plain file
d – directory
s – symbolic link
(others)

45. 45
rwx
Files:
 r - allowed to read.
 w - allowed to write
 x - allowed to execute
Directories:
 r - allowed to see the names of the
contents
 w - allowed to add and remove files.
 x - allowed to “enter” the directory

46. 46
Changing Permissions
chmod - changes the permissions
associated with a file or directory.
The simplest chmod looks like this:
chmod mode file

47. 47
chmod – numeric modes
Consider permission for each set of users
(user, group, other) as a 3-bit no.
 r – 4
 w – 2
 x – 1
A permission (mode) for all 3 classes is a 3-
digit octal no.
 755 – rwxr-xr-x
 644 – rw-r—r--
 700 – rwx------

48. 48
chmod - examples
$ chmod 700 CS571
$ ls –o Personal
drwx------ 10 kschmidt 4096 Dec 19 2004 CS571/
$ chmod 755 public_html
$ chmod 644 public_html/index.html
$ ls –ao public_html
drwxr-xr-x 16 kschmidt 4096 Jan 8 10:15 .
drwx--x--x 92 kschmidt 8192 Jan 8 13:36 ..
-rw-r--r-- 5 kschmidt 151 Nov 16 19:18 index.html
$ chmod 644 .plan
$ ls –o .plan
-rw-r--r-- 5 kschmidt 151 Nov 16 19:18 .plan

49. 49
chmod – symbolic modes
Can be used to set, add, or remove
permissions
Mode has the following form:
[ugoa][+-=][rwx]
 u – user g – group o – other a – all
 + add permission - remove permission =
set permission

50. 50
chmod examples
$ ls -al foo
-rwxrwx--x 1 hollingd grads foo
$ chmod g-wx foo
$ ls -al foo
-rwxr----x 1 hollingd grads foo
$ chmod u-r .
$ ls
ls: .: Permission denied

51. 51
grep
l Searches its input for a pattern.
l The pattern can be a simple substring or a complex
regular expression.
l If a line matches, it’s directed to STDOUT; otherwise,
it’s discarded.
l % echo “blah-foo” | grep blah
» Will print the matching line
l % echo “blah-foo” | grep zee
» Will not.

52. 52
Shell as a user interface
A shell is a command interpreter, an
interface between a human (or
another program) and the OS
 runs a program, perhaps the ls program.
 allows you to edit a command line.
 can establish alternative sources of input
and destinations for output for programs.
Is, itself, just another program

53. 53
Bourne-again Shell (bash)
Extension of the Bourne Shell (sh)
Contains many of the Korn Shell (ksh)
extensions
You may use the shell of your choice
(tcsh, zsh, etc.).

54. 54
Session Startup
Once you log in, your shell will be
started and it will display a prompt.
 (for our examples, we will use $ as the
prompt. It is not part of the input)
When the shell is started it looks in
your home directory for some
customization files.
 You can change the shell prompt, your
PATH, and a bunch of other things by
creating customization files.

55. 55
Incorrect login
You will receive the “Password:”
prompt even if you type an incorrect or
nonexistent login name
– Can you guess why?
Nothing will happen while you type your
password. It's fine

56. 56
Logging off
exit builtin (command)
 Exits the shell
 If it is the login (top-level) shell, then it
disconnects you
A shell is just another program that
is running. Can recursively invoke
shells
Please don’t just disconnect w/out
exiting

57. 57
Standard I/O
When you enter a command the shell
creates a subshell to run the process
or script.
The shell establishes 3 I/O channels:
 Standard Input (0) – keyboard
 Standard Output (1) – screen
 Standard Error (2) – screen
These streams may be redirected
to/from a file, or even another command

58. 58
Programs and Standard I/O
Program
Standard Input
(STDIN)
Standard Output
(STDOUT)
Standard Error
(STDERR)

59. 59
Terminating Standard Input
If standard input is your keyboard, you can type
stuff in that goes to a program.
To signal the end of input press Ctrl-D (^D), the
EOF signal, on a line by itself, this closes the
input stream.
The shell is a program that reads from standard
input.

60. 60
Shell metacharacters
Some characters have special meaning
to the shell. These are just a few:
 I/O redirection
< > |
 wildcards
* ? [ ]
 others
& ; $ ! ( ) space tab newline
These must be escaped or quoted to
inhibit special behavior

61. 61
Wildcards
* – matches 0 or more characters
? – matches exactly 1 character
[<list>] – matches any single character in
<list>
E.g.
ls *.cc – list all C++ source files in directory
ls a* – list all files that start w/’a’
ls a*.jpeg – list all JPEGs that start w/’a’
ls * - (make sure you have a subdirectory, and try it)

62. 62
Wildcards (more examples)
ls file?
- matches file1, file2, but not
file nor file22
ls file?.*.DEL
- matches file1.h.DEL,
file9.cc.DEL, file3..DEL but not
file8.DEL nor file.html.DEL
These are not regular expressions!

63. 63
Wildcards - classes
[abc] matches any of the enclosed characters
ls T[eE][sS][tT].doc
[a-z] matches any character in a range
ls [a-zA-Z]*
[!abc…] matches any character except those
listed.
ls [!0-9]*

64. 64
Shell Variables
bash uses shell variables to store
information
Shell variables are used to affect the
behavior of the shell, and many other
programs
We can access these variables:
 set new values for some to customize the
shell.
 find out the value of some to help
accomplish a task.

65. 65
Setting/Viewing Variables
To assign (in sh, ksh, bash):
VAR=someString
OTHER_VAR=“I have whitespace”
 Note, no whitespace around the ‘=‘!
To view (dereference) a variable:
$ echo $VAR
someString
$ echo $OTHER_VAR
I have whitespace

66. 66
Shell maintains some variables
Some common ones:
PATH – list of directories shell searches for
non-shell commands
PS1 – Primary prompt
USER – user's login name
HOME – user’s home directory
PWD – current working directory

67. 67
Other Useful Ones
SHELL – the login shell
TERM – the type of terminal interface
HISTFILE – where your command
history is saved
EDITOR – holds user's preferred editor
HOSTNAME – machine's hostname
SHELLOPTS – status of various shell
options (see Bash's set built-in)

68. 68
Displaying Shell Variables
Prefix the name of a shell variable with
"$".
The echo command will do:
$ echo $HOME
$ echo $PATH
You can use these variables on any
command line:
$ ls -al $HOME

69. 69
Setting Shell Variables
You can change the value of a shell
variable with an assignment command
(this is a shell builtin command):
HOME=/etc
PATH=/usr/bin:/usr/etc:/sbin
NEWVAR="blah blah blah"