MH
Uploaded byMathias Homann
4,420 views

UNIX and Linux - an introduction by Mathias Homann

The document outlines a 2-day intensive training course on Unix and Linux by Mathias Homann, aiming to provide insights into the significance of Unix, teach basic commands, and encourage self-study. It covers a history of Unix, differences between Unix and Linux, accessing Unix systems via SSH, and various commands and concepts such as filesystems, processes, permissions, and scripting. The training also includes practical exercises to solidify understanding and application of the material.

Embed presentation

UNIX and Linux an Introduction 2-day intensive training developed by Mathias Homann
Goals • provide insights on why UNIX matters • learn about what makes UNIX special • practice executing basic concepts • spark interest for deeper self-study UNIX and Linux an Introduction, developed 2 of 71 by Mathias Homann
Chapter Index • A short history of UNIX • UNIX today • Linux is not UNIX • How to access a UNIX system • Secure shell – SSH • SSH key management • Building tunnels with SSH UNIX and Linux an Introduction, developed 3 of 71 by Mathias Homann
Chapter Index (2) • Learning details about UNIX commands • Shells and the environment • Internal and external commands • Interactive shells and startup scripts • Wildcards • Command aliases UNIX and Linux an Introduction, developed 4 of 71 by Mathias Homann
Chapter Index (3) • Paths • Finding files – locate vs find • Filesystems • Mounting • Links • Devices and drivers • Permissions and ownership UNIX and Linux an Introduction, developed 5 of 71 by Mathias Homann
Chapter Index (4) • Background processes • Scheduled processes • Processes and threads • Output and input redirection • Pipes • Shell history UNIX and Linux an Introduction, developed 6 of 71 by Mathias Homann
Chapter Index (5) • The power of grep • Less is more • The vi editor • Regular expressions • Sed – the stream editor • Awk – pattern scanner for text files • Writing your own scripts • Syslog and log files UNIX and Linux an Introduction, developed 7 of 71 by Mathias Homann
Chapter Index (6) • Finding facts about a system • Gathering statistics • Writing advanced scripts • More exercises • Useful commands • Higher wizardry – perl • Discussion and closing UNIX and Linux an Introduction, developed 8 of 71 by Mathias Homann
A short history of UNIX Graphic credit: http://en.wikipedia.org/wiki/File:Unix.svg UNIX and Linux an Introduction, developed 9 of 71 by Mathias Homann
A short history of UNIX (2) • 1969: Thompson, Ritchie and others start to “play with that unused PDP-7 in the corner” at Bell Labs, develop a system called “Unicks” based on Multics • 1971: UNIX was developed; first released version on March 11, 1971 • 1975: Bourne shell introduced • 1982: SGI introduces IRIX, their commercial flavour of UNIX for their workstation line • 1983: SCO delivers their first packaged UNIX for Intel architecture PCs; Richard Stallman introduces the GNU operating system on September 27 UNIX and Linux an Introduction, developed 10 of 71 by Mathias Homann
A short history of UNIX (3) • 1986: HP releases HP-UX 1.0 • 1987: SUN and AT&T form an alliance to work on what will be UNIX System V Release 4.0 • 1990: IBM releases AIX • 1991: SUN releases Solaris OS, specially designed for symmetric multiprocessing • 1991: Linux Thorvalds sends his famous email to the minix developers mailing list, it starts with the words “I'm doing a (free) operating system ...” UNIX and Linux an Introduction, developed 11 of 71 by Mathias Homann
UNIX today • today, UNIX is the operating system of choice for both large server farms and small embedded devices • Amazon, Google, your android phone and your media center that is hooked up to your TV all run on Linux, a free UNIX-like operating system UNIX and Linux an Introduction, developed 12 of 71 by Mathias Homann
Linux is not UNIX • the open source community takes great joy in using “recursive acronyms”: – “GNU is not UNIX” – “Linux is not UNIX” • Linux is “UNIX-Like” • a lot in common, “UNIX-Compatible”, but also: – Not a serious, commercial development history / process – In a constant flow • UNIX is: – “carved in stone” before first deployment – commercially developed – Not free to use, unlike Linux, UNIX costs money UNIX and Linux an Introduction, developed 13 of 71 by Mathias Homann
How to access a UNIX system • a “traditional” UNIX host usually has one local console, and several “dumb” terminals attached • today, such “dumb terminals” are usually PCs running terminal emulation software, instead of the old fashioned “true” serial consoles • a Linux system has one local physical console (keyboard and screen), with several virtual consoles available (usually six, to switch between them use the keyboard shortcuts ctrl-alt-F1 to ctrl-alt-F6) • virtual console #7 is usually occupied by the graphical desktop environment if such is used UNIX and Linux an Introduction, developed 14 of 71 by Mathias Homann
Secure shell – SSH • a better way to access a UNIX system is through SSH – The most common SSH client for Windows is called putty, available from http://www.chiark.greenend.org.uk/~sgtatham/putty/ • SSH uses strong encryption – communication between the user and the system can not be eavesdropped on – the administrator password can not be intercepted – on the contrary, Telnet sends plaintext, unencrypted data over the network do not use telnet … ever • to access a UNIX system you need – a user account, and either have the password for that account – or have your SSH public key installed in the allowed keys list for that account UNIX and Linux an Introduction, developed 15 of 71 by Mathias Homann
Secure shell – SSH (2) • to login to a UNIX system through SSH, you use a command like this: ssh joe@example.host.com • useful commandline options for SSH: -1 and -2 to enforce SSH protocol version 1 or 2 -4 and -6 to enforce IPv4 or IPv6 -A enabled SSH agent forwarding (-a to disable) -X enable X11 forwarding (-x to disable) -C compresses data (useful on slower connections) UNIX and Linux an Introduction, developed 16 of 71 by Mathias Homann
SSH key management • SSH uses an asymmetric key pair • to generate a key pair, use ssh-keygen on UNIX, or puttygen on Windows • the public key is what you install on servers that you need to access • the private key needs a passphrase to be used, and is the part that you need to keep safe UNIX and Linux an Introduction, developed 17 of 71 by Mathias Homann
SSH key management (2) • the pass phrase for unlocking your private key can be significantly longer and more complex than a system password • to register your key for access to a system under one user account, append the content of your public key to ~/.ssh/authorized_keys • verify host keys by comparing the fingerprint of the host key with the fingerprint of the host key in your local known_hosts file • host keys do not change randomly UNIX and Linux an Introduction, developed 18 of 71 by Mathias Homann
Building tunnels with SSH • a tunnel is an encrypted “hole” that data can flow through • tunnels are used to create secure connections through firewalls • tunnels can be used to access services securely • a tunnel can: – forward a local port to the remote machine – a port on the remote machine to the local machine – a port on a machine in the same network as the remote machine to a port on the local machine UNIX and Linux an Introduction, developed 19 of 71 by Mathias Homann
Building tunnels with SSH (2) • parameters for tunnels: -L x:host:y – forward localhost:x to host:y -R x:host:y – forward host:x to localhost:y • example: – a tunnel could be used to map the web server on a remote system to the local port 8080 • this way, entering http://localhost:8080/ in your browser would access the remote server UNIX and Linux an Introduction, developed 20 of 71 by Mathias Homann
Building tunnels with SSH (3) • exercise – Generate your key pair with putty – Connect to (hostname) as (username) ← will have to be adapted for actual training environment – Forward the webserver running on the loopback interface on (hostname) to port 8080 on your local computer – Verify with your web browser that the tunnel works UNIX and Linux an Introduction, developed 21 of 71 by Mathias Homann
Learning about UNIX commands • UNIX has two built-in documentation systems: – info pages – man pages • to read the man page (manual page) of a given command, use man command – Try this out with man man • to look if there is a command that does something, try the apropos command • the apropos command lists the summary lines of all man pages that contain the term you specified UNIX and Linux an Introduction, developed 22 of 71 by Mathias Homann
Learning about UNIX commands (2) • to look at what a certain command does without reading the whole man page, use whatis program • this prints out the summary only of the command's man page • to find where in the filesystem a command is, use the which command UNIX and Linux an Introduction, developed 23 of 71 by Mathias Homann
Shells and the environment • the shell is the interface to the system • the shell executes commands and shows the results • the “environment” is the total of stored settings inside a shell • important shell variables on UNIX systems: $PATH $PS1 $CWD UNIX and Linux an Introduction, developed 24 of 71 by Mathias Homann
Shells and the environment (2) • UNIX knows several different shells: sh bash ksh zsh csh • the default shell on most Linux systems is bash • the set command shows all environment variables • change environment variables by assigning and exporting them • all exercises will use bash if not stated otherwise UNIX and Linux an Introduction, developed 25 of 71 by Mathias Homann
Internal and external commands • shells have built-in versions of most “system commands” • to “force” the use of a system command instead of the builtin, use the absolute path to the binary: /bin/ls • external commands are standardized • builtins differ from shell to shell UNIX and Linux an Introduction, developed 26 of 71 by Mathias Homann
Interactive shells + startup scripts • an “interactive” shell is one that you type commands into • “non-interactive” shells are the ones that execute scripts • a “login” shell is the first interactive shell that gets started when you log in • the “profile” or “rc file” is a script that gets executed by a shell on startup • most shells execute more than one startup script, and execute a different set of scripts when started non- interactive, and yet another for the login shell UNIX and Linux an Introduction, developed 27 of 71 by Mathias Homann
Interactive shells + startup scripts (2) • examples - the bash shell executes on startup as a login shell (in this order): /etc/profile ~/.bash_profile ~/.bash_login ~/.profile • if the bash shell is started interactively but not as a login shell it only executes ~/.bashrc • a non-interactive bash shell executes the file listed in $BASH_ENV if the variable is set and the file exists • note: a login bash executes ~/.bash_logout on exit UNIX and Linux an Introduction, developed 28 of 71 by Mathias Homann
Wildcards • a wildcard is a pattern that describes a path or filename • wildcard expansion in shell commands happens on the shell level, before the actual command gets executed • different shells handle wildcards in different ways • exercises: – Which wildcard would mean “all jpeg files in this folder”? – Which pattern means “all jpeg files in this folder that start with the letter a”? UNIX and Linux an Introduction, developed 29 of 71 by Mathias Homann
Command aliases • a command alias is a “placeholder” for a command and optionally one or more commonly used options • for example, on openSUSE Linux there is an alias “dir” which calls ls -l • a list of all aliases is shown by entering the alias command with no parameters • to see the aliases and wildcards expansion before commands get executed, enter set -x UNIX and Linux an Introduction, developed 30 of 71 by Mathias Homann
Paths • paths on UNIX systems are all inside a “tree” of folders that starts at the “root” folder or / – the user's home folder is usually held under /home and named after the username, e.g. /home/fred – the “search path” describes which folders to look in when a shell tries to find a command – the search path is stored in the environment variable $PATH – the minimal search path should contain /bin and /usr/bin – directories in the search path are separated by the colon: PATH=/bin:/usr/bin UNIX and Linux an Introduction, developed 31 of 71 by Mathias Homann
Everything is a file ... • ... not just a file • /proc/12345/status is a “virtual file” containing status information about process 12345 • /dev/mem is a “device file” that is actually an interface to the system's memory • /dev/sg0 is a “device file” for the first “generic” SCSI device in a system • /dev/sdb5 is a “device file” that references the first logical partition on the second external, SCSI or SATA disk UNIX and Linux an Introduction, developed 32 of 71 by Mathias Homann
Finding files – locate vs find • there are two ways to find a file, locate and find • the locate command uses a database of existing files which needs to be updated regularly, which usually happens once a day • the find command actually searches through the filesystem – this can take a while, but does not rely on a database that can be outdated locate can only search by shell wildcards find can search for anything, not just names • exercise: – find all files that changed since yesterday UNIX and Linux an Introduction, developed 33 of 71 by Mathias Homann
Filesystems • UNIX knows several different filesystems • one of the main differences is whether a filesystem is “journaling” or not • A journaling filesystem keeps track of the changes that will be made in a journal, before committing them to the main filesystem. In the event of a system outage, such filesystems are quicker to bring back online and less likely to be corrupted. • commonly used filesystems on Linux are: – ext2, ext3, ext4, jfs, xfs, reiserfs – Ext2 is the only of these that is not journaling UNIX and Linux an Introduction, developed 34 of 71 by Mathias Homann
Filesystems (2) • filesystem limits: names paths filesize volume size ext2 255 unlimited 16GB - 2TB 2TB-32TB ext3 255 unlimited 16GB - 2TB 2TB-32TB ext4 256 unlimited 16GB - 16TB 1EB jfs 255 unlimited 8TB 512TB-4PB xfs 255 unlimited 8EB 8EB reiserfs 4032/226 unlimited 8EB 16TB UNIX and Linux an Introduction, developed 35 of 71 by Mathias Homann
Mounting • all disk partitions need to be mounted to be accessible • a disk partition gets mounted into a folder, called the “mount point” • a UNIX system has a file with a list of all mount points that need to be mounted when the system starts: /etc/fstab – to mount a filesystem that is not listed in /etc/fstab you need superuser privileges – to access a partition you need to know its device node – device nodes are special files in /dev UNIX and Linux an Introduction, developed 36 of 71 by Mathias Homann
Mounting (2) • for example, the device node for a partition on a hard disk would be /dev/hda2 - meaning “the second primary partition on the first hard disk” • exercise: – explore what is mounted, and what is mounted on boot UNIX and Linux an Introduction, developed 37 of 71 by Mathias Homann
Links • a link is a “pointer” to a file – can be a “soft” or a “hard” link – a softlink is a pointer at a file, and can link across filesystems – a hardlink is another directory entry that points at the same starting sector of the filesystem • hardlinks are only possible within one filesystem, and cannot point at directories • links are created with the ln command ln creates hardlinks by default – for softlinks use the -s option • exercise: – create a link to /usr/bin/bash in ~/bin/ UNIX and Linux an Introduction, developed 38 of 71 by Mathias Homann
Devices and drivers • every piece of hardware is accessed through a device file under /dev • examples: /dev/hda - the first IDE harddisk /dev/sdb - the second SCSI or SATA disk /dev/nvidia – an NVIDIA graphics card /dev/dsp0 – the sound card UNIX and Linux an Introduction, developed 39 of 71 by Mathias Homann
Devices and drivers (2) • access to hardware is governed by the file permissions on those special files • for example, the permissions on a graphics card could be set to allow only root and anyone in the group “video” access to write to it, therefore only users in the group “video” would be able to run 3d accelerated applications • exercise: – find out which group has write access to the cd/rw drive, e.g. would have sufficient permissions to use a CDRW writer on the system UNIX and Linux an Introduction, developed 40 of 71 by Mathias Homann
Permissions and ownership • there are 3 levels of ownership of any file: – “owner” – “group” – “others” • permissions on a file: – read, write, execute, separately for all three owner types • example: – a file usually is read/write for the owner, and read-only for anyone else: -rw-r--r-- • change the owner: chown username file • change the group: chgrp groupname file • change permissions: chmod <permissions> file UNIX and Linux an Introduction, developed 41 of 71 by Mathias Homann
Background processes • UNIX can run commands in the background so that you can do something else – To execute a program in the background, end the commandline with & – To send a running program to the background, press ctrl-z and enter bg – To view the list of running background processes, enter jobs • when you logout while you still have background processes running, they will be killed unless you started them with nohup: nohup sleep 10000 will run sleep 10000 in the background – the output of the command will be written to nohup.out UNIX and Linux an Introduction, developed 42 of 71 by Mathias Homann
Scheduled processes • to run a program at regular intervals, use cron • cron keeps lists of scheduled jobs • edit the list with crontab -e • to run a program at a certain time once, use at at 23:00 asks for a sequence of commands • the job gets deleted once it runs UNIX and Linux an Introduction, developed 43 of 71 by Mathias Homann
Processes and threads • a process is one running executable – A single process can have more than one running thread – To see all processes, use top, htop, or the ps command – To send signals to processes, use the kill command • several “levels” of “kill signal”, ordered by “lethality”: kill -HUP tells a process to shut down nicely kill -QUIT – the same but a bit more urgent kill -KILL kills a process kill -l lists all possible signals with name and numeric value kill -9 is equivalent to kill -KILL • combined exercise: – start a process in the background that waits for 10 minutes – kill it before the time is over UNIX and Linux an Introduction, developed 44 of 71 by Mathias Homann
Output and input redirection • in- and Output can be redirected with the < and > symbol • redirect input: cat < /tmp/file.txt • redirect output: cat < /tmp/file.txt > /tmp/foo.bar • redirect the error output: sort < foo 2>/dev/null UNIX and Linux an Introduction, developed 45 of 71 by Mathias Homann
Pipes • pipes are used to direct the output of one command to the input of another • pipes use the pipe symbol: | • example: ls -la | less pipes the output of the ls command into the less pager • exercise: sort the content of the passwd file by alphabet UNIX and Linux an Introduction, developed 46 of 71 by Mathias Homann
Shell history • UNIX shells keep track of your commands • to browse through this list use the cursor up and cursor down keys • to re-execute a previous command, find it in your history and press enter • to find a previous command more quickly, press ctrl-r and start typing; the history will “jump” to the newest match of what you're typing UNIX and Linux an Introduction, developed 47 of 71 by Mathias Homann
Shell history (2) • to view the whole history, use the “history” command • to re-execute a certain command, find its number in the history, then enter !number, e.g. !514 to re-run number 514 from your history • to search in your history: press ctrl-r, then start typing • exercise: – use a few commands (ls, man, ls -l, ls -la) then view the history, then re-run some of them, then search in your history UNIX and Linux an Introduction, developed 48 of 71 by Mathias Homann
The power of grep • grep searches for text in files • grep can use regular expressions • grep has many options - read the man page • most useful options: – -i for “search case insensitive”, and -v for “inverse” or “search for anything that does not match my search string” • example: grep myusername /var/log/messages – find all lines in logfile that contain myusername • exercise: – filter the main system logfile for all lines from last Monday UNIX and Linux an Introduction, developed 49 of 71 by Mathias Homann
Less is more • less is a “pager” to view large text files on screen • less can search for text: – press : then enter a slash followed by your text • less can switch to “continuous refresh” to refresh a file that is still being written; press F to switch • less can read from “stdin”; pipe commands into it to paginate the output • g and g jump to the end and the start of the file • to search in less, type /searchterm • to jump to a line number, type :number • example: less /var/log/messages • exercise: – display the content of /usr/bin in less UNIX and Linux an Introduction, developed 50 of 71 by Mathias Homann
The vi editor • the editor vi exists on every UNIX system • vi has two modes: insertion mode and command mode – The editor begins in command mode, where the cursor movement and text deletion and pasting occur – Insertion mode begins upon entering an insertion or change command – [ESC] returns the editor to command mode (where you can quit, for example by typing :q!) • most commands execute as soon as you type them except for "colon" commands which execute when you press the return key • nifty: vi does syntax highlighting; activate with :syntax UNIX and Linux an Introduction, developed 51 of 71 by Mathias Homann
The vi editor (2) • quitting vi: :x Exit, saving changes :q Exit as long as there have been no changes ZZ Exit and save changes if any have been made :q!Exit and ignore any changes • switching to edit mode: i – insert r – replace a – append UNIX and Linux an Introduction, developed 52 of 71 by Mathias Homann
The vi editor (3) • cursor movement • cursor keys, and: ^ or 0 – move to start of line $ - move to end of line • block and line operations: dd – delete line yy – copy line p – paste line UNIX and Linux an Introduction, developed 53 of 71 by Mathias Homann
Regular expressions • regular expressions are powerful string matching patterns • examples: . - any one character * - any repetitions of the pattern left of the star .* - any number of any characters [:upper:] - all UPPER CASE characters n – newline [a-q] – any letter between a and q UNIX and Linux an Introduction, developed 54 of 71 by Mathias Homann
Regular expressions (2) • more info: • the “Regular expressions cheat sheet v2” – http://www.addedbytes.com/cheat-sheets/regular-expressions- cheat-sheet/ • example: – a regular expression cat.*dog matches any string that has “cat” and “dog” in it with any other characters in between them, but only in that order! UNIX and Linux an Introduction, developed 55 of 71 by Mathias Homann
Sed – the stream editor • Sed reads text from stdin, does edit operations on it, and prints the result on stdout • examples: sed -e 's/red/green/g' <colors.txt >colors1.txt – replaces all occurrences of “red” with “green” • exercise: – read the man page, then write a text file, then replace all “the” with “THE” • exercise: – print out each directory in the path variable in a new line UNIX and Linux an Introduction, developed 56 of 71 by Mathias Homann
Awk – pattern scanner for text files • Awk scans text files for patterns and operates on them • examples: awk '{print $2“n“}' < file.txt – print the second word of every line of file.txt • exercise: – read the man page, then use vi to write a text file with a few lines of the format: Username:Firstname:Lastname:Telephone number – then, use awk to convert this into ldap syntax: dn: uid=Username, dc=example, dc=com cn: Firstname Lastname sn: Lastname telephoneNumber: Telephone number UNIX and Linux an Introduction, developed 57 of 71 by Mathias Homann
Writing your own scripts • scripts can be written with any text editor • scripts don't need a special filename suffix • scripts need to be executable (hint: chmod) • scripts can explicitly state which shell should be used to execute it by putting the “hashbang” for the shell in the first line: #!/bin/bash – a bash script #!/bin/csh – a c shell script #!/bin/sh – a “sh” shell script • “sh” is the most basic shell; exists on any UNIX • exercise: – write a script that prints “hello world” ten times UNIX and Linux an Introduction, developed 58 of 71 by Mathias Homann
Syslog and log files • syslog is a UNIX daemon that collects and handles log file messages • syslog handles messages according to severity and facility • details: see man page • syslog can write messages to one or more files depending on severity and facility UNIX and Linux an Introduction, developed 59 of 71 by Mathias Homann
Syslog and log files (2) • logfiles usually exist in /var/log • the syslog configuration file is in /etc and states which facility.severity goes in which file • exercise: – check the syslog configuration to see which files hold messages of failed logins – then check if there were any recent failed login attempts UNIX and Linux an Introduction, developed 60 of 71 by Mathias Homann
Finding facts about a system • most Linux systems have a file in /etc/ that contains information about which distribution is running – for example, /etc/redhat-Release uname -a prints out information about the system • use mount and df to find out about disk sizes and mount points • a lot of information is in /proc • almost all configuration files are in /etc and its subdirectories UNIX and Linux an Introduction, developed 61 of 71 by Mathias Homann
Finding facts about a system (2) • most UNIX systems use some form of software package management – the list of installed packages can tell you a lot about the purpose of a system – on Linux there are two widely used package management systems, rpm and apt/dpkg – rpm is used on openSUSE, redhat, and their derivatives and commercial versions – on debian, ubuntu and derivatives, apt and dpkg is used – commands such as rpm -qa and dpkg –get-selections show the list of installed packages – these lists are long, less is your friend UNIX and Linux an Introduction, developed 62 of 71 by Mathias Homann
Finding facts about a system (3) • to list all loaded kernel modules lsmod • use insmod or modprobe to load additional kernel modules into a running system • use modprobe -r to remove a module and all dependencies from a running system • most of them are hardware drivers or drivers for “features” UNIX and Linux an Introduction, developed 63 of 71 by Mathias Homann
Gathering statistics • commands to explore the status of the system vmstat and free show memory usage iostat shows drive accesses netstat shows network statistics netstat can also show active network connections and listening processes; read the man page • exercise: – check how much memory the machine has, and how much is used; then check how many active network connections exist that are not SSH connections UNIX and Linux an Introduction, developed 64 of 71 by Mathias Homann
Writing advanced scripts • scripts can do math: a=$(($b+$c)) • scripts can have flow control – if (condition) then {command; command;}; else { command; command;}; endif – do { command; command; } while (condition); – for (iterator); do { command; command; } done; • scripts can have functions • scripts can return values to the calling process UNIX and Linux an Introduction, developed 65 of 71 by Mathias Homann
Writing advanced scripts (2) • exercise: – write a script that: – finds all log files under /var log – takes all files older than 2 days – prints their name, size and age – (size in 'human readable' format, 1024k = 1 M etc) UNIX and Linux an Introduction, developed 66 of 71 by Mathias Homann
More exercises • create a file “a”, write “hello world” into it, then hardlink it as “b” • check the inodes of both files – (hint: there's an option for ls that does that) • create a softlink “c” to “a”, check inodes • run tail -f /var/log/messages, send it to the background, edit a different file in vi, exit vi, bring the tail command back to the front • compare the differences between the various ps output formats: ps –ax ps -ef ps -axf – sort /etc/passwd alphabetically – sort it by uid UNIX and Linux an Introduction, developed 67 of 71 by Mathias Homann
Useful commands • cd – change directory • pushd – change directory but it stores the previous directory on a fifo stack • popd – returns to previous directory from the stack • cat – read from file or stdin, write to file or stdout (example: cat /etc/motd to print that file to screen) • cp – copy files • mv – move / rename files • head & tail – print the first or last lines of a file • find – search for files UNIX and Linux an Introduction, developed 68 of 71 by Mathias Homann
Useful commands (2) • less – display a file in a pager • grep – search for text in files • sed – command-line editing of files • wc – count lines/words/characters in files • df – disk free (shows free space in partitions) • du – disk usage (adds up file sizes, more accurate than df, takes some time to finish) • gzip – compress files, use gunzip to decompress • diff – compare files UNIX and Linux an Introduction, developed 69 of 71 by Mathias Homann
Higher wizardry – perl • perl is disputedly the most powerful, simple, complicated, and obscurely easy to understand scripting language • perl is enough for its own workshop day :) UNIX and Linux an Introduction, developed 70 of 71 by Mathias Homann
Discussion and closing • questions • comments • Instructor Evaluation Form UNIX and Linux an Introduction, developed 71 of 71 by Mathias Homann

More Related Content

PPTX
Introduction to Linux basic
byf114n
 
PPTX
Introduction to linux ppt
byOmi Vichare
 
PPT
Linux - Introductions to Linux Operating System
byVibrant Technologies & Computers
 
ODP
Introduction to linux
byplarsen67
 
PPT
Linux command ppt
bykalyanineve
 
PPTX
A general Overview of linux !!
byjainema23
 
PPTX
Introduction 2 linux
byPapu Kumar
 
ODP
Introduction to Linux OS
byMohammed Safwat
 
Introduction to Linux basic
byf114n
 
Introduction to linux ppt
byOmi Vichare
 
Linux - Introductions to Linux Operating System
byVibrant Technologies & Computers
 
Introduction to linux
byplarsen67
 
Linux command ppt
bykalyanineve
 
A general Overview of linux !!
byjainema23
 
Introduction 2 linux
byPapu Kumar
 
Introduction to Linux OS
byMohammed Safwat
 

What's hot

PDF
Linux introduction
byMd. Zahid Hossain Shoeb
 
PPT
History of Linux
byGray-chan Desu
 
PPTX
Linux introduction, class 1
byAdrian Mikeliunas
 
PDF
Linux programming lecture_notes
byIMRAN KHAN
 
PPT
Linux basics
bySantosh Khadsare
 
PPTX
Linux
byRittikaBaksi
 
PPTX
Introduction to linux
byStephen Ahiante
 
PPTX
what is LINUX ? presentation.
bysaad_khan1122
 
ODP
An Introduction to Linux
byanandvaidya
 
PPTX
Linux Simple Introduction
byJohnMihaya
 
PPT
comparing windows and linux ppt
byShivalik college of engineering
 
PPTX
ZendCon - Linux 101
byJustin Reock
 
PPTX
UNIX Operating System
byUnless Yuriko
 
PPT
Unit 1-a-brief-history-of-unix-ppt
byRahul Mashal
 
PPTX
A History of Linux
byDamian T. Gordon
 
PDF
Linux introduction Class 02
byAdrian Mikeliunas
 
PPTX
Operating systems linux
bywilliam_morg
 
PPTX
Operating Systems: Linux in Detail
byDamian T. Gordon
 
PDF
Linux Introduction
byLibsoul Technologies Pvt. Ltd.
 
PDF
Linux OS presentation
bySahilGothoskar
 
Linux introduction
byMd. Zahid Hossain Shoeb
 
History of Linux
byGray-chan Desu
 
Linux introduction, class 1
byAdrian Mikeliunas
 
Linux programming lecture_notes
byIMRAN KHAN
 
Linux basics
bySantosh Khadsare
 
Linux
byRittikaBaksi
 
Introduction to linux
byStephen Ahiante
 
what is LINUX ? presentation.
bysaad_khan1122
 
An Introduction to Linux
byanandvaidya
 
Linux Simple Introduction
byJohnMihaya
 
comparing windows and linux ppt
byShivalik college of engineering
 
ZendCon - Linux 101
byJustin Reock
 
UNIX Operating System
byUnless Yuriko
 
Unit 1-a-brief-history-of-unix-ppt
byRahul Mashal
 
A History of Linux
byDamian T. Gordon
 
Linux introduction Class 02
byAdrian Mikeliunas
 
Operating systems linux
bywilliam_morg
 
Operating Systems: Linux in Detail
byDamian T. Gordon
 
Linux Introduction
byLibsoul Technologies Pvt. Ltd.
 
Linux OS presentation
bySahilGothoskar
 

Viewers also liked

PPTX
UNIX/Linux training
byMichael Olafusi
 
PPTX
Unix Operating System
bysubhsikha
 
ODP
intro unix/linux 02
byduquoi
 
PDF
Unix - An Introduction
byDeepanshu Gahlaut
 
PPT
Unix/Linux Basic Commands and Shell Script
bysbmguys
 
PPT
Shell Scripting in Linux
byAnu Chaudhry
 
PPTX
Unix
byErm78
 
PPS
QSpiders - Unix Operating Systems and Commands
byQspiders - Software Testing Training Institute
 
PPT
Unix(introduction)
bymeashi
 
PDF
Unix Command Line Productivity Tips
byKeith Bennett
 
PPTX
Unix OS & Commands
byMohit Belwal
 
PPT
Unix memory management
byTech_MX
 
PPT
Basic Unix
byRajesh Kumar
 
PPS
Unix
byyoga.kathirvelu
 
PPT
Unix command-line tools
byEric Wilson
 
PPTX
Unix operating system
byABhay Panchal
 
PDF
Unix Training - 1
byankitmehta21
 
PPT
Linux+03
byDuong Hieu
 
PDF
Red hat enterprise_linux-5.5-release_notes-en-us
byDuong Hieu
 
PDF
Tu hoc su dung linux 2
byIntegrated Circuit Design Research & Education Center (ICDREC)
 
UNIX/Linux training
byMichael Olafusi
 
Unix Operating System
bysubhsikha
 
intro unix/linux 02
byduquoi
 
Unix - An Introduction
byDeepanshu Gahlaut
 
Unix/Linux Basic Commands and Shell Script
bysbmguys
 
Shell Scripting in Linux
byAnu Chaudhry
 
Unix
byErm78
 
QSpiders - Unix Operating Systems and Commands
byQspiders - Software Testing Training Institute
 
Unix(introduction)
bymeashi
 
Unix Command Line Productivity Tips
byKeith Bennett
 
Unix OS & Commands
byMohit Belwal
 
Unix memory management
byTech_MX
 
Basic Unix
byRajesh Kumar
 
Unix
byyoga.kathirvelu
 
Unix command-line tools
byEric Wilson
 
Unix operating system
byABhay Panchal
 
Unix Training - 1
byankitmehta21
 
Linux+03
byDuong Hieu
 
Red hat enterprise_linux-5.5-release_notes-en-us
byDuong Hieu
 
Tu hoc su dung linux 2
byIntegrated Circuit Design Research & Education Center (ICDREC)
 

Similar to UNIX and Linux - an introduction by Mathias Homann

PDF
Introduction to Networking | Linux-Unix and System Administration | Docker an...
byandega
 
PPT
ITE117 Konikoff Week1 wrap
byRobert Konikoff
 
PPTX
Linux
byNikhil Dhiman
 
PDF
Introduction to linux
byQIANG XU
 
PPT
Intro tounix
byHamza Abdolateef
 
PDF
2018-Summer-Tutorial-Intro-to-Linux.pdf
bysanjeevkuraganti
 
PPTX
Introduction to Linux
byPRIYATHAMDARISI
 
PPT
IntroToUnix.ppt
byPecific University
 
PPT
IntroToUnix.ppt.intro.to.linuxlinuxintroduction
byNagavelliMadhavi
 
PPTX
KMSUnix and Linux.pptx
byGanesh Bhosale
 
PPTX
A beginners introduction to unix
byzafarali1981
 
PPT
Intro tounix
bydjprince
 
PDF
Linux Installation Configuration and Command Line Basics 1st Edition Nathan C...
byarilakeguono
 
PPT
Operating Systems 1
byhutchison
 
PPT
Chapter09 -- networking with unix and linux
byRaja Waseem Akhtar
 
PDF
Unix Shell Scripting
byMustafa Qasim
 
PDF
Kernel linux lab manual feb (1)
byjohny shaik
 
PPT
Linux forensics
bySantosh Khadsare
 
PPT
Linux operating system by Quontra Solutions
byQUONTRASOLUTIONS
 
DOCX
UNIT I LINUX.docx
byBhuvanaR13
 
Introduction to Networking | Linux-Unix and System Administration | Docker an...
byandega
 
ITE117 Konikoff Week1 wrap
byRobert Konikoff
 
Linux
byNikhil Dhiman
 
Introduction to linux
byQIANG XU
 
Intro tounix
byHamza Abdolateef
 
2018-Summer-Tutorial-Intro-to-Linux.pdf
bysanjeevkuraganti
 
Introduction to Linux
byPRIYATHAMDARISI
 
IntroToUnix.ppt
byPecific University
 
IntroToUnix.ppt.intro.to.linuxlinuxintroduction
byNagavelliMadhavi
 
KMSUnix and Linux.pptx
byGanesh Bhosale
 
A beginners introduction to unix
byzafarali1981
 
Intro tounix
bydjprince
 
Linux Installation Configuration and Command Line Basics 1st Edition Nathan C...
byarilakeguono
 
Operating Systems 1
byhutchison
 
Chapter09 -- networking with unix and linux
byRaja Waseem Akhtar
 
Unix Shell Scripting
byMustafa Qasim
 
Kernel linux lab manual feb (1)
byjohny shaik
 
Linux forensics
bySantosh Khadsare
 
Linux operating system by Quontra Solutions
byQUONTRASOLUTIONS
 
UNIT I LINUX.docx
byBhuvanaR13
 

Recently uploaded

PDF
Top Benefits of Using KVM VPS Hosting for Growing Businesses
byEthernet Servers
 
PDF
Artificial Intelligence and Barbarism - Conceptual Map
byalfadix
 
PDF
IAI-Unit1-notes useful.............................
bydinesh620610
 
PDF
Azure Identity, Governance, and Monitoring solutions
byVICTOR MAESTRE RAMIREZ
 
PDF
Strengthening cybern resilience for a leading indian Financial Services group
bypandeydevika621
 
PDF
Small Business Automation: A Comprehensive Cost and ROI Guide
byAmelia Swank
 
PPTX
Information about Trusted Platform Module(TPM)
bynewtoknowtechs
 
PPTX
Scrape YouTube Video Data for Influencer Analytics.pptx
byWeb Data Crawler
 
PPTX
Digital transformation success powered by EPM and NexInfo.pptx
byjayasuryanexinfo
 
PPTX
Large Language Model. LLM Audit Artificial Intelligence
byMANASCHOUDHARY22
 
PPTX
AI Bot Traffic Surge: Retail Fraud Threat for Age-Restricted Websites
byFTx Identity
 
PDF
Techbrains Baku 2025 by GoUP - all speaker session
byHusseinMalikMammadli
 
PPT
HDTV and DTV Standards: The United States Opts for a Digital HDTV Standard
byJeffrey Hart
 
PDF
A Brief Introduction About Christopher Elwell Woburn
byChristopher Elwell Woburn
 
PPTX
Cesium formate brine - A brief history - prepared by John Downs in May 2011
byJohn Downs
 
PPTX
Search Engine Responses to Conspiratorial Search Practices AANZCA 2025 Presen...
bykkasianenko
 
PDF
Benefits of Using the IAC 500 Sensor for Ambient Conditions
byCS Instruments
 
PPTX
Pizza Chain Market Data Scraping for Better Insights Report.pptx
byWeb Data Crawler
 
PDF
The map to conquer linear algebra for IT engineer
byakipii ogaoga
 
PDF
AI Driven & AI Native Development AI native development
byZainKarim7
 
Top Benefits of Using KVM VPS Hosting for Growing Businesses
byEthernet Servers
 
Artificial Intelligence and Barbarism - Conceptual Map
byalfadix
 
IAI-Unit1-notes useful.............................
bydinesh620610
 
Azure Identity, Governance, and Monitoring solutions
byVICTOR MAESTRE RAMIREZ
 
Strengthening cybern resilience for a leading indian Financial Services group
bypandeydevika621
 
Small Business Automation: A Comprehensive Cost and ROI Guide
byAmelia Swank
 
Information about Trusted Platform Module(TPM)
bynewtoknowtechs
 
Scrape YouTube Video Data for Influencer Analytics.pptx
byWeb Data Crawler
 
Digital transformation success powered by EPM and NexInfo.pptx
byjayasuryanexinfo
 
Large Language Model. LLM Audit Artificial Intelligence
byMANASCHOUDHARY22
 
AI Bot Traffic Surge: Retail Fraud Threat for Age-Restricted Websites
byFTx Identity
 
Techbrains Baku 2025 by GoUP - all speaker session
byHusseinMalikMammadli
 
HDTV and DTV Standards: The United States Opts for a Digital HDTV Standard
byJeffrey Hart
 
A Brief Introduction About Christopher Elwell Woburn
byChristopher Elwell Woburn
 
Cesium formate brine - A brief history - prepared by John Downs in May 2011
byJohn Downs
 
Search Engine Responses to Conspiratorial Search Practices AANZCA 2025 Presen...
bykkasianenko
 
Benefits of Using the IAC 500 Sensor for Ambient Conditions
byCS Instruments
 
Pizza Chain Market Data Scraping for Better Insights Report.pptx
byWeb Data Crawler
 
The map to conquer linear algebra for IT engineer
byakipii ogaoga
 
AI Driven & AI Native Development AI native development
byZainKarim7
 

UNIX and Linux - an introduction by Mathias Homann

  • 1.
    UNIX and Linux an Introduction 2-day intensive training developed by Mathias Homann
  • 2.
    Goals • provide insights on why UNIX matters • learn about what makes UNIX special • practice executing basic concepts • spark interest for deeper self-study UNIX and Linux an Introduction, developed 2 of 71 by Mathias Homann
  • 3.
    Chapter Index • A short history of UNIX • UNIX today • Linux is not UNIX • How to access a UNIX system • Secure shell – SSH • SSH key management • Building tunnels with SSH UNIX and Linux an Introduction, developed 3 of 71 by Mathias Homann
  • 4.
    Chapter Index (2) • Learning details about UNIX commands • Shells and the environment • Internal and external commands • Interactive shells and startup scripts • Wildcards • Command aliases UNIX and Linux an Introduction, developed 4 of 71 by Mathias Homann
  • 5.
    Chapter Index (3) • Paths • Finding files – locate vs find • Filesystems • Mounting • Links • Devices and drivers • Permissions and ownership UNIX and Linux an Introduction, developed 5 of 71 by Mathias Homann
  • 6.
    Chapter Index (4) • Background processes • Scheduled processes • Processes and threads • Output and input redirection • Pipes • Shell history UNIX and Linux an Introduction, developed 6 of 71 by Mathias Homann
  • 7.
    Chapter Index (5) • The power of grep • Less is more • The vi editor • Regular expressions • Sed – the stream editor • Awk – pattern scanner for text files • Writing your own scripts • Syslog and log files UNIX and Linux an Introduction, developed 7 of 71 by Mathias Homann
  • 8.
    Chapter Index (6) • Finding facts about a system • Gathering statistics • Writing advanced scripts • More exercises • Useful commands • Higher wizardry – perl • Discussion and closing UNIX and Linux an Introduction, developed 8 of 71 by Mathias Homann
  • 9.
    A short historyof UNIX Graphic credit: http://en.wikipedia.org/wiki/File:Unix.svg UNIX and Linux an Introduction, developed 9 of 71 by Mathias Homann
  • 10.
    A short historyof UNIX (2) • 1969: Thompson, Ritchie and others start to “play with that unused PDP-7 in the corner” at Bell Labs, develop a system called “Unicks” based on Multics • 1971: UNIX was developed; first released version on March 11, 1971 • 1975: Bourne shell introduced • 1982: SGI introduces IRIX, their commercial flavour of UNIX for their workstation line • 1983: SCO delivers their first packaged UNIX for Intel architecture PCs; Richard Stallman introduces the GNU operating system on September 27 UNIX and Linux an Introduction, developed 10 of 71 by Mathias Homann
  • 11.
    A short historyof UNIX (3) • 1986: HP releases HP-UX 1.0 • 1987: SUN and AT&T form an alliance to work on what will be UNIX System V Release 4.0 • 1990: IBM releases AIX • 1991: SUN releases Solaris OS, specially designed for symmetric multiprocessing • 1991: Linux Thorvalds sends his famous email to the minix developers mailing list, it starts with the words “I'm doing a (free) operating system ...” UNIX and Linux an Introduction, developed 11 of 71 by Mathias Homann
  • 12.
    UNIX today • today,UNIX is the operating system of choice for both large server farms and small embedded devices • Amazon, Google, your android phone and your media center that is hooked up to your TV all run on Linux, a free UNIX-like operating system UNIX and Linux an Introduction, developed 12 of 71 by Mathias Homann
  • 13.
    Linux is notUNIX • the open source community takes great joy in using “recursive acronyms”: – “GNU is not UNIX” – “Linux is not UNIX” • Linux is “UNIX-Like” • a lot in common, “UNIX-Compatible”, but also: – Not a serious, commercial development history / process – In a constant flow • UNIX is: – “carved in stone” before first deployment – commercially developed – Not free to use, unlike Linux, UNIX costs money UNIX and Linux an Introduction, developed 13 of 71 by Mathias Homann
  • 14.
    How to accessa UNIX system • a “traditional” UNIX host usually has one local console, and several “dumb” terminals attached • today, such “dumb terminals” are usually PCs running terminal emulation software, instead of the old fashioned “true” serial consoles • a Linux system has one local physical console (keyboard and screen), with several virtual consoles available (usually six, to switch between them use the keyboard shortcuts ctrl-alt-F1 to ctrl-alt-F6) • virtual console #7 is usually occupied by the graphical desktop environment if such is used UNIX and Linux an Introduction, developed 14 of 71 by Mathias Homann
  • 15.
    Secure shell –SSH • a better way to access a UNIX system is through SSH – The most common SSH client for Windows is called putty, available from http://www.chiark.greenend.org.uk/~sgtatham/putty/ • SSH uses strong encryption – communication between the user and the system can not be eavesdropped on – the administrator password can not be intercepted – on the contrary, Telnet sends plaintext, unencrypted data over the network do not use telnet … ever • to access a UNIX system you need – a user account, and either have the password for that account – or have your SSH public key installed in the allowed keys list for that account UNIX and Linux an Introduction, developed 15 of 71 by Mathias Homann
  • 16.
    Secure shell –SSH (2) • to login to a UNIX system through SSH, you use a command like this: ssh joe@example.host.com • useful commandline options for SSH: -1 and -2 to enforce SSH protocol version 1 or 2 -4 and -6 to enforce IPv4 or IPv6 -A enabled SSH agent forwarding (-a to disable) -X enable X11 forwarding (-x to disable) -C compresses data (useful on slower connections) UNIX and Linux an Introduction, developed 16 of 71 by Mathias Homann
  • 17.
    SSH key management •SSH uses an asymmetric key pair • to generate a key pair, use ssh-keygen on UNIX, or puttygen on Windows • the public key is what you install on servers that you need to access • the private key needs a passphrase to be used, and is the part that you need to keep safe UNIX and Linux an Introduction, developed 17 of 71 by Mathias Homann
  • 18.
    SSH key management(2) • the pass phrase for unlocking your private key can be significantly longer and more complex than a system password • to register your key for access to a system under one user account, append the content of your public key to ~/.ssh/authorized_keys • verify host keys by comparing the fingerprint of the host key with the fingerprint of the host key in your local known_hosts file • host keys do not change randomly UNIX and Linux an Introduction, developed 18 of 71 by Mathias Homann
  • 19.
    Building tunnels withSSH • a tunnel is an encrypted “hole” that data can flow through • tunnels are used to create secure connections through firewalls • tunnels can be used to access services securely • a tunnel can: – forward a local port to the remote machine – a port on the remote machine to the local machine – a port on a machine in the same network as the remote machine to a port on the local machine UNIX and Linux an Introduction, developed 19 of 71 by Mathias Homann
  • 20.
    Building tunnels withSSH (2) • parameters for tunnels: -L x:host:y – forward localhost:x to host:y -R x:host:y – forward host:x to localhost:y • example: – a tunnel could be used to map the web server on a remote system to the local port 8080 • this way, entering http://localhost:8080/ in your browser would access the remote server UNIX and Linux an Introduction, developed 20 of 71 by Mathias Homann
  • 21.
    Building tunnels withSSH (3) • exercise – Generate your key pair with putty – Connect to (hostname) as (username) ← will have to be adapted for actual training environment – Forward the webserver running on the loopback interface on (hostname) to port 8080 on your local computer – Verify with your web browser that the tunnel works UNIX and Linux an Introduction, developed 21 of 71 by Mathias Homann
  • 22.
    Learning about UNIXcommands • UNIX has two built-in documentation systems: – info pages – man pages • to read the man page (manual page) of a given command, use man command – Try this out with man man • to look if there is a command that does something, try the apropos command • the apropos command lists the summary lines of all man pages that contain the term you specified UNIX and Linux an Introduction, developed 22 of 71 by Mathias Homann
  • 23.
    Learning about UNIXcommands (2) • to look at what a certain command does without reading the whole man page, use whatis program • this prints out the summary only of the command's man page • to find where in the filesystem a command is, use the which command UNIX and Linux an Introduction, developed 23 of 71 by Mathias Homann
  • 24.
    Shells and theenvironment • the shell is the interface to the system • the shell executes commands and shows the results • the “environment” is the total of stored settings inside a shell • important shell variables on UNIX systems: $PATH $PS1 $CWD UNIX and Linux an Introduction, developed 24 of 71 by Mathias Homann
  • 25.
    Shells and theenvironment (2) • UNIX knows several different shells: sh bash ksh zsh csh • the default shell on most Linux systems is bash • the set command shows all environment variables • change environment variables by assigning and exporting them • all exercises will use bash if not stated otherwise UNIX and Linux an Introduction, developed 25 of 71 by Mathias Homann
  • 26.
    Internal and externalcommands • shells have built-in versions of most “system commands” • to “force” the use of a system command instead of the builtin, use the absolute path to the binary: /bin/ls • external commands are standardized • builtins differ from shell to shell UNIX and Linux an Introduction, developed 26 of 71 by Mathias Homann
  • 27.
    Interactive shells +startup scripts • an “interactive” shell is one that you type commands into • “non-interactive” shells are the ones that execute scripts • a “login” shell is the first interactive shell that gets started when you log in • the “profile” or “rc file” is a script that gets executed by a shell on startup • most shells execute more than one startup script, and execute a different set of scripts when started non- interactive, and yet another for the login shell UNIX and Linux an Introduction, developed 27 of 71 by Mathias Homann
  • 28.
    Interactive shells +startup scripts (2) • examples - the bash shell executes on startup as a login shell (in this order): /etc/profile ~/.bash_profile ~/.bash_login ~/.profile • if the bash shell is started interactively but not as a login shell it only executes ~/.bashrc • a non-interactive bash shell executes the file listed in $BASH_ENV if the variable is set and the file exists • note: a login bash executes ~/.bash_logout on exit UNIX and Linux an Introduction, developed 28 of 71 by Mathias Homann
  • 29.
    Wildcards • a wildcardis a pattern that describes a path or filename • wildcard expansion in shell commands happens on the shell level, before the actual command gets executed • different shells handle wildcards in different ways • exercises: – Which wildcard would mean “all jpeg files in this folder”? – Which pattern means “all jpeg files in this folder that start with the letter a”? UNIX and Linux an Introduction, developed 29 of 71 by Mathias Homann
  • 30.
    Command aliases • acommand alias is a “placeholder” for a command and optionally one or more commonly used options • for example, on openSUSE Linux there is an alias “dir” which calls ls -l • a list of all aliases is shown by entering the alias command with no parameters • to see the aliases and wildcards expansion before commands get executed, enter set -x UNIX and Linux an Introduction, developed 30 of 71 by Mathias Homann
  • 31.
    Paths • paths onUNIX systems are all inside a “tree” of folders that starts at the “root” folder or / – the user's home folder is usually held under /home and named after the username, e.g. /home/fred – the “search path” describes which folders to look in when a shell tries to find a command – the search path is stored in the environment variable $PATH – the minimal search path should contain /bin and /usr/bin – directories in the search path are separated by the colon: PATH=/bin:/usr/bin UNIX and Linux an Introduction, developed 31 of 71 by Mathias Homann
  • 32.
    Everything is afile ... • ... not just a file • /proc/12345/status is a “virtual file” containing status information about process 12345 • /dev/mem is a “device file” that is actually an interface to the system's memory • /dev/sg0 is a “device file” for the first “generic” SCSI device in a system • /dev/sdb5 is a “device file” that references the first logical partition on the second external, SCSI or SATA disk UNIX and Linux an Introduction, developed 32 of 71 by Mathias Homann
  • 33.
    Finding files –locate vs find • there are two ways to find a file, locate and find • the locate command uses a database of existing files which needs to be updated regularly, which usually happens once a day • the find command actually searches through the filesystem – this can take a while, but does not rely on a database that can be outdated locate can only search by shell wildcards find can search for anything, not just names • exercise: – find all files that changed since yesterday UNIX and Linux an Introduction, developed 33 of 71 by Mathias Homann
  • 34.
    Filesystems • UNIX knowsseveral different filesystems • one of the main differences is whether a filesystem is “journaling” or not • A journaling filesystem keeps track of the changes that will be made in a journal, before committing them to the main filesystem. In the event of a system outage, such filesystems are quicker to bring back online and less likely to be corrupted. • commonly used filesystems on Linux are: – ext2, ext3, ext4, jfs, xfs, reiserfs – Ext2 is the only of these that is not journaling UNIX and Linux an Introduction, developed 34 of 71 by Mathias Homann
  • 35.
    Filesystems (2) • filesystemlimits: names paths filesize volume size ext2 255 unlimited 16GB - 2TB 2TB-32TB ext3 255 unlimited 16GB - 2TB 2TB-32TB ext4 256 unlimited 16GB - 16TB 1EB jfs 255 unlimited 8TB 512TB-4PB xfs 255 unlimited 8EB 8EB reiserfs 4032/226 unlimited 8EB 16TB UNIX and Linux an Introduction, developed 35 of 71 by Mathias Homann
  • 36.
    Mounting • all diskpartitions need to be mounted to be accessible • a disk partition gets mounted into a folder, called the “mount point” • a UNIX system has a file with a list of all mount points that need to be mounted when the system starts: /etc/fstab – to mount a filesystem that is not listed in /etc/fstab you need superuser privileges – to access a partition you need to know its device node – device nodes are special files in /dev UNIX and Linux an Introduction, developed 36 of 71 by Mathias Homann
  • 37.
    Mounting (2) • forexample, the device node for a partition on a hard disk would be /dev/hda2 - meaning “the second primary partition on the first hard disk” • exercise: – explore what is mounted, and what is mounted on boot UNIX and Linux an Introduction, developed 37 of 71 by Mathias Homann
  • 38.
    Links • a linkis a “pointer” to a file – can be a “soft” or a “hard” link – a softlink is a pointer at a file, and can link across filesystems – a hardlink is another directory entry that points at the same starting sector of the filesystem • hardlinks are only possible within one filesystem, and cannot point at directories • links are created with the ln command ln creates hardlinks by default – for softlinks use the -s option • exercise: – create a link to /usr/bin/bash in ~/bin/ UNIX and Linux an Introduction, developed 38 of 71 by Mathias Homann
  • 39.
    Devices and drivers •every piece of hardware is accessed through a device file under /dev • examples: /dev/hda - the first IDE harddisk /dev/sdb - the second SCSI or SATA disk /dev/nvidia – an NVIDIA graphics card /dev/dsp0 – the sound card UNIX and Linux an Introduction, developed 39 of 71 by Mathias Homann
  • 40.
    Devices and drivers(2) • access to hardware is governed by the file permissions on those special files • for example, the permissions on a graphics card could be set to allow only root and anyone in the group “video” access to write to it, therefore only users in the group “video” would be able to run 3d accelerated applications • exercise: – find out which group has write access to the cd/rw drive, e.g. would have sufficient permissions to use a CDRW writer on the system UNIX and Linux an Introduction, developed 40 of 71 by Mathias Homann
  • 41.
    Permissions and ownership •there are 3 levels of ownership of any file: – “owner” – “group” – “others” • permissions on a file: – read, write, execute, separately for all three owner types • example: – a file usually is read/write for the owner, and read-only for anyone else: -rw-r--r-- • change the owner: chown username file • change the group: chgrp groupname file • change permissions: chmod <permissions> file UNIX and Linux an Introduction, developed 41 of 71 by Mathias Homann
  • 42.
    Background processes • UNIXcan run commands in the background so that you can do something else – To execute a program in the background, end the commandline with & – To send a running program to the background, press ctrl-z and enter bg – To view the list of running background processes, enter jobs • when you logout while you still have background processes running, they will be killed unless you started them with nohup: nohup sleep 10000 will run sleep 10000 in the background – the output of the command will be written to nohup.out UNIX and Linux an Introduction, developed 42 of 71 by Mathias Homann
  • 43.
    Scheduled processes • to run a program at regular intervals, use cron • cron keeps lists of scheduled jobs • edit the list with crontab -e • to run a program at a certain time once, use at at 23:00 asks for a sequence of commands • the job gets deleted once it runs UNIX and Linux an Introduction, developed 43 of 71 by Mathias Homann
  • 44.
    Processes and threads •a process is one running executable – A single process can have more than one running thread – To see all processes, use top, htop, or the ps command – To send signals to processes, use the kill command • several “levels” of “kill signal”, ordered by “lethality”: kill -HUP tells a process to shut down nicely kill -QUIT – the same but a bit more urgent kill -KILL kills a process kill -l lists all possible signals with name and numeric value kill -9 is equivalent to kill -KILL • combined exercise: – start a process in the background that waits for 10 minutes – kill it before the time is over UNIX and Linux an Introduction, developed 44 of 71 by Mathias Homann
  • 45.
    Output and inputredirection • in- and Output can be redirected with the < and > symbol • redirect input: cat < /tmp/file.txt • redirect output: cat < /tmp/file.txt > /tmp/foo.bar • redirect the error output: sort < foo 2>/dev/null UNIX and Linux an Introduction, developed 45 of 71 by Mathias Homann
  • 46.
    Pipes • pipes areused to direct the output of one command to the input of another • pipes use the pipe symbol: | • example: ls -la | less pipes the output of the ls command into the less pager • exercise: sort the content of the passwd file by alphabet UNIX and Linux an Introduction, developed 46 of 71 by Mathias Homann
  • 47.
    Shell history • UNIXshells keep track of your commands • to browse through this list use the cursor up and cursor down keys • to re-execute a previous command, find it in your history and press enter • to find a previous command more quickly, press ctrl-r and start typing; the history will “jump” to the newest match of what you're typing UNIX and Linux an Introduction, developed 47 of 71 by Mathias Homann
  • 48.
    Shell history (2) •to view the whole history, use the “history” command • to re-execute a certain command, find its number in the history, then enter !number, e.g. !514 to re-run number 514 from your history • to search in your history: press ctrl-r, then start typing • exercise: – use a few commands (ls, man, ls -l, ls -la) then view the history, then re-run some of them, then search in your history UNIX and Linux an Introduction, developed 48 of 71 by Mathias Homann
  • 49.
    The power ofgrep • grep searches for text in files • grep can use regular expressions • grep has many options - read the man page • most useful options: – -i for “search case insensitive”, and -v for “inverse” or “search for anything that does not match my search string” • example: grep myusername /var/log/messages – find all lines in logfile that contain myusername • exercise: – filter the main system logfile for all lines from last Monday UNIX and Linux an Introduction, developed 49 of 71 by Mathias Homann
  • 50.
    Less is more •less is a “pager” to view large text files on screen • less can search for text: – press : then enter a slash followed by your text • less can switch to “continuous refresh” to refresh a file that is still being written; press F to switch • less can read from “stdin”; pipe commands into it to paginate the output • g and g jump to the end and the start of the file • to search in less, type /searchterm • to jump to a line number, type :number • example: less /var/log/messages • exercise: – display the content of /usr/bin in less UNIX and Linux an Introduction, developed 50 of 71 by Mathias Homann
  • 51.
    The vi editor •the editor vi exists on every UNIX system • vi has two modes: insertion mode and command mode – The editor begins in command mode, where the cursor movement and text deletion and pasting occur – Insertion mode begins upon entering an insertion or change command – [ESC] returns the editor to command mode (where you can quit, for example by typing :q!) • most commands execute as soon as you type them except for "colon" commands which execute when you press the return key • nifty: vi does syntax highlighting; activate with :syntax UNIX and Linux an Introduction, developed 51 of 71 by Mathias Homann
  • 52.
    The vi editor(2) • quitting vi: :x Exit, saving changes :q Exit as long as there have been no changes ZZ Exit and save changes if any have been made :q!Exit and ignore any changes • switching to edit mode: i – insert r – replace a – append UNIX and Linux an Introduction, developed 52 of 71 by Mathias Homann
  • 53.
    The vi editor(3) • cursor movement • cursor keys, and: ^ or 0 – move to start of line $ - move to end of line • block and line operations: dd – delete line yy – copy line p – paste line UNIX and Linux an Introduction, developed 53 of 71 by Mathias Homann
  • 54.
    Regular expressions • regularexpressions are powerful string matching patterns • examples: . - any one character * - any repetitions of the pattern left of the star .* - any number of any characters [:upper:] - all UPPER CASE characters n – newline [a-q] – any letter between a and q UNIX and Linux an Introduction, developed 54 of 71 by Mathias Homann
  • 55.
    Regular expressions (2) •more info: • the “Regular expressions cheat sheet v2” – http://www.addedbytes.com/cheat-sheets/regular-expressions- cheat-sheet/ • example: – a regular expression cat.*dog matches any string that has “cat” and “dog” in it with any other characters in between them, but only in that order! UNIX and Linux an Introduction, developed 55 of 71 by Mathias Homann
  • 56.
    Sed – thestream editor • Sed reads text from stdin, does edit operations on it, and prints the result on stdout • examples: sed -e 's/red/green/g' <colors.txt >colors1.txt – replaces all occurrences of “red” with “green” • exercise: – read the man page, then write a text file, then replace all “the” with “THE” • exercise: – print out each directory in the path variable in a new line UNIX and Linux an Introduction, developed 56 of 71 by Mathias Homann
  • 57.
    Awk – patternscanner for text files • Awk scans text files for patterns and operates on them • examples: awk '{print $2“n“}' < file.txt – print the second word of every line of file.txt • exercise: – read the man page, then use vi to write a text file with a few lines of the format: Username:Firstname:Lastname:Telephone number – then, use awk to convert this into ldap syntax: dn: uid=Username, dc=example, dc=com cn: Firstname Lastname sn: Lastname telephoneNumber: Telephone number UNIX and Linux an Introduction, developed 57 of 71 by Mathias Homann
  • 58.
    Writing your ownscripts • scripts can be written with any text editor • scripts don't need a special filename suffix • scripts need to be executable (hint: chmod) • scripts can explicitly state which shell should be used to execute it by putting the “hashbang” for the shell in the first line: #!/bin/bash – a bash script #!/bin/csh – a c shell script #!/bin/sh – a “sh” shell script • “sh” is the most basic shell; exists on any UNIX • exercise: – write a script that prints “hello world” ten times UNIX and Linux an Introduction, developed 58 of 71 by Mathias Homann
  • 59.
    Syslog and logfiles • syslog is a UNIX daemon that collects and handles log file messages • syslog handles messages according to severity and facility • details: see man page • syslog can write messages to one or more files depending on severity and facility UNIX and Linux an Introduction, developed 59 of 71 by Mathias Homann
  • 60.
    Syslog and logfiles (2) • logfiles usually exist in /var/log • the syslog configuration file is in /etc and states which facility.severity goes in which file • exercise: – check the syslog configuration to see which files hold messages of failed logins – then check if there were any recent failed login attempts UNIX and Linux an Introduction, developed 60 of 71 by Mathias Homann
  • 61.
    Finding facts abouta system • most Linux systems have a file in /etc/ that contains information about which distribution is running – for example, /etc/redhat-Release uname -a prints out information about the system • use mount and df to find out about disk sizes and mount points • a lot of information is in /proc • almost all configuration files are in /etc and its subdirectories UNIX and Linux an Introduction, developed 61 of 71 by Mathias Homann
  • 62.
    Finding facts abouta system (2) • most UNIX systems use some form of software package management – the list of installed packages can tell you a lot about the purpose of a system – on Linux there are two widely used package management systems, rpm and apt/dpkg – rpm is used on openSUSE, redhat, and their derivatives and commercial versions – on debian, ubuntu and derivatives, apt and dpkg is used – commands such as rpm -qa and dpkg –get-selections show the list of installed packages – these lists are long, less is your friend UNIX and Linux an Introduction, developed 62 of 71 by Mathias Homann
  • 63.
    Finding facts abouta system (3) • to list all loaded kernel modules lsmod • use insmod or modprobe to load additional kernel modules into a running system • use modprobe -r to remove a module and all dependencies from a running system • most of them are hardware drivers or drivers for “features” UNIX and Linux an Introduction, developed 63 of 71 by Mathias Homann
  • 64.
    Gathering statistics • commandsto explore the status of the system vmstat and free show memory usage iostat shows drive accesses netstat shows network statistics netstat can also show active network connections and listening processes; read the man page • exercise: – check how much memory the machine has, and how much is used; then check how many active network connections exist that are not SSH connections UNIX and Linux an Introduction, developed 64 of 71 by Mathias Homann
  • 65.
    Writing advanced scripts •scripts can do math: a=$(($b+$c)) • scripts can have flow control – if (condition) then {command; command;}; else { command; command;}; endif – do { command; command; } while (condition); – for (iterator); do { command; command; } done; • scripts can have functions • scripts can return values to the calling process UNIX and Linux an Introduction, developed 65 of 71 by Mathias Homann
  • 66.
    Writing advanced scripts(2) • exercise: – write a script that: – finds all log files under /var log – takes all files older than 2 days – prints their name, size and age – (size in 'human readable' format, 1024k = 1 M etc) UNIX and Linux an Introduction, developed 66 of 71 by Mathias Homann
  • 67.
    More exercises • createa file “a”, write “hello world” into it, then hardlink it as “b” • check the inodes of both files – (hint: there's an option for ls that does that) • create a softlink “c” to “a”, check inodes • run tail -f /var/log/messages, send it to the background, edit a different file in vi, exit vi, bring the tail command back to the front • compare the differences between the various ps output formats: ps –ax ps -ef ps -axf – sort /etc/passwd alphabetically – sort it by uid UNIX and Linux an Introduction, developed 67 of 71 by Mathias Homann
  • 68.
    Useful commands • cd – change directory • pushd – change directory but it stores the previous directory on a fifo stack • popd – returns to previous directory from the stack • cat – read from file or stdin, write to file or stdout (example: cat /etc/motd to print that file to screen) • cp – copy files • mv – move / rename files • head & tail – print the first or last lines of a file • find – search for files UNIX and Linux an Introduction, developed 68 of 71 by Mathias Homann
  • 69.
    Useful commands (2) • less – display a file in a pager • grep – search for text in files • sed – command-line editing of files • wc – count lines/words/characters in files • df – disk free (shows free space in partitions) • du – disk usage (adds up file sizes, more accurate than df, takes some time to finish) • gzip – compress files, use gunzip to decompress • diff – compare files UNIX and Linux an Introduction, developed 69 of 71 by Mathias Homann
  • 70.
    Higher wizardry –perl • perl is disputedly the most powerful, simple, complicated, and obscurely easy to understand scripting language • perl is enough for its own workshop day :) UNIX and Linux an Introduction, developed 70 of 71 by Mathias Homann
  • 71.
    Discussion and closing •questions • comments • Instructor Evaluation Form UNIX and Linux an Introduction, developed 71 of 71 by Mathias Homann