An introduction to UNIX What is UNIX?


Published on

Published in: Technology
1 Comment
  • Unix Video Training Tutorial DVDs (2 DVDs) Sale Price 12.95 More than 5.9 GB of Video files
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

An introduction to UNIX What is UNIX?

  1. 1. An introduction to UNIX <ul><li>What is UNIX? </li></ul><ul><li>Why do we use UNIX? </li></ul><ul><li>The UNIX file system </li></ul><ul><li>Logging in and getting started </li></ul><ul><li>Getting help! </li></ul><ul><li>Data storage and file sizes </li></ul><ul><li>Some useful programs </li></ul><ul><li>Shell scripts and awk </li></ul>
  2. 2. 1. What is UNIX? UNIX is an Operating System (OS) derived from an earlier OS called MULTICS. Multiuser Multitasking More than one user can use the same computer at the same time. More than one task (computer program) can be run at the same time. The OS is a collection of ~250 commands. Each command generally has a relatively simple function; for example, the cp command copies a file, the rm command removes a file. Commands can be combined to perform more complex tasks. As a trivial example, a file could be renamed by copying it to a new file name then removing the old file. In practice, this function is performed with the move command mv .
  3. 3. 2. Why do we use UNIX? when we could use a Windows PC or Apple Mac ? <ul><li>Because it is multiuser and multitasking it is worthwhile having UNIX computers with very fast processor(s) and large amounts of memory, hence capable of solving very complex problems, since they can be shared amongst many users. </li></ul><ul><li>Software stability. </li></ul><ul><li>Data security. Users are required to log in before they can do anything. Logging in restricts unauthorised use, allows all files created by a user to be identified as belonging to them and, if necessary, only visible to that user (NB features emulated by latest Windows and Mac OS). </li></ul><ul><li>Hardware reliability. </li></ul><ul><li>It has the programs and tools that allow us, as scientists, to perform the tasks we require! </li></ul>
  4. 4. 3. The UNIX file system <ul><li>The UNIX file system is very similar to that of a Windows PC or an Apple Mac. There are some significant exceptions: </li></ul><ul><li>The concept of “local drives” and “network drives” is unnecessary in UNIX. All disks, local or networked, appear as a part of a single file structure as subdirectories of a single “root” directory. </li></ul><ul><li>All files and directories have an owner – the user that originally created them. </li></ul><ul><li>All files and directories have a series of “permissions” assigned to them by their owner. </li></ul>Important point: You are allocated a homespace in which you should store your files. This space belongs to you. Your homespace is a disk which is automatically available, and is exactly the same, on every UNIX machine in SOC.
  5. 5. MS Windows and UNIX comparison C: A: D: N: sue ollie dan mail exec sue ollie dan mail exec local cdrom users floppy / N:danexec /users/dan/exec
  6. 6. File paths sue ollie dan mail exec local cdrom users floppy / my.doc [1] Absolute location: / mail dan users /users/dan/mail/my.doc / sue dan mail users / [2] Relative to the directory “ sue ” sue dan mail users / ../dan/mail/my.doc dan mail users sue dan mail users [3] For Dan, relative to his homespace: sue dan users ~/mail/my.doc mail dan mail [4] For everyone else, relative to Dan’s homespace: ~dan/mail/my.doc mail
  7. 7. 4. Logging in and getting started <ul><li>There are two ways in which you may choose to use the UNIX computers at SOC. </li></ul><ul><ul><li>Console log in – to physically sit in front of a particular UNIX computer and work on it “locally”. </li></ul></ul><ul><ul><li>Remote log in – from a PC you can use the Exceed program to log in to a UNIX computer somewhere else in the building and view any output across the network. </li></ul></ul>For much of the practical work associated with this course, you will be using a remote login to the UNIX computer “oriana” (or other machines).
  8. 8. Console login
  9. 9. Splash screen
  10. 10. CDE The command prompt
  11. 11. Remote login A remote login to a UNIX computer is done using “telnet” or “xterm”. These programs will give you a window with a command prompt exactly the same as the “terminal” window used when locally logged in to the UNIX computer. The command prompt
  12. 12. Using the command prompt The general structure is as follows: command [ options ] file or directory name(s) options modify how the command works and are used either to supply extra information required by the command or to use it in a way other than its default manner. The options are usually specified using letters preceded by a “-”. Possible options are listed in the on-line help for each command which will be described in a bit… The UNIX commands entered at the prompt are interpreted into actions by the “shell interpreter”. The shell requires commands to have the correct syntax so that they are unambiguous. UNIX commands are CaSe SeNsItIvE. Option, file names or directory names must be separated by a spaces.
  13. 13. Basic commands pwd print working directory (to screen not printer…!) cd name change directory to name ls lists the contents of the current directory ls -la detailed listing of the current directory cp file1 file2 copy a file mv file1 file2 move or rename a file rm name remove (delete) the file called name mkdir name make a new directory called name rmdir name remove (delete) the directory called name mail/ exec/ my.doc 2dan@sowhat> 1dan@sowhat> ls -la 1dan@sowhat> ls Example for ls: drwxr-x--- dan ugrad 4096 Jan 1 12:00 ./ drwxr-x--- dan ugrad 12288 Jan 1 12:00 ../ -rwxr-x--- dan ugrad 12288 Jan 1 12:00 .gmtdefaults drwxr-x--- dan ugrad 15 Jan 1 12:00 mail/ drwxr-x--- dan ugrad 15 Jan 1 12:00 exec/ -rwxr-x--- dan ugrad 51264 Jan 1 12:00 my.doc 2dan@sowhat> Notice that directories can be identified be a trailing “ / ” in both cases
  14. 14. File permissions <ul><li>Each kind of permission can be set with three levels of security: </li></ul><ul><li>User - the owner of the file </li></ul><ul><li>Group - a subset of the users on the local network (i.e., within SOC) with whom you generally work. You will be in group “ugrad”. Your group is decided, and can only be changed, by the system administrator. </li></ul><ul><li>Other or “World” – everyone else! </li></ul>For example, you may want to keep your email private so that only you can read it. In this case, you would set your email file so that the user (you) can read it, but that group and other cannot. If you want to make changes or delete a file write permission must be set. A directory must have both read and execute permissions set for you to see the contents. There are three kinds of permissions : These determine what you can actually do with a file. <ul><li>Write permission </li></ul><ul><li>Executable permission </li></ul><ul><li>Read permission </li></ul>
  15. 15. Setting file permissions ls –la -rw------- dan ugrad 51264 Jan 1 12:00 my.doc 4dan@sowhat> Permissions are set with the command chmod . The syntax is: chmod [-R] ugo+/-rwx name where + adds permissions and - removes permissions and name is the file or directory being set. When name is a directory, the option -R will change the permissions of all files and subdirectories within that directory. user group other permissions are denied where a “-” occurs r = read w = write x = execute 2dan@sowhat> chmod go-r my.doc 3dan@sowhat> 1dan@sowhat> ls –la -rw-r--r-- dan ugrad 51264 Jan 1 12:00 my.doc Example:
  16. 16. 5. Getting help <ul><li>Help is available for each UNIX command by type “ man ” followed by the command you want help on. </li></ul><ul><li>The man pages are often long, but follow a standard structure: </li></ul><ul><ul><li>A brief description of the command with a list of all the possible options </li></ul></ul><ul><ul><li>Detailed description of each option, usually alphabetically </li></ul></ul><ul><ul><li>One or more examples </li></ul></ul><ul><ul><li>A list of related commands </li></ul></ul><ul><li>It is possible to do a “keyword” search of all the man pages using the command “ man -k keyword ”, although this often turns up a very large number of possibilities… </li></ul>
  17. 17. 6. Data storage and file sizes <ul><li>The size of data files. </li></ul><ul><li>Very often any data you will be working with will be in the form of an array. An array can be anything from a single list of numbers representing, for example, gravity measurements along a profile, to a multidimensional array, such as changes in sea water temperature in a 3D volume with time. </li></ul><ul><li>The size of an array, in terms of computer memory, is dependent upon: </li></ul><ul><ul><li>The number of dimensions (x, y, z, time, etc.) </li></ul></ul><ul><ul><li>The number of values in each dimension </li></ul></ul><ul><ul><li>The numerical precision to which the contents of the array are stored </li></ul></ul>
  18. 18. Example 1 Gravity measurements from a satellite over 2 ° of latitude and longitude, sampled every 1’ forms a grid of 121  121=14,641 values. The precision with which the measurements are made dictate how many bytes are required to store it: A precision of 6 significant figures requires 4 bytes per value. 14,641  4 = 58,564 bytes or ~60 KB The entire Earth, at the same resolution would require just under 1 GB or 927,936,000 bytes.
  19. 19. Example 2 A commercial 3D marine seismic survey using a ship with 6 streamers, 320 receiver channels per streamer, and a 25 m shot spacing will create huge amounts of data. A 10 km square grid with 5 s record would produce : Number of source shots per 10 km line = 400 Number of lines per 10 km grid = 80 Number of receivers (6  320) = 1920 Number of data points per receiver assuming 2 ms (0.002 s) sample rate (5 / 0.002) = 2500 Using 4 bytes per sample would require 400  80  1920  2500  4 = 6.144  10 11 KB or 614.4 GB !
  20. 20. b) UNIX commands for checking file sizes and disk usage ls -la we have already seen shows file sizes in bytes du –sk name returns the size of the sum of the contents of a directory called name in KB df –k . returns the size, used space, and remaining available space on the disk on which the current directory is located in Example :
  21. 21. c) Data storage devices <ul><li>Fixed (hard) disks – either “internal” or “local” to the computer you are working on or “networked” on a file server </li></ul><ul><li>Removable disk </li></ul><ul><ul><li>Rewritable : 3 1/2 ” floppy disk, Zip, CD-RW, DVD-RW </li></ul></ul><ul><ul><li>Write-once : CD-R, DVD-R </li></ul></ul><ul><li>Tape – 4 mm DAT, 8 mm EXABYTE, DLT etc. </li></ul>Day-to-day work : Fixed disks have high rates of access to data Archival of data : Tape has high storage capacity and low cost
  22. 22. 7. Some useful programs <ul><li>nedit is a text editor you can use for writing shell scripts and FORTRAN programs. It understands and highlights the syntax for many computer languages (including FORTRAN, C and HTML), automatic line numbering and indentation, and has a spell checker. </li></ul><ul><li>gs or ghostscript displays a PostScript file on screen. Postscript is used for printing in UNIX. </li></ul><ul><li>setup – not a program as such, but typing “setup” on a UNIX computer at SOC will give you a list of all the software packages installed by the IT group. Typing “setup” followed by the name of the package will make it available to you. You will need to do this to access FORTRAN and GMT in the practicals for this course. </li></ul>
  23. 23. 8. Shell scripts and awk <ul><li>Aim : To use UNIX to manipulate data files </li></ul><ul><ul><li>Reformatting the contents of data files </li></ul></ul><ul><ul><ul><li>Reordering columns of data </li></ul></ul></ul><ul><ul><ul><li>Math functions on columns of data </li></ul></ul></ul><ul><ul><li>Extracting subsets of data from files </li></ul></ul><ul><ul><ul><li>Data that matches chosen criteria </li></ul></ul></ul><ul><ul><ul><li>Data decimation </li></ul></ul></ul>Question : “I could do this in Excel!” Answer : “Not with a one line command and not on a million data points”
  24. 24. Some more commands more name reads the file called name and displays the contents at the command prompt, one page at a time cat name1 name2 read the files called name1 and name2 and displays the contents at the command prompt grep ' text ' outputs only lines that contain the text text grep -v ' text ' outputs only lines that do not contain the text text sort –k n name sort the lines in the file called name so that values in the n th column are in ascending order tr ' text1 ' ' text2 ' exchange any character in text1 with its corresponding character in text2 Example: grep 'line1' : Input file: 30 50-line1 10 1-line1 3 11-line2 42 20-line2 7 1-line1 13 13-line3 Output file: 30 50-line1 10 1-line1 7 1-line1 Example: grep -v 'line1' : Input file: 30 50-line1 10 1-line1 3 11-line2 42 20-line2 7 1-line1 13 13-line3 Output file: 3 11-line2 42 20-line2 13 13-line3 Example: sort –k 1 Input_file : Input file: 30 50-line1 10 1-line1 3 11-line2 42 20-line2 7 1-line1 13 13-line3 Output file: 3 11-line2 7 1-line1 10 1-line1 13 13-line3 30 50-line1 42 20-line2 Example: tr '-' ' ' : Input file: 30 50-line1 10 1-line1 3 11-line2 42 20-line2 7 1-line1 13 13-line3 Output file: 3 11 line2 7 1 line1 10 1 line1 13 13 line3 30 50 line1 42 20 line2
  25. 25. awk <ul><li>awk is a command that provides many powerful functions for manipulating data files. We will use it for reformatting and math functions on columns of data. </li></ul><ul><li>The awk function “print” </li></ul><ul><ul><li>Columns in the input data file are referred to by their number preceded by a “$”. </li></ul></ul><ul><ul><li>Math functions are written in the same manner as FORTRAN using +,-,/,*, with brackets etc. </li></ul></ul><ul><ul><li>To raise one number to the power n the notation is ^ n . </li></ul></ul>The basic command structure is: awk '{print $1, $2,…}' For example, if we have a file with latitude and longitude in degree, minutes and seconds and want to convert it into decimal degrees: Input: 30. 45. 24. 17. 30. 0. awk '{print $1+($2/60)+($3/60^2), $4+($5/60)+($6/60^2)}' : 30.756666 17.5
  26. 26. Piping data and redirecting output When combining UNIX commands, a method for passing data from one command to another is needed. This is achieved with a pipe, represented by the vertical line symbol ( | ). <ul><li>By default, the output from a UNIX command appears at the command prompt. This is useful for looking at the contents of a file with the more command, for example. </li></ul><ul><li>To direct output to a file, it must be redirected using the “ > ” symbol. </li></ul><ul><ul><li>“ > ” creates a new file </li></ul></ul><ul><ul><li>“ >> ” appends to an existing file </li></ul></ul>Example: more input.dat|awk '{print $1, $2}' > output.dat
  27. 27. Putting this all together… We can continue piping data from one command to another as many times as is required to modify a data file to our requirements. The disadvantage is that the commands we create are getting complicated to remember and take time to type in – how can we make things easier? <ul><li>Piping the data, rather than outputting a new file at each step is advantageous: </li></ul><ul><ul><li>Speed – writing to and reading from disk is slow, memory is fast </li></ul></ul><ul><ul><li>Saves disk space </li></ul></ul>
  28. 28. The shell script <ul><li>A shell script is a file in which UNIX commands are written in plain text. </li></ul><ul><li>Each line in the shell script contains a UNIX command exactly as it would be typed in at the command prompt. </li></ul><ul><li>A shell script is created and edited in a text editor such as nedit. </li></ul><ul><li>The advantage of a shell script is that it makes it easy to combine many UNIX commands, which together perform complex or repetitive tasks, without having to type them in every time. </li></ul><ul><li>You will use shell scripts to manipulate data files and produce plots of them using GMT. </li></ul>
  29. 29. Making a text file into a shell script UNIX needs to know that a text file is a shell script. There is more than one way to do this, all are easy, and you can choose which suits you best: <ul><li>Give the file the extension “ .csh ” and give it user execute permission ( chmod u+x ) </li></ul><ul><li>Enter the text “ #! /bin/csh ” in the first line of the file and give it user execute permission </li></ul><ul><li>Execute the script by typing “ csh script_name ” </li></ul>
  30. 30. A example shell script the file is a shell script lots of familiar awk commands sets up a variable called “infile” in which the name of the input file is stored creates lots of temporary files from the input file combines all the temporary files deletes the temporary files invokes gs to display a PostScript file