Published on

Published in: Economy & Finance, Technology
1 Like
  • Shell Scripting: How to Automate Command Line Tasks Using Bash Scripting and Shell Programming --- http://amzn.to/1WzJYZm
    Are you sure you want to  Yes  No
    Your message goes here
  • Linux Command Line and Shell Scripting Bible --- http://amzn.to/1LyurZd
    Are you sure you want to  Yes  No
    Your message goes here
  • Classic Shell Scripting --- http://amzn.to/1R98HlK
    Are you sure you want to  Yes  No
    Your message goes here
  • http://www.dbmanagement.info/Tutorials/Unix.htm Unix Video Training Tutorial DVDs (2 DVDs) Sale Price 12.95 More than 5.9 GB of Video files
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. HSBC Global Technology UNIX -Kirtikumar Shinde
  2. 2. Topics <ul><li>1. Introduction </li></ul><ul><li>Unix Filesystem </li></ul><ul><li>File Permissions </li></ul><ul><li>Advanced Commands </li></ul><ul><li>Shell </li></ul><ul><li>Shell Programming </li></ul><ul><li>AWK </li></ul><ul><li>My Learnings </li></ul><ul><li>Questions? </li></ul>
  3. 3. Introduction <ul><li>Multi-user and Multitasking OS </li></ul><ul><li>Mostly Written in C at Bell Labs </li></ul><ul><li>Most of the servers are on Unix </li></ul><ul><li>Unix Flavors </li></ul><ul><ul><li>Solaris </li></ul></ul><ul><ul><li>HP </li></ul></ul><ul><ul><li>AIX </li></ul></ul><ul><ul><li>Linux </li></ul></ul><ul><li>Current DS Server – Which Unix? </li></ul>
  4. 4. Introduction <ul><li>Architecture </li></ul>
  5. 5. Unix File System <ul><li>An upside-down Tree </li></ul>/ etc bin export home user ad s3910120 dev tmp exam.txt work hobby.c proj1 date cal . . . . . . . . . . . . . . . . . . . . . . . . .
  6. 6. File permissions <ul><li>Changing permissions </li></ul><ul><ul><li>chmod g+w empl.txt </li></ul></ul><ul><ul><li>chmod 754 empl.txt </li></ul></ul><ul><li>Changing ownership </li></ul><ul><ul><li>chown gcdv_dev empl.txt </li></ul></ul><ul><li>Changing group </li></ul><ul><ul><li>chgrp develp empl.txt </li></ul></ul>reading others group owner 1 0 0 1 0 1 1 1 1 - r - r - x r x w execution writing
  7. 7. Advanced Commands <ul><li>Sort </li></ul><ul><ul><li>Sorts the contents of a file . </li></ul></ul><ul><ul><ul><li>sort [-b f n r o t] [file name(s)] </li></ul></ul></ul><ul><ul><li>Takes the contents of a file(s) and displays it in sorted order. </li></ul></ul><ul><ul><li>Flags: </li></ul></ul><ul><ul><ul><li>-b ignores blanks </li></ul></ul></ul><ul><ul><ul><li>-f change all lower case letter to upper case before comparing </li></ul></ul></ul><ul><ul><ul><li>-n numeric sort </li></ul></ul></ul><ul><ul><ul><li>-r reverse usual order </li></ul></ul></ul><ul><ul><ul><li>-o sends output of command to some file </li></ul></ul></ul><ul><ul><ul><li>-t field delimiter </li></ul></ul></ul><ul><ul><li>E.g. To sort the Emp.txt on 2 nd field in reverse order </li></ul></ul><ul><ul><li>$sort -t, -n -r +2 Emp.txt </li></ul></ul>
  8. 8. Advanced Commands <ul><li>grep </li></ul><ul><ul><li>Searches a file for pattern . </li></ul></ul><ul><ul><ul><li>grep [-c i l v] <pattern> [file name(s)] </li></ul></ul></ul><ul><ul><li>Takes the contents of a file(s) and displays it in sorted order. </li></ul></ul><ul><ul><li>Flags: </li></ul></ul><ul><ul><ul><li>-c displays count of matching lines. </li></ul></ul></ul><ul><ul><ul><li>-i ignore the case while searching </li></ul></ul></ul><ul><ul><ul><li>-l lists the names of files that matches pattern </li></ul></ul></ul><ul><ul><ul><li>-v displays all lines that don’t contain the pattern </li></ul></ul></ul><ul><ul><li>E.g. To display all lines in the ‘Emp.txt’ that match either the abc or xyz string: </li></ul></ul><ul><ul><li>$grep &quot;abc|xyz&quot; Emp.txt </li></ul></ul>
  9. 9. Advanced Commands <ul><li>tar </li></ul><ul><ul><li>Compression and decompression of files. </li></ul></ul><ul><ul><ul><li>tar -(x|c) [ v f r] <tarfile> [file name(s)] </li></ul></ul></ul><ul><ul><li>Flags: </li></ul></ul><ul><ul><ul><li>-x extract from file. </li></ul></ul></ul><ul><ul><ul><li>-c create new extract file. </li></ul></ul></ul><ul><ul><ul><li>-v displays list files </li></ul></ul></ul><ul><ul><ul><li>-f name of file follows </li></ul></ul></ul><ul><ul><ul><li>-r append to old extract. </li></ul></ul></ul><ul><ul><li>E.g . </li></ul></ul><ul><ul><ul><li>To compress file Emp.txt $tar -cvf Emp.tar Emp.txt </li></ul></ul></ul><ul><ul><ul><li>To decompress file Emp.tar $tar -xvf Emp.tar </li></ul></ul></ul>
  10. 10. Shell <ul><li>Command interpreter that waits for commands, executes them and displays the results </li></ul><ul><li>Bourne shell </li></ul><ul><ul><li>Developed by Steve Bourne at AT&T </li></ul></ul><ul><li>Korn shell </li></ul><ul><ul><li>Developed by David Korn at AT&T </li></ul></ul><ul><li>C-shell </li></ul><ul><ul><li>Developed by Bill Joy for Berkeley Unix </li></ul></ul>
  11. 11. Shell <ul><li>Shell works as follows: </li></ul><ul><ul><li>Shell displays a prompt. </li></ul></ul><ul><ul><li>You type in a command. </li></ul></ul><ul><ul><li>You press the return key. </li></ul></ul><ul><ul><li>The shell interprets the commands typed and tries to find the correct programs to run. </li></ul></ul><ul><ul><li>The kernel runs the requested programs and returns the results to the shell. </li></ul></ul><ul><ul><li>The shell displays the command prompt again </li></ul></ul>
  12. 12. Shell <ul><li>Which Shell I am in ? </li></ul><ul><ul><li>finger –m myusername </li></ul></ul><ul><li>Know your shell – Korn Shell – its features </li></ul><ul><ul><li>Aliases – It allows shorthand for commands. </li></ul></ul><ul><ul><li>Command history – Lets you recall previously entered commands. </li></ul></ul><ul><ul><li>Command line editing – Allows us to edit commands vi style. </li></ul></ul><ul><ul><li>Integrated programming features – It enables common programming tasks to be done cleanly & without creating extra processes. </li></ul></ul><ul><ul><li>Support regular expressions. </li></ul></ul><ul><ul><li>Advanced I/O features – Ability to carry out two way communication with concurrent processes. </li></ul></ul><ul><ul><li>Increased speed of shell code execution. </li></ul></ul><ul><ul><li>Has highly robust security features. </li></ul></ul>
  13. 13. <ul><li>Shell system variables </li></ul><ul><ul><li>PATH = Defines path shell must search in order to execute any command or file. </li></ul></ul><ul><ul><li>HOME = Indicates default working directory of the user. </li></ul></ul><ul><ul><li>PS1 = System prompt 1. </li></ul></ul><ul><ul><li>PS2 = System prompt 2, default value is “>”. </li></ul></ul>Shell
  14. 14. UNIX S hell Programming <ul><li>Shell Script (Shell Procedure) </li></ul><ul><ul><li>A program written in shell programming language is known as a shell script or shell procedure. </li></ul></ul><ul><li>Shell Programming Language </li></ul><ul><ul><li>The shell programming language is a command language with a lot of features common to many computer programming languages, including the structured language constructs: sequence, selection, and iteration. </li></ul></ul><ul><li>Command Languages </li></ul><ul><ul><li>Command languages are interpreted language s </li></ul></ul><ul><ul><li>It allows then use of Unix commands in scripts </li></ul></ul>
  15. 15. UNIX S hell Programming <ul><li>Shell scripting keywords. </li></ul><ul><ul><li>Please Note : Script variables shouldn’t be same as keywords. </li></ul></ul><ul><ul><ul><li>Looping constructs, to be discussed in coming slides. </li></ul></ul></ul><ul><ul><ul><li>set, unset </li></ul></ul></ul><ul><ul><ul><li>Readonly </li></ul></ul></ul><ul><ul><ul><li>Exit </li></ul></ul></ul><ul><ul><ul><li>Ulimit </li></ul></ul></ul><ul><ul><ul><li>Umask </li></ul></ul></ul>
  16. 16. UNIX S hell Programming <ul><li>Create - Simple Hello World shell script. </li></ul><ul><ul><ul><li>#!/bin/sh </li></ul></ul></ul><ul><ul><ul><li>echo &quot;Hello World“ </li></ul></ul></ul><ul><li>Make Files Executable </li></ul><ul><ul><ul><li>$ chmod 777 HelloWorld.sh </li></ul></ul></ul><ul><li>Execute the shell script </li></ul><ul><ul><li>Observe the output ‘Hello World’ will be between two command prompts. </li></ul></ul><ul><ul><ul><li>$ HelloWorld.sh </li></ul></ul></ul>
  17. 17. UNIX S hell Programming <ul><li>Assignments </li></ul><ul><ul><li>Value assignments. </li></ul></ul><ul><ul><li>String assignment – </li></ul></ul><ul><ul><ul><li>var=“Your Name” </li></ul></ul></ul><ul><ul><ul><li>Think if not within double quotes. </li></ul></ul></ul><ul><ul><ul><li>$echo $var </li></ul></ul></ul><ul><ul><ul><li>think if not $ </li></ul></ul></ul><ul><ul><ul><li>Making variable readonly. a=50 readonly a </li></ul></ul></ul>
  18. 18. UNIX S hell Programming <ul><li>echo and Escape Characters </li></ul><ul><ul><li>Display strings as “printf” in C </li></ul></ul><ul><ul><li>The echo command recognizes escape characters. Escape characters all start with the backslash ( ), and you can use them as part of the echo command argument string. </li></ul></ul><ul><ul><li>E.g. </li></ul></ul><ul><ul><ul><li>$ echo “ Hello ” </li></ul></ul></ul>
  19. 19. UNIX S hell Programming <ul><li>Passing parameters to a Shell script </li></ul><ul><ul><li>Consider Welcome.sh, & we are sending name as parameter to a Shell Script. Welcome.sh Vikram </li></ul></ul><ul><ul><li>think If you want to send First name & Last name as one parameter. In above case 0 th parameter value considered will be script name. 1 st parameter value considered will be name, i.e.Vikram. </li></ul></ul><ul><li>Accessing parameters in a Shell script </li></ul><ul><ul><li>Parameter to a shell script starts from index 1. Its value is retrieved by using $1 & so on. … Think how you will access parameter beyond 9 th parameter. </li></ul></ul>
  20. 20. UNIX S hell Programming <ul><li>Shell Positional Variable </li></ul><ul><ul><li>$0 : command itself </li></ul></ul><ul><ul><li>$1 : first parameter </li></ul></ul><ul><ul><li>$n : nth parameter </li></ul></ul><ul><ul><li>$# : no. of parameters </li></ul></ul><ul><ul><li>$? : exit status of last command executed </li></ul></ul><ul><ul><li>$* : all parameters </li></ul></ul><ul><ul><li>$$ : process number of shell </li></ul></ul><ul><ul><li>$! : PID of last background process </li></ul></ul>
  21. 21. UNIX S hell Programming <ul><li>if-then and if-then-else </li></ul><ul><ul><li>The if-then Construct </li></ul></ul><ul><ul><ul><li>if [ condition ] </li></ul></ul></ul><ul><ul><ul><li>then </li></ul></ul></ul><ul><ul><ul><li>true-commands </li></ul></ul></ul><ul><ul><ul><li>f i </li></ul></ul></ul><ul><ul><li>The if-then-else Construct </li></ul></ul><ul><ul><ul><li>if [ condition ] </li></ul></ul></ul><ul><ul><ul><li>then </li></ul></ul></ul><ul><ul><ul><li>true-commands </li></ul></ul></ul><ul><ul><ul><li>else </li></ul></ul></ul><ul><ul><ul><li>false-command(s) </li></ul></ul></ul><ul><ul><ul><li>fi </li></ul></ul></ul>
  22. 22. UNIX S hell Programming <ul><li>if-then-elif </li></ul><ul><ul><ul><li>if [ condition1 ] </li></ul></ul></ul><ul><ul><ul><li>then </li></ul></ul></ul><ul><ul><ul><li>commands_1 </li></ul></ul></ul><ul><ul><ul><li>elif [ condition2 ] </li></ul></ul></ul><ul><ul><ul><li>then </li></ul></ul></ul><ul><ul><ul><li>commands_2 </li></ul></ul></ul><ul><ul><ul><li>else </li></ul></ul></ul><ul><ul><ul><li>commands_n </li></ul></ul></ul><ul><ul><ul><li>fi </li></ul></ul></ul>
  23. 23. UNIX S hell Programming <ul><li>test Command </li></ul><ul><ul><li>The test command is a built-in shell command that evaluates the expression given to it as an argument and return true if the expression is true, if otherwise, false. </li></ul></ul><ul><ul><li>You can use square brackets ( [ ] ) instead of the word test. </li></ul></ul><ul><ul><li>Example </li></ul></ul><ul><ul><ul><li>if test $str1 = $str2 then echo “Something” fi </li></ul></ul></ul><ul><ul><ul><li>if [$str1 = $str2] then echo “Something” fi </li></ul></ul></ul>
  24. 24. UNIX S hell Programming <ul><li>Logical Operators </li></ul><ul><ul><li>-a AND Operator </li></ul></ul><ul><ul><li>-o OR Operator </li></ul></ul><ul><ul><li>! NOT Operator </li></ul></ul><ul><li>Numeric Test Operators </li></ul><ul><ul><li>-eq Is number1 equal to number2 ? </li></ul></ul><ul><ul><li>-ne Is number 1 not equal to number2 ? </li></ul></ul><ul><ul><li>-gt Is number1 great than number2 ? </li></ul></ul><ul><ul><li>-ge Is number1 great than or equal to number2 ? </li></ul></ul><ul><ul><li>-lt Is number1 less than number2 ? </li></ul></ul><ul><ul><li>-le Is number1 less than or equal to number2 ? </li></ul></ul><ul><li>E.g </li></ul><ul><ul><li>if test $var –lt 10 then echo “Something” fi </li></ul></ul>
  25. 25. UNIX S hell Programming <ul><li>String Test Operators </li></ul><ul><ul><li>string1 = string2 Does string1 match string2? </li></ul></ul><ul><ul><li>string1 != string2 Does string1 not match string2? </li></ul></ul><ul><ul><li>-n string Does string contain characters? </li></ul></ul><ul><ul><li>-z string Is string an empty string? </li></ul></ul><ul><li>E.g. </li></ul><ul><ul><li>if test $str1 = $str2 then echo “Something” fi </li></ul></ul><ul><ul><li>Think if no space on both sides of an operator </li></ul></ul>
  26. 26. UNIX S hell Programming <ul><li>File Test Operators </li></ul><ul><ul><li>-s file True if File size is greater than 0 </li></ul></ul><ul><ul><li>-f file True if File exists & not a directory </li></ul></ul><ul><ul><li>-d file True if file exists & is a directory file. </li></ul></ul><ul><ul><li>-r file True if file exists & you have read permission on it. </li></ul></ul><ul><ul><li>-w file True if file exists & you have write permission on it. </li></ul></ul><ul><ul><li>-x file True if file exists & you have execute permission on it. </li></ul></ul><ul><li>E.g </li></ul><ul><ul><li>if [ –f $filename ] then echo “Something” fi </li></ul></ul><ul><ul><li>Think if no space on both sides of an option. </li></ul></ul>
  27. 27. UNIX S hell Programming <ul><li>The expr Command </li></ul><ul><ul><li>Arithmetic Operators </li></ul></ul><ul><ul><li>+ : Addition operator </li></ul></ul><ul><ul><li>- : Subtraction operator </li></ul></ul><ul><ul><li>/ : Division operator </li></ul></ul><ul><ul><li>* : Multiplication operator </li></ul></ul><ul><ul><li>% : Remainder operator </li></ul></ul><ul><li>Expressions </li></ul><ul><ul><li>Arithmetic expression a=`expr $a + $b` … think if no space both the sides of + & space both the sides of =. </li></ul></ul><ul><ul><li>Floating point arithmetic expression a=`expr $a + $b | bc ` </li></ul></ul><ul><ul><li>String expression path=$path1”/”$path2 </li></ul></ul>
  28. 28. UNIX S hell Programming <ul><li>While Loop </li></ul><ul><ul><li>while [ condition ] </li></ul></ul><ul><ul><li>do </li></ul></ul><ul><ul><li>command(s) </li></ul></ul><ul><ul><li>done </li></ul></ul><ul><li>For Loop </li></ul><ul><ul><li>for variable in list-of-value </li></ul></ul><ul><ul><li>do </li></ul></ul><ul><ul><li>command(s) </li></ul></ul><ul><ul><li>done </li></ul></ul>
  29. 29. AWK <ul><li>Introduction to awk. </li></ul><ul><ul><li>awk is a programming language designed to search for, match patterns, and perform actions on files. awk programs are generally quite small, and are interpreted. </li></ul></ul><ul><ul><li>awk scans input lines one after the other, searching each line to see if it matches a set of patterns or conditions specified in the awk program. For each pattern, an action is specified. The action is performed when the pattern matches that of the input line. </li></ul></ul><ul><ul><li>Sample awk command: awk ‘/pattern/’’{print $0}’ file1 </li></ul></ul>
  30. 30. My Learnings <ul><li>When I will go for Shell scripting. </li></ul><ul><ul><li>Automating my regular tasks. </li></ul></ul><ul><ul><li>Customizing my work environment. </li></ul></ul><ul><ul><li>Task is pretty simple </li></ul></ul><ul><li>When I will avoid Shell scripting . </li></ul><ul><ul><li>It needs interaction with multiple applications. </li></ul></ul><ul><ul><li>Problem is relatively complex involves more than one tool. </li></ul></ul><ul><ul><li>Lookups or finding data . </li></ul></ul>
  31. 31. <ul><li>Questions ? </li></ul>
  32. 32. Thank You