Unix Shell Scripting
Instructor              Mustafa Qasim              Certified Ethical Hacker (CEH)  Oracle Certified Expert Solaris Securit...
Catch MeLinkedIn   pk.linkedin.com/in/mustafaqasimGoogle+    gplus.to/mustuTwitter    twitter.com/mustafaqasim
Introduction●    Name●    Job Description●    Professional Experience●    Expectations from Unix Shell Scripting course
Ideal ParticipantsHardworking              ?
Ideal Participants        ?Hardworking       ŊÖ
Ideal Participants     LaaazzZZzzyyy....
Lets Start...
Unix●    AT&T Employees at Bell Labs in 1969●    Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy,    Michae...
GNU●    GNU Not Unix●    Richard Stallman, 1983●    GNU General Public License (GPL)●    GNU Compiler Collection (GCC), th...
GNU/Linux●    In April 1991, Linus Torvalds, a 21-year-old    student at the University of Helsinki, Finland    started wo...
To DoMake yourself familiar with Unix, Unix Derivatives, GNU, Linux andsoftware license GPL and Free & Open Source Softwar...
To DoMake yourself familiar with Unix, Unix Derivatives, GNU, Linux andsoftware license GPL and Free & Open Source Softwar...
So, Where comes the Shell?
Shell?
Shell?
Compiler vs InterpretedComplete Program > Compiler > ExecutableProgram Fragment > Interpreter > Instant Output
Compiler vs Interpreted (Contd.)●    Compiled Languages    –   C/C++    –   Pascal    –   FORTRAN●    Interpreted Language...
Unix Shell●    First shell for Version 6 Unix was written by Ken Thompson    (Bell Labs) in 1971●    It was only Interacti...
Bourne Shell●    Created by Stephen Bourne at AT&T Bell Labs for V7 UNIX●    Goals    –   Interpreted Language    –   Scri...
Bourne Shell (Contd.)●    Bourne Shell Derivatives    –   Korn Shell (ksh)    –   Almquist shell (ash)    –   Bourne Again...
C Shell (csh)●    Bill Joy in 1978 for BSD●    Create a scripting language similar to C●    Prominent Feature    –   Comma...
Korn Shell (ksh)●    By David Korn●    Backward compatibility with Bourne Shell (sh)●    Derivative of Bourne Shell (sh)
Bourne Again Shell (Bash)●    GNU Project●    Superset of sh and ksh
POSIX●    Institute of Electrical and Electronics Engineering (IEEE)●    International Organization for Standardization (I...
Responsibilities of Shell1. Reading input and parsing the command line2. Evaluating special characters, such as wildcards ...
To Do●    Perform Login and Logout on a Unix shell●    Perform and understand the following commands    –   ls, dir, touch...
Unix Startup●    OS is booted and very first process is born    –   Init, PID 1●    OS authenticates the user and provides...
Parsing Commands●    Lexical Analysis    –   The process of breaking the line up into tokens is called lexical        anal...
Types of Commands1. Aliases2. Keywords3. Functions4. Built-in commands5. Executable programs
Shell Processes●    PID●    ps●    Linux: pstree●    Solaris: ptree
System Calls                    }            forkexit                         Kernel             exec   wait
fork System Call●    Parent/Child●    Inheritance of environment, open files, real and user    identifications, umask, cur...
fork System Call
wait System Call●    Waits for Childs PID & Exit Code●    Zombie Process●    Orphan Zombie Process
exec System Call●    exec call for executable●    Executable becomes the child
exit System Call●    sigchild signal to parent●    Exit code 0 - 255
To Do●    Perform and Understand the following commands    –   top, ps, pgrep, pstree/ptree, kill●    Learn to hunt a spec...
Environment and Inheritance●    UID, GID●    Ownership●    Permission●    Variables●    Working Directory
User & Group IDs$ iduid=502(mustu) gid=502(mustu)
File Creation Mask●    Umask         777 (Directory)         666 (File)       –022 (umask value)      –022 (umask value)  ...
Set Ownership & Permission●    chown●    chmod
Variables●    Local Variables●    Environment Variables    –   $env
File Descriptor●    0 stdin●    1 stdout●    2 stderr
I/O Redirection
I/O Redirection
I/O Redirection
Pipe
Signals
To Do●    Understand & Practice the following    –   Bash Environmental variables    –   Check your current environment va...
Lets write a Bash script ;-)
Writing Bash Script●    Shebang    –   #!/bin/bash●    Comments    –   #Getting input for value of age●    Make script exe...
Simple Bash Script#!/bin/bash# This is a commentecho “Khamoshi Ka Boycot”
Use Variable#!/bin/bashname=”Telenor”echo “$name Khamoshi Ka Boycot”
Pass Arguments./myscript ali 15 Lahore#!/bin/bashecho “Mr. $1 is $2 years old and belongs to $3”
Get Input#!/bin/bashecho "What is your name?"read unameecho ” Welcome $uname”
Arithmetic Operators   ==           Equal to   !=         Not equal to   >          Greater then   >=     Greater then, eq...
Logical Operators  &&          and  ||          or  !           not
Conditional Statementsif Construct:if commandthen       block of statementsfi
Conditional Statementsif/else Construct:if commandthen       block of statementselse       block of statementsfi
Conditional Statementsif/else if/else Construct:if commandthen       block of statementselif commandthen       block of st...
Iterative Statementsfor variable in word_listdo       block of statementsdone
Schedule Script●    Cron    –   crontan -l   #List Exiting Crons    –   crontab -e   #Edit Cron Scheduling File
REMEMBER●    Environment Variable Differ in Cron●    Use the MAIL parameter to receive cron errors●    Use the PATH parame...
Tip :)Your best buddyhttp://www.corntab.com/
Regular Expressions●    Pattern of characters used to match the same characters in a    search●    Enclosed by forward sla...
Regular ExpressionsHi Asad,Weve a upcoming training on Unix shell Programming later this week.As youve to deal with progra...
Regular Expressions:1,$s/programming/scripting/g
Regular Expression Metacharacters:1,$s/<[Pp]rogramming>/scripting/g1,$        First Line to Last Lines          Substitute...
Regular Expression MetacharactersMeta Character^FunctionBeginning of line anchorExample/^lahore/What It MatchedMatches all...
Regular Expression MetacharactersMeta Character$FunctionEnd of line anchorExample/lahore$/What It MatchedMatches all lines...
Regular Expression MetacharactersMeta Character.FunctionMatches one characterExample/la...e/What It MatchedMatches all sta...
Regular Expression MetacharactersMeta Character*FunctionMatches zero or more of the preceding charactersExample/ *lahore/W...
Regular Expression MetacharactersMeta Character[]FunctionMatches one in the setExample/[Ll]ahore/What It MatchedMatches li...
Regular Expression MetacharactersMeta Character[x-y]FunctionMatches one character within a range in the setExample/[A-Z]ah...
Regular Expression MetacharactersMeta Character[^]FunctionMatches one character not in the setExample/[^A-Z]ahore/What It ...
Regular Expression MetacharactersMeta CharacterFunctionUsed to escape a metacharacterExample/lahore./What It MatchedMatche...
Regular Expression MetacharactersMeta Character<FunctionBeginning-of-word anchorExample/<lahore/What It MatchedMatches lin...
Regular Expression Metacharacters                    AdditionalMeta Character>FunctionEnding-of-word anchorExample/lahore>...
Regular Expression Metacharacters                       AdditionalMeta Character(..)FunctionTags match characters to be us...
Regular Expression Metacharacters                       AdditionalMeta Character(..)FunctionTags match characters to be us...
Regular Expression Metacharacters                     AdditionalMeta Characterx{m} or x{m,} or x{m,n}FunctionRepetition of...
Sed●    Streamlined, non-interactive editor
How Sed Worksabc               abc           abcText File        Sed Buffer   Sed Output
Awk●    UNIX programming language used for manipulating data and    generating reports
How Awk Works                                                  $0Ahmad       33   LHRShakir      35   ISB                 ...
How Awk Works                                 Ahmad          33   LHR                                 Shakir         35   ...
To Do●    Practice Regular Expressions with grep, sed and awk.
Upcoming SlideShare
Loading in...5
×

Unix Shell Scripting

1,434

Published on

1 Comment
1 Like
Statistics
Notes
  • Thank you my friend, it was really helpful for newbies like me.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
1,434
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
94
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Unix Shell Scripting

  1. 1. Unix Shell Scripting
  2. 2. Instructor Mustafa Qasim Certified Ethical Hacker (CEH) Oracle Certified Expert Solaris Security AdministratorOracle Certified Professional Solaris System Administrator Oracle Certified Associate Solaris 10 Operating System Oracle Certified Associate MySQL 5.5/5 Cisco Certified Network Associate (CCNA)
  3. 3. Catch MeLinkedIn pk.linkedin.com/in/mustafaqasimGoogle+ gplus.to/mustuTwitter twitter.com/mustafaqasim
  4. 4. Introduction● Name● Job Description● Professional Experience● Expectations from Unix Shell Scripting course
  5. 5. Ideal ParticipantsHardworking ?
  6. 6. Ideal Participants ?Hardworking ŊÖ
  7. 7. Ideal Participants LaaazzZZzzyyy....
  8. 8. Lets Start...
  9. 9. Unix● AT&T Employees at Bell Labs in 1969● Ken Thompson, Dennis Ritchie, Brian Kernighan, Douglas McIlroy, Michael Lesk and Joe Ossanna● Unix Family OS – University of California, Berkeleys BSD – Solaris – HP-UX – AIX – Sequent and as well as Darwin
  10. 10. GNU● GNU Not Unix● Richard Stallman, 1983● GNU General Public License (GPL)● GNU Compiler Collection (GCC), the GNU C library (glibc), and GNU Core Utilities (coreutils),[1] but also the GNU Debugger (GDB), GNU Binary Utilities (binutils),[18] and the bash shell.● GNU Hurd Kernel● Free Software Foundation (FSF)
  11. 11. GNU/Linux● In April 1991, Linus Torvalds, a 21-year-old student at the University of Helsinki, Finland started working on some simple ideas for an operating system● Kernel vs OS Linus Torvalds
  12. 12. To DoMake yourself familiar with Unix, Unix Derivatives, GNU, Linux andsoftware license GPL and Free & Open Source Software (FOSS).http://en.wikipedia.org/wiki/Unixhttp://www.gnu.org/philosophy/philosophy.htmlhttp://en.wikipedia.org/wiki/GNUhttp://www.gnu.org/gnu/why-gnu-linux.htmlhttp://en.wikipedia.org/wiki/Linux
  13. 13. To DoMake yourself familiar with Unix, Unix Derivatives, GNU, Linux andsoftware license GPL and Free & Open Source Software (FOSS).http://en.wikipedia.org/wiki/Unixhttp://www.gnu.org/philosophy/philosophy.htmlhttp://en.wikipedia.org/wiki/GNUhttp://www.gnu.org/gnu/why-gnu-linux.htmlhttp://en.wikipedia.org/wiki/Linux
  14. 14. So, Where comes the Shell?
  15. 15. Shell?
  16. 16. Shell?
  17. 17. Compiler vs InterpretedComplete Program > Compiler > ExecutableProgram Fragment > Interpreter > Instant Output
  18. 18. Compiler vs Interpreted (Contd.)● Compiled Languages – C/C++ – Pascal – FORTRAN● Interpreted Languages – Unix Shell – Perl – Tcl/Tk
  19. 19. Unix Shell● First shell for Version 6 Unix was written by Ken Thompson (Bell Labs) in 1971● It was only Interactive Shell
  20. 20. Bourne Shell● Created by Stephen Bourne at AT&T Bell Labs for V7 UNIX● Goals – Interpreted Language – Scripting● New Features – Control Flows, Loops, Variables● Lacked Functions
  21. 21. Bourne Shell (Contd.)● Bourne Shell Derivatives – Korn Shell (ksh) – Almquist shell (ash) – Bourne Again Shell (Bash)
  22. 22. C Shell (csh)● Bill Joy in 1978 for BSD● Create a scripting language similar to C● Prominent Feature – Command History● Tenex Enhancements (tcsh) – Command completion – Command line editing
  23. 23. Korn Shell (ksh)● By David Korn● Backward compatibility with Bourne Shell (sh)● Derivative of Bourne Shell (sh)
  24. 24. Bourne Again Shell (Bash)● GNU Project● Superset of sh and ksh
  25. 25. POSIX● Institute of Electrical and Electronics Engineering (IEEE)● International Organization for Standardization (ISO)● POSIX Compliance● Bash, gawk
  26. 26. Responsibilities of Shell1. Reading input and parsing the command line2. Evaluating special characters, such as wildcards and the history character3. Setting up pipes, redirection, and background processing4. Handling signals5. Setting up programs for execution
  27. 27. To Do● Perform Login and Logout on a Unix shell● Perform and understand the following commands – ls, dir, touch, mkdir, cd, rm, mv, cat, less, more, which, pwd, man, info● Use vi editor to create and edit files● Understand Unix File System hierarchy especially the following directories – /bin, /sbin, /home, /root, /var
  28. 28. Unix Startup● OS is booted and very first process is born – Init, PID 1● OS authenticates the user and provides access to shell – /bin/login● Data Streams are set – Stdout, stderr, stdin● A process is forked to run Graphical Desktop
  29. 29. Parsing Commands● Lexical Analysis – The process of breaking the line up into tokens is called lexical analysis.● Tokens● Command? – Built-in command => Execute Internally – Executable command => Fork a Child Shell● Forking Child Shel
  30. 30. Types of Commands1. Aliases2. Keywords3. Functions4. Built-in commands5. Executable programs
  31. 31. Shell Processes● PID● ps● Linux: pstree● Solaris: ptree
  32. 32. System Calls } forkexit Kernel exec wait
  33. 33. fork System Call● Parent/Child● Inheritance of environment, open files, real and user identifications, umask, current working directory, and signals.
  34. 34. fork System Call
  35. 35. wait System Call● Waits for Childs PID & Exit Code● Zombie Process● Orphan Zombie Process
  36. 36. exec System Call● exec call for executable● Executable becomes the child
  37. 37. exit System Call● sigchild signal to parent● Exit code 0 - 255
  38. 38. To Do● Perform and Understand the following commands – top, ps, pgrep, pstree/ptree, kill● Learn to hunt a specific process and kill it● Learn to hunt a specific instance of a binary running multiple times simultaneous
  39. 39. Environment and Inheritance● UID, GID● Ownership● Permission● Variables● Working Directory
  40. 40. User & Group IDs$ iduid=502(mustu) gid=502(mustu)
  41. 41. File Creation Mask● Umask 777 (Directory) 666 (File) –022 (umask value) –022 (umask value) ------- --------- 755 644 drwxr-xr-x -rw-r--r--
  42. 42. Set Ownership & Permission● chown● chmod
  43. 43. Variables● Local Variables● Environment Variables – $env
  44. 44. File Descriptor● 0 stdin● 1 stdout● 2 stderr
  45. 45. I/O Redirection
  46. 46. I/O Redirection
  47. 47. I/O Redirection
  48. 48. Pipe
  49. 49. Signals
  50. 50. To Do● Understand & Practice the following – Bash Environmental variables – Check your current environment variables (hint: $env) – Check your users UID & GID – Check a files permission set for owner, group and others – Change a file/folder permission using both numerical and character format – Change a file/folder owner and group – Redirect stdout and stderr into a file – Pipe or Redirect output of programs
  51. 51. Lets write a Bash script ;-)
  52. 52. Writing Bash Script● Shebang – #!/bin/bash● Comments – #Getting input for value of age● Make script executable – Chmod +x myscript.sh● Execute Script – ./myscript.sh
  53. 53. Simple Bash Script#!/bin/bash# This is a commentecho “Khamoshi Ka Boycot”
  54. 54. Use Variable#!/bin/bashname=”Telenor”echo “$name Khamoshi Ka Boycot”
  55. 55. Pass Arguments./myscript ali 15 Lahore#!/bin/bashecho “Mr. $1 is $2 years old and belongs to $3”
  56. 56. Get Input#!/bin/bashecho "What is your name?"read unameecho ” Welcome $uname”
  57. 57. Arithmetic Operators == Equal to != Not equal to > Greater then >= Greater then, equal to < Less then <= Less then, equalt o
  58. 58. Logical Operators && and || or ! not
  59. 59. Conditional Statementsif Construct:if commandthen block of statementsfi
  60. 60. Conditional Statementsif/else Construct:if commandthen block of statementselse block of statementsfi
  61. 61. Conditional Statementsif/else if/else Construct:if commandthen block of statementselif commandthen block of statementselse block of statementsfi
  62. 62. Iterative Statementsfor variable in word_listdo block of statementsdone
  63. 63. Schedule Script● Cron – crontan -l #List Exiting Crons – crontab -e #Edit Cron Scheduling File
  64. 64. REMEMBER● Environment Variable Differ in Cron● Use the MAIL parameter to receive cron errors● Use the PATH parameter to set environment variables in cron● If your program needs to use neighbor files to run properly then in cron script first change directory (cd) to your programs path then run the program.
  65. 65. Tip :)Your best buddyhttp://www.corntab.com/
  66. 66. Regular Expressions● Pattern of characters used to match the same characters in a search● Enclosed by forward slash /telenor/
  67. 67. Regular ExpressionsHi Asad,Weve a upcoming training on Unix shell Programming later this week.As youve to deal with programming tasks frequently I would highlyrecommend you to participate in this training. It will improve yourprogramming skills and help you perform daily tasks efficiently.Thanks
  68. 68. Regular Expressions:1,$s/programming/scripting/g
  69. 69. Regular Expression Metacharacters:1,$s/<[Pp]rogramming>/scripting/g1,$ First Line to Last Lines Substitute<, > Beginning and End of Word[Pp] Match one of the word<[Pp]rogramming> Programming OR programmingg Globally
  70. 70. Regular Expression MetacharactersMeta Character^FunctionBeginning of line anchorExample/^lahore/What It MatchedMatches all lines beginning with lahore
  71. 71. Regular Expression MetacharactersMeta Character$FunctionEnd of line anchorExample/lahore$/What It MatchedMatches all lines ending with lahore
  72. 72. Regular Expression MetacharactersMeta Character.FunctionMatches one characterExample/la...e/What It MatchedMatches all starting with la, followed by three characters, followed by e
  73. 73. Regular Expression MetacharactersMeta Character*FunctionMatches zero or more of the preceding charactersExample/ *lahore/What It MatchedMatches lines with zero or more spaces, followed by the pattern lahore
  74. 74. Regular Expression MetacharactersMeta Character[]FunctionMatches one in the setExample/[Ll]ahore/What It MatchedMatches lines containing Lahore or lahore
  75. 75. Regular Expression MetacharactersMeta Character[x-y]FunctionMatches one character within a range in the setExample/[A-Z]ahore/What It MatchedMatches letters from A through Z followed by ahore
  76. 76. Regular Expression MetacharactersMeta Character[^]FunctionMatches one character not in the setExample/[^A-Z]ahore/What It MatchedMatches any character not in the range between A and Z
  77. 77. Regular Expression MetacharactersMeta CharacterFunctionUsed to escape a metacharacterExample/lahore./What It MatchedMatches lines containing lahore, followed by a literal period; Normally the period matches one of anycharacter
  78. 78. Regular Expression MetacharactersMeta Character<FunctionBeginning-of-word anchorExample/<lahore/What It MatchedMatches lines containing a word that begins with lahore (supported by vi and grep)
  79. 79. Regular Expression Metacharacters AdditionalMeta Character>FunctionEnding-of-word anchorExample/lahore>/What It MatchedMatches lines containing a word that ends with lahore (supported by vi and grep)
  80. 80. Regular Expression Metacharacters AdditionalMeta Character(..)FunctionTags match characters to be used laterExample/(love)able 1er/What It MatchedMay use up to nine tags, starting with the first tag at the leftmost part of the pattern. For example, the pattern loveis saved as tag 1, to be referenced later as 1. In this example, the search pattern consists of lovable followed by lover(supported by sed, vi, and grep)
  81. 81. Regular Expression Metacharacters AdditionalMeta Character(..)FunctionTags match characters to be used laterExample/(love)able 1er/What It MatchedMay use up to nine tags, starting with the first tag at the leftmost part of the pattern. For example, the pattern loveis saved as tag 1, to be referenced later as 1. In this example, the search pattern consists of lovable followed by lover(supported by sed, vi, and grep)
  82. 82. Regular Expression Metacharacters AdditionalMeta Characterx{m} or x{m,} or x{m,n}FunctionRepetition of character x, m times, at least m times, at least m and not more than n timesExampleo{5,10}What It MatchedMatches if line contains between 5 and 10 consecutive occurrences of the letter o (supported by vi and grep)
  83. 83. Sed● Streamlined, non-interactive editor
  84. 84. How Sed Worksabc abc abcText File Sed Buffer Sed Output
  85. 85. Awk● UNIX programming language used for manipulating data and generating reports
  86. 86. How Awk Works $0Ahmad 33 LHRShakir 35 ISB Ahmad 33 LHRQasim 28 KHI Awk takes a line and put in an internal variable $0 Text File $1 $2 $3 Ahmad 33 LHR Line is broken into fields separated by spaces and stored into internal numbered variables starting from $1
  87. 87. How Awk Works Ahmad 33 LHR Shakir 35 ISB Qasim 28 KHI data.txt#awk {print $1, $3} data.txtAhmad LHRShakir ISBQasim KHI
  88. 88. To Do● Practice Regular Expressions with grep, sed and awk.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×