Unix Shell Scripting Basics

27,413 views

Published on

Published in: Technology, News & Politics
5 Comments
63 Likes
Statistics
Notes
  • Shell Scripting: How to Automate Command Line Tasks Using Bash Scripting and Shell Programming --- http://amzn.to/1WzJYZm
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Linux Command Line and Shell Scripting Bible --- http://amzn.to/1LyurZd
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Classic Shell Scripting --- http://amzn.to/1R98HlK
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • http://www.dbmanagement.info/Tutorials/Unix.htm Unix Video Training Tutorial DVDs (2 DVDs) Sale Price 12.95 More than 5.9 GB of Video files
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Nice content
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
27,413
On SlideShare
0
From Embeds
0
Number of Embeds
735
Actions
Shares
0
Downloads
0
Comments
5
Likes
63
Embeds 0
No embeds

No notes for slide
  • Unix Shell Scripting Basics

    1. 1. UNIX Shell-Scripting Basics
    2. 2. Agenda <ul><li>What is a shell? A shell script? </li></ul><ul><li>Introduction to bash </li></ul><ul><li>Running Commands </li></ul><ul><li>Applied Shell Programming </li></ul>
    3. 3. What is a shell? <ul><li>% </li></ul>▌
    4. 4. What is a shell? <ul><li>/bin/bash </li></ul>
    5. 5. What is a shell? <ul><li>#!/bin/bash </li></ul>
    6. 6. What is a shell? INPUT shell OUTPUT ERROR
    7. 7. What is a shell? <ul><li>Any Program </li></ul><ul><li>But there are a few popular shells… </li></ul>
    8. 8. Bourne Shells <ul><li>/bin/sh </li></ul><ul><li>/bin/bash “Bourne-Again Shell” </li></ul>Steve Bourne
    9. 9. Other Common Shells <ul><li>C Shell ( /bin/csh ) </li></ul><ul><li>Turbo C Shell ( /bin/tcsh ) </li></ul><ul><li>Korn Shell ( /bin/ksh ) </li></ul>
    10. 10. An aside: What do I mean by /bin ? <ul><li>C Shell ( /bin/csh ) </li></ul><ul><li>Turbo C Shell ( /bin/tcsh ) </li></ul><ul><li>Korn Shell ( /bin/ksh ) </li></ul>
    11. 11. An aside: What do I mean by /bin ? <ul><li>/bin, /usr/bin, /usr/local/bin </li></ul><ul><li>/sbin, /usr/sbin, /usr/local/sbin </li></ul><ul><li>/tmp </li></ul><ul><li>/dev </li></ul><ul><li>/home/borwicjh </li></ul>
    12. 12. What is a Shell Script? <ul><li>A Text File </li></ul><ul><li>With Instructions </li></ul><ul><li>Executable </li></ul>
    13. 13. What is a Shell Script? <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
    14. 14. What is a Shell Script? A Text File <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
    15. 15. An aside: Redirection <ul><li>cat > /tmp/myfile </li></ul><ul><li>cat >> /tmp/myfile </li></ul><ul><li>cat 2> /tmp/myerr </li></ul><ul><li>cat < /tmp/myinput </li></ul><ul><li>cat <<INPUT Some input INPUT </li></ul><ul><li>cat > /tmp/x 2>&1 </li></ul>0 1 2 INPUT env OUTPUT ERROR
    16. 16. What is a Shell Script? How To Run <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
    17. 17. What is a Shell Script? What To Do <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
    18. 18. What is a Shell Script? Executable <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
    19. 19. What is a Shell Script? Running it <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
    20. 20. Finding the program: PATH <ul><li>% ./hello.sh </li></ul><ul><li>echo vs. /usr/bin/echo </li></ul><ul><li>% echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin </li></ul><ul><li>% which echo /usr/bin/echo </li></ul>
    21. 21. Variables and the Environment <ul><li>% hello.sh </li></ul><ul><li>bash: hello.sh: Command not found </li></ul><ul><li>% PATH=“$PATH:.” </li></ul><ul><li>% hello.sh </li></ul><ul><li>Hello, world </li></ul>
    22. 22. An aside: Quoting <ul><li>% echo ‘ $USER ’ </li></ul><ul><li>$USER </li></ul><ul><li>% echo “ $USER ” </li></ul><ul><li>borwicjh </li></ul><ul><li>% echo “ ” ” </li></ul><ul><li>” </li></ul><ul><li>% echo “deacnet sct” </li></ul><ul><li>deacnetsct </li></ul><ul><li>% echo ‘ ” ’ </li></ul><ul><li>” </li></ul>
    23. 23. Variables and the Environment <ul><li>% env </li></ul><ul><li>[…variables passed to sub-programs…] </li></ul><ul><li>% NEW_VAR=“Yes” </li></ul><ul><li>% echo $NEW_VAR </li></ul><ul><li>Yes </li></ul><ul><li>% env </li></ul><ul><li>[…PATH but not NEW_VAR…] </li></ul><ul><li>% export NEW_VAR </li></ul><ul><li>% env </li></ul><ul><li>[…PATH and NEW_VAR…] </li></ul>
    24. 24. Welcome to Shell Scripting! Shebang! The Environment PATH Input, Output, and Error chmod
    25. 25. How to Learn <ul><li>man </li></ul><ul><ul><li>man bash </li></ul></ul><ul><ul><li>man cat </li></ul></ul><ul><ul><li>man man </li></ul></ul><ul><li>man –k </li></ul><ul><ul><li>man –k manual </li></ul></ul><ul><li>Learning the Bash Shell , 2 nd Ed. </li></ul><ul><li>“ Bash Reference” Cards </li></ul><ul><li>http://www.tldp.org/LDP/abs/html/ </li></ul>
    26. 26. Introduction to bash
    27. 27. Continuing Lines: <ul><li>% echo This </li></ul><ul><li>Is </li></ul><ul><li>A </li></ul><ul><li>Very </li></ul><ul><li>Long </li></ul><ul><li>Command Line </li></ul><ul><li>This Is A Very Long Command Line </li></ul><ul><li>% </li></ul>
    28. 28. Exit Status <ul><li>$? </li></ul><ul><li>0 is True </li></ul><ul><li>% ls /does/not/exist </li></ul><ul><li>% echo $? </li></ul><ul><li>1 </li></ul><ul><li>% echo $? </li></ul><ul><li>0 </li></ul>
    29. 29. Exit Status: exit <ul><li>% cat > test.sh <<_TEST_ </li></ul><ul><li>exit 3 </li></ul><ul><li>_TEST_ </li></ul><ul><li>% chmod +x test.sh </li></ul><ul><li>% ./test.sh </li></ul><ul><li>% echo $? </li></ul><ul><li>3 </li></ul>
    30. 30. Logic: test <ul><li>% test 1 -lt 10 </li></ul><ul><li>% echo $? </li></ul><ul><li>0 </li></ul><ul><li>% test 1 == 10 </li></ul><ul><li>% echo $? </li></ul><ul><li>1 </li></ul>
    31. 31. Logic: test <ul><li>test </li></ul><ul><li>[ ] </li></ul><ul><ul><li>[ 1 –lt 10 ] </li></ul></ul><ul><li>[[ ]] </li></ul><ul><ul><li>[[ “this string” =~ “this” ]] </li></ul></ul><ul><li>(( )) </li></ul><ul><ul><li>(( 1 < 10 )) </li></ul></ul>
    32. 32. Logic: test <ul><li>[ -f /etc/passwd ] </li></ul><ul><li>[ ! –f /etc/passwd ] </li></ul><ul><li>[ -f /etc/passwd –a –f /etc/shadow ] </li></ul><ul><li>[ -f /etc/passwd –o –f /etc/shadow ] </li></ul>
    33. 33. An aside: $(( )) for Math <ul><li>% echo $(( 1 + 2 )) </li></ul><ul><li>3 </li></ul><ul><li>% echo $(( 2 * 3 )) </li></ul><ul><li>6 </li></ul><ul><li>% echo $(( 1 / 3 )) </li></ul><ul><li>0 </li></ul>
    34. 34. Logic: if <ul><li>if something </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li># “elif” a contraction of “else if”: </li></ul><ul><li>elif something-else </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li>else </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li>fi </li></ul>
    35. 35. Logic: if <ul><li>if [ $USER –eq “borwicjh” ] </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li># “elif” a contraction of “else if”: </li></ul><ul><li>elif ls /etc/oratab </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li>else </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li>fi </li></ul>
    36. 36. Logic: if <ul><li># see if a file exists </li></ul><ul><li>if [ -e /etc/passwd ] </li></ul><ul><li>then </li></ul><ul><li>echo “/etc/passwd exists” </li></ul><ul><li>else </li></ul><ul><li>echo “/etc/passwd not found!” </li></ul><ul><li>fi </li></ul>
    37. 37. Logic: for <ul><li>for i in 1 2 3 </li></ul><ul><li>do </li></ul><ul><li>echo $i </li></ul><ul><li>done </li></ul>
    38. 38. Logic: for <ul><li>for i in /* </li></ul><ul><li>do </li></ul><ul><li>echo “Listing $i:” </li></ul><ul><li>ls -l $i </li></ul><ul><li>read </li></ul><ul><li>done </li></ul>
    39. 39. Logic: for <ul><li>for i in /* </li></ul><ul><li>do </li></ul><ul><li>echo “Listing $i:” </li></ul><ul><li>ls -l $i </li></ul><ul><li>read </li></ul><ul><li>done </li></ul>
    40. 40. Logic: for <ul><li>for i in /* </li></ul><ul><li>do </li></ul><ul><li>echo “Listing $i:” </li></ul><ul><li>ls -l $i </li></ul><ul><li>read </li></ul><ul><li>done </li></ul>
    41. 41. Logic: C-style for <ul><li>for (( expr1 ; </li></ul><ul><li>expr2 ; </li></ul><ul><li>expr3 )) </li></ul><ul><li>do </li></ul><ul><li>list </li></ul><ul><li>done </li></ul>
    42. 42. Logic: C-style for <ul><li>LIMIT=10 </li></ul><ul><li>for (( a=1 ; </li></ul><ul><li>a<=LIMIT ; </li></ul><ul><li>a++ )) </li></ul><ul><li>do </li></ul><ul><li>echo –n “$a ” </li></ul><ul><li>done </li></ul>
    43. 43. Logic: while <ul><li>while something </li></ul><ul><li>do </li></ul><ul><li>: </li></ul><ul><li>done </li></ul>
    44. 44. Logic: while <ul><li>a=0; LIMIT=10 </li></ul><ul><li>while [ &quot;$a&quot; -lt &quot;$LIMIT&quot; ] </li></ul><ul><li>do </li></ul><ul><li>echo -n &quot;$a ” </li></ul><ul><li>a=$(( a + 1 )) </li></ul><ul><li>done </li></ul>
    45. 45. Counters <ul><li>COUNTER=0 </li></ul><ul><li>while [ -e “$FILE.COUNTER” ] </li></ul><ul><li>do </li></ul><ul><li>COUNTER=$(( COUNTER + 1)) </li></ul><ul><li>done </li></ul><ul><li>Note: race condition </li></ul>
    46. 46. Reusing Code: “Sourcing” <ul><li>% cat > /path/to/my/passwords <<_PW_ </li></ul><ul><li>FTP_USER=“sct” </li></ul><ul><li>_PW_ </li></ul><ul><li>% echo $FTP_USER </li></ul><ul><li>% . /path/to/my/passwords </li></ul><ul><li>% echo $FTP_USER </li></ul><ul><li>sct </li></ul><ul><li>% </li></ul>
    47. 47. Variable Manipulation <ul><li>% FILEPATH=/path/to/my/output.lis </li></ul><ul><li>% echo $FILEPATH </li></ul><ul><li>/path/to/my/output.lis </li></ul><ul><li>% echo ${FILEPATH %.lis } </li></ul><ul><li>/path/to/my/output </li></ul><ul><li>% echo ${FILEPATH #*/ } </li></ul><ul><li>path/to/my/output.lis </li></ul><ul><li>% echo ${FILEPATH ##*/ } </li></ul><ul><li>output.lis </li></ul>
    48. 48. <ul><li>It takes a long time to become a bash guru… </li></ul>
    49. 49. Running Programs
    50. 50. Reasons for Running Programs <ul><li>Check Return Code </li></ul><ul><ul><li>$? </li></ul></ul><ul><li>Get Job Output </li></ul><ul><ul><li>OUTPUT=`echo “Hello”` </li></ul></ul><ul><ul><li>OUTPUT=$(echo “Hello”) </li></ul></ul><ul><li>Send Output Somewhere </li></ul><ul><ul><li>Redirection: < , > </li></ul></ul><ul><ul><li>Pipes </li></ul></ul>
    51. 51. Pipes <ul><li>Lots of Little Tools </li></ul><ul><li>echo “Hello” | </li></ul><ul><li>wc -c </li></ul>INPUT echo OUTPUT ERROR 0 1 2 INPUT wc OUTPUT ERROR 0 1 2 A Pipe!
    52. 52. Email Notification <ul><li>% echo “Message” | </li></ul><ul><li>mail –s “Here’s your message” </li></ul><ul><li>[email_address] </li></ul>
    53. 53. Dates <ul><li>% DATESTRING=`date +%Y%m%d` </li></ul><ul><li>% echo $DATESTRING </li></ul><ul><li>20060125 </li></ul><ul><li>% man date </li></ul>
    54. 54. FTP the Hard Way <ul><li>ftp –n –u server.wfu.edu <<_FTP_ </li></ul><ul><li>user username password </li></ul><ul><li>put FILE </li></ul><ul><li>_FTP_ </li></ul>
    55. 55. FTP with wget <ul><li>wget ftp://user:pass@server.wfu.edu/file </li></ul><ul><li>wget –r ftp://user:pass@server.wfu.edu/dir/ </li></ul>
    56. 56. FTP with curl <ul><li>curl –T upload-file </li></ul><ul><li>-u username:password </li></ul><ul><li>ftp://server.wfu.edu/dir/file </li></ul>
    57. 57. Searching: grep <ul><li>% grep rayra /etc/passwd </li></ul><ul><li>% grep –r rayra /etc </li></ul><ul><li>% grep –r RAYRA /etc </li></ul><ul><li>% grep –ri RAYRA /etc </li></ul><ul><li>% grep –rli rayra /etc </li></ul>
    58. 58. Searching: find <ul><li>% find /home/borwicjh </li></ul><ul><li>-name ‘*.lis’ </li></ul><ul><li>[all files matching *.lis] </li></ul><ul><li>% find /home/borwicjh </li></ul><ul><li>-mtime -1 –name ‘*.lis’ </li></ul><ul><li>[*.lis, if modified within 24h] </li></ul><ul><li>% man find </li></ul>
    59. 59. Searching: locate <ul><li>% locate .lis </li></ul><ul><li>[files with .lis in path] </li></ul><ul><li>% locate log </li></ul><ul><li>[also finds “/var/log/messages”] </li></ul>
    60. 60. Applied Shell Programming
    61. 61. Make Your Life Easier <ul><li>TAB completion </li></ul><ul><li>Control+R </li></ul><ul><li>history </li></ul><ul><li>cd - </li></ul><ul><li>Study a UNIX Editor </li></ul>
    62. 62. pushd/popd <ul><li>% cd /tmp </li></ul><ul><li>% pushd /var/log </li></ul><ul><li>/var/log /tmp </li></ul><ul><li>% cd .. </li></ul><ul><li>% pwd </li></ul><ul><li>/var </li></ul><ul><li>% popd </li></ul><ul><li>/tmp </li></ul>
    63. 63. Monitoring processes <ul><li>ps </li></ul><ul><li>ps –ef </li></ul><ul><li>ps –u oracle </li></ul><ul><li>ps –C sshd </li></ul><ul><li>man ps </li></ul>
    64. 64. “DOS” Mode Files <ul><li>#!/usr/bin/bash^M </li></ul><ul><li>FTP transfer in ASCII, or </li></ul><ul><li>dos2unix infile > outfile </li></ul>
    65. 65. sqlplus <ul><li>JOB=“ZZZTEST” </li></ul><ul><li>PARAMS=“ZZZTEST_PARAMS” </li></ul><ul><li>PARAMS_USER=“BORWICJH” </li></ul><ul><li>sqlplus $BANNER_USER/$BANNER_PW << _EOF_ </li></ul><ul><li>set serveroutput on </li></ul><ul><li>set sqlprompt &quot;&quot; </li></ul><ul><li>EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); </li></ul><ul><li>_EOF_ </li></ul>
    66. 66. sqlplus <ul><li>sqlplus $USER/$PASS @$FILE_SQL </li></ul><ul><li>$ARG1 $ARG2 $ARG3 </li></ul><ul><li>if [ $? –ne 0 ] </li></ul><ul><li>then </li></ul><ul><li>exit 1 </li></ul><ul><li>fi </li></ul><ul><li>if [ -e /file/sql/should/create ] </li></ul><ul><li>then </li></ul><ul><li>[…use SQL-created file…] </li></ul><ul><li>fi </li></ul><ul><li>Ask Amy Lamy!  </li></ul>
    67. 67. Passing Arguments <ul><li>% cat > test.sh <<_TEST_ </li></ul><ul><li>echo “Your name is $1 $2 ” </li></ul><ul><li>_TEST_ </li></ul><ul><li>% chmod +x test.sh </li></ul><ul><li>% ./test.sh John Borwick ignore-this </li></ul><ul><li>Your name is John Borwick </li></ul>
    68. 68. INB Job Submission Template <ul><li>$1 : user ID </li></ul><ul><li>$2 : password </li></ul><ul><li>$3 : one-up number </li></ul><ul><li>$4 : process name </li></ul><ul><li>$5 : printer name </li></ul><ul><li>% /path/to/your/script $UI $PW </li></ul><ul><li>$ONE_UP $JOB $PRNT </li></ul>
    69. 69. Scheduling Jobs <ul><li>% crontab -l </li></ul><ul><li>0 0 * * * daily-midnight-job.sh </li></ul><ul><li>0 * * * * hourly-job.sh </li></ul><ul><li>* * * * * every-minute.sh </li></ul><ul><li>0 1 * * 0 1AM-on-sunday.sh </li></ul><ul><li>% EDITOR=vi crontab –e </li></ul><ul><li>% man 5 crontab </li></ul>
    70. 70. It's Over!
    71. 71. Other Questions? <ul><li>Shells and Shell Scripts </li></ul><ul><li>bash </li></ul><ul><li>Running Commands </li></ul><ul><li>bash and Banner in Practice </li></ul>

    ×