Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Unix shell scripting basics

1,287 views

Published on

Published in: Technology
  • Be the first to comment

Unix shell scripting basics

  1. 1. UNIX Shell-Scripting Basics
  2. 2. Agenda <ul><li>What is a shell? A shell script? </li></ul><ul><li>Introduction to bash </li></ul><ul><li>Running Commands </li></ul><ul><li>Applied Shell Programming </li></ul>
  3. 3. What is a shell? <ul><li>% </li></ul>▌
  4. 4. What is a shell? <ul><li>/bin/bash </li></ul>
  5. 5. What is a shell? <ul><li>#!/bin/bash </li></ul>
  6. 6. What is a shell? INPUT shell OUTPUT ERROR
  7. 7. What is a shell? <ul><li>Any Program </li></ul><ul><li>But there are a few popular shells… </li></ul>
  8. 8. Bourne Shells <ul><li>/bin/sh </li></ul><ul><li>/bin/bash “Bourne-Again Shell” </li></ul>Steve Bourne
  9. 9. Other Common Shells <ul><li>C Shell ( /bin/csh ) </li></ul><ul><li>Turbo C Shell ( /bin/tcsh ) </li></ul><ul><li>Korn Shell ( /bin/ksh ) </li></ul>
  10. 10. An aside: What do I mean by /bin ? <ul><li>C Shell ( /bin/csh ) </li></ul><ul><li>Turbo C Shell ( /bin/tcsh ) </li></ul><ul><li>Korn Shell ( /bin/ksh ) </li></ul>
  11. 11. An aside: What do I mean by /bin ? <ul><li>/bin, /usr/bin, /usr/local/bin </li></ul><ul><li>/sbin, /usr/sbin, /usr/local/sbin </li></ul><ul><li>/tmp </li></ul><ul><li>/dev </li></ul><ul><li>/home/borwicjh </li></ul>
  12. 12. What is a Shell Script? <ul><li>A Text File </li></ul><ul><li>With Instructions </li></ul><ul><li>Executable </li></ul>
  13. 13. What is a Shell Script? <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
  14. 14. What is a Shell Script? A Text File <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
  15. 15. An aside: Redirection <ul><li>cat > /tmp/myfile </li></ul><ul><li>cat >> /tmp/myfile </li></ul><ul><li>cat 2> /tmp/myerr </li></ul><ul><li>cat < /tmp/myinput </li></ul><ul><li>cat <<INPUT Some input INPUT </li></ul><ul><li>cat > /tmp/x 2>&1 </li></ul>0 1 2 INPUT env OUTPUT ERROR
  16. 16. What is a Shell Script? How To Run <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
  17. 17. What is a Shell Script? What To Do <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
  18. 18. What is a Shell Script? Executable <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
  19. 19. What is a Shell Script? Running it <ul><li>% cat > hello.sh <<MY_PROGRAM </li></ul><ul><li>#!/bin/sh </li></ul><ul><li>echo ‘Hello, world’ </li></ul><ul><li>MY_PROGRAM </li></ul><ul><li>% chmod +x hello.sh </li></ul><ul><li>% ./hello.sh </li></ul><ul><li>Hello, world </li></ul>
  20. 20. Finding the program: PATH <ul><li>% ./hello.sh </li></ul><ul><li>echo vs. /usr/bin/echo </li></ul><ul><li>% echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin </li></ul><ul><li>% which echo /usr/bin/echo </li></ul>
  21. 21. Variables and the Environment <ul><li>% hello.sh </li></ul><ul><li>bash: hello.sh: Command not found </li></ul><ul><li>% PATH=“$PATH:.” </li></ul><ul><li>% hello.sh </li></ul><ul><li>Hello, world </li></ul>
  22. 22. An aside: Quoting <ul><li>% echo ‘ $USER ’ </li></ul><ul><li>$USER </li></ul><ul><li>% echo “ $USER ” </li></ul><ul><li>borwicjh </li></ul><ul><li>% echo “ ” ” </li></ul><ul><li>” </li></ul><ul><li>% echo “deacnet sct” </li></ul><ul><li>deacnetsct </li></ul><ul><li>% echo ‘ ” ’ </li></ul><ul><li>” </li></ul>
  23. 23. Variables and the Environment <ul><li>% env </li></ul><ul><li>[…variables passed to sub-programs…] </li></ul><ul><li>% NEW_VAR=“Yes” </li></ul><ul><li>% echo $NEW_VAR </li></ul><ul><li>Yes </li></ul><ul><li>% env </li></ul><ul><li>[…PATH but not NEW_VAR…] </li></ul><ul><li>% export NEW_VAR </li></ul><ul><li>% env </li></ul><ul><li>[…PATH and NEW_VAR…] </li></ul>
  24. 24. Welcome to Shell Scripting! Shebang! The Environment PATH Input, Output, and Error chmod
  25. 25. How to Learn <ul><li>man </li></ul><ul><ul><li>man bash </li></ul></ul><ul><ul><li>man cat </li></ul></ul><ul><ul><li>man man </li></ul></ul><ul><li>man –k </li></ul><ul><ul><li>man –k manual </li></ul></ul><ul><li>Learning the Bash Shell , 2 nd Ed. </li></ul><ul><li>“ Bash Reference” Cards </li></ul><ul><li>http://www.tldp.org/LDP/abs/html/ </li></ul>
  26. 26. Introduction to bash
  27. 27. Continuing Lines: <ul><li>% echo This </li></ul><ul><li>Is </li></ul><ul><li>A </li></ul><ul><li>Very </li></ul><ul><li>Long </li></ul><ul><li>Command Line </li></ul><ul><li>This Is A Very Long Command Line </li></ul><ul><li>% </li></ul>
  28. 28. Exit Status <ul><li>$? </li></ul><ul><li>0 is True </li></ul><ul><li>% ls /does/not/exist </li></ul><ul><li>% echo $? </li></ul><ul><li>1 </li></ul><ul><li>% echo $? </li></ul><ul><li>0 </li></ul>
  29. 29. Exit Status: exit <ul><li>% cat > test.sh <<_TEST_ </li></ul><ul><li>exit 3 </li></ul><ul><li>_TEST_ </li></ul><ul><li>% chmod +x test.sh </li></ul><ul><li>% ./test.sh </li></ul><ul><li>% echo $? </li></ul><ul><li>3 </li></ul>
  30. 30. Logic: test <ul><li>% test 1 -lt 10 </li></ul><ul><li>% echo $? </li></ul><ul><li>0 </li></ul><ul><li>% test 1 == 10 </li></ul><ul><li>% echo $? </li></ul><ul><li>1 </li></ul>
  31. 31. Logic: test <ul><li>test </li></ul><ul><li>[ ] </li></ul><ul><ul><li>[ 1 –lt 10 ] </li></ul></ul><ul><li>[[ ]] </li></ul><ul><ul><li>[[ “this string” =~ “this” ]] </li></ul></ul><ul><li>(( )) </li></ul><ul><ul><li>(( 1 < 10 )) </li></ul></ul>
  32. 32. Logic: test <ul><li>[ -f /etc/passwd ] </li></ul><ul><li>[ ! –f /etc/passwd ] </li></ul><ul><li>[ -f /etc/passwd –a –f /etc/shadow ] </li></ul><ul><li>[ -f /etc/passwd –o –f /etc/shadow ] </li></ul>
  33. 33. An aside: $(( )) for Math <ul><li>% echo $(( 1 + 2 )) </li></ul><ul><li>3 </li></ul><ul><li>% echo $(( 2 * 3 )) </li></ul><ul><li>6 </li></ul><ul><li>% echo $(( 1 / 3 )) </li></ul><ul><li>0 </li></ul>
  34. 34. Logic: if <ul><li>if something </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li># “elif” a contraction of “else if”: </li></ul><ul><li>elif something-else </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li>else </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li>fi </li></ul>
  35. 35. Logic: if <ul><li>if [ $USER –eq “borwicjh” ] </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li># “elif” a contraction of “else if”: </li></ul><ul><li>elif ls /etc/oratab </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li>else </li></ul><ul><li>then </li></ul><ul><li>: </li></ul><ul><li>fi </li></ul>
  36. 36. Logic: if <ul><li># see if a file exists </li></ul><ul><li>if [ -e /etc/passwd ] </li></ul><ul><li>then </li></ul><ul><li>echo “/etc/passwd exists” </li></ul><ul><li>else </li></ul><ul><li>echo “/etc/passwd not found!” </li></ul><ul><li>fi </li></ul>
  37. 37. Logic: for <ul><li>for i in 1 2 3 </li></ul><ul><li>do </li></ul><ul><li>echo $i </li></ul><ul><li>done </li></ul>
  38. 38. Logic: for <ul><li>for i in /* </li></ul><ul><li>do </li></ul><ul><li>echo “Listing $i:” </li></ul><ul><li>ls -l $i </li></ul><ul><li>read </li></ul><ul><li>done </li></ul>
  39. 39. Logic: for <ul><li>for i in /* </li></ul><ul><li>do </li></ul><ul><li>echo “Listing $i:” </li></ul><ul><li>ls -l $i </li></ul><ul><li>read </li></ul><ul><li>done </li></ul>
  40. 40. Logic: for <ul><li>for i in /* </li></ul><ul><li>do </li></ul><ul><li>echo “Listing $i:” </li></ul><ul><li>ls -l $i </li></ul><ul><li>read </li></ul><ul><li>done </li></ul>
  41. 41. Logic: C-style for <ul><li>for (( expr1 ; </li></ul><ul><li>expr2 ; </li></ul><ul><li>expr3 )) </li></ul><ul><li>do </li></ul><ul><li>list </li></ul><ul><li>done </li></ul>
  42. 42. Logic: C-style for <ul><li>LIMIT=10 </li></ul><ul><li>for (( a=1 ; </li></ul><ul><li>a<=LIMIT ; </li></ul><ul><li>a++ )) </li></ul><ul><li>do </li></ul><ul><li>echo –n “$a ” </li></ul><ul><li>done </li></ul>
  43. 43. Logic: while <ul><li>while something </li></ul><ul><li>do </li></ul><ul><li>: </li></ul><ul><li>done </li></ul>
  44. 44. Logic: while <ul><li>a=0; LIMIT=10 </li></ul><ul><li>while [ &quot;$a&quot; -lt &quot;$LIMIT&quot; ] </li></ul><ul><li>do </li></ul><ul><li>echo -n &quot;$a ” </li></ul><ul><li>a=$(( a + 1 )) </li></ul><ul><li>done </li></ul>
  45. 45. Counters <ul><li>COUNTER=0 </li></ul><ul><li>while [ -e “$FILE.COUNTER” ] </li></ul><ul><li>do </li></ul><ul><li>COUNTER=$(( COUNTER + 1)) </li></ul><ul><li>done </li></ul><ul><li>Note: race condition </li></ul>
  46. 46. Reusing Code: “Sourcing” <ul><li>% cat > /path/to/my/passwords <<_PW_ </li></ul><ul><li>FTP_USER=“sct” </li></ul><ul><li>_PW_ </li></ul><ul><li>% echo $FTP_USER </li></ul><ul><li>% . /path/to/my/passwords </li></ul><ul><li>% echo $FTP_USER </li></ul><ul><li>sct </li></ul><ul><li>% </li></ul>
  47. 47. Variable Manipulation <ul><li>% FILEPATH=/path/to/my/output.lis </li></ul><ul><li>% echo $FILEPATH </li></ul><ul><li>/path/to/my/output.lis </li></ul><ul><li>% echo ${FILEPATH %.lis } </li></ul><ul><li>/path/to/my/output </li></ul><ul><li>% echo ${FILEPATH #*/ } </li></ul><ul><li>path/to/my/output.lis </li></ul><ul><li>% echo ${FILEPATH ##*/ } </li></ul><ul><li>output.lis </li></ul>
  48. 48. <ul><li>It takes a long time to become a bash guru… </li></ul>
  49. 49. Running Programs
  50. 50. Reasons for Running Programs <ul><li>Check Return Code </li></ul><ul><ul><li>$? </li></ul></ul><ul><li>Get Job Output </li></ul><ul><ul><li>OUTPUT=`echo “Hello”` </li></ul></ul><ul><ul><li>OUTPUT=$(echo “Hello”) </li></ul></ul><ul><li>Send Output Somewhere </li></ul><ul><ul><li>Redirection: < , > </li></ul></ul><ul><ul><li>Pipes </li></ul></ul>
  51. 51. Pipes <ul><li>Lots of Little Tools </li></ul><ul><li>echo “Hello” | </li></ul><ul><li>wc -c </li></ul>INPUT echo OUTPUT ERROR 0 1 2 INPUT wc OUTPUT ERROR 0 1 2 A Pipe!
  52. 52. Email Notification <ul><li>% echo “Message” | </li></ul><ul><li>mail –s “Here’s your message” </li></ul><ul><li>[email_address] </li></ul>
  53. 53. Dates <ul><li>% DATESTRING=`date +%Y%m%d` </li></ul><ul><li>% echo $DATESTRING </li></ul><ul><li>20060125 </li></ul><ul><li>% man date </li></ul>
  54. 54. FTP the Hard Way <ul><li>ftp –n –u server.wfu.edu <<_FTP_ </li></ul><ul><li>user username password </li></ul><ul><li>put FILE </li></ul><ul><li>_FTP_ </li></ul>
  55. 55. FTP with wget <ul><li>wget ftp://user:pass@server.wfu.edu/file </li></ul><ul><li>wget –r ftp://user:pass@server.wfu.edu/dir/ </li></ul>
  56. 56. FTP with curl <ul><li>curl –T upload-file </li></ul><ul><li>-u username:password </li></ul><ul><li>ftp://server.wfu.edu/dir/file </li></ul>
  57. 57. Searching: grep <ul><li>% grep rayra /etc/passwd </li></ul><ul><li>% grep –r rayra /etc </li></ul><ul><li>% grep –r RAYRA /etc </li></ul><ul><li>% grep –ri RAYRA /etc </li></ul><ul><li>% grep –rli rayra /etc </li></ul>
  58. 58. Searching: find <ul><li>% find /home/borwicjh </li></ul><ul><li>-name ‘*.lis’ </li></ul><ul><li>[all files matching *.lis] </li></ul><ul><li>% find /home/borwicjh </li></ul><ul><li>-mtime -1 –name ‘*.lis’ </li></ul><ul><li>[*.lis, if modified within 24h] </li></ul><ul><li>% man find </li></ul>
  59. 59. Searching: locate <ul><li>% locate .lis </li></ul><ul><li>[files with .lis in path] </li></ul><ul><li>% locate log </li></ul><ul><li>[also finds “/var/log/messages”] </li></ul>
  60. 60. Applied Shell Programming
  61. 61. Make Your Life Easier <ul><li>TAB completion </li></ul><ul><li>Control+R </li></ul><ul><li>history </li></ul><ul><li>cd - </li></ul><ul><li>Study a UNIX Editor </li></ul>
  62. 62. pushd/popd <ul><li>% cd /tmp </li></ul><ul><li>% pushd /var/log </li></ul><ul><li>/var/log /tmp </li></ul><ul><li>% cd .. </li></ul><ul><li>% pwd </li></ul><ul><li>/var </li></ul><ul><li>% popd </li></ul><ul><li>/tmp </li></ul>
  63. 63. Monitoring processes <ul><li>ps </li></ul><ul><li>ps –ef </li></ul><ul><li>ps –u oracle </li></ul><ul><li>ps –C sshd </li></ul><ul><li>man ps </li></ul>
  64. 64. “DOS” Mode Files <ul><li>#!/usr/bin/bash^M </li></ul><ul><li>FTP transfer in ASCII, or </li></ul><ul><li>dos2unix infile > outfile </li></ul>
  65. 65. sqlplus <ul><li>JOB=“ZZZTEST” </li></ul><ul><li>PARAMS=“ZZZTEST_PARAMS” </li></ul><ul><li>PARAMS_USER=“BORWICJH” </li></ul><ul><li>sqlplus $BANNER_USER/$BANNER_PW << _EOF_ </li></ul><ul><li>set serveroutput on </li></ul><ul><li>set sqlprompt &quot;&quot; </li></ul><ul><li>EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); </li></ul><ul><li>_EOF_ </li></ul>
  66. 66. sqlplus <ul><li>sqlplus $USER/$PASS @$FILE_SQL </li></ul><ul><li>$ARG1 $ARG2 $ARG3 </li></ul><ul><li>if [ $? –ne 0 ] </li></ul><ul><li>then </li></ul><ul><li>exit 1 </li></ul><ul><li>fi </li></ul><ul><li>if [ -e /file/sql/should/create ] </li></ul><ul><li>then </li></ul><ul><li>[…use SQL-created file…] </li></ul><ul><li>fi </li></ul><ul><li>Ask Amy Lamy!  </li></ul>
  67. 67. Passing Arguments <ul><li>% cat > test.sh <<_TEST_ </li></ul><ul><li>echo “Your name is $1 $2 ” </li></ul><ul><li>_TEST_ </li></ul><ul><li>% chmod +x test.sh </li></ul><ul><li>% ./test.sh John Borwick ignore-this </li></ul><ul><li>Your name is John Borwick </li></ul>
  68. 68. INB Job Submission Template <ul><li>$1 : user ID </li></ul><ul><li>$2 : password </li></ul><ul><li>$3 : one-up number </li></ul><ul><li>$4 : process name </li></ul><ul><li>$5 : printer name </li></ul><ul><li>% /path/to/your/script $UI $PW </li></ul><ul><li>$ONE_UP $JOB $PRNT </li></ul>
  69. 69. Scheduling Jobs <ul><li>% crontab -l </li></ul><ul><li>0 0 * * * daily-midnight-job.sh </li></ul><ul><li>0 * * * * hourly-job.sh </li></ul><ul><li>* * * * * every-minute.sh </li></ul><ul><li>0 1 * * 0 1AM-on-sunday.sh </li></ul><ul><li>% EDITOR=vi crontab –e </li></ul><ul><li>% man 5 crontab </li></ul>
  70. 70. It's Over!
  71. 71. Other Questions? <ul><li>Shells and Shell Scripts </li></ul><ul><li>bash </li></ul><ul><li>Running Commands </li></ul><ul><li>bash and Banner in Practice </li></ul>

×