UNIX Shell-Scripting Basics
Agenda
What is a shell? A shell script?
Introduction to bash
Running Commands
Applied Shell Programming
What is a shell?
%▌
What is a shell?
/bin/bash
What is a shell?
#!/bin/bash
What is a shell?
INPUT
shell
OUTPUT ERROR
What is a shell?
Any Program
But there are a few popular shells…
Bourne Shells
 /bin/sh
 /bin/bash
“Bourne-Again Shell”
Steve Bourne
Other Common Shells
C Shell (/bin/csh)
Turbo C Shell (/bin/tcsh)
Korn Shell (/bin/ksh)
An aside: What do I mean by /bin ?
C Shell (/bin/csh)
Turbo C Shell (/bin/tcsh)
Korn Shell (/bin/ksh)
An aside: What do I mean by /bin ?
 /bin, /usr/bin, /usr/local/bin
 /sbin, /usr/sbin, /usr/local/sbin
 /tmp
 /dev
 /h...
What is a Shell Script?
A Text File
With Instructions
Executable
What is a Shell Script?
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo ‘Hello, world’
MY_PROGRAM
% chmod +x hello.sh
% ./hel...
What is a Shell Script? A Text File
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo ‘Hello, world’
MY_PROGRAM
% chmod +x hell...
An aside: Redirection
 cat > /tmp/myfile
 cat >> /tmp/myfile
 cat 2> /tmp/myerr
 cat < /tmp/myinput
 cat <<INPUT
Some...
What is a Shell Script? How To Run
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo ‘Hello, world’
MY_PROGRAM
% chmod +x hello...
What is a Shell Script? What To Do
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo ‘Hello, world’
MY_PROGRAM
% chmod +x hello...
What is a Shell Script? Executable
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo ‘Hello, world’
MY_PROGRAM
% chmod +x hello...
What is a Shell Script? Running it
% cat > hello.sh <<MY_PROGRAM
#!/bin/sh
echo ‘Hello, world’
MY_PROGRAM
% chmod +x hello...
Finding the program: PATH
% ./hello.sh
echo vs. /usr/bin/echo
% echo $PATH
/bin:/usr/bin:/usr/local/bin:
/home/borwicjh...
Variables and the Environment
% hello.sh
bash: hello.sh: Command not found
% PATH=“$PATH:.”
% hello.sh
Hello, world
An aside: Quoting
% echo ‘$USER’
$USER
% echo “$USER”
borwicjh
% echo “””
”
% echo “deacnetsct”
deacnetsct
% echo ‘”’
”
Variables and the Environment
% env
[…variables passed to sub-programs…]
% NEW_VAR=“Yes”
% echo $NEW_VAR
Yes
% env
[…PATH ...
Welcome to Shell Scripting!
How to Learn
 man
 man bash
 man cat
 man man
 man –k
 man –k manual
 Learning the Bash Shell, 2nd
Ed.
 “Bash Refe...
Introduction to bash
Continuing Lines: 
% echo This 
Is 
A 
Very 
Long 
Command Line
This Is A Very Long Command Line
%
Exit Status
$?
0 is True
% ls /does/not/exist
% echo $?
1
% echo $?
0
Exit Status: exit
% cat > test.sh <<_TEST_
exit 3
_TEST_
% chmod +x test.sh
% ./test.sh
% echo $?
3
Logic: test
% test 1 -lt 10
% echo $?
0
% test 1 == 10
% echo $?
1
Logic: test
test
[ ]
 [ 1 –lt 10 ]
[[ ]]
 [[ “this string” =~ “this” ]]
(( ))
 (( 1 < 10 ))
Logic: test
 [ -f /etc/passwd ]
 [ ! –f /etc/passwd ]
 [ -f /etc/passwd –a –f /etc/shadow ]
 [ -f /etc/passwd –o –f /e...
An aside: $(( )) for Math
% echo $(( 1 + 2 ))
3
% echo $(( 2 * 3 ))
6
% echo $(( 1 / 3 ))
0
Logic: if
if something
then
:
# “elif” a contraction of “else if”:
elif something-else
then
:
else
then
:
fi
Logic: if
if [ $USER –eq “borwicjh” ]
then
:
# “elif” a contraction of “else if”:
elif ls /etc/oratab
then
:
else
then
:
fi
Logic: if
# see if a file exists
if [ -e /etc/passwd ]
then
echo “/etc/passwd exists”
else
echo “/etc/passwd not found!”
fi
Logic: for
for i in 1 2 3
do
echo $i
done
Logic: for
for i in /*
do
echo “Listing $i:”
ls -l $i
read
done
Logic: for
for i in /*
do
echo “Listing $i:”
ls -l $i
read
done
Logic: for
for i in /*
do
echo “Listing $i:”
ls -l $i
read
done
Logic: C-style for
for (( expr1 ;
expr2 ;
expr3 ))
do
list
done
Logic: C-style for
LIMIT=10
for (( a=1 ;
a<=LIMIT ;
a++ ))
do
echo –n “$a ”
done
Logic: while
while something
do
:
done
Logic: while
a=0; LIMIT=10
while [ "$a" -lt "$LIMIT" ]
do
echo -n "$a ”
a=$(( a + 1 ))
done
Counters
COUNTER=0
while [ -e “$FILE.COUNTER” ]
do
COUNTER=$(( COUNTER + 1))
done
Note: race condition
Reusing Code: “Sourcing”
% cat > /path/to/my/passwords <<_PW_
FTP_USER=“sct”
_PW_
% echo $FTP_USER
% . /path/to/my/passwor...
Variable Manipulation
% FILEPATH=/path/to/my/output.lis
% echo $FILEPATH
/path/to/my/output.lis
% echo ${FILEPATH%.lis}
/p...
It takes a long time to
become a bash
guru…
Running Programs
Reasons for Running Programs
Check Return Code
 $?
Get Job Output
 OUTPUT=`echo “Hello”`
 OUTPUT=$(echo “Hello”)
Sen...
Pipes
Lots of Little Tools
echo “Hello” | 
wc -c
INPUT
echo
OUTPUT ERROR
0
1 2
INPUT
wc
OUTPUT ERROR
0
1 2
A Pipe!
Email Notification
% echo “Message” | 
mail –s “Here’s your message” 
borwicjh@wfu.edu
Dates
% DATESTRING=`date +%Y%m%d`
% echo $DATESTRING
20060125
% man date
FTP the Hard Way
ftp –n –u server.wfu.edu <<_FTP_
user username password
put FILE
_FTP_
FTP with wget
 wget 
ftp://user:pass@server.wfu.edu/file
 wget –r 
ftp://user:pass@server.wfu.edu/dir/
FTP with curl
curl –T upload-file 
-u username:password 
ftp://server.wfu.edu/dir/file
Searching: grep
% grep rayra /etc/passwd
% grep –r rayra /etc
% grep –r RAYRA /etc
% grep –ri RAYRA /etc
% grep –rli rayra...
Searching: find
% find /home/borwicjh 
-name ‘*.lis’
[all files matching *.lis]
% find /home/borwicjh 
-mtime -1 –name ‘*....
Searching: locate
% locate .lis
[files with .lis in path]
% locate log
[also finds “/var/log/messages”]
Applied Shell Programming
Make Your Life Easier
TAB completion
Control+R
history
cd -
Study a UNIX Editor
pushd/popd
% cd /tmp
% pushd /var/log
/var/log /tmp
% cd ..
% pwd
/var
% popd
/tmp
Monitoring processes
ps
ps –ef
ps –u oracle
ps –C sshd
man ps
“DOS” Mode Files
#!/usr/bin/bash^M
FTP transfer in ASCII, or
dos2unix infile > outfile
sqlplus
JOB=“ZZZTEST”
PARAMS=“ZZZTEST_PARAMS”
PARAMS_USER=“BORWICJH”
sqlplus $BANNER_USER/$BANNER_PW << _EOF_
set serverou...
sqlplus
sqlplus $USER/$PASS @$FILE_SQL 
$ARG1 $ARG2 $ARG3
if [ $? –ne 0 ]
then
exit 1
fi
if [ -e /file/sql/should/create ]...
Passing Arguments
% cat > test.sh <<_TEST_
echo “Your name is $1 $2”
_TEST_
% chmod +x test.sh
% ./test.sh John Borwick ig...
INB Job Submission Template
$1: user ID
$2: password
$3: one-up number
$4: process name
$5: printer name
% /path/to/your/s...
Scheduling Jobs
% crontab -l
0 0 * * * daily-midnight-job.sh
0 * * * * hourly-job.sh
* * * * * every-minute.sh
0 1 * * 0 1...
Other Questions?
Shells and Shell Scripts
bash
Running Commands
bash and Banner in Practice
Unix Shell Scripting Basics
Upcoming SlideShare
Loading in …5
×

Unix Shell Scripting Basics

5,307 views

Published on

Published in: Education, Technology
1 Comment
2 Likes
Statistics
Notes
  • http://www.dbmanagement.info/Tutorials/Unix.htm Unix Video Training Tutorial DVDs (2 DVDs) Sale Price 12.95 More than 5.9 GB of Video files
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
5,307
On SlideShare
0
From Embeds
0
Number of Embeds
1,412
Actions
Shares
0
Downloads
188
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide
  • First match wins
    Security
  • Unix Shell Scripting Basics

    1. 1. UNIX Shell-Scripting Basics
    2. 2. Agenda What is a shell? A shell script? Introduction to bash Running Commands Applied Shell Programming
    3. 3. What is a shell? %▌
    4. 4. What is a shell? /bin/bash
    5. 5. What is a shell? #!/bin/bash
    6. 6. What is a shell? INPUT shell OUTPUT ERROR
    7. 7. What is a shell? Any Program But there are a few popular shells…
    8. 8. Bourne Shells  /bin/sh  /bin/bash “Bourne-Again Shell” Steve Bourne
    9. 9. Other Common Shells C Shell (/bin/csh) Turbo C Shell (/bin/tcsh) Korn Shell (/bin/ksh)
    10. 10. An aside: What do I mean by /bin ? C Shell (/bin/csh) Turbo C Shell (/bin/tcsh) Korn Shell (/bin/ksh)
    11. 11. An aside: What do I mean by /bin ?  /bin, /usr/bin, /usr/local/bin  /sbin, /usr/sbin, /usr/local/sbin  /tmp  /dev  /home/borwicjh
    12. 12. What is a Shell Script? A Text File With Instructions Executable
    13. 13. What is a Shell Script? % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world
    14. 14. What is a Shell Script? A Text File % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world
    15. 15. An aside: Redirection  cat > /tmp/myfile  cat >> /tmp/myfile  cat 2> /tmp/myerr  cat < /tmp/myinput  cat <<INPUT Some input INPUT  cat > /tmp/x 2>&1 INPUT env OUTPUT ERROR 0 1 2
    16. 16. What is a Shell Script? How To Run % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world
    17. 17. What is a Shell Script? What To Do % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world
    18. 18. What is a Shell Script? Executable % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world
    19. 19. What is a Shell Script? Running it % cat > hello.sh <<MY_PROGRAM #!/bin/sh echo ‘Hello, world’ MY_PROGRAM % chmod +x hello.sh % ./hello.sh Hello, world
    20. 20. Finding the program: PATH % ./hello.sh echo vs. /usr/bin/echo % echo $PATH /bin:/usr/bin:/usr/local/bin: /home/borwicjh/bin % which echo /usr/bin/echo
    21. 21. Variables and the Environment % hello.sh bash: hello.sh: Command not found % PATH=“$PATH:.” % hello.sh Hello, world
    22. 22. An aside: Quoting % echo ‘$USER’ $USER % echo “$USER” borwicjh % echo “”” ” % echo “deacnetsct” deacnetsct % echo ‘”’ ”
    23. 23. Variables and the Environment % env […variables passed to sub-programs…] % NEW_VAR=“Yes” % echo $NEW_VAR Yes % env […PATH but not NEW_VAR…] % export NEW_VAR % env […PATH and NEW_VAR…]
    24. 24. Welcome to Shell Scripting!
    25. 25. How to Learn  man  man bash  man cat  man man  man –k  man –k manual  Learning the Bash Shell, 2nd Ed.  “Bash Reference” Cards  http://www.tldp.org/LDP/abs/html/
    26. 26. Introduction to bash
    27. 27. Continuing Lines: % echo This Is A Very Long Command Line This Is A Very Long Command Line %
    28. 28. Exit Status $? 0 is True % ls /does/not/exist % echo $? 1 % echo $? 0
    29. 29. Exit Status: exit % cat > test.sh <<_TEST_ exit 3 _TEST_ % chmod +x test.sh % ./test.sh % echo $? 3
    30. 30. Logic: test % test 1 -lt 10 % echo $? 0 % test 1 == 10 % echo $? 1
    31. 31. Logic: test test [ ]  [ 1 –lt 10 ] [[ ]]  [[ “this string” =~ “this” ]] (( ))  (( 1 < 10 ))
    32. 32. Logic: test  [ -f /etc/passwd ]  [ ! –f /etc/passwd ]  [ -f /etc/passwd –a –f /etc/shadow ]  [ -f /etc/passwd –o –f /etc/shadow ]
    33. 33. An aside: $(( )) for Math % echo $(( 1 + 2 )) 3 % echo $(( 2 * 3 )) 6 % echo $(( 1 / 3 )) 0
    34. 34. Logic: if if something then : # “elif” a contraction of “else if”: elif something-else then : else then : fi
    35. 35. Logic: if if [ $USER –eq “borwicjh” ] then : # “elif” a contraction of “else if”: elif ls /etc/oratab then : else then : fi
    36. 36. Logic: if # see if a file exists if [ -e /etc/passwd ] then echo “/etc/passwd exists” else echo “/etc/passwd not found!” fi
    37. 37. Logic: for for i in 1 2 3 do echo $i done
    38. 38. Logic: for for i in /* do echo “Listing $i:” ls -l $i read done
    39. 39. Logic: for for i in /* do echo “Listing $i:” ls -l $i read done
    40. 40. Logic: for for i in /* do echo “Listing $i:” ls -l $i read done
    41. 41. Logic: C-style for for (( expr1 ; expr2 ; expr3 )) do list done
    42. 42. Logic: C-style for LIMIT=10 for (( a=1 ; a<=LIMIT ; a++ )) do echo –n “$a ” done
    43. 43. Logic: while while something do : done
    44. 44. Logic: while a=0; LIMIT=10 while [ "$a" -lt "$LIMIT" ] do echo -n "$a ” a=$(( a + 1 )) done
    45. 45. Counters COUNTER=0 while [ -e “$FILE.COUNTER” ] do COUNTER=$(( COUNTER + 1)) done Note: race condition
    46. 46. Reusing Code: “Sourcing” % cat > /path/to/my/passwords <<_PW_ FTP_USER=“sct” _PW_ % echo $FTP_USER % . /path/to/my/passwords % echo $FTP_USER sct %
    47. 47. Variable Manipulation % FILEPATH=/path/to/my/output.lis % echo $FILEPATH /path/to/my/output.lis % echo ${FILEPATH%.lis} /path/to/my/output % echo ${FILEPATH#*/} path/to/my/output.lis % echo ${FILEPATH##*/} output.lis
    48. 48. It takes a long time to become a bash guru…
    49. 49. Running Programs
    50. 50. Reasons for Running Programs Check Return Code  $? Get Job Output  OUTPUT=`echo “Hello”`  OUTPUT=$(echo “Hello”) Send Output Somewhere  Redirection: <, >  Pipes
    51. 51. Pipes Lots of Little Tools echo “Hello” | wc -c INPUT echo OUTPUT ERROR 0 1 2 INPUT wc OUTPUT ERROR 0 1 2 A Pipe!
    52. 52. Email Notification % echo “Message” | mail –s “Here’s your message” borwicjh@wfu.edu
    53. 53. Dates % DATESTRING=`date +%Y%m%d` % echo $DATESTRING 20060125 % man date
    54. 54. FTP the Hard Way ftp –n –u server.wfu.edu <<_FTP_ user username password put FILE _FTP_
    55. 55. FTP with wget  wget ftp://user:pass@server.wfu.edu/file  wget –r ftp://user:pass@server.wfu.edu/dir/
    56. 56. FTP with curl curl –T upload-file -u username:password ftp://server.wfu.edu/dir/file
    57. 57. Searching: grep % grep rayra /etc/passwd % grep –r rayra /etc % grep –r RAYRA /etc % grep –ri RAYRA /etc % grep –rli rayra /etc
    58. 58. Searching: find % find /home/borwicjh -name ‘*.lis’ [all files matching *.lis] % find /home/borwicjh -mtime -1 –name ‘*.lis’ [*.lis, if modified within 24h] % man find
    59. 59. Searching: locate % locate .lis [files with .lis in path] % locate log [also finds “/var/log/messages”]
    60. 60. Applied Shell Programming
    61. 61. Make Your Life Easier TAB completion Control+R history cd - Study a UNIX Editor
    62. 62. pushd/popd % cd /tmp % pushd /var/log /var/log /tmp % cd .. % pwd /var % popd /tmp
    63. 63. Monitoring processes ps ps –ef ps –u oracle ps –C sshd man ps
    64. 64. “DOS” Mode Files #!/usr/bin/bash^M FTP transfer in ASCII, or dos2unix infile > outfile
    65. 65. sqlplus JOB=“ZZZTEST” PARAMS=“ZZZTEST_PARAMS” PARAMS_USER=“BORWICJH” sqlplus $BANNER_USER/$BANNER_PW << _EOF_ set serveroutput on set sqlprompt "" EXECUTE WF_SATURN.FZ_Get_Parameters('$JOB', '$PARAMS', '$PARAMS_USER'); _EOF_
    66. 66. sqlplus sqlplus $USER/$PASS @$FILE_SQL $ARG1 $ARG2 $ARG3 if [ $? –ne 0 ] then exit 1 fi if [ -e /file/sql/should/create ] then […use SQL-created file…] fi  Ask Amy Lamy! 
    67. 67. Passing Arguments % cat > test.sh <<_TEST_ echo “Your name is $1 $2” _TEST_ % chmod +x test.sh % ./test.sh John Borwick ignore- this Your name is John Borwick
    68. 68. INB Job Submission Template $1: user ID $2: password $3: one-up number $4: process name $5: printer name % /path/to/your/script $UI $PW $ONE_UP $JOB $PRNT
    69. 69. Scheduling Jobs % crontab -l 0 0 * * * daily-midnight-job.sh 0 * * * * hourly-job.sh * * * * * every-minute.sh 0 1 * * 0 1AM-on-sunday.sh % EDITOR=vi crontab –e % man 5 crontab
    70. 70. Other Questions? Shells and Shell Scripts bash Running Commands bash and Banner in Practice

    ×