Ch23 system administration


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ch23 system administration

  1. 1. Automating System Administration Tasks Chapter 23
  2. 2. Chapter Goals <ul><li>Understand what tasks lend themselves to automation. </li></ul><ul><li>Understand the tools available to automate system administration tasks. </li></ul><ul><li>Understand differences between shells, and shell programming languages. </li></ul>
  3. 3. What to automate? <ul><li>As we have seen, there are several repetitive tasks in system administration: </li></ul><ul><ul><li>Downloading and installing patches </li></ul></ul><ul><ul><li>Installing accounts </li></ul></ul><ul><ul><li>Managing configuration files </li></ul></ul><ul><ul><li>Managing printers </li></ul></ul><ul><ul><li>Scanning log files </li></ul></ul><ul><ul><li>Monitoring security </li></ul></ul><ul><ul><li>Cleaning out temporary files. </li></ul></ul><ul><li>These tasks lend themselves to automation using a variety of tools. </li></ul>
  4. 4. What to automate? <ul><li>Patches </li></ul><ul><ul><li>RedHat up2date </li></ul></ul><ul><ul><li>Notre Dame’s “nd.installpatch” </li></ul></ul><ul><li>Installing accounts </li></ul><ul><ul><li>Acmaint </li></ul></ul><ul><ul><li>grep user_name /afs/ >>/etc/passwd ; pwconv </li></ul></ul><ul><li>Managing Configuration Files </li></ul><ul><ul><li>rdist </li></ul></ul><ul><ul><li>expect </li></ul></ul><ul><ul><li>rsync </li></ul></ul><ul><ul><li>package </li></ul></ul>
  5. 5. What to automate? <ul><li>Managing printers </li></ul><ul><ul><li>Write a script to add printers for you </li></ul></ul><ul><li>Scanning log files and monitoring security </li></ul><ul><ul><li>Swatch </li></ul></ul><ul><ul><li>Logwatch </li></ul></ul><ul><ul><li>Write a script to watch for things of interest to you. </li></ul></ul><ul><li>Cleaning out temporary files </li></ul><ul><ul><li>cron </li></ul></ul><ul><ul><li>find </li></ul></ul>
  6. 6. Information Distribution <ul><li>Distributing system information (files) is a problem. </li></ul><ul><ul><li>Small site --> small problem. </li></ul></ul><ul><ul><li>Large site --> LARGE problem. </li></ul></ul><ul><ul><ul><li>Consider: Site with 4 system architectures: </li></ul></ul></ul><ul><ul><ul><ul><li>Sun (250 hosts) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>HP (250 hosts) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>IBM (250 hosts) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>PC’s with Linux (250 hosts) </li></ul></ul></ul></ul><ul><ul><ul><li>How would you distribute a new version of sendmail to each host? </li></ul></ul></ul>
  7. 7. Information Distribution <ul><li>The first problem to tackle is “what should we distribute”? </li></ul><ul><ul><li>Try to determine which files may be distributed to multiple system architectures (usually text files). </li></ul></ul><ul><ul><li>Try to determine what binaries are located on the system disks (not on network file systems). </li></ul></ul><ul><ul><li>Try to determine which files are scripts that would work on multiple architectures. </li></ul></ul><ul><ul><li>Try to determine which files are common to all hosts of a particular architecture. </li></ul></ul>
  8. 8. Information Distribution <ul><li>Some typical examples: </li></ul><ul><ul><li>System Binaries </li></ul></ul><ul><ul><ul><li>sendmail </li></ul></ul></ul><ul><ul><ul><li>login – common to a single OS/architecture </li></ul></ul></ul><ul><ul><ul><li>ftpd </li></ul></ul></ul><ul><ul><ul><li>tcpd </li></ul></ul></ul><ul><ul><li>Common configuration files </li></ul></ul><ul><ul><ul><li>/etc/motd </li></ul></ul></ul><ul><ul><ul><li> </li></ul></ul></ul><ul><ul><ul><li>password file </li></ul></ul></ul><ul><ul><ul><li>shadow file – may not be directly usable </li></ul></ul></ul>
  9. 9. Information Distribution <ul><li>The next problem to tackle is which hosts you want to distribute files to: </li></ul><ul><ul><li>All hosts of an architecture? (Sparc, HP) </li></ul></ul><ul><ul><li>All hosts which are clients of a particular server? </li></ul></ul><ul><ul><li>All hosts which are null clients of a particular mail server? </li></ul></ul><ul><ul><li>All hosts in engineering? </li></ul></ul><ul><li>The more files/architectures you have to distribute information to, the more complex the setup will be for the distribution method (and the more likely you will mess something up). </li></ul>
  10. 10. Information Distribution <ul><li>There are two methods of distributing system information: </li></ul><ul><ul><li>The push method - a server pushes the information to the clients. </li></ul></ul><ul><ul><ul><li>Advantages: </li></ul></ul></ul><ul><ul><ul><ul><li>It is usually easy to run “unscheduled” push operations. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>You have some control over when the process runs. </li></ul></ul></ul></ul><ul><ul><ul><li>Disadvantages: </li></ul></ul></ul><ul><ul><ul><ul><li>More difficult to update a single client. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Tainted files will be distributed just like good files. </li></ul></ul></ul></ul><ul><ul><ul><li>Setup: </li></ul></ul></ul><ul><ul><ul><ul><li>Need to build/maintain copies of files to distribute. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Need to build/maintain the distribution control file. </li></ul></ul></ul></ul>
  11. 11. Information Distribution <ul><li>There are two methods of distributing system information: </li></ul><ul><ul><li>The pull method - a client pulls the information from a server. </li></ul></ul><ul><ul><ul><li>Advantages: </li></ul></ul></ul><ul><ul><ul><ul><li>It is easy to run “unscheduled” updates. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Systems can be reset to a known state upon reboot. </li></ul></ul></ul></ul><ul><ul><ul><li>Disadvantages: </li></ul></ul></ul><ul><ul><ul><ul><li>More difficult to update clients all at once. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Administrator has no idea which hosts have updated. </li></ul></ul></ul></ul><ul><ul><ul><li>Setup: </li></ul></ul></ul><ul><ul><ul><ul><li>Need to build/maintain copies of files to distribute. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Need to build/maintain the distribution control file. </li></ul></ul></ul></ul>
  12. 12. Information Distribution <ul><li>The push method </li></ul><ul><ul><li>The rdist program is one example of a “push method” file distribution system. </li></ul></ul><ul><ul><ul><li>Need to build a distfile </li></ul></ul></ul><ul><ul><ul><ul><li>tells what systems to distribute the files to </li></ul></ul></ul></ul><ul><ul><ul><ul><li>tells which files to distribute to the systems </li></ul></ul></ul></ul><ul><ul><ul><ul><li>allows special operations to be performed </li></ul></ul></ul></ul><ul><ul><ul><ul><li>allows notification of success/failures </li></ul></ul></ul></ul><ul><ul><ul><ul><li>allows exceptions to the rules </li></ul></ul></ul></ul><ul><ul><ul><ul><li>may be run from a cron entry </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Has a handy “no action” version that allows rules testing. </li></ul></ul></ul></ul>
  13. 13. Information Distribution <ul><ul><li>Rdist </li></ul></ul><ul><ul><ul><li>Need to create a repository of files to distribute. </li></ul></ul></ul><ul><ul><ul><ul><li>Good idea to use RCS/SCCS/CVS to keep track of versions, and changes. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Good idea to keep the repository “read only” so that bad versions of files do not get distributed. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Special caveat: </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Rdist distributes the files with the ownership of the user who ran the rdist command. </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>This can be a major problem if not handled by the settings in the distfile! </li></ul></ul></ul></ul></ul>
  14. 14. Sample rdist file <ul><li># </li></ul><ul><li># distribute something to lab hosts </li></ul><ul><li># </li></ul><ul><li>BASE=“/afs/” </li></ul><ul><li>SOL_CLIENTS=( snowwhite boole branin eckert noyce turing ) </li></ul><ul><li># </li></ul><ul><li># Files to distribute </li></ul><ul><li># </li></ul><ul><li>lab6: </li></ul><ul><li>$BASE/courses/cse444/lab6.init -> ${SOL_CLIENTS} </li></ul><ul><li>install /etc/init.d/lab6; </li></ul><ul><li>special &quot;chmod 755 /etc/init.d/lab6&quot;; </li></ul>
  15. 15. Rdist Example <ul><li># </li></ul><ul><li># distribute Solaris sendmail to grumpy, mail.cse), mail clients </li></ul><ul><li># distribute HPUX sendmail to geo </li></ul><ul><li># distribute AIX4 sendmail to yoyo and euclid </li></ul><ul><li>BASE=/afs/ </li></ul><ul><li>SRC=/afs/ </li></ul><ul><li>SOL_CLIENTS=( aliens.dcrl atanasoff athena backus bashful berry boole branin) </li></ul><ul><li>STANDALONE=( grumpy ) </li></ul><ul><li>AIX_CLIENTS=(euclid yoyo) </li></ul><ul><li>HPUX_CLIENTS=( geo ) </li></ul><ul><li>SUNS=(${SOL_CLIENTS} ${STANDALONE} ${SERVER}) </li></ul><ul><li>SERVER=( wizard ) </li></ul><ul><li>ALL=(${SERVER} ${STANDALONE} ${SOL_CLIENTS} ${AIX_CLIENTS} ) </li></ul>
  16. 16. Rdist example <ul><li># </li></ul><ul><li># Files to distribute </li></ul><ul><li># </li></ul><ul><li> </li></ul><ul><li>$BASE/common/etc/mail/ -> ${SOL_CLIENTS} </li></ul><ul><li>special &quot;cp /etc/mail/ /etc/mail/;; </li></ul><ul><li>install /etc/mail/; </li></ul><ul><li>special &quot;cp /etc/mail/ /etc/mail/;; </li></ul><ul><li>special &quot;ln -s /etc/mail/ /etc/;; </li></ul><ul><li>special &quot;rm /etc/mail/;; </li></ul><ul><li> </li></ul><ul><li>$BASE/server/etc/mail/ -> ${SERVER} </li></ul><ul><li>special &quot;cp /etc/mail/ /etc/mail/;; </li></ul><ul><li>install /etc/mail/; </li></ul><ul><li>special &quot;cp /etc/mail/ /etc/mail/;; </li></ul><ul><li>special &quot;ln -s /etc/mail/ /etc/;; </li></ul><ul><li>special &quot;rm /etc/mail/;; </li></ul>
  17. 17. Rdist example <ul><li> </li></ul><ul><li>$BASE/grumpy/etc/mail/ -> ${STANDALONE} </li></ul><ul><li>special &quot;cp /etc/mail/ /etc/mail/;; </li></ul><ul><li>install /etc/mail/; </li></ul><ul><li>special &quot;cp /etc/mail/ /etc/mail/;; </li></ul><ul><li>special &quot;ln -s /etc/mail/ /etc/;; </li></ul><ul><li>special &quot;rm /etc/mail/;; </li></ul><ul><li>sendmail.sun: </li></ul><ul><li>$BASE/common/usr/lib/sendmail -> ${SUNS} </li></ul><ul><li>special &quot;cp /usr/lib/sendmail /usr/lib/sendmail.old&quot;; </li></ul><ul><li>install /usr/lib/; </li></ul><ul><li>special &quot;chgrp sys /usr/lib/sendmail&quot;; </li></ul><ul><li>special &quot;chmod 6755 /usr/lib/sendmail&quot;; </li></ul><ul><li>special &quot;/usr/local/bin/killj sendmail&quot;; </li></ul><ul><li>special &quot;cp /usr/lib/ /usr/lib/sendmail&quot;; </li></ul><ul><li>special &quot;/usr/lib/sendmail -bd -q1h&quot;; </li></ul><ul><li>special &quot;rm /usr/lib/;; </li></ul>
  18. 18. Rdist example <ul><li> </li></ul><ul><li>$SRC/sendmail/cf/cf/ -> ${AIX_CLIENTS} </li></ul><ul><li>special &quot;cp /etc/mail/ /etc/mail/;; </li></ul><ul><li>install /etc/mail/; </li></ul><ul><li>special &quot;ln -s /etc/mail/ /etc/;; </li></ul><ul><li>sendmail.aix: </li></ul><ul><li>$SRC/sendmail/src/obj.AIX.4.000044673500/sendmail -> ${AIX_CLIENTS} </li></ul><ul><li>special &quot;cp /usr/lib/sendmail /usr/lib/sendmail.old&quot;; </li></ul><ul><li>install /usr/lib/sendmail; </li></ul><ul><li>special &quot;/usr/local/bin/killj sendmail&quot;; </li></ul><ul><li>special &quot;/usr/lib/sendmail -bd -q1h&quot;; </li></ul><ul><li> </li></ul><ul><li>$SRC/sendmail/cf/cf/ -> ${HPUX_CLIENTS} </li></ul><ul><li>special &quot;cp /etc/mail/ /etc/mail/;; </li></ul><ul><li>install /etc/mail/; </li></ul><ul><li>special &quot;ln -s /etc/mail/ /etc/;; </li></ul>
  19. 19. Rdist example <ul><li>sendmail.hpux: </li></ul><ul><li>$SRC/sendmail/src/obj.HP-UX.10.2/sendmail -> ${HPUX_CLIENTS} </li></ul><ul><li>special &quot;cp /usr/lib/sendmail /usr/lib/sendmail.old&quot;; </li></ul><ul><li>install /usr/lib/sendmail; </li></ul><ul><li>special &quot;/usr/local/bin/killj sendmail&quot;; </li></ul><ul><li>special &quot;/usr/lib/sendmail -bd -q1h&quot;; </li></ul>
  20. 20. Information Distribution <ul><li>The pull method </li></ul><ul><ul><li>There are two common “pull” packages: </li></ul></ul><ul><ul><ul><li>expect </li></ul></ul></ul><ul><ul><ul><ul><li>expect uses ftp to copy files to the client </li></ul></ul></ul></ul><ul><ul><ul><ul><li>expect provides some directives to manage the ftp session. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>can be run from a bootup script </li></ul></ul></ul></ul><ul><ul><ul><ul><li>can be run from a cron entry </li></ul></ul></ul></ul>
  21. 21. Sample expect script <ul><li>#!/usr/local/src/gnu/bin/expect </li></ul><ul><li>spawn /bin/ftp </li></ul><ul><li>expect &quot;ftp> &quot; {send &quot;open dopey.cselab.nd.edur&quot;} </li></ul><ul><li>expect &quot;): &quot; {send &quot;gorpr&quot;} </li></ul><ul><li>expect &quot;Password:&quot; {send &quot;c@Ntheh@r&quot;} </li></ul><ul><li>expect &quot;ftp> &quot; {send &quot;binr&quot;} </li></ul><ul><li>expect &quot;ftp> &quot; {send &quot;get np.tar /dev/nullr&quot;} </li></ul><ul><li>expect &quot;ftp> &quot; {send &quot;quitr&quot;} </li></ul><ul><li>exit </li></ul>
  22. 22. Information Distribution <ul><ul><li>Another pull technology is the “package” command used with AFS. </li></ul></ul><ul><ul><ul><li>package requires the administrator to set up a control file which contains a list of files which will be distributed. </li></ul></ul></ul><ul><ul><ul><li>package allows a noaction mode to test rules. </li></ul></ul></ul><ul><ul><ul><li>package could be run from cron (dangerous?). </li></ul></ul></ul><ul><ul><ul><li>package is usually run at system reboot. </li></ul></ul></ul>
  23. 23. Sample package file <ul><li># </li></ul><ul><li># CSE supplemental package file </li></ul><ul><li># Need these directories defined so that we can get to other entries </li></ul><ul><li># </li></ul><ul><li>D / root root 755 </li></ul><ul><li>D /afs root root 2755 </li></ul><ul><li>D /etc root staff 755 </li></ul><ul><li>D /etc/mail sys mail 755 </li></ul><ul><li>D /etc/init.d root sys 775 </li></ul><ul><li>D /etc/inet root sys 755 </li></ul><ul><li>D /etc/rc3.d root sys 775 </li></ul><ul><li>D /etc/rc2.d root sys 775 </li></ul><ul><li>DR /var/tmp bin sys 1777 </li></ul><ul><li>DR /tmp bin sys 1777 </li></ul><ul><li>D /usr root sys 775 </li></ul><ul><li>D /usr/bin root bin 775 </li></ul><ul><li>D /usr/sbin root bin 775 </li></ul><ul><li>D /usr/include root sys 775 </li></ul><ul><li>D /usr/vice root other 755 </li></ul><ul><li>D /usr/vice/etc root other 755 </li></ul><ul><li>D /var root sys 755 </li></ul><ul><li>D /var/spool root bin 755 </li></ul><ul><li>D /var/spool/cron root sys 755 </li></ul><ul><li>D /var/spool/cron/crontabs root sys 755 </li></ul>
  24. 24. Sample package file <ul><li># </li></ul><ul><li># Some necessary links </li></ul><ul><li># Eases installation of new machines </li></ul><ul><li># </li></ul><ul><li>LA /usr/include/X11 /afs/ </li></ul><ul><li>LA /usr/lib/X11 /afs/ </li></ul><ul><li># </li></ul><ul><li># Particular to this cluster </li></ul><ul><li># Package file, MOTD, and AFS cache size </li></ul><ul><li># </li></ul><ul><li>F /etc/motd /afs/ root sys 644 </li></ul><ul><li># </li></ul><ul><li># For all CSE clients (i.e. non-wizard) </li></ul><ul><li># </li></ul><ul><li>F /.rhosts /afs/ root sys 444 </li></ul><ul><li>F /.cshrc /afs/ root sys 444 </li></ul><ul><li>F /.profile /afs/ root sys 444 </li></ul><ul><li>F /var/spool/cron/crontabs/root /afs/ root other 400 </li></ul>
  25. 25. Sample package file <ul><li># Replace some /etc </li></ul><ul><li>F /etc/passwd /afs/ root other 644 </li></ul><ul><li>F /etc/resolv.conf /afs/ root other 644 </li></ul><ul><li>F /etc/pam.conf /afs/ root other 644 </li></ul><ul><li>FQ /etc/inet/hosts /afs/ root other 444 </li></ul><ul><li>LA /etc/hosts /etc/inet/hosts root other 644 </li></ul><ul><li>F /etc/inet/services /afs/ root other 644 </li></ul><ul><li>LA /etc/services /etc/inet/services root other 644 </li></ul><ul><li>F /etc/syslog.conf /afs/ root other 644 </li></ul><ul><li>F /etc/init.d/cse_client /afs/ root other 755 </li></ul><ul><li>LA /etc/rc3.d/S99cse_client /etc/init.d/cse_client root other 755 </li></ul><ul><li>F /usr/etc/ifstatus /afs/ root other 755 </li></ul><ul><li>F /etc/inet/inetd.conf /afs/ root sys 444 </li></ul><ul><li>LA /etc/inetd.conf /etc/inet/inetd.conf root other 444 </li></ul><ul><li>F /usr/sbin/in.ftpd /afs/ bin bin 555 </li></ul><ul><li>F /etc/ftpaccess /afs/ bin bin 555 </li></ul><ul><li>F /usr/bin/top /afs/ bin bin 4755 </li></ul><ul><li>## </li></ul><ul><li>FAQ /usr/lib/sendmail /afs/ root other 4555 </li></ul><ul><li>F /usr/lib/sendmail.hf /afs/ root other 555 </li></ul><ul><li>F /etc/mail/ /afs/ root other 444 </li></ul><ul><li>LA /etc/ /etc/mail/ root other 444 </li></ul>
  26. 26. Information Distribution <ul><li>Other distribution methods </li></ul><ul><ul><li>NIS - centralized database, not really a way to “distribute” files. The files are available to other hosts, but they are located on a (few) server(s). </li></ul></ul><ul><ul><li>MIT’s Hesoid - never caught on, difficult to build configuration files. Uses DNS to provide information from a central server. </li></ul></ul><ul><ul><li>Netinfo - NeXT computer’s version of NIS. </li></ul></ul><ul><ul><li>Rsync </li></ul></ul><ul><ul><li>Cfengine </li></ul></ul>
  27. 27. Shells <ul><li>Many automation tasks are handled using shell (or other) scripts. </li></ul><ul><li>Shell scripts are generally portable across multiple versions of operating systems. </li></ul><ul><li>Other scripting languages, such as Python, or Perl are much more powerful, but not always installed on systems by default. </li></ul>
  28. 28. Shell Programming <ul><li>Shells </li></ul><ul><ul><li>UNIX provides several shell programs for the users. </li></ul></ul><ul><ul><ul><li>These programs are the user’s interface with the system software. </li></ul></ul></ul><ul><ul><ul><li>The shell programs read commands from the user terminal and interpret or execute them as required. </li></ul></ul></ul><ul><ul><ul><li>Some of the more popular shells available are: </li></ul></ul></ul><ul><ul><ul><ul><li>sh - the original Bourne shell </li></ul></ul></ul></ul><ul><ul><ul><ul><li>csh - the Berkeley C shell </li></ul></ul></ul></ul><ul><ul><ul><ul><li>bash - the Bourne Again SHell (an updated sh) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>ksh - the Korn shell </li></ul></ul></ul></ul><ul><ul><ul><ul><li>tcsh - an updated version of the C shell. </li></ul></ul></ul></ul>
  29. 29. Shell Programming <ul><li>Shells </li></ul><ul><ul><li>Shell selection is a highly subjective user decision. </li></ul></ul><ul><ul><li>While the shells are primarily a command interpreter, they also provide an interpreted programming language which users can employ to make their lives easier. </li></ul></ul><ul><ul><li>Different shells implement different programming languages. </li></ul></ul><ul><ul><li>Unfortunately, system administrators have to be proficient with several shell languages in order to manage the system. </li></ul></ul><ul><ul><li>Over the next few lectures we will look at the “sh” and “csh” shells. In particular we will look at using these shells to write shell scripts which will assist us with our system administration tasks. </li></ul></ul>
  30. 30. Shell Programming <ul><li>Shells </li></ul><ul><ul><li>Most of you use a variant of the Cshell (csh or tcsh). </li></ul></ul><ul><ul><ul><li>The Cshell provides some job control, and command line editing options not available under the Bourne shell (sh). </li></ul></ul></ul><ul><ul><ul><li>Tcsh provides more advanced command line editing and parsing than csh...but tcsh is not a standard shell on all versions of Unix. </li></ul></ul></ul><ul><ul><li>Under Solaris the root account uses the bourne shell located in /sbin/sh. </li></ul></ul><ul><ul><ul><li>The bourne shell is not my favorite shell, but the examples I give will be sh based. </li></ul></ul></ul><ul><ul><ul><li>If you are serious about system administration, I suggest you become familiar with both sh and csh environments. </li></ul></ul></ul>
  31. 31. Shell Programming <ul><li>Common Shell Features </li></ul><ul><ul><li>All shells have some common features: </li></ul></ul><ul><ul><ul><li>Shells allow interactive or background processing. </li></ul></ul></ul><ul><ul><ul><li>Shells allow input/output redirection. </li></ul></ul></ul><ul><ul><ul><li>Shells allow “pipes” between commands. </li></ul></ul></ul><ul><ul><ul><li>Shells allow wildcard matching. </li></ul></ul></ul><ul><ul><ul><li>Shells assign special meaning to some characters. </li></ul></ul></ul><ul><ul><ul><li>Shells use variables to store values. </li></ul></ul></ul><ul><ul><li>The common invocation of a shell command is: </li></ul></ul><ul><ul><ul><li>command [options] arguments </li></ul></ul></ul>
  32. 32. Shell Programming <ul><li>Example Script Invocations </li></ul><ul><ul><li>nispopulate -u -F -p /nisstuff/update -d </li></ul></ul><ul><ul><li>/etc/init.d/lab10 start </li></ul></ul><ul><ul><li>which talk </li></ul></ul><ul><ul><li>cd /tmp </li></ul></ul><ul><ul><li>basename /afs/ </li></ul></ul>
  33. 33. Shell Programming <ul><li>Built-in Shell Operations </li></ul><ul><ul><li>Shells all have some built-in commands which we can use when writing shell scripts: </li></ul></ul><ul><ul><ul><li>echo - write a message on standard output (CSH). </li></ul></ul></ul><ul><ul><ul><ul><li>Been hit by root kit? Use echo! </li></ul></ul></ul></ul><ul><ul><ul><li>read - read something from standard input </li></ul></ul></ul><ul><ul><ul><li>test - test various conditions </li></ul></ul></ul><ul><ul><ul><li>expr - evaluate an expression </li></ul></ul></ul><ul><ul><ul><li>true - return a true value </li></ul></ul></ul><ul><ul><ul><li>false - return a false value </li></ul></ul></ul><ul><ul><ul><li>wait - wait for something (good) to happen </li></ul></ul></ul><ul><ul><ul><li>sleep - stop for some period of time </li></ul></ul></ul>
  34. 34. Shell Programming <ul><li>Shell Variables (Built-in) </li></ul><ul><ul><li>All shells have some built-in variables (reserved names) which can be useful when we write shell scripts: </li></ul></ul><ul><ul><ul><li>PATH - The system search path </li></ul></ul></ul><ul><ul><ul><li>LOGNAME - The users login name </li></ul></ul></ul><ul><ul><ul><li>TERM - The kind of terminal the user is at </li></ul></ul></ul><ul><ul><ul><li>HOME - The users home directory </li></ul></ul></ul><ul><ul><ul><li>MAIL - The location of the users mailbox </li></ul></ul></ul><ul><ul><ul><li>PS1 - The primary prompt string for the user </li></ul></ul></ul><ul><ul><ul><li>LD_LIBRARY_PATH - The path to system libraries. </li></ul></ul></ul>
  35. 35. Shell Programming <ul><li>Shell Variables (user assigned) </li></ul><ul><ul><li>To assign a values to a variables: </li></ul></ul><ul><ul><ul><li>variable=value (set variable=value for CSH) </li></ul></ul></ul><ul><ul><ul><ul><li>for example fruit=apple , or i=1, set name=$USER </li></ul></ul></ul></ul><ul><ul><li>To access the value of the variable, prepend a “$” : </li></ul></ul><ul><ul><ul><li>echo fruit </li></ul></ul></ul><ul><ul><ul><ul><li>fruit </li></ul></ul></ul></ul><ul><ul><ul><li>echo $fruit </li></ul></ul></ul><ul><ul><ul><ul><li>apple </li></ul></ul></ul></ul><ul><ul><ul><li>echo “User: $USER” </li></ul></ul></ul><ul><ul><ul><ul><li>User: curt </li></ul></ul></ul></ul>
  36. 36. Shell Programming <ul><li>Shell Variables </li></ul><ul><ul><li>You can also set variables by executing commands: </li></ul></ul><ul><ul><ul><li>menu=`cat food` </li></ul></ul></ul><ul><ul><ul><li>now=`date` </li></ul></ul></ul><ul><ul><li>It is also possible to assign string values to variables: </li></ul></ul><ul><ul><ul><li>PS1=“`uname -n` - Yes Master: “ </li></ul></ul></ul><ul><ul><ul><ul><li>echo $PS1 </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>grumpy - Yes Master: </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Must place multi-word strings in double quotes </li></ul></ul></ul></ul>
  37. 37. Shell Programming <ul><li>Metacharacters </li></ul><ul><ul><li>All shells treat certain characters as special. </li></ul></ul><ul><ul><ul><li>$ ^ ; & { } [ ] ‘ ` * ( ) | < > newline space tab </li></ul></ul></ul><ul><ul><ul><li>To use these characters in scripts they must be quoted or escaped. </li></ul></ul></ul><ul><ul><ul><ul><li>escape a character by placing a backslash in front of it. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>single quotes protect text from any substitutions. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>back quotes cause the shell to perform command substitution </li></ul></ul></ul></ul><ul><ul><ul><ul><li>double quotes protect against wildcard substitution </li></ul></ul></ul></ul>
  38. 38. Shell Programming <ul><li>Escaping Characters: </li></ul><ul><li>$ ( ) </li></ul><ul><li>syntax error: `)' unexpected </li></ul><ul><li>  </li></ul><ul><li>$ () </li></ul><ul><li>(): not found </li></ul><ul><li>  </li></ul><ul><li>Single Quotes </li></ul><ul><li>$ it's $cmd </li></ul><ul><li>> ' </li></ul><ul><li>its $cmd^J: not found </li></ul><ul><li>  </li></ul><ul><li>$ it's $cmd </li></ul><ul><li>it's: not found </li></ul>
  39. 39. Shell Programming <ul><li>Back Quotes </li></ul><ul><li>$ HN=uname -n </li></ul><ul><li>-n: not found </li></ul><ul><li>$ HN=`uname -n` </li></ul><ul><li>$ echo $HN </li></ul><ul><li>grumpy </li></ul><ul><li>  </li></ul><ul><li>$ who am i </li></ul><ul><li>curt pts/5 Nov 11 13:47 </li></ul><ul><li>  </li></ul><ul><li>$ 'who am i' </li></ul><ul><li>who am i: not found </li></ul><ul><li>$ `who am i` </li></ul><ul><li>curt: not found </li></ul>
  40. 40. Shell Programming <ul><li>Double Quotes </li></ul><ul><li>$ grep Things to do todolist </li></ul><ul><li>grep: can't open to </li></ul><ul><li>grep: can't open do </li></ul><ul><li>todolist:Things to do: </li></ul><ul><li>  </li></ul><ul><li>$ grep &quot;Things to do&quot; todolist </li></ul><ul><li>Things to do: </li></ul>
  41. 41. Shell Programming <ul><li>Shell test features </li></ul><ul><ul><li>All shells provide test mechanisms: </li></ul></ul><ul><ul><ul><li>This feature allows us to do conditional processing within shell scripts. </li></ul></ul></ul><ul><ul><li>In sh we can test for: </li></ul></ul><ul><ul><ul><li>numeric values: </li></ul></ul></ul><ul><ul><ul><ul><li>if [ $# -eq 0 ] ; then if ( $# = 0 ) then (CSH version) </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>general form: N <primitive> M </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>-eq, -ne, -gt, -lt, -ge, -le </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>= != > < >= <= </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>-o (or) -a (and) </li></ul></ul></ul></ul></ul>
  42. 42. Shell Programming <ul><li>Test Conditions </li></ul><ul><ul><li>The shells also provides a way to test the external environment: </li></ul></ul><ul><ul><ul><li>-r file (read permission) </li></ul></ul></ul><ul><ul><ul><li>-w file (write permission) </li></ul></ul></ul><ul><ul><ul><li>-x file (execute permission) </li></ul></ul></ul><ul><ul><ul><li>-e file (file exists) </li></ul></ul></ul><ul><ul><ul><li>-o file (user owns file) </li></ul></ul></ul><ul><ul><ul><li>-z file (file is empty) </li></ul></ul></ul><ul><ul><ul><li>-f file (plain file) </li></ul></ul></ul><ul><ul><ul><li>-d file (file is a directory) </li></ul></ul></ul><ul><ul><ul><ul><li>if [ -e /usr/lab/tmp/”$AFSID”.start ] ; then </li></ul></ul></ul></ul><ul><ul><ul><ul><li>if [ ! -d /usr/lab/tmp/”$AFSID” ] ; then </li></ul></ul></ul></ul>
  43. 43. Shell Programming <ul><li>Shell test features </li></ul><ul><ul><ul><li>string equality </li></ul></ul></ul><ul><ul><ul><ul><li>if [ $LOGNAME = “root” ] ; then </li></ul></ul></ul></ul><ul><ul><ul><ul><li>general form: S <primitive> R ; or <primitive> S </li></ul></ul></ul></ul><ul><ul><ul><ul><li>primitives: =, != </li></ul></ul></ul></ul><ul><ul><ul><ul><li>-z - is string zero length </li></ul></ul></ul></ul><ul><ul><ul><ul><li>-n - is string non zero length </li></ul></ul></ul></ul><ul><ul><ul><li>Testing strings can produce “odd” results: </li></ul></ul></ul><ul><ul><ul><li>number=1 </li></ul></ul></ul><ul><ul><ul><li>nombre=‘ 1’ </li></ul></ul></ul><ul><ul><ul><ul><li>test $number = $nombre evaluates true (spaces eaten) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>test “$number” = “$nombre” evaluates false </li></ul></ul></ul></ul>
  44. 44. Shell Programming <ul><li>Shell Math </li></ul><ul><ul><li>The shell also allows mathematical operations on variables: </li></ul></ul><ul><ul><ul><li>The expr command evaluates it’s arguments. </li></ul></ul></ul><ul><ul><ul><ul><li>General form: expr variable operation variable </li></ul></ul></ul></ul><ul><ul><ul><ul><li>operations: +, -, *, /, % (remainder) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>sum=`expr $1 + $2` </li></ul></ul></ul></ul>
  45. 45. Shell Programming <ul><li>Conditional Processing </li></ul><ul><ul><li>The shells all provide a way to do conditional processing: </li></ul></ul><ul><ul><ul><li>if [ condition ] ; then if ( condition) then </li></ul></ul></ul><ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>else (or else if) </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>fi endif </li></ul></ul></ul></ul>
  46. 46. Shell Programming <ul><li>Conditional Processing </li></ul><ul><ul><li>for variable in list # BOURNE SHELL </li></ul></ul><ul><ul><ul><li>do </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><li>done </li></ul></ul></ul><ul><ul><li>foreach i ( list ) # CSH </li></ul></ul><ul><ul><ul><li>statements </li></ul></ul></ul><ul><ul><ul><li>end </li></ul></ul></ul>
  47. 47. Shell Programming <ul><li>Conditional Processing </li></ul><ul><ul><li>while condition #BOURNE while condition # CSH </li></ul></ul><ul><ul><ul><li>do statements </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><li>done end </li></ul></ul></ul><ul><ul><li>until condition </li></ul></ul><ul><ul><ul><li>do </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><li>done </li></ul></ul></ul>
  48. 48. Shell Programming <ul><li>Conditional Processing SH </li></ul><ul><ul><li>case variable in </li></ul></ul><ul><ul><ul><li>pattern 1) </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><ul><li>;; </li></ul></ul></ul></ul><ul><ul><ul><li>pattern 2) </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><ul><li>;; </li></ul></ul></ul></ul><ul><ul><ul><li>*) #default case </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><ul><li>;; </li></ul></ul></ul></ul><ul><ul><li>esac </li></ul></ul>
  49. 49. Shell Programming <ul><li>Conditional Processing CSH </li></ul><ul><ul><li>switch ( variable ) </li></ul></ul><ul><ul><ul><li>case pattern 1: </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><ul><li>breaksw </li></ul></ul></ul></ul><ul><ul><ul><li>pattern 2: </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><ul><li>breaksw </li></ul></ul></ul></ul><ul><ul><ul><li>default : </li></ul></ul></ul><ul><ul><ul><ul><li>statements </li></ul></ul></ul></ul><ul><ul><ul><ul><li>breaksw </li></ul></ul></ul></ul><ul><ul><li>endsw </li></ul></ul>
  50. 50. Shell Programming <ul><li>Argument Handling </li></ul><ul><ul><li>The shell defines a method of handling arguments: </li></ul></ul><ul><ul><ul><li>$$ is the process id of the shell program </li></ul></ul></ul><ul><ul><ul><li>The name of the shell program itself is $0 </li></ul></ul></ul><ul><ul><ul><li>Argument variables are numbered in the order passed: </li></ul></ul></ul><ul><ul><ul><ul><li>$1 $2 $3 $4 ... </li></ul></ul></ul></ul><ul><ul><ul><li>$* and $@ access all variables </li></ul></ul></ul><ul><ul><ul><ul><li>$* lumps all variables into a single string </li></ul></ul></ul></ul><ul><ul><ul><ul><li>$@ keeps variables as separate entities </li></ul></ul></ul></ul><ul><ul><ul><li>$# is set to the argument count </li></ul></ul></ul>
  51. 51. Shell Programming <ul><li>Argument Handling </li></ul><ul><ul><li>When you pass an argument to a shell script how do you access it? For example : </li></ul></ul><ul><ul><ul><li>/etc/init.d/lab8 start </li></ul></ul></ul><ul><ul><li>In the example above we can test $1 to see if it is “start” or “stop” and continue accordingly. </li></ul></ul>
  52. 52. Shell Programming <ul><li>I/O Redirection </li></ul><ul><ul><li>Input </li></ul></ul><ul><ul><ul><li>< file -- causes shell to read from file </li></ul></ul></ul><ul><ul><ul><li><&n -- causes shell to read from file descriptor n </li></ul></ul></ul><ul><ul><ul><li><&- -- causes shell to close standard input </li></ul></ul></ul><ul><ul><ul><li><tag -- causes shell to read up to tag </li></ul></ul></ul><ul><ul><ul><li><~tag -- same as <tag, but discard white space </li></ul></ul></ul><ul><ul><li>Output </li></ul></ul><ul><ul><ul><li>> file -- causes shell to write to file </li></ul></ul></ul><ul><ul><ul><li>>>file -- causes shell to append to file </li></ul></ul></ul><ul><ul><ul><li>>&n -- causes shell to write to file descriptor n </li></ul></ul></ul><ul><ul><ul><li>>&- -- causes shell to close standard output </li></ul></ul></ul><ul><ul><ul><li>| -- pipe the output of command and feed it to next command </li></ul></ul></ul>
  53. 53. Shell Programming <ul><li>I/O Redirection </li></ul><ul><ul><li>echo $1 $2 $3 > $4 # overwrite $4 </li></ul></ul><ul><ul><li>echo $1 $2 $3 >> $4 # append to $4 </li></ul></ul><ul><ul><li>10 03 * * * cd /tmp ; find . ! -name . -type d -mtime +1 -exec rmdir {} ; >/dev/null 2>&1 </li></ul></ul><ul><ul><li>ps -ef | grep curt </li></ul></ul><ul><ul><li>read answer </dev/tty </li></ul></ul><ul><ul><li>cat < EOF </li></ul></ul><ul><ul><ul><li>FATAL ERROR: $SPOOL/etc/setup.cache is missing! </li></ul></ul></ul><ul><ul><ul><li>The file $SPOOL/etc/setup.cache is not present. This </li></ul></ul></ul><ul><ul><ul><li>probably means the machine has not been setup using the faxsetup(1M) </li></ul></ul></ul><ul><ul><ul><li>command. Read the documentation on setting up HylaFAX before you </li></ul></ul></ul><ul><ul><ul><li>startup a server system. </li></ul></ul></ul><ul><ul><ul><li>EOF </li></ul></ul></ul>
  54. 54. Shell Programming <ul><li>Other Shell Features </li></ul><ul><ul><li>The : command is a null command. </li></ul></ul><ul><ul><li>Shells allow I/O operations: </li></ul></ul><ul><ul><ul><li>echo “Enter your afsid: “ </li></ul></ul></ul><ul><ul><ul><ul><li>read AFSID </dev/tty </li></ul></ul></ul></ul><ul><ul><ul><li>cat file.dat | while read x y z </li></ul></ul></ul><ul><ul><ul><ul><li>>do echo $x $y $z ; done </li></ul></ul></ul></ul>
  55. 55. Shell Programming <ul><li>Other Shell Features </li></ul><ul><ul><li>All shells provide a means of assigning temporary filenames for their use: </li></ul></ul><ul><ul><ul><li>sort -u +1 -2 $* |tr -d 0-9 >/tmp/$0$$ </li></ul></ul></ul><ul><ul><li>All shells provide a means of catching interrupts for cleanup processing: </li></ul></ul><ul><ul><ul><li>trap ‘command arguments’ signal </li></ul></ul></ul><ul><ul><li>All shells provide a means of debugging shell programs (-x). </li></ul></ul><ul><ul><li>All shells allow for those dreaded comments: </li></ul></ul><ul><ul><ul><li># rest of line is a comment </li></ul></ul></ul>
  56. 56. Shell Programming <ul><li>Other Shell Features </li></ul><ul><ul><li>The set command allows you to set $1 through $n to the words given as its arguments: </li></ul></ul><ul><ul><ul><li>who -r </li></ul></ul></ul><ul><ul><ul><ul><li>. run-level 3 Nov 21 09:13 2 0 S </li></ul></ul></ul></ul><ul><ul><ul><li>set `who -r` </li></ul></ul></ul><ul><ul><ul><ul><li>echo $6 </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>09:13 </li></ul></ul></ul></ul></ul>
  57. 57. Shell Programming <ul><li>Other Shell features </li></ul><ul><ul><li>The shells also provide a utility to execute arguments. </li></ul></ul><ul><ul><li>The eval command executes commands generated by multiple levels of indirection: </li></ul></ul><ul><ul><ul><li>$ a=c ; b=m ; c=d ; cmd=date </li></ul></ul></ul><ul><ul><ul><ul><li>$ eval $`echo $a$b$c` </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Fri Nov 21 09:18:45 EST 1997 </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>$ $cmd </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Fri Nov 21 09:18:49 EST 1997 </li></ul></ul></ul></ul></ul>
  58. 58. Shell Programming <ul><li>#!/bin/sh </li></ul><ul><li>#ident &quot;@(#)nfs.client 1.8 93/07/07 SMI&quot; </li></ul><ul><li>if [ ! -d /usr/bin ] </li></ul><ul><li>then # /usr not mounted </li></ul><ul><li>exit </li></ul><ul><li>fi </li></ul><ul><li>killproc() { # kill the named process(es) </li></ul><ul><li>pid=`/usr/bin/ps -e | </li></ul><ul><li> /usr/bin/grep $1 | </li></ul><ul><li> /usr/bin/sed -e 's/^ *//' -e 's/ .*//'` </li></ul><ul><li>[ &quot;$pid&quot; != &quot;&quot; ] && kill $pid </li></ul><ul><li>} </li></ul>
  59. 59. Shell Programming <ul><li># Start/stop processes required for client NFS </li></ul><ul><li>case &quot;$1&quot; in </li></ul><ul><li>'start') </li></ul><ul><li>if [ -x /usr/lib/nfs/statd -a -x /usr/lib/nfs/lockd ] then </li></ul><ul><li>/usr/lib/nfs/statd > /dev/console 2>&1 </li></ul><ul><li>/usr/lib/nfs/lockd > /dev/console 2>&1 </li></ul><ul><li>fi </li></ul><ul><li># obtain kerberos ticket before attempting kerberos NFS mounts </li></ul><ul><li>/usr/bin/sed -e 's/#.*//' /etc/vfstab | </li></ul><ul><li> /usr/bin/grep kerberos >/dev/null 2>&1 </li></ul><ul><li>if [ $? = 0 -a -x /usr/bin/ksrvtgt -a -x /usr/bin/uname ] </li></ul><ul><li>then </li></ul><ul><li> HNAME=`/usr/bin/uname -n` </li></ul><ul><li> WARN=`/usr/bin/ksrvtgt root $HNAME 2>&1` </li></ul><ul><li> if [ $? != 0 ] then </li></ul><ul><li> echo &quot;Warning: cannot get kerb ticket for root.$HNAME&quot; </li></ul><ul><li> echo &quot; ksrvtgt error: $WARN&quot; </li></ul><ul><li> fi </li></ul><ul><li>fi </li></ul>
  60. 60. Shell Programming <ul><li>/sbin/mountall -F nfs </li></ul><ul><li>/sbin/mountall -F cachefs </li></ul><ul><li>/sbin/swapadd </li></ul><ul><li>;; </li></ul><ul><li>'stop') </li></ul><ul><li>killproc lockd </li></ul><ul><li>killproc statd </li></ul><ul><li>/sbin/umountall -F cachefs </li></ul><ul><li>/sbin/umountall -F nfs </li></ul><ul><li>;; </li></ul><ul><li>*) </li></ul><ul><li>echo &quot;Usage: /etc/init.d/nfs.client { start | stop }&quot; </li></ul><ul><li>;; </li></ul><ul><li>esac </li></ul>
  61. 61. Shell Programming <ul><li>Shell Programming </li></ul><ul><ul><li>All shells provide us with programming constructs (variables, flow control, conditional tests, ...). with these facilities we can write programs to make our lives easier. </li></ul></ul><ul><ul><li>When writing shell programs we also have easy access to all of the system utilities. </li></ul></ul><ul><ul><li>By combining shell code and utility calls it is easy to write very powerful tools! </li></ul></ul>
  62. 62. Find Command <ul><li>Cleanup Tools </li></ul><ul><ul><li>Find </li></ul></ul><ul><ul><ul><li>The find command can be a very useful tool for administrators to police disk usage. </li></ul></ul></ul><ul><ul><ul><ul><li>You can find and remove cruft from system directories (core files, a.out, .o files, junk in /tmp). </li></ul></ul></ul></ul><ul><ul><ul><ul><li>You can locate files owned by a particular user. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>You can locate files with particular permissions. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>You can locate files containing a specific string. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>You can locate files which have been modified since a particular date. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>You can locate files of a particular size. </li></ul></ul></ul></ul>
  63. 63. Find Command <ul><li>WARNING!!! </li></ul><ul><li>Removing files from a users directory may be illegal! </li></ul><ul><ul><li>Files in a user’s account are considered property of the user unless otherwise stated in site policies. </li></ul></ul><ul><ul><li>User must be informed that files in their account are not personal/private. </li></ul></ul><ul><li>Files in /tmp are exempt from this, as /tmp is a Public file system. Other public file systems (/scratch, /usr/tmp, /var/tmp, /pub) are also typically considered exempt. </li></ul>
  64. 64. Find Command <ul><li>Find </li></ul><ul><ul><li>The generic invocation of find is: </li></ul></ul><ul><li>find pathlist expression </li></ul><ul><ul><ul><li>The pathlist is a directory [or list of directories and/or files] you wish to search. </li></ul></ul></ul><ul><ul><ul><li>The expression is a directive which tells find what to look for, and what to do when it finds what it is looking for. </li></ul></ul></ul>
  65. 65. Find Command <ul><li>Find pathlist expression </li></ul><ul><ul><li>Some of the more useful expressions for the find command include: </li></ul></ul><ul><ul><ul><li>name - search for files called name </li></ul></ul></ul><ul><ul><ul><li>atime n - search for files which were accessed n days ago. </li></ul></ul></ul><ul><ul><ul><li>ls - print the current path and related statistics </li></ul></ul></ul><ul><ul><ul><li>mtime n - search for files which were modified n days ago. </li></ul></ul></ul><ul><ul><ul><li>exec command - execute command when the file is found. </li></ul></ul></ul><ul><ul><ul><li>print - causes the filename to be printed when found. </li></ul></ul></ul><ul><ul><ul><li>type c - search for files of type “c” (file, directory, link, device files, fifo, socket) </li></ul></ul></ul><ul><ul><ul><li>fstype type - search for filesystems of type </li></ul></ul></ul>
  66. 66. Find Command <ul><li>Find examples </li></ul><ul><ul><li>Suppose you want to make a tape of your files before you leave Notre Dame. You want to keep your source code, your mail addresses, and other files, but you do not want to carry along your a.out’s, any of the .o files, or any netscape cache files. </li></ul></ul><ul><li>find $HOME (-name a.out -o -name ‘*.o’ -o -name ‘cache*’) -exec rm{} ; </li></ul><ul><ul><li>If you were a system administrator who wanted to remove the same files from all directories on the system you could modify the command a little: </li></ul></ul><ul><li>find / (-name a.out -o -name ‘*.o’ -o -name ‘cache*’) -exec rm{} ; </li></ul><ul><li>SEE WARNING!! </li></ul>
  67. 67. Find Command <ul><li>Find examples </li></ul><ul><ul><li>Suppose you had a series of users complain that their .login files had been modified yesterday by someone other than themselves. If you wanted to find all instances of modified .login files you could use: </li></ul></ul><ul><ul><ul><li>find /home -name .login -mtime +1 -print </li></ul></ul></ul><ul><ul><li>Similarly, suppose you knew that someone had compromised your system and replaced the shell program with their own version. You found the string “floobydust” in their version of the shell program, so you want to look and find out if they left other copies of their program on the system: </li></ul></ul><ul><ul><ul><li>find / -type f -exec grep -l “floobydust” {} ; </li></ul></ul></ul>
  68. 68. Find Command <ul><li>Find </li></ul><ul><ul><li>How do the security tools we discussed (SATAN, COPS) find suid programs? </li></ul></ul><ul><ul><ul><li>find / (-perm 4755 -o -perm 2755 -o -perm 1755) -print </li></ul></ul></ul><ul><ul><li>Suppose an employee (UID 666) leaves the company and another employee (UID 999) takes over the code managed by the old employee. You have been asked to make sure the new employee has full permissions on the files the old employee owned: </li></ul></ul><ul><ul><ul><li>find / -user 666 -exec chown 999 {} ; </li></ul></ul></ul><ul><ul><ul><li>SEE WARNING!!! </li></ul></ul></ul>
  69. 69. Find Command <ul><li>Find </li></ul><ul><ul><li>Find can also be a useful tool for forgetful users (or professors) who want to find where they left something. </li></ul></ul><ul><ul><ul><li>find $HOME -name homework6 -print </li></ul></ul></ul><ul><ul><ul><li>find $HOME -exec grep -l “lab combination” {} ; </li></ul></ul></ul><ul><ul><li>As a system administrator you need to think about a couple of questions: </li></ul></ul><ul><ul><ul><li>How often should these housekeeping functions be performed? </li></ul></ul></ul><ul><ul><ul><li>Is there a method that will automate these functions without requiring my intervention. </li></ul></ul></ul>
  70. 70. Find Command <ul><li>Cleanup Tools </li></ul><ul><ul><li>Many places create special scripts to locate and remove “cruft” from the system. </li></ul></ul><ul><ul><li>Skulker </li></ul></ul><ul><ul><li>SpaceGripe </li></ul></ul><ul><ul><li>DiskClean </li></ul></ul><ul><ul><ul><li>Most are based on find. </li></ul></ul></ul>
  71. 71. AWK <ul><li>AWK </li></ul><ul><ul><li>AWK is a powerful pattern matching and scanning tool used by system administrators. </li></ul></ul><ul><ul><ul><li>AWK is typically used to scan an input string, grab certain portions of the string, then output the information in another format. </li></ul></ul></ul><ul><ul><li>AWK was developed by Alfred A ho, Peter W einberger, and Brian K ernighan (hence the name AWK ). </li></ul></ul><ul><ul><li>There are several variants of awk: standard awk (awk), Gnu awk (gawk), and new awk (nawk) are the most common. </li></ul></ul>
  72. 72. AWK <ul><li>AWK </li></ul><ul><ul><li>Awk allows us to: </li></ul></ul><ul><ul><ul><li>view a text file as if it were made up of records and fields in a textual database </li></ul></ul></ul><ul><ul><ul><li>Use variables to manipulate the database </li></ul></ul></ul><ul><ul><ul><li>Use loops and conditional processing </li></ul></ul></ul><ul><ul><ul><li>Generate formatted reports </li></ul></ul></ul><ul><ul><li>Nawk adds several capabilities: </li></ul></ul><ul><ul><ul><li>Define functions </li></ul></ul></ul><ul><ul><ul><li>Execute UNIX commands </li></ul></ul></ul><ul><ul><ul><li>Process the results of UNIX commands </li></ul></ul></ul><ul><ul><ul><li>Better command line argument processing </li></ul></ul></ul><ul><ul><ul><li>Work with multiple input streams </li></ul></ul></ul>
  73. 73. AWK <ul><ul><li>Generic invocations: </li></ul></ul><ul><ul><ul><li>awk ‘instructions’ files </li></ul></ul></ul><ul><ul><ul><li>awk -f ‘script’ files </li></ul></ul></ul><ul><ul><li>If you have a file (filename list) with the following entries: </li></ul></ul><ul><ul><ul><li>John Doe, 999 Sorin Hall </li></ul></ul></ul><ul><ul><ul><li>Jeff Walker, 234 Cushing Hall </li></ul></ul></ul><ul><ul><ul><li>Amy Vanderbilt, 876 Hesburgh Library </li></ul></ul></ul><ul><ul><li>awk ‘{print $1}’ list </li></ul></ul><ul><ul><ul><li>will output : </li></ul></ul></ul><ul><ul><ul><ul><li>John </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Jeff </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Amy </li></ul></ul></ul></ul>
  74. 74. AWK <ul><li>If you wanted to make mail labels from the file “list” you could do: </li></ul><ul><ul><li>awk -F, ‘{print $1 print $2 print “Notre Dame, IN 46656”}’ </li></ul></ul><ul><ul><li>The output would be: </li></ul></ul><ul><ul><ul><li>John Doe </li></ul></ul></ul><ul><ul><ul><li>999 Sorin Hall </li></ul></ul></ul><ul><ul><ul><li>Notre Dame, IN 46656 </li></ul></ul></ul><ul><ul><ul><li>Jeff Walker </li></ul></ul></ul><ul><ul><ul><li>234 Cushing Hall </li></ul></ul></ul><ul><ul><ul><li>Notre Dame, IN 46656 </li></ul></ul></ul><ul><ul><ul><li>Amy Vanderbilt </li></ul></ul></ul><ul><ul><ul><li>876 Hesburgh Library </li></ul></ul></ul><ul><ul><ul><li>Notre Dame, IN 46656 </li></ul></ul></ul>
  75. 75. AWK <ul><li>Awk </li></ul><ul><ul><li>You can set shell variables with the output of the awk command. </li></ul></ul><ul><ul><ul><li>input: name address phone </li></ul></ul></ul><ul><ul><ul><ul><li>John Doe, 123 Sorin Hall, 634-4444 </li></ul></ul></ul></ul><ul><ul><ul><ul><li>name=`awk ‘{print $1}’ input` </li></ul></ul></ul></ul><ul><ul><ul><ul><li>echo $name </li></ul></ul></ul></ul><ul><ul><ul><ul><li>John </li></ul></ul></ul></ul>
  76. 76. AWK <ul><li>Awk </li></ul><ul><ul><li>Awk does a lot of the “programming” for you. </li></ul></ul><ul><ul><li>An AWK program has three parts: </li></ul></ul><ul><ul><ul><li>An optional “begin” routine which is executed once before the main loop. </li></ul></ul></ul><ul><ul><ul><li>The main loop. </li></ul></ul></ul><ul><ul><ul><li>An optional “end” routine that is executed after all of the main loop code is done. </li></ul></ul></ul><ul><ul><li>AWK provides the main loop. When you call AWK is goes into a loop reading input and providing it to your code. </li></ul></ul><ul><ul><ul><li>You do not have to write code to read the just provide pattern matching statements. </li></ul></ul></ul>
  77. 77. AWK <ul><li>AWK </li></ul><ul><ul><li>AWK acts on patterns.. </li></ul></ul><ul><ul><ul><li>Your code is a series of patterns and actions . </li></ul></ul></ul><ul><ul><ul><li>When AWK detects the pattern, it executes the action. </li></ul></ul></ul><ul><ul><ul><li>For example: </li></ul></ul></ul><ul><ul><ul><ul><li>awk ‘{print $1}’ file </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>tells awk to print the first field of the input line. </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>$1 is the pattern </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>print $1 is the action </li></ul></ul></ul></ul></ul>
  78. 78. AWK <ul><li>AWK Regular Expressions </li></ul><ul><ul><ul><ul><li>/^$/ {print “Input is a blank line” } </li></ul></ul></ul></ul><ul><ul><ul><ul><li>/[0-9]+$/ {print “Input is an integer” } </li></ul></ul></ul></ul><ul><ul><ul><ul><li>/[A-z]+/ {print “Input is a string” } </li></ul></ul></ul></ul><ul><ul><ul><li>AWK works with internal variables much like the Bourne shell does: </li></ul></ul></ul><ul><ul><ul><ul><li>x = 1 assigns the value 1 to x </li></ul></ul></ul></ul><ul><ul><ul><ul><li>z = “Hello” assigns the string “Hello” to z </li></ul></ul></ul></ul><ul><ul><ul><ul><li>w = $1 assigns the value of the first field of the input to the variable w. </li></ul></ul></ul></ul>
  79. 79. AWK <ul><li>AWK </li></ul><ul><ul><ul><li>If we want to perform mathematical operations on the variables in an AWK script, the following operators are available: </li></ul></ul></ul><ul><ul><ul><ul><li>+, -, *, /, %, ^ </li></ul></ul></ul></ul><ul><ul><ul><li>AWK also has a group of assignment operators: </li></ul></ul></ul><ul><ul><ul><ul><li>+=, ++, --, -=, *=, /=, %= </li></ul></ul></ul></ul><ul><ul><ul><ul><li>A simple script to count blank lines in a file: </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>awk ‘/^$/ { ++x} END { print x }’ file </li></ul></ul></ul></ul></ul>
  80. 80. AWK <ul><li>AWK </li></ul><ul><ul><ul><ul><li>AWK has some handy built-in variables: </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>FS is the input field separator. (Default space or tab) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>OFS is the output field separator </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>RS is the input record separator (Default newline) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>FILENAME is the name of the input file </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>NR is the number of input records read </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>AWK has relational operators and Boolean operators: </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li><, >, <=, >=, ==, !=, ~, !~ </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>||, && </li></ul></ul></ul></ul></ul>
  81. 81. AWK <ul><li>AWK </li></ul><ul><ul><ul><li>So what is all of this useful for (besides lab9)? </li></ul></ul></ul><ul><ul><ul><ul><li>Suppose your site has a shell called the new shell (nsh). You are discontinuing support for this shell because it is very similar to the zork shell. You need to scan the password file, convert all accounts with nsh to use zork, and you need to keep track of how many users you convert over to the zork shell. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>The password file consists of 7 colon separated fields. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>The users shell is held in the 7th field of the input. </li></ul></ul></ul></ul>
  82. 82. AWK <ul><li>awk '{ </li></ul><ul><li># Set delimiters </li></ul><ul><li>BEGIN { FS=&quot;:&quot; ; OFS=&quot;:&quot; } </li></ul><ul><li>  </li></ul><ul><li># if shell ($7) is /bin/nsh convert it to /bin/zork </li></ul><ul><li>{ if ( $7 == &quot;/bin/nsh&quot; ) { </li></ul><ul><li>nnsh++ ; print $1,$2,$3,$4,$5,$6,&quot;/bin/zork&quot; } </li></ul><ul><li>  </li></ul><ul><li># Otherwise don’t change the line </li></ul><ul><li> else </li></ul><ul><li>{ print $1,$2,$3,$4,$5,$6,$7 } </li></ul><ul><li>  </li></ul><ul><li># Print number of users assimilated </li></ul><ul><li>} </li></ul><ul><li>END { print nnsh “ users converted from /bin/nsh to /bin/zork” } </li></ul><ul><li>}' </li></ul>
  83. 83. AWK <ul><li>Another stupid example: </li></ul><ul><ul><li>You work for a Veterinarian. The Dr. wants to send a postcard to the owners of all dogs informing them of a new parvo-virus immunization law. The information format in the Dr’s database is: </li></ul></ul><ul><ul><ul><li>Species,Owner Name, Address, City, State, Zip, Phone, Animal’s Name, Animals Age, Immunized,Last Visit, Notes </li></ul></ul></ul>
  84. 84. AWK <ul><li>nawk -F, '{ </li></ul><ul><li># If canine and not immunized, print postcard, else ignore the line </li></ul><ul><li>if ( $1 == &quot;Canine&quot; && $10 == &quot;No&quot; ) </li></ul><ul><li>{print $2 ; print $3 ; print $4 &quot;, &quot; $5 &quot; &quot; $6 </li></ul><ul><li>print &quot;&quot; </li></ul><ul><li>print &quot;Dear &quot; $2 &quot;,&quot; </li></ul><ul><li>print &quot;A recently enacted state law requires that all dog&quot; ; </li></ul><ul><li>print &quot;owners get their animals immunized for parvo-virus &quot; ; </li></ul><ul><li>print &quot;as soon as possible. Our records indicate that &quot; $8 </li></ul><ul><li>print &quot;needs to be immunized. Please call our office to &quot; ; </li></ul><ul><li>print &quot;make an appointment as soon as possible.&quot; </li></ul><ul><li>print &quot;&quot; </li></ul><ul><li>print &quot;&quot; </li></ul><ul><li>} </li></ul><ul><li>}' </li></ul>
  85. 85. Other Useful Tools <ul><li>Some other tools you might want to experiment with: </li></ul><ul><ul><li>cut - cut bytes from the input. </li></ul></ul><ul><ul><ul><li>first=Curt last=Freeland </li></ul></ul></ul><ul><ul><ul><li>f1=`echo $first | cut -b1` f1=C </li></ul></ul></ul><ul><ul><ul><li>ln=`echo $last | cut -b1-7` ln=Freelan </li></ul></ul></ul><ul><ul><ul><li>try=curt:x:1234:40:GECOS:/home/curt:/bin/csh </li></ul></ul></ul><ul><ul><ul><li>found=`echo $try | cut -d: -f1` found=curt </li></ul></ul></ul>
  86. 86. Other Useful Tools <ul><li>It’s pretty easy to convert strings from upper to lower case in C. How would you do the same thing in shell scripts? </li></ul><ul><ul><li>The tr command translates (or transforms) characters on the input stream into different characters on the output stream. </li></ul></ul><ul><ul><li>echo $f1$ln </li></ul></ul><ul><ul><ul><li>CFreelan </li></ul></ul></ul><ul><ul><li>lname=`echo $f1$ln | tr “[:upper:]” “[:lower:]”` </li></ul></ul><ul><ul><li>echo $lname </li></ul></ul><ul><ul><ul><li>cfreelan </li></ul></ul></ul>
  87. 87. Other Useful Tools <ul><li>There are many variants of the grep command. The “fgrep” command has an interesting flag: </li></ul><ul><ul><li>Consider: </li></ul></ul><ul><ul><ul><li>a password file contains the following entries: </li></ul></ul></ul><ul><ul><ul><ul><li>curt:x:1234:40:GECOS:/home/curt:/bin/csh </li></ul></ul></ul></ul><ul><ul><ul><ul><li>curt1:x:1235:40:GECOS:/home/curt1:/bin/csh </li></ul></ul></ul></ul><ul><ul><ul><ul><li>curt2:x:1236:40:GECOS:/home/curt2:/bin/csh </li></ul></ul></ul></ul><ul><ul><ul><li>grep curt /etc/passwd will return all three lines! </li></ul></ul></ul><ul><ul><ul><li>fgrep -x curt /etc/passwd </li></ul></ul></ul><ul><ul><ul><ul><li>returns nothing! The -x flag tells grep that it has to match the entire input string exactly. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>awk -F: ‘{print $1}’ /etc/passwd | fgrep -x curt </li></ul></ul></ul></ul>
  88. 88. Other Useful Tools <ul><li>The Streams editor (sed) is a very handy tool. sed uses regular expression matching to convert the input stream to a new output stream one line at a time. </li></ul><ul><ul><li>SED works on a temporary buffer called the pattern space - it does not change the original input file. </li></ul></ul><ul><ul><li>SED uses the same commands as the ex and vi editor (when run in line mode). </li></ul></ul><ul><ul><ul><li>sed ‘1,3d’ file </li></ul></ul></ul><ul><ul><ul><ul><li>Deletes lines 1-3 from file and displays result on screen. </li></ul></ul></ul></ul><ul><ul><ul><li>sed ‘[Jj]ohn/p’ datafile </li></ul></ul></ul><ul><ul><ul><ul><li>Prints any lines from the input file that contain John or john </li></ul></ul></ul></ul>
  89. 89. Other Useful Tools <ul><li>#!/bin/sh </li></ul><ul><li>#ident &quot;@(#)nfs.client 1.8 93/07/07 SMI&quot; </li></ul><ul><li>if [ ! -d /usr/bin ] </li></ul><ul><li>then # /usr not mounted </li></ul><ul><li>exit </li></ul><ul><li>fi </li></ul><ul><li>killproc() { # kill the named process(es) </li></ul><ul><li>pid=`/usr/bin/ps -e | </li></ul><ul><li> /usr/bin/grep -w $1 | </li></ul><ul><li> /usr/bin/sed -e 's/^ *//' -e 's/ .*//'` </li></ul><ul><li>[ &quot;$pid&quot; != &quot;&quot; ] && kill $pid </li></ul><ul><li>} </li></ul>
  90. 90. Summary <ul><li>Automating repetitive tasks allows the administrator time to get other things done. </li></ul><ul><li>The administrator needs to understand what can be automated, and what cannot be automated. </li></ul><ul><li>The administrator should understand how to write shell scripts using the command line utilities used to perform daily tasks. </li></ul>