Unix tips and tricks
Upcoming SlideShare
Loading in...5
×
 

Unix tips and tricks

on

  • 1,609 views

Useful UNIX and Linux tips and tricks (:

Useful UNIX and Linux tips and tricks (:

Statistics

Views

Total Views
1,609
Views on SlideShare
1,588
Embed Views
21

Actions

Likes
2
Downloads
34
Comments
0

2 Embeds 21

http://www.techgig.com 16
https://twitter.com 5

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Unix tips and tricks Unix tips and tricks Presentation Transcript

  • Unix Tips & Tricks Aleksandar Bilanovic Ericsson Telekommunikation GmbH
  • MotivationUNIX interface is made of simple commands that can becombined into useful programs. Number of combination isendlessOur infrastructure and product is on UNIX (Linux) platformWhat this presentation isThis presentation is collection of few useful Unix tips and tricksWhat this presentation is NOTThis presentation is not comprehensive guide to UnixThis presentation is not how-to
  • TipShell● Using control operators. ● Good habit in interactive shell usage ● MUST when writing shell scripts $ cd /dir && cmd $ cd /dir || mkdir /dir● Using mkdir –p to create directory structure ● Bad habit $ mkdir /dir $ cd /dir $ mkdir dir1 $ mkdir dir2 $ cd dir1 $ mkdir a $ mkdir b $ mkdir c ● Good habit $ mkdir -p /dir/{dir1/{a,b,c},dir2}
  • TipShell● Breaking long input lines by using ● Bad habit $ find /bin /lib /sbin /usr/sbin /usr/lib /usr/bin /disk2 /disk1 -perm / 4000 –exec ls –la {} ; ● Good habit $ find /bin > /lib > /sbin > /usr/sbin > /usr/lib > /usr/bin > /disk2 > /disk1 > -perm /4000 > –exec ls –la {} ;● Command substitution $ mkdir $(basename $PWD) $ mkdir `basename $PWD`
  • TipShell● History helpers ● bck-i-search $ cd /tmp bck-i-search: cd /_ ● history command editing $ fc $ fc –l ● $_ - last argument $ ls -l /usr/home/dmr/somefile $ vi $_ $ pwd /home/bilke $ cd – ¡ cd /tmp $ cd - ~ $ pwd /home/bilke
  • TipShell● for loop ● using shell built in $ for i in {1..10}; do echo $i; done # bash $ for i in {01..10}; do echo $i; done # padding with zeroes, only in ZSH! ● using external program (not recommended – slow and not portable) $ for i in $(seq 1 10); do echo $i; done $ for i in $(seq –w 1 10); do echo $i; done● greping /etc/passwd file by columns using awk(1) $ awk -F":" $3 >= 1000 && $7 != "/bin/false" /etc/passwd $ ls -l| grep Dec -rw-r--r-- 1 bilke bilke 0 Dec 2 17:45 archive.tar -rw-r--r-- 1 bilke bilke 0 Jul 7 2001 December_Report.pdf -rw-r--r-- 1 bilke bilke 0 Dec 2 17:45 README $ ls -l| awk $6 == "Dec" -rw-r--r-- 1 bilke bilke 0 Dec 2 17:45 archive.tar -rw-r--r-- 1 bilke bilke 0 Dec 2 17:45 README
  • TipTerm● Term screwed up – reset(1) is your friend $ cat /bin/ls #¾xºßÀct{/ Ð¥ûdßÿÿÿé|F¤ëppðqÑÄêí5¸ÁXÀ!9ùí.oa Ø =ðÉàü`ñ÷9Ï reset $● script(1) - make typescript of terminal session $ script $ ls typescript $ exit Script done, file is typescript $ cat typescript Script started on Wed 02 Dec 2009 05:01:13 PM CET $ ls typescript $ Script done on Wed 02 Dec 2009 05:01:36 PM CET $
  • TipTerm● Need to run command in background with terminal detached?● Use screen(1) & nohup(1) ● screen - screen manager with VT100/ANSI terminal emulation ● nohup - run a command immune to hangups, with output to a non-tty ● Difference – while nohup is detaching command from terminal and redirecting output to file, screen is detaching itself (not command) from terminal.
  • TrickRemove files starting with dash● File which name starts with – (dash) can be “hard” to remove: $ > -i # Ctrl + D $ ls –l total 0 -rw-r--r-- 1 bilke users 0 2009-12-02 01:15 –i $ rm –i rm: missing operand Try `rm --help for more information. $ rm -i rm: missing operand Try `rm --help for more information. $ rm ‘-i’ rm: missing operand Try `rm --help for more information. $ rm “-i” rm: missing operand Try `rm --help for more information. $ rm “-i” rm: cannot remove `-i: No such file or directory
  • TrickRemove files starting with dash● A - Delimit the option list. Later arguments, if any, are treated as operands even if they begin with `-. $ rm -- -i $ ls –l total 0● B – Find out file’s inode number and delete it using find $ ls –li total 0 12180 -rw-r--r-- 1 bilke users 0 2009-12-02 01:25 –i $ find . -inum 12180 -exec rm {} ; $ ls –l total 0● C – Using absolute or relative path $ rm ./-i $ ls –l total 0
  • TrickRemoving open files● File is not deleted until last hard link is deleted and last FD is closed. $ less /tmp/file $ rm /tmp/file # from another shell $ pgrep less 17395 $ ls –l /proc/17395/fd total 0 lrwx------ 1 bilke users 64 2009-12-02 01:02 0 -> /dev/pts/0 lrwx------ 1 bilke users 64 2009-12-02 01:02 1 -> /dev/pts/0 lrwx------ 1 bilke users 64 2009-12-02 01:02 2 -> /dev/pts/0 lr-x------ 1 bilke users 64 2009-12-02 01:02 3 -> /dev/tty lr-x------ 1 bilke users 64 2009-12-02 01:02 4 -> /tmp/file (deleted) $ cp /proc/17395/fd/4 /tmp/file # or $ stat -L /proc/17395/fd/4 # (get inode and user FS specific tools to restore in place – thanks Dr. Reinhard)● Potential problems with rotation of logs! ● file reference removed from directory, but process is still writing into it – usually until all space on drive is consumed. ● Symptom - discrepancy between df and du.
  • TipI/O Redirection● Redirecting I/O $ ls /tmp /lala >stdout_file 2>stderr_file $ ls /tmp /lala > both_out_and_err 2>&1 $ ls /tmp /lala >> out_append $ ls /tmp /lala 2>> err_append $ cmd M>N (M – file descriptor (defaults to 1); N – file name) $ cmd M>&N (M – file descriptor (defualts to 1); N – another FD) $ tr “a-z” “A-Z” < /etc/services● Can be placed anywhere in command line $ echo a b >c $ echo >c a b $ >c echo a b $ </var/log/messages grep bar
  • TrickI/O Redirection● Here document $ tr “a-z” “A-Z” << EOF > one two three > uno dos tres > EOF ONE TWO THREE UNO DOS TRES ● Escape special characters $ cat << EOF > Work dir is $PWD > EOF Work dir is /home/bilke $ cat << “EOF” > Work dir is $PWD > EOF Work dir is $PWD
  • TipI/O Redirection● Emptying file using I/O redirection $ cat /dev/null >file $ echo -n >file $ :> file● Redirecting shell I/O $ exec 2> err_file $ ls /tmp /lala /tmp: lost+found/ $ cat err_file ls: cannot access /lala: No such file or directory $ echo 1234567890 > File # Write string to "File“. $ exec 3<> File # Open "File“ for rw and assign fd 3 to it. $ read -n 4 <&3 # Read only 4 characters. $ echo -n . >&3 # Write a decimal point there. $ exec 3>&- # Close fd 3. $ cat File 1234.67890
  • TipI/O Redirection● tee (1) - Copy standard input to file(s), and also to standard output. $ make all | tee make_out rm -f mksyntax gcc -DPROGRAM="bash" -DCONF_HOSTTYPE="i686" -DCONF_OSTYPE="linux-gnu" -DCONF_MACHTYPE="i686-pc- linux-gnu" -DCONF_VENDOR="pc" -DLOCALEDIR="/usr/local/share/locale" -DPACKAGE="bash" -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -g -o mksyntax ./mksyntax.c rm -f syntax.c ./mksyntax -o syntax.c /bin/sh ./support/mkversion.sh -b -S . -s release -d 3.2 -o newversion.h …
  • TipInput processing● xargs(1) – build and execute command line from stdin $ find /tmp -name core -type f -print | xargs /bin/rm –f calls rm once ● ! Be careful, can’t handle long files list. Use find’s built in –exec instead (slower): calls rm as many $ find /tmp -name core -type f –exec rm –f {} ; times as file is matched (slower but safer) ● Getting compact list of users using xargs : $ cut -d: -f1 < /etc/passwd | sort | xargs echo avahi backup bilke bin bind daemon Debian-exim festival ftp games gdm gnats haldaemon identd irc libuuid list lp mail man messagebus mysql nemanja news nobody ntop ntp postfix postgres proxy root saned snmp sshd statd sync sys uucp www-data
  • TipInput processing● read(1) – shell built in reads from standard input $ ls -1 /4Mfiles_dir| head –n 600000| while read file; do rm delete first 600k -rf $file; done files from dir with 4M files ● Use I/O redirection instead of pipe (faster!) $ while read line; do echo LINE: $line; done < /etc/services good habit bad habit! $ cat /etc/services| while read line; do echo LINE: $line; do not cat files, done do not use pipe If not necessary ● read(1) can also process space/tab delimited data: egrep -v ^#|^$ /etc/protocols | while read protocol number rest ; do echo $protocol:$number; done
  • TipDealing with remote● Execute command on remote server $ ssh bilanova@siegburg “ls –la”● Execute command that requires term on remote server $ ssh –t bilanova@lublin “top”● Redirect output on remote server ls -la / | ssh bilanova@lublin cat > lublin_ls_out● Making backup on remote machine tar vcf - /data | ssh bmachine‘(cd /destdir; cat |bzip2 -9 > b.tar.bz) subshell
  • Trick& More● dd(1) - convert and copy a file ● allocate data blocks as the file grows. $ dd if=/dev/zero of=server.img oflag=direct bs=1M seek=2047 count=1Works only on systems where file systems supports sparse files (mostUNIXes) and where dd can do ftruncate(2). If available truncate(1) canbe used too.● Using tar(1) to copy files When recursive copying, cp (cp -Rip, etc.) may not be the best tool for the job. For example, cp copies hard links as separate files, which is probably not what you want. To get a true copy of a directory, try: $ tar cf - <dir> | (cd <destdir>; tar xf -)
  • Trick& More● lsof(8) – list open files ● Since tcp/udp connection is also file we can list it too: $ /usr/sbin/lsof -i:22 COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME ssh 15856 bilke 3u IPv4 1662638 TCP cunningplan.ba.uk:49111- >saga.lhs-systems.com:ssh (ESTABLISHED) ssh 29587 bilke 3u IPv4 3756273 TCP cunningplan.ba.uk:60326- >70.42.235.86:ssh (ESTABLISHED)● pstree(1) - display a tree of processes (also ps auxf)
  • ReferencesMastering Unix Shell Scripting (Randal K. Michael)http://www.ibm.com/developerworks/aix/library/au-badunixhabits.htmlhttp://www.nesbitt.ca/unix-tips.htmlhttp://sial.org/howto/shell/http://nl.ijs.si/gnusl/tex/tunix/tips/tips.htmlhttp://osxfaq.com/tips/unix-tricks/