Downloaded 79 times
![Text Manipulation (sed)Text Manipulation (sed)
5. > sed 's/[ t]*$//'
If you don't like spaces...
6. > sed 's/([`"$])/1/g'
Use escape judiciously with meta-characters
7. > sed 10 | sed 's/^/ /; s/ *(.{7,})/1/'
Handy for number alignments
8. > sed –n '1000{p;q}'
9. > sed –n '10,20p;20q'
Printing with sed
June 28, 2010 21Tusharadri Sarkar](https://image.slidesharecdn.com/aquickintroductiontolinux-130704131815-phpapp02/85/A-Quick-Introduction-to-Linux-21-320.jpg)
![Text Manipulation (tr)Text Manipulation (tr)
tr: Stands for “Translate”
> echo 'Test' | tr '[:lower:]' '[:upper:]'
Case conversion
> tr –dc '[:print:]' < /dev/urandom
Handle non printable characters
> tr –s '[:blank:]' 't' < /proc/diskstats | cut –f4
Combine with 'cut' to manipulate fields
June 28, 2010 22Tusharadri Sarkar](https://image.slidesharecdn.com/aquickintroductiontolinux-130704131815-phpapp02/85/A-Quick-Introduction-to-Linux-22-320.jpg)
![Basic NetworkingBasic Networking
netstat: Working with kernel routing table
> netstat –r
> netstat –i
> netstat –l
> netstat –tup
route: Modifying the kernel routing table
> route add [route addr] [interface]
> route delete [route addr] [interface]
June 28, 2010 26Tusharadri Sarkar](https://image.slidesharecdn.com/aquickintroductiontolinux-130704131815-phpapp02/85/A-Quick-Introduction-to-Linux-26-320.jpg)
Uploaded byTusharadri Sarkar
A Quick Introduction to Linux
This document provides an introduction to useful Linux commands. It begins with a brief history of UNIX and Linux and describes the Linux directory structure. It then discusses commands for navigating directories, searching for files, manipulating text, basic networking, system information, disk usage monitoring, and more. The goal is to compile some essential Linux commands for newcomers to learn and get comfortable with.
![Unix environment [autosaved]](https://cdn.slidesharecdn.com/ss_thumbnails/unixenvironmentautosaved-151015111256-lva1-app6892-thumbnail.jpg?width=640&height=640&fit=bounds)
A Quick Introduction to Linux
- 1. A Quick IntroductiontoA Quick Introduction to LinuxLinux (A compilation of some useful Linux commands)(A compilation of some useful Linux commands) Tusharadri Sarkar June 28, 2010 IBM June 28, 2010 1Tusharadri Sarkar
- 2. A Brief Historyof UNIXA Brief History of UNIX Bell Labs : 1970 The C programming language Support for simultaneous users Minimum effort for platform compatibility Simple, elegant, and easy* to use * Of course, we are talking about the CLI folks !! June 28, 2010 2Tusharadri Sarkar
- 3. A Brief Historyof LinuxA Brief History of Linux The GNU project - Richard Stallman 1984 MINIX - Professor Andrew Tanenbaum Linus Torvalds - inspired by MINIX April 1991 Linux 0.01 released September 1991 June 28, 2010 3Tusharadri Sarkar
- 4. Linux Directory StructureLinuxDirectory Structure June 28, 2010 4Tusharadri Sarkar
- 5. Main DirectoriesMain Directories /The parent (root) directory bin Compiled executable files (ls, grep) boot Linux Kernel images and configs etc System configuration files dev Device files for all system H/Ws home Stores home directories of users lib System library files June 28, 2010 5Tusharadri Sarkar
- 6. Main DirectoriesMain Directories usr Storesfiles and executables that all users can access var Stores files whose size might vary tmp Stores temporary files sbin Stores executables for system administrator, user can also access proc Provides Kernel windows to view system status (memory, IO etc.) June 28, 2010 6Tusharadri Sarkar
- 7. Users and GroupsUsersand Groups Each “user” is a member of at least one “group” in Linux Each user has a home directory What are there in /home and /usr/users? UNIX permissions: both users and groups > ls –l : see all permissions June 28, 2010 7Tusharadri Sarkar Note: From now on, the > at the beginning of each command represents the command prompt. The command starts after that
- 8. Notes on FilePermissionNotes on File Permission In UNIX/Linux system everything is a file File has 3 permissions : read (r), write (w) and execute (x) File has 3 sets of permissions: user – group – others For each, permissions have weights: r = 4 w = 2 x = 1 So, what does permissions 755 means ? June 28, 2010 8Tusharadri Sarkar
- 9. Alias: To makelife easy...Alias: To make life easy... Pipes |: to chain multiple commands Alias: Write commands once and execute them whenever you need An example: > alias jtest=`find <JAVA_TEST_PATH> -name *.java | grep 'Test' | sed 's/Test/TestCase/g'` What does it do? The .bashrc file: Your own set of aliases June 28, 2010 9Tusharadri Sarkar
- 10. Directory NavigationDirectory Navigation June28, 2010 10Tusharadri Sarkar pwd Present Working Directory cd Login (home) dir. of the user cd – User’s last (previous) dir. cd ~/<path> Here, ~ stands for ‘home’ cd /<path> Using absolute path cd .. One dir. level up '.' (dot) and '..' (dot dot) indicate current and parent directories, just like Windows
- 11. Creating & RemovingDirectoriesCreating & Removing Directories Check your permissions carefully: write permission – to create and remove directories Check your permissions again: not only user but group permissions also matters > mkdir <dname> : Create directory > rmdir <dname> : Remove directory June 28, 2010 11Tusharadri Sarkar
- 12. File Searching (ls)FileSearching (ls) ls: Stands for “List” > ls –a : Expose the hidden files > ls –lrt : Very useful when searching by last modification dates Use regular expressions: ls –lrt lib*.so Create alias of your own listing commands to make things easy for you again: Example: > alias l=`ls –l --color=auto` June 28, 2010 12Tusharadri Sarkar
- 13. File Searching (find)FileSearching (find) find: A very powerful command for file searching operations > find <path> -name *.txt Typical usage with regular expressions > find <path> -type d when you are after the directories!! > find <path> -type f when you are after the regular files!! June 28, 2010 13Tusharadri Sarkar
- 14. File Searching (find)FileSearching (find) find is more powerful when chained with other commands like xgrep or grep For example: > find <path> -name "*.*" -exec grep "<pattern>" '{}' ; -print 2>/dev/null What does the above command do? June 28, 2010 14Tusharadri Sarkar
- 15. File Searching (grep)FileSearching (grep) grep: When you are interested in the contents of the files Typical usage for search operations: > grep '<pattern>' <files> When the pattern position also matters: > grep –a N '<pattern>' <file> > grep –b N '<pattern>' <file> > grep –C N '<pattern>' <file> June 28, 2010 15Tusharadri Sarkar
- 16. File Searching (grep)FileSearching (grep) When you are looking for the line numbers: > grep –c '<pattern>' <file> Remember: C and c are different options for 'grep' The --color option: Highlight <pattern> Enhance power of 'grep' : 'find' and 'sed' Extremely useful if you are interested in Shell scripting June 28, 2010 16Tusharadri Sarkar
- 17. File Operations (CopyMove Delete)File Operations (Copy Move Delete) cp – mv – rm: Straight forward and simple They have similar syntax and options: > cp <source> <destination> > mv <source> <destination> > rm <filename> -i & -r : Do it 'interactively' or 'recursively' Be careful while using rm and move. Use > alias rm=`rm –i`; alias mv=`mv –i` June 28, 2010 17Tusharadri Sarkar
- 18. Archives and CompressionsArchivesand Compressions Need to create archive and compress files? It’s easy with tar and gzip: > tar –cf demo.tar <file1> <file2> … > tar –xvf demo.tar - like verbose? Zip and Unzip a file: gzip & gunzip :: bzip2 & bunzip2 Combine archiving and zipping together: > tar –czvf myfiles.tar.gz *.sh > tar –xzvf myfiles.tar.gz June 28, 2010 18Tusharadri Sarkar
- 19. Text Manipulation (sed)TextManipulation (sed) sed: Stands for “Stream Editor” Powerful but also complex When it comes to text manipulation: Combine powers of 'sed' and 'grep' General format of 'sed' for substitution: > sed 's/<pat1>/<pat2>/<switch>' file1 > file2 How does the <switch> influence the output? June 28, 2010 19Tusharadri Sarkar
- 20. Text Manipulation (sed)TextManipulation (sed) 9 typical ‘sed’ usage examples Pattern substitutions: 1. > sed 's/string1/string2/g' – very basic 2. > sed 's/(.*)1/12/g' – a little tricky ! 3. > sed '/ *#/d; /^ *$/d' – trickier !! 4. > sed ':a; /$/N; s/n//; ta' – too cryptic !!! What are all those '' required for? June 28, 2010 20Tusharadri Sarkar
- 21. Text Manipulation (sed)TextManipulation (sed) 5. > sed 's/[ t]*$//' If you don't like spaces... 6. > sed 's/([`"$])/1/g' Use escape judiciously with meta-characters 7. > sed 10 | sed 's/^/ /; s/ *(.{7,})/1/' Handy for number alignments 8. > sed –n '1000{p;q}' 9. > sed –n '10,20p;20q' Printing with sed June 28, 2010 21Tusharadri Sarkar
- 22. Text Manipulation (tr)TextManipulation (tr) tr: Stands for “Translate” > echo 'Test' | tr '[:lower:]' '[:upper:]' Case conversion > tr –dc '[:print:]' < /dev/urandom Handle non printable characters > tr –s '[:blank:]' 't' < /proc/diskstats | cut –f4 Combine with 'cut' to manipulate fields June 28, 2010 22Tusharadri Sarkar
- 23. Set Operations (sort)SetOperations (sort) sort: is helpful with text files uniq helps refining you sorting > sort file1 file2 | uniq > sort file1 file2 | uniq –d > sort file1 file1 file2 | uniq –u > sort file1 file2 | uniq –u What are the differences in output? June 28, 2010 23Tusharadri Sarkar
- 24. Set Operation (join)SetOperation (join) join: works best on previously sorted files: > join –t'0' –a1 –a2 file1 file2 > join –t'0' file1 file2 > join –t'0' –v2 file1 file2 > join –t'0' –v1 –v2 file1 file2 On unsorted files join works just like concatenating files June 28, 2010 24Tusharadri Sarkar
- 25. Basic NetworkingBasic Networking ifconfig:An equivalent of 'ipconfig' on DOS, but it does much more > ifconfig > ifconfig –< interface> iwconfig: 'w' for wireless network hostname: When you want to know about the host/ the system you are on > hostname > hostname –i June 28, 2010 25Tusharadri Sarkar
- 26. Basic NetworkingBasic Networking netstat:Working with kernel routing table > netstat –r > netstat –i > netstat –l > netstat –tup route: Modifying the kernel routing table > route add [route addr] [interface] > route delete [route addr] [interface] June 28, 2010 26Tusharadri Sarkar
- 27. Basic System InformationBasicSystem Information > uname –a Which OS, Kernel version, hardware and system architecture am I running on? For specific information use the following: -s, -o, -r, -v, -m, or -n Know who all are working with you: > w > who > whoami > who am i (Yes, the output is different!!) June 28, 2010 27Tusharadri Sarkar
- 28. Basic System InformationBasicSystem Information > head –n1 /etc/issue What distribution version and release am I using? > cat /proc/partitions What about my partitions? > grep 'MemTotal' /proc/meminfo Am I running out of RAM? > mount | column –t What file systems is currently in use? June 28, 2010 28Tusharadri Sarkar
- 29. Disk Space UsageDiskSpace Usage > ls –lSr 'ls' command coming to rescue again!! du – Stands for “Disc Usage” > du –s * | sort –k1,1rn | head > du –hk . > du –h <file> df – Stands for “Disc Free” > df –h > df –i June 28, 2010 29Tusharadri Sarkar
- 30. Basic Monitoring &DebuggingBasic Monitoring & Debugging If you want to monitor a file continuously: > tail –f <filename> You will need Ctrl+C too!! When you want to deal with processes: > ps –ef <processname> > ps –p pid1, pid2,... Be sure of PID & PPID !! > ps –e –o pid, args --forest Hierarchical list Some miscellaneous commands: > free –m > last reboot June 28, 2010 30Tusharadri Sarkar
- 31. Basic Monitoring &DebuggingBasic Monitoring & Debugging 3 Advance commands (for sysadmin) Every CPU cycle is costly... > ps –e –o pcpu, cpu, nice, state, cputime, args --sort pcpu | sed '/^ 0.0 /d' And every byte of memory too... > ps –e –orss=, args= | sort –b –k1, 1n | pr – TW$COLUMNS Locked threads can cause troubles... > ps –C <process> -L –o pid, tid, pcpu, state June 28, 2010 31Tusharadri Sarkar
- 32. MiscellaneousMiscellaneous Get familiar withthe time keepers: cal: “Calendar”, no one can live without it... > cal (current month’s calendar) > cal <mm> <yyyy> (for specific month) date: Of course you want to know this... > date (Date with IST) > date “%d/%m%y %H:%M:%S” (A more familiar date format) June 28, 2010 32Tusharadri Sarkar
- 33. MiscellaneousMiscellaneous Where are yourcommands resting? > which <command> > whereis <command> Read a file without opening it: > cat <filename> Finally, the grand “Manual” of Linux: > man <item> The <item> list is really large!! Try it... June 28, 2010 33Tusharadri Sarkar
- 34. Thank You !!ThankYou !! 34
Editor's Notes
- #3 Created at Bell Labs in 1970 Written in the C programming language which was developed at the same time Supports large numbers of simultaneous users Runs with few alterations on many hardware platforms Simple, elegant, and easy to use (at least compared to its predecessors)
- #4 1984 - Richard Stallman started the GNU project to create a free operating system Professor Andrew Tanenbaum wrote the operating system MINIX from scratch to teach his students the inner workings of a real operating system April 1991 - Linus Torvalds starts working on a new operating system inspired by MINIX September 1991 - Linux version 0.01 released.
- #8 Each user has a home directory, which is where personal files and preferences are stored. These are typically in /home or in /usr/users Use the 'ls -l' or long listing command to see all the permissions assigned to a file. It is similar to the 'dir' command in DOS.
- #9 Each permission is assigned a different weight r=4, w=2 & x=1 If a file is assigned permissions 755 it means the current user has all permissions while group and others have only read and execute permissions
- #10 For example: alias jtest=`find . -name *.java | grep 'Test' | sed 's/Test//'` The entire command will execute when you run 'jtest’ To create permanent aliases add them in the .bashrc file in you home or login directory.
- #11 pwd: shows the current directory path cd: go to the home directory of the user cd - : go to the previous directory cd ~/<path> : go to any directory <path> under the home directory cd /<path> : go to any directory <path> under root (/). This is the absolute path name cd ../ : go one directory up
- #12 mkdir <dname> : Creates directory <dname> rmdir <dname> : Removes directory <dname> Remember: In Unix everything is 'file'
- #13 ls : Lists directory contents (The command that you will probably use the most) ls -a : Lists directory contents including hidden files ls -lrt : Lists directory contents by date of modification in reverse order Regular expressions blends efficiently with ls: For example: ls -lrt lib*.so --color Option lists the filenames with different color according to filetypes
- #14 find <path> -name *.txt : lists all the .txt files under the directory <path> find <path> -type d : lists all the sub-directories under <path> find <path> -type f: lists all the regular files under <path> You can get the same result of the above commands by using 'ls' with some tricks, but 'find' is far more powerful and easy to use
- #15 find <path> -name "*.*" -exec grep "<pattern>" '{}' \; -print 2>/dev/null The above example will look for every instance of <pattern> even inside file contents under <path> recursively
- #16 grep '<pattern>' *.* : Searches for <pattern> inside all regulare files in current directory grep -a/-b N '<pattern>' *.* : Same as above but prints N lines including the matching lines either after or before it grep -C N '<pattern>' *.* : Same as above but prints N lines including the matching lines both before and after it
- #17 grep -c '<pattern>‘ *.* Searches for <pattern> inside all the regular files in the current directory and prints the file names and the line numbers at which <pattern> is present
- #18 cp <source> <destination> : To copy a file <source> to <destination> move <source> <destination> : To move a file <source> to <destination>. This also doubles as the command to rename the file rm <fname> : To remove / delete file <fname>
- #19 tar -cf demo.tar <files>: Create an archive demo.tar from <files> tar -xvf demo.tar : Extracts the files from archive demo.tar (-v gives verbose output) gzip & gunzip : To Zip and Unzip a file bzip2 & bunzip2 : Same as above with a different compression algorithm tar -czvf tushar.tar.gz *.sh and tar -xzvf tushar.tar.gz (-z refers to zipping)
- #20 General format of 'sed' for substitution operations: sed 's/<pat1>/<pat2>/<switch>' file1 > file2 It means, in file file1 replace occurrences of <pat1> with <pat2> depending on <switch> and put the outcome in file2 <switch> = ‘g’ or ‘c’ which stands for ‘global’ and ‘conditional’ substitution
- #21 sed 's/string1/string2/g' : Replace all occurrence of string1 with string2 sed 's/\(.*\)1/\12/g' : Modify anystring1 to anystring2. '\' is used to escape special chars sed '/ *#/d; /^ *$/d' : Remove comments and blank lines. ';' joins two sed operations sed ':a; /\\$/N; s/\\\n//; ta' : Concatenate lines with trailing \
- #22 sed 's/[ \t]*$//' : Remove trailing spaces from lines sed 's/\([`"$\]\)/\\\1/g' : Escape shell meta characters active within double quotes seq 10 | sed 's/^/ /; s/ *\(.\{7,\}\)/\1/' : Right align numbers (Not applicable for all patterns) sed -n '1000{p;q}' : Print 1000th line sed -n '10,20p;20q' : Print lines 10 to 20
- #23 echo 'Test' | tr '[:lower:]' '[:upper:]' : Convert uppercase letters to lower case letters tr -dc '[:print:]' < /dev/urandom : Filter non printable characters tr -s '[:blank:]' '\t' < /proc/diskstats | cut -f4 : cut fields separated by blanks
- #24 sort file1 file2 | uniq : Shows union of unsorted files sort file1 file2 | uniq -d : Shows intersection of unsorted files sort file1 file1 file2 | uniq -u : Shows difference of unsorted files sort file1 file2 | uniq -u : Shows symmetric Difference of unsorted files
- #25 join -t'\0' -a1 -a2 file1 file2 : Shows union of sorted files join -t'\0' file1 file2 : Shows intersection of sorted files join -t'\0' -v2 file1 file2 : Shows the difference of sorted files join -t'\0' -v1 -v2 file1 file2 : Shows the symmetric Difference of sorted files
- #26 ifconfig : Lists details of all the available network interfaces of your system ifconfig -<IFACE> : Shows details about interface <IFACE> iwconfig : Same as 'ifconfig' but lists only the wireless network interfaces hostname : Shows the system host name hostname -i : Shows host IP address
- #27 netstat -r : Lists the routing table of the kernel netstat -i : List interface details netstat -l: Lists all the binded listening ports netstat -tup : Lists all active connections route add : To add a new route to the table route delete : To delete a route from the table
- #28 uname -a : Shows kernel version and the system architecture information To get specific information use following options: -r (release) -n (distribution) -m (Hardware ) -v (version) -s (OS name) -o (OS category) who : Lists all the users currently logged in into the system whoami : Gives the current user name who am i : Lists current user login info and time
- #29 head -n1 /etc/issue : Shows the name and the version of distribution cat /proc/partitions : Lists all the partitions registered on the system grep 'MemTotal' /proc/meminfo : Shows the total RAM registered to the system Mount | column -t : Lists all the mounted file systems on the system (Aligns the output)
- #30 ls -lSr : Show files by size, biggest last du -s * | sort -k1,1rn | head : Show top disk users in current directory du -h : Shows the file sizes in easy to interpret disk usage (-h stands for Human Readable) df -h : Show free space on mounted filesystems df -i : Show free inodes on mounted filesystems
- #31 tail -f <fname> : Continuously monitors file <fname> until terminated by Ctrl+C ps -ef : Lists all the processes in the system ps -p id1,id2,... : Lists information for particular process IDs id1,id2,… ps -e -o pid,args --forest : List processes and displays them arranged in a hierarchy free -m : Shows amount of free memory in MB last reboot : Show system reboot history
- #32 ps -e -o pcpu,cpu,nice,state,cputime,args --sort pcpu | sed '/^ 0.0 /d' : Lists processes by % cpu usage ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS : Lists processes by mem (KB) usage. See also ps_mem.py ps -C <process> -L -o pid,tid,pcpu,state : Lists all threads for a particular <process>