Practical unix utilities for text processing

grep | sed | awk | xargs | etc
Practical *nix utilities(for text processing)

whoami

/
awk
cat
grep
tac
sed
echo
ls
du
test
mv
split
tail
dir
join
wc
head
vim
tr
sort
sum
cut
expr
uniq
paste
kill
tee

/
Log
Mega App
Files
DB

pwd | ls | find | tee
GNU
Coreutils
http://www.gnu.org/software/coreutils/
The takeaway command:man
> info coreutils

List of files:
ls –l
ls–1
ls –latr
find . –name *.txt

Seek for a string in a file:
grep“cat” file.txt
grep –v “dog” file.txt
grep –i “PaTtErN” file.txt
egrep“cat|dog” file.txt
zgrep“cat” file.txt.gz

for / xargs
Do something with each file:
for file in `find . –name *tmp`
do
rm$file
done
find . –name *tmp| xargsrm

find + grep
find . -name '*txt' -exec grep-l aaa{} ;
find . -name '*txt' | xargsgrep-l aaa

cat
grep
tac
echo
ls
du
test
mv
split
tail
dir
join
wc
head
tr
sort
sum
cut
expr
uniq
paste
kill
tee

sort | uniq

csplit

awk
sed

sed
sfor substitution
sed‘s/cat/dog/’
# cat -> dog
sed ‘s/(a)(b)/21/’
# ab-> ba

sed
pfor printing
sed –n ‘/dog/p’
# print lines that match ‘dog’
sed–n ‘/start/,/end/p’
# print range

sed
dto delete
sed ‘/dog/d’
# delete lines that match ‘dog’
sed ‘1,/pattern/d’
# delete range

sed
| and –e for invocation
sed ‘s/a/A/’ | sed ‘s/b/B/’
#
sed –e ‘s/a/A/’ –e ‘s/b/B/’
#

sed
{ .. } to group the commands
sed ‘/pattern/ {
s/p/P/
s/e/E/
}’
#pattern -> PattErn

sed
rto read a file
sed ‘/include/ r file.txt’
# insert file.txt after include
wto write to a file
sed‘/pattern/ w file.txt’
# write matched lines to a file

sedtris

awk
aaabbb ccc
aaabbbzzz
awk '/zzz/' 1.txt
grepzzz 1.txt
aaabbbzzz

awk
awk
'BEGIN{<initializations>}
<pattern 1> {<actions>}
<pattern 2> {<actions>}
...
END{<final actions>}'

awk
awk
'BEGIN{a=0, b=0}
/aaa/{a++}
/bbb/ {b++}
END{printf “%d %d”,a,b}'

awk
awk
'{arr[$2]+=$1}
END
{ for (id in arr)
printf "%s %d ",id,arr[id]}'

exit
@antonarhipov

http://arhipov.blogspot.com	1910
http://www.devclub.eu	165
http://arhipov.blogspot.in	86
http://arhipov.blogspot.de	59
http://feeds.feedburner.com	30
http://arhipov.blogspot.co.uk	23
http://arhipov.blogspot.fr	23
http://arhipov.blogspot.com.es	17
http://arhipov.blogspot.com.au	14
http://arhipov.blogspot.ca	12
http://arhipov.blogspot.it	10
http://arhipov.blogspot.se	9
http://arhipov.blogspot.com.br	8
http://arhipov.blogspot.pt	4
http://arhipov.blogspot.mx	2
http://translate.googleusercontent.com	2
http://arhipov.blogspot.co.nz	2
http://webcache.googleusercontent.com	2
http://www.arhipov.blogspot.com	2
http://www.hanrss.com	1
http://paper.li	1
http://arhipov.blogspot.com.ar	1
http://arhipov.blogspot.jp	1

Practical unix utilities for text processing

by Anton Arhipov on Oct 02, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

23 Embeds 2,384

Statistics

Practical unix utilities for text processing — Presentation Transcript