SlideShare a Scribd company logo
1 of 29
grep | sed | awk | xargs | etc Practical *nix utilities(for text processing)
whoami
/ awk cat grep tac sed echo ls du test mv split tail dir join wc head vim tr sort sum cut expr uniq paste kill tee
/ Log Mega App Files DB
pwd | ls | find | tee GNU  Coreutils http://www.gnu.org/software/coreutils/ The takeaway command:man > info coreutils
pwd | ls | find | tee List of files: ls –l ls–1 ls –latr find . –name *.txt
pwd | ls | find | tee Seek for a string in a file: grep“cat” file.txt grep –v “dog” file.txt grep –i “PaTtErN” file.txt egrep“cat|dog” file.txt zgrep“cat” file.txt.gz
for / xargs Do something with each file: for file in `find . –name *tmp`  do rm$file  done find . –name *tmp| xargsrm
pwd | ls | find | tee find + grep find . -name '*txt' -exec grep-l aaa{}  find . -name '*txt' | xargsgrep-l aaa
pwd | ls | find | tee cat grep tac echo ls du test mv split tail dir join wc head tr sort sum cut expr uniq paste kill tee
paste
join
sort | uniq
wc
cut
csplit
awk sed
sed sfor substitution sed‘s/cat/dog/’  # cat -> dog sed ‘s/ab//’ # ab-> ba
sed pfor printing sed –n ‘/dog/p’  # print lines that match ‘dog’ sed–n ‘/start/,/end/p’ # print range
sed dto delete sed ‘/dog/d’ # delete lines that match ‘dog’ sed ‘1,/pattern/d’ # delete range
sed | and –e for invocation sed ‘s/a/A/’ | sed ‘s/b/B/’ # sed –e ‘s/a/A/’ –e ‘s/b/B/’  #
sed { .. } to group the commands sed ‘/pattern/ {        s/p/P/        s/e/E/     }’ #pattern -> PattErn
sed rto read a file sed ‘/include/ r file.txt’ # insert file.txt after include wto write to a file sed‘/pattern/ w file.txt’ # write matched lines to a file
sedtris
awk aaabbb ccc aaabbbzzz awk '/zzz/'  1.txt   grepzzz    1.txt aaabbbzzz
awk awk 'BEGIN{<initializations>} <pattern 1> {<actions>} <pattern 2> {<actions>} ...  END{<final actions>}'
awk awk 'BEGIN{a=0, b=0}  /aaa/{a++}  /bbb/   {b++} END{printf “%d%d”,a,b}'
awk awk '{arr[$2]+=$1}  END  { for (id in arr)  printf "%s%d",id,arr[id]}'
exit @antonarhipov

More Related Content

What's hot

Fast and cost effective geospatial analysis pipeline with AWS lambda
Fast and cost effective geospatial analysis pipeline with AWS lambdaFast and cost effective geospatial analysis pipeline with AWS lambda
Fast and cost effective geospatial analysis pipeline with AWS lambdaMila Frerichs
 
2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekinge2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekingeProf. Wim Van Criekinge
 
Why Python (for Statisticians)
Why Python (for Statisticians)Why Python (for Statisticians)
Why Python (for Statisticians)Matt Harrison
 
Course 102: Lecture 7: Simple Utilities
Course 102: Lecture 7: Simple Utilities Course 102: Lecture 7: Simple Utilities
Course 102: Lecture 7: Simple Utilities Ahmed El-Arabawy
 
Parse, scale to millions
Parse, scale to millionsParse, scale to millions
Parse, scale to millionsFlorent Vilmart
 
Introduction to Perl
Introduction to PerlIntroduction to Perl
Introduction to PerlSway Wang
 
Value protocols and codables
Value protocols and codablesValue protocols and codables
Value protocols and codablesFlorent Vilmart
 
Advanced linux chapter ix-shell script
Advanced linux chapter ix-shell scriptAdvanced linux chapter ix-shell script
Advanced linux chapter ix-shell scriptEliezer Moraes
 
7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)Steven Francia
 
Go初心者がGoでコマンドラインツールの作成に挑戦した話
Go初心者がGoでコマンドラインツールの作成に挑戦した話Go初心者がGoでコマンドラインツールの作成に挑戦した話
Go初心者がGoでコマンドラインツールの作成に挑戦した話dcubeio
 
Ansible ハンズオン on AWS - DevelopersIO 2017
Ansible ハンズオン on AWS - DevelopersIO 2017Ansible ハンズオン on AWS - DevelopersIO 2017
Ansible ハンズオン on AWS - DevelopersIO 2017Shuji Watanabe
 
JIP Pipeline System Introduction
JIP Pipeline System IntroductionJIP Pipeline System Introduction
JIP Pipeline System Introductionthasso23
 
Logrotate sh
Logrotate shLogrotate sh
Logrotate shBen Pope
 
Introduction to source{d} Engine and source{d} Lookout
Introduction to source{d} Engine and source{d} Lookout Introduction to source{d} Engine and source{d} Lookout
Introduction to source{d} Engine and source{d} Lookout source{d}
 
File handling in c language
File handling in c languageFile handling in c language
File handling in c languageHarish Gyanani
 
Rotzy - Building an iPhone Photo Sharing App on Google App Engine
Rotzy - Building an iPhone Photo Sharing App on Google App EngineRotzy - Building an iPhone Photo Sharing App on Google App Engine
Rotzy - Building an iPhone Photo Sharing App on Google App Enginegeehwan
 

What's hot (20)

Cscope and ctags
Cscope and ctagsCscope and ctags
Cscope and ctags
 
Fast and cost effective geospatial analysis pipeline with AWS lambda
Fast and cost effective geospatial analysis pipeline with AWS lambdaFast and cost effective geospatial analysis pipeline with AWS lambda
Fast and cost effective geospatial analysis pipeline with AWS lambda
 
2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekinge2015 bioinformatics python_strings_wim_vancriekinge
2015 bioinformatics python_strings_wim_vancriekinge
 
Txjs
TxjsTxjs
Txjs
 
Why Python (for Statisticians)
Why Python (for Statisticians)Why Python (for Statisticians)
Why Python (for Statisticians)
 
Course 102: Lecture 7: Simple Utilities
Course 102: Lecture 7: Simple Utilities Course 102: Lecture 7: Simple Utilities
Course 102: Lecture 7: Simple Utilities
 
Parse, scale to millions
Parse, scale to millionsParse, scale to millions
Parse, scale to millions
 
Introduction to Perl
Introduction to PerlIntroduction to Perl
Introduction to Perl
 
Value protocols and codables
Value protocols and codablesValue protocols and codables
Value protocols and codables
 
Advanced linux chapter ix-shell script
Advanced linux chapter ix-shell scriptAdvanced linux chapter ix-shell script
Advanced linux chapter ix-shell script
 
7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)7 Common Mistakes in Go (2015)
7 Common Mistakes in Go (2015)
 
Go初心者がGoでコマンドラインツールの作成に挑戦した話
Go初心者がGoでコマンドラインツールの作成に挑戦した話Go初心者がGoでコマンドラインツールの作成に挑戦した話
Go初心者がGoでコマンドラインツールの作成に挑戦した話
 
Ansible ハンズオン on AWS - DevelopersIO 2017
Ansible ハンズオン on AWS - DevelopersIO 2017Ansible ハンズオン on AWS - DevelopersIO 2017
Ansible ハンズオン on AWS - DevelopersIO 2017
 
JIP Pipeline System Introduction
JIP Pipeline System IntroductionJIP Pipeline System Introduction
JIP Pipeline System Introduction
 
Logrotate sh
Logrotate shLogrotate sh
Logrotate sh
 
Introduction to source{d} Engine and source{d} Lookout
Introduction to source{d} Engine and source{d} Lookout Introduction to source{d} Engine and source{d} Lookout
Introduction to source{d} Engine and source{d} Lookout
 
My life as a beekeeper
My life as a beekeeperMy life as a beekeeper
My life as a beekeeper
 
File handling in c language
File handling in c languageFile handling in c language
File handling in c language
 
A Python Crash Course
A Python Crash CourseA Python Crash Course
A Python Crash Course
 
Rotzy - Building an iPhone Photo Sharing App on Google App Engine
Rotzy - Building an iPhone Photo Sharing App on Google App EngineRotzy - Building an iPhone Photo Sharing App on Google App Engine
Rotzy - Building an iPhone Photo Sharing App on Google App Engine
 

Viewers also liked

Defeating The Network Security Infrastructure V1.0
Defeating The Network Security Infrastructure  V1.0Defeating The Network Security Infrastructure  V1.0
Defeating The Network Security Infrastructure V1.0Philippe Bogaerts
 
Unix Command Line Productivity Tips
Unix Command Line Productivity TipsUnix Command Line Productivity Tips
Unix Command Line Productivity TipsKeith Bennett
 
Sed & awk the dynamic duo
Sed & awk   the dynamic duoSed & awk   the dynamic duo
Sed & awk the dynamic duoJoshua Thijssen
 
Web Application Security: Introduction to common classes of security flaws an...
Web Application Security: Introduction to common classes of security flaws an...Web Application Security: Introduction to common classes of security flaws an...
Web Application Security: Introduction to common classes of security flaws an...Thoughtworks
 
Web Application Security with PHP
Web Application Security with PHPWeb Application Security with PHP
Web Application Security with PHPjikbal
 
Learning sed and awk
Learning sed and awkLearning sed and awk
Learning sed and awkYogesh Sawant
 
Practical Example of grep command in unix
Practical Example of grep command in unixPractical Example of grep command in unix
Practical Example of grep command in unixJavin Paul
 
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAPVirtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAPMichael Coates
 
Secure Shell(ssh)
Secure Shell(ssh)Secure Shell(ssh)
Secure Shell(ssh)Pina Parmar
 
Top 100 Linux Interview Questions and Answers 2014
Top 100 Linux Interview Questions and Answers 2014Top 100 Linux Interview Questions and Answers 2014
Top 100 Linux Interview Questions and Answers 2014iimjobs and hirist
 
RHCE FINAL Questions and Answers
RHCE FINAL Questions and AnswersRHCE FINAL Questions and Answers
RHCE FINAL Questions and AnswersRadien software
 
Introduction to SSH
Introduction to SSHIntroduction to SSH
Introduction to SSHHemant Shah
 
Linux Systems Performance 2016
Linux Systems Performance 2016Linux Systems Performance 2016
Linux Systems Performance 2016Brendan Gregg
 

Viewers also liked (20)

Defeating The Network Security Infrastructure V1.0
Defeating The Network Security Infrastructure  V1.0Defeating The Network Security Infrastructure  V1.0
Defeating The Network Security Infrastructure V1.0
 
class12_Networking2
class12_Networking2class12_Networking2
class12_Networking2
 
Unix Command Line Productivity Tips
Unix Command Line Productivity TipsUnix Command Line Productivity Tips
Unix Command Line Productivity Tips
 
PHP Secure Programming
PHP Secure ProgrammingPHP Secure Programming
PHP Secure Programming
 
Sed & awk the dynamic duo
Sed & awk   the dynamic duoSed & awk   the dynamic duo
Sed & awk the dynamic duo
 
Web Application Security: Introduction to common classes of security flaws an...
Web Application Security: Introduction to common classes of security flaws an...Web Application Security: Introduction to common classes of security flaws an...
Web Application Security: Introduction to common classes of security flaws an...
 
Secure SHell
Secure SHellSecure SHell
Secure SHell
 
Secure shell protocol
Secure shell protocolSecure shell protocol
Secure shell protocol
 
How to Setup A Pen test Lab and How to Play CTF
How to Setup A Pen test Lab and How to Play CTF How to Setup A Pen test Lab and How to Play CTF
How to Setup A Pen test Lab and How to Play CTF
 
Web Application Security with PHP
Web Application Security with PHPWeb Application Security with PHP
Web Application Security with PHP
 
Learning sed and awk
Learning sed and awkLearning sed and awk
Learning sed and awk
 
Practical Example of grep command in unix
Practical Example of grep command in unixPractical Example of grep command in unix
Practical Example of grep command in unix
 
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAPVirtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
Virtual Security Lab Setup - OWASP Broken Web Apps, Webgoat, & ZAP
 
SSH - Secure Shell
SSH - Secure ShellSSH - Secure Shell
SSH - Secure Shell
 
Secure Shell(ssh)
Secure Shell(ssh)Secure Shell(ssh)
Secure Shell(ssh)
 
SSH
SSHSSH
SSH
 
Top 100 Linux Interview Questions and Answers 2014
Top 100 Linux Interview Questions and Answers 2014Top 100 Linux Interview Questions and Answers 2014
Top 100 Linux Interview Questions and Answers 2014
 
RHCE FINAL Questions and Answers
RHCE FINAL Questions and AnswersRHCE FINAL Questions and Answers
RHCE FINAL Questions and Answers
 
Introduction to SSH
Introduction to SSHIntroduction to SSH
Introduction to SSH
 
Linux Systems Performance 2016
Linux Systems Performance 2016Linux Systems Performance 2016
Linux Systems Performance 2016
 

Similar to Practical unix utilities for text processing

Unit 12 finding and processing files
Unit 12 finding and processing filesUnit 12 finding and processing files
Unit 12 finding and processing filesroot_fibo
 
Linux commands
Linux commandsLinux commands
Linux commandsshekhar70
 
Basic shell commands by Jeremy Sanders
Basic shell commands by Jeremy SandersBasic shell commands by Jeremy Sanders
Basic shell commands by Jeremy SandersDevanand Gehlot
 
Linux Basic Commands
Linux Basic CommandsLinux Basic Commands
Linux Basic CommandsHanan Nmr
 
Old school hacking : Exploiting UNIX Wildcards
Old school hacking : Exploiting UNIX WildcardsOld school hacking : Exploiting UNIX Wildcards
Old school hacking : Exploiting UNIX WildcardsUNIDEES Algérie
 
Productivity tips - Introduction to linux for bioinformatics
Productivity tips - Introduction to linux for bioinformaticsProductivity tips - Introduction to linux for bioinformatics
Productivity tips - Introduction to linux for bioinformaticsBITS
 
Unit 8
Unit 8Unit 8
Unit 8siddr
 
Linux advanced privilege escalation
Linux advanced privilege escalationLinux advanced privilege escalation
Linux advanced privilege escalationJameel Nabbo
 
Unix Basics 04sp
Unix Basics 04spUnix Basics 04sp
Unix Basics 04spDr.Ravi
 
101 3.3 perform basic file management
101 3.3 perform basic file management101 3.3 perform basic file management
101 3.3 perform basic file managementAcácio Oliveira
 
3.1.a linux commands reference
3.1.a linux commands reference3.1.a linux commands reference
3.1.a linux commands referenceAcácio Oliveira
 

Similar to Practical unix utilities for text processing (20)

Unit 12 finding and processing files
Unit 12 finding and processing filesUnit 12 finding and processing files
Unit 12 finding and processing files
 
Linux commands
Linux commandsLinux commands
Linux commands
 
unix_commands.ppt
unix_commands.pptunix_commands.ppt
unix_commands.ppt
 
Basic shell commands by Jeremy Sanders
Basic shell commands by Jeremy SandersBasic shell commands by Jeremy Sanders
Basic shell commands by Jeremy Sanders
 
Linux Commands.pptx
Linux Commands.pptxLinux Commands.pptx
Linux Commands.pptx
 
Linux Basic Commands
Linux Basic CommandsLinux Basic Commands
Linux Basic Commands
 
Old school hacking : Exploiting UNIX Wildcards
Old school hacking : Exploiting UNIX WildcardsOld school hacking : Exploiting UNIX Wildcards
Old school hacking : Exploiting UNIX Wildcards
 
Linux Commands
Linux CommandsLinux Commands
Linux Commands
 
Productivity tips - Introduction to linux for bioinformatics
Productivity tips - Introduction to linux for bioinformaticsProductivity tips - Introduction to linux for bioinformatics
Productivity tips - Introduction to linux for bioinformatics
 
Linux
LinuxLinux
Linux
 
Unit 8
Unit 8Unit 8
Unit 8
 
Unix for Librarians
Unix for LibrariansUnix for Librarians
Unix for Librarians
 
Linux midterm quiz
Linux midterm quizLinux midterm quiz
Linux midterm quiz
 
Linux advanced privilege escalation
Linux advanced privilege escalationLinux advanced privilege escalation
Linux advanced privilege escalation
 
Unix Basics 04sp
Unix Basics 04spUnix Basics 04sp
Unix Basics 04sp
 
101 3.3 perform basic file management
101 3.3 perform basic file management101 3.3 perform basic file management
101 3.3 perform basic file management
 
Unix Basics Commands
Unix Basics CommandsUnix Basics Commands
Unix Basics Commands
 
3.1.a linux commands reference
3.1.a linux commands reference3.1.a linux commands reference
3.1.a linux commands reference
 
Red Hat Linux cheat sheet
Red Hat Linux cheat sheetRed Hat Linux cheat sheet
Red Hat Linux cheat sheet
 
Unix 2 en
Unix 2 enUnix 2 en
Unix 2 en
 

More from Anton Arhipov

JavaZone 2022 - Building Kotlin DSL.pdf
JavaZone 2022 - Building Kotlin DSL.pdfJavaZone 2022 - Building Kotlin DSL.pdf
JavaZone 2022 - Building Kotlin DSL.pdfAnton Arhipov
 
TechTrain 2019 - (Не)адекватное техническое интервью
TechTrain 2019 - (Не)адекватное техническое интервьюTechTrain 2019 - (Не)адекватное техническое интервью
TechTrain 2019 - (Не)адекватное техническое интервьюAnton Arhipov
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCityAnton Arhipov
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCityAnton Arhipov
 
Devoxx Ukraine 2018 - Kotlin DSL in under an hour
Devoxx Ukraine 2018 - Kotlin DSL in under an hourDevoxx Ukraine 2018 - Kotlin DSL in under an hour
Devoxx Ukraine 2018 - Kotlin DSL in under an hourAnton Arhipov
 
GeeCON Prague 2018 - Kotlin DSL in under an hour
GeeCON Prague 2018 - Kotlin DSL in under an hourGeeCON Prague 2018 - Kotlin DSL in under an hour
GeeCON Prague 2018 - Kotlin DSL in under an hourAnton Arhipov
 
Build pipelines with TeamCity and Kotlin DSL
Build pipelines with TeamCity and Kotlin DSLBuild pipelines with TeamCity and Kotlin DSL
Build pipelines with TeamCity and Kotlin DSLAnton Arhipov
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCityAnton Arhipov
 
JavaDay Kiev 2017 - Integration testing with TestContainers
JavaDay Kiev 2017 - Integration testing with TestContainersJavaDay Kiev 2017 - Integration testing with TestContainers
JavaDay Kiev 2017 - Integration testing with TestContainersAnton Arhipov
 
GeeCON Prague 2017 - TestContainers
GeeCON Prague 2017 - TestContainersGeeCON Prague 2017 - TestContainers
GeeCON Prague 2017 - TestContainersAnton Arhipov
 
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingJavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingAnton Arhipov
 
JavaOne 2017 - TestContainers: integration testing without the hassle
JavaOne 2017 - TestContainers: integration testing without the hassleJavaOne 2017 - TestContainers: integration testing without the hassle
JavaOne 2017 - TestContainers: integration testing without the hassleAnton Arhipov
 
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingJavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingAnton Arhipov
 
JavaZone 2017 - The Hitchhiker’s guide to Java class reloading
JavaZone 2017 - The Hitchhiker’s guide to Java class reloadingJavaZone 2017 - The Hitchhiker’s guide to Java class reloading
JavaZone 2017 - The Hitchhiker’s guide to Java class reloadingAnton Arhipov
 
JUG.ua 20170225 - Java bytecode instrumentation
JUG.ua 20170225 - Java bytecode instrumentationJUG.ua 20170225 - Java bytecode instrumentation
JUG.ua 20170225 - Java bytecode instrumentationAnton Arhipov
 
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloading
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloadingRiga DevDays 2017 - The hitchhiker’s guide to Java class reloading
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloadingAnton Arhipov
 
GeeCON 2017 - TestContainers. Integration testing without the hassle
GeeCON 2017 - TestContainers. Integration testing without the hassleGeeCON 2017 - TestContainers. Integration testing without the hassle
GeeCON 2017 - TestContainers. Integration testing without the hassleAnton Arhipov
 
JEEConf 2017 - The hitchhiker’s guide to Java class reloading
JEEConf 2017 - The hitchhiker’s guide to Java class reloadingJEEConf 2017 - The hitchhiker’s guide to Java class reloading
JEEConf 2017 - The hitchhiker’s guide to Java class reloadingAnton Arhipov
 
JEEConf 2017 - Having fun with Javassist
JEEConf 2017 - Having fun with JavassistJEEConf 2017 - Having fun with Javassist
JEEConf 2017 - Having fun with JavassistAnton Arhipov
 

More from Anton Arhipov (20)

JavaZone 2022 - Building Kotlin DSL.pdf
JavaZone 2022 - Building Kotlin DSL.pdfJavaZone 2022 - Building Kotlin DSL.pdf
JavaZone 2022 - Building Kotlin DSL.pdf
 
Idiomatic kotlin
Idiomatic kotlinIdiomatic kotlin
Idiomatic kotlin
 
TechTrain 2019 - (Не)адекватное техническое интервью
TechTrain 2019 - (Не)адекватное техническое интервьюTechTrain 2019 - (Не)адекватное техническое интервью
TechTrain 2019 - (Не)адекватное техническое интервью
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCity
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCity
 
Devoxx Ukraine 2018 - Kotlin DSL in under an hour
Devoxx Ukraine 2018 - Kotlin DSL in under an hourDevoxx Ukraine 2018 - Kotlin DSL in under an hour
Devoxx Ukraine 2018 - Kotlin DSL in under an hour
 
GeeCON Prague 2018 - Kotlin DSL in under an hour
GeeCON Prague 2018 - Kotlin DSL in under an hourGeeCON Prague 2018 - Kotlin DSL in under an hour
GeeCON Prague 2018 - Kotlin DSL in under an hour
 
Build pipelines with TeamCity and Kotlin DSL
Build pipelines with TeamCity and Kotlin DSLBuild pipelines with TeamCity and Kotlin DSL
Build pipelines with TeamCity and Kotlin DSL
 
Build pipelines with TeamCity
Build pipelines with TeamCityBuild pipelines with TeamCity
Build pipelines with TeamCity
 
JavaDay Kiev 2017 - Integration testing with TestContainers
JavaDay Kiev 2017 - Integration testing with TestContainersJavaDay Kiev 2017 - Integration testing with TestContainers
JavaDay Kiev 2017 - Integration testing with TestContainers
 
GeeCON Prague 2017 - TestContainers
GeeCON Prague 2017 - TestContainersGeeCON Prague 2017 - TestContainers
GeeCON Prague 2017 - TestContainers
 
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingJavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
 
JavaOne 2017 - TestContainers: integration testing without the hassle
JavaOne 2017 - TestContainers: integration testing without the hassleJavaOne 2017 - TestContainers: integration testing without the hassle
JavaOne 2017 - TestContainers: integration testing without the hassle
 
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloadingJavaOne 2017 - The hitchhiker’s guide to Java class reloading
JavaOne 2017 - The hitchhiker’s guide to Java class reloading
 
JavaZone 2017 - The Hitchhiker’s guide to Java class reloading
JavaZone 2017 - The Hitchhiker’s guide to Java class reloadingJavaZone 2017 - The Hitchhiker’s guide to Java class reloading
JavaZone 2017 - The Hitchhiker’s guide to Java class reloading
 
JUG.ua 20170225 - Java bytecode instrumentation
JUG.ua 20170225 - Java bytecode instrumentationJUG.ua 20170225 - Java bytecode instrumentation
JUG.ua 20170225 - Java bytecode instrumentation
 
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloading
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloadingRiga DevDays 2017 - The hitchhiker’s guide to Java class reloading
Riga DevDays 2017 - The hitchhiker’s guide to Java class reloading
 
GeeCON 2017 - TestContainers. Integration testing without the hassle
GeeCON 2017 - TestContainers. Integration testing without the hassleGeeCON 2017 - TestContainers. Integration testing without the hassle
GeeCON 2017 - TestContainers. Integration testing without the hassle
 
JEEConf 2017 - The hitchhiker’s guide to Java class reloading
JEEConf 2017 - The hitchhiker’s guide to Java class reloadingJEEConf 2017 - The hitchhiker’s guide to Java class reloading
JEEConf 2017 - The hitchhiker’s guide to Java class reloading
 
JEEConf 2017 - Having fun with Javassist
JEEConf 2017 - Having fun with JavassistJEEConf 2017 - Having fun with Javassist
JEEConf 2017 - Having fun with Javassist
 

Recently uploaded

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLScyllaDB
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxLoriGlavin3
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 

Recently uploaded (20)

Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Developer Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQLDeveloper Data Modeling Mistakes: From Postgres to NoSQL
Developer Data Modeling Mistakes: From Postgres to NoSQL
 
The State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptxThe State of Passkeys with FIDO Alliance.pptx
The State of Passkeys with FIDO Alliance.pptx
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 

Practical unix utilities for text processing

  • 1. grep | sed | awk | xargs | etc Practical *nix utilities(for text processing)
  • 3. / awk cat grep tac sed echo ls du test mv split tail dir join wc head vim tr sort sum cut expr uniq paste kill tee
  • 4. / Log Mega App Files DB
  • 5. pwd | ls | find | tee GNU Coreutils http://www.gnu.org/software/coreutils/ The takeaway command:man > info coreutils
  • 6. pwd | ls | find | tee List of files: ls –l ls–1 ls –latr find . –name *.txt
  • 7. pwd | ls | find | tee Seek for a string in a file: grep“cat” file.txt grep –v “dog” file.txt grep –i “PaTtErN” file.txt egrep“cat|dog” file.txt zgrep“cat” file.txt.gz
  • 8. for / xargs Do something with each file: for file in `find . –name *tmp` do rm$file done find . –name *tmp| xargsrm
  • 9. pwd | ls | find | tee find + grep find . -name '*txt' -exec grep-l aaa{} find . -name '*txt' | xargsgrep-l aaa
  • 10. pwd | ls | find | tee cat grep tac echo ls du test mv split tail dir join wc head tr sort sum cut expr uniq paste kill tee
  • 11. paste
  • 12. join
  • 14. wc
  • 15. cut
  • 18. sed sfor substitution sed‘s/cat/dog/’ # cat -> dog sed ‘s/ab//’ # ab-> ba
  • 19. sed pfor printing sed –n ‘/dog/p’ # print lines that match ‘dog’ sed–n ‘/start/,/end/p’ # print range
  • 20. sed dto delete sed ‘/dog/d’ # delete lines that match ‘dog’ sed ‘1,/pattern/d’ # delete range
  • 21. sed | and –e for invocation sed ‘s/a/A/’ | sed ‘s/b/B/’ # sed –e ‘s/a/A/’ –e ‘s/b/B/’ #
  • 22. sed { .. } to group the commands sed ‘/pattern/ { s/p/P/ s/e/E/ }’ #pattern -> PattErn
  • 23. sed rto read a file sed ‘/include/ r file.txt’ # insert file.txt after include wto write to a file sed‘/pattern/ w file.txt’ # write matched lines to a file
  • 25. awk aaabbb ccc aaabbbzzz awk '/zzz/' 1.txt grepzzz 1.txt aaabbbzzz
  • 26. awk awk 'BEGIN{<initializations>} <pattern 1> {<actions>} <pattern 2> {<actions>} ... END{<final actions>}'
  • 27. awk awk 'BEGIN{a=0, b=0} /aaa/{a++} /bbb/ {b++} END{printf “%d%d”,a,b}'
  • 28. awk awk '{arr[$2]+=$1} END { for (id in arr) printf "%s%d",id,arr[id]}'