awk is a very versatile programming language for working on text files. It is more powerful than sed but less complex than C. It is an excellent filter and report writer. In this class I will go over the elements and features of gawk, (the Free Software foundation version of awk), examples and a few one-liners.
sed, known as the “streams editor”, is a powerful tool for text manipulation on UNIX systems. Unlike vim, sed can operate on files of limitless size. The session will cover some of the intermediate concepts such as grouping and back referencing, regular expressions, replacing the nth occurrence of a pattern, operation on every nth line and so on. We will spice up the session with a few “sed one-liners/idioms”
This course provides you with skills to
* Develop sed and awk scripts
* Use sed and awk to automate common tasks
* Use sed and awk to create formatted reports
Prerequisites
* Basic understanding of UNIX / Linux Operating System
* Knowledge of basic UNIX / Linux commands
Intended Audience
* System Administrators, Testing Professionals, and Software Developers working in the UNIX / Linux environment
sed, known as the “streams editor”, is a powerful tool for text manipulation on UNIX systems. Unlike vim, sed can operate on files of limitless size. The session will cover some of the intermediate concepts such as grouping and back referencing, regular expressions, replacing the nth occurrence of a pattern, operation on every nth line and so on. We will spice up the session with a few “sed one-liners/idioms”
This course provides you with skills to
* Develop sed and awk scripts
* Use sed and awk to automate common tasks
* Use sed and awk to create formatted reports
Prerequisites
* Basic understanding of UNIX / Linux Operating System
* Knowledge of basic UNIX / Linux commands
Intended Audience
* System Administrators, Testing Professionals, and Software Developers working in the UNIX / Linux environment
Oracle database supports perl- and POSIX-compatible regular expressions with five elegant and powerful functions: REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_INSTR, REGEXP_LIKE, and REGEXP_COUNT.
This session will demonstrate their nuances and how to use them effectively for data cleansing, manipulation and selection, for validating things such as Social Security Numbers, credit cards, IP addresses, phone numbers, DNAs, XMLs, for extracting things such as email-ids, hostnames from URLs and strings, and for transposing delimited columns to rows. There will be a demo of a few tricky examples taken from forums.oracle.com and asktom.oracle.com.
The session will end with fuzzy matching and optimization techniques, and things to watch out for.
http://docs.oracle.com/cd/E11882_01/appdev.112/e25518/adfns_regexp.htm
It will provide brief knowledge regarding different types of constructs such as conditional statement execution and looping etc, which is crucial for shell Scripting.
This is the seventh set of slightly updated slides from a Perl programming course that I held some years ago.
I want to share it with everyone looking for intransitive Perl-knowledge.
A table of content for all presentations can be found at i-can.eu.
The source code for the examples and the presentations in ODP format are on https://github.com/kberov/PerlProgrammingCourse
Oracle database supports perl- and POSIX-compatible regular expressions with five elegant and powerful functions: REGEXP_REPLACE, REGEXP_SUBSTR, REGEXP_INSTR, REGEXP_LIKE, and REGEXP_COUNT.
This session will demonstrate their nuances and how to use them effectively for data cleansing, manipulation and selection, for validating things such as Social Security Numbers, credit cards, IP addresses, phone numbers, DNAs, XMLs, for extracting things such as email-ids, hostnames from URLs and strings, and for transposing delimited columns to rows. There will be a demo of a few tricky examples taken from forums.oracle.com and asktom.oracle.com.
The session will end with fuzzy matching and optimization techniques, and things to watch out for.
http://docs.oracle.com/cd/E11882_01/appdev.112/e25518/adfns_regexp.htm
It will provide brief knowledge regarding different types of constructs such as conditional statement execution and looping etc, which is crucial for shell Scripting.
This is the seventh set of slightly updated slides from a Perl programming course that I held some years ago.
I want to share it with everyone looking for intransitive Perl-knowledge.
A table of content for all presentations can be found at i-can.eu.
The source code for the examples and the presentations in ODP format are on https://github.com/kberov/PerlProgrammingCourse
This is the first book I have written. It's called Awk One-Liners Explained and it teaches the awk utility through 70 well-explained examples, such as numbering lines in a file, printing certain lines, deleting certain lines, counting words and lines, and others. Get your book copy at: www.catonmat.net/blog/awk-book/
This is the second book I have written. It's called Sed One-Liners Explained and it teaches the powerful sed UNIX utility. Sed is a text stream editor, meaning it's made for editing text files. In this book I teach you sed through 100 well-explained examples. They include various common tasks you'd do with text files, such as deleting, changing, inserting, printing lines from text files, changing certain words or paragraphs, numbering lines, and it includes various special applications, such as printing email headers, printing the IP address on Linux systems by changing the output of `ifconfig`, and many others. Get your book copy at: www.catonmat.net/blog/sed-book/
In the Notes on Programming Language Syntax page, an example par.docxmecklenburgstrelitzh
In the
Notes on Programming Language Syntax
page, an example parser for a simple language is given, using C syntax. Write the parser using F#, but you may only use functional programming and immutable date. Create the list of tokens as a discriminated union, which (in the simplest case) looks like an enumeration.
type TERMINAL = IF|THEN|ELSE|BEGIN|END|PRINT|SEMICOLON|ID|EOF
With this type declared, you can use the terminals like you would use enumerated values in Java.
Use immutable data. The C-code example uses mutable data. Pass the program into the start symbol function. Pass the input yet to be processed to each non-terminal function.
The main function might look like this:
let test_program program =
let result = program |> S
match result with
| [] -> failwith "Early termination or missing EOF"
| x::xs -> if x = EOF then accept() else error()
You do not have to parse input strings. Assume that the parsing has been done. Pass a list of tokens that represent a program into the start symbol. Try these program examples:
[IF;ID;THEN;BEGIN;PRINT;ID;SEMICOLON;PRINT;ID;END;ELSE;PRINT;ID;EOF]
[IF;ID;THEN;IF;ID;THEN;PRINT;ID;ELSE;PRINT;ID;ELSE;BEGIN;PRINT;ID;END;EOF]
Causes error:
[IF;ID;THEN;BEGIN;PRINT;ID;SEMICOLON;PRINT;ID;SEMICOLON;END;ELSE;PRINT;ID;EOF]
Print an accept message when the input is valid and completely consumed. Generate appropriate error messages for incorrect symbols, not enough input, and too much input.
Once you have the parser recognizing input, generate a parse tree using a discriminated type.
Implement a parser using functional programming and immutable data for the unambiguous grammar for arithmetic expressions, from the
Notes on Programming Language Syntax.
E -> E + T | E - T | T
T -> T * F | T / F | F
F -> i | (E)
Use the suggestion in the notes to get around the fact that this grammar appears to need more than one lookahead token.
Once you have the parser recognizing input, generate a parse tree using a discriminated type.
Recall that an F# function that takes two arguments can be coded in either uncurried form (in which case it takes a pair as its input) or curried form (in which case it takes the first argument and returns a function that takes the second argument). In fact it is easy to convert from one form to the other in F#. To this end, define an F# function
curry f
that converts an uncurried function to a curried function, and an F# function
uncurry f
that does the opposite conversion. For example,
> (+);;
val it : (int -> int -> int) =
[email protected]
>
> let plus = uncurry (+);;
val plus : (int * int -> int)
> plus (2,3);;
val it : int = 5
> let cplus = curry plus;;
val cplus : (int -> int -> int)
> let plus3 = cplus 3;;
val plus3 : (int -> int)
> plus3 10;;
val it : int = 13
What are the types of
curry
and
uncurry
?
Given vectors
u = (u
1
, u
2
,..., u
n
)
and .
Regular Expressions in PHP, MySQL by programmerblog.netProgrammer Blog
This PPT explains, how to use regular expressions in PHP. PHP has two types of regular expressions Perl Style and Posix Style.
Read detailed tutorials on http://programmerblog.net
When performing an investigation it is helpful to be reminded of the powerful options available to the investigator. This document is aimed to be a reference to the tools that could be used.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
2. Agenda
2
Elements of awk
Optional Bio Break
Examples and one-liners
Q&A
3. What is awk
3
An acronym of the last names of the three authors
General purpose pattern-scanning and processing
language
Used for filtering, transforming and reporting
More advanced than sed, but less complicated than
C; less cryptic than Perl.
gawk, nawk
4. awk syntax
4
awk [-Ffield_sep] 'cmd' infile(s)
awk [-Ffield_sep] –f cmd_file infile(s)
infile can be the output of pipeline.
Space is the default field_sep
5. awk mechanics
5
[pattern] [{action} …]
Input files processed a line at a time
Every line is processed if there is no pattern
Lines split into fields based on field-sep
Print is the default action.
Input files not affected in anyway
6. Field Names
6
Lines split into fields based on field-sep
$0 represents the whole line
$1, $2, … $n represent different fields
Field names could be used as variables
7. Built-in variables.
7
Variable Explanation
FS Field separator variable for input lines. Defaults to space or tab
NR Number of input lines processed so far
NF Number of fields in the current input line
FILENAME Name of the current input file
OFMT Default format for output numbers
OFS Output field separator. Defaults to space
ORS Output record separator. Defaults to new-line character
RS Input Record Separator. Defaults to new-line character.
FNR Same as NR; but gets reset after each file unlike NR
RSTART, Variables set by the match() function which indicates where the
RLENGTH match starts and how long the match is
SUBSEP Subscript separator. Used in multi-dimensional arrays
8. Operators
8
Operator Explanation
+, -, *, / Addition, Subtraction, Multiplication, Division,
% Remainder/Modulo operation
++ Unary increment (var++ same as var=var+1)
-- Unary decrement
^ or ** Exponentaion
+=, -=, *=, /=, Assignment operator preceded by arithmetic operation (var+=5
%= same as var=var+5)
No operator String concatenation (newstr=“new” $3)
?: Ternary operator (expr1 ? expr2 : expr3)
9. Relational Operators.
9
Operator Explanation
== Equality operator
!= Not equal to
< Less than
<= Less than or equal to
> Greater than
>= Greater than equal to
~ Contains regular expression
!~ Doesn‟t contain regular expression
10. awk patterns
10
Can match either particular lines or ranges of lines
Regular expression patterns
Relational expression patterns
BEGIN and END patterns
11. Regular Expressions
11
Meta character Meaning
. Matches any single character except newline
* Matches zero or more of the character preceding it
e.g.: bugs*, table.*
^ Denotes the beginning of the line. ^A denotes lines starting
with A
$ Denotes the end of the line. :$ denotes lines ending with :
Escape character (., *, [, , etc)
[] matches one or more characters within the brackets. e.g.
[aeiou], [a-z], [a-zA-Z], [0-9], [[:alpha:]], [a-z?,!]
[^] matches any characters others than the ones inside brackets.
eg. ^[^13579] denotes all lines not starting with odd numbers,
[^02468]$ denotes all lines not ending with even numbers
<, > Matches characters at the beginning or end of words
12. Extended Regular Expressions
12
Meta character Meaning
| alternation. e.g.: ho(use|me), the(y|m), (they|them)
+ one or more occurrences of previous character. a+ is same as
aa*)
? zero or one occurrences of previous character.
{n} exactly n repetitions of the previous char or group
{n,} n or more repetitions of the previous char or group
{n, m} n to m repetitions of previous char or group. For the above
four –re-interval option needs to be specified
(....) Used for grouping
13. Regular Expressions – Examples
13
Example Meaning
.{10,} 10 or more characters. Curly braces have to
escaped
[0-9]{3}-[0-9]{2}-[0-9]{4} Social Security number
([2-9][0-9]{2})[0-9]{3}-[0- Phone number (xxx)yyy-zzzz
9]{4}
[0-9]{3}[ ]*[0-9]{3} Postal code in India
[0-9]{5}(-[0-9]{4})? US ZIP Code with optional four-digit extension
14. Regular Expression Patterns.
14
Example Explanation
awk „/pat1/‟ infile Same as grep „pat1‟ infile
awk „/pat1/, /pat2/‟ infile Print all lines between pat1 and pat2 repetitively
awk „/pat1|pat2/‟ infile Print lines that have either pat1 or pat2
awk „/pat1.*pat2/‟ infile Print lines that have pat1 followed by pat2 with
something or nothing in between
15. Relational Expression Patterns.
15
Example Explanation
awk „$1==“USA”‟ infile Print the line if the first field is USA
awk „$2 !=“xyz”‟ infile Print all lines whose second field is not “xyz”
awk „$2 < $3‟ infile Print all lines whose third field is greater than the
second
awk „$5 ~ /USA/‟ infile Print if the fifth field contains USA
awk „$5 !~ /USA/‟ infile Print if the fifth field doesn‟t contain USA
awk „NF == 5‟ infile Print lines that have five fields
awk „NR == 5, NR==10‟ Print lines 5 to 10
infile
awk „NR%5==0‟ infile Print every fifth line (% is the modulo operator)
awk „NR%5‟ infile Print everything other than every fifth line
awk „$NF ~ /pat1/‟ infile Print if the last field contains pat1
16. awk compound-patterns
16
Compound patterns formed with Boolean operations
(&&, ||, !), and range patterns
pat1 && pat2 (compound AND)
pat1 || pat2 (compound OR)
!pat1 (Negation)
pat1, pat2 (range pattern)
17. Compound Pattern Examples
17
Example Explanation
awk „/pat1/ && $1==“str1”‟ infile Print lines that have pat1 and whose first
field equals str1
awk „/pat1/ || $2 >= 10‟ infile Print lines that have pat1 OR whose second
field is greater than or equal to 10
awk „!/pat1/‟ infile Same as grep –v “pat1” infile
awk „NF >=3 && NF <=6‟ infile Print lines that have between 3 and six
fields
awk „/pat1/ || /pat2/‟ infile Same as awk „/pat1|pat2/‟ infile
awk „/pat1/, /pat2/‟ infile Print all lines between pat1 and pat2
repetitively
awk „!/pat1|pat2/‟ infile Print lines that have neither pat1 nor pat2
awk „NR > 30 && $1 ~ /pat1|pat2/‟ Print lines beyond 30 that have first field
infile containing either pat1 or pat2
18. Compound Pattern Examples
18
Example Explanation
awk „/pat1/&&/pat2/‟ infile Print lines that have both pat1 and pat2.
awk „/pat1.*pat2/‟ infile How is this different from the one above?
awk „NR<10 || NR>20‟ infile Print all lines except lines 10 to 20
awk „!(NR >=10 && NR<=20)‟ infile Print lines between 10 and 20. Same as awk
‘NR==10, NR==20’ infile
19. BEGIN and END patterns
19
BEGIN allows actions before any lines are processed.
END allows actions after all lines have been
processed
Either or both optional
BEGIN {action}
[Pattern] {action}
END {action}
20. BEGIN
20
Use BEGIN to:
Set initial values for variables
Print headings
Set internal field separator (same as –F on command line)
awk „BEGIN {FS=“:”; print “File name”, FILENAME}‟ file2 file2
21. END
21
Use END to:
Perform any final calculations
Print report footers.
Do any thing that must be done after all lines have been
processed.
awk „END {print NR}‟ file2 file2
22. Creating Actions
22
Actions consist of one or more statements separated
by semicolon, newline, or a right-brace.
Types of statements:
Assignment statement (e.g.var1=1)
Flow-control statements
Print control statement
23. Flow-control statements
23
Statement Explanation
if (conditional) Perform statement_list1 if conditional is true.
{statement_list1} Otherwise statement_list2 if specified
[else {statement_listt2}]
while (conditional) Perform statement_list while conditional is true
{statement_list}
for Perform int_expr firt. While conditional_expr is
(int_expr;conditional_expr true, perform statement_list and execute ctrl_expr.
;ctrl_expr) {statement_list}
break Break from the containing loop and continue with
the next statement
continue Go to the next iteration of the containing loop without
executing the remaining statements in loop
next Skip remaining patterns on this line
exit Skip the rest of the input and go to the END pattern
if one exists or exit.
24. Print-control statements
24
Statement Explanation
print [expression_list] Print the expression on stdout unless redirected to
[>filename] filename.
printf format [, Prints the output as specified in format (like printf
expression_list] in C). Has a rich set of format specifiers.
[>filename]
25. Variables
25
Provide power and flexibility
Formed with letters, numbers and underscore
character.
Can be of either string or numeric type
No need to declare or initialize.
Type implied by the assignment. No $ in front of
variables. (e.g. var1=10; job_type=„clerk‟)
Field names ($1, $2, ..$n) are special form of
variables. Can be used like any other variable.
26. Arrays
26
One-dimensional arrays: array_name[index]
Index can be either numeric or string. Starts with 1
if numeric
No special declaration needed. Simply assign
values to an array element.
No set size. Limited only by the amount of memory
on the machine.
phone[“home”], phone[“mobile”], phone[var1],
phone[$1], ranks[1]
27. Multi-Dimensional arrays
27
Arrays are one-dimensional.
Array_name[1,2] not supported
Concatenate the subscripts to form a string which
could be used as the index:
array_name[1”,”2]
Space is the concatenation operator. “1,2”, a three character
string is the index.
Use SUBSEP, subscript separator, variable to
eliminate the need to have double quotes around
the comma.
28. Built-in functions
28
Function Explanation
cos(awk_expr) Cosine of awk_expr
exp(awk_expr) Returns the exponential of awk_expr (as in e raised to the
power of awk_expr)
index(str1, str2) Returns the position of strt2 in str1.
length(str) Returns the length of str
log(awk_expr) Base-e log of awk_expr
sin(awk_expr) Sine of awk_expr
sprintf(frmt, awk_expr) Returns the value of awk_expr formatted as per frmt
sqrt(awk_expr) Square root of awk_expr
split(str, array, [field_sep]) Splits a string into its elements and stores into an array
substr(str, start, length) Returns a substring of str starting at position “start” for
“length” characters.
toupper(), tolower() Useful when doing case-insensitive searches
29. Built-in functions contd.
29
Function Explanation
sub(pat1, “pat2”, [string]) Substitute the first occurrence of pat1 with pat2 in string.
String by default is the entire line
gsub(pat1, “pat2”, [string]) Same as above, but replace all occurrences of pat1 with
pat2.
match(string, pat1) Finds the regular expression pat1, and sets two special
variables (RSTART, RLENGTH) that indicate where the
regular expression begins and ends
systime() returns the current time of day as the number of seconds
since Midnight, January 1, 1970
30. Case Insensitive Match
30
Case insensitive match:
awk „BEGIN {ignorecase=1} /PAT1/‟
awk „tolower($0) ~ /pat1/ …‟
32. Very Simple Examples
32
Find the average filesize in a directory
Find the users without password
Convert String to Word (string2word.awk)
List the file count and size for each user
(cnt_and_size.awk)
33. Awk one-liners
33
Example Explanation
awk‟{print $NF}‟ infile Print the last field in each line
awk‟{print $(NF-1)}‟ infile Print the field before the last field. What would
happen if () are removed? What happens if there is
only one field
awk‟NF‟ infile Print only non-blank lines. Same as awk „/./‟
awk „{print length, $0)‟ infile Print each line preceded by its length.
awk „BEGIN {while Print 1 to 10
(++x<11) print x}‟
awk „BEGIN {for (i=10; Print 10 to 50 in increments of 4
i<=50; i+=4) print i}‟
awk „{print; print “”}‟ infile Add a blank line after every line
awk „{print; if (NF>0) print Add a blank line after every non-blank line
“”}‟ infile
34. Awk one-liners
34
Example Explanation
awk‟NF !=0 {++cnt} END Count the number of non-blank lines
{print cnt}‟ infile
ls –l | awk „NR>1 {s+=$5} Return the average file size in a directory
END {print “Average:”
s/(NR-1)}‟
awk „/pat1/?/pat2/:/pat3/‟ uses ternary operator ?: Equivalent to awk „/pat1/ && /pat2/ ||
pat3‟ except for lines containing both pat1 and pat3
infile
awk „NF<10?/pat1/:/pat2/‟ Use pat1 if number of fields is less than 10;
infile otherwise use pat2
awk „ORS=NR%3?” ”:”n”‟ Join three adjacent lines. ORS is the output record
infile separator
awk „ORS=NR%3?”t”:”n” Print the first field three to row. ORS is the output
{print $1}‟ infile record separator
awk „FNR < 11‟ f1, f2, f3 Concatenate the first 10 lines of f1, f2, and f3.
35. Awk one-liners
35
Example Explanation
awk „length < 81‟ Print lines that are shorter than 81 characters
awk „/pat1/, 0‟ Print all lines between the line containing pat1 and
end of file
awk „NR==10, 0‟ Print lines 10 to the end of file. The end condition
“0” represents “false”.
awk '{ sub(/^[ t]+/, ""); Trim the leading tabs or spaces. Called ltrim
print }'
awk '{ sub(/[ t]+$/, ""); Trim the trailing tabs or spaces. Called rtrim
print }'
awk '{ gsub(/^[ t]+|[ Trim the white spaces on both sides
t]+$/, ""); print }'
36. Awk one-liners
36
Example Explanation
awk '/pat1/ { gsub(/pat2/, Replace pat2 with “str” on lines containing pat1
“str") }; { print }'
awk '{ $NF = ""; print }' Delete the last field on each line
41. Unanswered questions
41
How to print lines that are outside a block of lines?
(print lines that are not enclosed by /pat1/,/pat2/
Does awk support grouping and back-referencing
(e.g. identify adjacent duplicate words)?