The document provides information about using regular expressions and common Linux utilities like grep and sed to search text files. It covers the basic syntax and options for grep and sed, examples of how to use them to search and make substitutions in files, and an overview of common regular expression metacharacters. The key topics discussed are using grep to search for patterns in files, the s command and flags in sed for substitutions, and metacharacters in regular expressions like *, ., ^, $, [], etc. and what they are used to match in text searches.
This course provides you with skills to
* Develop sed and awk scripts
* Use sed and awk to automate common tasks
* Use sed and awk to create formatted reports
Prerequisites
* Basic understanding of UNIX / Linux Operating System
* Knowledge of basic UNIX / Linux commands
Intended Audience
* System Administrators, Testing Professionals, and Software Developers working in the UNIX / Linux environment
From my November 3, 2011 talk at MNPHP. Regular expressions are a powerful tool available in nearly every programming language or platform, including PHP. I go over the history of POSIX vs. PCRE, examples in PHP, and optimizations on how to write faster expressions.
sed, known as the “streams editor”, is a powerful tool for text manipulation on UNIX systems. Unlike vim, sed can operate on files of limitless size. The session will cover some of the intermediate concepts such as grouping and back referencing, regular expressions, replacing the nth occurrence of a pattern, operation on every nth line and so on. We will spice up the session with a few “sed one-liners/idioms”
This course provides you with skills to
* Develop sed and awk scripts
* Use sed and awk to automate common tasks
* Use sed and awk to create formatted reports
Prerequisites
* Basic understanding of UNIX / Linux Operating System
* Knowledge of basic UNIX / Linux commands
Intended Audience
* System Administrators, Testing Professionals, and Software Developers working in the UNIX / Linux environment
From my November 3, 2011 talk at MNPHP. Regular expressions are a powerful tool available in nearly every programming language or platform, including PHP. I go over the history of POSIX vs. PCRE, examples in PHP, and optimizations on how to write faster expressions.
sed, known as the “streams editor”, is a powerful tool for text manipulation on UNIX systems. Unlike vim, sed can operate on files of limitless size. The session will cover some of the intermediate concepts such as grouping and back referencing, regular expressions, replacing the nth occurrence of a pattern, operation on every nth line and so on. We will spice up the session with a few “sed one-liners/idioms”
Practical Example of grep command in unixJavin Paul
Grep command is one of most useful command in unix. having mastery in Grep and find means your productivity will be very high in unix. these grep command tutorials contains some examples of grep command in unix. It teaches how to leverage power of grep command in unix or linux. This presentation contains some of most useful example of grep command in unix.
for more tutorial see my blog
http://javarevisited.blogspot.com/2011/03/10-find-command-in-unix-examples-basic.html
http://javarevisited.blogspot.com/2011/06/10-examples-of-grep-command-in-unix-and.html
This lecture discusses the concept of Regular Expressions along with its usage in different tools such as grep, sed, and awk
Check the other Lectures and courses in
http://Linux4EnbeddedSystems.com
or Follow our Facebook Group at
- Facebook: @LinuxforEmbeddedSystems
Lecturer Profile:
- https://www.linkedin.com/in/ahmedelarabawy
AssignmentModify the Python program by adding the create_dictiona.pdftsekar2004
Assignment:
Modify the Python program by adding the create_dictionary function. The parameter file_name
is the name of the csv file that contains several lines of ASCII encodings. The format of this file
is described in the link above. The function should return a dictionary where the keys are
characters and the values are the 8 bit representations. Note: use the characters for comma, quote,
and space characters, instead of the words comma, quote, and space.
Add the translate function. The parameter sentence consists of a number of characters. The
parameter dictionary is the dictionary returned by the create_dictionary function. The parameter
file_name is the name of the output file where all output should be sent. In the output file, the
text should be a binary string (zeros and ones only) with each 8 bit ascii code appearing one after
the other. (see sample transcripts)
If a character does not appear in the dictionary, the string "\nUNDEFINED\n" should appear
instead of an 8 bit representation.
When the program is run, it should produce the following three output files: output-1.txt, output-
2.txt and output-3.txt.
Grading:
2 points - The create_dictionary function returns a dictionary that contains the correct binary
code for each character of length one.
1 point - The create_dictionary function returns a dictionary that contains the correct binary code
for comma, quote, and space.
3 points - When your program is tested with different values for the variable sentence, all
characters that exist in the dictionary are translated correctly.
1 point - When your program is tested with different values for the variable sentence, characters
that do NOT exist in the dictionary are translated correctly to the word UNDEFINED on a line of
its own.
1 point - The format of any output files created match the format of the sample output files
exactly. def create_dictionary(f): \# f is the filename; rename the parameter as you wish pass \#
open the file for reading and assign to a file handle variable* \# make a dictionary variable -- it
can be empty to start with \# use the file handle variable step through each line doing this.." \#
make the line into a list, splitting it up at the commas \# optional: temporarily try printing it out.
What you expected? \# assign the binary number to a variable as a string \# optional: temporarily
try printing it out. What you expected? \# assign the character to a variable as a string \#
optional: temporarily try printing it out. What you expected? \# use the two new variables to add
an entry to the dictionary \# optional: temporarily try printing it out. What you expected? \#
Note: you will want the comma, space, and quote mark keys to be those actual characters.". \#
"make sure that gets handled at some point before returning the dictionary \# *don't forget to
close the file \# return the dictionary to where it was called from def translate(s, d, f): \# rename
the parameters as you wish pass
010000010010000001101100011.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
2. Exam Objectives
Key Knowledge Areas
Create simple regular expressions containing several notational elements.
Use regular expression tools to perform searches through a filesystem or file content.
Objective 3: GNU and Unix Commands
Search text files using regular expressions Weight: 2
Terms and Utilities
grep
egrep
fgrep
sed
regex(7)
2
3. Search text files using regular expressions
Using the stream editor
3
4. Search text files using regular expressions
The sed utility is a stream editor that takes input either from a file or from data that
is piped into the utility.
Sed works globally within a file unless addressing symbols are used to limit scope
Using the stream editor
4
sed command can be used to make simple substitutions and more powerful changes to a file.
Simple substitutions throughout a file are made using the following syntax:
sed -option s/REGEXP/replacement/flag filename
sed command will work using text from standard input as well as text from specified files. Original file is
left intact, and the changes are written to a new file.
REGEXP stands for regular expression, which is a way of searching for particular characters
s/ command instructs sed to locate REGEXP and remove that while adding the replacement in its place.
sed -e ‘s/lisa/Lisa/’ -e ‘s/nikki/Nikki/’ myfriends
sed -e ‘s/lisa/Lisa/’; ‘s/nikki/Nikki/’ myfriends
Ex: search the my friends file and replace the characters lisa and nikki with Lisa and Nikki, respectively.
5. Search text files using regular expressions
Using the stream editor
5
6. Search text files using regular expressions
Using the stream editor
6
Options Used with sed
Option Use
-V Displays version information and then exits.
-h Displays help information and then exits.
-n Prevents the file from being displayed after it has been processed.
-e command Appends the commands to those being processed.
-f file Appends the commands in the specified script file to those being processed.
sed -f scriptname filename
Ex: run a script on a file by using the -f option. This enables you to store frequently used options and simplifies
larger commands
7. Search text files using regular expressions
Using the stream editor
7
Flags Used with s - flags allow further configuration of command
Flag Use
g Applies the changes globally.
p Prints all lines that contain a substitution. (Normally used with the -n option.)
NUMBER Replaces only the NUMBER match.
w filename Writes all lines with substitutions to the file specified.
I Ignores case when matching REGEXP.
Addressing Used with sed
Address Use
number Specifies the line number to match.
number, number Specifies the line numbers to match and includes all lines located between these numbers.
$ Specifies to match the last line.
! Matches all lines except for the lines specified.
8. Search text files using regular expressions
Using the stream editor
8
9. Search text files using regular expressions
Using the stream editor
9
10. Search text files using regular expressions
Using the stream editor
10
11. Search text files using regular expressions
Using the stream editor
11
14. Search text files using regular expressions
Using grep
14
grep utility is used to search files for the pattern specified.
•Default action of the utility is to print the matches to the search.
•Can accept filenames to search or it can search data from standard input.
Syntax: grep –options [-e searchpattern] [-f filename]
Used with 3 variants controlled by options:
Option Use
-G The default behavior which interprets the pattern as a basic regular expression.
-E Interprets the pattern as an extended regular expression.
This option functions the same as the –G option with GNU grep.
-F Interprets the pattern as a list of fixed strings.
19. Search text files using regular expressions
Regular expressions are patterns of characters, some with special meaning, that
are useful when using text filters.
Special characters are and can be used to represent other characters or groups of characters.
These special characters are known as metacharacters.
Regular expressions
19
It is important to understand that although these characters may appear to be the same as
wildcard characters used at a shell prompt, they function differently.
20. Search text files using regular expressions
metacharacters
20
Metacharacter Use
The slash is used to locate any of the characters following the slash.
* The asterisk matches zero or more occurrences of the preceding regular expression. The zero or more occurrence matching
is useful when using this character along with others. For example, when searching for *are, matches for are and stare are returned.
. A dot matches any single character; this character is used as a wildcard.
^ The caret is used to locate the start line; it is often followed by another character to locate a line starting with that
character. Using this metacharacter to search for ^A would locate all lines beginning with A.
$ The dollar sign locates the end of the line, and when proceeded by another character, it will locate lines ending with that
character. So, using a$ will locate all lines ending with a.
[ ] Brackets are used to locate specific characters; a range of characters can also be specified within brackets. When a range is specified such
as 1-5, the numbers 1, 2, 3, 4, and 5 are specified.
[^ ] Brackets with a caret as the first character contained between them search for all characters except those that are also contained within
the brackets. So, for [^1-9] all characters are found except for the numbers ranging from one to nine.
{ } These symbols are used to locate a range or specific number of instances. The expression a{3} will search for aaa while the expression a
{1,3} will locate a, aa, and aaa.
< > The slash and less than symbols are used with a set of characters followed by the slash and greater than symbols. The characters that are
located within the symbols are searched for at the word boundary. This allows you to locate complete words, regardless of where they are located within a sentence.