Perl Introduction

Perl Welcome Pratical Extraction and Reporting Language Marcos Rebelo oleber@gmail.com

Perl - Larry Wall

Larry Wall, programmer, linguist, author, born March 10, 1949 in Duncan, British Columbia, Canada. Perl computer language creation in 1987.
Quotes:

The three chief virtues of a programmer are: Laziness, Impatience and Hubris
There's More Than One Way to Do It

What is Perl?

Perl is a general-purpose programming language originally developed for text manipulation and now used for a wide range of tasks including web development, system administration, network programming, code generation and more.

It's intended to be practical rather than beautiful.
Supports both procedural and OO programming.
It has powerful built-in support for text processing
It has one of the world's most impressive collections of third-party modules (http://www.cpan.org).

Perl – Pros

Quick development phases
Open Source, and free licencing
Excellent text handling and regular expressions
Large experienced active user base
Fast, for an interpreted language
Code developed is generally cross plaform
Very easy to write powerful programs in a few lines of code

Perl – Cons

Limited GUI support
Can look complicated, initially, particularly if you're not familiar with regular expressions

Consider using for

Web applications
System administration
Most command line applications
Batch processing
Reports
Database access and manipulation
Any application involving significant text processing

Hello World

in a file:

print "Hello, World! "

on the shell

#> perl -e 'print "Hello, World! "'

with perl 5.10

#> perl -E 'say "Hello, World!"'

Basic syntax overview

Perl statements end in a semi-colon. say("Hello, world!") ;
You can use parentheses for subroutines arguments or omit them according to your personal taste. say "Hello, world!";
Comments start with a # and run to the end of the line. # This is a comment

Double quotes or single quotes may be used around literal strings: say( " Hello, world! " ); say( ' Hello, world! ' );
However, only double quotes interpolate variables and special characters such as newlines ( ): print("Hello, $name! "); # interpolate $name print('Hello, $name! '); # prints $name! literally

Whitespaces are irrelevant: print( "Hello world");
... except inside quoted strings: print("Hello world"); # this would print with a # line break in the middle.

Use feature qw(say)

Perl 5.10 as added a new feature

use feature qw(say); say 'Hello, world!';

Is equivalent to:

print 'Hello, world!', " ";

We will see later that the correct expression is:

{ local $ = " "; print 'Hello, world!' }

Running Perl programs

To run a Perl program from the Unix command line: #> perl progname.pl
Alternatively, put this shebang in your script : #!/usr/bin/perl and run your executable script #> progname.pl
-e : allows you to define the Perl code in the command line to be executed, -E to get 5.10 #> perl -e 'print("Hello, World! ")'

Perl variable types

Scalars my $ animal = 'camel'; my $ answer = 42; my $ scalar_ref = $scalar;
Arrays my @ animals = ('camel', 'lama'); my @ numbers = (23, 42, 69); my @ mixed = ('camel', 42, 1.23);
Associative Array / Hash Tables my % fruit_color = ( apple => 'red', banana => 'yellow' );

Scalar Type

Scalar values can be strings, integers, floating point numbers or references. The variable name starts with a $ .

my $ animal = 'camel';
my $ answer = 42;
my $ scalar_ref = $scalar;

Perl will automatically convert between types as required.

print('3' + 4.5); # prints 7.5

Array Type

An array represents a list of scalars. The variable name starts with a @ .

my @ animals = ('camel', 'llama');
my @ numbers = (23, 42, 69);
my @ mixed = ('camel', 42, 1.23);

Arrays are zero-indexed, negative index start by -1 at the end of the array.

print( $ animals [0] ); # prints "camel"
print( $ numbers [-1] ); # prints 69

Array Slice

To get multiple values from an array:

@ numbers [0,1] ; # gives (23, 42);
@ numbers [0..2] ; # gives (23, 42, 69);
@ numbers [1..$#numbers] ;

The single element we're getting from the array start with a $ , the sub-array start with @ .
@ numbers [2, 1] = @ numbers [1, 2] ;
($scalar1, $scalar2) = @numbers;

Array Type

The special variable $# array tells you the index of the last element of an array:

print($mixed[ $# mixed]); # prints 1.23

You might be tempted to use $#array+1 to tell you how many items there are in an array. Using @array in a scalar context will give you the number of elements:

if (@animals < 5) { ... }
if ( scalar( @animals ) < 5) { ... }

Associative Array Type Hash table

A hash represents a set of key/value pairs:

my % fruit_color = ( 'apple', 'red', 'banana', 'yellow' );

You can use whitespace and the => operator to lay them out more nicely:

my % fruit_color = ( apple => 'red', ' banana ' => 'yellow' );

Associative Array Type Hash table

To get at hash elements:

$ fruit_color{ ' apple ' }; # "red"
$ fruit_color{banana}; # "yellow"

You can get at lists of keys and values with keys() and values() .

my @fruits = keys (%fruit_colors);
my @colors = values (%fruit_colors);

Hashes have no particular internal order, though you can sort the keys and loop through them.

Hash Slice

To get/set multiple values from an hash

@ fruit_color { 'watermelon', 'orange' } = ('green', 'orange');

Removing repetition from an array

my @array = (1,2,9,5,2,5); my %hash ; @hash{@array} = (); @array = keys(%hash) ; say "@array"; # 1 9 2 5

Variable Reference

Scalar references

$ scalar, '' , 123

Array references

@ array, [ 'camel', 42, 1.23 ]

Hash references

\% hash, { apple => 'red', banana => 'yellow' }

Unref a variable

$$ scalar_ref, @$ array_ref, %$ hash_ref

Complex data types

More complex data types can be constructed using references:

my $cities = { IT => [ 'Milan', 'Rome', ... ] , PT => [ 'Lisbon', 'Oporto', ... ] , ... } ;

You can access it throw:

print($cities ->{ ' IT' } -> [ 1 ] ); print($cities ->{ ' IT' }[ 1 ] );
my @citiesPT = @{ $cities->{PT} } ;

Lexical Variables

Throughout the previous slides the examples have used the syntax:

my $var = 'value';

The my is actually not required, you could use:

$var = 'value';

However, the above usage will create global variables throughout your program.

my creates lexically scoped variables instead. The variables are scoped to the block in which they are defined.

Lexical Variables

Not having the variables scoped is usually a bad programming practice.

my $a = 'foo'; if ($some_condition) { my $b = 'bar'; print($a); # prints "foo" print($b); # prints "bar" } print($a); # prints "foo" print($b); # prints nothing; $b has # fallen out of scope

Lexical Variables

It’s recommended to use my in combination with a use strict at the top of your programs. Using the pragma strict is highly recommended.

use strict; my $first = "first"; # OK $second = "second"; # NOT OK

Also highly recommended is the pragma warnings for having aditional warnings. You can enable or deseable this warnings.

Magic variables

There are some ‘magic’ variables. These special variables are used for all kinds of purposes.

$_ : is the ‘default variable’.
@ARGV : the command line arguments to your script.
$ARGV : contains the name of the current file when reading from <> or readline() .
@_ : the arguments passed to a subroutine.
$a , $b : Special package variables when using sort() .
$/ : The input record separator, newline by default.

Most common operators

Arithmetic

+ += addition
- -= subtraction
* *= multiplication
/ /= division

Boolean logic

&& and
|| or
! not

Miscellaneous

= assignment
. string concatenation
x string multiplication
.. range operator (creates a list of numbers)

Conditional constructs

Perl has most of the usual conditional

if COND BLOCK
if COND BLOCK else BLOCK
if COND BLOCK elsif COND BLOCK
if COND BLOCK elsif COND BLOCK else BLOCK

The COND shall be conditional statement surround by ( and ) , and BLOCK shall be one ore more statements surround by { and } .

if ( is_valid( $value ) ) { … }

There's also a negated version of if (don't use it): unless ( is_valid( $value ) ) { ... }
This is provided as a 'more readable' version of if ( not( is_valid( $value ) ) ) { ... }
0, '0', '', () and undef are all false in a boolean context. All other values are true.

Note that the braces are required in Perl, even if you've only got one line in the block. However, there is a clever way of making your one-line conditional blocks more English like:

The traditional way if ($zippy) { print("Yow!"); }
The Perlish post-condition way print("Yow!") if $zippy; print("No cubes") unless $cubes;

while looping constructs

Perl has most of the usual loop constructs.

While Loop: while ( is_valid( $value ) ) { … }
There's also a negated version (don't use it): until ( is_valid( $value ) ) { … }
You can also use while in a post-condition: print("xpto ") while condition;
Going throw a hash: while (my ($key,$value) = each (%ENV)){ print "$key=$value "; }

for looping constructs

for (my $i=0; $i <= $max; $i++) { ... }

The C style for loop is rarely needed in Perl since Perl provides the more friendly foreach loop.

foreach my $i (0 .. $max) { ... }

foreach looping constructs

Passing all elements of an array, foreach is an alias to for .

foreach my $var (@array) { ... }
for my $value (values(%hash)) { ... }

By default the value goes on $_

foreach (@array) { print " $_ " }

Changing the variable, changes the value inside the array. $var is an alias.

for my $var (@array) { $var += $var }

Jump on loops

LINE: while ( defined(my $line = <>) ) { next LINE if not is_valid( $line ); #... }
Jump on loops:

last LABEL : immediately exits the loop in question
next LABEL : starts the next iteration of the loop
redo LABEL : restarts the loop block without evaluating the conditional again

If the LABEL is omitted, the command refers to the innermost enclosing loop.

Inverting the cycle

With a single command

print foreach (0..9)

With multiply commands

do { ... } while ($true)
Warning: last , next , and redo don't work in this case.

Exercises 1 - Scalars 1) Implement the ‘Guess the lucky number’. The program shall chose a random number between 0 and 100 and ask the user for a guess. Notice the user if the guess is bigger, smaller or equal to the random number. If the guess is correct the program shall leave otherwise re-ask for a number.

Exercises 1 - Array 2) Create an array that contains the names of 5 students of this class. 2a) Print the array. 2b) Create a new Array shifting the elements left by one positions (element 1 goes to 0, …) and setting the first element in the last position. Print the array. 2c) Ask a user to input a number. Print the name with that index.

Exercises 1 - Hash 3) Homer Family

my %relatives = ( Lisa => 'daughter', Bart => 'son', Maggie => 'daughter', Marge => 'mother', Homer => 'father', Santa => 'dog');

3a) Print all the characters name’s. 3b) Demand for a name and print is position. 3c) Print all the characters position’s, no repetitions. 3d) Demand for a position and print the name.

Subroutines

The Perl model for subroutine call and return values is simple:

all subroutine are passed as parameters one single flat list of scalars (goes to @_ )
all subroutines likewise return to their caller one scalar or a list of scalars.
lists or hashes in these lists will collapse, losing their identities - but you may always pass a reference.
The subroutine name start with a & , for simplification can be omitted.

Subroutine - example

sub max { my $mval = shift(@_) ; # my ($mval, @rest) = @_ ; # big copy foreach my $foo ( @_ ) { if ( $mval < $foo ) { $mval = $foo; } } return $mval; }
my $my_max = max( 1, 9, 3, 7 ) ; print $my_max; # prints 9

Subroutine input and output

The parameters are passed to the subroutine in the array @_ , changing the values of the array will change the values outside the subroutine call.

sub double { $_[0] *= 2 ; } my $b = 5; double($b); print($b); # prints 10

The last statement value is returned by default.

print(double($b)); # prints 20

Subroutine input and output

Using @_ is dangerouse and shall be carefully considered. It's always possible to do a copy.

sub double { my ($a) = @_; $a *= 2; return $a; } my $b = 5; print( double( $b ) ); # prints 10 print($b); # prints 5

Persistent Private Variables

Just because a lexical variable is statically scoped to its enclosing block, eval, or file.

{ my $secretVal = 0; sub gimmeAnother { return ++$secretVal; } } print gimmeAnother; # OK ++$secretVal; # NOT OK

state variables

From perl 5.10 you may use static variables.

use feature 'state'; sub gimmeAnother { state $secretVal = 0; return ++$secretVal; } print gimmeAnother; # OK

Some features in perl 5.10 have to be activated to avoid colisions with old code. Activating all them:

use feature ':5.10';

use v5.10; # use perl 5.10

Named Parameters

We can implement named parameters using a hash. This provides an elegant way to pass in parameters without having to define them formally.

sub login { my %param = @_; ... } login( user=>'User', pass=>'Pass' );

Named Parameters

We may pass the hash directly.

sub login { my ($param) = @_; ... } login({ user=>'User', pass=>'Pass' });

Named Parameters

We can easily give default values by checking the hash.

sub login { my ($param) = @_; $param->{user} = $DEFAULT_USER if not exists $param->{user}; $param -> {pass} = $DEFAULT_PASS if not exists $param->{pass}; $param->{host} = $DEFAULT_HOST if not exists $param->{host}; ... }

Named Parameters

We can easily give default values by checking the hash.

sub login { my $param = { 'user' => $DEFAULT_USER, 'pass' => $DEFAULT_PASS, 'host' => $DEFAULT_HOST, %{shift(@_)} }; ... }

Named Parameters

We can also write the subroutine so that it accepts both named parameters and a simple list.

sub login { my $param; if ( ref($_[0]) eq 'HASH' ) { $param = $_[0]; } else { @{$param}{qw(user pass host)}=@_; } ... } login('Login', 'Pass'); login({user => 'Login', pass => 'Pass'});

Exercises 2 1) Create a new subroutine that calculates the Fibonacci series. Using this subroutine, do a program that receives multiple numbers as argument and prints the Fibonacci value. F(0) = 0 F(1) = 1 F(n) = F(n-1) + F(n-2) 1a) with presistent variable 1b) with state variable

IO

Read a File

open( FH, '<', 'path/to/file' ) or die "can't open file: $! "; while ( defined( my $line = <FH> ) ) { chomp($line); … } close( FH ) ;

Open a Filehandler

Opening a file for input.

open (INFH, " < ", "input.txt") or die $!;
open (INFH, " < input.txt") or die $!;
open (INFH, "input.txt") or die $!;

Opening a file for output.

open (OUTFH, " > ", "output.txt") or die $!;
open (OUTFH, " > output.txt") or die $!;

Opening a file for appending

open (LOGFH, " >> ", "my.log") or die $!;
open (LOGFH, " >> my.log") or die $!;

Open a Filehandler

You can also use a scalar variable as filehandle:

open( my $inFH , "input.txt") or die $!;
a lexical scalar variable closes at the end of the block if it wasn't closed before

It's possible to pipe from a command:

open(my $net, "netstat | ") or die "Can't netstat: $!";

It's possible to pipe to a command:

open(my $sendmail, " | sendmail -t") or die "Can't open sendmail: $!";

Write to a Filehandle

We've already seen how to print to standard output using print(). However, print() can also take an optional first argument specifying which filehandle to print to:

print STDERR ('Are you there?');
print OUTFH $record;
print { $FH } $logMessage;
Note: There is no ‘,’ between filehandle and the text.

Closing File Handles:

close($inFH);

Read from a Filehandle

You can read from an open filehandle using the <> operator or the readline() subroutine.

Line by line: my $line = <$fh> ; my $line = readline($fh) ;
Slurp: my @lines = <$fh> ; my @lines = readline($fh) ;

Read from a Filehandle

Slurping a file can be useful but it may be a memory hog (usually if you are programming in Perl that is not a problem, the actual computers already have a lot of memory). Most text file processing can be done a line at a time with Perl's looping constructs.

while ( defined(my $line = <$fh>) ) { print "Just read: $line"; }
foreach my $line (<$fh>) { # slurps print "Just read: $line"; }

Input Record Separator

$/ is the input record separator (newline by default). You may set it to a multi-character string to match a multi-character terminator or to undef to read through the end of file.

open(my $fh, '<', $myfile) or die $!; my $txt = do{ local $/ = undef ; <$fh>}; close($fh);
my $txt = do { local (@ARGV, $/) = ($myfile); readline(); };

Read from @ARGV file list

The magic filehandle * ARGV can be used to eseally process the input. Perl actually does an implicit open on each file in @ARGV (process parameters). Usually <ARGV> is simplyfied to <> . In this case:

while (my $line = <> ) { # use $line }
while ( <> ) { # use $_ }

If @ARGV is empty when the loop first begins, Perl pretends you've opened STDIN .
$ARGV has the open file, or '-' if reading STDIN .

*DATA Filehandle

This special filehandle refers to anything following either the __END__ token or the __DATA__ token in the current file.

The __END__ token always opens the main::DATA filehandle, and so is used in the main program.
The __DATA__ token opens the DATA handle in whichever package is in effect at the time, so different modules can each have their own DATA filehandle, since they (presumably) have different package names.

Exercises 3

Create a program to print a file to the stdout. The program shall receive two flags (--file, --line) followed by a list of file names:

--file : printing the file name before the line.
--line : printing the line number before the line.
Create a version with open/close and one without.

The program shall be called like:

perl -w printFile.pl (--line|--file)* files

Note: $. is the line number for the last filehandle accessed.

Exercises 3

Create a program that has a mapping of integer to integer in the DATA section, 2 integer in each line separated by a space. From the ARGV/STDIN will get a TSV file. The program shall print the second file adding a second column with:

If the first column value is a key in the DATA section, set the value in the DATA section .
Otherwise, set 'MAPPING NOT FOUND'.

To split a string use: split(/ /, $str)

Regular Expressions

The simplest regex is simply a word.

"Hello World" =~ m/ World /
Or simply: "Hello World" =~ / World /

Expressions like this are useful in conditionals:

print "Matches" if $str =~ / World / ;

The sense of the match can be reversed by using !~ operator:

print "No match" if $str !~ / Word / ;

Regular Expressions

Regexps are treated mostly as double quoted strings, so variable substitution works:

$foo = 'house'; 'cathouse' =~ /cat $foo /; # matches 'housecat' =~ / ${foo} cat/; # matches

qr// compiles the Regular Expression.

foreach my $regexp (@regexps) { my $comp = qr/ $regexp / ; foreach my $str (@strs) { print '$str ' if $str =~ / $comp /; } }

To specify where the regex should match, we would use the anchor metacharacters ^ and $ .

The ^ means match at the beginning of the string. print 'Starts with Hello' if /^Hello/;
The $ means match at the end of the string or before a newline at the end of the string. print 'Ends with World!' if /World!$/;

Character Classes

A character class allows a set of possible characters, rather than just a single character. Character classes are denoted by brackets [] .

/ [bcr] at/ - matches 'bat', 'cat', or 'rat‘
/ [yY][eE][sS] / - match 'yes' case-insensitively.
/yes/ i - also match 'yes' in a case-insensitive way.

The special character '-' acts as a range operator, so [0123456789] become [0-9] :

/item [0-9] / - matches 'item0' or ... or 'item9‘
/ [0-9a-f] / i - matches a hexadecimal digit

Character Classes

The special character ^ in the first position of a character class denotes a negated character class, which matches any character but those in the brackets. Both [...] and [^...] must match one character, or the match fails. Then

/ [^a] at/ - doesn't match 'aat' or 'at', but matches all other 'bat', 'cat', ....
/ [^0-9] / - matches a non-numeric character
/ [a^] at/ - matches 'aat' or '^at'; here '^' is ordinary

Character Classes

Character classes also have ordinary and special characters, but the sets of ordinary and special characters inside a character class are different than those outside a character class. The special characters for a character class are -]^$ and are matched using an escape:

/ []c] def/ - matches ‘]def’ or ‘cdef’
$x = 'bcr'; / [$x] at/ - matches ‘bat’, ‘cat’, or ‘rat’ / [$x] at/ - matches ‘$at’ or ‘xat’ and not ‘bat’, … / [\$x] at/ - matches ‘at’, ‘bat’, ‘cat’, or ‘rat’

Character Classes

Perl has several abbreviations for common character classes:

d is a digit character [0-9]
D is a non-digit character [^d]
s is a whitespace character [ f]
S is non-whitespace character [^s]
w is a word character [0-9a-zA-Z_]
W is a non-word character [^w]
The period '.' matches any character but [^ ]

Character Classes

The dswDSW abbreviations can be used both inside and outside of character classes. Here are some in use:

/ dd : dd : dd / - matches a hh:mm:ss time format
/[ ds ]/ - matches any digit or whitespace character
/ .. rt/ - matches any two chars, followed by 'rt'
/end . / - matches 'end.'
/end [.] / - same thing, matches 'end.‘

Exercises 4

Create a Regular Expression that match with a string that contains ‘a’ or ‘b’ followed by any 2 characters followed by a digit. The strings ‘(adc34)’ and ‘rdb850’ matches, but ‘alfa’ doesn’t match.
Create a Regular Expression that match a 5 digit integer in octal format.
Create a program that receives one regexp as the first parameter and a file list and prints the lines matching the regexp.

Quantifiers

Quantifiers can be used to specify how many of the previous thing you want to match on, where “thing” means either a literal character, one of the metacharacters listed above, or a group of characters or metacharacters in parentheses.

* - zero or more
+ - one or more
? - zero or one
{3} - matches exactly 3
{3,6} - matches between 3 and 6
{3,} - matches 3 or more

Quantifiers

If you want it to match the minimum number of times possible, follow the quantifier with a ?.

'a,b,c,d' =~ /,( .+ ),/; # match 'b,c'
'a,b,c,d' =~ /,( .+? ),/; # match 'b'

Avoid unnecesary backtracing :

'[1234567890]' =~ /[ d++ ] /

Matching this or that

We can match different character strings with the alternation metacharacter '|'. Perl will try to match the regex at the earliest possible point in the string.

Even though dog is the first alternative in the second regex, cat is able to match earlier in the string. 'cats or dogs' =~ /cat | dog/; # matches 'cat' 'cats or dogs' =~ /dog | cat/; # matches 'cat'
At a given character position, the first alternative that allows the regex match to succeed will be the one that matches. 'cats' =~ /c | ca | cat | cats/; # matches 'c' 'cats' =~ /cats | cat | ca | c/; # matches 'cats'

Grouping things and hierarchical matching

The grouping metacharacters () allow a part of a regex to be treated as a single unit. Parts of a regex are grouped by enclosing them in parentheses.

/ ( a|b ) b/ - matches 'ab' or 'bb'.
/ ( ^a|b ) c/ - matches 'ac' at start of string or 'bc' anywhere.
/house ( cat| ) / - matches either 'housecat' or 'house'.
/house ( cat ( s| ) | ) / - matches either 'housecats' or 'housecat' or 'house'.

Extracting matches

The grouping metacharacters () also allow the extraction of the parts of a string that matched. For each grouping, the part that matched inside goes into the special variables $1 , $2 , etc.

# extract time in hh:mm:ss format $time =~ /(dd):(dd):(dd)/; my ($hour, $min, $sec) = ( $1 , $2 , $3 );

In list context, a match /regex/ with groupings will return the list of matched values ($1, $2, ...) .

my ($hour, $min, $sec) = ($time =~ /(dd):(dd):(dd)/);

Extracting matches

To get a list of matches we can use:

my @listOfNumber = ($txt =~ /(d+)/g);

If the groupings in a regex are nested, $1 gets the group with the leftmost opening parenthesis, $2 the next opening parenthesis, etc. For example, here is a complex regex and the matching variables indicated below it:

/ ( ab ( cd|ef ) (( gi)|j))/; 1 2 34

Named capture

Identical to normal capturing parentheses () but %+ or %- may be used after a successful match to refer to a named buffer.

'Michael Jackson' =~ /( ?<NAME> w+)s+( ?<NAME> w+)/
%+ is ('NAME' => 'Michael')
%- is ('NAME' => ['Michael','Jackson'])
$1 is 'Michael'
$2 is 'Jackson'

Search and replace

Search and replace is performed using s/regex/replacement/. The replacement replaces in the string whatever is matched with the regex.

$x = "'Good cat!'"; $x =~ s/ cat / dog / ; # "'Good dog!'" $x =~ s/ '(.*)' / $1 / ; # "Good dog!"

With the global modifier, s///g will search and replace all occurrences of the regex in the string:

$x = $y = '4 by 4'; $x =~ s/4/four/; # 'four by 4' $y =~ s/4/four/ g ; # 'four by four'

The split operator

split(/regex/, string) - splits the string into a list of substrings and returns that list. The regex determines the character sequence that string is split with respect to.

split(/s+/, 'Calvin and Hobbes'); # ('Calvin', 'and', 'Hobbes')

If the empty regex // is used, the string is split into individual characters.

split(//, 'abc'); # ('a', 'b', 'c')

The split operator

If the regex has groupings, then the list produced contains the matched substrings from the groupings as well:

split(m!(/)!, '/usr/bin'); # ('', '/', 'usr', '/', 'bin')

Since the first character of string matched the regex, split prepended an empty initial element to the list.

Magic Variables

We have already seen $1 , $2 , ... There is also:

$` - The string preceding whatever was matched.
$& - The string matched.
$' - The string following whatever was matched.

These variables are read-only and dynamically scoped to the current BLOCK.

'abcdef' =~ /cd/; print(" $` : $& : $' "); # prints ab:cd:ef

Make the Regexp slow

'abcdef'=~/^(.*?)(cd)(.*)$/; # $1, $2, $3

Switch use qw(switch say); given ($foo) { when (undef) {say '$foo is undefined'} when ('foo') {say '$foo is the str "foo"'} when (/Milan/) {say '$foo matches /Milan/'} when ([1,3,5,7,9]) { say '$foo is an odd digit'; continue ; # Fall through } when ($_ < 100) {say '$foo less than 100'} when (&check) {say 'check($foo) is true'} default {die 'what shall I do with $foo?'} }

Smart Matching

given(EXPR) will assign the value of EXPR to $_ within the lexical scope of the block.
when($foo) is equivalent to when($_ ~~ $foo)

Smart Matching $a $b Matching Code Code Code $a == $b # not empty prototype if any Any Code $b->($a) # not empty prototype if any Hash Hash [sort keys %$a]~~[sort keys %$b] Hash Array grep {exists $a->{$_}} @$b Hash Regex grep /$b/, keys %$a Hash Any exists $a->{$b} Array Array arrays are identical, value by value Array Regex grep /$b/, @$a Array Num grep $_ == $b, @$a Array Any grep $_ eq $b, @$a Any undef !defined $a Any Regex $a =~ /$b/ Code() Code() $a->() eq $b->() Any Code() $b->() # ignoring $a Num numish $a == $b Any Str $a eq $b Any Num $a == $b Any Any $a eq $b

Exercises 5

Create a program that printout the title inside a html file.
Create a program that printout all the lines in a file substituting the numbers by #.
In the printfile.pl add one adicional flag like:

--regexp=REGEXP - the program shall only print the lines that match with the REGEXP.

Exercises 5

For each of the strings , say which of the patterns it matches. Where there is a match, what would be the values of $MATCH , $1 , $2 , etc.?

'The Beatles (White Album) - Ob-La-Di, Ob-La-Da' 'Tel: 212945900' '(c) (.+)s*1'
RegExp /((.*))/ and /((.*?))/ /d{4,}/ /(ww)-(ww)-(ww)/ /W+/

Core subroutines

Useful scalar subroutines

lc(EXPR), lcfirst(EXPR), uc(EXPR), ucfirst(EXPR) - lower case, lower case first, upper case and uppercase first.
length(EXPR) - Returns the length in characters of the value of EXPR.
sprintf(FORMAT, LIST) - Returns a string formatted by the usual printf conventions of C.
abs(EXPR), cos(EXPR), int(EXPR), log(EXPR), sin(EXPR), sqrt(EXPR) - normal numeric subroutines.

chop/chomp

chop(VARIABLE), chop(LIST) : Chops off the last character of a string and returns the character chopped.
chomp(VARIABLE), chomp(LIST) : Removes any trailing string that corresponds to the current value of $/ .

chomp( $line ); chomp( $line = <> ); chomp( @lines = <> );

substr

substr EXPR,OFFSET,LENGTH,REPLACEMENT substr EXPR,OFFSET,LENGTH substr EXPR,OFFSET
Extracts a substring out of EXPR and returns it

my $var = 'Good dog'; say substr($var, 5); # 'dog' substr($var, 5) = 'cat'; # 'Good cat' substr($var, 5, 5, 'cow'); # 'Good cow'

Useful scalar subroutines

oct(EXPR) - Interprets EXPR as an octal string and returns the corresponding value. If EXPR happens to start off with:

0x - interprets it as a hex string.
0b - interpreted it as a binary string.

defined(EXPR) - Returns a boolean value telling whether EXPR has a value other than the undefined value undef.
Note: If EXPR is omitted, uses $_ .

Useful list subroutines

pop(ARRAY), push(ARRAY, LIST) - Pops/Pushes a value from/to the end of the ARRAY .
shift(ARRAY), unshift(ARRAY, LIST) - Pops/Pushes a value from/to the start of the ARRAY .
Note: in the pop and shift if ARRAY is omitted, pops the @ARGV array in the main program, and the @_ array in subroutines. Avoid to use it.

join

join(EXPR,LIST) - Joins the separate strings of LIST separated by the value of EXPR .

join (':', (1..5)) eq '1:2:3:4:5'

reverse

reverse(LIST)

In list context, returns a list value consisting of the elements of LIST in the opposite order.

print reverse (1,'ab'); # prints "ab1"

In scalar context, concatenates the elements of LIST and returns a string value with all characters in the opposite order.

print reverse (1,"ab").""; # prints "ba1"
print scalar( reverse (1,"ab"));

Inverting the keys/values in a hash.

%by_name = reverse (%by_address);

map

map BLOCK LIST or map(EXPR,LIST) - Evaluates the BLOCK or EXPR for each element of LIST (locally setting $_ to each element) and returns the list value composed of the results of each such evaluation.

@double = map {$_*2} @nums;
my @double; foreach (@nums) { push(@double, $_*2) }

Note: $_ is an alias to the list value.

grep

grep BLOCK LIST or grep(EXPR,LIST) – Filters the elements in LIST using the BLOCK or EXPR for each element of LIST (locally setting $_ to each element). In scalar context, returns the number of filtered elements.

my @even = grep {$_ % 2 == 0} (1..100);
my @even; foreach (1..100) { push(@even, $_) if $_ % 2 == 0; }

Note: $_ is an alias to the list value.

sort

sort(LIST) - In list context, this sorts the LIST and returns the sorted list value.

By default compares the elements as strings. sort (10,9,20) #10, 20, 9
Providing a closure, the elements come in $a and $b . sort {$a <=> $b} (10,9,20) # 9, 10, 20
Schwartzian transform

@sorted = map { $_->[0] } sort { $a->[1] cmp $b->[1] } map { [ $_, foo($_) ] } @unsorted;

each

each(HASH) - When called in list context, returns a 2-element list consisting of the key and value for the next element of a hash, so that you can iterate over it. When called in scalar context, returns only the key for the next element in the hash.

while (my ($key,$val) = each (%ENV)) { print("$key=$val "); }

exists

exists EXPR - Given an expression that specifies a hash or an array, returns true if the specified element has ever been initialized, even if the corresponding value is undefined.

my @a = (1, undef); $a[3] = undef; exists ($a[1]) # true exists ($a[3]) # true exists ($a[2]) # false
my %a = ('a' => 1); exists ($a{'a'}) # true exists ($a{'b'}) # false

delete

delete(EXPR) - Given an expression that specifies a hash element or array element deletes the specified element(s) from the hash or array.

my @array = (a => 1, b => 2, c => 3); delete($array[2]); # ('a',1,2,'c',3);
my %hash = (a => 1, b => 2, c => 3); delete($hash{b}); # (a => 1, c => 3);

In the case of an array, if the array elements happen to be at the end, the size of the array will shrink to the highest element that tests true for exists().

eval / die

eval EXPR : compiles and evaluates the expression and catch's the exception.
eval BLOCK : evaluates the expression and catch's the exception.
die EXPR : if out of eval prints the error and exits with the value of $!, otherwise sets the value in $@ and exits eval with undef.
eval { $answer = $a / $b; }; warn $@ if $@;

Exercises 6

Create a subroutine that receives an array and returns a hash reference with the elements as keys and the inversed string as values.
Create a subroutine that receives an array of scalars and return a new one just with the strings with length smaller than 20.
Create a program that reads a file and prints all the strings capitalized. This means:

First character in upper-case.
The remaining in lower-case.

Modules and OO

Package

The idea is to protected each package variables.

package Dog; our $c = 1; my $d = 1; sub inc {$c++; $d++} package main; our $c = 0; my $d = 0; sub inc {$c++; $d++} print("$d-$c-$Dog::d-$Dog::c ");# "0-0--1" Dog::inc(); print("$d-$c-$Dog::d-$Dog::c ");# "0-0--2" inc(); print("$d-$c-$Dog::d-$Dog::c ");# "1-1--2"

use Modules

The module is just a package defined in a file of the same name with .pm on the end.
Perl modules are typically included by saying:

use MODULE LIST;
BEGIN { require MODULE; MODULE->import( LIST ); }

use Data::Dumper; # exports Dumper print(Dumper({1 => 3}));

use Modules

Any double colons in the module name are translated into your system's directory separator, so if your module is named My::Mod , Perl might look for it as My/Mod.pm .
Perl will search for modules in each of the directories listed in the @INC array. If it is need new directories you can use the lib pragma.

use lib '/path/to/modules';

Write Modules

To start a traditional module create a file called Some/Module.pm and start with this template:

package Some::Module; # package name use strict; use warnings; # use always use Exporter; our @ISA = qw(Exporter); our $VERSION = 05.22; our @EXPORT = qw($var1 &func1); our @EXPORT_OK = qw($var2 &func2); our ( $var1, $var2 ) = ( 1, 2 ); sub func1() {print("func1 ");} sub func2() {print("func2 ");} 1; # has to finnish with a true value

Write Modules

use Exporter; our @ISA = qw(Exporter); Import Exporter module. Derive the methods of Exporter module. Each element of the @ISA array is just the name of another package, the packages are searched for missing methods in the order that they occur. use base qw(Exporter); # is similar
our $VERSION = 05.22; Sets the version number. Importing like: use Some::Module 6.15;

Write Modules

our @EXPORT = qw($var1 &func1); Lists of symbols that are going to be exported by default (avoid to use it).
our @EXPORT_OK = qw($var2 &func2); Lists of symbols that are going to be exported by request (better practice).
our ( $var1, $var2 ) = ( 1, 2 ); sub func1() {print("func1 ");} sub func2() {print("func2 ");} Definition of the symbols.

Write Modules

This is ugly but can be used to call subroutine as well.

my $name = "Some::Module";
The package name pass as the first parameter of the subroutine. Some::Module->func(); $name->func();
This will not pass the module name inside the subroutine Some::Module::func(); &{"${name}::func"}();

Perl Objects

There are three very simple definitions.

An object is simply a reference that happens to know which class it belongs to.
A class is simply a package that happens to provide methods to deal with object references.
A method is simply a subroutine that expects an object reference (or a package name, for class methods) as the first argument.

Object constructor

Perl doesn't provide any special syntax for constructors. A constructor is merely a subroutine that returns a reference to something ‘blessed’ into a class. Usually the same class the subroutine is defined on.

package Animal; sub new { bless({}) }

That word new isn't special.

package Animal; sub Animal { bless({}) }

Objects Inheritance

If you care about inheritance then you want to use the two-arg form of bless so that your constructors may be inherited.

package Animal; sub new {return bless({}, shift @_ );} package Dog; use base qw(Animal); # use 'Animal'; # +- true # push @ISA, 'Animal';

This would be called like:

my $dog = Dog->new();

get/put method

The get/put method in perl

sub property { my ($self, $value) = @_; $self->{property} = $value if @_>1; return $self->{property}; }

Use like: $obj->property(1); print $obj->property();

Method overwriting

The most common way to to call a method from an object is:

print("Dog: ".$dog->fly()." "); print("Bat: ".$bat->fly()." "); Perl will look to the scalar reference and see the package name of the blessed reference.

Method Implementation

package Animal; sub fly { return 0; }
package Bat; use base qw(Animal); sub fly { return 1; }

Method overwriting

If you need to, you can force Perl to start looking in some other package

$bat->Insect::fly(); # dangerous

As a special case of the above, you may use the SUPER pseudo-class to tell Perl to start looking for the method in the packages named in the current class's @ISA list:

$bat->SUPER::fly();

Object Destroy

The object is automatically destroyed when the last reference to an object goes away. If you want to capture control just before the object is freed, you may define a DESTROY method in your class. It will automatically be called and you can do any extra cleanup you need to do. Perl passes a reference to the object under destruction as the first argument.

Exercises 7

Create a module with the subroutines min , max and in .
Create a set of classes to represent the animal fly capabilities. Shall have two methods fly and name(get/put), the constructor receives the animal name. Consider the following rules: dog is a animal animal doesn’t fly bird is a animal bird flies penguin is a bird penguin doesn’t fly
Create a program to test them.

Standard Modules

pragma

The usual ones

strict - Restricts unsafe constructs
warnings - Control optional warnings
lib - Manipulate @INC at compile time
base - Establish an ISA relationship with base class
constant - Declare constants (consider Readonly)
subs - Predeclare sub names
integer - Forces integer math

Perl 5.10 gives new features, see perlpragma

Usual guilties

Data::Dumper - stringified perl data structures
Carp - better die() and warn()
Cwd - pathname of current working directory
Exporter - Implements default import method for modules
POSIX
IPC::Open3 - open a process for reading, writing, and error handling
Time::HiRes - High resolution alarm, sleep, gettimeofday, interval timers

World-Wide Web

LWP - The World-Wide Web library
LWP::UserAgent - Web user agent class
HTTP::Request - HTTP style request message
HTTP::Response - HTTP style response message
LWP::Simple - Simple procedural interface to LWP
LWP::Simple::Post - Single-method POST requests
HTTP::Async - process multiple HTTP requests in parallel without blocking.

World-Wide Web

WWW::Mechanize - Handy web browsing in a Perl object
Net::SLP - accessing the Service Location Protocol
Net::POP3 - Post Office Protocol 3 Client
Net::SMTP - Simple Mail Transfer Protocol Client
MIME::Lite - low-calorie MIME generator
JSON - JSON encoder/decoder
JSON::XS – fast JSON serialising/deserialising

Apache/mod_perl packages

Apache2::Const - Perl Interface for Apache Constants
Apache2::RequestIO - Perl API for Apache request record IO
Apache2::RequestRec - Perl API for Apache request record accessors
Apache2::RequestUtil - Perl API for Apache request record utils
CGI - Handle Common Gateway Interface requests and responses

Security

Data::UUID - Generating GUIDs/UUIDs
MIME::Base64 - Encoding and decoding of base64 strings
Digest::SHA - Perl extension for SHA-1/224/256/384/512
Digest::MD5 - Perl interface to the MD5 Algorithm
Crypt::DES - Perl DES encryption module
Net::SSH - Perl extension for secure shell

Test

Test - provides a simple framework for writing test scripts
Test::More - yet another framework for writing test scripts
Test::Exception - Test exception based code
Test::Output - Utilities to test STDOUT and STDERR messages.

Other

DBI - Database independent interface for Perl
DBD::* - Perl DBI Database Driver DBD::SQLite, DBD::CSV, DBD::Google, ...
Template – Template toolkit
HTML::Template - Perl module to use HTML Templates

Advanced Perl

DBI

use DBI ; my $dsn = "DBI:mysql:database=$database;" . "host=$hostname;port=$port"; my $dbh = DBI->connect ($dsn, $user, $password); my $sth = $dbh-> prepare ( "SELECT * FROM person WHERE name = ?") or die $dbh->errstr ; $sth-> execute ('oleber') or die $dbh->errstr ; while (my $ref = $sth-> fetchrow_hashref ()) { print Dumper $ref; } $sth-> finish (); $dbh-> disconnect ;

AUTOLOAD a method

When the method isn't found, the AUTOLOAD will be called.

sub AUTOLOAD { my ($self, @params) = @_; my $name = $AUTOLOAD ; $name =~ s/.*://; # strip name die "Can't access '$name' field" if not exists $self->{_p}->{$name}; ( $self->{$name} ) = @params if @params; return $self->{$name}; }

tie variable

package SConst; sub TIESCALAR { my ($pkg, $val) = @_; bless $val, $pkg; return $val; } sub FETCH { # read return ${shift()}; } sub STORE { # write die "No way"; } 1;