Perl Presentation

Slide 1: Perl Sopan Shewale, sopan.shewale@gmail.com

Slide 2: Hello World – First Program $ cat HelloWorld.pl #!/usr/bin/perl print "Welcome to the world of Perl Programming !n"; Make it executable $ chmod +x HelloWorld.pl Execute it $ ./HelloWorld.pl Welcome to the world of Perl Programming ! Another Way to Execute $ perl HelloWorld.pl Welcome to the world of Perl Programming !

Slide 3: Another Simple Example – Taking input from user [sopan@ps3724 tutorial]$ cat InteractiveHello.pl #!/usr/bin/perl my $user; print "Please write your name:"; $user = <STDIN>; print "Welcome to the world of perl, You are : $usern";

Slide 4: Simple example – way to execute external command [sopan@ps3724 tutorial]$ cat df.pl #!/usr/bin/perl print "Method1:n"; my $out1 = system("df -k"); print "The output of df -k command is :n $out1n"; Method1: Filesystem 1K-blocks Used Available Use% Mounted on /dev/hda8 19346964 10931664 7432528 60% / /dev/hda1 101089 14826 81044 16% /boot /dev/hda3 15116868 12328832 2020132 86% /home none 252056 0 252056 0% /dev/shm /dev/hda5 8064272 5384612 2270008 71% /usr /dev/hda6 3020140 307088 2559636 11% /var /dev/hda2 30233928 8830792 19867324 31% /usr/local The output of df -k command is : 0

Slide 5: Simple example – way to execute external command print "Method2:n"; my $out2 = `df -k`; print "The output of df -k command is :n $out2n"; Method2: The output of df -k command is : Filesystem 1K-blocks Used Available Use% Mounted on /dev/hda8 19346964 10931664 7432528 60% / /dev/hda1 101089 14826 81044 16% /boot /dev/hda3 15116868 12328832 2020132 86% /home none 252056 0 252056 0% /dev/shm /dev/hda5 8064272 5384612 2270008 71% /usr /dev/hda6 3020140 307088 2559636 11% /var /dev/hda2 30233928 8830792 19867324 31% /usr/local [sopan@ps3724 tutorial]$

Slide 6: An introduction to Simple Stuff Perl Variables – How perl treats integers, floating numbers or  strings? Give simple examples. Play with variables – assign, modify, print the values.  Explain about “my” syntax.  Arithmetic Operations (+, -, * etc) on int/float and strings.  Introduction to loops (if-else, while, for)  Comparison operators – (>, <, ==, eq, nq etc)  Introduction to special variable $_  How to use command line argument in the script –though  arrays will be introduced later.

Slide 7: Exercise Find the maximum from the given set of integers. 1. Take two numbers from user – print the addition of these numbers 2. Greatest Common Divisor – Implement following algorithm 3. gcd (a, b) { if (b < a ) { swap (a, b); } While (b) { r = b % a; a = b; b = r; } return a; }

Slide 8: Directory and File Operations File Operations – Read the file, Write the file, Append the file.  Introduction to $!  Introduction to die  Introduction to chomp  Unlink  Exercise– Write the program to monitor given set of hosts in the network.

Slide 9: Arrays and Functions Arrays, split, join functions, pop, push, shift, unshift functions.  Exercises Example 1. Counting number of lines from a given file.  Example 2. Counting number of words from a file (… what is word?)  Example 3. Various operations on /etc/password – e.g. Which userid do  not have user-gecos?

Slide 10: Exercise – long assignement 1975:Won:West Indies Example-Data 1975:Lost:Australia We would like to provide the following services: 1979:Won:West Indies 1979:Lost:England Q1. Given Year, won/lost category, print team name. 1983:Won:India Q2. Given Year, Print all entries for that year. 1983:Lost:West Indies Q3. For Won/Lost – print year and teams. 1987:Won:Australia Q4. Print all entries sorted by Won/Lost  or By Year. 1987:Lost:England 1992:Won:Pakistan 1992:Lost:England 1996:Won:Srilanka 1996:Lost:Australia

Slide 11: grep and map functions List Filtering with grep my @result = grep EXPR @input_list; my $count = grep EXPR @input_list; my @input_numbers = (1, 4, 50, 6 14); my @result = grep $_>10, @input_numbers; Transforming Lists with Map my @input_list = (1, 4, 10, 56, 100); my @result = map $_ + 100, @input_list; my @next_result = map ($_, 5*$_), @input_list; my %hash_result =map ($_, $_*$_), @input_list;

Slide 12: Exercise Q. 1 Write a program that takes a list of filenames on the command line and uses grep to select the ones whose size in bytes is less than 1000. Use map to transform the strings in this list, putting four space characters in front of each and a newline character after. Print the resulting list.

Slide 13: Regular Expression Q. What is a regular expression? Ans. A regular expression is simply a string that describes a pattern. Q. So what is pattern? Ans. Let us understand by example: ls *.txt, dir *.* are examples of patterns. author:sopan_shewale can be example of search pattern in for Search on content management site or for search engine. In Perl, the patterns described by regular expressions are used to search strings, extract desired parts of strings, and to do search and replace operations. Regular Expression is often abbrevated as regexp, regex.

Slide 14: Regular expression (Cont…) Let us look at the example if ("Hello World" =~ /World/) { print "It matchesn"; } else { print "It doesn't matchn"; } The sense of =~ is reversed by !~ operator.

Slide 15: Regular Expression (Cont…) The literal string in the regexp can be replaced by a variable: $greeting = "World"; if ("Hello World" =~ /$greeting/) { print "It matchesn"; } else { print "It doesn't matchn"; } Regexp’s are case sensitive

Slide 16: Regular Expression (Cont…) If a regexp matches in more than one place in the string, perl will always match at the earliest possible point in the string: "Hello World" =~ /o/; # matches 'o' in 'Hello' "That hat is red" =~ /hat/; # matches 'hat' in 'That' Some characters, called metacharacters, are reserved for use in regexp notation. List is : {}[]()^$.|*+? "2+2=4" =~ /2+2/; # doesn't match, + is a metacharacter "2+2=4" =~ /2+2/; # matches, + is treated like an ordinary +

Slide 17: Regular Expression (Cont…) %vi simple_grep Grep Kind of example #!/usr/bin/perl $regexp = shift; while (<>) { print if /$regexp/; } Where in the string the regexp should try to match – Use ^ and $ .  The anchor ^ : match at the beginning of the string  The anchor $ : match at the end of the string,  "housekeeper" =~ /keeper/; # matches "housekeeper" =~ /^keeper/; # doesn't match "housekeeper" =~ /keeper$/; # matches "housekeepern" =~ /keeper$/; # matches "keeper" =~ /^keep$/; # doesn't match "keeper" =~ /^keeper$/; # matches "" =~ /^$/; # ^$ matches an empty string

Slide 18: Regular Expression (Cont…) Using character classes A character class allows a set of possible characters, rather than just a single character, to match at a particular point in a regexp. /cat/; # matches 'cat' /[bcr]at/; # matches 'bat, 'cat', or 'rat' /item[0123456789]/; # matches 'item0' or ... or 'item9‘ "abc" =~ /[cab]/; # matches 'a' In the last statement, even though 'c' is the first character in the class, 'a‘ matches because the first character position in the string is the earliest point at which the regexp can match. $x = 'bcr'; /[$x]at/; # matches 'at', 'bat, 'cat', or 'rat' /item[0-9]/; # matches 'item0' or ... or 'item9' /[0-9bx-z]aa/; # matches '0aa', ..., '9aa',

Slide 19: Regular Expression (Cont…) d is a digit and represents [0-9]  s is a whitespace character and represents [ trnf]  w is a word character (alphanumeric or _) and represents [0-  9a-zA-Z_] D is a negated d; it represents any character but a digit [^0-  9] S is a negated s; it represents any non-whitespace character  [^s] W is a negated w; it represents any non-word character  [^w] The period '.' matches any character but "n" 

Slide 20: Regular Expression (Cont…) Matching this or that This is accomplished by using the alternation metacharacter | "cats and dogs" =~ /cat|dog|bird/; # matches "cat" "cats and dogs" =~ /dog|cat|bird/; # matches "cat"

Slide 21: Regular Expression (Cont…) Grouping things and hierarchical matching sometime we want alternatives for just part of a regexp The grouping metacharacters () solve this problem /(a|b)b/; # matches 'ab' or 'bb‘ /(ac|b)b/; # matches 'acb' or 'bb' /(^a|b)c/; # matches 'ac' at start of string or 'bc' anywhere /(a|[bc])d/; # matches 'ad', 'bd', or 'cd' /house(cat|)/; # matches either 'housecat' or 'house‘ /house(cat(s|)|)/; # matches either 'housecats' or 'housecat' or # 'house'. Note groups can be nested. /(19|20|)dd/; # match years 19xx, 20xx, or the Y2K problem, xx "20" =~ /(19|20|)dd/; # matches the null alternative '()dd', # because '20dd' can't match

Slide 22: Regular Expression (Cont…) The process of trying one alternative, seeing if it matches, and moving on to the next alternative if it doesn't, is called backtracking. To be concrete, here is a step-by-step analysis of what perl does when it tries to match the regexp "abcde" =~ /(abd|abc)(df|d|de)/;

Slide 23: Regular Expression (Cont…) [1]. Start with the first letter in the string 'a'. [2]. Try the first alternative in the first group 'abd'. [3]. Match 'a' followed by 'b'. So far so good. [4]. 'd' in the regexp doesn't match 'c' in the string – a dead end. So backtrack two characters and pick the second alternative in the first group 'abc'. [5]. Match 'a' followed by 'b' followed by 'c'. We are on a roll and have satisfied the first group. Set $1 to 'abc'. [6]. Move on to the second group and pick the first alternative 'df'. [7]. Match the 'd'. [8]. 'f' in the regexp doesn't match 'e' in the string, so a dead end. Backtrack one character and pick the second alternative in the second group 'd'. [9]. 'd' matches. The second grouping is satisfied, so set $2 to 'd'. [10]. We are at the end of the regexp, so we are done! We have matched 'abcd' out of the string "abcde".

Slide 24: Regular Expression (Cont…) Extracting Matches The grouping metacharacters () allow the extraction of the parts of the string that matched. The extracted stuff is put into the special variables $1, $2, etc # extract hours, minutes, seconds if ($time =~ /(dd):(dd):(dd)/) { # match hh:mm:ss format $hours = $1; $minutes = $2; $seconds = $3; }

Slide 25: Regular Expression (Cont…) Matching Repetitions metacharacters ?, * , + , and {} play a vital role in matchings.  a? = match 'a' 1 or 0 times  a* = match 'a' 0 or more times, i.e., any number of times  a+ = match 'a' 1 or more times, i.e., at least once  a{n,m} = match at least n times, but not more than m times.  a{n,} = match at least n or more times  a{n} = match exactly n times

Slide 26: Regular Expression (Cont…) A few Principles Principle 0: Taken as a whole, any regexp will be matched at the earliest possible position in the string. Principle 1: In an alternation a|b|c... , the leftmost alternative that allows a match for the whole regexp will be the one used. Principle 2: The maximal matching quantifiers ?, * , + and {n,m} will in general match as much of the string as possible while still allowing the whole regexp to match.

Slide 27: Regular Expression (Cont…) A few Principles Principle 3: If there are two or more elements in a regexp, the leftmost greedy quantifier, if any, will match as much of the string as possible while still allowing the whole regexp to match. The next leftmost greedy quantifier, if any, will try to match as much of the string remaining available to it as possible, while still allowing the whole regexp to match. And so on, until all the regexp elements are satisfied.

Slide 28: Regular Expression (Cont…) Principle 0 overrides the others - the regexp will be matched as early as possible, with the other principles determining how the regexp matches at that earliest character position. $x = "The programming republic of Perl"; $x =~ /^(.+)(e|r)(.*)$/; # matches, # $1 = 'The programming republic of Pe' # $2 = 'r' # $3 = 'l'

Slide 29: Exercise Problem 1: Let us match both integers and floating point numbers from  text, ignore other parts. Problem 2. Match Email Addresses or collect all email addresses from  given text topic. Problem 3. $string = “/some/path/in/directory/html”; print only last  part – last level directory name.

Slide 30: Regular Expression – Match the positive Integer Question: Determine whether a string contains a positive integer. Discussion: You might start with expression like /[0-9]+/ which is for "one or more digits".  Then you might do simplification – to /d+/ - but still it is wrong, why? Expression like  "abc123de“ fails - Opps So you are moving to add anchors like /^d+$/.  This this is not correct – the new line char troubles us- the expression can match "123n"  as well as "123“ - oops! Now the modern Perl versions provide the z anchor, - The best answer to our problem  could be /^d+z/. But still issues - Although deprecated, the $* variable controls the matching of ^ and $  to permit internal newline matches as well as end-of-string matches. The string "foobarn123" will also match our new regular expression - Oops again.  Let us go to the final answer – The best answer is /Ad+z/, which says, "beginning of  string" followed by "one or more digits" followed by "end of string". The answer is: /Ad+z/ - finally  

Slide 31: Hashes Defination: Hashes are like arrays but scalar indexes. Example Defination: my %fruit; $fruit{"red"} = "apple"; $fruit{"orange} = "orange"; $fruit{"purple"} = "grape"; %fruit = ("red", "apple", "orange", "orange", "purple", "grape"); %fruit = (         "red" => "apple",         "orange" => "orange",         "purple" => "grape", );

Slide 32: Functions on Hashes Functions on Hashes: #vi fruit.pl  delete my %fruit = (  Keys         "red" => "apple",  values each         "orange" => "orange",         "purple" => "grape", ); for (keys %fruit) { print “$_ : $fruit{$_} n”; } Why use Hash? Write a program which takes the input the city/capital name and returns the country name by Using Arrays and Hashes or any other method. So you know why use Hashes.

Slide 33: References # Create some variables $a = "mama mia"; @array = (10, 20); %hash = ("laurel" => "hardy", "nick" => "nora"); # Now create references to them $ra = $a; # $ra now "refers" to (points to) $a $rarray = @array; $rhash = %hash; #You can create references to constant scalars in a similar fashion: $ra = 10; $rs = "hello world";

Slide 34: References #Look nicely: @array = (10, 20); $ra = @array; is similar to $ra =[10, 20] # Notice the square braces. %hash = ("laurel" => "hardy", "nick" => "nora"); $rhash = %hash is similar to $rhash = { “laurel”=>”hardy”, “nick” => “nora” }; Dereferencing: Means getting at the value that a reference point to. $ra = $a; # First take a reference to $a $$ra += 2; # instead of $a += 2; print $$ra; # instead of print $a $rarray = @array; push (@array , "a", 1, 2); # Using the array as a whole push (@$rarray, "a", 1, 2); # Indirectly using the ref. to the array $rhash = %hash; print $hash{"key1"}; # Ordinary hash lookup print $$rhash{"key1"}; # hash replaced by $rhash

Slide 35: References Shortcuts with the Arrow Notation $rarray = @array; print $rarray->[1] ; # The "visually clean" way $rhash = %hash; print $rhash->{"k1"}; #instead of ........ print $$rhash{"k1"}; # or print ${$rhash}{"k1"};

Slide 36: Subroutines Q. Why Subroutines exists in Programming? Ans. Do I need to Answer? Defining subroutine: #Defining subroutine: sub mysubroutinename { #### code to do that } •How to execute subroutine? •How to pass the arguments to subroutines? •How to use those arguments passed to the subroutines? •Return Values?

Slide 37: Modules Perl Allows you to partition your code into one more reusable module. Modules is code reuse We will see:  Define Modules using the package keyword  Load pre-defined modules using “use” and “require” keywords.  Access package specific variables and subroutines using “::” notation.  Load Functions at Run Time.

Slide 39: Modules (Cont…) Remember that modules are not intended to be standalone applications we'll never actually be running our module by itself. Instead, we'll be using our module to provide added functionality to a larger script Let us look at the example: Filename is SayHello.pm Package SayHello; sub hello_package { return “Hello, Welcome to the world of Modules”; } 1;

Slide 40: use and require – what is that? Use of “use” keyword  This is simplest way to pull the module into your script  Searches all the paths in the array @INC until it finds a file  Use “require” command:  What’s the difference then? When you load a module using use, the module is loaded and executed at compile time, even before the remaining portion of the script is compiled. require pulls in the code of the module at run-time; and if the file cannot be found generates a run time error Note on variables declared with “my” you will not be able to access it from outside the module If you must access a module variable from outside the package you are declaring it in, then you can declare it using the our function

Slide 42: For the decent example – may be if you are thinking of contributing to cpan.org- Ha a look at http://en.wikipedia.org/wiki/Perl_module bless : This operator takes the reference and converts it into object. Some Useful work module – Employee.pm

Slide 44: CGI Check my other PPT  Explore the application available at  http://perl-md5-login.sourceforge.net/  MD5 Algorithm for Encryption  The module is used by Yahoo etc 

Slide 45: Example: Count Click Script #!/usr/bin/perl my $cookie = new CGI::Cookie(-name=>'CGISESSID', ######################### -value=>$sessionid, -path=>"/"); ## Author: Sopan Shewale print $q->header('text/html', -cookie=>$cookie); ## Company: Persistent Systems Pvt Ltd ## This script is created for giving demo on click count. print $q->start_html("Welcome to Click Count Demo"); ## The Script is support to display increse/decrease print "<h1>Welcome to Click Count Demo</h1>"; ##click's, handles back button of browser, does not ##handle reload stuff. my $count = $session->param('count'); ## also it's based on sessions. ## count-is click count variable ######################## if(!defined($count)) { $session->param('count', 0); $count=0;} ### if session is first time created, set count=0 use strict; use warnings; $session->param('count', $count); $count = $session->param('count'); use CGI; #print "<h1>The Click Count is: $count n"; use CGI::Session; use CGI::Cookie; ## Form stuff print $q->startform(-method=>'POST'); my $q = new CGI(); print $q->submit( -name=>"Increase", -value=>'Increase1'); my $sessionid = $q->cookie("CGISESSID") || undef; print $q->submit( -name=>"Decrease", -value=>'Decrease1'); my $session = new CGI::Session(undef, print $q->endform(); $sessionid, {Directory=>'/tmp'}); $sessionid = $session->id();

Slide 46: Example: Count Click Script (Cont…) ## increases the count by 1 ## Which button is being pressed sub increase_count { my $which_button = $q->param('Increase'); my $number = shift; if(defined ($which_button)) { $number = $number +1; print "Increase pressed"; return $number; $count = increase_count($count); } ## Increase the count since increase button is clicked $session->param('count', $count); ## decreases the count by 1 }else { sub decrease_count { $which_button=$q->param('Decrease'); my $number = shift; if(defined($which_button)){ $number = $number -1; print "Decrease pressed"; return $number; $count = decrease_count($count); ## Decrease the count since decrease button is clicked } $session->param('count', $count); } else {print "You have not pressed any button, seems you are typing/re-typing the same URL"; } } $count = $session->param('count'); print "<h1>The Click Count is: $count n"; print $q->end_html();

Slide 47: Using eval String Form: Expression Evaluation   The Block Form: Exception Handling  Possible to use eval for time-outs. A few more…

Slide 48: Data::Dumper

Slide 49: TWiki

Slide 50: tools: bin: view, edit, search etc mailnotify lib: modules and required libraries, plugin code template: Templates (view.tmpl), also has skin related data pub: Attachments of the topics and commonly shared data data: webs directories (e.g. Main,TWiki) and each directory inside contains the topics from that web. The topics are companied by there version history

Slide 51: TWiki (Cont…) Basic request in TWiki from browser side, calls some script from bin Directory (like views) which Loads setlib.cfg from the same directory. Which loads LocalSite.cfg from lib directory. The LocalSite.cfg has site related Configuration variables defined. Then the relevant modules are loaded, topic are read, macros expanded, the content converted into html page and sent back to browser, request is completed. Now a days the LocalSite.cfg is written using Web-Interface, For example http://mytwikiste.com/bin/configure

Slide 52: TWiki (Cont…) What is plugins?: One can add plugins to TWiki to extend the functionality of TWiki without altering the core code. Plugin approach lets us do following: add virtually unlimited features while keeping the main TWiki  code compact and efficient; heavily customize an installation and still do clean updates to new  versions of TWiki; rapidly develop new TWiki functions in Perl using the Plugin API. 

Slide 53: TWiki (Cont…) TWiki Community: The plugins are located at  http://twiki.org/cgi-bin/view/Plugins  Each plugin has couple of topics on the site for support, development, installation instruction. Let us look at example: http://twiki.org/cgi-bin/view/Plugins/CommentPlugin : Has comment Plugin, with installation  instructions, syntax etc. http://twiki.org/cgi-bin/view/Plugins/CommentPluginDev: Used for support activity and  comments from public. http://twiki.org/cgi-bin/view/Plugins/CommentPluginAppraisal: Appraisal purpose, any one  can appraise the plugin. The plugin developer must read: TWiki::Func (http://twiki.org/cgi-bin/view/TWiki/TWikiFuncModule) code. That’s the Plugin API.

Slide 54: TWiki (Cont…) General Instructions for creating Plugin The plugin name is in the format <name>Plugin.  The code goes in <TWikiRootDirectory>/lib/TWiki/Plugings/ directory.  TWiki provides plugin called EmptyPlugin which can be used as Template to develop  new plugin. The code of Empty Plugin is <TWikiRootDirectory>/lib/TWiki/Plugins/EmptyPlugin.pm  Create a <name>Plugin topic in the TWiki web. The topic contains the settings,  installation instructions and syntax of the plugin. Enable Plugin by using configure e.g. http://yourtwikiinstalled.com/bin/configure 

Slide 55: HelloPlugin %HELLO{format=“Dear $first $last, $brWelcome to the world of TWiki” first=“Hari” last=“Sadu”}% This should get expanded to: Dear Hari Sadu, Welcome to the world of TWiki

Slide 58: Database Connectivity DBI Module  General process used in database activity  Connect   Prepare sql query  Execute  Finish  Disconnect Best document available at  http://search.cpan.org/src/TIMB/DBI_AdvancedTalk_2004/index.htm

Slide 59: Exercise Q. 1: Given array, delete the duplicate entries – the order should be reserved e.g. For input Array - (“Proxy”, “Proxy”, “smtp”, “smtp”, “smtp”, “mail”, 25) - output should be (“proxy”, “smtp”, “mail” 25);

Slide 60: References Online   http://learn.perl.org/library/beginning_perl/ - Simon Cozens  http://ebb.org/PickingUpPerl/  http://gnosis.cx/publish/programming/regular_expressions.html  http://perldoc.perl.org/perlretut.html Books   Advanced Perl Programming – By Sriram Srininivasan  Learning Perl Objects, References, & Modules – Randal L. Schwartz  Intermediate Perl – Randal L. Schwartz  Perl for Systems Administration – David N. Blank  Mastering Algorithms with Perl – Jon Orwant et all People to Track   Randal L. Schwartz  Simon Cozens  Convey - ???

Slide 61: References (Cont…) Tools  Web Application Development - Frameworks  CGI::Application (http://cgiapp.erlbaum.net/)  Catalyst (http://www.catalystframework.org/ )  PageKit Web Application Framework (http://www.pagekit.org/)  Combust web site Framework (http://combust.develooper.com/)  Mason (http://www.masonhq.com/)  Embperl (http://perl.apache.org/embperl/ )  Template Tools – used in Web Application Development  Template Toolkit http://www.tt2.org/  HTML::Template http://html-template.sourceforge.net/  Petal - http://search.cpan.org/~bpostle/Petal-2.19/lib/Petal.pm  Other  TWiki (http://twiki.org) – Enterprise Collaboration Platform  Sympa - Mailing List  Mhon-Arch – Mail Archiving Soltion  Webmin – Unix System Administration Application 

Slide 62: Thank You