Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Perl Web Client

3,249 views

Published on

Come accedere al World Wide Web utilizzando Perl, dai primi passi ad un controllo avanzato sulle fasi di download e upload.

Published in: Self Improvement
  • Be the first to comment

Perl Web Client

  1. 1. I Client Web in Perl
  2. 2. http:// polettix.s3.amazonaws.com /tmp/pwc.tar.gz
  3. 3. LWP::Simple
  4. 4. LWP::Simple::getprint perl -MLWP::Simple -e 'getprint("http://www.perl.it")'
  5. 5. LWP::Simple::getprint perl -MLWP::Simple -e 'getprint("http://www.perl.it")'
  6. 6. LWP::Simple::getprint perl -MLWP::Simple -e 'getprint("http://www.perl.it")'
  7. 7. LWP::Simple::getstore perl -MLWP::Simple -e 'getstore("http://www.perl.it", "index.html")'
  8. 8. LWP::Simple::getstore perl -MLWP::Simple -e 'getstore("http://www.perl.it", "index.html")'
  9. 9. LWP::Simple::get use LWP::Simple 'get'; my $pagina = get('http://www.perl.it') // die "errore!"; say $pagina =~ 'polettix' ? 'OK!!!' : 'No!!!';
  10. 10. LWP::Simple::get use LWP::Simple 'get'; my $pagina = get('http://www.perl.it') // die "errore!"; say $pagina =~ 'polettix' ? 'OK!!!' : 'No!!!';
  11. 11. LWP::Simple::get use LWP::Simple 'get'; my $pagina = get('http://www.perl.it') // die "errore!"; say $pagina =~ 'polettix' ? 'OK!!!' : 'No!!!';
  12. 12. LWP::Simple::get use LWP::Simple 'get'; my $pagina = get('http://www.perl.it') // die "errore!"; say $pagina =~ 'polettix' ? 'OK!!!' : 'No!!!'; Perl 5.10
  13. 13. LWP::UserAgent
  14. 14. LWP::UserAgent use LWP::UserAgent; my $ua = LWP::UserAgent->new( user_agent => 'BrowsHer/1.0', timeout => 10, );
  15. 15. LWP::UserAgent::new use LWP::UserAgent; my $ua = LWP::UserAgent->new( user_agent => 'BrowsHer/1.0', timeout => 10, );
  16. 16. LWP::UserAgent::new use LWP::UserAgent; my $ua = LWP::UserAgent->new( user_agent => 'BrowsHer/1.0', timeout => 10, );
  17. 17. LWP::UserAgent::new use LWP::UserAgent; my $ua = LWP::UserAgent->new( user_agent => 'BrowsHer/1.0', timeout => 10, );
  18. 18. GET my $uri = 'http://www.perl.it'; my $risposta = $ua->get($uri);
  19. 19. GET my $uri = 'http://www.perl.it'; my $risposta = $ua->get($uri); HTTP::Response
  20. 20. HTTP::Response
  21. 21. HTTP::Response if ($response->is_success()) { say "OK, ho scaricato:n", $response->content(); }
  22. 22. HTTP::Response if ($response->is_success()) { say "OK, ho scaricato:n", $response->content(); } elsif ($response->is_error()) { say "errore: ", $response->status_line(); }
  23. 23. HTTP::Response
  24. 24. HTTP::Response is_success() is_redirect() is_info() is_error()
  25. 25. HTTP::Response is_success() is_error()
  26. 26. HTTP::Response is_success()
  27. 27. HTTP::Response if ($response->is_success()) { say "OK, ho scaricato:n", $response->content(); } elsif ($response->is_error()) { say "errore: ", $response->status_line(); }
  28. 28. HTTP::Response if ($response->is_success()) { say "OK, ho scaricato:n", $response->decoded_content(); } elsif ($response->is_error()) { say "errore: ", $response->status_line(); }
  29. 29. HTTP::Response if ($response->is_success()) { say "OK, ho scaricato:n", $response->decoded_content(); } elsif ($response->is_error()) { say "errore: ", $response->status_line(); }
  30. 30. HTTP Status Line
  31. 31. HTTP Status Line 200 OK
  32. 32. HTTP Status Line 200 OK 307 Temporary Redirect
  33. 33. HTTP Status Line 200 OK 307 Temporary Redirect 403 Forbidden
  34. 34. HTTP Status Line 200 OK 307 Temporary Redirect 403 Forbidden 404 Not Found
  35. 35. HTTP Status Line 200 OK 307 Temporary Redirect 403 Forbidden 404 Not Found 500 Internal Server Error
  36. 36. HTTP Status Line 200 OK 307 Temporary Redirect 403 Forbidden 404 Not Found 500 Internal Server Error ...
  37. 37. Form con GET
  38. 38. Form con GET my $base = 'http://www.google.com';
  39. 39. Form con GET my $base = 'http://www.google.com'; my $url = "$base?q=perl";
  40. 40. Form con GET my $base = 'http://www.google.com'; my $url = "$base?q=perl"; my $risposta = $ua->get($url);
  41. 41. Attenzione!
  42. 42. Attenzione! my $query = 'sam & max';
  43. 43. Attenzione! my $query = 'sam & max'; my $url = "$base?q=$query";
  44. 44. Attenzione! t a eo d r n b a m my $query = 'sam & max'; m te i t my $url = "$base?q=$query"; R e a S
  45. 45. Attenzione! my $query = 'sam & max'; my $url = "$base?q=$query";
  46. 46. Attenzione! Caratteri Speciali my $query = 'sam & max'; my $url = "$base?q=$query";
  47. 47. NN FUNZIONA!
  48. 48. Dov’è urlencode()?
  49. 49. Ma in PHP c’è!!!
  50. 50. A T O C I Ma in PHP c’è!!! C B O
  51. 51. use URI; use URI; my %parametri = ( nome => 'Flavio', cognome => 'Poletti', ); my $uri = URI->new( 'http://roma.pm.org/cerca.pl'); $uri->query_form(%parametri);
  52. 52. use URI; use URI; my %parametri = ( nome => 'Flavio', cognome => 'Poletti', ); my $uri = URI->new( 'http://roma.pm.org/cerca.pl'); $uri->query_form(%parametri);
  53. 53. use URI; use URI; my %parametri = ( nome => 'Flavio', cognome => 'Poletti', ); my $uri = URI->new( 'http://roma.pm.org/cerca.pl'); $uri->query_form(%parametri);
  54. 54. use URI; my $uri_string = $uri->as_string(); my $risposta = $ua->get($uri_string); # oppure... my $risposta = $ua->get("$uri");
  55. 55. use URI; my $uri_string = $uri->as_string(); my $risposta = $ua->get($uri_string); # oppure... my $risposta = $ua->get("$uri");
  56. 56. use URI; my $uri_string = $uri->as_string(); my $risposta = $ua->get($uri_string); # oppure... my $risposta = $ua->get("$uri");
  57. 57. use URI; my $uri_string = $uri->as_string(); my $risposta = $ua->get($uri_string); # oppure... my $risposta = $ua->get("$uri");
  58. 58. POST my %parametri = ( nome => 'Flavio', cognome => 'Poletti', ); my $risposta = $ua->post( 'http://www.perl.it/post.pl', %parametri, 'X-New' => 1);
  59. 59. POST my %parametri = ( nome => 'Flavio', cognome => 'Poletti', ); my $risposta = $ua->post( 'http://www.perl.it/post.pl', %parametri, 'X-New' => 1);
  60. 60. POST my %parametri = ( nome => 'Flavio', cognome => 'Poletti', ); my $risposta = $ua->post( 'http://www.perl.it/post.pl', %parametri, 'X-New' => 1);
  61. 61. POST my %parametri = ( nome => 'Flavio', cognome => 'Poletti', ); my $risposta = $ua->post( 'http://www.perl.it/post.pl', %parametri, 'X-New' => 1);
  62. 62. POST my %parametri = ( nome => 'Flavio', cognome => 'Poletti', ); my $risposta = $ua->post( 'http://www.perl.it/post.pl', %parametri, 'X-New' => 1);
  63. 63. HTTP::Request
  64. 64. F M HTTP::Request T R
  65. 65. F M HTTP::Request T R Friendly
  66. 66. Autenticazione?
  67. 67. credentials() my $server = 'roma.pm.org:80'; my $realm = 'Orticello'; my $utente = 'giardiniere'; my $pass = 'ciclamino'; $ua->credentials($server, $realm, $utente, $pass);
  68. 68. credentials() my $server = 'roma.pm.org:80'; my $realm = 'Orticello'; my $utente = 'giardiniere'; my $pass = 'ciclamino'; $ua->credentials($server, $realm, $utente, $pass);
  69. 69. credentials() my $server = 'roma.pm.org:80'; my $realm = 'Orticello'; my $utente = 'giardiniere'; my $pass = 'ciclamino'; $ua->credentials($server, $realm, $utente, $pass);
  70. 70. Proxy?
  71. 71. env_proxy # nella shell... bash$ export http_proxy='http://ex.com:8080' # nel codice... my $ua = LWP::UserAgent->new( # ... env_proxy => 1, );
  72. 72. env_proxy # nella shell... bash$ export http_proxy='http://ex.com:8080' # nel codice... my $ua = LWP::UserAgent->new( # ... env_proxy => 1, );
  73. 73. env_proxy() # nella shell... bash$ export http_proxy='http://ex.com:8080' # nel codice... $ua->env_proxy();
  74. 74. proxy() $ua->proxy(http => 'http://ex.com:80'); $ua->proxy( [qw( ftp gopher )] => 'http://dovetipare.it:80' );
  75. 75. proxy() $ua->proxy(http => 'http://ex.com:80'); $ua->proxy( [qw( ftp gopher )] => 'http://dovetipare.it:80' );
  76. 76. proxy() $ua->proxy(http => 'http://ex.com:80'); $ua->proxy( [qw( ftp gopher )] => 'http://dovetipare.it:80' );
  77. 77. SSL?
  78. 78. Crypt::SSLeay
  79. 79. Crypt::SSLeay na ! ortu a F Bu on
  80. 80. Cookies?
  81. 81. LWP::UserAgent::new
  82. 82. Re lo LWP::UserAgent::new ad ed my $ua = LWP::UserAgent->new( cookie_jar => {} );
  83. 83. cookie_jar() use HTTP::Cookies::Mozilla; my $jar = HTTP::Cookies::Mozilla->new( file => '/profilo/cookies.sqlite'); $ua->cookie_jar($jar);
  84. 84. cookie_jar() use HTTP::Cookies::Mozilla; my $jar = HTTP::Cookies::Mozilla->new( file => '/profilo/cookies.sqlite'); $ua->cookie_jar($jar);
  85. 85. cookie_jar() use HTTP::Cookies::Mozilla; my $jar = HTTP::Cookies::Mozilla->new( file => '/profilo/cookies.sqlite'); $ua->cookie_jar($jar);
  86. 86. WWW::Mechanize WWW::Mechanize www.z-design.it
  87. 87. new() use WWW::Mechanize; my $mech = WWW::Mechanize->new( user_agent => 'BrowsHer/1.0', timeout => 10, env_proxy => 1, autocheck => 1, stack_depth => 40, );
  88. 88. new() use WWW::Mechanize; my $mech = WWW::Mechanize->new( user_agent => 'BrowsHer/1.0', timeout => 10, env_proxy => 1, autocheck => 1, stack_depth => 40, );
  89. 89. autocheck eval { $mech->get($uri); # ... $mech->post($altra_uri); 1; } or say "errore: $@";
  90. 90. autocheck eval { $mech->get($uri); # ... $mech->post($altra_uri); 1; } or say "errore: $@";
  91. 91. autocheck eval { $mech->get($uri); # ... $mech->post($altra_uri); 1; } or say "errore: $@";
  92. 92. new() use WWW::Mechanize; my $mech = WWW::Mechanize->new( user_agent => 'BrowsHer/1.0', timeout => 10, env_proxy => 1, autocheck => 1, stack_depth => 40, );
  93. 93. $mech->back()
  94. 94. $mech->request() $mech->response()
  95. 95. $mech->success()
  96. 96. $mech->status()
  97. 97. $mech->content()
  98. 98. $mech->content()
  99. 99. $mech->links() for my $link ($mech->links()) { say $link->url_abs(); }
  100. 100. $mech->links() for my $link ($mech->links()) { say $link->url_abs(); }
  101. 101. find_link() my $perlit = $mech->find_link( text => 'Perl.it', );
  102. 102. find_link() my $perlit = $mech->find_link( text_regex => qr/perl.it/i, );
  103. 103. find_link() n text_regex url_regex url_abs_regex name_regex id_regex class_regex tag_regex
  104. 104. find_all_links() n text_regex url_regex url_abs_regex name_regex id_regex class_regex tag_regex
  105. 105. $mech->images() for my $immagine ($mech->images()) { $mech->get($immagine->url()); my $filename = $mech->response()->filename(); $mech->save_content($filename); }
  106. 106. $mech->images() for my $immagine ($mech->images()) { $mech->get($immagine->url()); my $filename = $mech->response()->filename(); $mech->save_content($filename); }
  107. 107. $mech->images() for my $immagine ($mech->images()) { $mech->get($immagine->url()); my $filename = $mech->response()->filename(); $mech->save_content($filename); }
  108. 108. $mech->images() for my $immagine ($mech->images()) { $mech->get($immagine->url()); my $filename = $mech->response()->filename(); $mech->save_content($filename); }
  109. 109. find_image() n alt_regex url_regex url_abs_regex tag_regex
  110. 110. find_all_images() n alt_regex url_regex url_abs_regex tag_regex
  111. 111. $mech->forms()
  112. 112. $mech ->form_number(3)
  113. 113. $mech ->form_name('Hi')
  114. 114. $mech ->form_id('C1P8')
  115. 115. $mech ->form_with_fields( qw( user pass ) )
  116. 116. Una volta selezionato... $mech->field( domain => 'polettix.it'); $mech->set_fields( username => 'flavio', password => 'poletti',); $mech->select( mongers => 'Roma.pm'); $mech->tick(mailing_list_subscriber => 1); $mech->untick(wants_spam => 1); $mech->submit();
  117. 117. Una volta selezionato... $mech->field( domain => 'polettix.it'); $mech->set_fields( username => 'flavio', password => 'poletti',); $mech->select( mongers => 'Roma.pm'); $mech->tick(mailing_list_subscriber => 1); $mech->untick(wants_spam => 1); $mech->submit();
  118. 118. Una volta selezionato... $mech->field( domain => 'polettix.it'); $mech->set_fields( username => 'flavio', password => 'poletti',); $mech->select( mongers => 'Roma.pm'); $mech->tick(mailing_list_subscriber => 1); $mech->untick(wants_spam => 1); $mech->submit();
  119. 119. Una volta selezionato... $mech->field( domain => 'polettix.it'); $mech->set_fields( username => 'flavio', password => 'poletti',); $mech->select( mongers => 'Roma.pm'); $mech->tick(mailing_list_subscriber => 1); $mech->untick(wants_spam => 1); $mech->submit();
  120. 120. Una volta selezionato... $mech->field( domain => 'polettix.it'); $mech->set_fields( username => 'flavio', password => 'poletti',); $mech->select( mongers => 'Roma.pm'); $mech->tick(mailing_list_subscriber => 1); $mech->untick(wants_spam => 1); $mech->submit();
  121. 121. Una volta selezionato... $mech->field( domain => 'polettix.it'); $mech->set_fields( username => 'flavio', password => 'poletti',); $mech->select( mongers => 'Roma.pm'); $mech->tick(mailing_list_subscriber => 1); $mech->untick(wants_spam => 1); $mech->submit();
  122. 122. Per pigri $mech->submit_form( with_fields => { domain => 'roma.pm.org', username => 'flavio', password => 'poletti', }, );
  123. 123. Controllo Maggiore
  124. 124. downlink
  125. 125. Applicazione Client Server
  126. 126. Applicazione Client Server
  127. 127. Applicazione Client Server
  128. 128. Applicazione Client Server
  129. 129. Applicazione Client Server
  130. 130. Applicazione Client Server
  131. 131. Applicazione Client Server
  132. 132. Applicazione Client Server
  133. 133. Applicazione Client Server
  134. 134. Applicazione Client Server
  135. 135. Applicazione Client Server
  136. 136. Applicazione Client Server
  137. 137. Applicazione Client Server
  138. 138. Applicazione Client Server
  139. 139. $ua->progress() package My::LWP::UserAgent; use base 'LWP::UserAgent'; sub progress { my ( $self, $status, $r ) = @_; print {*STDERR} "$statusn"; print {*STDOUT} $r->content() if $status eq 'end'; return 1 }
  140. 140. $ua->progress() package My::LWP::UserAgent; use base 'LWP::UserAgent'; sub progress { my ( $self, $status, $r ) = @_; print {*STDERR} "$statusn"; print {*STDOUT} $r->content() if $status eq 'end'; return 1 }
  141. 141. $ua->progress() package My::LWP::UserAgent; use base 'LWP::UserAgent'; sub progress { my ( $self, $status, $r ) = @_; print {*STDERR} "$statusn"; print {*STDOUT} $r->content() if $status eq 'end'; return 1 }
  142. 142. $ua->progress() package My::LWP::UserAgent; use base 'LWP::UserAgent'; sub progress { my ( $self, $status, $r ) = @_; print {*STDERR} "$statusn"; print {*STDOUT} $r->content() if $status eq 'end'; return 1 }
  143. 143. $ua->progress() package My::LWP::UserAgent; use base 'LWP::UserAgent'; sub progress { my ( $self, $status, $r ) = @_; print {*STDERR} "$statusn"; print {*STDOUT} $r->content() if $status eq 'end'; return 1 }
  144. 144. $ua->progress() package My::LWP::UserAgent; use base 'LWP::UserAgent'; sub progress { my ( $self, $status, $r ) = @_; print {*STDERR} "$statusn"; print {*STDOUT} $r->content() if $status eq 'end'; return 1 }
  145. 145. $ua->progress() package My::LWP::UserAgent; use base 'LWP::UserAgent'; sub progress { my ( $self, $status, $r ) = @_; print {*STDERR} "$statusn"; print {*STDOUT} $r->content() if $status eq 'end'; return 1 }
  146. 146. :content_cb
  147. 147. callback
  148. 148. callback sub produci_callback { my ($filename) = @_; open my $fh, ’>’, $filename or die "open(’$filename’): $!"; binmode $fh; return sub { my ($data, $response, $protocol) = @_; print {$fh} $data; }; }
  149. 149. :content_cb my $response = $ua->get( 'http://roma.pm.org/roma.pm.png', ':content_cb' => produci_callback('logo.png'));
  150. 150. callback sub produci_callback { # ... return sub { my ($data, $response, $protocol) = @_; # ... }; }
  151. 151. uplink
  152. 152. Applicazione Client Server
  153. 153. Applicazione Client Server
  154. 154. Applicazione Client Server
  155. 155. Upload di un file my $uri = 'http://localhost/upload.pl'; my %parametri = ( blah => ’questo-server’, data => scalar(localtime()), file => [ ’/etc/passwd’ ], );
  156. 156. Upload controllato my $r; # per la risposta { local $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD; $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1; my $request = HTTP::Request::Common::POST($uri, %parametri, ’Content-Type’ => ’form-data’); # ...
  157. 157. Upload controllato my $r; # per la risposta { local $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD; $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1; my $request = HTTP::Request::Common::POST($uri, %parametri, ’Content-Type’ => ’form-data’); # ...
  158. 158. Upload controllato my $r; # per la risposta { local $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD; $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1; my $request = HTTP::Request::Common::POST($uri, %parametri, ’Content-Type’ => ’form-data’); # ...
  159. 159. Upload controllato my $content_closure = $request->content(); $request->content(sub { my $porzione = $content_closure->(); return unless defined $porzione; # ... return $porzione; });
  160. 160. Upload controllato $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1 my $content_closure = $request->content(); $request->content(sub { my $porzione = $content_closure->(); return unless defined $porzione; # ... return $porzione; });
  161. 161. Upload controllato my $content_closure = $request->content(); $request->content(sub { my $porzione = $content_closure->(); return unless defined $porzione; # ... return $porzione; });
  162. 162. Upload controllato $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1 my $content_closure = $request->content(); $request->content(sub { my $porzione = $content_closure->(); return unless defined $porzione; # ... return $porzione; });
  163. 163. Upload controllato my $content_closure = $request->content(); $request->content(sub { my $porzione = $content_closure->(); return unless defined $porzione; # ... return $porzione; });
  164. 164. Upload controllato my $content_closure = $request->content(); $request->content(sub { my $porzione = $content_closure->(); return unless defined $porzione; # ... return $porzione; });
  165. 165. Upload controllato my $content_closure = $request->content(); $request->content(sub { my $porzione = $content_closure->(); return unless defined $porzione; # ... return $porzione; });
  166. 166. Upload controllato $r = $ua->request($request); }
  167. 167. Upload controllato $r = $ua->request($request); }
  168. 168. Upload controllato $r = $ua->request($request); ! }
  169. 169. Upload controllato my $r; # per la risposta { local $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD; $HTTP::Request::Common::DYNAMIC_FILE_UPLOAD = 1; #... $r = $ua->request($request); }
  170. 170. WWW::Mechanize::Shell
  171. 171. W::M::Shell shell$ perl -MWWW::Mechanize::Shell -e shell (no url)>
  172. 172. W::M::Shell (no url)> get http://www.zooomr.com/login/ Retrieving http://www.zooomr.com/login/(200) http://www.zooomr.com/login/>
  173. 173. W::M::Shell http://www.zooomr.com/login/>forms Form [1] GET http://www.zooomr.com/search/people/ q=Find Friends! (text) Form [2] POST http://www.zooomr.com/recovery/ [f1] email= (text) <NONAME>=Retrieve Password (submit) gogogo=1 (hidden readonly) Form [3] POST http://www.zooomr.com/login/ [f2] gogogo=1 (hidden readonly) redirect_to=http://www.zooomr.com/ (hidden readonly) username= (text) password= (password) <NONAME>=Let’s Go Exploring! (submit) <NONAME>=<UNDEF> (button) processlogin=1 (hidden readonly)
  174. 174. W::M::Shell http://www.zooomr.com/login/>form 3 POST http://www.zooomr.com/login/ [f2] gogogo=1 (hidden readonly) redirect_to=http://www.zooomr.com/ (hidden readonly) username= (text) password= (password) <NONAME>=Let’s Go Exploring! (submit) <NONAME>=<UNDEF> (button) processlogin=1 (hidden readonly)
  175. 175. W::M::Shell http://www.zooomr.com/login/>value username polettix http://www.zooomr.com/login/>value password pippo http://www.zooomr.com/login/>form 3 POST http://www.zooomr.com/login/ [f2] gogogo=1 (hidden readonly) redirect_to=http://www.zooomr.com/ (hidden readonly) username=polettix (text) password=pippo (password) <NONAME>=Let’s Go Exploring! (submit) <NONAME>=<UNDEF> (button) processlogin=1 (hidden readonly)
  176. 176. W::M::Shell ...>click
  177. 177. W::M::Shell ...>script session.pl
  178. 178. HTTP::Recorder

×