?
Who's afraid
    of the

Big Bad Wolf
       Hakim.Cassimally@staff.dada.net
     2007­08­30 DADA Spa., Firenze, Italy
 ...
Debugging web applications
        Debugging is hard
    ●



        Complex, legacy applications...
    ●



        Sup...
This metaphor is brought to
                            you by
        firenze.pm
    ●



        perl.it
    ●



      ...
What happens when a little
    piggy goes in the forest?




            The Forest



                  
                ...
What happens when a little
    piggy goes in the forest?




            CGI
            The Forest



                  
...
What happens when a little
    piggy goes in the forest?



              mod_perl
            The Forest
           CGI

...
What happens when a little
    piggy goes in the forest?


                 mod_perl

            Perl
            The For...
What happens when a little
    piggy goes in the forest?


                  mod_perl
           Perl
                 DBI...
What happens when a little
    piggy goes in the forest?


                  mod_perl
                  modules
          ...
What happens when a little
    piggy goes in the forest?


             our  mod_perl
           Perl
                  mo...
The Forest
        And more!
    ●


            Network
        –

            HTML
        –

            XML
        –
...
Werewolves!




          
                                      
             13   
                     30 agosto 2007
 ...
Werewolves!




                     There!
                     Wolves!
          
                                      ...
What happens when a little
    piggy goes in the forest?




                 
                                           ...
What happens when a little
      piggy goes in the forest?




              The Forest
       pig




16                 ...
What happens when a little
    piggy goes in the forest?


YARGGH!

            The Forest



                  
         ...
What happens when a little
    piggy goes in the forest?


    404
            The Forest



                  
          ...
What happens when a little
    piggy goes in the forest?




            The Forest                   another pig




    ...
What happens when a little
    piggy goes in the forest?




            The Forest



                  
                ...
What happens when a little
        piggy goes in the forest?
        It never came out!
    ●



        It came out limpi...
What happens when a little
        piggy goes in the forest?
        It never came out!
    ●



        It came out limpi...
Werewolves!
        Round up the villagers for a lynch­mob!
    ●



        (Bring a pitchfork!)
    ●



        Burn th...
Engineers
        Luckily things are (a little) better if we know 
    ●


        what we're doing




                  ...
We shouldn't see this




         Scary Forest
           (eeek!)


                
                                    ...
But this




        
                                    
           26   
                   30 agosto 2007
               
Logging
          Trace the piggy as he goes through the forest
      ●




27                               
            ...
Logging
PIG1: I'm scared




                   The Forest
     pig




                         
                        ...
Logging
PIG1: I'm scared
PIG1: I'm very scared




                 The Forest



                         
              ...
Logging
PIG1: I'm scared
PIG1: I'm very scared
PIG1: This is Cruelty to Animals!




                 The Forest



      ...
Logging
PIG1: I'm scared
PIG1: I'm very scared
PIG1: This is Cruelty to Animals!
PIG1: Speaking to the Database




      ...
Logging
PIG1: I'm scared
PIG1: I'm very scared
PIG1: This is Cruelty to Animals!
PIG1: Speaking to the Database
PIG1: Look...
Logging (print statements)
        Print statements
    ●


            print “I AM HERE!”;
        –




                ...
Logging (warnings)
        warn
    ●


            warn “THIS HAPPENED!”;
        –

            (this will end up in the...
Logging (apache error log)
        $ tail ­f error.log.2007­04­11
    ●


                 | grep “Media”   # your module
...
Logging (apache error log)
        $ tail ­f error.log.2007­04­11
    ●


               | grep ­v “uninitialized”
       ...
Logging (print statements)
        Print to HTML
    ●


            content­type: text/html   (or text/plain)
        –

...
Logging (data structures)
        Complex data structures
    ●


            Data::Dumper
        –

            $VAR1 = ...
Logging (manual trace)
        When did the script crash?
    ●


            $this­>printDebug(“HC 1 got here”);
        ...
Logging (manual trace)
        Yes, that's quite stupid
    ●


            (but it beats using a pitchfork)
        –



...
Logging
            (end of html debug)
        To avoid messing up headers you can get your 
    ●


        framework to...
Logging (to file)
        save to a file
    ●


            open file
        –

            write
        –

           ...
Logging
        use Log::Log4Perl qw(:easy);
    ●



        Log::Log4Perl­>easy_init($ERROR);
    ●



        my $log =...
Logging
        Log files eventually get really big
    ●



        Sysadmins will want to rotate them
    ●



        T...
Error handling
                         If a pig screams in
                           the forest and
                    ...
Error handling
        Sometimes it's OK to die
    ●


            use CGI::Carp qw( fatalsToBrowser );
        –

      ...
Error handling (CSI!)
        But how did the piggy die?
    ●



            Bad parameter at TestModule.pm line 10.
    ...
Error handling (Carp)
        package TestModule;
    ●


        use Carp qw( croak );
    ●



        ...  
    ●


   ...
Error handling (stacktraces)
        What if it was another subroutine or module 
    ●


        called?
        use Carp...
Error handling (framework)
        If your framework handles this, you may get 
    ●


        stacktraces by default
   ...
Error handling (Database)
        We don't want the user seeing DB errors in live!
    ●


            BUT
        –

    ...
Error handling (Database)
        Do you really want to fail silently...?
    ●


            $sth­>execute()
        –
  ...
Testing
                         I hope
                        someone
                       tested this
               ...
Burn the Witch!
        Sometimes we think we know where the 
    ●


        problem is
            Sometimes we're wrong...
How do we check
        A test!
    ●


            does it weigh more or less than a duck?
        –

        use Test::M...
Testing in the forest
        Sometimes we need to test:
    ●


            user clicks on link in browser, logs in
     ...
Testing in the forest
        But...
    ●


            what if we suspect that the problem is with 
        –
          ...
See the trees...
        Write a simple test
    ●


            use Test::More;
        –

            use MyModule;
    ...
See the trees...

        MVC!
    ●




                       
                                                   
     ...
Testing
        Helps clarify what the problem is
    ●



        Helps clarify where the problem is
    ●


            ...
Interactive Testing
        Perl has a debugger
    ●



        Handy for running little snippets of code to 
    ●


   ...
Interactive Testing
    ~$ perl ­debug

    Enter h or `h h' for help, or `man perldebug' for more help.

    main::(­e:1)...
Interactive Debugging
        Perl's debugger can also do much more
    ●


            Load modules
        –

          ...
Mapping the Forest




64             
                                           
                  64   
               ...
Mapping the Forest
        Documentation!
    ●


            not just a boring task
        –

            it's a way to ...
Timing (bonus)
     what's
      the
     hurry?

                  The Forest




                      
66              ...
Timing the piggies
        By hand
    ●


            my $timer = time;
        –

            ...do stuff...
        –

...
Timing the piggies
                             (accurately)
        By hand
    ●


            use Time::HiRes qw(time);...
What to optimize
        Sometimes a script/subroutine may not be the 
    ●


        guilty party
        We should meas...
What to optimize
        Devel::DProf
    ●


            Show which subroutines took most time to run in 
        –
     ...
test.pl
 #!/usr/bin/perl
 use strict; use warnings;


 print_hello();
 do_do_sleep();

 sub print_hello {
     for (1..100...
running test.pl under DProf
        $ perl ­d:DProf test.pl
    ●


        HELLOHELLOHELLOHELLOHELLOHELLOHELLO
    ●

   ...
dprofpp
$ dprofpp 
Total Elapsed Time = 2.379776 Seconds
  User+System Time = 0.059776 Seconds
Exclusive Times
%Time ExclS...
dprofpp timings
        Note that sleep wasn't counted
    ●


            real times not counted, just CPU of local machi...
dprofpp -r                         (real)
$ dprofpp  ­r
Total Elapsed Time = 2.379776 Seconds
         Real Time = 2.37977...
dprofpp -r timings
        Note that do_do_sleep wasn't counted
    ●


            inclusive times not counted (child cal...
dprofpp -rI
               (real, inclusive)
$ dprofpp  ­rI
Total Elapsed Time = 2.379776 Seconds
         Real Time = 2.3...
DProf stability
        Perl debugging hooks are... a little fragile
    ●



        dprofpp is particularly so
    ●



...
DProf in web scripts
        Not run from command line...
    ●



        but we can add most command line flags to 
    ...
DProf in web scripts
        By default, DProf will create a file
    ●


            $current_dir/tmon.out
        –

   ...
DProf in web scripts
                     (tmon.out)
        #!/usr/bin/perl 
    –

        #!/usr/bin/perl ­d:DProf
    ...
DProf in web scripts
        There are other modules!
    ●


            Devel::SmallProf
        –

            Devel::F...
Profiling: what next?
        You found a hotspot
    ●


            takes lots of time...
        –

            ... or ...
Profiling: what next?
        You found a hotspot
    ●


            takes lots of time...
        –

            ... or ...
testing optimization
        things to test
    ●


            1) does it still work
        –

            2) do other t...
testing optimization
        old version of conv
    ●


    sub conv {
        my $id = shift;

        my %hash = (
    ...
testing optimization
        new version of conv
    ●


    sub conv {
        my $id = shift;
        $id=~tr/0123456789...
testing optimization
        The test
    ●


    use Test::More tests=>1;

    # test the quot;convquot; function in sn_c...
testing optimization
             (3. is it faster?)
use Benchmark qw(:all);
use Test::More tests => 1;

my @LIST = '0000'...
testing optimization
             (3. is it faster?)
my $results = timethese( 1_000_000,
    {
        conv1 => &conv1,
  ...
testing optimization
             (3. is it faster?)
$ perl bench.pl 
1..1
ok 1 ­ structures are identical

Benchmark: tim...
testing optimization
             (3. is it faster?)
$ perl bench.pl 
1..1
ok 1 ­ structures are identical

Benchmark: tim...
testing optimization
                 (3. is it faster?)
        Benchmark compares on a micro level, running 
    ●


   ...
Grazie

Hakim Cassimally
Senior Dev, dada.net

hakim.cassimally@staff.dada.net

...
Grazie
Web Developer @ Dada Group
Dada Group is seeking Perl and/or PHP Developers
to join our passionate & international ...
Bigbadwolf
Bigbadwolf
Upcoming SlideShare
Loading in...5
×

Bigbadwolf

8,634

Published on

My talk at YAPC::EU::2007::Vienna on debugging web applications.
A version of this talk was originally given as part of DADA.net's in-house training.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
8,634
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
46
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Bigbadwolf

  1. 1. ? Who's afraid of the Big Bad Wolf Hakim.Cassimally@staff.dada.net 2007­08­30 DADA Spa., Firenze, Italy                1     30 agosto 2007
  2. 2. Debugging web applications Debugging is hard ● Complex, legacy applications... ● Superstition ● Panic ● A metaphor involving pigs and wolves to the  ● rescue!                3     30 agosto 2007
  3. 3. This metaphor is brought to you by firenze.pm ● perl.it ● beer ● (Belhaven Best, James Pub, Firenze) –                4     30 agosto 2007
  4. 4. What happens when a little piggy goes in the forest? The Forest                5     30 agosto 2007
  5. 5. What happens when a little piggy goes in the forest? CGI The Forest                6     30 agosto 2007
  6. 6. What happens when a little piggy goes in the forest? mod_perl The Forest CGI                7     30 agosto 2007
  7. 7. What happens when a little piggy goes in the forest? mod_perl Perl The Forest CGI                8     30 agosto 2007
  8. 8. What happens when a little piggy goes in the forest? mod_perl Perl DBI The Forest CGI                9     30 agosto 2007
  9. 9. What happens when a little piggy goes in the forest? mod_perl modules Perl The Forest CGI DBI                10    30 agosto 2007  
  10. 10. What happens when a little piggy goes in the forest? our mod_perl Perl modules code The Forest CGI DBI                11    30 agosto 2007  
  11. 11. The Forest And more! ● Network – HTML – XML – 3rd party services – AJAX – Configuration – Unix permissions –                12    30 agosto 2007  
  12. 12. Werewolves!                13    30 agosto 2007  
  13. 13. Werewolves! There! Wolves!                14    30 agosto 2007  
  14. 14. What happens when a little piggy goes in the forest?                15    30 agosto 2007  
  15. 15. What happens when a little piggy goes in the forest? The Forest pig 16               16    30 agosto 2007  
  16. 16. What happens when a little piggy goes in the forest? YARGGH! The Forest                17    30 agosto 2007  
  17. 17. What happens when a little piggy goes in the forest? 404 The Forest                18    30 agosto 2007  
  18. 18. What happens when a little piggy goes in the forest? The Forest another pig                19    30 agosto 2007  
  19. 19. What happens when a little piggy goes in the forest? The Forest                20    30 agosto 2007  
  20. 20. What happens when a little piggy goes in the forest? It never came out! ● It came out limping and wounded ● Dead!  Gored by werewolves! ● Takes a long time to come out ●                21    30 agosto 2007  
  21. 21. What happens when a little piggy goes in the forest? It never came out! ● It came out limping and wounded ● Part of the functionality is broken – Information wrong – Dead!  Gored by werewolves! ● User sees raw error on screen – Takes a long time to come out ● optimization needed –                22    30 agosto 2007  
  22. 22. Werewolves! Round up the villagers for a lynch­mob! ● (Bring a pitchfork!) ● Burn them! ● (maybe burn a few innocent villagers along the way  – too!)                23    30 agosto 2007  
  23. 23. Engineers Luckily things are (a little) better if we know  ● what we're doing                24    30 agosto 2007  
  24. 24. We shouldn't see this Scary Forest (eeek!)                25    30 agosto 2007  
  25. 25. But this                26    30 agosto 2007  
  26. 26. Logging Trace the piggy as he goes through the forest ● 27               27    30 agosto 2007  
  27. 27. Logging PIG1: I'm scared The Forest pig                28    30 agosto 2007  
  28. 28. Logging PIG1: I'm scared PIG1: I'm very scared The Forest                29    30 agosto 2007  
  29. 29. Logging PIG1: I'm scared PIG1: I'm very scared PIG1: This is Cruelty to Animals! The Forest                30    30 agosto 2007  
  30. 30. Logging PIG1: I'm scared PIG1: I'm very scared PIG1: This is Cruelty to Animals! PIG1: Speaking to the Database The Forest                31    30 agosto 2007  
  31. 31. Logging PIG1: I'm scared PIG1: I'm very scared PIG1: This is Cruelty to Animals! PIG1: Speaking to the Database PIG1: Look over there, a wol... The Forest                32    30 agosto 2007  
  32. 32. Logging (print statements) Print statements ● print “I AM HERE!”; –                33    30 agosto 2007  
  33. 33. Logging (warnings) warn ● warn “THIS HAPPENED!”; – (this will end up in the Apache error log – /var/log/apache2/error.log(.yyyy.mm.dd) ● errors also go to Apache error log ● always check the error log –                34    30 agosto 2007  
  34. 34. Logging (apache error log) $ tail ­f error.log.2007­04­11 ● | grep “Media”   # your module ● | grep “HC”      # your identifier ● Point to the right machine ● edit  /etc/hosts  ( ...driversetchosts  on Windows) – send to a server you have an account on – (restart Firefox, sigh) –                35    30 agosto 2007  
  35. 35. Logging (apache error log) $ tail ­f error.log.2007­04­11 ● | grep ­v “uninitialized” ● Too many warnings! ● Hard to see what are real errors ● (Also annoys the Sysadmins) ●                36    30 agosto 2007  
  36. 36. Logging (print statements) Print to HTML ● content­type: text/html   (or text/plain) – <pre> $info </pre> – You can wrap this in a call – $this­>printDebug(“pippo”); ● But this messes up headers! ● Label your own piggy! ● $this­>printDebug( –     “HC(contacts): pippo”);                37    30 agosto 2007  
  37. 37. Logging (data structures) Complex data structures ● Data::Dumper – $VAR1 = { – 'y' => 2, 'x' => 1 };                38    30 agosto 2007  
  38. 38. Logging (manual trace) When did the script crash? ● $this­>printDebug(“HC 1 got here”); – do something – $this­>printDebug(“HC 2 got here”); – do something – $this­>printDebug(“HC 3 got here”); – do something – $this­>printDebug(“HC 4 got here”); –                39    30 agosto 2007  
  39. 39. Logging (manual trace) Yes, that's quite stupid ● (but it beats using a pitchfork) –                40    30 agosto 2007  
  40. 40. Logging (end of html debug) To avoid messing up headers you can get your  ● framework to print a development footer at the  bottom of the HTML page instead                41    30 agosto 2007  
  41. 41. Logging (to file) save to a file ● open file – write – close file – Lots of people recommend Log::Log4perl ●                42    30 agosto 2007  
  42. 42. Logging use Log::Log4Perl qw(:easy); ● Log::Log4Perl­>easy_init($ERROR); ● my $log = Log::Log4perl­>get_logger('foo'); ● $log­>debug(“Only show this to logs”); ● $log­>info(“This is more important”); ● $log­>warn(“Oh dear”); ● $log­>error(“Eek! This error on  ● screen”); $log­>fatal(“Werewolves!”); ●                43    30 agosto 2007  
  43. 43. Logging Log files eventually get really big ● Sysadmins will want to rotate them ● This may mean appending ● .yyyy.mm.dd – to the filenames                44    30 agosto 2007  
  44. 44. Error handling If a pig screams in the forest and there's noone there to hear it... did it really die? The Forest 45    another pig            45    30 agosto 2007  
  45. 45. Error handling Sometimes it's OK to die ● use CGI::Carp qw( fatalsToBrowser ); – die “HC: eeeeeek!  The piggy is dead”;  –                46    30 agosto 2007  
  46. 46. Error handling (CSI!) But how did the piggy die? ● Bad parameter at TestModule.pm line 10. – If only we knew who had called TestModule! ●                47    30 agosto 2007  
  47. 47. Error handling (Carp) package TestModule; ● use Carp qw( croak ); ● ...   ● croak “Bad parameter”; ... Bad parameter at test.pl line 7 –                48    30 agosto 2007  
  48. 48. Error handling (stacktraces) What if it was another subroutine or module  ● called? use Carp qw(confess); ● confess “Ooops!”; ● Ooops! at Pippo.pm line 13 ● Pippo::three() called at Pluto.pm line 10 Pluto::two() called at Pinco.pm line 11 Pinco::one() called at test.pl line 7                49    30 agosto 2007  
  49. 49. Error handling (framework) If your framework handles this, you may get  ● stacktraces by default They aren't always easy to read, but they contain  – lots of useful information! Catalyst – In house framwork –                50    30 agosto 2007  
  50. 50. Error handling (Database) We don't want the user seeing DB errors in live! ● BUT – We probably want to see them in staging! – classic idiom – $sth­>execute()   or die $sth­>errstr; ● You may want to: – have a “fatal” function which either dies informatively (in  ● staging) or produces an appropriate error document (in  live)                51    30 agosto 2007  
  51. 51. Error handling (Database) Do you really want to fail silently...? ● $sth­>execute() – or warn $sth­>errstr; (or $logger­>log($sth­>errstr)); –                52    30 agosto 2007  
  52. 52. Testing I hope someone tested this parachute The Forest 54               53    30 agosto 2007  
  53. 53. Burn the Witch! Sometimes we think we know where the  ● problem is Sometimes we're wrong –                54    30 agosto 2007  
  54. 54. How do we check A test! ● does it weigh more or less than a duck? – use Test::More tests => 1; ● is( $possible_witch­>weight(), –     $duck­>weight(),     “It's the witch!” );                55    30 agosto 2007  
  55. 55. Testing in the forest Sometimes we need to test: ● user clicks on link in browser, logs in – user puts in information – the script talks to the Database – the script talks to a 3rd party API – the script talks to a module which returns some  – information the HTML is processed – (e.g. the forest) ●                56    30 agosto 2007  
  56. 56. Testing in the forest But... ● what if we suspect that the problem is with  – formatting of display name? MyModule:: createDisplayName(); – Just test that! –                57    30 agosto 2007  
  57. 57. See the trees... Write a simple test ● use Test::More; – use MyModule; – is ( – $m­>createDisplayName('robin'),      “Robin Hood”,      “display name returned correctly” ); If the test returns correctly, then maybe the  ● problem is before (or after) (we could use logging to help find out which!) ●                58    30 agosto 2007  
  58. 58. See the trees... MVC! ●                59    30 agosto 2007  
  59. 59. Testing Helps clarify what the problem is ● Helps clarify where the problem is ● (or isn't) – Also ● vampires are repelled by tests! –                60    30 agosto 2007  
  60. 60. Interactive Testing Perl has a debugger ● Handy for running little snippets of code to  ● check how it works                61    30 agosto 2007  
  61. 61. Interactive Testing ~$ perl ­debug Enter h or `h h' for help, or `man perldebug' for more help. main::(­e:1):   bug  DB<1> @a = (1,2,3,4,5) DB<2> splice(@a, 2,1) = (quot;onequot;, quot;twoquot;); Can't modify splice in scalar assignment at (eval  18)[/usr/share/perl/5.8/perl5db.pl:628] line 2, at EOF DB<3> splice(@a, 2,1, quot;onequot;, quot;twoquot;);    DB<4> x @a  0  1  1  2  2  'one'  3  'two'  4  4  5  5                62    30 agosto 2007  
  62. 62. Interactive Debugging Perl's debugger can also do much more ● Load modules – Set breakpoints – Step through file, etc. – It could also be connected to remotely, for  ● example in Apache sessions see also Devel::ebug ● ... and Devel::REPL ●                63    30 agosto 2007  
  63. 63. Mapping the Forest 64               64    30 agosto 2007  
  64. 64. Mapping the Forest Documentation! ● not just a boring task – it's a way to make sure the piggies don't get eaten  – by wolves next time POD is good! ● but ● # Watch out!  We're doing this because  # otherwise {...} would happen! is good too.                65    30 agosto 2007  
  65. 65. Timing (bonus) what's the hurry? The Forest     66            66    30 agosto 2007  
  66. 66. Timing the piggies By hand ● my $timer = time; – ...do stuff... – my $delta = time ­ $timer; –                67    30 agosto 2007  
  67. 67. Timing the piggies (accurately) By hand ● use Time::HiRes qw(time); – my $timer = time; – ...do stuff... – my $delta = time ­ $timer; –                68    30 agosto 2007  
  68. 68. What to optimize Sometimes a script/subroutine may not be the  ● guilty party We should measure... ● everything? – by hand? –                69    30 agosto 2007  
  69. 69. What to optimize Devel::DProf ● Show which subroutines took most time to run in  – your script                70    30 agosto 2007  
  70. 70. test.pl #!/usr/bin/perl use strict; use warnings; print_hello(); do_do_sleep(); sub print_hello {     for (1..100_000) {  print quot;HELLOquot;;  } } sub do_do_sleep {     for (1..2) { do_sleep(); } } sub do_sleep {     sleep 1; }                71    30 agosto 2007  
  71. 71. running test.pl under DProf $ perl ­d:DProf test.pl ● HELLOHELLOHELLOHELLOHELLOHELLOHELLO ● HELLOHELLOHELLOHELLOHELLOHELLOHELLO HELLOHELLOHELLOHELLOHELLOHELLOHELLO HELLOHELLOHELLOHELLOHELLOHELLOHELLO HELLOHELLOHELLOHELLOHELLOHELLOHELLO HELLOHELLOHELLOHELLOHELLOHELLOHELLO HELLOHELLOHELLOHELLOHELLOHELLOHELLO HELLOHELLOHELLOHELLOHELLOHELLOHELLO HELLOHELLOHELLOHELLOHELLOHELLOHELLO HELLOHELLOHELLOHELLOHELLOHELLOHELLO                72    30 agosto 2007  
  72. 72. dprofpp $ dprofpp  Total Elapsed Time = 2.379776 Seconds   User+System Time = 0.059776 Seconds Exclusive Times %Time ExclSec CumulS #Calls sec/call Csec/c  Name  66.9   0.040  0.040      1   0.0400 0.0400  main::print_hello  16.7   0.010  0.010      3   0.0033 0.0033  main::BEGIN  0.00       ­ ­0.000      1        ­      ­  strict::bits  0.00       ­ ­0.000      1        ­      ­  strict::import  0.00       ­ ­0.000      1        ­      ­  warnings::import  0.00       ­ ­0.000      1        ­      ­  bytes::import  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_load_fi  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_undef_s  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_find_sy  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_install  0.00       ­ ­0.000      1        ­      ­  warnings::BEGIN  0.00       ­ ­0.000      1        ­      ­  Data::Dumper::bootstra                73    30 agosto 2007  
  73. 73. dprofpp timings Note that sleep wasn't counted ● real times not counted, just CPU of local machine – sleep doesn't affect local CPU – neither does remote DB call ! –                74    30 agosto 2007  
  74. 74. dprofpp -r (real) $ dprofpp  ­r Total Elapsed Time = 2.379776 Seconds          Real Time = 2.379776 Seconds Exclusive Times %Time ExclSec CumulS #Calls sec/call Csec/c  Name  84.4   2.010  2.010      2   1.0050 1.0050  main::do_sleep  14.7   0.350  0.350      1   0.3500 0.3500  main::print_hello  0.42   0.010  0.010      1   0.0100 0.0100  warnings::BEGIN  0.42   0.010  0.010      1   0.0100 0.0100  overload::BEGIN  0.00       ­ ­0.000      1        ­      ­  strict::bits  0.00       ­ ­0.000      1        ­      ­  strict::import  0.00       ­ ­0.000      1        ­      ­  warnings::import  0.00       ­ ­0.000      1        ­      ­  bytes::import  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_load_fi  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_undef_s  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_find_sy  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_install  0.00       ­ ­0.000      1        ­      ­  warnings::BEGIN                75    30 agosto 2007  
  75. 75. dprofpp -r timings Note that do_do_sleep wasn't counted ● inclusive times not counted (child calls).   – Just the time a single subroutine takes to run. Usually, you want to know what all of your  – subroutine was doing!                76    30 agosto 2007  
  76. 76. dprofpp -rI (real, inclusive) $ dprofpp  ­rI Total Elapsed Time = 2.379776 Seconds          Real Time = 2.379776 Seconds Inclusive Times %Time ExclSec CumulS #Calls sec/call Csec/c  Name  84.4   2.010  2.010      2   1.0050 1.0050  main::do_sleep  84.4       ­  2.010      1        ­ 2.0100  main::do_do_sleep  14.7   0.350  0.350      1   0.3500 0.3500  main::print_hello  0.84       ­  0.020      3        ­ 0.0066  main::BEGIN  0.42   0.010  0.010      1   0.0100 0.0100  warnings::BEGIN  0.42   0.010  0.010      1   0.0100 0.0100  overload::BEGIN  0.00       ­ ­0.000      1        ­      ­  strict::bits  0.00       ­ ­0.000      1        ­      ­  strict::import  0.00       ­ ­0.000      1        ­      ­  warnings::import  0.00       ­ ­0.000      1        ­      ­  bytes::import  0.00       ­ ­0.000      1        ­      ­  DynaLoader::dl_load_fi  0.00       ­ ­0.000      1        ­      ­ DynaLoader::dl_undef_sy  0.00       ­ ­0.000      1        ­      ­ DynaLoader::dl_find_sym                77    30 agosto 2007  
  77. 77. DProf stability Perl debugging hooks are... a little fragile ● dprofpp is particularly so ● Depending on the host/perl you may find ● perl ­d:Whatever  will crash up to ~20% of time                78    30 agosto 2007  
  78. 78. DProf in web scripts Not run from command line... ● but we can add most command line flags to  ● shebang! Change ● #!/usr/bin/perl – #!/usr/bin/perl ­d:DProf – This works ● (except the 20% of time that script will just crash) –                79    30 agosto 2007  
  79. 79. DProf in web scripts By default, DProf will create a file ● $current_dir/tmon.out – (This may be an issue if you are developing  ● without individual sandboxes)                80    30 agosto 2007  
  80. 80. DProf in web scripts (tmon.out) #!/usr/bin/perl  – #!/usr/bin/perl ­d:DProf –                81    30 agosto 2007  
  81. 81. DProf in web scripts There are other modules! ● Devel::SmallProf – Devel::FastProf – Devel::DProfLB   (“Less Bad”) – Apache::DProf   (If running on mod_perl) – Search “prof” on http://search.cpan.org ● look at documentation – look at reviews! –                82    30 agosto 2007  
  82. 82. Profiling: what next? You found a hotspot ● takes lots of time... – ... or called lots of times – What next? ● make the hotspot faster – find a way to rewrite it – PROFIT? –                83    30 agosto 2007  
  83. 83. Profiling: what next? You found a hotspot ● takes lots of time... – ... or called lots of times – What next? ● make the hotspot faster – find a way to rewrite it – make sure everything still works! –                84    30 agosto 2007  
  84. 84. testing optimization things to test ● 1) does it still work – 2) do other things that used the same code still  – work 3) is it faster – 1. you can do by hand ● 2. it's nicer to have automated tests ●                85    30 agosto 2007  
  85. 85. testing optimization old version of conv ● sub conv {     my $id = shift;     my %hash = (         0 => 'a',        1 => 'b',         2 => 'c',        3 => 'd',         4 => 'e',        5 => 'f',         6 => 'g',        7 => 'h',         8 => 'i',        9 => 'l'     );     my @single = split //, $id;     my $res;     foreach (@single){         $res = $hash{$_}.$res;     }     return $res; }                86    30 agosto 2007  
  86. 86. testing optimization new version of conv ● sub conv {     my $id = shift;     $id=~tr/0123456789//cd;   # delete things not in the range     $id=~tr/0123456789/abcdefghil/d;  # do the transliteration     return reverse $id; }                87    30 agosto 2007  
  87. 87. testing optimization The test ● use Test::More tests=>1; # test the quot;convquot; function in sn_contacts/main.chm is (conv( quot;0123quot; ),              quot;dcbaquot; ); is (conv( quot;hello 0123 worldquot; ),  quot;dcbaquot; ); # sanity check is (conv( quot;890quot; ),               quot;aliquot; ); is (conv( quot;0123456789quot; ),        quot;lihgfedcbaquot; ); is (conv2( quot;0123quot; ),             quot;dcbaquot; ); is (conv2( quot;hello 0123 worldquot; ), quot;dcbaquot; ); # sanity check is (conv2( quot;890quot; ),              quot;aliquot; ); is (conv2( quot;0123456789quot; ),       quot;lihgfedcbaquot; );                88    30 agosto 2007  
  88. 88. testing optimization (3. is it faster?) use Benchmark qw(:all); use Test::More tests => 1; my @LIST = '0000'..'9999'; is_deeply(     [ conv1(@LIST) ],     [ conv2(@LIST) ],     quot;structures are identicalquot;, ); sub do_conv1 {     return map {         conv( $_ );     } @_; } sub do_conv2 {     return map {         conv2( $_ );     } @_;     }            89    30 agosto 2007  
  89. 89. testing optimization (3. is it faster?) my $results = timethese( 1_000_000,     {         conv1 => &conv1,         conv2 => &conv2,     },     quot;nonequot; ); cmpthese( $results );                90    30 agosto 2007  
  90. 90. testing optimization (3. is it faster?) $ perl bench.pl  1..1 ok 1 ­ structures are identical Benchmark: timing 1000000 iterations of conv1, conv2...      conv1: ­1 wallclock secs ( 0.16 usr +  0.00 sys =  0.16 CPU) @  6250000.00/s (n=1000000)             (warning: too few iterations for a reliable count)      conv2:  0 wallclock secs ( 0.16 usr +  0.00 sys =  0.16 CPU) @  6250000.00/s (n=1000000)             (warning: too few iterations for a reliable count)            Rate conv2 conv1 conv2 6250000/s    ­­   ­0% conv1 6250000/s    0%    ­­                91    30 agosto 2007  
  91. 91. testing optimization (3. is it faster?) $ perl bench.pl  1..1 ok 1 ­ structures are identical Benchmark: timing 1000000 iterations of conv1, conv2...      conv1: ­1 wallclock secs ( 0.16 usr +  0.00 sys =  0.16 CPU) @  6250000.00/s (n=1000000)             (warning: too few iterations for a reliable count)      conv2:  0 wallclock secs ( 0.16 usr +  0.00 sys =  0.16 CPU) @  6250000.00/s (n=1000000)             (warning: too few iterations for a reliable count)            Rate conv2 conv1 conv2 6250000/s    ­­   ­0% conv1 6250000/s    0%    ­­                92    30 agosto 2007  
  92. 92. testing optimization (3. is it faster?) Benchmark compares on a micro level, running  ● thousands (or millions) of times Now test on macro (single script) ● Redo dprofpp, compare results – (Or even save the old tmon.out) –                93    30 agosto 2007  
  93. 93. Grazie Hakim Cassimally Senior Dev, dada.net hakim.cassimally@staff.dada.net ...
  94. 94. Grazie Web Developer @ Dada Group Dada Group is seeking Perl and/or PHP Developers to join our passionate & international Production Team. Multiple locations are available in Italy, Spain, the United States, Brazil & China, for on-site work. Your profile: you have passion for the Internet, new technologies and systems integration. Moreover you have extensive experience developing applications for the web, especially in the consumer entertainment (web & mobile) field. If you think you are up to the challenge, please send us your resume !!! recruiting@staff.dada.net Contact: www.dada.net Web-site:
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×