My talk at YAPC::EU::2007::Vienna on debugging web applications. A version of this talk was originally given as part of DADA.net's in-house training.

1. ?
Who's afraid
of the
Big Bad Wolf
Hakim.Cassimally@staff.dada.net
2007­08­30 DADA Spa., Firenze, Italy
1
30 agosto 2007

3. Debugging web applications
Debugging is hard
●
Complex, legacy applications...
●
Superstition
●
Panic
●
A metaphor involving pigs and wolves to the
●
rescue!
3
30 agosto 2007

4. This metaphor is brought to
you by
firenze.pm
●
perl.it
●
beer
●
(Belhaven Best, James Pub, Firenze)
–
4
30 agosto 2007

5. What happens when a little
piggy goes in the forest?
The Forest
5
30 agosto 2007

6. What happens when a little
piggy goes in the forest?
CGI
The Forest
6
30 agosto 2007

7. What happens when a little
piggy goes in the forest?
mod_perl
The Forest
CGI
7
30 agosto 2007

8. What happens when a little
piggy goes in the forest?
mod_perl
Perl
The Forest
CGI
8
30 agosto 2007

9. What happens when a little
piggy goes in the forest?
mod_perl
Perl
DBI
The Forest
CGI
9
30 agosto 2007

10. What happens when a little
piggy goes in the forest?
mod_perl
modules
Perl
The Forest
CGI DBI
10
30 agosto 2007

11. What happens when a little
piggy goes in the forest?
our mod_perl
Perl
modules
code
The Forest
CGI DBI
11
30 agosto 2007

12. The Forest
And more!
●
Network
–
HTML
–
XML
–
3rd party services
–
AJAX
–
Configuration
–
Unix permissions
–
12
30 agosto 2007

13. Werewolves!
13
30 agosto 2007

14. Werewolves!
There!
Wolves!
14
30 agosto 2007

15. What happens when a little
piggy goes in the forest?
15
30 agosto 2007

16. What happens when a little
piggy goes in the forest?
The Forest
pig
16
16
30 agosto 2007

17. What happens when a little
piggy goes in the forest?
YARGGH!
The Forest
17
30 agosto 2007

18. What happens when a little
piggy goes in the forest?
404
The Forest
18
30 agosto 2007

19. What happens when a little
piggy goes in the forest?
The Forest another pig
19
30 agosto 2007

20. What happens when a little
piggy goes in the forest?
The Forest
20
30 agosto 2007

21. What happens when a little
piggy goes in the forest?
It never came out!
●
It came out limping and wounded
●
Dead! Gored by werewolves!
●
Takes a long time to come out
●
21
30 agosto 2007

22. What happens when a little
piggy goes in the forest?
It never came out!
●
It came out limping and wounded
●
Part of the functionality is broken
–
Information wrong
–
Dead! Gored by werewolves!
●
User sees raw error on screen
–
Takes a long time to come out
●
optimization needed
–
22
30 agosto 2007

23. Werewolves!
Round up the villagers for a lynch­mob!
●
(Bring a pitchfork!)
●
Burn them!
●
(maybe burn a few innocent villagers along the way
–
too!)
23
30 agosto 2007

24. Engineers
Luckily things are (a little) better if we know
●
what we're doing
24
30 agosto 2007

25. We shouldn't see this
Scary Forest
(eeek!)
25
30 agosto 2007

26. But this
26
30 agosto 2007

27. Logging
Trace the piggy as he goes through the forest
●
27
27
30 agosto 2007

28. Logging
PIG1: I'm scared
The Forest
pig
28
30 agosto 2007

29. Logging
PIG1: I'm scared
PIG1: I'm very scared
The Forest
29
30 agosto 2007

30. Logging
PIG1: I'm scared
PIG1: I'm very scared
PIG1: This is Cruelty to Animals!
The Forest
30
30 agosto 2007

31. Logging
PIG1: I'm scared
PIG1: I'm very scared
PIG1: This is Cruelty to Animals!
PIG1: Speaking to the Database
The Forest
31
30 agosto 2007

32. Logging
PIG1: I'm scared
PIG1: I'm very scared
PIG1: This is Cruelty to Animals!
PIG1: Speaking to the Database
PIG1: Look over there, a wol...
The Forest
32
30 agosto 2007

33. Logging (print statements)
Print statements
●
print “I AM HERE!”;
–
33
30 agosto 2007

34. Logging (warnings)
warn
●
warn “THIS HAPPENED!”;
–
(this will end up in the Apache error log
–
/var/log/apache2/error.log(.yyyy.mm.dd)
●
errors also go to Apache error log
●
always check the error log
–
34
30 agosto 2007

35. Logging (apache error log)
$ tail ­f error.log.2007­04­11
●
| grep “Media” # your module
●
| grep “HC” # your identifier
●
Point to the right machine
●
edit /etc/hosts ( ...driversetchosts on Windows)
–
send to a server you have an account on
–
(restart Firefox, sigh)
–
35
30 agosto 2007

36. Logging (apache error log)
$ tail ­f error.log.2007­04­11
●
| grep ­v “uninitialized”
●
Too many warnings!
●
Hard to see what are real errors
●
(Also annoys the Sysadmins)
●
36
30 agosto 2007

37. Logging (print statements)
Print to HTML
●
content­type: text/html (or text/plain)
–
<pre> $info </pre>
–
You can wrap this in a call
–
$this­>printDebug(“pippo”);
●
But this messes up headers!
●
Label your own piggy!
●
$this­>printDebug(
–
“HC(contacts): pippo”);
37
30 agosto 2007

38. Logging (data structures)
Complex data structures
●
Data::Dumper
–
$VAR1 = {
–
'y' => 2,
'x' => 1
};
38
30 agosto 2007

39. Logging (manual trace)
When did the script crash?
●
$this­>printDebug(“HC 1 got here”);
–
do something
–
$this­>printDebug(“HC 2 got here”);
–
do something
–
$this­>printDebug(“HC 3 got here”);
–
do something
–
$this­>printDebug(“HC 4 got here”);
–
39
30 agosto 2007

40. Logging (manual trace)
Yes, that's quite stupid
●
(but it beats using a pitchfork)
–
40
30 agosto 2007

41. Logging
(end of html debug)
To avoid messing up headers you can get your
●
framework to print a development footer at the
bottom of the HTML page instead
41
30 agosto 2007

42. Logging (to file)
save to a file
●
open file
–
write
–
close file
–
Lots of people recommend Log::Log4perl
●
42
30 agosto 2007

43. Logging
use Log::Log4Perl qw(:easy);
●
Log::Log4Perl­>easy_init($ERROR);
●
my $log = Log::Log4perl­>get_logger('foo');
●
$log­>debug(“Only show this to logs”);
●
$log­>info(“This is more important”);
●
$log­>warn(“Oh dear”);
●
$log­>error(“Eek! This error on
●
screen”);
$log­>fatal(“Werewolves!”);
●
43
30 agosto 2007

44. Logging
Log files eventually get really big
●
Sysadmins will want to rotate them
●
This may mean appending
●
.yyyy.mm.dd
–
to the filenames
44
30 agosto 2007

45. Error handling
If a pig screams in
the forest and
there's noone there
to hear it... did it
really die?
The Forest
45
another pig
45
30 agosto 2007

46. Error handling
Sometimes it's OK to die
●
use CGI::Carp qw( fatalsToBrowser );
–
die “HC: eeeeeek! The piggy is dead”;
–
46
30 agosto 2007

47. Error handling (CSI!)
But how did the piggy die?
●
Bad parameter at TestModule.pm line 10.
–
If only we knew who had called TestModule!
●
47
30 agosto 2007

48. Error handling (Carp)
package TestModule;
●
use Carp qw( croak );
●
...
●
croak “Bad parameter”;
...
Bad parameter at test.pl line 7
–
48
30 agosto 2007

49. Error handling (stacktraces)
What if it was another subroutine or module
●
called?
use Carp qw(confess);
●
confess “Ooops!”;
●
Ooops! at Pippo.pm line 13
●
Pippo::three() called at Pluto.pm line 10
Pluto::two() called at Pinco.pm line 11
Pinco::one() called at test.pl line 7
49
30 agosto 2007

50. Error handling (framework)
If your framework handles this, you may get
●
stacktraces by default
They aren't always easy to read, but they contain
–
lots of useful information!
Catalyst
–
In house framwork
–
50
30 agosto 2007

51. Error handling (Database)
We don't want the user seeing DB errors in live!
●
BUT
–
We probably want to see them in staging!
–
classic idiom
–
$sth­>execute() or die $sth­>errstr;
●
You may want to:
–
have a “fatal” function which either dies informatively (in
●
staging) or produces an appropriate error document (in
live)
51
30 agosto 2007

52. Error handling (Database)
Do you really want to fail silently...?
●
$sth­>execute()
–
or warn $sth­>errstr;
(or $logger­>log($sth­>errstr));
–
52
30 agosto 2007

53. Testing
I hope
someone
tested this
parachute
The Forest
54
53
30 agosto 2007

54. Burn the Witch!
Sometimes we think we know where the
●
problem is
Sometimes we're wrong
–
54
30 agosto 2007

55. How do we check
A test!
●
does it weigh more or less than a duck?
–
use Test::More tests => 1;
●
is( $possible_witch­>weight(),
–
$duck­>weight(),
“It's the witch!” );
55
30 agosto 2007

56. Testing in the forest
Sometimes we need to test:
●
user clicks on link in browser, logs in
–
user puts in information
–
the script talks to the Database
–
the script talks to a 3rd party API
–
the script talks to a module which returns some
–
information
the HTML is processed
–
(e.g. the forest)
●
56
30 agosto 2007

57. Testing in the forest
But...
●
what if we suspect that the problem is with
–
formatting of display name?
MyModule:: createDisplayName();
–
Just test that!
–
57
30 agosto 2007

58. See the trees...
Write a simple test
●
use Test::More;
–
use MyModule;
–
is (
–
$m­>createDisplayName('robin'),
“Robin Hood”,
“display name returned correctly” );
If the test returns correctly, then maybe the
●
problem is before (or after)
(we could use logging to help find out which!)
●
58
30 agosto 2007

59. See the trees...
MVC!
●
59
30 agosto 2007

60. Testing
Helps clarify what the problem is
●
Helps clarify where the problem is
●
(or isn't)
–
Also
●
vampires are repelled by tests!
–
60
30 agosto 2007

61. Interactive Testing
Perl has a debugger
●
Handy for running little snippets of code to
●
check how it works
61
30 agosto 2007

62. Interactive Testing
~$ perl ­debug
Enter h or `h h' for help, or `man perldebug' for more help.
main::(­e:1): bug
DB<1> @a = (1,2,3,4,5)
DB<2> splice(@a, 2,1) = (quot;onequot;, quot;twoquot;);
Can't modify splice in scalar assignment at (eval
18)[/usr/share/perl/5.8/perl5db.pl:628] line 2, at EOF
DB<3> splice(@a, 2,1, quot;onequot;, quot;twoquot;);
DB<4> x @a
0 1
1 2
2 'one'
3 'two'
4 4
5 5
62
30 agosto 2007

63. Interactive Debugging
Perl's debugger can also do much more
●
Load modules
–
Set breakpoints
–
Step through file, etc.
–
It could also be connected to remotely, for
●
example in Apache sessions
see also Devel::ebug
●
... and Devel::REPL
●
63
30 agosto 2007

64. Mapping the Forest
64
64
30 agosto 2007

65. Mapping the Forest
Documentation!
●
not just a boring task
–
it's a way to make sure the piggies don't get eaten
–
by wolves next time
POD is good!
●
but
●
# Watch out! We're doing this because
# otherwise {...} would happen!
is good too.
65
30 agosto 2007

66. Timing (bonus)
what's
the
hurry?
The Forest
66
66
30 agosto 2007

67. Timing the piggies
By hand
●
my $timer = time;
–
...do stuff...
–
my $delta = time ­ $timer;
–
67
30 agosto 2007

68. Timing the piggies
(accurately)
By hand
●
use Time::HiRes qw(time);
–
my $timer = time;
–
...do stuff...
–
my $delta = time ­ $timer;
–
68
30 agosto 2007

69. What to optimize
Sometimes a script/subroutine may not be the
●
guilty party
We should measure...
●
everything?
–
by hand?
–
69
30 agosto 2007

70. What to optimize
Devel::DProf
●
Show which subroutines took most time to run in
–
your script
70
30 agosto 2007

71. test.pl
#!/usr/bin/perl
use strict; use warnings;
print_hello();
do_do_sleep();
sub print_hello {
for (1..100_000) { print quot;HELLOquot;; }
}
sub do_do_sleep {
for (1..2) { do_sleep(); }
}
sub do_sleep {
sleep 1;
}
71
30 agosto 2007

72. running test.pl under DProf
$ perl ­d:DProf test.pl
●
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
●
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
HELLOHELLOHELLOHELLOHELLOHELLOHELLO
72
30 agosto 2007

73. dprofpp
$ dprofpp
Total Elapsed Time = 2.379776 Seconds
User+System Time = 0.059776 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
66.9 0.040 0.040 1 0.0400 0.0400 main::print_hello
16.7 0.010 0.010 3 0.0033 0.0033 main::BEGIN
0.00 ­ ­0.000 1 ­ ­ strict::bits
0.00 ­ ­0.000 1 ­ ­ strict::import
0.00 ­ ­0.000 1 ­ ­ warnings::import
0.00 ­ ­0.000 1 ­ ­ bytes::import
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_load_fi
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_undef_s
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_find_sy
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_install
0.00 ­ ­0.000 1 ­ ­ warnings::BEGIN
0.00 ­ ­0.000 1 ­ ­ Data::Dumper::bootstra
73
30 agosto 2007

74. dprofpp timings
Note that sleep wasn't counted
●
real times not counted, just CPU of local machine
–
sleep doesn't affect local CPU
–
neither does remote DB call !
–
74
30 agosto 2007

75. dprofpp -r (real)
$ dprofpp ­r
Total Elapsed Time = 2.379776 Seconds
Real Time = 2.379776 Seconds
Exclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
84.4 2.010 2.010 2 1.0050 1.0050 main::do_sleep
14.7 0.350 0.350 1 0.3500 0.3500 main::print_hello
0.42 0.010 0.010 1 0.0100 0.0100 warnings::BEGIN
0.42 0.010 0.010 1 0.0100 0.0100 overload::BEGIN
0.00 ­ ­0.000 1 ­ ­ strict::bits
0.00 ­ ­0.000 1 ­ ­ strict::import
0.00 ­ ­0.000 1 ­ ­ warnings::import
0.00 ­ ­0.000 1 ­ ­ bytes::import
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_load_fi
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_undef_s
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_find_sy
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_install
0.00 ­ ­0.000 1 ­ ­ warnings::BEGIN
75
30 agosto 2007

76. dprofpp -r timings
Note that do_do_sleep wasn't counted
●
inclusive times not counted (child calls).
–
Just the time a single subroutine takes to run.
Usually, you want to know what all of your
–
subroutine was doing!
76
30 agosto 2007

77. dprofpp -rI
(real, inclusive)
$ dprofpp ­rI
Total Elapsed Time = 2.379776 Seconds
Real Time = 2.379776 Seconds
Inclusive Times
%Time ExclSec CumulS #Calls sec/call Csec/c Name
84.4 2.010 2.010 2 1.0050 1.0050 main::do_sleep
84.4 ­ 2.010 1 ­ 2.0100 main::do_do_sleep
14.7 0.350 0.350 1 0.3500 0.3500 main::print_hello
0.84 ­ 0.020 3 ­ 0.0066 main::BEGIN
0.42 0.010 0.010 1 0.0100 0.0100 warnings::BEGIN
0.42 0.010 0.010 1 0.0100 0.0100 overload::BEGIN
0.00 ­ ­0.000 1 ­ ­ strict::bits
0.00 ­ ­0.000 1 ­ ­ strict::import
0.00 ­ ­0.000 1 ­ ­ warnings::import
0.00 ­ ­0.000 1 ­ ­ bytes::import
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_load_fi
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_undef_sy
0.00 ­ ­0.000 1 ­ ­ DynaLoader::dl_find_sym
77
30 agosto 2007

78. DProf stability
Perl debugging hooks are... a little fragile
●
dprofpp is particularly so
●
Depending on the host/perl you may find
●
perl ­d:Whatever
will crash up to ~20% of time
78
30 agosto 2007

79. DProf in web scripts
Not run from command line...
●
but we can add most command line flags to
●
shebang!
Change
●
#!/usr/bin/perl
–
#!/usr/bin/perl ­d:DProf
–
This works
●
(except the 20% of time that script will just crash)
–
79
30 agosto 2007

80. DProf in web scripts
By default, DProf will create a file
●
$current_dir/tmon.out
–
(This may be an issue if you are developing
●
without individual sandboxes)
80
30 agosto 2007

81. DProf in web scripts
(tmon.out)
#!/usr/bin/perl
–
#!/usr/bin/perl ­d:DProf
–
81
30 agosto 2007

82. DProf in web scripts
There are other modules!
●
Devel::SmallProf
–
Devel::FastProf
–
Devel::DProfLB (“Less Bad”)
–
Apache::DProf (If running on mod_perl)
–
Search “prof” on http://search.cpan.org
●
look at documentation
–
look at reviews!
–
82
30 agosto 2007

83. Profiling: what next?
You found a hotspot
●
takes lots of time...
–
... or called lots of times
–
What next?
●
make the hotspot faster
–
find a way to rewrite it
–
PROFIT?
–
83
30 agosto 2007

84. Profiling: what next?
You found a hotspot
●
takes lots of time...
–
... or called lots of times
–
What next?
●
make the hotspot faster
–
find a way to rewrite it
–
make sure everything still works!
–
84
30 agosto 2007

85. testing optimization
things to test
●
1) does it still work
–
2) do other things that used the same code still
–
work
3) is it faster
–
1. you can do by hand
●
2. it's nicer to have automated tests
●
85
30 agosto 2007

86. testing optimization
old version of conv
●
sub conv {
my $id = shift;
my %hash = (
0 => 'a', 1 => 'b',
2 => 'c', 3 => 'd',
4 => 'e', 5 => 'f',
6 => 'g', 7 => 'h',
8 => 'i', 9 => 'l'
);
my @single = split //, $id;
my $res;
foreach (@single){
$res = $hash{$_}.$res;
}
return $res;
}
86
30 agosto 2007

87. testing optimization
new version of conv
●
sub conv {
my $id = shift;
$id=~tr/0123456789//cd;
# delete things not in the range
$id=~tr/0123456789/abcdefghil/d;
# do the transliteration
return reverse $id;
}
87
30 agosto 2007

88. testing optimization
The test
●
use Test::More tests=>1;
# test the quot;convquot; function in sn_contacts/main.chm
is (conv( quot;0123quot; ), quot;dcbaquot; );
is (conv( quot;hello 0123 worldquot; ), quot;dcbaquot; ); # sanity check
is (conv( quot;890quot; ), quot;aliquot; );
is (conv( quot;0123456789quot; ), quot;lihgfedcbaquot; );
is (conv2( quot;0123quot; ), quot;dcbaquot; );
is (conv2( quot;hello 0123 worldquot; ), quot;dcbaquot; ); # sanity check
is (conv2( quot;890quot; ), quot;aliquot; );
is (conv2( quot;0123456789quot; ), quot;lihgfedcbaquot; );
88
30 agosto 2007

89. testing optimization
(3. is it faster?)
use Benchmark qw(:all);
use Test::More tests => 1;
my @LIST = '0000'..'9999';
is_deeply(
[ conv1(@LIST) ],
[ conv2(@LIST) ],
quot;structures are identicalquot;,
);
sub do_conv1 {
return map {
conv( $_ );
} @_;
}
sub do_conv2 {
return map {
conv2( $_ );
} @_;
}
89
30 agosto 2007

90. testing optimization
(3. is it faster?)
my $results = timethese( 1_000_000,
{
conv1 => &conv1,
conv2 => &conv2,
},
quot;nonequot;
);
cmpthese( $results );
90
30 agosto 2007

91. testing optimization
(3. is it faster?)
$ perl bench.pl
1..1
ok 1 ­ structures are identical
Benchmark: timing 1000000 iterations of conv1, conv2...
conv1: ­1 wallclock secs ( 0.16 usr + 0.00 sys = 0.16 CPU) @
6250000.00/s (n=1000000)
(warning: too few iterations for a reliable count)
conv2: 0 wallclock secs ( 0.16 usr + 0.00 sys = 0.16 CPU) @
6250000.00/s (n=1000000)
(warning: too few iterations for a reliable count)
Rate conv2 conv1
conv2 6250000/s ­­ ­0%
conv1 6250000/s 0% ­­
91
30 agosto 2007

92. testing optimization
(3. is it faster?)
$ perl bench.pl
1..1
ok 1 ­ structures are identical
Benchmark: timing 1000000 iterations of conv1, conv2...
conv1: ­1 wallclock secs ( 0.16 usr + 0.00 sys = 0.16 CPU) @
6250000.00/s (n=1000000)
(warning: too few iterations for a reliable count)
conv2: 0 wallclock secs ( 0.16 usr + 0.00 sys = 0.16 CPU) @
6250000.00/s (n=1000000)
(warning: too few iterations for a reliable count)
Rate conv2 conv1
conv2 6250000/s ­­ ­0%
conv1 6250000/s 0% ­­
92
30 agosto 2007

93. testing optimization
(3. is it faster?)
Benchmark compares on a micro level, running
●
thousands (or millions) of times
Now test on macro (single script)
●
Redo dprofpp, compare results
–
(Or even save the old tmon.out)
–
93
30 agosto 2007

94. Grazie
Hakim Cassimally
Senior Dev, dada.net
hakim.cassimally@staff.dada.net
...

95. Grazie
Web Developer @ Dada Group
Dada Group is seeking Perl and/or PHP Developers
to join our passionate & international Production Team.
Multiple locations are available in Italy, Spain, the United States, Brazil & China,
for on-site work.
Your profile:
you have passion for the Internet, new technologies and systems integration.
Moreover you have extensive experience developing applications for the web,
especially in the consumer entertainment (web & mobile) field.
If you think you are up to the challenge, please send us your resume !!!
recruiting@staff.dada.net
Contact:
www.dada.net
Web-site: