Working Effectively with
 Legacy Perl Code
       Erik Rantapaa
      Frozen Perl 2010
What is Legacy Code?
What is Legacy Code?

Uses old/obsolete perl and modules
Not well factored / organically grown
Original developers gone
No...
What is Legacy Code?


Michael Feathers - "Code without tests"

Ward Cunningham - "Accumulated technical debt"

Stuart Hal...
Example Legacy App

 e-commerce application
 over 10 years old
 "rewritten" at least a couple of times
 worked on by lots ...
Overview

 Unit Testing from a Perl perspective
 Working with the Code Base
 Instrumentation
 Future Directions
The Case for Testing


"... With tests we can change our code quickly and verifiably.
Without them we really don't know if...
Cost of Fixing Defects
The Feedback Cycle
Testing vs. Debugging

Debugging:
  manual set-up (set breakpoints, step code, etc.)
  temporarily modifies code (printf d...
Your Application
A Unit Test
Unit Testing

   isolates a small piece of functionality
   eliminates dependencies on other components through
   mocking...
Impediments to Unit Testing

Main impediment: the way the application is glued together.

use LWP;
...
sub notify_user {
 ...
Strongly Coupled Concerns

sub emit_html {




}
Dependency Breaking Techniques
Adapt Parameter                Parameterize Constructor
Break Out Method Object        Para...
Sprouting

   replace a block of code with a subroutine/method call (even
   for new code)

Benefits:

   simple and safe ...
Sprouting Example - DBI calls

Any DBI call is a good candidate for sprouting.

DBI call = an action on a business concept...
Dependency Injection

# use LWP;
...
sub notify_user {
  my ($user, $message, $ua ) = @_;
  ...
  # my $ua = LWP::UserAgen...
Preserving Signatures

use LWP;
...
sub notify_user {
  my ($user, $message, $ua ) = @_;
  ...
   $ua ||= LWP::UserAgent->...
Perl-Specific Mocking/Isolation
         Techniques
Replacing a Package

 alter @INC to load alternate code

 alter %INC to omit unneeded code
Replacing Subroutines

Monkey patch the symbol table:

  no warnings 'redefine';
  *Product::saveToDatabase = sub { $_[0]-...
Modifying Object Instances

Re-bless to a subclass:

package MockTemplateEngine;
our @ISA = qw(RealTemplateEngine);
sub pr...
Use the Stack

Define behavior based on caller().

Use in conjunction with monkey patching subs.
Exploiting Perl's Dynamicism

  Use the dynamic capabilities of Perl to help you isolate code
  and create mock objects.
 ...
Manual Testing

 Not always bad
 Some times it's the best option:
    automated verification is difficult / impossible
   ...
Manual Testing Example
Using an Wrapper Layer
Real-World Experience

 Writing the first test is very difficult
 It gets easier!
 Biggest win: automated verification
 Ne...
More Real-World Experience

 Isolating code will help you understand the dependency
 structure of your application
 Let yo...
Working with the Code Base
Source Control

Get everthing under source control:
   perl itself
   all CPAN modules
   shared libraries
   Apache, mod_...
Reliable Builds

  Automate building and deployment
  Break dependencies on absolute paths, port numbers
  Enable multiple...
Logging

   make logs easy to access
   import logs into a database
   automate a daily synopsis

SELECT count(*), substr(...
Wrappers

Create deployment-specific wrappers for perl and perldoc:

#!/bin/sh
PATH=...
PERL5LIB=...
LD_LIBRARY_PATH=...
O...
Create Tools

$ compile-check Formatter.pm

Runs app-perl -cw -MApp::PaymentService::Email::Formatter

Full module name de...
Automated Testing

 Continuous Integration
 Smoke Tests
 Regression Tests
Instrumentation
Devel::NYTProf

Pros:
   full instrumentation of your code
   timing statistics on a per-line basis
   call-graphs, heat m...
Devel::Leak::Object

   Original purpose: find leakage of objects due to cyclic
   references

Modifications:

   Record h...
dtrace


Pros:
   instrument events across the entire system
   runs at nearly full speed - usable in production

Cons:
  ...
Future Directions

More static analysis
  manually added soft type assertions
  more help in editors/IDEs
  more refactori...
Future Directions

More dynamic instrumention
  usable in production
  low performance hit
  mainly interested in:
      t...
Future Directions

Automated Instrumentation
   build a database of the runtime behavior of your program
   collect design...
A Refactoring Problem

Want to change $self->{PRICE} to $self->getPrice:

package Product;

sub foo {
  my ($self, ...) = ...
A Refactoring Problem

Within package Product → easy!

package Product;

sub getPrice { ... }

sub foo {
  my ($self, ...)...
A Refactoring Problem

Can we change this?

package SomeOtherClass;

sub bar {
  ...
  ... $product->{PRICE} ...
}

Need t...
A Refactoring Problem

Look at context for help.

package SomeOtherClass;

sub bar {
  ...
  my $product = Product->new(.....
A Refactoring Problem

Now what?

package SomeOtherClass;

sub bar {
  my ($self, $product, ...) = @_;
  ...
  ... $produc...
References

 "Working Effectively with Legacy Code" - Michael Feathers
 "Clean Code Talks" - googletechtalks channel on Yo...
Thanks!

Feedback, suggestions, experiences?

                  erantapaa@gmail.com
Working Effectively With Legacy Perl Code
Upcoming SlideShare
Loading in...5
×

Working Effectively With Legacy Perl Code

1,623

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,623
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Working Effectively With Legacy Perl Code

  1. 1. Working Effectively with Legacy Perl Code Erik Rantapaa Frozen Perl 2010
  2. 2. What is Legacy Code?
  3. 3. What is Legacy Code? Uses old/obsolete perl and modules Not well factored / organically grown Original developers gone No one fully understands it Uses outdated practices Hard to modify without breaking it Long new developer ramp-up time No documentation (or wrong documentation) No tests On the other hand... Doing useful work - generating revenue
  4. 4. What is Legacy Code? Michael Feathers - "Code without tests" Ward Cunningham - "Accumulated technical debt" Stuart Halloway - "Legacy is the degree to which code: - fails to capture intent - fails to communicate intent - captures irrelevant detail (ceremony)"
  5. 5. Example Legacy App e-commerce application over 10 years old "rewritten" at least a couple of times worked on by lots of different developers > 1000 .pm files > 150 database tables, >2000 columns 3 templating systems, 100's of template files used perl 5.8.0 until recently lots of old versions of CPAN modules (some customized) didn't compile with -cw until recently rm -rf not an option
  6. 6. Overview Unit Testing from a Perl perspective Working with the Code Base Instrumentation Future Directions
  7. 7. The Case for Testing "... With tests we can change our code quickly and verifiably. Without them we really don't know if our code is getting better or worse. ..." - Michael Feathers
  8. 8. Cost of Fixing Defects
  9. 9. The Feedback Cycle
  10. 10. Testing vs. Debugging Debugging: manual set-up (set breakpoints, step code, etc.) temporarily modifies code (printf debugging) manual verification pay for it every time Testing: no source code modification automated set-up and verification pay once when you create the test reap the rewards every time you run it
  11. 11. Your Application
  12. 12. A Unit Test
  13. 13. Unit Testing isolates a small piece of functionality eliminates dependencies on other components through mocking / substitution ideally runs quickly provides automated verification Benefits: safety net during refactoring after refactoring becomes a regression test speeds up the Edit-Compile-Run-Debug cycle
  14. 14. Impediments to Unit Testing Main impediment: the way the application is glued together. use LWP; ... sub notify_user { my ($user, $message) = @_; ... my $ua = LWP::UserAgent->new; ... $ua->request(...); }
  15. 15. Strongly Coupled Concerns sub emit_html { }
  16. 16. Dependency Breaking Techniques Adapt Parameter Parameterize Constructor Break Out Method Object Parameterize Method Definition Completion Primitive Parameter Encapsulate Global Pull Up Feature References Push Down Dependency Extract and Override Call Replace Function with Extract and Override Factory Function Pointer Method Replace Global Reference Extract and Override Getter with Getter Extract Implementer Subclass and Override Extract Interface Method Introduce Instance Delegator Supersede Instance Variable Introduce Static Setter Template Redefinition Link Substitution Text Redefinition ...
  17. 17. Sprouting replace a block of code with a subroutine/method call (even for new code) Benefits: simple and safe code transformation code block can be run independently permits the code to be redefined
  18. 18. Sprouting Example - DBI calls Any DBI call is a good candidate for sprouting. DBI call = an action on a business concept Sprouting permits: testing of SQL syntax testing of query operation removal of interaction with the database
  19. 19. Dependency Injection # use LWP; ... sub notify_user { my ($user, $message, $ua ) = @_; ... # my $ua = LWP::UserAgent->new; ... $ua->request(...); } "Ask for - Don't create"
  20. 20. Preserving Signatures use LWP; ... sub notify_user { my ($user, $message, $ua ) = @_; ... $ua ||= LWP::UserAgent->new; ... $ua->request(...); }
  21. 21. Perl-Specific Mocking/Isolation Techniques
  22. 22. Replacing a Package alter @INC to load alternate code alter %INC to omit unneeded code
  23. 23. Replacing Subroutines Monkey patch the symbol table: no warnings 'redefine'; *Product::saveToDatabase = sub { $_[0]->{saved} = 1 }; *CORE::GLOBAL::time = sub { ... } Create accessors for package lexicals
  24. 24. Modifying Object Instances Re-bless to a subclass: package MockTemplateEngine; our @ISA = qw(RealTemplateEngine); sub process { ... new behavior ... } ... $template = RealTemplateEngine->new(...); bless $template, 'MockTemplateEngine';
  25. 25. Use the Stack Define behavior based on caller(). Use in conjunction with monkey patching subs.
  26. 26. Exploiting Perl's Dynamicism Use the dynamic capabilities of Perl to help you isolate code and create mock objects. Not the cleanest techniques, but they can greatly simplify getting dependency-laden code under test.
  27. 27. Manual Testing Not always bad Some times it's the best option: automated verification is difficult / impossible automated test to costly to create
  28. 28. Manual Testing Example
  29. 29. Using an Wrapper Layer
  30. 30. Real-World Experience Writing the first test is very difficult It gets easier! Biggest win: automated verification Next biggest win: efficient tests Manual testing is ok (save what you did!)
  31. 31. More Real-World Experience Isolating code will help you understand the dependency structure of your application Let your tests guide refactoring Testable code ~ clean code ~ modular code
  32. 32. Working with the Code Base
  33. 33. Source Control Get everthing under source control: perl itself all CPAN modules shared libraries Apache, mod_perl, etc.
  34. 34. Reliable Builds Automate building and deployment Break dependencies on absolute paths, port numbers Enable multiple deployments to co-exist on the same machine
  35. 35. Logging make logs easy to access import logs into a database automate a daily synopsis SELECT count(*), substr(message, 0, 50) as "key" FROM log_message WHERE log_time BETWEEN ... AND ... GROUP BY key ORDER BY count(*) DESC;
  36. 36. Wrappers Create deployment-specific wrappers for perl and perldoc: #!/bin/sh PATH=... PERL5LIB=... LD_LIBRARY_PATH=... ORACLE_HOME=... ... /path/to/app/perl "$@"
  37. 37. Create Tools $ compile-check Formatter.pm Runs app-perl -cw -MApp::PaymentService::Email::Formatter Full module name determined from current working directory.
  38. 38. Automated Testing Continuous Integration Smoke Tests Regression Tests
  39. 39. Instrumentation
  40. 40. Devel::NYTProf Pros: full instrumentation of your code timing statistics on a per-line basis call-graphs, heat map useful for Cons: big performance hit (not suitable for production) you need to drive it Also see Aspect::Library::NYTProf for targeted profiling
  41. 41. Devel::Leak::Object Original purpose: find leakage of objects due to cyclic references Modifications: Record how (the stack) objects get created Count the number of each type of object created Determine where objects of a certain class are created Robust facility to instrument bless and DESTROY.
  42. 42. dtrace Pros: instrument events across the entire system runs at nearly full speed - usable in production Cons: only available for Mac OS and Solaris perl probes still under development
  43. 43. Future Directions More static analysis manually added soft type assertions more help in editors/IDEs more refactoring tools better compilation diagnostics catch mistyped subroutine names catch errors in calling methods/subs catch other type errors
  44. 44. Future Directions More dynamic instrumention usable in production low performance hit mainly interested in: the call chain (stack) types of variables / subroutine parameters / return values dtrace? modified perl interpreter?
  45. 45. Future Directions Automated Instrumentation build a database of the runtime behavior of your program collect design details not expressed in the source mine the database to infer internal rules: call graphs signature of subroutines use rules to aid static analysis
  46. 46. A Refactoring Problem Want to change $self->{PRICE} to $self->getPrice: package Product; sub foo { my ($self, ...) = @_; ... ... $self->{PRICE} ... ... }
  47. 47. A Refactoring Problem Within package Product → easy! package Product; sub getPrice { ... } sub foo { my ($self, ...) = @_; ... ... $self->getPrice ... # know type of $self ... }
  48. 48. A Refactoring Problem Can we change this? package SomeOtherClass; sub bar { ... ... $product->{PRICE} ... } Need to know if $product is from package Product (or a subclass)
  49. 49. A Refactoring Problem Look at context for help. package SomeOtherClass; sub bar { ... my $product = Product->new(...); ... ... $product->{PRICE} ... }
  50. 50. A Refactoring Problem Now what? package SomeOtherClass; sub bar { my ($self, $product, ...) = @_; ... ... $product->{PRICE} ... } Need to figure out where SomeOtherClass::bar() gets called. Instrumentation DB → just look up the answer
  51. 51. References "Working Effectively with Legacy Code" - Michael Feathers "Clean Code Talks" - googletechtalks channel on YouTube Writing Testable Code - Misko Hevery http://misko.hevery. com/code_reviewers_guide "Clean Code: A Handbook of Agile Software Craftsmanship," Robert C. Martin, ed.
  52. 52. Thanks! Feedback, suggestions, experiences? erantapaa@gmail.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×