Rescuing a-legacy-codebase

RESCUING LEGACY CODE
CURTIS “OVID” POE
CTO, ALL AROUND THE WORLD
OVID@ALLAROUNDTHEWORLD.FR
https://www.slideshare.net/Ovid/rescuing-alegacycodebase

ALL AROUND THE WORLD
CTO, ALL AROUND THE WORLD

QUESTION POLICY
CTO

SUBDUING MANAGERS
mardi 3 avril 2018©2018 ovid@allaroundtheworld.fr

TYPICAL LARGE TEST SUITE
• Hour or more to run
• Tests often fail
• Hard to see errors
• Developers don’t run them
• Or only run “some” tests
• Or check Facebook while running tests

IRONY
Large test suites can
lead to poor quality
code!

TYPICAL LARGE TEST SUITE
With apologies to XKCD: https://xkcd.com/303/

WHAT THE DEVELOPER SAID
“Yeah, the test suite is too slow and it’s hurting quality. I
want to fix this by running most tests in a single process.
It’s going to take me two to four weeks of not adding
new features, but we’ll make up for it in increased
developer productivity and a higher quality application.”

WHAT THE MANAGER HEARD
“Yeah, the test suite is too slow and it’s hurting quality. I
want to fix this by running most tests in a single process.
It’s going to take me two to four weeks of not adding
new features, but we’ll make up for it in increased
developer productivity and a higher quality application.”

Public domain. Erin Whittaker, U.S. National Park Service - Grand Canyon National Park Facebook page
Manager
s
Developers

WHY DON’T MANAGERS LISTEN?
• Managers trust other managers
• Developers are always asking to change things
• Managers focus on business needs
• Developers focus on technical needs
• Risk profile is different
• Developers like fun tasks

HOW DO WE GET PAST THIS?

THEORY
• CEO and others create goals
• Upper management creates a strategy
• Middle management develops tactics
• Developers implement those tactics

REALITY
• Everybody makes shit up uses their intuition

THE BUSINESS CASE
The “lite” version

FIRST QUESTIONS
1. What problem are you trying to solve?
2. Why are you trying to solve it?
3. What are your obstacles?
4. What is your solution?*
5. Why is it better than the alternatives?

THE SOLUTION
• What are the direct costs?
• What are the opportunity costs?
• What are the risks? (low, medium, high)
• What are the rewards? (low, medium, high)

TIPS
• Be honest
• Avoid loaded language
• Focus on business needs
• Catalyst vs. Mojolicious

CATALYST VERSUS
MOJOLICIOUS RISKS
Software appliances

CATALYST
• No explicit guarantees in the documentation
• Strong, proven track record of backwards-
compatibility
• Deemed “low risk”

MOJOLICIOUS
« Features may only be changed in a
major release, to fix a serious security
issue, or after being deprecated for at
least 3 months. »
http://mojolicious.org/perldoc/Mojolicious/Guides/Contributing

MOJOLICIOUS
« Only the two most recent stable
release series of Perl are supported. »
http://mojolicious.org/perldoc/Mojolicious/Guides/FAQ

MOJOLICIOUS
« We will also keep the distribution
installable up to a certain legacy
version that we deem worthy of
supporting, but not specifically optimize
for it, this is currently 5.10.1. »
http://mojolicious.org/perldoc/Mojolicious/Guides/FAQ

MOJO BREAKING CHANGES
• May 25: 4.01 release announcing the addition of a new reserved stash value in 3 months.
• June 3: 4.05 release announcing that all render methods will now return the invocant
instead of true/false.
• June 11: 4.09 release announcing that respond_to content negotiation will now honor other
previously ignored headers.
• …
• August 25: 4.21 release with a breaking change.
• September 3: 4.32 release with another breaking change.
• September 11: 4.33 release with yet another breaking change.
“We go from breaking changes once a year to breaking changes every week.” – Sebastian Riedel
https://groups.google.com/forum/#!topic/mojolicious/WnjvVC0bYjY

MOJOLICOUS
• Few explicit guarantees in the documentation
• Documentation suggests no commitment to backcompat
• History of developer complaints about breaking changes
• Deemed “medium to high risk”
This is not a criticism of Mojolicious!

RISK
CC BY-SA 3.0 https://commons.wikimedia.org/wiki/User:Guma89

BUSINESS CASE—TEST SUITES
PROS
• Easier to run tests
• Easier to find tests
• Fewer regressions =
lower maintenance
costs
• Increased productivity
= lower development
costs
CONS
• X? weeks of lost time
• Hard to get started

LOWER DEVELOPMENT COSTS
• 1 hour 20 minutes ⇒ 15 minutes
• Extra hour of developer productivity per day
• Equivalent of hiring having an extra developer
• Pays for itself in a month

REWRITE OR RESTRUCTURE?
Pros and Cons

OR THE THIRD OPTION …
• Rewriting
• Refactoring
• Reimagining

ROBERT X. CRINGELY SAYS …
“Cleaning up code” is a terrible thing. Redesigning
WORKING code into different WORKING code (also
known as refactoring) is terrible. The reason is that once
you touch WORKING code, it becomes NON-WORKING
code, and the changes you make (once you get it
working again) will never be known. It is basically a
programmer’s ego trip and nothing else. Cleaning up
code, which generally does not occur in nature, is a
prime example of amateur Open Source software.
https://web.archive.org/web/20121113182409/http://www.pbs.org/cringely/pulpit/2003/pulpit_20030501_000770.html

RESTRUCTURING …
Refactor [ree-fak-ter]
Noun:
The process of making a series of gradual
code changes without behavioral changes with
the goal of improving quality lowering
maintenance
costs.

REWRITE
PROS
• Cleaner architecture
• Easier to maintain
• More developers
• Make the impossible
possible
CONS
• Paying for two projects
• Extremely high risk
• Lost business
knowledge
• Lost technical
knowledge
• Larger scope of work

SPAGHETTI V. SPAGHETTI
PROS
• Cleaner architecture
• Easier to maintain
• More developers
• Make the impossible
possible
CONS
• Maintaining two
projects
• Extremely high risk
• Lost business
knowledge
• Lost technical
knowledge
• Larger scope of work

RESTRUCTURING
PROS
• Business knowledge
kept
• Technical knowledge
kept
• Only maintain one
system
• Smaller scope of work
CONS
• Technology lock-in
• “Impossible” needs
• Getting buy-in

GETTING BUY-IN
Business
needs
Technical
needs
Business
case

WOULD YOU PITCH THIS?
In a landmark 1995 study, the Standish Group
established that only about 17% of IT projects could be
considered “fully successful,” another 52% were
“challenged” (they didn't meet budget, quality or time
goals) and 30% were “impaired or failed.” In a recent
update of that study conducted for ComputerWorld,
Standish examined 3,555 IT projects between 2003 and
2012 that had labor costs of at least $10 million and
found that only 6.4% of [IT projects] were successful.
https://www.informationweek.com/strategic-cio/executive-insights-and-innovation/why-do-big-it-projects-fail-so-often/d/d-id/1112087

WHY REWRITES FAIL
• “But we’ve spent too much money to stop now!”
• Nobody wants to admit they’re wrong
• No capacity planning
• Unclear goals

BIGGEST REFACTORING
OBJECTION?

“WE DON’T KNOW HOW.”

THIS IS “HOW”.

WHAT’S THE PROBLEM?
• Too buggy?
• Too slow?
• Too unmaintainable?
• Something else?

OODA LOOP
• Observe
• Orient
• Decide
• Act

OODA LOOP
• Observe – Gather information
• Orient – Process information
• Decide – Take a decision
• Act – Implement that decision

OBSERVE: GATHER INFORMATION
• What are the functional requirements?
• What is the architecture?
• What documentation exists? (Danger!)
• What tests exists?
• What areas of the code are more fragile?
• What external resources does it require?
• …?

ORIENT: PROCESS INFORMATION
• Identify root cause of goal obstacles
• First facts, then opinions, then emotions
• Focus on root causes!

DECIDE: TAKE A DECISION
• Define success
• 50% test coverage ☹
• 50% monthly reduction in support calls ✓
• Decide your approach
• Set a timeframe

ACT: JUST DO IT!

DESIGNATED TECHNICAL EXPERT
• Understands business needs
• Understands system design/architecture
• Expert in primary language(s) of your codebase
• A strong automated testing background
• Very comfortable with code coverage tools
• A strong database background
• Ability to admit when they're wrong

ARCHITECTURE ROADMAP
• Determine the “ideal” architecture for your application
• Decide how to refactor towards this ideal
• Understand what must not change
• Integration tests against what must not change
• Pick a small initial target

PICKING YOUR TARGET
sub product {
my ($self, $customer_id, $product_id) = @_;
…
my $product = $self->dbh->selectall_arrayref(
“SELECT * FROM …”
);
…
print <<“END_HTML”;
<html><head>
…

PICKING YOUR TARGET
sub product ($self, $product_id) {
my $product = $self->model('Products')->fetch($product_id)
or $self->redirect('/status_not_found');
$self->stash( product => $product );
}

INITIAL TARGET
• SQL?
• HTML?
• Logic?

INITIAL TARGET
• Customer-facing?
• Backend?
• Smallest task/risk?

INTEGRATION TESTS
• Messy applications are hard to unit test
• Integration tests against the “view”
• Don’t stress about TDD

TESTS
• Restructuring should not change behavior
• Tests can help (not prevent) avoid changing behavior
• You probably need to rethink your testing strategy

TEST SUITE
• Use a test framework!
• Test database changes should be transparent
• High-level integration tests are high value
• Put fixtures in roles for easy sharing
• Tests must be discoverable
• Correctness first, then performance

CORRECTNESS EXAMPLE
• Same database as production
• Same version
• Same configuration

TEST FRAMEWORK
• Large websites use frameworks
• Large test suites should use frameworks
• Test::Class::Moose (TCM)

TCM BASE CLASS PSEUDO-CODE
package TestsFor::My::Project {
use Test::Class::Moose;
sub test_startup { begin transaction }
sub test_setup { begin savepoint }
sub test_teardown { rollback savepoint }
sub test_shutdown { rollback transaction }
}

WRITE TESTS!
sub test_vip_accounts_regenerate_stats_faster ( $test, $ ) {
my $winston = $test->character_winston;
ok !$winston->is_vip,
'Characters should not start as having a vip account';
my $yesterday = $test->now - period( days => 1 );
$winston->vip_until($yesterday);
ok !$winston->is_vip,
'... and vip_until in the past is not VIP';

PHASE TRANSITION
A change in a feature of a physical system that results in
a discrete transition of that system to another state.
Phase transitions often involve the absorption or
emission of energy from the system; ice, at 0 ° Celsius,
must absorb a considerable amount of heat energy to
become water.

ITERATING
1. Did the transition succeed?
2. What are our next steps towards our goal?
3. Plan next steps
4. Execute steps
5. Goto step1

ITERATING
• Separate restructuring and changing
• Remember your timeframe
• Remember your success criteria

PRIORITIZATION

PRIORITIZATION
• You already have buy-in
• But what to do next?
• Build mini ‘business cases’

A THREE-NUMBER BUSINESS CASE

THREE NUMBERS
• Complexity
• Monetization
• Utility

SCORING
score = (
(
(monetization * 40 + usability * 20)
-
(complexity * 40)
) + 500
)/10

BUILDING THE NUMBERS
• Automate spreadsheet creation
• Devs choose the complexity value
• Product owner chooses monetization and utility values
• Other factors may override!

NEXT STEPS
• Always focus on your measurable goal
• You can stop at any time
• You always have working code

QUESTIONS?

Rescuing a-legacy-codebase

More Related Content

Similar to Rescuing a-legacy-codebase

More from Curtis Poe

Recently uploaded

Rescuing a-legacy-codebase

Editor's Notes