A fun short piece on the outcome of the BCS National Championship and a topic that should be of interest to everyone, New Product Development Portfolio Management.

1. VIEWPOINT
BCS Rankings and New Product
Development Scoring Systems
A computer does not adequately account for heart and character
BY BILL POSTON
So my Longhorns lost the Bowl Championship Series (BCS) National Championship
game to a team named after an algae bloom. Not that I am bitter, but I believe
that Sandra Bullock deserves to win Best Actress for pretending to be all hot and
bothered when meeting Nick Saban in The Blind Side. I don’t like that guy and
neither does she.
While I am obviously disappointed with the outcome, the BCS pairings were
mostly free from controversy this year. The convoluted system that has been
continuously tweaked since its inception in 1998 seemed to work. The complex
algorithm that underlies the ranking methodology uses a series of Borda counts
and combines the results of a sports writers poll, a coaches poll, and an average of The problem occurs
six different computer scoring systems to arrive at its overall rankings (Borda was when these algorithm-
an eighteenth century French mathematician who didn’t know anything about driven systems
college football). Football people don’t really trust computers – or the French – so become the de facto
the human polls are now weighted more heavily. But the only reason we have decision making tool.
computer scoring systems is because we were, once upon a time, not happy with
Even the BCS gives the
the consistency of the results of the human polls. We have the LSU version of Nick
Saban to thank for that. computer only one-
third of the vote. If
The benefit of the computer ranking systems is that they use objective criteria to Excel can tell us which
evaluate the performance of teams and that they are not subject to “reputation new product concepts
bias” or any other bad ol’ form of subjective selection by mere human beings. I we should invest in
think that having six different computer models account for a third of the input is then why do we need
about right for college football and they appear to be doing a decent job of Vice Presidents?
predicting success. No one really thought that Cincinnati was going to beat
Florida. Did they?
Predicting success is also the objective of scoring systems used by companies in
the new product development process. I have worked with dozens of clients that
have extremely sophisticated systems for scoring and ranking new product
concepts. These systems identify characteristics of new product projects that are
historically correlated with winners and use these measures as predictors of
success.

2. VIEWPOINT
I like these systems and believe that they are, if properly constructed, a valuable
input into the project selection and portfolio prioritization process. The problem
occurs when these algorithm-driven systems become the de facto decision making
tool. Even the BCS gives the computer only one-third of the vote. If Excel can tell
us which new product concepts we should invest in then why do we need Vice
Presidents?
When we are talking about innovation we need to first consider that we are
dealing with tremendous amounts of uncertainty. No matter how hard we try to
quantify variables and assign probabilities to projections, we are still dealing with Give the computer a
forecasts and estimates. Just because we put these numbers into a fancy voice but don’t let it
algorithm doesn’t make them true. We can get better over time by tweaking the
set your priorities.
algorithm based on demonstrated in-market success, but we will always have to
The best models out
deal with – and embrace – uncertainty.
there are not perfect
predictors of success
That is where management judgment comes into play. There is no good way to
automate project selection and prioritization. Scoring systems should be an input and cannot replace a
into decision makers that aggressively question the assumptions behind the model seasoned executive’s
and parse the variables that go into it. In my opinion, an informed executive with knowledge of, and feel
a good gut beats a BASS (big-assed spreadsheet) every day. Give the computer a for, the marketplace.
voice but don’t let it set your priorities. The best models out there are not perfect
predictors of success and cannot replace a seasoned executive’s knowledge of,
and feel for, the marketplace.
So let’s make sure that our scoring models are used properly and improved over
time. The BCS ranking system will continue to evolve and I’ll eventually get over
the Longhorns’ loss. The computer does not adequately account for heart and
character. If it did, Texas would be the clear #1. Hook ’em Horns!
KALYPSO CONTACT
Bill Poston, Managing Partner
bill.poston@kalypso.com
www.kalypso.com