A presentation I delivered in 2006 at The Royal Society for the Encouragement of Arts, Manufactures and Commerce (RSA), a London-based, British organisation committed to finding practical solutions to social challenges.

1. © LOCRIS Limited 2006© LOCRIS Limited 2006
Uses and Abuses of ITUses and Abuses of IT
BenchmarkingBenchmarking
Outsourcing and ProcessOutsourcing and Process
ImprovementImprovement
John L. Evans –John L. Evans – FCMI FIMC FIMIS MBCS CITP MID CIPSFCMI FIMC FIMIS MBCS CITP MID CIPS
Director – LOCRIS LimitedDirector – LOCRIS Limited

2. Software
Measurement and
2
© LOCRIS Limited 2006© LOCRIS Limited 2006
LOCRISLOCRIS
• DWPDWP
• Cap GeminiCap Gemini
• RS ComponentsRS Components
• Royal & SunAllianceRoyal & SunAlliance
• AXAAXA
• AT&TAT&T
• The Pru’The Pru’
• Wincor NixdorfWincor Nixdorf
• WoolworthsWoolworths
• Centrica / British GasCentrica / British Gas
• British Energy – PETBritish Energy – PET
• BTBT
• BABA
• Rover GroupRover Group
• BBCBBC
• Boots The ChemistBoots The Chemist
• LIFFELIFFE
• BNP ParibasBNP Paribas
• BSiBSi
• BG plc – TranscoBG plc – Transco
• SEMA GroupSEMA Group
• Legal Aid BoardLegal Aid Board
• Legal & GeneralLegal & General
• UK National LotteryUK National Lottery
• PA ConsultingPA Consulting
• United UtilitiesUnited Utilities
• GlaxoGlaxo
• INMARSATINMARSAT
• BACSBACS

3. Software
Measurement and
3
© LOCRIS Limited 2006© LOCRIS Limited 2006
Is there really value inIs there really value in
Outsourcing your issues?Outsourcing your issues?
• Outsourcing - a short cut to improvement?Outsourcing - a short cut to improvement?
• suppliers and their promisessuppliers and their promises
• customer's termscustomer's terms
• governance and objectivitygovernance and objectivity
• how will you ever knowhow will you ever know
• benchmark databenchmark data
• the ability to rapidly realise benefitsthe ability to rapidly realise benefits

4. Software
Measurement and
4
© LOCRIS Limited 2006© LOCRIS Limited 2006
"When you can measure what you"When you can measure what you
are speaking about, and express itare speaking about, and express it
in numbers, you know somethingin numbers, you know something
about it; but when you cannotabout it; but when you cannot
measure it, when you cannotmeasure it, when you cannot
express it in numbers, yourexpress it in numbers, your
knowledge is of a meagre andknowledge is of a meagre and
unsatisfactory kind; it may be theunsatisfactory kind; it may be the
beginning of knowledge, but youbeginning of knowledge, but you
have scarcely in your thoughtshave scarcely in your thoughts
advanced to the state of Science,advanced to the state of Science,
whatever the matter may be." -whatever the matter may be." -
Lord KelvinLord Kelvin

5. Software
Measurement and
5
© LOCRIS Limited 2006© LOCRIS Limited 2006
IT performance measuresIT performance measures
‘ineffective’‘ineffective’ John Riley 23 May 2006John Riley 23 May 2006
• current measures ineffectivecurrent measures ineffective
• fewer measuresfewer measures
• stop measuring cost and activitystop measuring cost and activity
• measure efficiency, output and outcomemeasure efficiency, output and outcome
• no value in “value-add” or ROIno value in “value-add” or ROI
• IT cost vs profit or turnoverIT cost vs profit or turnover
• cost reduction not a good indicatorcost reduction not a good indicator

6. Software
Measurement and
6
© LOCRIS Limited 2006© LOCRIS Limited 2006
IT performance measuresIT performance measures
‘ineffective’‘ineffective’ John Riley 23 May 2006John Riley 23 May 2006
• IT is (generally) not a revenue generatorIT is (generally) not a revenue generator
• IT provides things that can’t be measuredIT provides things that can’t be measured
• fewer, more effective measuresfewer, more effective measures
• don’t leave a vacuumdon’t leave a vacuum
• others will set measurements for youothers will set measurements for you
• communicate with peerscommunicate with peers
• link to organisational objectiveslink to organisational objectives
• be credible to the top teambe credible to the top team

7. Software
Measurement and
7
© LOCRIS Limited 2006© LOCRIS Limited 2006
Scenario 1Scenario 1
• Suppose:Suppose:
– 450 AMS / Development staff450 AMS / Development staff
– £25M / annum cost base£25M / annum cost base
– Demanding, rapidly changingDemanding, rapidly changing
organisation and customers (users)organisation and customers (users)
– Morbidity in the departmentMorbidity in the department
– No performance dataNo performance data

8. Software
Measurement and
8
© LOCRIS Limited 2006© LOCRIS Limited 2006
Scenario 2Scenario 2
• Suppose:Suppose:
– 450 AMS / Development staff450 AMS / Development staff
– £25M / annum cost base£25M / annum cost base
– Demanding, rapidly changingDemanding, rapidly changing
organisation and customers (users)organisation and customers (users)
– Morbidity in the departmentMorbidity in the department
– Existing performance data – functionExisting performance data – function
point analysis going back some yearspoint analysis going back some years

9. Software
Measurement and
9
© LOCRIS Limited 2006© LOCRIS Limited 2006
ConsiderConsider
• 2 scenarios2 scenarios
– Without performance dataWithout performance data
– With performance dataWith performance data

10. Software
Measurement and
10
© LOCRIS Limited 2006© LOCRIS Limited 2006
Once Upon a TimeOnce Upon a Time
• A companyA company
• With 5 years performance dataWith 5 years performance data
• UnhappyUnhappy
• Outsourced – because it didn’t knowOutsourced – because it didn’t know
how to improvehow to improve

11. Software
Measurement and
11
© LOCRIS Limited 2006© LOCRIS Limited 2006
The DreamThe Dream
• From day 1:From day 1:
– 25% improvement in year 125% improvement in year 1
– Progressing to upper quartileProgressing to upper quartile
– From a standing startFrom a standing start

12. Software
Measurement and
12
© LOCRIS Limited 2006© LOCRIS Limited 2006
The RealityThe Reality
• Productivity – 20% down in year 1Productivity – 20% down in year 1
• And year 2And year 2
• Par by year 3Par by year 3
• Improving by………Improving by………

13. Software
Measurement and
13
© LOCRIS Limited 2006© LOCRIS Limited 2006
Sadness AboundsSadness Abounds
• The QueenThe Queen
shouted at theshouted at the
KnightKnight
• The KnightThe Knight
Laughed at Queen!Laughed at Queen!
• The Knight had aThe Knight had a
ContractContract
• What could theWhat could the
Queen do?Queen do?

14. Software
Measurement and
14
© LOCRIS Limited 2006© LOCRIS Limited 2006
The Happy EndingThe Happy Ending
• With Performance Data enabled:With Performance Data enabled:
– measurement of the actual changemeasurement of the actual change
– measurement of the actual lossmeasurement of the actual loss
– prediction of the actual prognosisprediction of the actual prognosis
– Renegotiation of the contractRenegotiation of the contract
– achievement of the intended savings /achievement of the intended savings /
productivity improvementsproductivity improvements
– stay out of courtstay out of court

15. Software
Measurement and
15
© LOCRIS Limited 2006© LOCRIS Limited 2006
Enlightenment?Enlightenment?

16. Software
Measurement and
16
© LOCRIS Limited 2006© LOCRIS Limited 2006
MineseekerMineseeker
• It will take hundreds ofIt will take hundreds of
years and £B to clearyears and £B to clear
• 70 countries affected70 countries affected
• Man takes a day to surveyMan takes a day to survey
404022
mtrsmtrs
• Mineseeker airship take 1Mineseeker airship take 1
second to survey 100second to survey 10022
mtrsmtrs
• www.mineseeker.comwww.mineseeker.com