Promise Keynote


Published on

Norman Fenton

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Joke about Savoy. Who I am – show my book. I’ll have something more to say about this book shortly. One of the best books on software metrics was written by Bob Grady of Hewlett Packard in 1987. Bob was responsible for what I believe was recognised as the first true company-wide metrics programme. And his book described the techniques and experiences associated with that at HP. A few years later I was at a meeting where Bob told an interesting story about that metrics programme. He said that one of the main objectives of the programme was to achieve process improvement by learning from metrics what process activities worked and what ones didn’t. To do this they looked at those projects that in metrics terms were considered most successful. These were the projects with especially low rates of customer-reported defects. The idea was to learn what processes characterised such successful projects. It turned out that what they learned from this was very different to what they had expected. I am not going to tell you what it was they learnt until the end of my presentation; by then you may have worked it out for yourselves.
  • Promise Keynote

    1. 1. New Directions for Software Metrics Norman Fenton Agena Ltd and Queen Mary University of London PROMISE 20 May 2007
    2. 2. Overview <ul><li>History of software metrics </li></ul><ul><li>Good and bad news </li></ul><ul><li>Hard project constraints </li></ul><ul><li>Project trade-offs </li></ul><ul><li>Decision-making and intervention </li></ul><ul><li>The true objective of software metrics? </li></ul><ul><li>Why we need a causal approach </li></ul><ul><li>Models in action </li></ul><ul><li>Results </li></ul>
    3. 3. Metrics History: Typical Approach What I really want to measure (Y) What I can measure (X) Y = f (X)
    4. 4. Metrics History: the drivers <ul><li>‘ productivity’=size/effort </li></ul><ul><li>‘ effort’=a*size b </li></ul><ul><li>‘ quality’=defects/size </li></ul>
    5. 5. Metrics history: size matters! <ul><li>LOC </li></ul><ul><li>Improved size metrics </li></ul><ul><li>Improved complexity metrics </li></ul>
    6. 6. Some Decent News About Metrics <ul><li>Empirical results/banchmarks </li></ul><ul><li>Significant industrial activity </li></ul><ul><li>Academic/research output </li></ul><ul><li>Metrics in programmer toolkits </li></ul>
    7. 7. ….But Now the Bad News <ul><li>Lack of commercial relevance </li></ul><ul><li>Programmes doomed by data </li></ul><ul><li>Activity poorly motivated </li></ul><ul><li>Failed to meet true objective of quantitative risk assessment </li></ul>
    8. 9. Regression models….?
    9. 10. Using metrics and fault data to predict quality 0 0 Post-release faults 10 20 30 40 80 120 160 Pre-release faults ?
    10. 11. Pre-release vs post-release faults: actual Post-release faults 0 10 20 30 0 40 80 120 160 Pre-release faults
    11. 12. What we need What I think is... ?
    12. 13. The Good News <ul><li>It is possible to use metrics to meet the real objective </li></ul><ul><li>Don’t need a heavyweight ‘metrics programme’ </li></ul><ul><li>A lot of the hard stuff has been done </li></ul>
    13. 14. A Causal Model (Bayesian net) Residual Defects Testing Effort Problem complexity Defects found and fixed Defects Introduced Design process quality Operational defects Operational usage
    14. 15. A Model in action
    15. 17.
    16. 20. A Model in action
    17. 23. A Model in action
    18. 29. Actual versus predicted defects
    19. 30. How did we build the models? <ul><li>Expert judgement </li></ul><ul><li>Published data </li></ul><ul><li>Company data </li></ul>
    20. 31. Conclusions <ul><li>Heavyweight data and classical statistics NOT the answer </li></ul><ul><li>Empirical studies laid groundwork </li></ul><ul><li>Causal models for quantitative risk </li></ul>
    21. 32. … And <ul><li>You can use the technology NOW </li></ul><ul><li> </li></ul>