Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
We Know Less Than You Think (But We Do Know Something) Copyright © 2009 Gregory V. Wilson. This presentation may be freely...
Computer “Science” (?) <ul><li>Some of computer science is applied mathematics </li></ul><ul><li>But what kind of “science...
Show Me <ul><li>Shaw 1990: “Prospects for a Discipline of Software Engineering” </li></ul><ul><ul><li>Called for evidence-...
There’s a Lot of Folklore… <ul><li>“ The best programmers are up to 28 times more productive than the worst.” (or 40, or 1...
Non-Linearity <ul><li>[WOO79]: 25% increase in problem complexity    100% increase in solution complexity </li></ul><ul><...
Glass’s Law <ul><li>Any new tool or technique initially makes you less productive </li></ul><ul><ul><li>Which is why so ma...
Project Failure <ul><li>#1 cause: poor estimation </li></ul><ul><ul><li>Short iterations cure this in the small, but don’t...
Boehm’s Curve <ul><li>[BOE01] and many others </li></ul>http://www.third-bit.com 00111
Re-Use <ul><li>“ It takes three times longer to build a reusable component than it does to build something used only once....
Lines of Code <ul><li>“ Productivity and reliability depend on the length of a program’s text, independent of language lev...
<ul><li>“ Maintenance makes up 40-80% of the total cost of a software project.” </li></ul><ul><ul><li>[BOE75] and many mor...
 
 
 
 
 
 
 
<ul><li>Lack of documentation was the #1 complaint for over 40% of respondents, and in the top three for almost 80% </li><...
<ul><li>Emphasis on grids and GPUs is inappropriate and unproductive </li></ul><ul><ul><li>Improving skills will yield gre...
References <ul><li>[BOE75] Boehm: “The High Cost of Software” </li></ul><ul><li>[COL95] Cole: “Runaway Projects—Causes and...
Recommended Reading <ul><li>Doar Practical Development Environments </li></ul><ul><li>Feathers Working Effectively with Le...
Upcoming SlideShare
Loading in …5
×

We Know Less Than You Think (But We Do Know Something)

4,585 views

Published on

A quick summary of empirical software engineering, followed by early results from a survey of how over 1900 scientists use computers in their research.

Published in: Technology
  • Be the first to comment

We Know Less Than You Think (But We Do Know Something)

  1. 1. We Know Less Than You Think (But We Do Know Something) Copyright © 2009 Gregory V. Wilson. This presentation may be freely used and distributed with attribution. Prof. Greg Wilson Dept. of Computer Science University of Toronto [email_address]
  2. 2. Computer “Science” (?) <ul><li>Some of computer science is applied mathematics </li></ul><ul><li>But what kind of “science” is software engineering? </li></ul><ul><ul><li>Most of it isn’t (yet?) as rigorous as bridge design </li></ul></ul><ul><ul><li>Practitioners would object to being called alchemists, organizational psychologists, or cheerleaders </li></ul></ul>http://www.third-bit.com 00001
  3. 3. Show Me <ul><li>Shaw 1990: “Prospects for a Discipline of Software Engineering” </li></ul><ul><ul><li>Called for evidence-based software engineering </li></ul></ul><ul><ul><li>Built on empirical work since the 1970s by Boehm, Basili, Glass, and others </li></ul></ul><ul><li>Empirical studies now expected by top journals and conferences </li></ul>http://www.third-bit.com 00010
  4. 4. There’s a Lot of Folklore… <ul><li>“ The best programmers are up to 28 times more productive than the worst.” (or 40, or 100, or…) </li></ul><ul><ul><li>Trace it back: [SAC68] had 12 (self-taught) subjects </li></ul></ul><ul><li>[BOE75]: best to worst is a factor of 5 </li></ul><ul><ul><li>Borne out by other studies </li></ul></ul><ul><ul><li>Consistent with other fields </li></ul></ul>http://www.third-bit.com 00011
  5. 5. Non-Linearity <ul><li>[WOO79]: 25% increase in problem complexity  100% increase in solution complexity </li></ul><ul><ul><li>Interactions grow as N 2 </li></ul></ul><ul><li>This can work in your favor </li></ul><ul><ul><li>Cut 1/4 of your features, problem because 1/2 as hard to solve </li></ul></ul>http://www.third-bit.com 00100
  6. 6. Glass’s Law <ul><li>Any new tool or technique initially makes you less productive </li></ul><ul><ul><li>Which is why so many senior students program with Notepad </li></ul></ul><ul><li>[GLA99]: 5-35% improvement is the best you can expect from changing tools </li></ul><ul><ul><li>Ah, but techniques… </li></ul></ul>http://www.third-bit.com 00101
  7. 7. Project Failure <ul><li>#1 cause: poor estimation </li></ul><ul><ul><li>Short iterations cure this in the small, but don’t help in the large </li></ul></ul><ul><li>#2 cause: unstable requirements </li></ul><ul><ul><li>But research code often doesn’t have “requirements” </li></ul></ul><ul><li>See [VGE91], [COL95], [HOF01] </li></ul>http://www.third-bit.com 00110
  8. 8. Boehm’s Curve <ul><li>[BOE01] and many others </li></ul>http://www.third-bit.com 00111
  9. 9. Re-Use <ul><li>“ It takes three times longer to build a reusable component than it does to build something used only once.” </li></ul><ul><li>“ If more than 25% of a component has to be revised, it’s better to rewrite it from scratch.” </li></ul><ul><li>See e.g. [THO97] </li></ul>http://www.third-bit.com 01000
  10. 10. Lines of Code <ul><li>“ Productivity and reliability depend on the length of a program’s text, independent of language level.” </li></ul><ul><li>See [PRE00] for a recent experiment </li></ul><ul><li>However, machine-independent code has machine-independent performance </li></ul>http://www.third-bit.com 01001
  11. 11. <ul><li>“ Maintenance makes up 40-80% of the total cost of a software project.” </li></ul><ul><ul><li>[BOE75] and many more since </li></ul></ul><ul><li>But roughly 60% of “maintenance” is enhancement </li></ul><ul><li>And 30% of maintenance time is spent figuring out things work </li></ul>Maintenance http://www.third-bit.com 01010
  12. 19. <ul><li>Lack of documentation was the #1 complaint for over 40% of respondents, and in the top three for almost 80% </li></ul><ul><li>… but ranked second to last in list of where respondents spend their time </li></ul>What Hurts? http://www.third-bit.com 01010
  13. 20. <ul><li>Emphasis on grids and GPUs is inappropriate and unproductive </li></ul><ul><ul><li>Improving skills will yield greater dividends on shorter timescales </li></ul></ul><ul><li>swc.scipy.org for current course </li></ul><ul><ul><li>software-carpentry.org for the next one </li></ul></ul><ul><ul><li>We’ll be webcasting July 13-31 </li></ul></ul>Thoughts http://www.third-bit.com 01011
  14. 21. References <ul><li>[BOE75] Boehm: “The High Cost of Software” </li></ul><ul><li>[COL95] Cole: “Runaway Projects—Causes and Effects” </li></ul><ul><li>[GLA99] Glass: “The Realities of Software Technology Payoffs” </li></ul><ul><li>[HOF01] Hofmann and Lehner: “Requirements Engineering as a Success Factor in Software Projects” </li></ul><ul><li>[PRE00] Prechelt: “An Empirical Comparison of Seven Programming Languages” </li></ul><ul><li>[SAC68] Sackman et al: “Exploratory Experimental Studies Comparing Online and Offline Programming Performances” </li></ul><ul><li>[THO97] Thomas et al: “An Analysis of Errors in a Reuse-Oriented Development Environment” </li></ul><ul><li>[VGE91] Van Genuchten: “Why Is Software Late?” </li></ul><ul><li>[WOO79] Woodfield: “An Experiment on Unit Increase in Problem Complexity” </li></ul>http://www.third-bit.com 01100
  15. 22. Recommended Reading <ul><li>Doar Practical Development Environments </li></ul><ul><li>Feathers Working Effectively with Legacy Code </li></ul><ul><li>Fogel Producing Open Source Software </li></ul><ul><li>Ford The Productive Programmer </li></ul><ul><li>Glass Facts and Fallacies of Software Engineering </li></ul><ul><li>Johnson GUI Bloopers </li></ul><ul><li>Kernighan almost anything </li></ul><ul><li>Kniberg Scrum and XP From the Trenches </li></ul><ul><li>Mak The Martian Principles </li></ul><ul><li>Nygard Release It! </li></ul><ul><li>Spinellis Code Reading and Code Quality </li></ul><ul><li>Weiner Time, Love, Memory </li></ul>01101 http://www.third-bit.com

×