Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Empirical, Evidence-Based Software Engineering

2,652 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Empirical, Evidence-Based Software Engineering

  1. 1. Cerezo
  2. 2. SH1998 H2009 S2010 SH1998CHEEP
  3. 3. N2008 Z2008 B2009On Architectural Misalignment
  4. 4. T2006] Z2006 PI
  5. 5. Global Software TeamsCheaper… but better? How?
  6. 6. Configuring Global Software Teams
  7. 7. C2008 J2003 C2008 MW C2010 H1999 E2007 GH1999 CH2010 S2004
  8. 8. J2007 MJ2003 JS K06 JM2006JC
  9. 9. Experiments
  10. 10. HJT2011 T2005J2004 JG2008 MJ2005
  11. 11. Languages & DSLsThe Sapir-Whorf Hypothesis
  12. 12. DrDobbs Language Hours per Function Point ASP 6.1Comparing Syntax Across Languages Visual Basic 8.5 Java 10.6 SQL 10.8 C++ 12.4 C 13 PL/1 15.5 COBOL 16.8 ABAP 19.9
  13. 13. Complexity MetricsThe measures you can’t reason about, control you
  14. 14. N2005
  15. 15. E2000 N2006E2001
  16. 16. N2005 G2000 K1996
  17. 17. DebuggingPrediction, prevention and removal
  18. 18. What Have We Learned About Fighting Defects O2002
  19. 19. Defect Defect Potentials Removal Efficiency Delivered Effects origins High-Quality Average Low High-Quality Average Low High-Quality Average Low Requirements 0,4 1 1,5 85% 77% 50% 0,08 0,23 0,75 Design 0,6 1,25 2,2 97% 85% 50% 0,02 0,19 1,1 Coding 1 1,75 2,5 99% 95% 80% 0,01 0,09 0,5 Documents 0,4 0,6 1 98% 80% 70% 0,01 0,12 0,3 Bad fixes 1 0,4 0,8 95% 70% 50% 0,01 0,12 0,4 Total 2,5 5 8 96% 85% 62% 0,13 0,75 3,05 System Commercial Informatio Military Outsource Web Embedded SEI-CMM 3 SEI-CMM1 Stat Overall Software Software n Software Software Software Software Software Software SoftwareDefectPotentials 6 5 4,5 7 5,2 4 5,5 5 5,75 5,1Defect RemovalEfficiency 94% 90% 73% 96% 92% 72% 95% 95% 83% 86%DeliveredDefects 0,36 0,5 1,22 0,28 0,42 1,12 0,3 0,25 0,9 0,68First YearDiscovery Rate 65% 70% 30% 75% 60% 95% 90% 60% 35% 64%First YearReportedRecord 0,23 0,35 0,36 0,21 0,25 1,06 0,25 0,15 0,34 0,42 SPR
  20. 20. Defect 1st Year 1st Year Defect Delivered Size Removal Discovery Reported Potential Defects Efficienct Rate Defects 1 1,85 95% 0,09 90% 0,08 10 2,45 92% 0,2 80% 0,16 100 3,68 90% 0,37 70% 0,26 1000 5 85% 0,75 50% 0,38 10000 7,6 78% 1,67 40% 0,67 100000 9,55 75% 2,39 30% 0,72 Average 5,02 85% 0,91 60% 0,38 Total Defect Method Removal Repairs Efficiency Removed Delivered Number Percent Costs RemovalInspections + Projects Projects Efficiencystatic analysis +testing $304.434 $6.289 $310.723 99% 990 10 >99 6 0%Inspections + 95-99 104 7%testing $321.590 $13.932 $335.522 96% 940 40 90-95 263 18%Static analysis + 85-90 559 37%testing $276.817 $41.796 $318.613 93% 925 75 80-85 408 27%Testing Only $326.089 $92.879 $418.968 85% 850 150 <80 161 11% COMPARISON OF DEFECT REMOVAL BY METHOD Total 1500 100% SPR
  21. 21. Defect 1st Year 1st Year Defect Delivered Size Removal Discovery Reported Potential Defects Efficienct Rate Defects 1 1,85 95% 0,09 90% 0,08 10 2,45 92% 0,2 80% 0,16 100 3,68 90% 0,37 70% 0,26 1000 5 85% 0,75 50% 0,38 10000 7,6 78% 1,67 40% 0,67 100000 9,55 75% 2,39 30% 0,72 Average 5,02 85% 0,91 60% 0,38 Software Defect Removal Delivered methods Potential Efficiency Defects Defect Removal DeliveredSEI CMMI Waterfall 5,5 80% 1,1 Potentials Efficiency Defects Iterative 4,75 87% 0,62CMMI 1 5,25 80% 1,05 Object-Oriented 4,5 88% 0,54CMMI 2 5 85% 0,75 Agile with Scrum 4 90% 0,4CMMI 3 4,75 90% 0,48 RUP 4,25 94% 0,26CMMI 4 4,5 93% 0,32 PSP and TSP 3,5 96% 0,14CMMI 5 4,25 96% 0,17 Model-based 3 98% 0,06 85% cert reuse 1,75 99% 0,02 Software Defect Removal Delivered Defect Removal Delivered methods Potential Efficiency DefectsSEI CMMI Potentials Efficiency Defects Waterfall 7 75% 1,75CMMI 1 6,5 75% 1,63 Iterative 6,25 82% 1,13CMMI 2 6,25 82% 1,13 Object-Oriented 5,75 85% 0,86CMMI 3 5,5 87% 0,71 Agile with Scrum 5,5 87% 0,72 RUP 5,5 90% 0,55CMMI 4 5,25 90% 0,53 PSP and TSP 5 94% 0,3CMMI 5 4,75 94% 0,29 Model-based 4 96% 0,15 85% cert reuse 2,25 96% 0,09 SPR
  22. 22. tlogaEclipse IDE, also see [L2010]
  23. 23. @Google R0211GitHub Impl NT2007 S2006 F1999 W2008 C2008 W2010
  24. 24. C2006 C2006 V1993M2002
  25. 25. Comparing Operating Systems (Linux,FreeBSD, Solaris & WindowsComparing sucessful software projects(Linux/GCC/Apache) to closed onesComparing metrics of 100 open sourceprograms to metrics of closed programsOpen Source as Lead User’s Make or BuyDecision
  26. 26. M2009
  27. 27. ArchitectureArchitecture is a means, not a end in itself
  28. 28. FBB2002
  29. 29. W2008 P2002 L2001 P2001 MWDEM
  30. 30. B2007C2005 U2010 R2009
  31. 31. Technical DebtThe Coccyx, Appendix and Dens Sapientiae of software:vestiges of code evolution
  32. 32. Empirical Report on Technical Debtacross Languages and Applications
  33. 33. Agile MethodologiesRun slow to get fast… or go fast to get slow
  34. 34. W2002
  35. 35. Systematic Review on Agile Software Practices EG
  36. 36. H2009S2008 E2004 E2005 H2005
  37. 37. W2003] J2006 Type BETTER WORSE INC TotalType BETTER WORSE INC Total NO- NO- DIFF DIFF Controlled 1 0 3 4Controlled 0 2 3 5 experimentexperiment Pilot study 5 1 1 7Pilot study 5 1 2 8Industrial 0 0 1 1 Industrial 0 1 0 1use useTotal 5 3 6 14 Total 6 2 4 12 TDD reduces the number of defects TDD improves test quality
  38. 38. EC2010 S2008 S2009 D2008C2012
  39. 39. RefactoringDoes it really add any value?
  40. 40. RW2011W2007
  41. 41. SoftwareDevelopment as Investment
  42. 42. Development as Investment Break-even saving/investment ratio for survival rates (black 90%, red 80%, blue 70% and green 60%) and development/maintenance ratios; survival rates 5.5, 6, 6.5, 7 and 7.5 years
  43. 43. Interaction of Development with Markets
  44. 44. Journal on Empirical Software EngineeringInternational Symposium on Empirical Software Engineering andMeasurementInternational Conference on Mining Software RepositoriesInternational Conference on Automated Software EngineeringIEEE Transactions on Software EngineeringInternational Conference on Software EngineeringInternational Symposium on the Foundations of Software EngineeringInternational Symposium on Software Testing & AnalysisInternational Conference on Global Software EngineeringWorking Conference on Reverse EngineeringWorkshop on Program Analysis for Software Tools and EngineeringInternational Symposium on Software Reliability EngineeringInternational Conference on Software MaintenanceJournal of Software Maintenance and EvolutionInternational Symposium on Engineering Secure Software and SystemsProceedings of the Agile Development ConferenceEvaluation and Assessment in Software EngineeringJournal of Systems and SoftwareEmpirical Software Engineering Group

×