0
Measuring & Analyzing           Things That Matter           When You Have Too          Many Things To Keep               ...
Thursday, February 23, 12
How does this happen?Thursday, February 23, 12
How do we fix it?                     • code reviews are hard                     • code reviews are time consuming        ...
information radiator                            “Coined around 2000 while standing in a Thoughtworks                      ...
“Chart what you care about, what you worry about,                       what you want other people to know.”              ...
http://www.exampler.com/old-blog/2005/04/02/Thursday, February 23, 12
1/2 Marathon TrainingThursday, February 23, 12
What Should We                              Measure?                     • Project progress                     • Code com...
Code “Quality”Thursday, February 23, 12
Rudolph FleschThursday, February 23, 12
Flesch-Kincaid Reading Ease                            Score                           Notes                            90...
competing algorithms                     • SMOG                     • Gunning-Fog             http://www.editcentral.com/g...
Maurice Halstead                             “Software Science”                            http://en.wikipedia.org/wiki/Ha...
Measure units of human                               thought used!!!      “Stroud’s (1955, 1967) original proposal, which ...
abc   Assignment – an explicit transfer of data into a variable   Branch – an explicit forward program branch out of scope...
Cyclomatic Complexity                  M = E − N + 2P            E = the number of edges of the graph            N = the n...
Basis Path Testing  http://users.csc.calpoly.edu/~jdalbey/206/Lectures/BasisPathTutorial/index.htmlThursday, February 23, 12
Complexity                                      Conservation                              “One of McCabes original applica...
Complexity &                             Cohesion         “Our results support the idea that there is a relationship      ...
saikuro                            http://saikuro.rubyforge.org/Thursday, February 23, 12
yupThursday, February 23, 12
flog                     • Written by Ryan Davis                     • opinionated                     • ruby specificThursd...
Thursday, February 23, 12
Thursday, February 23, 12
Flog Method Reading Ease *                            Score                          Notes                             1-2...
flogtime      Not quite done, but a        fun quick hack                            https://github.com/chad/flogtimeThursda...
code coverage                            https://github.com/colszowka/simplecovThursday, February 23, 12
code coverage                     • C0 - every line executed                     • C1 - every branch executed             ...
metric-fu roodi flog flay churn stats rails_best_practices reek saikuro coverage                            http://metric-fu...
Ward’s Signature                                Survey                             http://c2.com/doc/SignatureSurvey/Thurs...
something stupidThursday, February 23, 12
Thursday, February 23, 12
Bug Prediction at                                Google                            “Here at Google, we have thousands of  ...
Bug Prediction at Google                            https://github.com/igrigorik/bugspotsThursday, February 23, 12
$> cd /your/git/repo                       $>   git bugspots -d 500                            .. example output ..       ...
Hotspots:                                   0.9723 - ext/ed.cpp                                   0.3311 - ext/ed.h       ...
turbulence http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679&tth=DYN&tt=siteemail&iDy...
highest complexity                            turbulence   highest turbulenceThursday, February 23, 12
gitstats                            http://gitstats.sourceforge.net/Thursday, February 23, 12
gitstatsThursday, February 23, 12
The Church of GraphsThursday, February 23, 12
graphite                            Highly scalable open source real time                                       graphing e...
statsd                              Open source node-based daemon for                            stats aggregation. UDP fr...
statsd gemThursday, February 23, 12
Resque failuresThursday, February 23, 12
Error rates that matterThursday, February 23, 12
various key stats overlaidThursday, February 23, 12
looking at pictures only                                gets you so farThursday, February 23, 12
Shewhart Charts                            Measure process stabilityThursday, February 23, 12
doesn’t work for data like this thoughThursday, February 23, 12
Double Exponential                            Smoothing (Holt-Winters)     http://www.it.iitb.ac.in/~praj/acads/seminar/04...
Thursday, February 23, 12
What is QA?                            50 releases per day redefines “QA”Thursday, February 23, 12
Thanks!Thursday, February 23, 12
Upcoming SlideShare
Loading in...5
×

Measuring

1,985

Published on

Published in: Technology
1 Comment
5 Likes
Statistics
Notes
  • video at http://bostonrb.org/presentations/measuring-analyzing-things-that-matter-when-you-have-too-many-things-to-keep-track-of
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
1,985
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
20
Comments
1
Likes
5
Embeds 0
No embeds

No notes for slide

Transcript of "Measuring"

  1. 1. Measuring & Analyzing Things That Matter When You Have Too Many Things To Keep Track Of* (chocolate salmon) by @chadfowler * this title easily understood by 13-15 year old studentsThursday, February 23, 12
  2. 2. Thursday, February 23, 12
  3. 3. How does this happen?Thursday, February 23, 12
  4. 4. How do we fix it? • code reviews are hard • code reviews are time consuming • there are too many commits to review • bulleted lists are awesomeThursday, February 23, 12
  5. 5. information radiator “Coined around 2000 while standing in a Thoughtworks office looking at all the paper on the walls around me, “information radiator” refers to a publicly posted display that shows people walking by what is going on. Information radiators are best when they are big, very easy to see (e.g. not online, generally), and change often enough to be worth revisiting.” http://alistair.cockburn.us/Information+radiatorThursday, February 23, 12
  6. 6. “Chart what you care about, what you worry about, what you want other people to know.” http://xprogramming.com/articles/bigvisiblecharts/Thursday, February 23, 12
  7. 7. http://www.exampler.com/old-blog/2005/04/02/Thursday, February 23, 12
  8. 8. 1/2 Marathon TrainingThursday, February 23, 12
  9. 9. What Should We Measure? • Project progress • Code complexity • Developer “productivity” • Business “key performance indicators” • Bug/error rates • Everything importantThursday, February 23, 12
  10. 10. Code “Quality”Thursday, February 23, 12
  11. 11. Rudolph FleschThursday, February 23, 12
  12. 12. Flesch-Kincaid Reading Ease Score Notes 90-100 easily understood by an average 11-year-old student 60-70 easily understood by 13- to 15-year-old students 0-30 best understood by university graduatesThursday, February 23, 12
  13. 13. competing algorithms • SMOG • Gunning-Fog http://www.editcentral.com/gwt1/EditCentral.htmlThursday, February 23, 12
  14. 14. Maurice Halstead “Software Science” http://en.wikipedia.org/wiki/Halstead_complexity_measuresThursday, February 23, 12
  15. 15. Measure units of human thought used!!! “Stroud’s (1955, 1967) original proposal, which is usually called a discrete-moment model, claims that all incoming information is processed in nonoverlapping (i.e., temporarily discrete) samples or scans and that the temporal order of stimuli within a scan is not preserved.” T = ( n1N2( n1log2n1 + n2log2n2) log2n) / 2n2S http://yunus.hacettepe.edu.tr/~sencer/complexity.htmlThursday, February 23, 12
  16. 16. abc Assignment – an explicit transfer of data into a variable Branch – an explicit forward program branch out of scope Condition – a logical/Boolean test |ABC| = sqrt((A*A)+(B*B)+(C*C)) http://www.softwarerenovation.com/ABCMetric.pdfThursday, February 23, 12
  17. 17. Cyclomatic Complexity M = E − N + 2P E = the number of edges of the graph N = the number of nodes of the graph P = the number of connected componentsThursday, February 23, 12
  18. 18. Basis Path Testing http://users.csc.calpoly.edu/~jdalbey/206/Lectures/BasisPathTutorial/index.htmlThursday, February 23, 12
  19. 19. Complexity Conservation “One of McCabes original applications was to limit the complexity of routines during program development; he recommended that programmers should count the complexity of the modules they are developing, and split them into smaller modules whenever the cyclomatic complexity of the module exceeded 10.” NO MORE THAN TEN GALLONS OF CYCLOMATIC COMPLEXITY PER MODULE!!!!! http://en.wikipedia.org/wiki/Cyclomatic_complexity#cite_note-mccabe76-2Thursday, February 23, 12
  20. 20. Complexity & Cohesion “Our results support the idea that there is a relationship between complexity and cohesion, basically that a lack of cohesion is associated with high complexity”Exploring the Relationship between Cohesion and Complexity - Stein, Cox, Etzkorn - 2005Thursday, February 23, 12
  21. 21. saikuro http://saikuro.rubyforge.org/Thursday, February 23, 12
  22. 22. yupThursday, February 23, 12
  23. 23. flog • Written by Ryan Davis • opinionated • ruby specificThursday, February 23, 12
  24. 24. Thursday, February 23, 12
  25. 25. Thursday, February 23, 12
  26. 26. Flog Method Reading Ease * Score Notes 1-2 easily understood by an average Visual Basic developer 3-10 easily understood by PHP developers 10-20 best understood by Perl, Smalltalk, or Lisp developers 20+ rich hickey * all totally contrivedThursday, February 23, 12
  27. 27. flogtime Not quite done, but a fun quick hack https://github.com/chad/flogtimeThursday, February 23, 12
  28. 28. code coverage https://github.com/colszowka/simplecovThursday, February 23, 12
  29. 29. code coverage • C0 - every line executed • C1 - every branch executed • C2 - every code path across lines executed It doesn’t matter which one. Only direction matters.Thursday, February 23, 12
  30. 30. metric-fu roodi flog flay churn stats rails_best_practices reek saikuro coverage http://metric-fu.rubyforge.org/Thursday, February 23, 12
  31. 31. Ward’s Signature Survey http://c2.com/doc/SignatureSurvey/Thursday, February 23, 12
  32. 32. something stupidThursday, February 23, 12
  33. 33. Thursday, February 23, 12
  34. 34. Bug Prediction at Google “Here at Google, we have thousands of engineers working on our code base every day. In fact, as previously noted, 50% of the Google code base changes every month.” http://google-engtools.blogspot.com/2011/12/bug-prediction-at-google.htmlThursday, February 23, 12
  35. 35. Bug Prediction at Google https://github.com/igrigorik/bugspotsThursday, February 23, 12
  36. 36. $> cd /your/git/repo $> git bugspots -d 500 .. example output .. Scanning /git/eventmachine repo Found 31 bugfix commits, with 23 hotspots: Fixes: - Revert "Write maximum of 16KB of data to an SSL connection per tick (fixes #233)" for #273 - Do not close attached sockets (fixes #200) - Write maximum of 16KB of data to an SSL connection per tick (fixes #233) - Merge branch master into close_schedule_fix - Remove dependency on readbytes.rb for ruby 1.9 (fixes #167, #234) - Fix compilation on MSVC2008 (fixes #253) - EM::Deferrable#(callback|errback|timeout) now return self so you can chain them (closes #177) - Make EventMachine::Connection#get_peername and #get_sockname valid for IPv6 (closes #132) - reconnect DNS socket if closed - Use String#bytesize in EM::Connection#send_datagram for 1.9 (closes #153)Thursday, February 23, 12
  37. 37. Hotspots: 0.9723 - ext/ed.cpp 0.3311 - ext/ed.h 0.3271 - ext/em.cpp 0.3034 - lib/eventmachine.rb 0.2433 - lib/em/protocols/postgres3.rb 0.2403 - ext/project.h 0.0431 - lib/em/deferrable.rb 0.029 - ext/cmain.cpp 0.0278 - ext/rubymain.cpp 0.0277 - ext/eventmachine.h 0.0241 - lib/em/resolver.rb 0.0241 - tests/test_resolver.rb 0.0225 - lib/em/connection.rb 0.0013 - lib/em/protocols/smtpserver.rb 0.0003 - ext/extconf.rb 0.0002 - tests/test_basic.rb 0.0001 - ext/em.h 0.0001 - ext/cplusplus.cppThursday, February 23, 12
  38. 38. turbulence http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679&tth=DYN&tt=siteemail&iDyn=2 https://github.com/chad/turbulenceThursday, February 23, 12
  39. 39. highest complexity turbulence highest turbulenceThursday, February 23, 12
  40. 40. gitstats http://gitstats.sourceforge.net/Thursday, February 23, 12
  41. 41. gitstatsThursday, February 23, 12
  42. 42. The Church of GraphsThursday, February 23, 12
  43. 43. graphite Highly scalable open source real time graphing engine. http://graphite.wikidot.com/Thursday, February 23, 12
  44. 44. statsd Open source node-based daemon for stats aggregation. UDP front-end for fault- tolerance*. Pushes aggregated stats into graphite. https://github.com/etsy/statsdThursday, February 23, 12
  45. 45. statsd gemThursday, February 23, 12
  46. 46. Resque failuresThursday, February 23, 12
  47. 47. Error rates that matterThursday, February 23, 12
  48. 48. various key stats overlaidThursday, February 23, 12
  49. 49. looking at pictures only gets you so farThursday, February 23, 12
  50. 50. Shewhart Charts Measure process stabilityThursday, February 23, 12
  51. 51. doesn’t work for data like this thoughThursday, February 23, 12
  52. 52. Double Exponential Smoothing (Holt-Winters) http://www.it.iitb.ac.in/~praj/acads/seminar/04329008_ExponentialSmoothing.pdfThursday, February 23, 12
  53. 53. Thursday, February 23, 12
  54. 54. What is QA? 50 releases per day redefines “QA”Thursday, February 23, 12
  55. 55. Thanks!Thursday, February 23, 12
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×