Measuring

Measuring & Analyzing
Things That Matter
When You Have Too
Many Things To Keep
Track Of* (chocolate salmon)

by @chadfowler

* this title easily understood by 13-15 year old students
Thursday, February 23, 12

How does this happen?


How do we ﬁx it?

• code reviews are hard
• code reviews are time consuming
• there are too many commits to review
• bulleted lists are awesome


information radiator
“Coined around 2000 while standing in a Thoughtworks
office looking at all the paper on the walls around me,
“information radiator” refers to a publicly posted display
that shows people walking by what is going on.
Information radiators are best when they are big, very
easy to see (e.g. not online, generally), and change often
enough to be worth revisiting.”

http://alistair.cockburn.us/Information+radiator


“Chart what you care about, what you worry about,
what you want other people to know.”

http://xprogramming.com/articles/bigvisiblecharts/

http://www.exampler.com/old-blog/2005/04/02/

1/2 Marathon Training


What Should We
Measure?
• Project progress
• Code complexity
• Developer “productivity”
• Business “key performance indicators”
• Bug/error rates
• Everything important

Code “Quality”


Rudolph Flesch


Flesch-Kincaid Reading Ease
Score Notes

90-100 easily understood by an average 11-year-old student

60-70 easily understood by 13- to 15-year-old students

0-30 best understood by university graduates


competing algorithms

• SMOG
• Gunning-Fog

http://www.editcentral.com/gwt1/EditCentral.html


Maurice Halstead
“Software Science”

http://en.wikipedia.org/wiki/Halstead_complexity_measures


Measure units of human
thought used!!!
“Stroud’s (1955, 1967) original proposal, which is
usually called a discrete-moment model, claims that
all incoming information is processed in
nonoverlapping (i.e., temporarily discrete) samples
or scans and that the temporal order of stimuli
within a scan is not preserved.”

T ' = ( n1N2( n1log2n1 +
n2log2n2) log2n) / 2n2S

http://yunus.hacettepe.edu.tr/~sencer/complexity.html

abc

Assignment – an explicit transfer of data into a variable
Branch – an explicit forward program branch out of scope
Condition – a logical/Boolean test

|ABC| = sqrt((A*A)+(B*B)+(C*C))

http://www.softwarerenovation.com/ABCMetric.pdf

Cyclomatic Complexity
M = E − N + 2P

E = the number of edges of the graph
N = the number of nodes of the graph
P = the number of connected components


Basis Path Testing

http://users.csc.calpoly.edu/~jdalbey/206/Lectures/BasisPathTutorial/index.html


Complexity
Conservation
“One of McCabe's original applications was to limit the complexity of
routines during program development; he recommended that
programmers should count the complexity of the modules they are
developing, and split them into smaller modules whenever the
cyclomatic complexity of the module exceeded 10.”

NO MORE THAN TEN GALLONS OF CYCLOMATIC
COMPLEXITY PER MODULE!!!!!

http://en.wikipedia.org/wiki/Cyclomatic_complexity#cite_note-mccabe76-2


Complexity &
Cohesion
“Our results support the idea that there is a relationship
between complexity and cohesion, basically that a lack of
cohesion is associated with high complexity”

Exploring the Relationship between Cohesion and Complexity - Stein, Cox, Etzkorn
- 2005


saikuro

http://saikuro.rubyforge.org/

yup


ﬂog

• Written by Ryan Davis
• opinionated
• ruby speciﬁc


Flog Method Reading Ease *
Score Notes
1-2 easily understood by an average Visual Basic developer

3-10 easily understood by PHP developers

10-20 best understood by Perl, Smalltalk, or Lisp developers

20+ rich hickey

* all totally contrived


ﬂogtime

Not quite done, but a
fun quick hack

https://github.com/chad/ﬂogtime

code coverage

https://github.com/colszowka/simplecov

code coverage

• C0 - every line executed
• C1 - every branch executed
• C2 - every code path across lines executed

It doesn’t matter which one. Only direction matters.

metric-fu
roodi
ﬂog
ﬂay
churn
stats
rails_best_practices
reek
saikuro
coverage

http://metric-fu.rubyforge.org/

Ward’s Signature
Survey

http://c2.com/doc/SignatureSurvey/

something stupid


Bug Prediction at
Google
“Here at Google, we have thousands of
engineers working on our code base every
day. In fact, as previously noted, 50% of the
Google code base changes every month.”

http://google-engtools.blogspot.com/2011/12/bug-prediction-at-google.html


Bug Prediction at Google

https://github.com/igrigorik/bugspots


$> cd /your/git/repo
$> git bugspots -d 500

.. example output ..

Scanning /git/eventmachine repo
Found 31 bugfix commits, with 23 hotspots:

Fixes:
- Revert "Write maximum of 16KB of data to an SSL connection per tick (fixes #233)" for #273
- Do not close attached sockets (fixes #200)
- Write maximum of 16KB of data to an SSL connection per tick (fixes #233)
- Merge branch 'master' into close_schedule_fix
- Remove dependency on readbytes.rb for ruby 1.9 (fixes #167, #234)
- Fix compilation on MSVC2008 (fixes #253)
- EM::Deferrable#(callback|errback|timeout) now return self so you can chain them (closes #177)
- Make EventMachine::Connection#get_peername and #get_sockname valid for IPv6 (closes #132)
- reconnect DNS socket if closed
- Use String#bytesize in EM::Connection#send_datagram for 1.9 (closes #153)


Hotspots:
0.9723 - ext/ed.cpp
0.3311 - ext/ed.h
0.3271 - ext/em.cpp
0.3034 - lib/eventmachine.rb
0.2433 - lib/em/protocols/postgres3.rb
0.2403 - ext/project.h
0.0431 - lib/em/deferrable.rb
0.029 - ext/cmain.cpp
0.0278 - ext/rubymain.cpp
0.0277 - ext/eventmachine.h
0.0241 - lib/em/resolver.rb
0.0241 - tests/test_resolver.rb
0.0225 - lib/em/connection.rb
0.0013 - lib/em/protocols/smtpserver.rb
0.0003 - ext/extconf.rb
0.0002 - tests/test_basic.rb
0.0001 - ext/em.h
0.0001 - ext/cplusplus.cpp


turbulence
http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=16679&tth=DYN&tt=siteemail&iDyn=2

https://github.com/chad/turbulence

highest complexity
turbulence highest turbulence


gitstats

http://gitstats.sourceforge.net/

gitstats


The Church of Graphs


graphite

Highly scalable open source real time
graphing engine.

http://graphite.wikidot.com/

statsd
Open source node-based daemon for
stats aggregation. UDP front-end for fault-
tolerance*.

Pushes aggregated stats into graphite.

https://github.com/etsy/statsd


statsd gem


Resque failures


Error rates that matter


various key stats overlaid


looking at pictures only
gets you so far


Shewhart Charts

Measure process stability

doesn’t work for data like this though


Double Exponential
Smoothing (Holt-Winters)

http://www.it.iitb.ac.in/~praj/acads/seminar/04329008_ExponentialSmoothing.pdf


What is QA?

50 releases per day redeﬁnes “QA”

Thanks!


Measuring

Recommended

Recommended

More Related Content

Similar to Measuring

Similar to Measuring (20)

Recently uploaded

Recently uploaded (20)

Measuring