Code coverage is an easy way to measure if we have enough tests, yet many of us have yet to use it. This talk delves into the benefits of meaningful code coverage and how to avoid some of its pitfalls with a new tool called DeepCover.

4. Deep into Ruby Code Coverage
Code Coverage
1. What & Why?
2. How?
3. Under the hood

5. What & Why
Covered: ran by a least one test.
Code Coverage: identifies uncovered code.
tests missing
unused code
overly cautious code

6. How - Method Coverage
def foo(something: false)
if something
bar
end
end
# no test
Method coverage: 0%

7. How - Line Coverage
def foo(something: false)
if something
bar
end
end
# test:
expect(foo).to...
Method coverage: 100%
Line coverage: 75%

8. How - Node Coverage
def foo(something: false)
bar if something
end
# test:
expect(foo).to...
Method coverage: 100%
Line coverage: 100%
Node coverage: 80%

9. How - Branch coverage
def foo(something: false)
bar if something
end
# test:
expect(foo(something: true)).to...

10. How - Branch coverage
def foo(something: false)
bar if something
fail_unless_bar_was_run
end
# test:
expect(foo(something: true)).to...
Method coverage: 100%
Line coverage: 100%
Node coverage: 100%
Branch coverage: 50%

11. I want it
Sounds awesome

12. I want it

13. I want it

14. I want it

15. Live demo

16. Real world examples
Wiki Education Dashboard
100% loc => 96.75% loc
AsciiDoc - Kramdown
100% coverage => 99% of nodes, 77% of branches
found 3 bugs and counting

17. Typical examples
# Untested guard clauses
return unless Features.wiki_ed?
# Unused scopes
class Revision < ActiveRecord::Base
scope :after_date, ->(date) { where('date > ?', date) }
end
# Untested edge cases
siblings = (parent = opts[:parent]) && parent.children

20. DeepCover vs MRI
Feature MRI DeepCover
Line coverage partial ✓
Node coverage no ✓
Branch coverage partial ✓
Method coverage ✓ ~
Slowdown < 1% ~20%
Platform support Ruby 2.5+Ruby 2.1+, JRuby

21. DeepCover vs MRI
Branch coverage MRIDeepCover
if / unless / ?: ✓ ✓
case / when ✓ ✓
❘❘ / && no ✓
foo&.bar ✓ ✓
{❘foo = 42, bar:
43❘}
no ✓
while / until ✓ !

22. DeepCover vs MRI
require 'deep_cover/builtin_takeover'
Even compatible with secret 2.5+ API

23. Under the Hood
How to implement?
Counters

24. Under the Hood - MRI
static int
iseq_compile_each0(rb_iseq_t *iseq, LINK_ANCHOR *const ret, ...)
{
const int line = (int)nd_line(node);
const enum node_type type = nd_type(node);
if (ISEQ_COMPILE_DATA(iseq)->last_line == line) {
/* ignore */
}
else {
if (node->flags & NODE_FL_NEWLINE) {
ISEQ_COMPILE_DATA(iseq)->last_line = line;
ADD_TRACE_LINE_COVERAGE(ret, line);
ADD_TRACE(ret, RUBY_EVENT_LINE);
}
}
// ...
ADD_TRACE_LINE_COVERAGE: same API as TracePoint, just not acccessible
from Ruby.

25. Under the Hood - MRI
// ...
switch (type) {
case NODE_BLOCK:{
while (node && nd_type(node) == NODE_BLOCK) {
CHECK(COMPILE_(ret, "BLOCK body", node->nd_head,
(node->nd_next ? 1 : popped)));
node = node->nd_next;
}
if (node) {
CHECK(COMPILE_(ret, "BLOCK next", node->nd_next, popped));
}
break;
}
case NODE_IF:
case NODE_UNLESS:
CHECK(compile_if(iseq, ret, node, popped, type));
break;
… 1600 more lines…

26. Under the Hood - MRI
; a = 1
b = 2; c = 3
d = b + c
puts d

27. Under the Hood - MRI
ADD_TRACE_LINE_COVERAGE; a = 1
ADD_TRACE_LINE_COVERAGE; b = 2; c = 3
ADD_TRACE_LINE_COVERAGE; d = b + c
ADD_TRACE_LINE_COVERAGE; puts d

28. Under the Hood - DeepCover
$counter[0]+=1; a = 1
$counter[1]+=1; b = 2; c = 3
$counter[2]+=1; d = b + c
$counter[3]+=1; puts d

29. Under the Hood - DeepCover
$counter[0]+=1; a = 1
$counter[1]+=1; b = 2; $counter[2]+=1; c = 3
$counter[3]+=1; d = b + c
$counter[4]+=1; puts d

30. Under the Hood - DeepCover
$counter[0]+=1; a = 1
b = 2; c = 3
d = b + c; $counter[1]+=1;
puts d;

31. Under the Hood - DeepCover
use whitequark/parser gem
rewrite using rules for each type of node
only code insertions
no insertion of new lines
deduce execution from trackers
Either:
monkeypatch require, load and autoload
define RubyVM::InstructionSequence.load_iseq
clone project & rewrite

32. Rewriting rules
puts 'Hello'
# becomes
temp = puts 'hello'; tracker += 1 ; temp

33. Rewriting rules - most complex case
foo&.bar
# becomes
temp = foo
if nil != temp
branch_tracker += 1
temp = temp&.bar
tracker += 1
temp
end

34. Rewriting rules - actually most complex case
obj.a, b, obj.c = [...]
# becomes
# ???

35. Rewriting rules - actually most complex case
obj.a, b, obj.c = [...]
# becomes
(tracker += 1; obj).a, b,
(tracker2 += 1; obj).c = [...]; tracker3 += 1

36. Where can control flow be interrupted?
foo = 1
bar = 2
puts(foo + bar)

37. Where can control flow be interrupted?
require "active_support/core_ext/object/try"
module DateAndTime
module Calculations
WEEKEND_DAYS = [ 6, 0 ]
def on_weekend?
WEEKEND_DAYS.include?(wday)
end
def all_day
beginning_of_day..end_of_day
end
def end_of_week(start_day = Date.beginning_of_week)
# ...
end
alias :at_end_of_week :end_of_week
end
end

38. Future
Reporting for Coveralls/CodeCov
Better handling for complex setups (multi-version / CI / …)
100% coverage for DeepCover (!)

39. Thanks
Coauthor and original instigator
Maxime Lapointe (maxlap)
MRI branch coverage
Yusuke Endoh (mame)
Yuichiro Kaneko (yui-knk)
Early adopters
Sage Ross (ragesoss), Dan Allen (mojavelinux), Cedric (stripedpumpkin), Sylvain
Joyeux