Towards Tooling; A Look at What is Missing From the Ruby Toolbox

Towards
Tooling
what is missing
from our toolbox?
Loren Segal
@lsegal

Friday, November 8, 13

Are Rubyists
good at testing
because they
have good tools?

Do Rubyists have
good tools
because they are
good at testing?

Tools are
important

We have
good tools

...sometimes.

This talk is
about the

not-so-good

tools


1.
Introduce
different tools

2.
Find out which
tools we are
missing

3.
Write these
tools plz thx!
Be a garbage collector

Note:

Google
TOOL NAME + LANGUAGE
You should ﬁnd the tools
referenced in this talk

Kinds of Tools

Deployment / Ops
Documentation
Testing
Visualization

High Level

Debugging
Linting
Static Analysis

Low Level


Visualization

Some of
the most
important tools
are visualization tools

Know what
your code
is doing

Thread in a sealed box.
Is it dead or alive?


Visual Studio


VisualVM


Discoverability


Implementors

Call references


ECLIPSE

Not just
IDEs

I’ll prove it...


Firebug


Do you remember
web development
before Firebug?

Before: no visibility.


Ember Inspector


Smalltalk

Inherently
Visual

Where is
Ruby viz?

RubyMine


Proﬁlers?

memprof
Joe Damato
github/ice799/memprof

perftools.rb


NetBeans / JRuby


Use the
JVM

Lint
divide by zero: check
initialized vars: check
...
style: check (last!)

Reek/Flog/Flay
Does: detect code smells
Does not: ﬁnd common errors


Assumption:
Pretty code is
correct code


Ugly.
Not “correct”.

github.com/lsegal/my_fake_project


PS. I ♡
Code
Climate

Understand
your tools

Code Climate
does not
replace
testing

ruby-lint
Yorick Peterse
but it’s new

Nothing
comes
standard

Other
languages?

JSHint (JavaScript)
pylint (Python)
FindBugs (Java)
FxCop (C#)

Widely used.

Why not
Ruby?

Static
Analysis
lint++

is a
huge ﬁeld


Types of “static analysis”
-

Defect Finding

-

Memory Checking / Fuzz Testing

-

Extended Static Checking

-

Model Checking / Data Flow Analysis

-

Symbolic Execution


Defect
Finding

is basically lint,


but with less
emphasis on syntax.


The Usual
Suspects

Brakeman
Justin Collins
brakemanscanner.org
(Ruby on Rails)

Finds common ﬂaws
in Rails code
XSS, SQL injection, mass
assignment

Static detection of security vulnerabilities
in scripting languages
https://www.usenix.org/legacy/event/sec06/tech/full_papers/xie/xie_html/


Fuzz
Testing

garbage in...


Lots of tools.
C, Java, JS, Python, etc.


Lots of papers.


“Automated Whitebox
Fuzz Testing”
Microsoft Research

(used in SAGE)
http://research.microsoft.com/en-us/um/people/
pg/public_psﬁles/ndss2008.pdf


What about us?


Heckle
Ryan Davis, Kevin Clark

Mutant
Markus Schirp

github/mbj/mutant

We could use a real fuzz testing tool.


FuzzBert?
Martin Bosslet

github/krypt/FuzzBert

lots of papers out there
with algorithms to implement


LET’S GET


Symbolic
Execution

Run your code
with no immediate values


Similar to
Extended Static Checking
but...


Contracts not required
and
Can tell you which inputs
generated valid or invalid state

Think:
Automatic Test Case
Generation


// @example pow(2, 8) == 256
int pow(int x, int n) {
int v[32] = {x}, result = 0;
for (int i = 1; i < n; i++) {
v[i] = x * v[i-1];
}
return v[n-1];
}


SymExe report:
x=1,n=5,result=1
x=2,n=8,result=256
x=1,n=0,error: array out of bounds ←
x=1,n=33,error: array out of bounds ←


// @example pow(2, 8) == 256
// @requires n > 0
// @requires n < 32
int pow(int x, int n) {
int v[32] = {x}, result = 0;
for (int i = 1; i < n; i++) {
v[i] = x * v[i-1];
}
return v[n-1];
}


KLEE (LLVM)
Kudzu (JavaScript)
Kiasan (Java, SPARK)


Nothing for Ruby*

“Automatic Program Veriﬁcation and
Test Case Generation of Ruby Programs”
(*)


Ruby doesn’t really have a
scientiﬁc community.


Chicken and egg.


Python vs Ruby?
Big boy language?


We are great
at testing,
deployment,
web frameworks

Not so good at
visualization,
linting,
static analysis

We attract
web developers
because we have good
web tools


Could we
build tools
for other
communities?
science, engineering, math

Take responsibility.


Great tool ideas are
waiting to be implemented


Tons of research papers
in ﬁelds I mentioned
scholar.google.com

I had a whole section on my
favourite research papers.


Come ﬁnd me if you want titles.


Thank you.
Slides will be linked on Twitter

@lsegal


Towards Tooling; A Look at What is Missing From the Ruby Toolbox

Recommended

Recommended

More Related Content

Recently uploaded

Recently uploaded (20)

Featured

Featured (20)

Towards Tooling; A Look at What is Missing From the Ruby Toolbox