Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Towards
Tooling
what is missing
from our toolbox?
Loren Segal
@lsegal

Friday, November 8, 13
Are Rubyists
good at testing
because they
have good tools?
Friday, November 8, 13
Do Rubyists have
good tools
because they are
good at testing?
Friday, November 8, 13
Do Rubyists have
good tools
because they are
good at testing?
Friday, November 8, 13
Friday, November 8, 13
Tools are
important
Friday, November 8, 13
We have
good tools
Friday, November 8, 13
...sometimes.
Friday, November 8, 13
This talk is
about the

not-so-good

tools

Friday, November 8, 13
Goals
Friday, November 8, 13
1.
Introduce
different tools
Friday, November 8, 13
2.
Find out which
tools we are
missing
Friday, November 8, 13
3.
Write these
tools plz thx!
Be a garbage collector
Friday, November 8, 13
Note:

Google
TOOL NAME + LANGUAGE
You should find the tools
referenced in this talk
Friday, November 8, 13
Kinds of Tools
Friday, November 8, 13
Deployment / Ops
Documentation
Testing
Visualization

High Level

Debugging
Linting
Static Analysis

Low Level

Friday, No...
Visualization
Friday, November 8, 13
Some of
the most
important tools
are visualization tools
Friday, November 8, 13
Know what
your code
is doing
Friday, November 8, 13
Thread in a sealed box.
Is it dead or alive?

Friday, November 8, 13
Visual Studio

Friday, November 8, 13
Visual Studio

Friday, November 8, 13
XCode

Friday, November 8, 13
VisualVM

Friday, November 8, 13
Discoverability

Friday, November 8, 13
Implementors

Call references

Friday, November 8, 13

ECLIPSE
Not just
IDEs
Friday, November 8, 13
I’ll prove it...

Friday, November 8, 13
Firebug

Friday, November 8, 13
Do you remember
web development
before Firebug?
Friday, November 8, 13
Before: no visibility.

Friday, November 8, 13
Ember Inspector

Friday, November 8, 13
Smalltalk
Friday, November 8, 13
Friday, November 8, 13
Inherently
Visual
Friday, November 8, 13
Where is
Ruby viz?
Friday, November 8, 13
RubyMine

Friday, November 8, 13
Profilers?
Friday, November 8, 13
memprof
Joe Damato
github/ice799/memprof
Friday, November 8, 13
perftools.rb

Friday, November 8, 13
NetBeans / JRuby

Friday, November 8, 13
Use the
JVM
Friday, November 8, 13
Lintng
Friday, November 8, 13
Lint
divide by zero: check
initialized vars: check
...
style: check (last!)
Friday, November 8, 13
Ruby?
Friday, November 8, 13
Reek/Flog/Flay
Does: detect code smells
Does not: find common errors

Friday, November 8, 13
Assumption:
Pretty code is
correct code

Friday, November 8, 13
Friday, November 8, 13
Ugly.
Not “correct”.
Friday, November 8, 13
github.com/lsegal/my_fake_project

Friday, November 8, 13
PS. I ♡
Code
Climate
Friday, November 8, 13
Understand
your tools
Friday, November 8, 13
Code Climate
does not
replace
testing
Friday, November 8, 13
ruby-lint
Yorick Peterse
but it’s new
Friday, November 8, 13
Nothing
comes
standard
Friday, November 8, 13
Other
languages?
Friday, November 8, 13
JSHint (JavaScript)
pylint (Python)
FindBugs (Java)
FxCop (C#)
Friday, November 8, 13
Widely used.
Friday, November 8, 13
Why not
Ruby?
Friday, November 8, 13
Friday, November 8, 13
Static
Analysis
lint++
Friday, November 8, 13
is a
huge field

Friday, November 8, 13
Friday, November 8, 13
Types of “static analysis”
-

Defect Finding

-

Memory Checking / Fuzz Testing

-

Extended Static Checking

-

Model Che...
Defect
Finding
Friday, November 8, 13
is basically lint,

Friday, November 8, 13
but with less
emphasis on syntax.

Friday, November 8, 13
The Usual
Suspects
Friday, November 8, 13
Brakeman
Justin Collins
brakemanscanner.org
(Ruby on Rails)
Friday, November 8, 13
Finds common flaws
in Rails code
XSS, SQL injection, mass
assignment
Friday, November 8, 13
Friday, November 8, 13
Static detection of security vulnerabilities
in scripting languages
https://www.usenix.org/legacy/event/sec06/tech/full_pa...
Fuzz
Testing
Friday, November 8, 13
garbage in...

Friday, November 8, 13
Lots of tools.
C, Java, JS, Python, etc.

Friday, November 8, 13
Lots of papers.

Friday, November 8, 13
“Automated Whitebox
Fuzz Testing”
Microsoft Research

(used in SAGE)
http://research.microsoft.com/en-us/um/people/
pg/pub...
What about us?

Friday, November 8, 13
Heckle
Ryan Davis, Kevin Clark
Friday, November 8, 13
Friday, November 8, 13
Mutant
Markus Schirp

github/mbj/mutant
Friday, November 8, 13
We could use a real fuzz testing tool.

Friday, November 8, 13
FuzzBert?
Martin Bosslet

github/krypt/FuzzBert
Friday, November 8, 13
lots of papers out there
with algorithms to implement

Friday, November 8, 13
LET’S GET

Friday, November 8, 13
Symbolic
Execution
Friday, November 8, 13
Run your code
with no immediate values

Friday, November 8, 13
Similar to
Extended Static Checking
but...

Friday, November 8, 13
Contracts not required
and
Can tell you which inputs
generated valid or invalid state
Friday, November 8, 13
Think:
Automatic Test Case
Generation

Friday, November 8, 13
// @example pow(2, 8) == 256
int pow(int x, int n) {
int v[32] = {x}, result = 0;
for (int i = 1; i < n; i++) {
v[i] = x *...
SymExe report:
x=1,n=5,result=1
x=2,n=8,result=256
x=1,n=0,error: array out of bounds ←
x=1,n=33,error: array out of bound...
// @example pow(2, 8) == 256
// @requires n > 0
// @requires n < 32
int pow(int x, int n) {
int v[32] = {x}, result = 0;
f...
Tools?
Friday, November 8, 13
KLEE (LLVM)
Kudzu (JavaScript)
Kiasan (Java, SPARK)

Friday, November 8, 13
Nothing for Ruby*

“Automatic Program Verification and
Test Case Generation of Ruby Programs”
(*)

Friday, November 8, 13
Ruby doesn’t really have a
scientific community.

Friday, November 8, 13
Chicken and egg.

Friday, November 8, 13
Python vs Ruby?
Big boy language?

Friday, November 8, 13
RECAP

Friday, November 8, 13
We are great
at testing,
deployment,
web frameworks
Friday, November 8, 13
Not so good at
visualization,
linting,
static analysis
Friday, November 8, 13
We attract
web developers
because we have good
web tools

Friday, November 8, 13
Could we
build tools
for other
communities?
science, engineering, math
Friday, November 8, 13
Take responsibility.

Friday, November 8, 13
Great tool ideas are
waiting to be implemented

Friday, November 8, 13
Tons of research papers
in fields I mentioned
scholar.google.com
Friday, November 8, 13
I had a whole section on my
favourite research papers.

Friday, November 8, 13
Come find me if you want titles.

Friday, November 8, 13
Thank you.
Slides will be linked on Twitter

@lsegal

Friday, November 8, 13
Upcoming SlideShare
Loading in …5
×

Towards Tooling; A Look at What is Missing From the Ruby Toolbox

1,405 views

Published on

You can usually judge the maturity of a programming language ecosystem by the breadth of its tooling. For example, Java has a plethora of IDEs that each, in turn, have many well maintained refactoring and code quality plugins (like FindBugs and PMD). C/C++ is equally well established in this space. Even JavaScript is becoming well represented, with a number of static code analyzers and language supersets (like Dart and TypeScript) aimed at improving tooling in the language. But where is Ruby in all of this? This talk will shed some light on the existing tools available in the Ruby world as well as some new tools just starting to be built out in the areas of static analysis, formal verification, and code quality checking. We will look at what kind of tools the Ruby community is good at building, what kind of tools we are bad at, and most of all, some of the tooling we should be working on to really improve our ecosystem and drive more developers to this wonderful language.

Published in: Technology
  • Be the first to comment

Towards Tooling; A Look at What is Missing From the Ruby Toolbox

  1. 1. Towards Tooling what is missing from our toolbox? Loren Segal @lsegal Friday, November 8, 13
  2. 2. Are Rubyists good at testing because they have good tools? Friday, November 8, 13
  3. 3. Do Rubyists have good tools because they are good at testing? Friday, November 8, 13
  4. 4. Do Rubyists have good tools because they are good at testing? Friday, November 8, 13
  5. 5. Friday, November 8, 13
  6. 6. Tools are important Friday, November 8, 13
  7. 7. We have good tools Friday, November 8, 13
  8. 8. ...sometimes. Friday, November 8, 13
  9. 9. This talk is about the not-so-good tools Friday, November 8, 13
  10. 10. Goals Friday, November 8, 13
  11. 11. 1. Introduce different tools Friday, November 8, 13
  12. 12. 2. Find out which tools we are missing Friday, November 8, 13
  13. 13. 3. Write these tools plz thx! Be a garbage collector Friday, November 8, 13
  14. 14. Note: Google TOOL NAME + LANGUAGE You should find the tools referenced in this talk Friday, November 8, 13
  15. 15. Kinds of Tools Friday, November 8, 13
  16. 16. Deployment / Ops Documentation Testing Visualization High Level Debugging Linting Static Analysis Low Level Friday, November 8, 13
  17. 17. Visualization Friday, November 8, 13
  18. 18. Some of the most important tools are visualization tools Friday, November 8, 13
  19. 19. Know what your code is doing Friday, November 8, 13
  20. 20. Thread in a sealed box. Is it dead or alive? Friday, November 8, 13
  21. 21. Visual Studio Friday, November 8, 13
  22. 22. Visual Studio Friday, November 8, 13
  23. 23. XCode Friday, November 8, 13
  24. 24. VisualVM Friday, November 8, 13
  25. 25. Discoverability Friday, November 8, 13
  26. 26. Implementors Call references Friday, November 8, 13 ECLIPSE
  27. 27. Not just IDEs Friday, November 8, 13
  28. 28. I’ll prove it... Friday, November 8, 13
  29. 29. Firebug Friday, November 8, 13
  30. 30. Do you remember web development before Firebug? Friday, November 8, 13
  31. 31. Before: no visibility. Friday, November 8, 13
  32. 32. Ember Inspector Friday, November 8, 13
  33. 33. Smalltalk Friday, November 8, 13
  34. 34. Friday, November 8, 13
  35. 35. Inherently Visual Friday, November 8, 13
  36. 36. Where is Ruby viz? Friday, November 8, 13
  37. 37. RubyMine Friday, November 8, 13
  38. 38. Profilers? Friday, November 8, 13
  39. 39. memprof Joe Damato github/ice799/memprof Friday, November 8, 13
  40. 40. perftools.rb Friday, November 8, 13
  41. 41. NetBeans / JRuby Friday, November 8, 13
  42. 42. Use the JVM Friday, November 8, 13
  43. 43. Lintng Friday, November 8, 13
  44. 44. Lint divide by zero: check initialized vars: check ... style: check (last!) Friday, November 8, 13
  45. 45. Ruby? Friday, November 8, 13
  46. 46. Reek/Flog/Flay Does: detect code smells Does not: find common errors Friday, November 8, 13
  47. 47. Assumption: Pretty code is correct code Friday, November 8, 13
  48. 48. Friday, November 8, 13
  49. 49. Ugly. Not “correct”. Friday, November 8, 13
  50. 50. github.com/lsegal/my_fake_project Friday, November 8, 13
  51. 51. PS. I ♡ Code Climate Friday, November 8, 13
  52. 52. Understand your tools Friday, November 8, 13
  53. 53. Code Climate does not replace testing Friday, November 8, 13
  54. 54. ruby-lint Yorick Peterse but it’s new Friday, November 8, 13
  55. 55. Nothing comes standard Friday, November 8, 13
  56. 56. Other languages? Friday, November 8, 13
  57. 57. JSHint (JavaScript) pylint (Python) FindBugs (Java) FxCop (C#) Friday, November 8, 13
  58. 58. Widely used. Friday, November 8, 13
  59. 59. Why not Ruby? Friday, November 8, 13
  60. 60. Friday, November 8, 13
  61. 61. Static Analysis lint++ Friday, November 8, 13
  62. 62. is a huge field Friday, November 8, 13
  63. 63. Friday, November 8, 13
  64. 64. Types of “static analysis” - Defect Finding - Memory Checking / Fuzz Testing - Extended Static Checking - Model Checking / Data Flow Analysis - Symbolic Execution Friday, November 8, 13
  65. 65. Defect Finding Friday, November 8, 13
  66. 66. is basically lint, Friday, November 8, 13
  67. 67. but with less emphasis on syntax. Friday, November 8, 13
  68. 68. The Usual Suspects Friday, November 8, 13
  69. 69. Brakeman Justin Collins brakemanscanner.org (Ruby on Rails) Friday, November 8, 13
  70. 70. Finds common flaws in Rails code XSS, SQL injection, mass assignment Friday, November 8, 13
  71. 71. Friday, November 8, 13
  72. 72. Static detection of security vulnerabilities in scripting languages https://www.usenix.org/legacy/event/sec06/tech/full_papers/xie/xie_html/ Friday, November 8, 13
  73. 73. Fuzz Testing Friday, November 8, 13
  74. 74. garbage in... Friday, November 8, 13
  75. 75. Lots of tools. C, Java, JS, Python, etc. Friday, November 8, 13
  76. 76. Lots of papers. Friday, November 8, 13
  77. 77. “Automated Whitebox Fuzz Testing” Microsoft Research (used in SAGE) http://research.microsoft.com/en-us/um/people/ pg/public_psfiles/ndss2008.pdf Friday, November 8, 13
  78. 78. What about us? Friday, November 8, 13
  79. 79. Heckle Ryan Davis, Kevin Clark Friday, November 8, 13
  80. 80. Friday, November 8, 13
  81. 81. Mutant Markus Schirp github/mbj/mutant Friday, November 8, 13
  82. 82. We could use a real fuzz testing tool. Friday, November 8, 13
  83. 83. FuzzBert? Martin Bosslet github/krypt/FuzzBert Friday, November 8, 13
  84. 84. lots of papers out there with algorithms to implement Friday, November 8, 13
  85. 85. LET’S GET Friday, November 8, 13
  86. 86. Symbolic Execution Friday, November 8, 13
  87. 87. Run your code with no immediate values Friday, November 8, 13
  88. 88. Similar to Extended Static Checking but... Friday, November 8, 13
  89. 89. Contracts not required and Can tell you which inputs generated valid or invalid state Friday, November 8, 13
  90. 90. Think: Automatic Test Case Generation Friday, November 8, 13
  91. 91. // @example pow(2, 8) == 256 int pow(int x, int n) { int v[32] = {x}, result = 0; for (int i = 1; i < n; i++) { v[i] = x * v[i-1]; } return v[n-1]; } Friday, November 8, 13
  92. 92. SymExe report: x=1,n=5,result=1 x=2,n=8,result=256 x=1,n=0,error: array out of bounds ← x=1,n=33,error: array out of bounds ← Friday, November 8, 13
  93. 93. // @example pow(2, 8) == 256 // @requires n > 0 // @requires n < 32 int pow(int x, int n) { int v[32] = {x}, result = 0; for (int i = 1; i < n; i++) { v[i] = x * v[i-1]; } return v[n-1]; } Friday, November 8, 13
  94. 94. Tools? Friday, November 8, 13
  95. 95. KLEE (LLVM) Kudzu (JavaScript) Kiasan (Java, SPARK) Friday, November 8, 13
  96. 96. Nothing for Ruby* “Automatic Program Verification and Test Case Generation of Ruby Programs” (*) Friday, November 8, 13
  97. 97. Ruby doesn’t really have a scientific community. Friday, November 8, 13
  98. 98. Chicken and egg. Friday, November 8, 13
  99. 99. Python vs Ruby? Big boy language? Friday, November 8, 13
  100. 100. RECAP Friday, November 8, 13
  101. 101. We are great at testing, deployment, web frameworks Friday, November 8, 13
  102. 102. Not so good at visualization, linting, static analysis Friday, November 8, 13
  103. 103. We attract web developers because we have good web tools Friday, November 8, 13
  104. 104. Could we build tools for other communities? science, engineering, math Friday, November 8, 13
  105. 105. Take responsibility. Friday, November 8, 13
  106. 106. Great tool ideas are waiting to be implemented Friday, November 8, 13
  107. 107. Tons of research papers in fields I mentioned scholar.google.com Friday, November 8, 13
  108. 108. I had a whole section on my favourite research papers. Friday, November 8, 13
  109. 109. Come find me if you want titles. Friday, November 8, 13
  110. 110. Thank you. Slides will be linked on Twitter @lsegal Friday, November 8, 13

×