Ruby AST Tools

Ruby AST Tools The ParseTree Family Brian Landau Wednesday, March 25, 2009

The Ruby AST Code Code Style 1 Style 2 AST Code Code Style 3 Style 4 Wednesday, March 25, 2009

The Ruby AST Nodes Wednesday, March 25, 2009

The Ruby AST Nodes def (a,b) def (a,b) c= a.to_s c= a.to_s return end return end def (a,b) def (a,b) c= a.to_s c= a.to_s Wednesday, March 25, 2009

Parsing Ruby in Ruby ParseTree ruby_parser Wednesday, March 25, 2009

ParseTree Wednesday, March 25, 2009

ParseTree RubyInline based Wednesday, March 25, 2009

ParseTree RubyInline based Input sources: Class Method Proc String Wednesday, March 25, 2009

ruby_parser Wednesday, March 25, 2009

ruby_parser Uses Racc Wednesday, March 25, 2009

ruby_parser Uses Racc Retrives line numbers Wednesday, March 25, 2009

ruby_parser Uses Racc Retrives line numbers Slower than ParseTree Wednesday, March 25, 2009

ruby_parser Uses Racc Retrives line numbers Slower than ParseTree Sexp output is ParseTree compatible Wednesday, March 25, 2009

Sexp Nodes Wednesday, March 25, 2009

Sexp Nodes class Simple def say(*args) puts args * ' ' end end Wednesday, March 25, 2009

Sexp Nodes class Simple def say(*args) puts args * ' ' end end s(:class, :Simple, nil, s(:scope, s(:defn, :say, s(:args, :quot;*argsquot;), s(:scope, s(:block, s(:call, nil, :puts, s(:arglist, s(:call, s(:lvar, :args), :*, s(:arglist, s(:str, quot; quot;)))))))))) Wednesday, March 25, 2009

Sexp Objects Inherits from Array `structure` method `each_of_type` method `sexp_type` method `sexp_body` method Wednesday, March 25, 2009

SexpProcessors Or How I Learned to Love ParseTree Wednesday, March 25, 2009

Making a SexpProcessors Wednesday, March 25, 2009

Making a SexpProcessors Subclass `SexpProcessor` Wednesday, March 25, 2009

Making a SexpProcessors Subclass `SexpProcessor` Call `super` from initialize Wednesday, March 25, 2009

Making a SexpProcessors Subclass `SexpProcessor` Call `super` from initialize Deﬁne `process_xxx` methods Wednesday, March 25, 2009

Making a SexpProcessors Subclass `SexpProcessor` Call `super` from initialize Deﬁne `process_xxx` methods Shift off Sexp nodes Wednesday, March 25, 2009

Making a SexpProcessors Subclass `SexpProcessor` Call `super` from initialize Deﬁne `process_xxx` methods Shift off Sexp nodes Default process method Wednesday, March 25, 2009

Making a SexpProcessors Subclass `SexpProcessor` Call `super` from initialize Deﬁne `process_xxx` methods Shift off Sexp nodes Default process method `rewrite_xxx` methods Wednesday, March 25, 2009

Making a SexpProcessors Subclass SexpProcessor class MyProcessor < SexpProcessor end Wednesday, March 25, 2009

Making a SexpProcessors Call “super” in “initialize” class MyProcessor < SexpProcessor def initialize super self.warn_on_default = false self.strict = false end end Wednesday, March 25, 2009

Making a SexpProcessors class MyProcessor < SexpProcessor # ... Process “call” node def process_call(exp) recv = process exp.shift name = exp.shift args = process exp.shift return s() end end Wednesday, March 25, 2009

Making a SexpProcessors class MyProcessor < SexpProcessor # ... def process_call(exp) recv = process exp.shift name = exp.shift args = process exp.shift return s() end Must shift o all Sexp Nodes end Wednesday, March 25, 2009

Making a SexpProcessors class MyProcessor < SexpProcessor # ... def process_call(exp) recv = process exp.shift name = exp.shift args = process exp.shift return s() end Must return a Sexp end Wednesday, March 25, 2009

Making a SexpProcessors class MyProcessor < SexpProcessor # ... def default_process(exp) until exp.size == 0 exp.shift end return s() A default process method end end Wednesday, March 25, 2009

UniﬁedRuby pt = ParseTree.new parse_tree = pt.parse_tree(Simple) ast = pt.process(parse_tree) # OR ast = pt.process(File.read('simple.rb')) Wednesday, March 25, 2009

UniﬁedRuby class MyProcessor < SexpProcessor include UniﬁedRuby # ... end Wednesday, March 25, 2009

Ruby2Ruby Ruby2Ruby.translate(Simple, :say) # => quot;def say(*args)...quot; Wednesday, March 25, 2009

ParseTree extensions require 'parse_tree' require 'ruby2ruby' require 'parse_tree_extensions' bomb = proc do quot;BOOM!quot; end puts bomb.to_ruby # => quot;proc { quot;BOOM!quot; }quot; Wednesday, March 25, 2009

ParseTree Family Flog ambition Parse Tree roodi Wednesday, March 25, 2009

ParseTree Family Code Analytics Custom DSL Wednesday, March 25, 2009

Flog Takes a set of ﬁles as input Processes for “bad code” Wednesday, March 25, 2009

Flog Decends from SexpProcessor Includes UniﬁedRuby Wednesday, March 25, 2009

Flog Assigns weighted scores to: Node Types Node Constructions Method calls Wednesday, March 25, 2009

Flog Reports scores by: Class Method Wednesday, March 25, 2009

Flog > flog lib 377.6: flog total 11.1: flog/method average 150.2: ClassMethods#acts_as_markup 24.7: HTML#array_to_html 21.5: ClassMethods#get_markdown_class 18.4: main#none 17.9: ActsAsMarkup#none Wednesday, March 25, 2009

Flog > flog -g lib 377.6: flog total 11.1: flog/method average 171.7: ClassMethods total 150.2: ClassMethods#acts_as_markup 21.5: ClassMethods#get_markdown_class 24.7: HTML total 24.7: HTML#array_to_html ... Wednesday, March 25, 2009

Flog > flog -d lib 377.6: flog total 11.1: flog/method average 150.2: ClassMethods#acts_as_markup 50.9: [] 42.6: branch 24.0: send 16.5: define_method 11.9: to_s 11.8: assignment ... Wednesday, March 25, 2009

self-ﬂagellation Choose your own weightings Wednesday, March 25, 2009

roodi Ruby Object Oriented Design Inferometer Wednesday, March 25, 2009

roodi Ruby Object Oriented Design Inferometer Cyclomatic complexity Wednesday, March 25, 2009

roodi Ruby Object Oriented Design Inferometer Cyclomatic complexity Assignment in conditional Wednesday, March 25, 2009

roodi Ruby Object Oriented Design Inferometer Cyclomatic complexity Assignment in conditional Class/Method line count Wednesday, March 25, 2009

roodi Ruby Object Oriented Design Inferometer Cyclomatic complexity Assignment in conditional Class/Method line count Empty rescue body Wednesday, March 25, 2009

roodi Ruby Object Oriented Design Inferometer Cyclomatic complexity Assignment in conditional Class/Method line count Empty rescue body No for loops Wednesday, March 25, 2009

roodi Ruby Object Oriented Design Inferometer Cyclomatic complexity Assignment in conditional Class/Method line count Empty rescue body No for loops Number of parameters Wednesday, March 25, 2009

roodi > sudo gem install roodi > roodi “lib/**/*.rb” Wednesday, March 25, 2009

roodi: How it works Wednesday, March 25, 2009

roodi: How it works Parse ﬁles Wednesday, March 25, 2009

roodi: How it works Parse ﬁles Move node-by-node Wednesday, March 25, 2009

roodi: How it works Parse ﬁles Move node-by-node Evaluate each “Check” Wednesday, March 25, 2009

roodi: How it works Parse ﬁles Move node-by-node Evaluate each “Check” Do Recursively Wednesday, March 25, 2009

Ambition Wednesday, March 25, 2009

Ambition DSL for Querying as Ruby Wednesday, March 25, 2009

Ambition DSL for Querying as Ruby Framework for API Adapters Wednesday, March 25, 2009

Ambition DSL for Querying as Ruby Framework for API Adapters Generator for new Adapters Wednesday, March 25, 2009

Ambition ActiveRecord Adapter Queries only run when needed to_s = SQL to_hash = ﬁnd-compatible hash Wednesday, March 25, 2009

Ambition: How it works Processors parameters: Context/Owner Block Wednesday, March 25, 2009

Ambition: How it works Context/Owner User.select { |m| m.name == 'jon' } Processor Block Wednesday, March 25, 2009

Ambition: How it works Parse the block Process nodes recursively Translate “clauses” to query string Wednesday, March 25, 2009

Ambition: How it works Adapters: Translator for Processors Make API calls with a Query Class Wednesday, March 25, 2009

The Future: Ruby 1.9 YARV A virtual machine based stack architecture implementation Ripper Offers similar functionality as ParseTree and SexpProcessor Wednesday, March 25, 2009

Ripper Effects YARV behavior Returns array of arrays & literals Line and column for each token Ripper::Filter Wednesday, March 25, 2009

Ripper: Lexer require 'ripper' require 'pp' p Ripper.lex(quot;def m(a) nil endquot;) #=> [[[1, 0], :on_kw, quot;defquot;], # [[1, 3], :on_sp, quot; quot; ], # [[1, 4], :on_ident, quot;mquot; ], # [[1, 5], :on_lparen, quot;(quot; ], # [[1, 6], :on_ident, quot;aquot; ], # [[1, 7], :on_rparen, quot;)quot; ], # [[1, 8], :on_sp, quot; quot; ], # [[1, 9], :on_kw, quot;nilquot;], # [[1, 12], :on_sp, quot; quot; ], # [[1, 13], :on_kw, quot;endquot;]] Wednesday, March 25, 2009

Ripper: Sexp Tree require 'ripper' require 'pp' pp Ripper.sexp(quot;def m(a) nil endquot;) #=> [:program, # [:stmts_add, # [:stmts_new], # [:def, # [:@ident, quot;mquot;, [1, 4]], # [:paren, [:params, [[:@ident, quot;aquot;, [1, 6]]], nil, nil, nil]], # [:bodystmt, # [:stmts_add, [:stmts_new], [:var_ref, [:@kw, quot;nilquot;, [1, 9]]]], # nil, # nil, # nil]]]] Wednesday, March 25, 2009

Ripper: Filter require 'ripper/ﬁlter' class CommentStripper < Ripper::Filter def self.strip(src) new(src).parse end def on_default(event, token, data) print token end def on_comment(token, data) puts end end CommentStripper.strip(ARGF) Wednesday, March 25, 2009

Q&A Rate at: http://speakerrate.com/talks/594-ruby-ast-tools brian.landau@viget.com http://www.viget.com/extend http://www.websideattractions.com/ Wednesday, March 25, 2009

http://websideattractions.com	114
http://www.websideattractions.com	84
http://www.slideshare.net	14

Ruby AST Tools

by Brian Landau on Mar 25, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 212

Statistics

Ruby AST Tools — Presentation Transcript