Your SlideShare is downloading. ×
Tork03 LT
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Tork03 LT

670

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
670
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. Ruby Masaya TARUI
    • 2. ••• tokyu.rb #ruby-ja
    • 3.
    • 4. • Regexp• Rex• PEG ( Treetop , Citrus , Parslet )• Ripper• Bootstrap• M17N• ERB• Racc
    • 5. ••••
    • 6. • Racc strscan Ruby C → Rex
    • 7. Rex• C• Regexp!• Ruby /(match1)|(match2)|.../ Marshal capture [ match1,match1’s capture1, match1’s capture2...• Cature Capture 1.9 Lost Ruby parsec
    • 8. • tork05 ( tork03 )• @kawabata PEG : Parsing Expression Grammar https://github.com/kawabata/aozora-proc•• Ruby M17N Treetop, Citrus →(Rule Class )• M17N
    • 9. PEG•• Packrat Parser O(n) Packrat Parser =•
    • 10. # -*- coding: utf-8 -*-require pprequire ./peg.rbexpr_peg = <<EOTparser expr(top) top ← Expr !. ->$1 Expr ← Sum Sum ← Product ((+ / -) Product)* -> { restruct(r) } Product ← Value ((* / /) Value)* -> { restruct(r) } Value ← Sp n:[0-9]+ Sp -> {n.join.to_i} / Sp ( e:Expr ) Sp -> e Sp ← ( " " / "t" / "n" )*EOTdef restruct(r) r[1].inject(r[0]){|a,b| [b[0],a,b[1]] }endexpr_parser = PEG::ParserGenerator.parse(expr_peg)require ppexpr = " 1+ 2 * (3 - 4) / 5 + 6 "pp exprpp expr_parser.parse(expr)=>" 1+ 2 * (3 - 4) / 5 + 6 "["+", ["+", 1, ["/", ["*", 2, ["-", 3, 4]], 5]], 6]StackOverFlow orz
    • 11. PEG• PEG• M17N OK
    • 12. -> { restruct(r) }• Ruby → Ruby
    • 13. parse.y$ wc parse.y 10745 27247 234392 parse.y•
    • 14. Ruby Scanner parse “a <%= ‘%%>’ %>b”
    • 15. -> { restruct(r) }• Ruby → Ruby• 8 10
    • 16. ERB Ruby Scanner class TrimScanner < Scanner # :nodoc: def initialize(src, trim_mode, percent) super @trim_mode = trim_mode @percent = percent if @trim_mode == > @scan_line = self.method(:trim_line1) elsif @trim_mode == <> @scan_line = self.method(:trim_line2) elsif @trim_mode == - @scan_line = self.method(:explicit_trim_line) else @scan_line = self.method(:scan_line) end end attr_accessor :stag def scan(&block) @stag = nil if @percent @src.each_line do |line| percent_line(line, &block) end else @scan_line.call(@src, &block) end nil end def percent_line(line, &block) if @stag || line[0] != ?% return @scan_line.call(line, &block) end line[0] = if line[0] == ?% @scan_line.call(line, &block) else yield(PercentLine.new(line.chomp)) end end
    • 17. def trim_line2(line) head = nil line.scan(/(.*?)(<%%|%%>|<%=|<%#|<%|%>n|%>|n|z)/m) do |tokens| tokens.each do |token| next if token.empty? head = token unless head ERB Ruby Scanner if token == "%>n" yield(%>) if is_erb_stag?(head) yield(:cr) else yield("n") end head = nil else yield(token) head = nil if token == "n" end end end end def explicit_trim_line(line) line.scan(/(.*?)(^[ t]*<%-|<%-|<%%|%%>|<%=|<%#|<%|-%>n|-%>|%>|z)/m) do |tokens| tokens.each do |token| next if token.empty? if @stag.nil? && /[ t]*<%-/ =~ token yield(<%) elsif @stag && token == "-%>n" yield(%>) yield(:cr) elsif @stag && token == -%> yield(%>) else yield(token) end end end end ERB_STAG = %w(<%= <%# <%) def is_erb_stag?(s) ERB_STAG.member?(s) end end Scanner.default_scanner = TrimScanner Scanner
    • 18. Racc Ruby Scannerdef scan_action buf = nest = 1 pre = nil @in_block = action begin pre = nil if s = reads(/As+/) # does not set pre buf << s end until @line.empty? if s = reads(/A[^"`{}%#/$]+/) buf << (pre = s) next end case ch = read(1) when { nest += 1 buf << (pre = ch) when } nest -= 1 if nest == 0 @in_block = nil return buf end buf << (pre = ch) when # # comment buf << ch << @line break when "", ", ` buf << (pre = scan_quoted(ch)) when % if literal_head? pre, @line # % string, regexp, array buf << ch case ch = read(1) when /[qQx]/n buf << ch << (pre = scan_quoted(read(1), %string)) when /wW/n
    • 19. # % string, regexp, array buf << ch case ch = read(1) when /[qQx]/n buf << ch << (pre = scan_quoted(read(1), %string))Racc Ruby Scanner when /wW/n buf << ch << (pre = scan_quoted(read(1), %array)) when /s/n buf << ch << (pre = scan_quoted(read(1), %symbol)) when /r/n buf << ch << (pre = scan_quoted(read(1), %regexp)) when /[a-zA-Z0-9= ]/n # does not include "_" scan_error! "unknown type of % literal %#{ch}" else buf << (pre = scan_quoted(ch, %string)) end else # operator buf << ||op-> if $raccs_print_type buf << (pre = ch) end when / if literal_head? pre, @line # regexp buf << (pre = scan_quoted(ch, regexp)) else # operator buf << ||op-> if $raccs_print_type buf << (pre = ch) end when $ # gvar buf << ch << (pre = read(1)) else raise racc: fatal: must not happen end end buf << "n" end while next_line() raise racc: fatal: scan finished before parser finishedend
    • 20.
    • 21. Ripper!• parse.y 1.9 Ruby•irb(main):001:0> Ripper.parse("a = b + 1")=> nil
    • 22. Ripper• test/ripper/test_files.rbclass Parser < Ripper PARSER_EVENTS.each {|n| eval "def on_#{n}(*args) r = [:#{n}, *args]; r.inspect; Object.new end" } SCANNER_EVENTS.each {|n| eval "def on_#{n}(*args) r = [:#{n}, *args]; r.inspect; Object.new end" }end•
    • 23. Ripper Scanner class RubyCodeDetector < Ripper class ParseError < Exception attr_reader :message,:result def initialize(r,msg) @result = r @message = msg end end def on_parse_error(msg) l = @lines[lineno-1][0,column-1] src = (@lines[0,lineno-1]+[l])*"n" src.force_encoding(@src.encoding) raise ParseError.new(src,msg) end def initialize(*args) @src=args[0] @lines=@src.split(/n/).map{|i| i.force_encoding("ASCII-8BIT") } super end def parse super raise ParseError.new(@src,"syntax error, unexpected $end, expecting }") nil rescue ParseError => e if e.message =~ /G syntax error, unexpected }/ e.result else raise $! nil end end end
    • 24. Ripper Scanner class RubyCodeDetector < Ripper class ParseError < Exception attr_reader :message,:result def initialize(r,msg) @result = r @message = msg end end def on_parse_error(msg) l = @lines[lineno-1][0,column-1] src = (@lines[0,lineno-1]+[l])*"n" src.force_encoding(@src.encoding) raise ParseError.new(src,msg) end def initialize(*args) @src=args[0] @lines=@src.split(/n/).map{|i| i.force_encoding("ASCII-8BIT") } super end def parse super raise ParseError.new(@src,"syntax error, unexpected $end, expecting }") nil rescue ParseError => e if e.message =~ /G syntax error, unexpected }/ e.result else raise $! nil end end end
    • 25. • Regexp• Rex• PEG ( Treetop , Citrus , Parslet )• Ripper• Bootstrap• M17N• ERB• Racc
    • 26. • Ripper 1.9• Ripper

    ×