Creating own language made easy

5,393 views
4,993 views

Published on

Slides from my talk from #jQueryBulgaria 2013

0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,393
On SlideShare
0
From Embeds
0
Number of Embeds
1,199
Actions
Shares
0
Downloads
31
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

Creating own language made easy

  1. 1. Creating own language made easy Ingvar Stepanyan @RReverser
  2. 2. Everything sucks
  3. 3. Everything sucks – human version
  4. 4. Everything sucks – developer version
  5. 5. Scary magic ??? ?? ?
  6. 6. Not so scary magic JS JS JS Parser Generator
  7. 7. Parsers
  8. 8. Parser generators  jison Bison in javascript, used by Coffeescript  PEG.js parser generator for JavaScript based on the parsing expression grammar formalism  OMeta/JS (source) metacompiler for many languages to many targets, including js.  languagejs - PEG parser written in JavaScript with first class errors  Canopy Self-hosting PEG parser compiler for JavaScript, influenced by Ruby parser generators such as Treetop and Citrus  JS/CC LALR(1) parser generator  jsparse PEG by Grandmaster Chris Double  ReParse parser combinator library for Javascript like Haskell's Parsec  p4js Monadic parser library for JavaScript  JSGLR Scannerless, Generalized Left-to-right Rightmost (SGLR) derivation parser for JavaScript  antlr has a javascript target  Cruiser.Parse LL(k) parser
  9. 9. Parser generators Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  10. 10. Choice ordering Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  11. 11. Ambiguity if a then (if b then s) else s2 or if a then if b then s else s2: if a then (if b then s else s2) Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  12. 12. Left recursion x=1-2-3 Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  13. 13. Left recursion x=1-2-3 Top-down (Jison) Bottom-up (PEG.js) [ [ "x", "=", [ [ "1", "-", "2" ], "-", "3" ] "x", "=", [ "1", "-", [ "2", "-", "3" ] ] ] ]
  14. 14. Summary choice Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  15. 15. Jison syntax: helpers %{ var scope = {}; %} …
  16. 16. Jison syntax: lexer … %lex %% s+ /* skip whitespace */ [A-Za-z_]w+ return 'ID'; d+ return ‘NUMBER’; [+*;=] return yytext; <<EOF>> return 'EOF'; /lex …
  17. 17. Jison syntax: operator precedence … %left ';‘ %right ‘=‘ %left ‘+’ %left ‘*’ … /* “x=a*b+c” -> assign(“x”, “a*b+c”) -> assign(“x”, add(“a*b”, “c”)) -> assign(“x”, add(mul(“a”, “b”), “c”)) */
  18. 18. Jison syntax: rules %start program program : stmt* EOF { return $1 } ; stmt : expr ‘;’ -> $1 ; expr : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | NUMBER -> $1 ;
  19. 19. Code generation
  20. 20. Methods  string concatenation  building AST object + escodegen (http://github.com/Constellation/escodegen)  using SourceNode from source-map (https://github.com/mozilla/source-map)
  21. 21. Debugging: source maps
  22. 22. Methods  string concatenation  building AST object + escodegen (http://github.com/Constellation/escodegen)  using SourceNode from source-map (https://github.com/mozilla/source-map)
  23. 23. AST way
  24. 24. Methods  string concatenation  building AST object + escodegen (http://github.com/Constellation/escodegen)  using SourceNode from source-map (https://github.com/mozilla/source-map)
  25. 25. SourceNode  new SourceNode(line, column, filename, jsChunk)  line, column – position in original file  filename – name of original file  jsChunk – JavaScript code string, another SourceNode instance or array of those
  26. 26. Resulting stack JS JS JS Jison sourcemap
  27. 27. Live demo

×