Your SlideShare is downloading. ×
0
Creating own language
made easy
Ingvar Stepanyan
@RReverser
Everything sucks
Everything sucks – human version
Everything sucks – developer version
Scary magic

???
??

?
Not so scary magic

JS

JS

JS
Parser
Generator
Parsers
Parser generators


jison Bison in javascript, used by Coffeescript



PEG.js parser generator for JavaScript based on t...
Parser generators
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
= left:multiplicati...
Choice ordering
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
= left:multiplicative...
Ambiguity
if a then (if b then s) else s2
or
if a then if b then s else s2:
if a then (if b then s else s2)

Top-down (Jis...
Left recursion
x=1-2-3
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
= left:multipl...
Left recursion
x=1-2-3
Top-down (Jison)

Bottom-up (PEG.js)
[

[

"x",
"=",
[
[
"1",
"-",
"2"
],
"-",
"3"
]

"x",
"=",
[
"...
Summary choice
Top-down (Jison)

Bottom-up (PEG.js)
start
= additive

%left "+"
%left "*"

additive
= left:multiplicative ...
Jison syntax: helpers
%{

var scope = {};
%}
…
Jison syntax: lexer
…

%lex
%%
s+

/* skip whitespace */

[A-Za-z_]w+

return 'ID';

d+

return ‘NUMBER’;

[+*;=]

return ...
Jison syntax: operator precedence
…

%left ';‘
%right ‘=‘
%left ‘+’
%left ‘*’

…
/*
“x=a*b+c”
-> assign(“x”, “a*b+c”)
-> a...
Jison syntax: rules
%start program
program
: stmt* EOF { return $1 }
;
stmt
: expr ‘;’ -> $1
;
expr
: expression "+" expre...
Code generation
Methods
 string concatenation
 building AST object + escodegen (http://github.com/Constellation/escodegen)
 using Sourc...
Debugging: source maps
Methods
 string concatenation
 building AST object + escodegen (http://github.com/Constellation/escodegen)
 using Sourc...
AST way
Methods
 string concatenation
 building AST object + escodegen (http://github.com/Constellation/escodegen)
 using Sourc...
SourceNode
 new SourceNode(line, column, filename, jsChunk)
 line, column – position in original file
 filename – name ...
Resulting stack

JS

JS

JS
Jison
sourcemap
Live demo
Creating own language made easy
Upcoming SlideShare
Loading in...5
×

Creating own language made easy

4,037

Published on

Slides from my talk from #jQueryBulgaria 2013

Transcript of "Creating own language made easy"

  1. 1. Creating own language made easy Ingvar Stepanyan @RReverser
  2. 2. Everything sucks
  3. 3. Everything sucks – human version
  4. 4. Everything sucks – developer version
  5. 5. Scary magic ??? ?? ?
  6. 6. Not so scary magic JS JS JS Parser Generator
  7. 7. Parsers
  8. 8. Parser generators  jison Bison in javascript, used by Coffeescript  PEG.js parser generator for JavaScript based on the parsing expression grammar formalism  OMeta/JS (source) metacompiler for many languages to many targets, including js.  languagejs - PEG parser written in JavaScript with first class errors  Canopy Self-hosting PEG parser compiler for JavaScript, influenced by Ruby parser generators such as Treetop and Citrus  JS/CC LALR(1) parser generator  jsparse PEG by Grandmaster Chris Double  ReParse parser combinator library for Javascript like Haskell's Parsec  p4js Monadic parser library for JavaScript  JSGLR Scannerless, Generalized Left-to-right Rightmost (SGLR) derivation parser for JavaScript  antlr has a javascript target  Cruiser.Parse LL(k) parser
  9. 9. Parser generators Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  10. 10. Choice ordering Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  11. 11. Ambiguity if a then (if b then s) else s2 or if a then if b then s else s2: if a then (if b then s else s2) Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  12. 12. Left recursion x=1-2-3 Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  13. 13. Left recursion x=1-2-3 Top-down (Jison) Bottom-up (PEG.js) [ [ "x", "=", [ [ "1", "-", "2" ], "-", "3" ] "x", "=", [ "1", "-", [ "2", "-", "3" ] ] ] ]
  14. 14. Summary choice Top-down (Jison) Bottom-up (PEG.js) start = additive %left "+" %left "*" additive = left:multiplicative "+" right:additive { return left + right; } / multiplicative %start program; multiplicative = left:primary "*" right:multiplicative { return left * right; } / primary program : expression { return $$ } ; primary = integer / "(" additive:additive ")" { return additive; } integer "integer" = digits:[0-9]+ { return parseInt(digits.join(""), 10); } %% expression : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | "(" expression ")" -> $2 | NUMBER -> $1 ;
  15. 15. Jison syntax: helpers %{ var scope = {}; %} …
  16. 16. Jison syntax: lexer … %lex %% s+ /* skip whitespace */ [A-Za-z_]w+ return 'ID'; d+ return ‘NUMBER’; [+*;=] return yytext; <<EOF>> return 'EOF'; /lex …
  17. 17. Jison syntax: operator precedence … %left ';‘ %right ‘=‘ %left ‘+’ %left ‘*’ … /* “x=a*b+c” -> assign(“x”, “a*b+c”) -> assign(“x”, add(“a*b”, “c”)) -> assign(“x”, add(mul(“a”, “b”), “c”)) */
  18. 18. Jison syntax: rules %start program program : stmt* EOF { return $1 } ; stmt : expr ‘;’ -> $1 ; expr : expression "+" expression -> $1 + $3 | expression "*" expression -> $1 * $3 | NUMBER -> $1 ;
  19. 19. Code generation
  20. 20. Methods  string concatenation  building AST object + escodegen (http://github.com/Constellation/escodegen)  using SourceNode from source-map (https://github.com/mozilla/source-map)
  21. 21. Debugging: source maps
  22. 22. Methods  string concatenation  building AST object + escodegen (http://github.com/Constellation/escodegen)  using SourceNode from source-map (https://github.com/mozilla/source-map)
  23. 23. AST way
  24. 24. Methods  string concatenation  building AST object + escodegen (http://github.com/Constellation/escodegen)  using SourceNode from source-map (https://github.com/mozilla/source-map)
  25. 25. SourceNode  new SourceNode(line, column, filename, jsChunk)  line, column – position in original file  filename – name of original file  jsChunk – JavaScript code string, another SourceNode instance or array of those
  26. 26. Resulting stack JS JS JS Jison sourcemap
  27. 27. Live demo
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×