Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20130530-PEGjs

1,923 views

Published on

Published in: Software, Technology, Education
  • Be the first to comment

  • Be the first to like this

20130530-PEGjs

  1. 1. PEG.js - Javascript Parser Generator - Hidetomo Suzuki
  2. 2. 2013 / 5 / 30 Scope Page 2 This MTG Target Understand What’s PEG.js Know How to Use PEG.js Make New DSL and Parser
  3. 3. 2013 / 5 / 30 Agenda Page 3 Table of Contents 1.How to make parser 2. What’s PEG.js 3. Try to used to be PEG 4. Let’s make DSL 5.Applications
  4. 4. 2013 / 5 / 30 Agenda Page 4 Table of Contents 1.How to make parser 2. What’s PEG.js 3. Try to used to be PEG 4. Let’s make DSL 5.Applications
  5. 5. 2013 / 5 / 30 Page 5 1. How to make parser 1. Lexical Analysis(字句解析) 2. Syntactic Parsing(構文解析) 3. Semantic Analysis(意味解析) 4. Intermidiate Code Generation(中間コード生成) 5. Code Optimization(コード最適化) 6. Code Generation(コード生成) General Step of Processing with Compiler Parser
  6. 6. 2013 / 5 / 30 Page 6 1. How to make parser 1. Lexical Analysis(字句解析) 2. Syntactic Parsing(構文解析) 3. Semantic Analysis(意味解析) Analysis for Make Parser Make strings(minimum unit of string has semantic) from characters Make tree structure from strings which result of Lexical Analysis Type Check ※ A string of Lexical Analysis is called “Token”.
  7. 7. 2013 / 5 / 30 Page 7 1. How to make parser 1. Lexical Analysis position_x = position_x + 2.0 * time  Identifier : position_x  Substitution Symbol : =  Identifier : position_x  Addition Symbol : +  Number : 2.0  Multiple Symbol : *  Identifier : time
  8. 8. 2013 / 5 / 30 Page 8 1. How to make parser 2. Syntactic Parsing  Identifier : position_x Substitution Symbol : =  Identifier : position_x Addition Symbol : +  Number : 2.0 Multiple Symbol : *  Identifier : time Substitution Symbol Identifier position_x Expression ExpressionExpression Expression Expression Identifier position_x Number 2.0 Identifier time = + *
  9. 9. 2013 / 5 / 30 Page 9 1. How to make parser 3. Semantic Analysis Substitution Symbol Identifier position_x Expression ExpressionExpression Expression Expression Identifier position_x Number Identifier = +  Correct Case : Real Number * Integer Number  Wrong Case : Real Number * Function Pointer 2.0 time *
  10. 10. 2013 / 5 / 30 Agenda Page 10 Table of Contents 1.How to make parser 2. What’s PEG.js 3. Try to used to be PEG 4. Let’s make DSL 5.Applications
  11. 11. 2013 / 5 / 30 Here comes your footer  2. What‘s PEG.js?  Parser Generator for JavaScript  Use PEG (Parsing Expression Grammar)  Easy to try with the web page Outline of PEG.js http://pegjs.majda.cz/
  12. 12. 2013 / 5 / 30 Here comes your footer  2. What‘s PEG.js? PEG(Parsing Expression Grammar) is one of grammar for artificial language. Focus point is how input will be analyzed. What‘s PEG Ex) Simple Expression Grammar which Has Four Arithmetic Operations expression <- addexp addexp <- multiexp (“+” multiexp / “-” multiexp)* multiexp <- number (“*” number / “/” number)* number <- [0-9]+ Accept : 2*3+6/2-5*3 -> multiexp + multiexp – multiexp -> ・・・ Decline : (1+1)*3+6/2-(7-2)*3 -> ?*3+6/2-?*3 -> can’t recognize…
  13. 13. 2013 / 5 / 30 Here comes your footer  2. What‘s PEG.js? • Nothing Confusion • Different : “a b / a”, “a / a b” • Easy to use compared with CFG • Don’t need to implement scanner (Lexical Analyzer) • Don’t Express Left Recursive PEG vs CFG(Context Free Grammar) • Confusion Existing • Same : “a b | a”, “a | a b” • Need to implement scanner • Can Express Left Recursive PEG CFG
  14. 14. 2013 / 5 / 30 Page 14 2. What‘s PEG.js? 1. Make Grammar with PEG 2. Generate Parser with “pegjs” Command 3. Use the Parser Loaded as a Library Step of Make Parser with PEG.js vim filename.pegjs Pegjs filename.pegjs parser.js <script src=“./parser.js” type=“text/javascript”></script> <script type=“text/javascript”> parser.parse(“something”);
  15. 15. 2013 / 5 / 30 Page 15 2. What‘s PEG.js? Install PEG.js # Install Pythonbrew $ curl -kL http://xrl.us/pythonbrewinstall | bash $ vim ~/.bash_profile [[ -s $HOME/.pythonbrew/etc/bashrc ]] && source $HOME/.pythonbrew/etc/bashrc $ source ~/.bashrc $ pythonbrew install 2.7.2 $ pythonbrew switch 2.7.2 # Install Node $ git clone git://github.com/creationix/nvm.git ~/.nvm $ nvm install v0.8.7 $ source ~/.nvm/nvm.sh $ nvm use v0.8.7 $ vim ~/.npmrc $ registry = http://registry.npmjs.org/
  16. 16. 2013 / 5 / 30 Page 16 2. What‘s PEG.js? 1. Make Grammar with PEG 2. Generate Parser with “pegjs” Command 3. Use the Parser Loaded as a Library Let’s use PEG.js a little bit $ vim ffirst.pegjs start = addexp addexp = integer ("+" integer / "-" integer)* integer = [0-9]+ $ ./node_modules/pegjs/bin/pegjs first.pegjs parser.js $ vim parser.js 1st line: module.export = … -> var parser = … $ vim index.html <script type=“text/javascript” src=“./parser.js”></script> <script type=“text/javascript”> alert(parser.parse(“1+2+3-5”)); </script>
  17. 17. 2013 / 5 / 30 Agenda Page 17 Table of Contents 1.How to make parser 2. What’s PEG.js 3. Try to used to be PEG 4. Let’s make DSL 5.Applications
  18. 18. 2013 / 5 / 30 Page 18 3.Try to used to be PEG PEG.js Online Version (http://pegjs.majda.cz/online)
  19. 19. 2013 / 5 / 30 Page 19 3. Try to used to be PEG Very Simple PEG.js Grammar(calc : integer) Grammar Test Input Test Input Result start = integer integer = digits:[0-9]+ { return parseInt(digits.join(“”), 10); } 123456789 123456789
  20. 20. 2013 / 5 / 30 Page 20 3. Try to used to be PEG Very Simple PEG.js Grammar(calc : add two integer) Grammar Test Input Test Input Result start = target1:integer "+" target2:integer { return target1+target2; } integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); } 81+19 100
  21. 21. 2013 / 5 / 30 Page 21 3. Try to used to be PEG Very Simple PEG.js Grammar(calc : add and minus two integer) Grammar Test Input Test Input Result start = target1:integer "+" target2:integer { return target1+target2; } / target1:integer "-" target2:integer { return target1-target2; } integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); } 100-19 81
  22. 22. 2013 / 5 / 30 Page 22 3. Try to used to be PEG Very Simple PEG.js Grammar(calc : add and minus integers) Grammar Test Input Test Input Result start = expression expression = ope1:integer "+" ope2:expression { return ope1+ope2; } / ope1:integer "-" ope2:expression { return ope1-ope2; } / ope:integer { return ope; } integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); } 1+2+3+4+5-6+7-8+9-10 9
  23. 23. 2013 / 5 / 30 Page 23 3. Try to used to be PEG Very Simple PEG.js Grammar(calc : four arithmetic) Grammar Test Input Test Input Result start = expression expression = ope1:integer "+" ope2:expression { return ope1+ope2; } / ope1:integer "-" ope2:expression { return ope1-ope2; } / ope1:integer "*" ope2:expression { return ope1*ope2; } / ope1:integer "/" ope2:expression { return ope1/ope2; } / ope:integer { return ope; } integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); } 2*3/2 3
  24. 24. 2013 / 5 / 30 Page 24 3. Try to used to be PEG Very Simple PEG.js Grammar(calc : four arithmetic with priority) Grammar Test Input Test Input Result start = expression expression = ope1:multiple "+" ope2:expression { return ope1+ope2; } / ope1:multiple "-" ope2:expression { return ope1-ope2; } / ope:multiple { return ope; } multiple = ope1:integer "*" ope2:multiple { return ope1*ope2; } / ope1:integer "/" ope2:multiple { return ope1/ope2; } / ope:integer { return ope; } integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); } 2*3/2+8/4 5
  25. 25. 2013 / 5 / 30 Page 25 3. Try to used to be PEG Very Simple PEG.js Grammar(calc : four arithmetic with priority) Grammar Test Input Test Input Result start = expression expression = ope1:multiple "+" ope2:expression { return ope1+ope2; } / ope1:multiple "-" ope2:expression { return ope1-ope2; } / ope:multiple { return ope; } multiple = ope1:bracket "*" ope2:multiple { return ope1*ope2; } / ope1:bracket "/" ope2:multiple { return ope1/ope2; } / ope:bracket { return ope; } bracket = "(" exp:expression ")" {return exp; } / ope:integer {return ope; } integer = digits:[0-9]+ { return parseInt(digits.join(""), 10); } (7+3)*3/2+8/(6-2) 17
  26. 26. 2013 / 5 / 30 Agenda Page 26 Table of Contents 1.How to make parser 2. What’s PEG.js 3. Try to used to be PEG 4. Let’s make DSL 5.Applications
  27. 27. 2013 / 5 / 30 Page 27 4. Let‘s make DSL Config File Format Parser Result Web Page Image Target <LogicalExpression?>Y---[function param] N---[function param] if (isLogicalExpression) function(param); else function(param); Ex) <Hydea?>Y---[show ‘hydea’] N---[show ‘hydeb’] Ex) if (isHydea) show(‘hydea’); else show(‘hydeb’); URL: ex.com/?p=hydea URL: ex.com/?p=hydeb
  28. 28. 2013 / 5 / 30 Page 28 4. Let‘s make DSL Just write over specification Make Grammar start = "<" logic "?>Y---[" function " " param "]nN---[" function " " param "]" logic = [a-zA-Z]+ function = [a-zA-Z]+ param = "'" [a-zA-Z]+ "'"
  29. 29. 2013 / 5 / 30 Page 29 4. Let‘s make DSL Refactoring a little bit Make Grammar start = "<" identifier "?>Y---[" identifier " " param "]nN---[" identifier " " param "]" param = "'" identifier "'" identifier = [a-zA-Z]+
  30. 30. 2013 / 5 / 30 Page 30 4. Let‘s make DSL Finish to make param and identifier part Make Grammar start = "<" identifier "?>Y---[" identifier " " param "]nN---[" identifier " " param "]" param = "'" id:identifier "'" {return "'" + id + "'"; } identifier = chars:[a-zA-Z]+ {return chars.join(""); }
  31. 31. 2013 / 5 / 30 Page 31 4. Let‘s make DSL Organize Function Part Make Grammar start = "<" identifier "?>Y---[" function "]nN---[" function "]" function = id:identifier " " param:param {return id + "(" + param + ")"; } param = "'" id:identifier "'" {return "'" + id + "'"; } identifier = chars:[a-zA-Z]+ {return chars.join(""); }
  32. 32. 2013 / 5 / 30 Page 32 4. Let‘s make DSL Finish to make Make Grammar start = "<" id:identifier "?>Y---[" f1:function "]nN---[" f2:function "]" {return "if (is" + id + ") " + f1 + ";nelse " + f2 + ";"; } function = id:identifier " " param:param {return id + "(" + param + ")"; } param = "'" id:identifier "'" {return "'" + id + "'"; } identifier = chars:[a-zA-Z]+ {return chars.join(""); }
  33. 33. 2013 / 5 / 30 Agenda Page 33 Table of Contents 1.How to make parser 2. What’s PEG.js 3. Try to used to be PEG 4. Let’s make DSL 5.Applications
  34. 34. 2013 / 5 / 30 Page 34 5. Application Show My Demo(Demo’s grammar) start = tree tree = stat:statement? { return stat; } statement = stat:if_statement { return stat.toString(); } / stat:proc_statement { return stat.toString(); } if_statement = fac:if_factor 'Y' edge branch1:proc_statement 'n' 'N' 'n' edge branch2:statement { return branch2.toString().match(/if/) ? "if (" + fac + ")nt" + branch1 + 'nelse ' + branch2 : 'if (' + fac + ')nt' + branch1 + 'nelsent' + branch2;} proc_statement = fac1:proc_factor [-|]+ fac2:proc_statement { return fac1 + '.then(' + fac2.slice(0,fac2.length-1) + ');'; } / factor:proc_factor { return factor + ';'; } proc_factor = '[' procexp:expression ']' { return procexp; } if_factor = '<' ifexp:expression '>' { return ifexp; } expression = elem:element ' ' cdr:arguments { return elem.toString() + '(' +cdr + ')'; } / elem:element { return elem.toString(); } arguments = elem:element ',' arg:arguments { return elem.toString() + ',' + arg; } / elem:element { return elem.toString(); } element = characters:[a-zA-Z0-9-_."]+ { return characters.join('') } edge = symbols:[-|n]+ { return symbols.join('') }
  35. 35. 2013 / 5 / 30 Page 35 5. Application Recommendation Conf1 Conf2 Conf3 ・・・ Recommendation Engine Choose a config file
  36. 36. 2013 / 5 / 30 Reference Page 36 Famous Book for Compiler
  37. 37. 2013 / 5 / 30 Thank you Page 37 Thank you for your listening

×