Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lint, coverage, doc, autocompletion, transpilation, minification... powered by ast

251 views

Published on

Today's Web Development tooling is now a very rich ecosystem that allows very professional workflows, and the fun thing is that they all share a very interesting technology called AST, for "Abstract syntax tree".

That's so powerful that some people see it as a very complex magic thing. Now the fun part... It is in fact quite simple to understand., and Today's libraries (Arcon, Esprima, Espree, Babylon) allows anyone to create their own tools, fork existing ones, or write plugins matching our own desires.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Lint, coverage, doc, autocompletion, transpilation, minification... powered by ast

  1. 1. Lint, Coverage, Doc, Autocompletion, Transpilation, Minification... powered by AST Alexandre Morgaut NantesJS Meetup #39 - April 2019
  2. 2. ● Ealy JavaScript Adopter (1996) ● Former Wakanda Server-Side JS platform Team (2008) ● JS & Web Standards Evangelist since 2009 ● Former member of CommonJS (2010) ● Former W3C Advisory Committee member (2012) ● Initiated NantesJS (2012) ● Joined the Wiztivi company (2015) Presentation @amorgaut wiztivi.comAMorgaut
  3. 3. Summary ● Compilers ● BinAST ● Editors ● JavaScript AST ● Using AST in Plugins ● Task Managers ● AST Explorer
  4. 4. Compilers
  5. 5. http://www.cs.man.ac.uk/~pjj/farrell/comp3.html Anatomy of a Compiler
  6. 6. http://www-igm.univ-mlv.fr/~forax/ens/java-avance/cours/pdf/old/Generation%20de%20code%201.pdf
  7. 7. Some Token Types Name Examples identifiers x, color, UP​ keywords if, while, return​ punctuation }, (, ; separators +, <, =​ literals true, 6.02e23, "music"​ comments // en ligne, /* en bloc */​
  8. 8. BinAST
  9. 9. Binary AST ● Give the JS engine a pre-tokenized, pre-parsed, script content ● Give it in binary format to make it lighter ● Current advanced prototypes already show JS parsing improvements of 30%-50% on all the most common frameworks https://github.com/tc39/proposal-binary-ast https://github.com/binast/binjs-ref
  10. 10. https://yoric.github.io/Fosdem-2019/
  11. 11. Editors
  12. 12. Editor Features ● With Tokenizer Help ○ Syntax highlighting ○ Collapse / Expand ● With AST Help ○ Lint ○ Find Usages ○ Code Completion ○ Reformat ○ Statement level breakpoints
  13. 13. ACE - The Cloud9 Editor // ### Abstract Syntax Tree // // The default AST structure is inspired by the Mozilla Parser API but can // easily be customized by overriding these functions. // Lexer // ----- // // The lexer, or the tokenizer reads the input string character by character // and derives a token left-right. To be as efficient as possible the lexer // prioritizes the common cases such as identifiers. It also works with // character codes instead of characters as string comparisons was the // biggest bottleneck of the parser. https://github.com/ajaxorg/ace/blob/9b5b63d1dc7c1b81b58d30c87d14b5905d030ca5/lib/ace/mode/lua/luaparse.js
  14. 14. ternjs code-analysis engine for JavaScript ● Autocompletion on variables and properties ● Function argument hints ● Querying the type of an expression ● Finding the definition of something ● Automatic refactoring Built-in in the Bracket Editor (Adobe) https://ternjs.net/
  15. 15. JavaScript AST
  16. 16. SpiderMonkey engine's JavaScript parser API https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/Parser_API
  17. 17. Parsers ● espree ○ ESLint, estraverse, hint, javascript-obfuscator, JSDoc, protobuf.js, v8/web-tooling-benchmark ● babylon / babel-parser ○ babel, ESDoc ● esprima ○ previous ESLint, flow, istanbul ● acorn ○ espree, buble, browserify, ternjs, webpack, optimize-js
  18. 18. Espree ● Fork of Esprima (v1.2.2) ○ Primary goal is to produce the exact same AST structure and tokens as Esprima ○ The AST structure being the ESTree API with JSX extensions ● Built on top of Acorn ○ use its modular architecture -> extensions of core functionalities ● Maintained by the ESLint project https://github.com/eslint/espree const ast = espree.parse(code);
  19. 19. ESTree "This site will serve as a community standard" "keep up with the evolution of the JavaScript language" https://github.com/estree/estree
  20. 20. AST Example
  21. 21. Using AST in Plugins
  22. 22. ESLint Rule Enforce a maximum number of parameters in function definitions https://eslint.org/docs/rules/max-params https://github.com/eslint/eslint/blob/master/lib/rules/max-params.js
  23. 23. Rule Option Management Rule Validation Code AST Node Type Binding
  24. 24. Babel Transform Plugin babel-plugin-transform-class A minimalist ES6 class babel transformer ● Remove ES6 forbidden actions controls ○ ex: throw on this modification before calling super() ○ Encourage to check that at edit time using ESLint rules ● Remove Polyfill generations ○ Generate Full ES5 compatible code ○ Even compatible with ES3 using very minimalist Polyfills https://github.com/AMorgaut/babel-plugin-transform-class
  25. 25. Return a visitor object ● keys: AST Node types ● values: callback or map of callbacks ○ enter: beginning of the node block ○ exit: end of the node block babel.types ● provides useful helpers https://github.com/AMorgaut/babel-plugin-transform-class/blob/master/src/index.js
  26. 26. Task Managers
  27. 27. Task Manager Grunt, Gulp, npm, ... webpack ● $ npm run build ○ lint > transpile* > concat* > minify* ○ * with SourceMap ○ each step using it's own AST ● Some of those tools (ex: babel) can use an existing AST
  28. 28. AST Explorer
  29. 29. https://astexplorer.net https://astexplorer.net/#/gist/0178b41edea28820b2452e3422059cbd/latest Input Source Transform Code Input AST Output Source
  30. 30. AST Explorer 1. Select (or use default) a: a. Source Type (JavaScript, CSS, HTML, …) b. Parser Lib (babel, eslint, …) c. (Optional) Transformer Lib 2. Provide a Source Code a. A Browsable matching AST is automatically generated 3. Provide a Transform Code a. An Output Source is automatically updated b. Error messages code be shown instead to help fixing the code
  31. 31. Questions ?
  32. 32. Thank you !

×