The document discusses abstract syntax trees (ASTs) and parsing of code. It provides examples of parsing JavaScript code into tokens and then building a syntax tree. It also discusses common JavaScript parsers like Esprima and Acorn, and AST specifications like ESTree. Homomorphic parsing is shown where code and AST are equivalent. Common uses of ASTs include transpilation, linting, minification, and code transformation.

1. YOUR CODE IS NOT A STRING
INGVAR STEPANYAN

2. YOUR CODE IS NOT A STRING
SOURCE MAPS - EPIC WIN IN DEBUGGING

3. YOUR CODE IS NOT A STRING
SOURCE MAPS - EPIC WIN IN DEBUGGING

4. YOUR CODE IS NOT A STRING
BUILDERS - EPIC FAIL FOR DEBUGGING
umdify:
// UMD definition
output += '(function(root, factory) {n';
output += ' if (typeof define === "function" && define.amd) {n';
output += ' define([' + depNames.join(', ') + '], factory);n';
output += ' } else if (typeof exports === "object") {n';
output += ' module.exports = factory(require);n';
…

5. YOUR CODE IS NOT A STRING
BUILDERS - EPIC FAIL FOR DEBUGGING
grunt-amd-wrap:
var srcText = grunt.file.read(file.src[0]);
var destText = amdWrap(srcText);

6. YOUR CODE IS NOT A STRING
BUILDERS - EPIC FAIL FOR DEBUGGING
super-duper-universal-transformer:
function transform(srcText) {
return 'var answer = 42;';
}

7. IT HAS A SOUL
YOUR CODE IS NOT A STRING

8. YOUR CODE IS NOT A STRING
YOU ARE PERFORMING PARSING RIGHT NOW

9. YOUR CODE IS NOT A STRING
YOU ARE PERFORMING PARSING RIGHT NOW

10. YOUR CODE IS NOT A STRING
WE ARE ALL PARSERS
In computer science, an abstract syntax tree (AST), or just
syntax tree, is a tree representation of the abstract syntactic
structure of source code written in a programming
language. Each node of the tree denotes a construct
occurring in the source code.
The syntax is "abstract" in not representing every detail
appearing in the real syntax. For instance, grouping
parentheses are implicit in the tree structure, and a syntactic
construct like an if-condition-then expression may be
denoted by means of a single node with three branches.

11. YOUR CODE IS NOT A STRING
WE ARE ALL PARSERS
In computer science, an abstract syntax tree (AST), or just
syntax tree, is a tree representation of the abstract syntactic
structure of source code written in a programming
language. Each node of the tree denotes a construct
occurring in the source code.
The syntax is "abstract" in not representing every detail
appearing in the real syntax. For instance, grouping
parentheses are implicit in the tree structure, and a syntactic
construct like an if-condition-then expression may be
denoted by means of a single node with three branches.
Paragraphs

12. YOUR CODE IS NOT A STRING
WE ARE ALL PARSERS
In computer science, an abstract syntax tree (AST), or just
syntax tree, is a tree representation of the abstract syntactic
structure of source code written in a programming
language. Each node of the tree denotes a construct
occurring in the source code.
The syntax is "abstract" in not representing every detail
appearing in the real syntax. For instance, grouping
parentheses are implicit in the tree structure, and a syntactic
construct like an if-condition-then expression may be
denoted by means of a single node with three branches.
Paragraphs
Sentences

13. YOUR CODE IS NOT A STRING
WE ARE ALL PARSERS
In computer science, an abstract syntax tree (AST), or just
syntax tree, is a tree representation of the abstract syntactic
structure of source code written in a programming
language. Each node of the tree denotes a construct
occurring in the source code.
The syntax is "abstract" in not representing every detail
appearing in the real syntax. For instance, grouping
parentheses are implicit in the tree structure, and a syntactic
construct like an if-condition-then expression may be
denoted by means of a single node with three branches.
Paragraphs
Words and Punctuators

14. YOUR CODE IS NOT A STRING
HOW DO WE READ TEXTS?
WORDS AND PUNCTUATION SENTENCES PARAGRAPHSPHRASES

15. YOUR CODE IS NOT A STRING
HOW DO WE READ CODE?
WORDS AND PUNCTUATION STATEMENTS FUNCTIONSEXPRESSIONS

16. YOUR CODE IS NOT A STRING
HOW DO COMPILERS READ CODE?
WORDS AND PUNCTUATION STATEMENTS FUNCTIONSEXPRESSIONS

17. YOUR CODE IS NOT A STRING
HOW DO COMPILERS READ CODE?
WORDS AND PUNCTUATION STATEMENTS FUNCTIONSEXPRESSIONS
LEXING PARSING
(TOKENS) (SYNTACTIC NODES)

18. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
var answer = 42;

19. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
76 61 72 20 61 6E 73 77 65 72 20 3D 20 34 32 3B

20. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ v

21. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ va

22. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var

23. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var

24. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ a

25. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ an

26. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ ans

27. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answ

28. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answe

29. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answer

30. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answer

31. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answer
‣ =

32. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answer
‣ =

33. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answer
‣ =
‣ 4

34. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answer
‣ =
‣ 42

35. YOUR CODE IS NOT A STRING
LEXING INTO TOKENS
v a r a n s w e r = 4 2 ;
‣ var
‣ answer
‣ =
‣ 42
‣ ;

36. YOUR CODE IS NOT A STRING
PARSING INTO SYNTAX TREE
‣ var
‣ answer
‣ =
‣ 42
‣ ;
Literal
Identifier
VariableDeclarator VariableDeclaration

37. YOUR CODE IS NOT A STRING
FILE-BASED BUILDERS
Transforming
AST
Generating
code
Parsing
code
Reading
file
Writing
file
Plugin

38. YOUR CODE IS NOT A STRING
STREAMING BUILDERS
Transforming
AST
Generating
code
Parsing
code
Reading
file
Writing
file
Plugin

39. YOUR CODE IS NOT A STRING
TRANSPILERS
Transforming
AST
Generating
code
Parsing
code
Reading
file
Writing
file
Plugin

40. YOUR CODE IS NOT A STRING
AST TOOLING
▸ Minifiers (UglifyJS supports transforming AST)
▸ Linters (you can write own rules for ESLint!)
▸ Babel (you can write own transpiler plugins!)
▸ Facebook Codemod (you can patch entire codebase!)
▸ Any-to-JS compilers (CoffeeScriptRedux, JSX, …)
▸ Bundlers (WebPack, Browserify combine ASTs)

41. YOUR CODE IS NOT A STRING
HOMOICONICITY (CODE == AST)
(+ (+ 2 3) 7)

42. YOUR CODE IS NOT A STRING
HOMOICONICITY (CODE == AST)
<html>
<head>
<title>your title here</title>
</head>
<body bgcolor="ffffff">
<center>
<img src="clouds.jpg" align="bottom" />
</center>
<hr />
<a href="http://somegreatsite.com">link name</a> is a link
<h1>this is a header</h1>
<h2>this is a medium header</h2>
<a href="mailto:support@yourcompany.com">support@yourcompany.com</a>
</body>
</html>

43. YOUR CODE IS NOT A STRING
ECMASCRIPT SYNTAX SPEC
http://rreverser.com/ecmascript-syntax-graph/

44. YOUR CODE IS NOT A STRING
ECMASCRIPT SYNTAX SPEC

45. YOUR CODE IS NOT A STRING
PARSERS PRODUCING JAVASCRIPT ASTS
‣ JavaScript
‣ SpiderMonkey: Reflect.parse – Mozilla's Parser API
‣ Esprima – most popular ECMAScript parser in JS
‣ Acorn – faster alternative ECMAScript parser in JS
‣ UglifyJS – has own parser with custom AST format
‣ Traceur – has ES6 parser that can be used separately as well
‣ …
‣ CoffeeScript
‣ CoffeeScriptRedux – rewrite of CS compiler that internally uses CS AST with conversion
to JS AST
‣ JSX
‣ esprima-fb – Facebook's fork of Esprima Harmony branch

46. YOUR CODE IS NOT A STRING
SPIDERMONKEY PARSER API

47. YOUR CODE IS NOT A STRING
ESTREE - JAVASCRIPT AST SPECIFICATION

48. YOUR CODE IS NOT A STRING
ESTREE - JAVASCRIPT AST SPECIFICATION
https://github.com/estree/estree
‣ Literal { value, raw }
‣ Identifier { name }
‣ MemberExpression { object, property, computed }
‣ BinaryExpression { left, operator, right }
‣ …

49. YOUR CODE IS NOT A STRING
ASTEXPLORER - AST PLAYGROUND
http://astexplorer.net/

50. YOUR CODE IS NOT A STRING
BABEL PLUGIN TEMPLATE
https://github.com/RReverser/babel-plugin-hello-world

51. QUESTIONS?
@RReverser