1.
JavaScript Parser Infrastructure forCode Quality Analysis Ariya Hidayat Twitter: @AriyaHidayat http://ariya.ofilabs.com

2.
Who Am I

3.
Do you...  have a coding style/guidelines?  use code quality tool such as JSLint or JSHint?  actively prevent performance & coverage regressions?  wish you could write your own quality rules?

4.
Code Quality

5.
High Quality: Practical Aspects Do not provoke ambiguities Avoid silly mistakes Learn better code pattern Write readable code Improve future maintenance

6.
Better Use of the Tools Not everyone is a JavaScript ninja Engineer Tools •Boring •Repetitive Feedback Cycle •Time-consuming

7.
From Spelling Checker to Grammar Enforcement Agent Your so wrong, therefore you loose! No misspelled word. Wrong choice of words!

8.
Adaptive Quality Criteria Explicit ImplicitCustomize analysis options Infer from high-quality sampleDefine new sets of rules Observe the engineer’s behavior

9.
Next-Generation Code Quality Tools To boldly analyze what no man has analyzed before...

10.
Parser Infrastructure

11.
JavaScript in the Browser User Interface Data Persistence Browser Engine Render Engine JavaScript Graphics Networking I/O Engine Stack

12.
JavaScript Engine Building Blocks Built-in objects, Runtime host objects, ... Syntax Tree Virtual Source Parser Machine/ Interpreter Fast and conservative

13.
Tokenization identifier number var answer = 42; keyword equal sign semicolon

14.
Syntax Tree Variable Declaration Identifier Literal Constant answer 42

15.
JavaScript Parser Written in JavaScript Esprima UglifyJS Narcissus ZeParser Traceur Es-Lab

16.
Produce Correct Output  ECMA-262 compliant use strict;  Automatic semicolon insertion var 日本語 = 1  Strict Mode, e.g. “use strict” return 日本語  Unicode for identifiers

17.
Heavily Tested  > 500 unit tests  Compatibility tests  100% code coverage  Performance tests Enforced during development

18.
Sensible Syntax Tree https://developer.mozilla.org/en/SpiderMonkey/Parser_API { type: "Program", body: [ { type: "ExpressionStatement", expression: { answer = 42 type: "AssignmentExpression", operator: "=", left: { type: "Identifier", name: "answer" }, right: { type: "Literal", value: 42 } http://esprima.org/demo/parse.html } } ] Try online! }

19.
Syntax Tree Visualization answer = 42

20.
Specification, Parser Code, Syntax Tree ECMA-262 Annex A.4 while ( Expression ) Statement function parseWhileStatement() { var test, body; expectKeyword(while); expect((); test = parseExpression(); expect()); body = parseStatement(); return { type: Syntax.WhileStatement, test: test, body: body }; }

21.
High Performance http://esprima.org/test/compare.html Speed Comparison Chrome 18 Internet Explorer 9 Esprima 233 ms 567 ms UglifyJS parse-js 620 ms 922 ms Benchmark corpus: jQuery, Prototype, MooTools, ExtCore, ...

22.
Syntax Node Location { type: "ExpressionStatement", expression: { type: "AssignmentExpression", operator: "=", left: { answer = 42 type: "Identifier", name: "answer", range: [0, 6] }, right: { type: "Literal", value: 42, range: [9, 11] }, range: [0, 11] }, range: [0, 11] }

23.
Fit for Code Regeneration https://github.com/Constellation/escodegen Source Esprima Escodegen Source Syntax Tree Shorten variable name Syntax Transformation Automatically inline short function Obfuscate

24.
Easily Tweaked LLJS: Low Level JavaScript http://mbebenita.github.com/LLJS/ Block scope let x = 0; let u8 flag; Data types let i32 position; struct Point { int x, y; }; Pointers let u16 *p = q;

25.
Error Tolerant Useful for IDE, editors, ... Mismatched quote var msg = "Hello’; Too many dots person..age = 18; Incomplete, still typing if (person. Strict mode violation use strict; with (person) { }

26.
Handle the Comments // Life, Universe, and Everything answer = 42 comments: [ { range: [0, 34], type: "Line", value: " Life, Universe, and Everything" } ] Documentation tool Code annotation https://github.com/thejohnfreeman/jfdoc https://github.com/goatslacker/node-typedjs

27.
Forward Looking Experimental ‘harmony’ branch Object initializer shorthand Destructuring assignment var point = {x, y}; point = {14, 3, 77}; {x, y, z} = point; Module Block scope module MathExtra { // do something { let x; } const y = 0; }

28.
Code Quality Tools

29.
Syntax Tree Visualization Block statement http://esprima.org/demo/parse.html

30.
Most Popular Keywords this 3229 function 3108 if 3063 return 2878 var 2116 var fs = require(fs), else 562 esprima = require(esprima), for 436 files = process.argv.splice(2); new 232 in 225 files.forEach(function (filename) { typeof 188 var content = fs.readFileSync(filename, utf-8), while 143 tokens = esprima.parse(content, { tokens: true }).tokens; case 122 break 115 tokens.forEach(function (token) { try 84 if (token.type === Keyword) { catch 84 console.log(token.value); delete 72 } throw 38 }); switch 35 }); continue 25 default 15instanceof 14 do 12 void 10 finally 4 http://ariya.ofilabs.com/2012/03/most-popular-javascript-keywords.html

31.
Most Popular Statements var fs = require(fs),ExpressionStatement 6728 esprima = require(esprima), BlockStatement 6353 files = process.argv.splice(2); IfStatement 3063 ReturnStatement 2878 files.forEach(function (filename) { VariableDeclaration 2116 var content = fs.readFileSync(filename, utf-8), syntax = esprima.parse(content); FunctionDeclaration 371 ForStatement 293 JSON.stringify(syntax, function (key, value) { ForInStatement 143 if (key === type) { WhileStatement 131 if (value.match(/Declaration$/) || BreakStatement 115 value.match(/Statement$/)) { TryStatement 84 console.log(value); EmptyStatement 66 } } ThrowStatement 38 return value; SwitchStatement 35 }); ContinueStatement 25 }); DoWhileStatement 12 LabeledStatement 6 http://ariya.ofilabs.com/2012/04/most-popular-javascript-statements.html

32.
Identifier Length Distribution750 mean of the identifier length is 8.27 characters500 prototype-1.7.0.0.js SCRIPT_ELEMENT_REJECTS_TEXTNODE_APPENDING prototype-1.7.0.0.js MOUSEENTER_MOUSELEAVE_EVENTS_SUPPORTED jquery-1.7.1.js subtractsBorderForOverflowNotVisible jquery.mobile-1.0.js getClosestElementWithVirtualBinding250 prototype-1.7.0.0.js HAS_EXTENDED_CREATE_ELEMENT_SYNTAX 0 0 5 10 15 20 25 30 35 40 45 http://ariya.ofilabs.com/2012/05/javascript-identifier-length-distribution.html

33.
“Code Linting” var fs = require(fs), esprima = require(./esprima), files = process.argv.splice(2); Not a strict equal files.forEach(function (filename) { var content = fs.readFileSync(filename, utf-8), syntax = esprima.parse(content, { loc: true });if (x == 9) { // do Something JSON.stringify(syntax, function (key, value) {} if (key === test && value.operator === ==) console.log(Line, value.loc.start.line); return value; }); }); Another example: next-generation JSHint https://github.com/jshint/jshint-next

34.
Strict Mode Checking Duplicate data property in object literal not allowed in strict mode use strict; block = { color: blue, height: 20, width: 10, color: red }; http://ariya.ofilabs.com/2012/03/strict-mode-checks-with-esprima.html

35.
http://ariya.ofilabs.com/2011/08/hall-of-api-shame-boolean-trap.html“Boolean Trap” Finder var volumeSlider = new Slider(false); Obfuscated choice component.setHidden(false); Double-negative filter.setCaseInsensitive(false);Can you make up your mind? treeItem.setState(true, false); event.initKeyEvent("keypress", true, true, null, null, The more the merrier? false, false, false, false, 9, 0);

36.
Style Formatter https://github.com/fawek/codepainter Sample code Source CodePainter Formatted source Infer coding styles Indentation Quote for string literal Whitespace

37.
https://github.com/Constellation/escodegenRewrite and Regenerate answer = 42; var syntax = esprima.parse(answer = 42;); syntax.body[0].expression.right.value = 1337; escodegen.generate(syntax) answer = 1337;

38.
Code Coverage https://github.com/itay/node-cover https://github.com/coveraje/coveraje x = 42; https://github.com/pmlopes/coberturajs if (false) x = -1; http://ariya.ofilabs.com/2012/03/javascript-code-coverage-and-esprima.html

39.
http://itay.github.com/snug_codecoverage_slides/Instrumentation for Coverage { __statement_ZyyqFc(1); var a = 5; Statement var a = 5; } { __statement_ZyyqFc(2); Expression foo(); __expression_kC$jur(3), foo(); } function foo() { function foo() { Block ... __block_n53cJc(1); }; ... }

40.
Non-Destructive Partial Source Modification Do not remove comments Intact Preserve indentation & other formatting Modified Add “contextual” information Inject or change function invocation

41.
String Literal Quotes May need proper escaping console.log("Hello") console.log(Hello) List of tokens [ { type: "Identifier", value: "console", range: [0, 7] }, { type: "Punctuator", value: ".", range: [7, 8] }, { type: "Identifier", value: "log", range: [8, 11] }, { type: "Punctuator", value: "(", range: [11, 12] }, { type: "String", value: ""Hello"", range: [12, 19] }, { type: "Punctuator", value: ")", range: [19, 19] } ] http://ariya.ofilabs.com/2012/02/from-double-quotes-to-single-quotes.html

42.
Tracking the Scalability Array.prototype.swap = function (i, j) { var k = this[i]; this[i] = this[j]; this[j] = k; } Array.prototype.swap = function (i, j) { Log({ name: Array.prototype.swap, lineNumber: 1, range: [23, 94] }); var k = this[i]; this[i] = this[j]; this[j] = k; } http://ariya.ofilabs.com/2012/01/scalable-web-apps-the-complexity-issue.html

43.
Execution Tracing 4640 function calls jQuery Mobile startup log https://gist.github.com/1823129 jquery.js 26 jQuery jquery.js 103 init undefined, undefined, [object Object] jquery.js 274 each (Function) jquery.js 631 each [object Object], (Function), undefined jquery.js 495 isFunction [object Object] jquery.js 512 type [object Object] jquery.mobile.js 1857 [Anonymous] jquery.mobile.js 642 [Anonymous] jquery.mobile.js 624 enableMouseBindings jquery.mobile.js 620 disableTouchBindings http://ariya.ofilabs.com/2012/02/tracking-javascript-execution-during-startup.html

44.
http://esprima.googlecode.com/git-history/harmony/demo/transpile.htmlTranspilation // Object literal property shorthand. function Point(x, y) { Harmony return { x, y }; } Intact // Object literal property shorthand. ES 5.1 function Point(x, y) { return { x: x, y: y }; } Inserted fragment

45.
Application Structure Class manifestExt.define(My.sample.Person, { name: Mr. Unknown, age: 42, { className: My.sample.Person, constructor: function(name) {}, functions: [walk, run], properties: [name, age] walk: function(steps) {} } run: function(steps) {}}); http://www.sencha.com/learn/sencha-class-system/

46.
Code Outline EclipseFunctions Variables

47.
Content Assist (aka Autocomplete, aka “Intellisense”) Eclipse http://contraptionsforprogramming.blogspot.com/2012/02/better-javascript-content-assist-in.html

48.
Copy Paste Mistakefunction inside(point, rect) { return (point.x >= rect.x1) && (point.y >= rect.y1) && (point.x <= rect.x2) && (point.y <= rect.y1);} Wrong check

49.
Refactoring Helper // Add two numbers // Add two numbers function add(firt, two) { function add(first, two) { return firt + two; return first + two; } }

50.
Future Tree traversal Scope analysis “Richer” syntax treeSyntax transformation Pattern Matching

51.
Conclusion

52.
Smart editing Source transformation Instrumentation Modern Minification & obfuscation ParserCode coverage Documentation generator Conditional contracts Dependency analysis

53.
Thank You! ariya.hidayat@gmail.com ariya.ofilabs.com @AriyaHidayat