Strategies for End-to-End                            Ariya Hidayat       Web Apps Testing
Amazon KindleCloud ReaderFinancial Times
Edit       Develop                 QA
Code Editing
Do not provoke ambiguitiesAvoid silly mistakes                        Learn better code patternWrite readable code        ...
JavaScript Engine Building Blocks                               Built-in objects,                               host objec...
Parsing Stage                        identifier           number                var answer = 42;            keyword        ...
JavaScript Parser (Written in JavaScript)          Esprima          UglifyJS                        Narcissus        ZePar...
Syntax Tree Visualization  answer = 42
Style Formatter                     https://github.com/fawek/codepainter                  Sample code  Source          Cod...
Code Outline   EclipseFunctions  Variables
Application StructureExt.define(My.sample.Person, {                Class manifest    name: Mr. Unknown,    age: 42,       ...
Content Assist (aka Intellisense aka Autocomplete)                                                                        ...
Error Tolerant          Mismatched quote         var msg =                                   "Hello’;              Too man...
Code Linting                      var fs = require(fs), Not a strict equal       esprima = require(./esprima),            ...
Copy Paste Mistake function inside(point, rect) {     return (point.x >= rect.x1) && (point.y >= rect.y1) &&            (p...
“Boolean Trap” Finder              http://ariya.ofilabs.com/2011/08/hall-of-api-shame-boolean-trap.html         Obfuscated ...
Refactoring Helper // Add two numbers          // Add two numbers function add(firt, two) {   function add(first, two) {  ...
Syntax = Messagevar Foo = (function () {   return {                   module Foo {     bar: function (x, y) {     export f...
Transpilation                // Object literal property shorthand.                function Point(x, y) {    Harmony       ...
Development
Unit Test Frameworkhttp://www.trollquotes.org/image/5-gandalf-troll-quote.jpg
Automatic Reload     Modified source   Reload content
Remote Interaction         http://www.sencha.com/blog/remote-javascript-debugging-on-android/    Weinre, Adobe Shadow, Rem...
Fast Smoke Test                  Feedback                    Cycle
Precommit CheckTypical Scenario                            This is                           awesome!                     ...
Zombie.js  var Browser = require("zombie");     browser = new Browser();  browser.visit("http://mobileconference.nl", func...
PhantomJS (Headless WebKit)                              Paint Normal Browser                     Layout           Display...
Screen Capture = Pixel-Perfect Comparison      phantomjs rasterize.js URL output
CasperJS var casper = require(casper).create(); casper.start(http://www.mobileconference.nl/, function() {     this.echo(P...
Postcommit Quality Metrics                             Quality Factor 1        Check-in                              Quali...
Identifier Length Distribution                                                    mean of the identifier length is 8.27 char...
Statement DistributionExpressionStatement                                        6728     BlockStatement                  ...
QA
Platform Proliferation
BusterJS                              Test Suite           Server Interface             Test Server
Selenium                            Firefox            Opera          WebDriver       Chrome           WebDriver          ...
Code Coverage                                 https://github.com/itay/node-cover                                          ...
Instrumentation for Code Coverage                                            {                      var a = 5;    Statemen...
Performance via Benchmarks.js       var suite = new Benchmark.Suite;       suite.add(String#indexOf, function() {         ...
Performance Baseline                                             Baseline     Execution time                      Applicat...
Scalability via Run-time Complexity Analysis       Array.prototype.swap = function (i, j) {           var k = this[i]; thi...
Execution Tracing                                           4640 function calls                                           ...
Wrap-up
Coding style/lint, semantic outline, autocomplete, Edit     API usage, refactoring, better syntax          Test framework,...
To boldly analyze what no man has analyzed before...
Thank You            ariya.hidayat@gmail.com            ariya.ofilabs.com            @AriyaHidayat
Upcoming SlideShare
Loading in...5
×

Strategies for End-to-End Web Apps Testing

16,711

Published on

Developing complex web applications without systematic strategies for its quality assurance will lead to maintenance nightmares. End-to-end testing, from the moment the implementation is being planned up to the stage of deployment, requires careful planning and a suitable set of tools. In this talk, the use of various tools to assist such testing strategies will be uncovered. This involves the use of scriptable headless page automation (PhantomJS, Zombie.js, Selenium) for fast smoke testing as part of the precommit process and for more comprehensive preflight checks in the continuous integration system. In addition, a wide spectrum of testing possibilities will be achieved with extra tools such as static code analyzer, full-stack execution logger, profiler, and code coverage/instrumentation.

Published in: Technology
0 Comments
7 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
16,711
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
73
Comments
0
Likes
7
Embeds 0
No embeds

No notes for slide

Strategies for End-to-End Web Apps Testing

  1. 1. Strategies for End-to-End Ariya Hidayat Web Apps Testing
  2. 2. Amazon KindleCloud ReaderFinancial Times
  3. 3. Edit Develop QA
  4. 4. Code Editing
  5. 5. Do not provoke ambiguitiesAvoid silly mistakes Learn better code patternWrite readable code Improve future maintenance
  6. 6. JavaScript Engine Building Blocks Built-in objects, host objects, ... Runtime Syntax Tree Virtual Source Parser Machine/ Interpreter Fast and conservative
  7. 7. Parsing Stage identifier number var answer = 42; keyword equal sign semicolon Variable Declaration Identifier Literal Constant
  8. 8. JavaScript Parser (Written in JavaScript) Esprima UglifyJS Narcissus ZeParser Traceur Es-Lab
  9. 9. Syntax Tree Visualization answer = 42
  10. 10. Style Formatter https://github.com/fawek/codepainter Sample code Source CodePainter Formatted source Indentation Infer coding Quote for string styles literal Whitespace
  11. 11. Code Outline EclipseFunctions Variables
  12. 12. Application StructureExt.define(My.sample.Person, { Class manifest name: Mr. Unknown,  age: 42, { constructor: function(name) {}, className: My.sample.Person,  functions: [walk, run], walk: function(steps) {} properties: [name, age] run: function(steps) {} }});
  13. 13. Content Assist (aka Intellisense aka Autocomplete) Eclipse http://contraptionsforprogramming.blogspot.com/2012/02/better-javascript-content-assist-in.html
  14. 14. Error Tolerant Mismatched quote var msg = "Hello’; Too many dots person..age = 18;   Incomplete, still typing if (person.   use strict; Strict mode violation with (person) { }
  15. 15. Code Linting var fs = require(fs), Not a strict equal esprima = require(./esprima), files = process.argv.splice(2);   files.forEach(function (filename) {if (x == 9) { var content = fs.readFileSync(filename, utf-8), // do Something syntax = esprima.parse(content, { loc: true });  } JSON.stringify(syntax, function (key, value) { if (key === test && value.operator === ==) console.log(Line, value.loc.start.line); return value; }); });
  16. 16. Copy Paste Mistake function inside(point, rect) { return (point.x >= rect.x1) && (point.y >= rect.y1) && (point.x <= rect.x2) && (point.y <= rect.y1); } Wrong check
  17. 17. “Boolean Trap” Finder http://ariya.ofilabs.com/2011/08/hall-of-api-shame-boolean-trap.html Obfuscated choice var volumeSlider = new Slider(false); Double-negative component.setHidden(false); filter.setCaseInsensitive(false);Can you make up your mind? treeItem.setState(true, false); The more the merrier event.initKeyEvent("keypress", true, true, null, null, false, false, false, false, 9, 0);
  18. 18. Refactoring Helper // Add two numbers // Add two numbers function add(firt, two) { function add(first, two) { return firt + two; return first + two; } }
  19. 19. Syntax = Messagevar Foo = (function () { return { module Foo { bar: function (x, y) { export function bar (x, y) { // do Something // do something } } }; }}; ES 5.1 Harmony Today ES 6/7, Future
  20. 20. Transpilation // Object literal property shorthand. function Point(x, y) { Harmony return { x, y }; } // Object literal property shorthand. ES 5.1 function Point(x, y) { return { x: x, y: y }; }
  21. 21. Development
  22. 22. Unit Test Frameworkhttp://www.trollquotes.org/image/5-gandalf-troll-quote.jpg
  23. 23. Automatic Reload Modified source Reload content
  24. 24. Remote Interaction http://www.sencha.com/blog/remote-javascript-debugging-on-android/ Weinre, Adobe Shadow, Remote Web Inspector,...
  25. 25. Fast Smoke Test Feedback Cycle
  26. 26. Precommit CheckTypical Scenario This is awesome! un t to r forge tests the Git: precommit hook http://ariya.ofilabs.com/2012/03/git-pre-commit-hook-and-smoke-testing.html
  27. 27. Zombie.js var Browser = require("zombie");   browser = new Browser(); browser.visit("http://mobileconference.nl", function () { console.log(browser.text("title")); });
  28. 28. PhantomJS (Headless WebKit) Paint Normal Browser Layout Display Paint XHeadless Operation Layout Display
  29. 29. Screen Capture = Pixel-Perfect Comparison phantomjs rasterize.js URL output
  30. 30. CasperJS var casper = require(casper).create(); casper.start(http://www.mobileconference.nl/, function() { this.echo(Page title is: + this.evaluate(function() { return document.title; }), INFO); }); casper.run();
  31. 31. Postcommit Quality Metrics Quality Factor 1 Check-in Quality Factor 2
  32. 32. Identifier Length Distribution mean of the identifier length is 8.27 characters 750 500 250 0 0 5 10 15 20 25 30 35 40 45 prototype-1.7.0.0.js SCRIPT_ELEMENT_REJECTS_TEXTNODE_APPENDING prototype-1.7.0.0.js MOUSEENTER_MOUSELEAVE_EVENTS_SUPPORTED jquery-1.7.1.js subtractsBorderForOverflowNotVisible jquery.mobile-1.0.js getClosestElementWithVirtualBinding prototype-1.7.0.0.js HAS_EXTENDED_CREATE_ELEMENT_SYNTAX http://ariya.ofilabs.com/2012/05/javascript-identifier-length-distribution.html
  33. 33. Statement DistributionExpressionStatement 6728 BlockStatement 6353 IfStatement 3063 ReturnStatement 2878 var fs = require(fs), VariableDeclaration 2116 esprima = require(esprima), FunctionDeclaration 371 files = process.argv.splice(2);   ForStatement 293 files.forEach(function (filename) { ForInStatement 143 var content = fs.readFileSync(filename, utf-8), WhileStatement 131 syntax = esprima.parse(content); BreakStatement 115   JSON.stringify(syntax, function (key, value) { TryStatement 84 if (key === type) { EmptyStatement 66 if (value.match(/Declaration$/) || ThrowStatement 38 value.match(/Statement$/)) { SwitchStatement 35 console.log(value); ContinueStatement 25 } } DoWhileStatement 12 return value; LabeledStatement 6 }); }); http://ariya.ofilabs.com/2012/04/most-popular-javascript-statements.html
  34. 34. QA
  35. 35. Platform Proliferation
  36. 36. BusterJS Test Suite Server Interface Test Server
  37. 37. Selenium Firefox Opera WebDriver Chrome WebDriver WebDriver WebDriver WebDriver Client
  38. 38. Code Coverage https://github.com/itay/node-cover https://github.com/coveraje/coveraje https://github.com/pmlopes/coberturajs x = 42; if (false) x = -1; http://ariya.ofilabs.com/2012/03/javascript-code-coverage-and-esprima.html
  39. 39. Instrumentation for Code Coverage { var a = 5; Statement __statement_ZyyqFc(1); var a = 5; } { __statement_ZyyqFc(2); foo(); Expression __expression_kC$jur(3), foo(); } function foo() { function foo() { __block_n53cJc(1); Block ... ... }; } http://itay.github.com/snug_codecoverage_slides/
  40. 40. Performance via Benchmarks.js var suite = new Benchmark.Suite; suite.add(String#indexOf, function() { Hello World!.indexOf(o) > -1; }) .on(complete, function() { console.log(Fastest is + this.filter(fastest).pluck(name)); }) .run(); JSPerf.com
  41. 41. Performance Baseline Baseline Execution time Application revision
  42. 42. Scalability via Run-time Complexity Analysis Array.prototype.swap = function (i, j) { var k = this[i]; this[i] = this[j]; this[j] = k; } Array.prototype.swap = function (i, j) { Log({ name: Array.prototype.swap, lineNumber: 1, range: [23, 94] }); var k = this[i]; this[i] = this[j]; this[j] = k; } http://ariya.ofilabs.com/2012/01/scalable-web-apps-the-complexity-issue.html
  43. 43. Execution Tracing 4640 function calls https://gist.github.com/1823129 jQuery Mobile startup log jquery.js 26 jQuery jquery.js 103 init undefined, undefined, [object Object] jquery.js 274 each (Function) jquery.js 631 each [object Object], (Function), undefined jquery.js 495 isFunction [object Object] jquery.js 512 type [object Object] jquery.mobile.js 1857 [Anonymous] jquery.mobile.js 642 [Anonymous] jquery.mobile.js 624 enableMouseBindings jquery.mobile.js 620 disableTouchBindingshttp://ariya.ofilabs.com/2012/02/tracking-javascript-execution-during-startup.html
  44. 44. Wrap-up
  45. 45. Coding style/lint, semantic outline, autocomplete, Edit API usage, refactoring, better syntax Test framework, remote control, precommit check,Develop automatic reload, fast smoke testing, preflight metrics Comprehensive browser driver, code coverage, QA performance baseline, execution tracking
  46. 46. To boldly analyze what no man has analyzed before...
  47. 47. Thank You ariya.hidayat@gmail.com ariya.ofilabs.com @AriyaHidayat
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×