Understanding JavaScript Testing

  • 43,375 views
Uploaded on

A talk given at the Boston Developer Day and at the 2009 jQuery Conference.

A talk given at the Boston Developer Day and at the 2009 jQuery Conference.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • While interesting, there didn’t seem to be any real takeaway for me as a developer (other than, maybe, that I have no hope for effective JavaScript testing...)
    -Brian Dukes
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
43,375
On Slideshare
0
From Embeds
0
Number of Embeds
36

Actions

Shares
Downloads
1,451
Comments
1
Likes
114

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. JavaScript Testing John Resig http://ejohn.org/ - http://twitter.com/jeresig
  • 2. Why Test JavaScript? ✦ Cross-browser issues. ✦ The possibility for causing an unforeseen problem is simply too great.
  • 3. What should I use?
  • 4. Looong Tail 250.0 187.5 125.0 62.5 0 Other
  • 5. People don’t test. :-( 900 675 450 225 0 None
  • 6. Basic Components ✦ Writing and understanding a JavaScript test suite is easy. ✦ Test Suite ✦ Tests ✦ Assertions ✦ Async Tests ✦ Test Runner
  • 7. Assertions  <html>   <head>     <title>Test Suite</title>     <script>     function assert( value, desc ) {       var li = document.createElement("li");       li.className = value ? "pass" : "fail";       li.appendChild( document.createTextNode( desc ) );       document.getElementById("results").appendChild( li );     }          window.onload = function(){       assert( true, "The test suite is running." );     };     </script>     <style>       #results li.pass { color: green; }       #results li.fail { color: red; }     </style>   </head>   <body>     <ul id="results"></ul>   </body>   </html> 
  • 8. Tests      test("A test.", function(){         assert( true, "First assertion completed" );         assert( true, "Second assertion completed" );         assert( true, "Third assertion completed" );       });              test("Another test.", function(){         assert( true, "First test completed" );         assert( false, "Second test failed" );         assert( true, "Third assertion completed" );       }); 
  • 9. Tests    var results;          function assert( value, desc ) {       var li = document.createElement("li");       li.className = value ? "pass" : "fail";       li.appendChild( document.createTextNode( desc ) );       results.appendChild( li );       if ( !value ) {         li.parentNode.parentNode.className = "fail";       }       return li;     }          function test(name, fn){       results = document.getElementById("results");       results = assert( true, name ).appendChild(         document.createElement("ul") );       fn();     } 
  • 10. Async Tests  test(function(){     pause();     setTimeout(function(){       assert( true, "First test completed" );       resume();     }, 100);   });      test(function(){     pause();     setTimeout(function(){       assert( true, "Second test completed" );       resume();     }, 200);   }); 
  • 11. Async Tests  (function(){     var queue = [], paused = false;          this.test = function(fn){       queue.push( fn );       runTest();     };          this.pause = function(){       paused = true;     };          this.resume = function(){       paused = false;       setTimeout(runTest, 1);     };          function runTest(){       if ( !paused && queue.length ) {         queue.shift()();         if ( !paused ) {           resume();         }       }     }   })(); 
  • 12. TestRunner ✦ Responsible for loading an executing tests. ✦ Sometimes individual test groups can run standalone (Dojo) sometimes they require the test runner (QUnit, JSUnit).
  • 13. Popular Test Frameworks 250.0 187.5 125.0 62.5 0 QUnit JSUnit Selenium YUITest FireUnit Screw.Unit JSSpec
  • 14. Unit Testing ✦ Break code into logical chucks for testing. ✦ Focus on one method at a time. ✦ Popular Frameworks: ✦ QUnit ✦ JSUnit ✦ YUITest
  • 15. QUnit ✦ Unit Testing framework built for jQuery. ✦ Additional features: ✦ Supports asynchronous testing. ✦ Can break code into modules. ✦ Supports test timeouts. ✦ http://docs.jquery.com/QUnit
  • 16. QUnit Style  test("a basic test example", function() {     ok( true, "this test is fine" );     var value = "hello";     equals( "hello", value, "We expect value to be hello" );   });      module("Module A");      test("first test within module", function() {     ok( true, "all pass" );   });      test("second test within module", function() {     ok( true, "all pass" );   });      module("Module B");      test("some other test", function() {     expect(2);     equals( true, false, "failing test" );     equals( true, true, "passing test" );   }); 
  • 17. QUnit
  • 18. JSUnit ✦ One of the oldest JavaScript testing frameworks. ✦ A port of JUnit to JavaScript, circa 2001. ✦ Code feels very 2001 (frames!) ✦ http://www.jsunit.net/
  • 19. JSUnit        function coreSuite() {             var result = new top.jsUnitTestSuite();             result.addTestPage("tests/jsUnitAssertionTests.html");             result.addTestPage("tests/jsUnitSetUpTearDownTests.html");             result.addTestPage("tests/jsUnitRestoredHTMLDivTests.html");             result.addTestPage("tests/jsUnitFrameworkUtilityTests.html");             result.addTestPage("tests/jsUnitOnLoadTests.html");             result.addTestPage("tests/jsUnitUtilityTests.html");             return result;         }                  function serverSuite() {             var result = new top.jsUnitTestSuite();             result.addTestPage("tests/server/jsUnitVersionCheckTests.html");             result.addTestPage("tests/server/jsUnitServerAjaxTests.html");             return result;         }                  function librariesSuite() {             var result = new top.jsUnitTestSuite();             result.addTestPage("tests/jsUnitMockTimeoutTest.html");             return result;         }                  function suite() {             var newsuite = new top.jsUnitTestSuite();             newsuite.addTestSuite(coreSuite());             newsuite.addTestSuite(serverSuite());             newsuite.addTestSuite(librariesSuite());             return newsuite;         }
  • 20. JSUnit  function testAssertNotUndefined() {       assertNotUndefined("1 should not be undefined", 1);       assertNotUndefined(1);   }      function testAssertNaN() {       assertNaN("a string should not be a number", "string");       assertNaN("string");   }      function testAssertNotNaN() {       assertNotNaN("1 should not be not a number", 1);       assertNotNaN(1);   }      function testFail() {       var excep = null;       try {           fail("Failure message");       } catch (e) {           excep = e;       }       assertJsUnitException("fail(string) should throw a JsUnitException", excep);   }      function testTooFewArguments() {       var excep = null;       try {           assert();       } catch (e1) {           excep = e1;       }       assertNonJsUnitException("Calling an assertion function with too  few arguments should throw an exception", excep);   }
  • 21. JSUnit
  • 22. YUITest (2 & 3) ✦ Testing framework built and developed by Yahoo (released Oct 2008). ✦ Completely overhauled to go with YUI v3. ✦ Features: ✦ Supports async tests. ✦ Has good event simulation. ✦ v2: http://developer.yahoo.com/yui/ examples/yuitest/ ✦ v3: http://developer.yahoo.com/yui/3/test/
  • 23. YUITest 2   YAHOO.example.yuitest.ArrayTestCase = new YAHOO.tool.TestCase({       name : "Array Tests",              setUp : function () {           this.data = [0,1,2,3,4]       },              tearDown : function () {           delete this.data;       },                testPop : function () {           var Assert = YAHOO.util.Assert;                    var value = this.data.pop();                      Assert.areEqual(4, this.data.length);           Assert.areEqual(4, value);                   },                      testPush : function () {           var Assert = YAHOO.util.Assert;                      this.data.push(5);                      Assert.areEqual(6, this.data.length);           Assert.areEqual(5, this.data[5]);                   }  }); 
  • 24. YUITest 2
  • 25. YUITest 3  Y.example.test.DataTestCase = new Y.Test.Case({       name : "Data Tests",              setUp : function () {           this.data = {               name: "test",               year: 2007,               beta: true           };       },              tearDown : function () {           delete this.data;       },              testName : function () {           var Assert = Y.Assert;                      Assert.isObject(this.data);           Assert.isString(this.data.name);           Assert.areEqual("test", this.data.name);                   },              testYear : function () {           var Assert = Y.Assert;                      Assert.isObject(this.data);           Assert.isNumber(this.data.year);           Assert.areEqual(2007, this.data.year);                   }  }); 
  • 26. YUITest 3
  • 27. Behavior Testing ✦ Similar to unit testing, but broken up by task. ✦ Functionally very similar to unit testing, uses different terminology ✦ Popular frameworks: ✦ Screw.Unit ✦ JSSpec
  • 28. Screw.Unit ✦ Popular BDD framework. ✦ http://github.com/nathansobo/screw-unit/ tree/master
  • 29. Screw.Unit  describe("a nested describe", function() {     var invocations = [];        before(function() {       invocations.push("before");     });        describe("a doubly nested describe", function() {       before(function() {         invocations.push('inner before');       });          it("runs befores in all ancestors prior to an it", function() {         expect(invocations).to(equal, ["before", "inner before"]);       });     });   }); 
  • 30. Screw.Unit
  • 31. JSSpec ✦ Used by MooTools as their testing framework. ✦ http://jania.pe.kr/aw/moin.cgi/JSSpec
  • 32. JSSpec  describe('"Should have"s', {     'String length': function() {       expect("Hello").should_have(4, "characters");     },     'Array length': function() {       expect([1,2,3]).should_have(4, "items");     },     'Object's item length': function() {       expect({name:'Alan Kang', email:'jania902@gmail.com', accounts:['A', 'B']}).should_have(3, "accounts");     },     'No match': function() {       expect("This is a string").should_have(5, "players");     },     'Exactly': function() {       expect([1,2,3]).should_have_exactly(2, "items");     },     'At least': function() {       expect([1,2,3]).should_have_at_least(4, "items");     },     'At most': function() {       expect([1,2,3]).should_have_at_most(2, "items");     }   }) 
  • 33. JSSpec
  • 34. Automation ✦ Functional Testing ✦ Browser launching ✦ Server-Side
  • 35. Functional Testing ✦ Selenium IDE ✦ There may be others by Selenium is far and away the best.
  • 36. Selenium IDE ✦ Records and automates actions performed by a user. ✦ An extension for Firefox that records the actions. ✦ Can play them back in all browsers (limited by cross-domain issues). ✦ Primarily for testing web applications - everyone should use it, though! ✦ http://seleniumhq.org/projects/ide/
  • 37. Selenium IDE
  • 38. Browser Launching ✦ Automates the process of opening browser windows, running tests, and getting results. ✦ Frequently require a specific framework. ✦ Popular frameworks: ✦ WebDriver http://code.google.com/p/ webdriver/ (Java) ✦ Waitr http://wtr.rubyforge.org/ (Ruby) ✦ JsTestDriver http://code.google.com/p/ js-test-driver/ (Java) ✦ Selenium RC http://seleniumhq.org/ projects/remote-control/ (Java)
  • 39. Browser Launching
  • 40. Server-Side ✦ Ignore the browser! Simulate it on the server-side. ✦ Almost always uses Java + Rhino to construct a browser. ✦ Some frameworks: ✦ Crosscheck ✦ Env.js ✦ Blueridge
  • 41. Server-Side ✦ Crosscheck ✦ Pure Java, even simulates browser bugs. ✦ http://www.thefrontside.net/crosscheck ✦ Env.js ✦ Pure JavaScript, focuses on standards support. ✦ http://github.com/thatcher/env-js/tree/ master ✦ Blueridge ✦ Env.js + Screw.Unit + Rhino ✦ http://github.com/relevance/blue-ridge/
  • 42. Env.js $ java -jar build/js.jar Rhino 1.6 release 6 2007 06 28 js> load('build/runtest/env.js'); js> window.location = 'test/index.html'; test/index.html js> load('dist/jquery.js'); // Add pretty printing to jQuery objects: js> jQuery.fn.toString = DOMNodeList.prototype.toString; js> $('span').remove(); [ <span#å°åŒ—Taibei>, <span#å°åŒ—>, <span#utf8class1>, <span#utf8class2>, <span#foo:bar>, <span#test.foo[5]bar> ] // Yes - UTF-8 is supported in DOM documents! js> $('span') [ ] js> $('div').append('<span><b>hello!</b> world</span>'); [ <div#main>, <div#foo> ] js> $('span') [ <span>, <span> ] js> $('span').text() hello! worldhello! world
  • 43. Distributed ✦ Selenium Grid ✦ Push Selenium tests out to many machines (that you manage), simultaneously. ✦ Collect and store the results. ✦ http://selenium-grid.seleniumhq.org/ ✦ TestSwarm ✦ Push tests to a distributed swarm of clients. ✦ Results viewable on the server. ✦ http://testswarm.com/
  • 44. TestSwarm
  • 45. Choose Your Browsers
  • 46. Cost / Benefit IE 7 IE 6 FF 3 Safari 3 Opera 9.5 Cost Benefit Draw a line in the sand.
  • 47. Graded Support Yahoo Browser Compatibility
  • 48. Browser Support Grid IE Firefox Safari Opera Chrome Previous 6.0 2.0 3.0 9.5 Current 7.0 3.0 3.2 9.6 1.0 Next 8.0 3.1 4.0 10.0 2.0 jQuery Browser Support
  • 49. Browser Support Grid IE Firefox Safari Opera Chrome Previous 6.0 2.0 3.0 9.5 Current 7.0 3.0 3.2 9.6 1.0 Next 8.0 3.1 4.0 10.0 2.0 jQuery 1.3 Browser Support
  • 50. The Scaling Problem ✦ The Problem: ✦ jQuery has 6 test suites ✦ Run in 11 browsers ✦ (Not even including multiple platforms!) ✦ All need to be run for every commit, patch, and plugin. ✦ JavaScript testing doesn’t scale well.
  • 51. Distributed Testing ✦ Hub server ✦ Clients connect and help run tests ✦ A simple JavaScript client that can be run in all browsers ✦ Including mobile browsers! ✦ TestSwarm
  • 52. FF 3.5 FF 3.5 FF 3.5 IE 6 IE 6 FF 3 IE 6 Op 9 FF 3 IE 7 TestSwarm IE 7 Test Suite Test Suite Test Suite
  • 53. Manual Testing ✦ Push tests to users who follow pre-defined steps ✦ Answer ‘Yes’/’No’ questions which are pushed back to the server. ✦ An effective way to distribute manual test load to dozens of clients.
  • 54. TestSwarm.com ✦ Incentives for top testers (t-shirts, books) ✦ Will be opening for alpha testing very soon ✦ Help your favorite JavaScript library become better tested! ✦ http://testswarm.com
  • 55. Questions ✦ Contact: ✦ John Resig ✦ http://ejohn.org/ ✦ http://twitter.com/jeresig ✦ More info: ✦ http://jsninja.com/Overview ✦ http://ejohn.org/blog/javascript-testing- does-not-scale/