Nicholas C. Zakas
Yahoo!, Inc.
YUIConf
November 9, 2010
Who's this guy?
@captain_picard
@slicknet
Who's this guy?
Principal Front End
Engineer
Contributor,
Creator of YUI Test
Author Lead Author Contributor Lead Author
YUI Test
How'd we get here?
Automated Integration Testing
with YUI Test (Talk)
October 2009
YUI Test Debut
July 2007 (YUI 2.3.0)
Async Testing
TestRep...
Today
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage
Today
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage
YUI Test Standalone Library has no
YUI dependencies
You can load it completely on its own – outside of YUI 2.x and YUI 3.x
Why?
To address a series of ongoing
problems
Problem
Inconsistencies across versions
YUI Test for YUI 2.x
YAHOO.tool.TestCase
YAHOO.tool.TestSuite
YAHOO.tool.TestRunner
YAHOO.tool.TestReporter
YAHOO.tool.Tes...
YUI Test for YUI 3.x
Y.Test.Case
Y.Test.Suite
Y.Test.Runner
Y.Test.Reporter
Y.Test.Format
Y.Assert
Y.ArrayAssert
Y.DateAss...
Hmmmm...
Y.Test.Case
Y.Test.Suite
Y.Test.Runner
Y.Test.Reporter
Y.Test.Format
Y.Assert
Y.ArrayAssert
Y.DateAssert
Y.Object...
YUI Test Standalone 1.0.0
YUITest.TestCase
YUITest.TestSuite
YUITest.TestRunner
YUITest.Reporter
YUITest.PageManager
YUITe...
Problem
I'd like to use YUI Test, but not YUI
Problem
Everyone else is doing it
What's it look like?
<script type="text/javascript"
src="yuitest.js"></script>
<script type="text/javascript">
var testCase = new YUITest.TestC...
var testSuite = new YUITest.TestSuite("name");
testSuite.add(testCase1);
testSuite.add(testCase2);
YUITest.TestRunner.add(...
Not just a port!
API cleanup & bug fixes
YUITest.ObjectAssert
areEqual()
hasKey()
hasKeys()
ownsKey()
ownsKeys()
ownsNoKeys()
inheritsKey()
inheritsKeys()
ownsOrIn...
Friendlier Test Names
var testCase = new YUITest.TestCase({
name: "My Test Case",
"This should be a test": function(){
//t...
Counting Asserts
var testCase = new YUITest.TestCase({
name: "My Test Case",
"This should be a test": function(){
//this t...
Beyond _should
var testCase = new YUITest.TestCase({
name: "My Test Case",
_should: { //old
"Another test": "Error message...
Beyond _should
var testCase = new YUITest.TestCase({
name: "My Test Case",
"Another test": function(){ //new
YUITest.Asser...
What about the YUI 2.x and 3.x
versions?
YUI Test Standalone becomes the
core implementation of both
Today
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage
Easy automation for continuous
integration
Hudson
Continuous build/testing system
Used to periodically build from source and
validate by running tests
http://hudson-...
?JUnit
Selenium
TAP
SelNG
Screen scrape
Hack
Hudson
?JUnit
TAP
SelNG
Screen scrape
Hack
Hudson
Selenium
Selenium
Web application testing system
Primarily used by QA organizations for
functional testing
http://seleniumhq.org/
http://seleniumhq.org/projects/remote-control/
Selenium Remote Control
Hudson Selenium
RC
?
Screen scraping to gather results = ick
Hudson Selenium
Driver
Selenium
RC
Selenium Driver
Three primary goals
Completely handle communication between
Hudson and Selenium RC
Yes, Uhura once wore yellow – focus, people
#1
#2
Extract as much data as possible from tests
Return the results in a format that Hudson can consume and report on
#3
Don't force already-existing tests to change
Seriously, it's just wrong
Test Pages
var testCase = new YUITest.TestCase({
name: "My Test Case",
//...
});
YUITest.TestRunner.add(testCase);
YUITest...
Command Line
Usage: java -jar yuitest-selenium-driver-x.y.z.jar [options] [test files]
Global Options
-h, --help Displays ...
Examples
java -jar yuitest-selenium-driver.jar
http://www.example.com/tests/test_ui
java -jar yuitest-selenium-driver.jar ...
Tests File
<?xml version="1.0"?>
<yuitest>
<tests base="http://www.example.com/tests/" timeout="10000">
<url>test_core</ur...
Demo
(Sorry Slideshare viewers)
Today
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage
What is code coverage?
I dunna know what
happened, all my tests
passed!
What was it you were
testing, Mr. Scott?
...
Code coverage tells you what code
executed during testing
Code can have multiple paths
function doSomething(){
if (condition){
doSomethingElse();
} else {
doAThirdThing();
}
}
The ...
doSomething()
condition
doSomethingElse() doAThirdThing()
doSomething()
condition
doSomethingElse() doAThirdThing()
doSomething()
condition
doAThirdThing()doSomethingElse()
Step One
Instrument JavaScript
Coverage covered-file.jsfile.js
Command Line
Usage: java -jar yuitest-coverage-x.y.z.jar [options] [file|dir]
Global Options
-h, --help Displays this info...
Step Two
Run tests with instrumented
JavaScript
Step Three
Generate reports
Extract Coverage Data Yourself
//object results
results = YUITest.TestRunner.getCoverage();
//standard JSON results format...
Coverage
Report
Reportresults.json
Command Line
Usage: java -jar yuitest-coverage-report-x.y.z.jar [options] [file]
Global Options
-h, --help Displays this i...
CoverageSelenium
Driver
Selenium Driver automatically pulls in coverage data
Complete Testing Solution
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage
JavaScript continuous int...
Compatibility
• YUI Test Standalone Library 1.0.0
• YUI Test for YUI 3.1.1+
• YUI Test for YUI 2.8.1+
Contribute
• GitHub:
http://github.com/yui/yuitest/
• YUILibrary:
http://yuilibrary.com/projects/yuitest/
• YUIBlog Announ...
Questions?
Etcetera
• My blog:
www.nczonline.net
• Twitter:
@slicknet
• These Slides:
http://slideshare.net/nzakas/
• Rate Me:
http:/...
Creative Commons Images Used
• http://www.flickr.com/photos/anks/3311228772/
• http://www.flickr.com/photos/eschipul/47168...
YUI Test The Next Generation (YUIConf 2010)
YUI Test The Next Generation (YUIConf 2010)
YUI Test The Next Generation (YUIConf 2010)
YUI Test The Next Generation (YUIConf 2010)
Upcoming SlideShare
Loading in...5
×

YUI Test The Next Generation (YUIConf 2010)

6,417

Published on

JavaScript testing has grown by leaps and bounds over the past few years. When YUI Test was first introduced in 2007, it was just the first step in a long process of bringing test-driven development to the front end. YUI Test evolved with the release of YUI 3 to introduce mock objects as feedback indicated a need. As feedback continued to come in, YUI Test continued to evolve. Learn about the next version of YUI Test, how it makes testing any JavaScript code easier, and the brand new tools that allow you to integrate your testing into a continuous integration environment.

Published in: Technology
0 Comments
6 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,417
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
88
Comments
0
Likes
6
Embeds 0
No embeds

No notes for slide

YUI Test The Next Generation (YUIConf 2010)

  1. 1. Nicholas C. Zakas Yahoo!, Inc. YUIConf November 9, 2010
  2. 2. Who's this guy? @captain_picard @slicknet
  3. 3. Who's this guy? Principal Front End Engineer Contributor, Creator of YUI Test Author Lead Author Contributor Lead Author
  4. 4. YUI Test How'd we get here?
  5. 5. Automated Integration Testing with YUI Test (Talk) October 2009 YUI Test Debut July 2007 (YUI 2.3.0) Async Testing TestReporter XML and JSON Test Formats December 2007 (YUI 2.4.0) Mock Objects Friendly Test Names September 2009 (YUI 3.0.0) Test-Driven Development with YUI Test (Talk) October 2008 YUI Test Debut July 2007 (YUI 2.3.0) Getting started with YUI Test (YUIBlog) December 2008 Writing effective tests (YUIBlog) January 2009 Automation Hooks JUnit XML and TAP Formats April 2010 (YUI 3.1.0) ZOMG!!! Today Yeti Released August 2010
  6. 6. Today YUI Test Standalone Library YUI Test Selenium Driver YUI Test Coverage
  7. 7. Today YUI Test Standalone Library YUI Test Selenium Driver YUI Test Coverage
  8. 8. YUI Test Standalone Library has no YUI dependencies You can load it completely on its own – outside of YUI 2.x and YUI 3.x
  9. 9. Why? To address a series of ongoing problems
  10. 10. Problem Inconsistencies across versions
  11. 11. YUI Test for YUI 2.x YAHOO.tool.TestCase YAHOO.tool.TestSuite YAHOO.tool.TestRunner YAHOO.tool.TestReporter YAHOO.tool.TestManager YAHOO.tool.TestFormat YAHOO.util.Assert YAHOO.util.ArrayAssert YAHOO.util.DateAssert YAHOO.util.ObjectAssert
  12. 12. YUI Test for YUI 3.x Y.Test.Case Y.Test.Suite Y.Test.Runner Y.Test.Reporter Y.Test.Format Y.Assert Y.ArrayAssert Y.DateAssert Y.ObjectAssert Y.Mock
  13. 13. Hmmmm... Y.Test.Case Y.Test.Suite Y.Test.Runner Y.Test.Reporter Y.Test.Format Y.Assert Y.ArrayAssert Y.DateAssert Y.ObjectAssert Y.Mock YAHOO.tool.TestCase YAHOO.tool.TestSuite YAHOO.tool.TestRunner YAHOO.tool.TestReporter YAHOO.tool.TestManager YAHOO.tool.TestFormat YAHOO.util.Assert YAHOO.util.ArrayAssert YAHOO.util.DateAssert YAHOO.util.ObjectAssert
  14. 14. YUI Test Standalone 1.0.0 YUITest.TestCase YUITest.TestSuite YUITest.TestRunner YUITest.Reporter YUITest.PageManager YUITest.TestFormat YUITest.Assert YUITest.ArrayAssert YUITest.DateAssert YUITest.ObjectAssert YUITest.Mock
  15. 15. Problem I'd like to use YUI Test, but not YUI
  16. 16. Problem Everyone else is doing it
  17. 17. What's it look like?
  18. 18. <script type="text/javascript" src="yuitest.js"></script> <script type="text/javascript"> var testCase = new YUITest.TestCase({ name: "Same ol' test case", testSomething: function(){ YUITest.Assert.isTrue(true); }, "This should work": function(){ YUITest.Assert.isTrue(works); } }); YUITest.TestRunner.add(testCase); YUITest.TestRunner.run(); </script> Familiar Syntax!
  19. 19. var testSuite = new YUITest.TestSuite("name"); testSuite.add(testCase1); testSuite.add(testCase2); YUITest.TestRunner.add(testSuite); YUITest.TestRunner.run(); Familiar Syntax!
  20. 20. Not just a port! API cleanup & bug fixes
  21. 21. YUITest.ObjectAssert areEqual() hasKey() hasKeys() ownsKey() ownsKeys() ownsNoKeys() inheritsKey() inheritsKeys() ownsOrInheritsKey() ownsOrInheritsKeys()
  22. 22. Friendlier Test Names var testCase = new YUITest.TestCase({ name: "My Test Case", "This should be a test": function(){ //this is a test in YUI Test for YUI 3.x }, "As is this one": function(){ //this is not a test in YUI Test for YUI 3.x //but is in YUI Test Standalone 1.0.0 } });
  23. 23. Counting Asserts var testCase = new YUITest.TestCase({ name: "My Test Case", "This should be a test": function(){ //this test passes in YUI Test for YUI 3.x //but fails in YUI Test Standalone 1.0.0 } });
  24. 24. Beyond _should var testCase = new YUITest.TestCase({ name: "My Test Case", _should: { //old "Another test": "Error message" }, "Another test": function(){ //new someFunction(); } });
  25. 25. Beyond _should var testCase = new YUITest.TestCase({ name: "My Test Case", "Another test": function(){ //new YUITest.Assert.throwsError("Error message", someFunction); } });
  26. 26. What about the YUI 2.x and 3.x versions? YUI Test Standalone becomes the core implementation of both
  27. 27. Today YUI Test Standalone Library YUI Test Selenium Driver YUI Test Coverage
  28. 28. Easy automation for continuous integration
  29. 29. Hudson Continuous build/testing system Used to periodically build from source and validate by running tests http://hudson-ci.org/
  30. 30. ?JUnit Selenium TAP SelNG Screen scrape Hack Hudson
  31. 31. ?JUnit TAP SelNG Screen scrape Hack Hudson Selenium
  32. 32. Selenium Web application testing system Primarily used by QA organizations for functional testing http://seleniumhq.org/
  33. 33. http://seleniumhq.org/projects/remote-control/ Selenium Remote Control
  34. 34. Hudson Selenium RC ? Screen scraping to gather results = ick
  35. 35. Hudson Selenium Driver Selenium RC
  36. 36. Selenium Driver Three primary goals
  37. 37. Completely handle communication between Hudson and Selenium RC Yes, Uhura once wore yellow – focus, people #1
  38. 38. #2 Extract as much data as possible from tests Return the results in a format that Hudson can consume and report on
  39. 39. #3 Don't force already-existing tests to change Seriously, it's just wrong
  40. 40. Test Pages var testCase = new YUITest.TestCase({ name: "My Test Case", //... }); YUITest.TestRunner.add(testCase); YUITest.TestRunner.run(); Test pages must automatically begin running tests without additional interaction
  41. 41. Command Line Usage: java -jar yuitest-selenium-driver-x.y.z.jar [options] [test files] Global Options -h, --help Displays this information. --browsers <browsers> Run tests in these browsers (comma-delimited). --conf <file> Load options from <file>. --coveragedir <dir> Output coverage files to <dir>. --erroronfail Indicates that a test failure should cause an error to be reported to the console. --host <host> Use the Selenium host <host>. --port <port> Use <port> port on the Selenium host. --resultsdir <dir> Output test result files to <dir>. --silent Don't output test results to the console. --tests <file> Loads test info from <file>. -v, --verbose Display informational messages and warnings.
  42. 42. Examples java -jar yuitest-selenium-driver.jar http://www.example.com/tests/test_ui java -jar yuitest-selenium-driver.jar --host selenium.example.com --port 9080 http://www.example.com/tests/test_ui java -jar yuitest-selenium-driver.jar --browsers *firefox,*iexplore http://www.example.com/tests/test_ui http://www.example.com/tests/test_util java -jar yuitest-selenium-driver.jar --tests tests.xml
  43. 43. Tests File <?xml version="1.0"?> <yuitest> <tests base="http://www.example.com/tests/" timeout="10000"> <url>test_core</url> <url timeout="30000">test_util</url> <url>test_ui</url> </tests> </yuitest> Easiest way to specify multiple tests to execute on the same host machine
  44. 44. Demo (Sorry Slideshare viewers)
  45. 45. Today YUI Test Standalone Library YUI Test Selenium Driver YUI Test Coverage
  46. 46. What is code coverage?
  47. 47. I dunna know what happened, all my tests passed!
  48. 48. What was it you were testing, Mr. Scott?
  49. 49. ...
  50. 50. Code coverage tells you what code executed during testing
  51. 51. Code can have multiple paths function doSomething(){ if (condition){ doSomethingElse(); } else { doAThirdThing(); } } The goal of unit testing is to exercise each path The goal of code coverage is to identify if you did
  52. 52. doSomething() condition doSomethingElse() doAThirdThing()
  53. 53. doSomething() condition doSomethingElse() doAThirdThing()
  54. 54. doSomething() condition doAThirdThing()doSomethingElse()
  55. 55. Step One Instrument JavaScript
  56. 56. Coverage covered-file.jsfile.js
  57. 57. Command Line Usage: java -jar yuitest-coverage-x.y.z.jar [options] [file|dir] Global Options -h, --help Displays this information. --charset <charset> Read the input file using <charset>. -d, --dir Input and output (-o) are both directories. -v, --verbose Display informational messages and warnings. -o <file|dir> Place the output into <file|dir>. Defaults to stdout.
  58. 58. Step Two Run tests with instrumented JavaScript
  59. 59. Step Three Generate reports
  60. 60. Extract Coverage Data Yourself //object results results = YUITest.TestRunner.getCoverage(); //standard JSON results format results = YUITest.TestRunner.getCoverage( YUITest.CoverageFormat.JSON); //alternate format results = YUITest.TestRunner.getCoverage( YUITest.CoverageFormat.XebugJSON);
  61. 61. Coverage Report Reportresults.json
  62. 62. Command Line Usage: java -jar yuitest-coverage-report-x.y.z.jar [options] [file] Global Options -h, --help Displays this information. --format <format> Output reports in <format>. Defaults to HTML. -v, --verbose Display informational messages and warnings. -o <file|dir> Place the output into <file|dir>.
  63. 63. CoverageSelenium Driver Selenium Driver automatically pulls in coverage data
  64. 64. Complete Testing Solution YUI Test Standalone Library YUI Test Selenium Driver YUI Test Coverage JavaScript continuous integration testing system
  65. 65. Compatibility • YUI Test Standalone Library 1.0.0 • YUI Test for YUI 3.1.1+ • YUI Test for YUI 2.8.1+
  66. 66. Contribute • GitHub: http://github.com/yui/yuitest/ • YUILibrary: http://yuilibrary.com/projects/yuitest/ • YUIBlog Announcement: http://www.yuiblog.com/blog/2010/11/09/intro ducing-the-new-yui-test
  67. 67. Questions?
  68. 68. Etcetera • My blog: www.nczonline.net • Twitter: @slicknet • These Slides: http://slideshare.net/nzakas/ • Rate Me: http://spkr8.com/t/4999
  69. 69. Creative Commons Images Used • http://www.flickr.com/photos/anks/3311228772/ • http://www.flickr.com/photos/eschipul/4716821041/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×