This document summarizes a presentation given by Nicholas C. Zakas on the evolution of YUI Test and introducing the new standalone version. Some key points: - YUI Test was originally developed as a testing framework for YUI but inconsistencies arose between YUI 2.x and 3.x versions. - A new standalone version was created to address these issues and allow YUI Test to be used without YUI dependencies. It provides a familiar syntax and API improvements. - Additional related libraries were introduced, including a Selenium driver for browser automation and code coverage to identify untested code paths. - Together these provide a complete JavaScript testing solution for continuous integration with features like Hudson integration and reporting

1. Nicholas C. Zakas
Yahoo!, Inc.
YUIConf
November 9, 2010

2. Who's this guy?
@captain_picard
@slicknet

3. Who's this guy?
Principal Front End
Engineer
Contributor,
Creator of YUI Test
Author Lead Author Contributor Lead Author

4. YUI Test
How'd we get here?

5. Automated Integration Testing
with YUI Test (Talk)
October 2009
YUI Test Debut
July 2007 (YUI 2.3.0)
Async Testing
TestReporter
XML and JSON Test Formats
December 2007 (YUI 2.4.0)
Mock Objects
Friendly Test Names
September 2009 (YUI 3.0.0)
Test-Driven Development
with YUI Test (Talk)
October 2008
YUI Test Debut
July 2007 (YUI 2.3.0)
Getting started with
YUI Test (YUIBlog)
December 2008
Writing effective tests
(YUIBlog)
January 2009
Automation Hooks
JUnit XML and TAP Formats
April 2010 (YUI 3.1.0)
ZOMG!!!
Today
Yeti Released
August 2010

6. Today
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage

7. Today
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage

8. YUI Test Standalone Library has no
YUI dependencies
You can load it completely on its own – outside of YUI 2.x and YUI 3.x

9. Why?
To address a series of ongoing
problems

10. Problem
Inconsistencies across versions

11. YUI Test for YUI 2.x
YAHOO.tool.TestCase
YAHOO.tool.TestSuite
YAHOO.tool.TestRunner
YAHOO.tool.TestReporter
YAHOO.tool.TestManager
YAHOO.tool.TestFormat
YAHOO.util.Assert
YAHOO.util.ArrayAssert
YAHOO.util.DateAssert
YAHOO.util.ObjectAssert

12. YUI Test for YUI 3.x
Y.Test.Case
Y.Test.Suite
Y.Test.Runner
Y.Test.Reporter
Y.Test.Format
Y.Assert
Y.ArrayAssert
Y.DateAssert
Y.ObjectAssert
Y.Mock

13. Hmmmm...
Y.Test.Case
Y.Test.Suite
Y.Test.Runner
Y.Test.Reporter
Y.Test.Format
Y.Assert
Y.ArrayAssert
Y.DateAssert
Y.ObjectAssert
Y.Mock
YAHOO.tool.TestCase
YAHOO.tool.TestSuite
YAHOO.tool.TestRunner
YAHOO.tool.TestReporter
YAHOO.tool.TestManager
YAHOO.tool.TestFormat
YAHOO.util.Assert
YAHOO.util.ArrayAssert
YAHOO.util.DateAssert
YAHOO.util.ObjectAssert

14. YUI Test Standalone 1.0.0
YUITest.TestCase
YUITest.TestSuite
YUITest.TestRunner
YUITest.Reporter
YUITest.PageManager
YUITest.TestFormat
YUITest.Assert
YUITest.ArrayAssert
YUITest.DateAssert
YUITest.ObjectAssert
YUITest.Mock

15. Problem
I'd like to use YUI Test, but not YUI

17. Problem
Everyone else is doing it

19. What's it look like?

20. <script type="text/javascript"
src="yuitest.js"></script>
<script type="text/javascript">
var testCase = new YUITest.TestCase({
name: "Same ol' test case",
testSomething: function(){
YUITest.Assert.isTrue(true);
},
"This should work": function(){
YUITest.Assert.isTrue(works);
}
});
YUITest.TestRunner.add(testCase);
YUITest.TestRunner.run();
</script>
Familiar
Syntax!

21. var testSuite = new YUITest.TestSuite("name");
testSuite.add(testCase1);
testSuite.add(testCase2);
YUITest.TestRunner.add(testSuite);
YUITest.TestRunner.run();
Familiar
Syntax!

22. Not just a port!
API cleanup & bug fixes

23. YUITest.ObjectAssert
areEqual()
hasKey()
hasKeys()
ownsKey()
ownsKeys()
ownsNoKeys()
inheritsKey()
inheritsKeys()
ownsOrInheritsKey()
ownsOrInheritsKeys()

24. Friendlier Test Names
var testCase = new YUITest.TestCase({
name: "My Test Case",
"This should be a test": function(){
//this is a test in YUI Test for YUI 3.x
},
"As is this one": function(){
//this is not a test in YUI Test for YUI 3.x
//but is in YUI Test Standalone 1.0.0
}
});

25. Counting Asserts
var testCase = new YUITest.TestCase({
name: "My Test Case",
"This should be a test": function(){
//this test passes in YUI Test for YUI 3.x
//but fails in YUI Test Standalone 1.0.0
}
});

26. Beyond _should
var testCase = new YUITest.TestCase({
name: "My Test Case",
_should: { //old
"Another test": "Error message"
},
"Another test": function(){ //new
someFunction();
}
});

27. Beyond _should
var testCase = new YUITest.TestCase({
name: "My Test Case",
"Another test": function(){ //new
YUITest.Assert.throwsError("Error message",
someFunction);
}
});

28. What about the YUI 2.x and 3.x
versions?
YUI Test Standalone becomes the
core implementation of both

29. Today
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage

30. Easy automation for continuous
integration

31. Hudson
Continuous build/testing system
Used to periodically build from source and
validate by running tests
http://hudson-ci.org/

32. ?JUnit
Selenium
TAP
SelNG
Screen scrape
Hack
Hudson

33. ?JUnit
TAP
SelNG
Screen scrape
Hack
Hudson
Selenium

34. Selenium
Web application testing system
Primarily used by QA organizations for
functional testing
http://seleniumhq.org/

35. http://seleniumhq.org/projects/remote-control/
Selenium Remote Control

36. Hudson Selenium
RC
?
Screen scraping to gather results = ick

37. Hudson Selenium
Driver
Selenium
RC

38. Selenium Driver
Three primary goals

39. Completely handle communication between
Hudson and Selenium RC
Yes, Uhura once wore yellow – focus, people
#1

40. #2
Extract as much data as possible from tests
Return the results in a format that Hudson can consume and report on

41. #3
Don't force already-existing tests to change
Seriously, it's just wrong

42. Test Pages
var testCase = new YUITest.TestCase({
name: "My Test Case",
//...
});
YUITest.TestRunner.add(testCase);
YUITest.TestRunner.run();
Test pages must automatically begin
running tests without additional interaction

43. Command Line
Usage: java -jar yuitest-selenium-driver-x.y.z.jar [options] [test files]
Global Options
-h, --help Displays this information.
--browsers <browsers> Run tests in these browsers (comma-delimited).
--conf <file> Load options from <file>.
--coveragedir <dir> Output coverage files to <dir>.
--erroronfail Indicates that a test failure should cause
an error to be reported to the console.
--host <host> Use the Selenium host <host>.
--port <port> Use <port> port on the Selenium host.
--resultsdir <dir> Output test result files to <dir>.
--silent Don't output test results to the console.
--tests <file> Loads test info from <file>.
-v, --verbose Display informational messages and warnings.

44. Examples
java -jar yuitest-selenium-driver.jar
http://www.example.com/tests/test_ui
java -jar yuitest-selenium-driver.jar --host
selenium.example.com --port 9080
http://www.example.com/tests/test_ui
java -jar yuitest-selenium-driver.jar --browsers
*firefox,*iexplore http://www.example.com/tests/test_ui
http://www.example.com/tests/test_util
java -jar yuitest-selenium-driver.jar --tests tests.xml

45. Tests File
<?xml version="1.0"?>
<yuitest>
<tests base="http://www.example.com/tests/" timeout="10000">
<url>test_core</url>
<url timeout="30000">test_util</url>
<url>test_ui</url>
</tests>
</yuitest>
Easiest way to specify multiple tests to
execute on the same host machine

46. Demo
(Sorry Slideshare viewers)

47. Today
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage

48. What is code coverage?

49. I dunna know what
happened, all my tests
passed!

50. What was it you were
testing, Mr. Scott?

51. ...

53. Code coverage tells you what code
executed during testing

54. Code can have multiple paths
function doSomething(){
if (condition){
doSomethingElse();
} else {
doAThirdThing();
}
}
The goal of unit testing is to exercise each path
The goal of code coverage is to identify if you did

55. doSomething()
condition
doSomethingElse() doAThirdThing()

56. doSomething()
condition
doSomethingElse() doAThirdThing()

57. doSomething()
condition
doAThirdThing()doSomethingElse()

58. Step One
Instrument JavaScript

59. Coverage covered-file.jsfile.js

60. Command Line
Usage: java -jar yuitest-coverage-x.y.z.jar [options] [file|dir]
Global Options
-h, --help Displays this information.
--charset <charset> Read the input file using <charset>.
-d, --dir Input and output (-o) are both directories.
-v, --verbose Display informational messages and warnings.
-o <file|dir> Place the output into <file|dir>. Defaults to
stdout.

61. Step Two
Run tests with instrumented
JavaScript

62. Step Three
Generate reports

63. Extract Coverage Data Yourself
//object results
results = YUITest.TestRunner.getCoverage();
//standard JSON results format
results = YUITest.TestRunner.getCoverage(
YUITest.CoverageFormat.JSON);
//alternate format
results = YUITest.TestRunner.getCoverage(
YUITest.CoverageFormat.XebugJSON);

64. Coverage
Report
Reportresults.json

65. Command Line
Usage: java -jar yuitest-coverage-report-x.y.z.jar [options] [file]
Global Options
-h, --help Displays this information.
--format <format> Output reports in <format>. Defaults to HTML.
-v, --verbose Display informational messages and warnings.
-o <file|dir> Place the output into <file|dir>.

66. CoverageSelenium
Driver
Selenium Driver automatically pulls in coverage data

67. Complete Testing Solution
YUI Test
Standalone Library
YUI Test
Selenium Driver
YUI Test
Coverage
JavaScript continuous integration testing system

68. Compatibility
• YUI Test Standalone Library 1.0.0
• YUI Test for YUI 3.1.1+
• YUI Test for YUI 2.8.1+

70. Contribute
• GitHub:
http://github.com/yui/yuitest/
• YUILibrary:
http://yuilibrary.com/projects/yuitest/
• YUIBlog Announcement:
http://www.yuiblog.com/blog/2010/11/09/intro
ducing-the-new-yui-test

71. Questions?

72. Etcetera
• My blog:
www.nczonline.net
• Twitter:
@slicknet
• These Slides:
http://slideshare.net/nzakas/
• Rate Me:
http://spkr8.com/t/4999

73. Creative Commons Images Used
• http://www.flickr.com/photos/anks/3311228772/
• http://www.flickr.com/photos/eschipul/4716821041/