YUI Test The Next Generation (YUIConf 2010)

Nicholas C. Zakas YUIConf Yahoo!, Inc. November 9, 2010

Who's this guy? @captain_picard @slicknet

Who's this guy? Principal Front End Contributor, Engineer Creator of YUI Test Author Lead Author Contributor Lead Author

YUI Test How'd we get here?

Mock Objects YUI Test Debut Friendly Test Names July 2007 (YUI 2.3.0) September 2009 (YUI 3.0.0) Automation Hooks Test-Driven Development JUnit XML and TAP Formats with YUI Test (Talk) April 2010 (YUI 3.1.0) October 2008 Writing effective tests ZOMG!!! (YUIBlog) Today January 2009 Getting started with Yeti Released YUI Test (YUIBlog) August 2010 December 2008 Async Testing Automated Integration Testing TestReporter with YUI Test (Talk) XML and JSON Test Formats October 2009 December 2007 (YUI 2.4.0)

Today YUI Test YUI Test YUI Test Standalone Library Selenium Driver Coverage

YUI Test Standalone Library has no YUI dependencies You can load it completely on its own – outside of YUI 2.x and YUI 3.x

Why? To address a series of ongoing problems

Problem Inconsistencies across versions

YUI Test for YUI 2.x YAHOO.tool.TestCase YAHOO.tool.TestSuite YAHOO.tool.TestRunner YAHOO.tool.TestReporter YAHOO.tool.TestManager YAHOO.tool.TestFormat YAHOO.util.Assert YAHOO.util.ArrayAssert YAHOO.util.DateAssert YAHOO.util.ObjectAssert

YUI Test for YUI 3.x Y.Test.Case Y.Test.Suite Y.Test.Runner Y.Test.Reporter Y.Test.Format Y.Assert Y.ArrayAssert Y.DateAssert Y.ObjectAssert Y.Mock

Hmmmm... YAHOO.tool.TestCase Y.Test.Case YAHOO.tool.TestSuite Y.Test.Suite YAHOO.tool.TestRunner Y.Test.Runner YAHOO.tool.TestReporter Y.Test.Reporter YAHOO.tool.TestManager YAHOO.tool.TestFormat Y.Test.Format YAHOO.util.Assert Y.Assert YAHOO.util.ArrayAssert Y.ArrayAssert YAHOO.util.DateAssert Y.DateAssert YAHOO.util.ObjectAssert Y.ObjectAssert Y.Mock

YUI Test Standalone 1.0.0 YUITest.TestCase YUITest.TestSuite YUITest.TestRunner YUITest.Reporter YUITest.PageManager YUITest.TestFormat YUITest.Assert YUITest.ArrayAssert YUITest.DateAssert YUITest.ObjectAssert YUITest.Mock

Problem I'd like to use YUI Test, but not YUI

Problem Everyone else is doing it

What's it look like?

var testSuite = new YUITest.TestSuite("name"); testSuite.add(testCase1); testSuite.add(testCase2); YUITest.TestRunner.add(testSuite); YUITest.TestRunner.run(); Familiar Syntax!

Not just a port! API cleanup & bug fixes

YUITest.ObjectAssert areEqual() hasKey() hasKeys() ownsKey() ownsKeys() ownsNoKeys() inheritsKey() inheritsKeys() ownsOrInheritsKey() ownsOrInheritsKeys()

Friendlier Test Names var testCase = new YUITest.TestCase({ name: "My Test Case", "This should be a test": function(){ //this is a test in YUI Test for YUI 3.x }, "As is this one": function(){ //this is not a test in YUI Test for YUI 3.x //but is in YUI Test Standalone 1.0.0 } });

Counting Asserts var testCase = new YUITest.TestCase({ name: "My Test Case", "This should be a test": function(){ //this test passes in YUI Test for YUI 3.x //but fails in YUI Test Standalone 1.0.0 } });

Beyond _should var testCase = new YUITest.TestCase({ name: "My Test Case", _should: { //old "Another test": "Error message" }, "Another test": function(){ //new someFunction(); } });

Beyond _should var testCase = new YUITest.TestCase({ name: "My Test Case", "Another test": function(){ //new YUITest.Assert.throwsError("Error message", someFunction); } });

What about the YUI 2.x and 3.x versions? YUI Test Standalone becomes the core implementation of both

Easy automation for continuous integration

Hudson Continuous build/testing system Used to periodically build from source and validate by running tests http://hudson-ci.org/

Hack Selenium Hudson SelNG ? JUnit TAP Screen scrape

Selenium Web application testing system Primarily used by QA organizations for functional testing http://seleniumhq.org/

Selenium Remote Control http://seleniumhq.org/projects/remote-control/

Hudson ? Selenium RC Screen scraping to gather results = ick

Hudson Selenium Selenium Driver RC

Selenium Driver Three primary goals

#1 Completely handle communication between Hudson and Selenium RC Yes, Uhura once wore yellow – focus, people

#2 Extract as much data as possible from tests Return the results in a format that Hudson can consume and report on

#3 Don't force already-existing tests to change Seriously, it's just wrong

Test Pages var testCase = new YUITest.TestCase({ name: "My Test Case", //... }); YUITest.TestRunner.add(testCase); YUITest.TestRunner.run(); Test pages must automatically begin running tests without additional interaction

Command Line Usage: java -jar yuitest-selenium-driver-x.y.z.jar [options] [test files] Global Options -h, --help Displays this information. --browsers <browsers> Run tests in these browsers (comma-delimited). --conf <file> Load options from <file>. --coveragedir <dir> Output coverage files to <dir>. --erroronfail Indicates that a test failure should cause an error to be reported to the console. --host <host> Use the Selenium host <host>. --port <port> Use <port> port on the Selenium host. --resultsdir <dir> Output test result files to <dir>. --silent Don't output test results to the console. --tests <file> Loads test info from <file>. -v, --verbose Display informational messages and warnings.

Examples java -jar yuitest-selenium-driver.jar http://www.example.com/tests/test_ui java -jar yuitest-selenium-driver.jar --host selenium.example.com --port 9080 http://www.example.com/tests/test_ui java -jar yuitest-selenium-driver.jar --browsers *firefox,*iexplore http://www.example.com/tests/test_ui http://www.example.com/tests/test_util java -jar yuitest-selenium-driver.jar --tests tests.xml

Tests File <?xml version="1.0"?> <yuitest> <tests base="http://www.example.com/tests/" timeout="10000"> <url>test_core</url> <url timeout="30000">test_util</url> <url>test_ui</url> </tests> </yuitest> Easiest way to specify multiple tests to execute on the same host machine

Demo (Sorry Slideshare viewers)

What is code coverage?

I dunna know what happened, all my tests passed!

What was it you were testing, Mr. Scott?

Code coverage tells you what code executed during testing

Code can have multiple paths function doSomething(){ if (condition){ doSomethingElse(); } else { doAThirdThing(); } } The goal of unit testing is to exercise each path The goal of code coverage is to identify if you did

doSomething() condition doSomethingElse() doAThirdThing()

Step One Instrument JavaScript

file.js Coverage covered-file.js

Command Line Usage: java -jar yuitest-coverage-x.y.z.jar [options] [file|dir] Global Options -h, --help Displays this information. --charset <charset> Read the input file using <charset>. -d, --dir Input and output (-o) are both directories. -v, --verbose Display informational messages and warnings. -o <file|dir> Place the output into <file|dir>. Defaults to stdout.

Step Two Run tests with instrumented JavaScript

Step Three Generate reports

Extract Coverage Data Yourself //object results results = YUITest.TestRunner.getCoverage(); //standard JSON results format results = YUITest.TestRunner.getCoverage( YUITest.CoverageFormat.JSON); //alternate format results = YUITest.TestRunner.getCoverage( YUITest.CoverageFormat.XebugJSON);

results.json Coverage Report Report

Command Line Usage: java -jar yuitest-coverage-report-x.y.z.jar [options] [file] Global Options -h, --help Displays this information. --format <format> Output reports in <format>. Defaults to HTML. -v, --verbose Display informational messages and warnings. -o <file|dir> Place the output into <file|dir>.

Selenium Coverage Driver Selenium Driver automatically pulls in coverage data

Complete Testing Solution YUI Test YUI Test YUI Test Standalone Library Selenium Driver Coverage JavaScript continuous integration testing system

Compatibility • YUI Test Standalone Library 1.0.0 • YUI Test for YUI 3.1.1+ • YUI Test for YUI 2.8.1+

Contribute • GitHub: http://github.com/yui/yuitest/ • YUILibrary: http://yuilibrary.com/projects/yuitest/ • YUIBlog Announcement: http://www.yuiblog.com/blog/2010/11/09/intro ducing-the-new-yui-test

Questions?

Etcetera • My blog: www.nczonline.net • Twitter: @slicknet • These Slides: http://slideshare.net/nzakas/ • Rate Me: http://spkr8.com/t/4999

Creative Commons Images Used • http://www.flickr.com/photos/anks/3311228772/ • http://www.flickr.com/photos/eschipul/4716821041/

YUI Test The Next Generation (YUIConf 2010)

by Nicholas Zakas

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 43

Statistics