Automated Frontend Testing

Automated Frontend Testing NeilCrosby (Image from “The Korean Robot” - http://www.ﬂickr.com/photos/zebrapares/1344995547/)

Who am I? @NeilCrosby European Frontend Architect for Search at Yahoo! I help look after frontend code quality.

What’s this talk about? • Automatically and regularly testing frontend code against known standards. • Why? What? How? Where?

Why? What? How? Where?

Why test? • We all want to write code that works. • Testing improves reliability. • Repeatedly testing minimises regressions. • Automatic testing can pick up “dumb-ass” mistakes.

That sounds lovely • Normally that doesn’t happen though. • Normally what happens is...

1. We build things. They adhere to our standards.

2. We go to the pub.

3. Later, we add more code.

4. Our code stops adhering to our standards.

5. Things become less easy to maintain.

6. Bugs creep in.

7. We go to the pub in despair.

8. Repeat.

So... • Test automatically and often. • By adhering to a known standard, and constantly testing against it, it becomes easier to spot problems with the code that we're writing.

And that saves money Fewer bugs == Less time ﬁxing bugs == More time developing new features.

What’s normally tested Backend Frontend In-Browser (API, functions) (HTML, CSS, JS) (Functional) Validators, JsLint, *Unit Selenium YSlow etc. Sometimes Automatic via CI Manual automatic via CI Adhoc, when Well deﬁned Fairly well deﬁned remembered

Why not use Selenium? • Selenium tests code after it’s been interpreted by the browser. • Good for functional testing, not so good for testing discrete units of code.

Discrete Units? • I’m looking at testing the quality of our HTML, CSS and JavaScript. • It should all adhere to a known standard.

So we’re testing what? • The code that leaves the server. • Before it is interpreted by the browser.

What should we test? • Currently I’m testing against: • W3C HTML Validator validator.w3.org • W3C CSS Validator jigsaw.w3.org/css-validator • JsLint jslint.com

Validation is a dirty word • People don’t like the word. • They always ask “what if?”.

Custom DTD • Custom DTDs can help you create your own internal standard.

Custom DTDs are Easy <!ENTITY % HTML.strict SYSTEM quot;strict.dtdquot;> %HTML.strict; <!ATTLIST OL %attrs; -- %coreattrs, %i18n, %events -- start NUMBER #IMPLIED -- starting sequence number -- >

How do I do this testing? • I’ve been writing a test suite! • http://github.com/NeilCrosby/frontend- test-suite (CC Attribution-Share Alike)

Currently in Phase 1 • Tests can easily be run against known units of HTML, CSS and JavaScript. • I’m using phpunit as the framework.

HTML Testing • By default tests against HTML 4.01 Strict. • Test against custom DTDs. • Module and full page testing. • Test ﬁles, URLs or strings.

CSS Testing • By default tests against CSS 2.1. • Known exceptions to the standards can be allowed to pass validation. • Test ﬁles, URLs or strings.

JavaScript Testing • Test against any conﬁguration JsLint allows. • Test ﬁles, URLs or strings.

Add to your CI • Run these tests in your hourly builds. • Run them as a pre-commit hook.

Be a good citizen • You’ll be running these tests frequently. • Set up your own local versions of the HTML Validator, CSS Validator and JsLint. • Instructions available on the web.

Running these tests • If you need to test a simple site, it’s easy - pass a conﬁg object to TheCodeTrainEasyFrontendTestSuite. • Otherwise, write custom TestCase extensions (still not hard).

Including the Suite require_once( 'TheCodeTrainEasyFrontendTestSuite.php' );

Creating the Tests class SomeTestSuite extends TheCodeTrainEasyFrontendTestSuite { public static function suite() { return parent::suite(array( ‘html’ => array( ... ), ‘css’ => array( ... ), ‘js’ => array( ... ), )); } }

The ‘html/css/js’ arrays ‘html’ => array( ‘validator’ => ‘http://yourvalidator’, ‘tests’ => array( ‘http://someurl.to/test’, ‘file://some/file/to/test’, ‘<p>Some string to test</p>’, ), ‘options’ => array( ... ) )

The Options Array ‘options’ => array( ‘doctype_override’ => ‘<!DOCTYPE HTML SYSTEM quot;http://site/doctype.dtdquot;>’, ‘document_section’ => TheCodeTrainHtmlValidator::HTML_CHUNK, ‘document_section_position’ => TheCodeTrainHtmlValidator::POSITION_HEAD )

Options? • Custom doctypes, full page or module, position on page. • Options can also be given for individual tests. array( ‘file://some/file/to/test’, array( your_options ) )

Running these tests > phpunit SomeTestSuite PHPUnit 3.3.1 by Sebastian Bergmann. ......... Time: 7 seconds OK (9 tests, 9 assertions)

Some failures ....F..... Failed asserting that Array ( [0] => Array ( [line] => 7 [errortype] => [error] => syntax of attribute value does not conform to declared value [original_line] => <label for=quot;quot;>Some label</label> ) ) is false.

Some failures .....F.... Failed asserting that Array ( [0] => Array ( [line] => 1 [errortype] => parse-error [error] => Value Error : display (http://www.w3.org/TR/CSS21/ visuren.html#propdef-display) inlin is not a display value : [original_line] => p ) ) is false.

Where’s this used? • Yahoo! - SearchMonkey • TheCodeTrain.co.uk’s WordPress theme.

It’s found problems • Security holes. • Form usability issues. • Other suspect HTML.

The Future!

Phase 2 - The Future • Test for things a simple DTD check can’t pick up on. • CodeSniffer. • Whitelisting sections of code - e.g. adverts. • Basic accessibility testing - working with the experts at Yahoo!.

Available Online • This talk: http://icanhaz.com/aft • The Code (please fork and contribute): http://github.com/NeilCrosby/frontend- test-suite • Twitter: automated frontend testing, @NeilCrosby • My blog hub: http://neilcrosby.com

http://static.slidesharecdn.com	230
http://thecodetrain.co.uk	172
http://lanyrd.com	48
http://www.slideshare.net	40
http://themindstorms.blogspot.com	36
http://iamleu.tumblr.com	6
http://webcache.googleusercontent.com	3
http://translate.googleusercontent.com	2
http://voxpopdesign.com	1
http://www.kaichanvong.com	1
http://www.mashme.tv	1

Automated Frontend Testing

by Neil Crosby on Mar 27, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

11 Embeds 540

Statistics

Automated Frontend Testing — Webinar Transcript