One mistake many people make is running their selenium test suites with every browser combination... you don't need to! Rendering problems are probably the biggest cross browser compatibility issue, and you can't actually test for these with Functional Tests. So by running Selenium tests across many browsers you are really testing JavaScript and DOM compatibility. Running our Selenium tests across many browsers was our JavaScript compatibility strategy, but it proved inefficient as we ended up with bugs in Internet Explorer; how embarrassing!

1. Decapitating
Selenium
Alan Parkinson
CEO, Hindsight Software

3. Selenium WebDriver test
suites take a long time……

4. and then we make it worse
by running them again…
and again… on every
browser

5. Who has done this?

6. In many cases we have all got
it wrong

7. Single Page JavaScript Apps
are becoming pervasive

8. JavaScript problem space
browser compatibility issues…
both JavaScript and DOM

9. No visual testing
Low JavaScript code coverage
Rendering Compatibility?
JavaScript/DOM Compatibility?
What do we achieve with
WebDriver…
Difficult to test caching and error handling

10. Run all JavaScript unit tests cross browser
Simple Answer

11. 17 minute test suite with 11 seconds per test
average
Functional Tests
JavaScript Unit Tests
Looking at the numbers
21 second test suite with 0.073 seconds per
test average

12. Unit
Component/Integration
Functional
Automation Pyramid

13. Unit
Component/Integration
Functional
Cost and Time Pyramid

14. Do we meet our compatibility objective?
Code Coverage has a use

15. Too many JavaScript files
Order of files
*The qualifier
Still need some `smoke` WebDriver tests
for these issues

16. JAVASCRIPT TEST
RUNNERS

17. • Testing Framework Agnostic
• Continuous Integration
• Code coverage with Instanbul
Spectacular Test Runner for
JavaScript
• Launch Browsers
• Manually connect browsers
Real Web Browsers

18. $ npm install karma --save-dev
$ npm install -g karma-cli
$ npm install karma-mocha --save-dev
$ npm install karma-chrome-launcher --
save-dev

19. It`s all about karma.conf.js
Configuring Karma
$ karma init
$ karma start
$ karma run

20. Reusing your Selenium Infrastructure
Karma WebDriver
Launcher
$ npm install karma-webdriver-launcher --
save-dev

22. customLaunchers: {
'IE8': { … },
'IE9': { … },
'IE10': { … },
'Chrome-Windows': { … },
'Chrome-Mac': { … }
}

23. customLaunchers: {
'IE8': {
base: 'WebDriver',
config: gridConfig,
browserName: 'internet
explorer',
platform: 'Windows XP',
version: '8',
name: 'Karma'
}
}

24. var gridConfig = {
hostname: '127.0.0.1',
port: 4444
};

25. browsers: ['IE8', 'IE9', 'IE10', 'Chrome-
Windows'],

26. You can override `server` mode from the cli
Continuous Integration Karma
$ karma start--reporters junit,progress --
single-run

27. What about the REST
of the application?

28. RESTful services have
an implicit contract
between provider and
consumer

29. This contract allows server
functionality to be checked via
REST only automation

30. Semantic Versioning 2.0.0
Version REST APIs:
//rest/1.1/my/resource
Given a version number
MAJOR.MINOR.PATCH
1. MAJOR version when you make incompatible
API changes,
2. MINOR version when you add functionality in
a backwards-compatible manner, and
3. PATCH version when you make backwards-
compatible bug fixes.

31. • JavaScript Code Coverage
• Long Feedback time
• Meeting our objective -
Compatibility
Functional tests have poor
performance:
Cross-browser unit test
JavaScript:
Summary
• Karma Runner
• Use your existing WebDriver
infrastructure
• Use code coverage

32. Questions
http://karma-
runner.github.io/
https://github.com/hindsigh
tsoftware/karma-webdriver-
example