1. Pitfalls and tips withPitfalls and tips with
browser based testIngbrowser based testIng

2. My Background
several projects with functional testing
consulted at organizations to introduce functional testing
and builds
heavily involved in trying to improve effectiveness and
reduce maintenance
developed a framework over several projects that made use
of my learnings

3. Common uses of functional
tests
smoke/sanity tests for most valuable and risky features
black box/regression tests
post deployment checks and rollbacks
multi browser testing

4. what do we want from
tests?
correctness (when my tests work, my application is not
broken)
robustness (when my tests break, my application is broken)

5. what do we focus on?
what’s valuable? (it matters to us that people can do this)
what’s risky? (if this breaks, it really affects us)

6. so whats the problem?so whats the problem?

7. five problemsfive problems

8. 1. slow tests1. slow tests

9. BecauseBecause

10. uses theuses the
entire stackentire stack

11. low spec hardwarelow spec hardware

12. tons of setuptons of setup
and teardownand teardown

13. talking to thetalking to the
browser is slowbrowser is slow

14. soso

15. very long feedback loopvery long feedback loop

16. can delay deploymentscan delay deployments

17. 2. Brittle tests2. Brittle tests

18. UIUI
changeschanges
a lota lot

19. tests overtests over
specifyspecify

20. 3. Random failures3. Random failures

21. race conditionsrace conditions

22. unreliable dependenciesunreliable dependencies

23. 4. tests are hard to debug4. tests are hard to debug

24. too manytoo many
tests breaktests break

25. error messageserror messages
tell you nothingtell you nothing

26. 5. test code becomes5. test code becomes
unmaintainableunmaintainable

27. insufficient commitmentinsufficient commitment
to testingto testing

28. inadequateinadequate
understandingunderstanding
of testingof testing

29. what happens
there is an initial stream of tests
tests break unnecessarily
tests become hard to maintain
team loses confidence and motivation
some teams even abandon testing via the UI

30. tips and trickstips and tricks

31. Treat test code
like real code
like real code
speed
maintainability

32. A lot of the techniques we useA lot of the techniques we use
with real code work with testwith real code work with test
codecode

33. similar levels ofsimilar levels of
commitmentcommitment

34. testtest
creation/maintenancecreation/maintenance
can be part of the devcan be part of the dev
cyclecycle

35. Use Ruby
maintainability
development time

36. use a headless browser
when possible
speed

37. Use selenium RC
(if you use selenium)
avoid brittleness
maintainability

38. Don’t record and replay
(if you use selenium)
avoid brittleness
maintainability
development time

39. Introduce an Abstraction
layer
speed
maintainability

40. Categorize/tag tests
speed
maintainability

41. Use the
page object pattern
maintainability

42. Parallelize
speed

43. separate data
for tests
speed
avoid random failures

44. clump data into common
location
maintainability

45. Push setup and teardown
out of tests
speed
avoid brittleness
development time
debuggability

46. Only exercise parts of the
application you want to test
speed
avoid brittleness
debuggability

47. Build stub services
speed
avoid random failures

48. don’t over specify
avoid brittleness
debuggability

49. run on better hardware
speed

50. slice vertically
debuggability

51. Use the right locator
maintainability
avoid brittleness
speed

52. learn xpath/css selectors
avoid brittleness
maintainability

53. fail later
speed

54. use polls over sleep
speed
avoid random failures

55. use existing waits of
selenium
speed
avoid random failures
avoid brittleness
avoid random

56. Don’t use assertions to
identify transitions
avoid brittleness
debuggability
avoid brittleness

57. screenshots and html
dumps
debuggability

58. xpath/css selector tester
development time

59. firebug or w/e
development time
maintainability

60. talk to me if you’retalk to me if you’re
testing tabular datatesting tabular data

61. I’m always available forI’m always available for
questionsquestions

62. Questions?Questions?

63. thank youthank you