The document is a presentation about WebTest, an open source tool for automated functional web testing. It discusses how WebTest works, how to extend its capabilities using Groovy, common myths about automated web testing, and the tool's future integration with WebDriver. The presentation aims to demonstrate how WebTest can be used to efficiently test web applications.

1. WebTest Efficient Functional Web Testing with HtmlUnit and Beyond Marc Guillemot JUG Cologne 25.08.2008

2. Your speaker French developer exiled in Germany Independent consultant since 2002 Focus on Java, Groovy, Grails and ... web testing HtmlUnit & WebTest lead developer Committer to Groovy & NekoHTML Blog: http://mguillem.wordpress.com Site (coming): http://www.efficient-webtesting.com

3. Agenda Introduction to WebTest Test automation: a software engineering activity Extending WebTest Myths and best practices WebTest and AJAX WebTest's future(s) Conclusion Q & A

4. What I won't say Why you should automate your web application tests How to install WebTest How to integrate into your build Why Selenium sucks ;-)

5. What is WebTest Automated web functional testing tool Open Source (Apache 2 license) Founded in 2001 Currently 4 committers (Germany, Switzerland, USA, Australia) Built on top of Apache Ant http://webtest.canoo.com

6. Who uses WebTest? Open Source project => no idea! Let's look at the mailing list: 700 subscribers [email_address] *@apache.org *@*.gov Akamai, AMD, Axa, Bayer, Boeing, BT, Citigroup, Crédit-Suisse, CSC, DHL, EDS, IBM, Lexmark, Novartis, Oracle, Paypal, Sun, T-Mobile, Thalesgroup, Verisign, Zuehlke, ... => everybody from small to huge

7. demo (on a strange site)

8. Over 100 WebTest Steps General <invoke .../> <clickLink .../> ... Forms <setInputField .../> <setRadioButton .../> <clickButton .../> ... Verification <verifyTitle .../> <verifyXPath .../> <verifyInputField .../> ... PDF <pdfDecryptDocument .../> <pdfVerifyField .../> ... Excel documents <excelFindRow .../> <excelVerifyCellValue .../> ... Email, Applets, ... Comprehensive online documentation http://webtest.canoo.com

9. A software engineering activity

10. Apply rules of software engineering Remove duplications Extract modules Use source control Consider modifications and extensions Ant, Java, Groovy

11. XPath Learn XPath Learn XPath!!! Apply it wisely /html/body/div[2]/center/table[3]/tr/td[@id='total'] //*[@id='total'] Write HTML code allowing simple XPath expressions!!!

12. Apply the right testing paradigm Capture / replay Model based testing Data driven testing Scripted automation

13. demo WebTestRecorder

14. Model based testing Specification: modelling expected behavior Be tolerant to accidental behavior Example: bad XPath: /html/body/div[2]/center/table[3] better: //*[@id='total'] Fault model: modelling unexpected behavior < not > < verifyText text= &quot;NullPointerException&quot;/> </ not >

15. demo data driven

16. Scripted automation Maximum flexibility and power Maximum responsibility

17. Extending WebTest

18. Why? Go beyond built-in features Customize for special needs Make your tests easier to write Make your tests results easier to read

19. Building blocks WebTest Groovy Apache ANT Apache POI ... HtmlUnit Jakarta commons-httpclient NekoHTML Mozilla Rhino ...

20. WebTest uses HtmlUnit A “browser for Java programms” 100% pure java, headless Able to simulate FF or IE Very fast and leightweight Used in WebTest but as well in JSFUnit, WebDriver, JWebUnit, Cactus, Celerity, Schnell, ... Very good JS support... but not yet perfect

21. Go beyond built-in features ... < webtest > < groovy description =&quot; configure NTLM proxy credentials &quot;> step.context.webClient.credentialsProvider.addNTLMProxyCredentials( &quot;michael&quot;, &quot;secret&quot;, &quot;testserver&quot;, 12345, &quot;BlueGene&quot;, &quot;my.windows.domain&quot;) </ groovy > < invoke url =&quot; http://testserver/ &quot;/> ..... </ webtest > ...

22. Hack the web connection

23. Define custom steps Works but: too low level not reusable ... < verifyXPath xpath= &quot;//*[@id='navPath1']” text= &quot;Home”/> < verifyXPath xpath= &quot;//*[@id='navPath2']” text= &quot;Functional testing”/> < verifyXPath xpath= &quot;//*[@id='navPath3']” text= &quot;WebTest”/> ...

24. Define custom steps < groovyScript name= &quot;verifyNavPath”><![CDATA[ class VerifyNavigationPath extends com.canoo.webtest.steps.Step { String level1, level2, level3, level4 void doExecute() { def ant = new AntBuilder(project) def levels = [0, level1, level2, level3, level4] for (i in 1..<levels.size()) { if (levels[i]) ant.verifyXPath(xpath: &quot;//a[@id='navPath${i}']/text()&quot;, text: levels[i], description: &quot;Verify level ${i}&quot;) } } } project.addTaskDefinition('verifyNavPath', VerifyNavigationPath) ]]></ groovyScript > ... < verifyNavPath level1= &quot;Home”/> < verifyNavPath level1= &quot;Home” level2= &quot;Functional testing” level3= &quot;WebTest”/>

25. Myths & Best Practices

26. “ UI tests are brittle by nature”

27. Determine what you want to test! How to test addition into the basket? clickLink ”Einkaufskorb”? clickLink “Ajax” ? clickLink xpath=”//*[text() = 'Top Angebot']//a” ? clickLink xpath=”//*[text() = '6.60']/following-sibling::a” ? => it depends! These 4 examples don't test the same thing !

28. wrong! “ UI tests are brittle by nature”

29. “ Tests ensure that the application works”

30. demo http://new.jugcologne.org

31. wrong! => tests allow to find errors, that's all “ Tests ensure that the application works”

32. “ UI tests must run in the browser to act like real users”

33. wrong! => a test pilot is not a standard pilot “ UI tests must run in the browser to act like real users”

34. “ UI tests are slow”

35. Speed matters! Execution speed WebTest very fast: no rendering, single process, HtmlUnit's API allows a better control HtmlUnit vs Watir Celerity benchmarks: 69%-99% time reduction Schnell benchmark: 97% time reduction Report analyse speed Very rich reports Debugging (mostly) useless

36. demo wt.parallel.nbWorkers

37. wrong! “ UI tests are slow”

38. WebTest & AJAX

39. AJAX testing AJAX supported XMLHttpRequest supported since 2005 But AJAX = complex JS libraries AJAX = testing challenge WebTest JS support not yet perfect Simulated browser offers testing facilities

40. AJAX = testing challenge Selenium & Co' solution: WebTest's solution: BUT not yet perfect - works well with XHR and setTimeout - doesn't work well with setInterval ;-( nothing here! clickButton &quot;Update&quot; click &quot;Update&quot; waitForElementPresent &quot;some elt&quot; click

41. JS support not yet perfect HtmlUnit JS support continuously improved HtmlUnit/WebTest already used in numerous projects to test AJAX applications Currently supported libraries: GWT, Sarissa, ...

42. Testing Facility example: mashup unavailability other.server.com www.myserver.com ?

43. Testing Facility example: mashup unavailability

44. WebTest's future(s)

45. WebTest's future(s) Integration of WebDriver Integration with JMeter for load testing ...

46. Integration with WebDriver N o w WebTest HtmlUnit HtmlUnit WebDriver Firefox IE ... C o m i n g WebTest

47. demo wt.withFirefox=1

48. demo WebTest Sampler

49. Conclusion

50. Lessons learned Use recorder only to give you a jump start Write testable code write coherent html code add nodes, ids, ... to make testing easier Test early Modulize / Refactor / Reuse code Customize your test tool Distinguish guaranted from accidental behavior: which changes in the application should make a step fail which changes in the application should NOT make it fail

51. WebTest key properties simple fast excellent reporting very low TCO runs everywhere no display needed easy to extend straightforward integration js support not as good as in “normal” browser doesn't accept (too) badly formatted html code doesn't accept (too) badly formatted html code

52. References WebTest http://webtest.canoo.com WebTestRecorder http://webtestrecorder.canoo.com Mailing list http://lists.canoo.com/mailman/listinfo/webtest WebTest screencasts Creating a first WebTest project http://opensource.basehaus.com/webtest/screencasts/creating-a-first-webtest-project.htm Data driven WebTest http://opensource.basehaus.com/webtest/screencasts/data-driven-webtest.htm HtmlUnit http://htmlunit.sf.net WebDriver http://code.google.com/p/webdriver/ JMeter http://jakarta.apache.org/jmeter/ Why Selenium sucks (Mark Striebeck, Selenium User Meetup) http://br.youtube.com/watch?v=EDb8yOM3Vpw Groovy http://groovy.codehaus.org/ AntBuilder http://groovy.codehaus.org/Using+Ant+from+Groovy Watir http://wtr.rubyforge.org/ Schnell http://code.google.com/p/schnell-jruby/ Celerity http://celerity.rubyforge.org/ JSFUnit http://www.jboss.org/jsfunit/

53. Happy testing!