0
WebTest Efficient Functional Web Testing with HtmlUnit and Beyond <ul><ul><li>Marc Guillemot </li></ul></ul><ul><ul><li>JU...
Your speaker <ul><li>French developer exiled in Germany </li></ul><ul><li>Independent consultant since 2002 </li></ul><ul>...
Agenda <ul><li>Introduction to WebTest </li></ul><ul><li>Test automation: a software engineering activity </li></ul><ul><l...
What I won't say <ul><li>Why you should automate your web application tests </li></ul><ul><li>How to install WebTest </li>...
What is WebTest <ul><li>Automated web functional testing tool </li></ul><ul><li>Open Source (Apache 2 license) </li></ul><...
Who uses WebTest? <ul><li>Open Source project => no idea! </li></ul><ul><li>Let's look at the mailing list: 700 subscriber...
demo (on a strange site)
Over 100 WebTest Steps <ul><li>General </li></ul><ul><ul><li><invoke .../> </li></ul></ul><ul><ul><li><clickLink .../> </l...
A software engineering activity
Apply rules of software engineering <ul><li>Remove duplications </li></ul><ul><li>Extract modules </li></ul><ul><li>Use so...
XPath <ul><li>Learn XPath </li></ul><ul><li>Learn XPath!!!  </li></ul><ul><li>Apply it wisely </li></ul><ul><ul><li>/html/...
Apply the right testing paradigm <ul><li>Capture / replay </li></ul><ul><li>Model based testing </li></ul><ul><li>Data dri...
demo WebTestRecorder
Model based testing <ul><li>Specification: modelling expected behavior </li></ul><ul><li>Be tolerant to accidental behavio...
demo data driven
Scripted automation <ul><li>Maximum flexibility and power </li></ul><ul><li>Maximum responsibility </li></ul>
Extending WebTest
Why? <ul><li>Go beyond built-in features </li></ul><ul><li>Customize for special needs </li></ul><ul><li>Make your tests e...
Building blocks WebTest Groovy Apache ANT Apache POI ... HtmlUnit Jakarta commons-httpclient NekoHTML Mozilla Rhino ...
WebTest uses HtmlUnit <ul><li>A “browser for Java programms” </li></ul><ul><li>100% pure java, headless </li></ul><ul><li>...
Go beyond built-in features <ul><li>... </li></ul><ul><li>< webtest > </li></ul><ul><li>< groovy description =&quot; confi...
Hack the web connection
Define custom steps <ul><li>Works but: </li></ul><ul><ul><li>too low level </li></ul></ul><ul><ul><li>not reusable </li></...
Define custom steps <ul><li>< groovyScript   name= &quot;verifyNavPath”><![CDATA[ </li></ul><ul><li>class  VerifyNavigatio...
Myths & Best Practices
“ UI tests  are brittle by nature”
Determine what you want to test! <ul><li>How to test addition into the basket? </li></ul><ul><ul><li>clickLink ”Einkaufsko...
wrong! “ UI tests  are brittle by nature”
“ Tests ensure that the application works”
demo http://new.jugcologne.org
wrong! => tests allow to find errors, that's all “ Tests ensure that the application works”
“ UI tests must run in the browser  to act like real users”
wrong! => a test pilot is not a standard pilot “ UI tests must run in the browser  to act like real users”
“ UI tests are slow”
Speed matters! <ul><li>Execution speed </li></ul><ul><ul><li>WebTest very fast: no rendering, single process, HtmlUnit's A...
demo wt.parallel.nbWorkers
wrong! “ UI tests are slow”
WebTest & AJAX
AJAX testing <ul><li>AJAX supported </li></ul><ul><ul><li>XMLHttpRequest supported since 2005 </li></ul></ul><ul><ul><li>B...
AJAX = testing challenge <ul><li>Selenium & Co' solution: </li></ul><ul><li>WebTest's solution: </li></ul>BUT not yet perf...
JS support not yet perfect <ul><li>HtmlUnit JS support continuously improved </li></ul><ul><li>HtmlUnit/WebTest already us...
Testing Facility example: mashup unavailability other.server.com www.myserver.com ?
Testing Facility example: mashup unavailability
WebTest's future(s)
WebTest's future(s) <ul><li>Integration of WebDriver </li></ul><ul><li>Integration with JMeter for load testing </li></ul>...
Integration with WebDriver N o w WebTest HtmlUnit HtmlUnit WebDriver Firefox IE ... C o m i n g WebTest
demo wt.withFirefox=1
demo WebTest Sampler
Conclusion
Lessons learned <ul><li>Use recorder only to give you a jump start </li></ul><ul><li>Write testable code </li></ul><ul><ul...
WebTest key properties <ul><li>simple </li></ul><ul><li>fast </li></ul><ul><li>excellent reporting </li></ul><ul><li>very ...
References <ul><li>WebTest http://webtest.canoo.com </li></ul><ul><ul><li>WebTestRecorder  http://webtestrecorder.canoo.co...
<ul><ul><li>Happy testing! </li></ul></ul>
Upcoming SlideShare
Loading in...5
×

WebTest - Efficient Functional Web Testing with HtmlUnit and Beyond

11,040

Published on

JUG Cologne presentation 25.08.08

Published in: Technology
1 Comment
6 Likes
Statistics
Notes
No Downloads
Views
Total Views
11,040
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
164
Comments
1
Likes
6
Embeds 0
No embeds

No notes for slide

Transcript of "WebTest - Efficient Functional Web Testing with HtmlUnit and Beyond"

  1. 1. WebTest Efficient Functional Web Testing with HtmlUnit and Beyond <ul><ul><li>Marc Guillemot </li></ul></ul><ul><ul><li>JUG Cologne </li></ul></ul><ul><ul><li>25.08.2008 </li></ul></ul>
  2. 2. Your speaker <ul><li>French developer exiled in Germany </li></ul><ul><li>Independent consultant since 2002 </li></ul><ul><li>Focus on Java, Groovy, Grails and ... web testing </li></ul><ul><li>HtmlUnit & WebTest lead developer </li></ul><ul><li>Committer to Groovy & NekoHTML </li></ul><ul><li>Blog: http://mguillem.wordpress.com </li></ul><ul><li>Site (coming): http://www.efficient-webtesting.com </li></ul>
  3. 3. Agenda <ul><li>Introduction to WebTest </li></ul><ul><li>Test automation: a software engineering activity </li></ul><ul><li>Extending WebTest </li></ul><ul><li>Myths and best practices </li></ul><ul><li>WebTest and AJAX </li></ul><ul><li>WebTest's future(s) </li></ul><ul><li>Conclusion </li></ul><ul><li>Q & A </li></ul>
  4. 4. What I won't say <ul><li>Why you should automate your web application tests </li></ul><ul><li>How to install WebTest </li></ul><ul><li>How to integrate into your build </li></ul><ul><li>Why Selenium sucks ;-) </li></ul>
  5. 5. What is WebTest <ul><li>Automated web functional testing tool </li></ul><ul><li>Open Source (Apache 2 license) </li></ul><ul><li>Founded in 2001 </li></ul><ul><li>Currently 4 committers (Germany, Switzerland, USA, Australia) </li></ul><ul><li>Built on top of Apache Ant </li></ul><ul><li>http://webtest.canoo.com </li></ul>
  6. 6. Who uses WebTest? <ul><li>Open Source project => no idea! </li></ul><ul><li>Let's look at the mailing list: 700 subscribers </li></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>*@apache.org </li></ul></ul><ul><ul><li>*@*.gov </li></ul></ul><ul><ul><li>Akamai, AMD, Axa, Bayer, Boeing, BT, Citigroup, Crédit-Suisse, CSC, DHL, EDS, IBM, Lexmark, Novartis, Oracle, Paypal, Sun, T-Mobile, Thalesgroup, Verisign, Zuehlke, ... </li></ul></ul><ul><li>=> everybody from small to huge </li></ul>
  7. 7. demo (on a strange site)
  8. 8. Over 100 WebTest Steps <ul><li>General </li></ul><ul><ul><li><invoke .../> </li></ul></ul><ul><ul><li><clickLink .../> </li></ul></ul><ul><ul><li>... </li></ul></ul><ul><li>Forms </li></ul><ul><ul><li><setInputField .../> </li></ul></ul><ul><ul><li><setRadioButton .../> </li></ul></ul><ul><ul><li><clickButton .../> </li></ul></ul><ul><ul><li>... </li></ul></ul><ul><li>Verification </li></ul><ul><ul><li><verifyTitle .../> </li></ul></ul><ul><ul><li><verifyXPath .../> </li></ul></ul><ul><ul><li><verifyInputField .../> </li></ul></ul><ul><ul><li>... </li></ul></ul><ul><li>PDF </li></ul><ul><ul><li><pdfDecryptDocument .../> </li></ul></ul><ul><ul><li><pdfVerifyField .../> </li></ul></ul><ul><ul><li>... </li></ul></ul><ul><li>Excel documents </li></ul><ul><ul><li><excelFindRow .../> </li></ul></ul><ul><ul><li><excelVerifyCellValue .../> </li></ul></ul><ul><ul><li>... </li></ul></ul><ul><li>Email, Applets, ... </li></ul><ul><li>Comprehensive online documentation http://webtest.canoo.com </li></ul>
  9. 9. A software engineering activity
  10. 10. Apply rules of software engineering <ul><li>Remove duplications </li></ul><ul><li>Extract modules </li></ul><ul><li>Use source control </li></ul><ul><li>Consider modifications and extensions </li></ul><ul><ul><li>Ant, Java, Groovy </li></ul></ul>
  11. 11. XPath <ul><li>Learn XPath </li></ul><ul><li>Learn XPath!!! </li></ul><ul><li>Apply it wisely </li></ul><ul><ul><li>/html/body/div[2]/center/table[3]/tr/td[@id='total'] </li></ul></ul><ul><ul><li>//*[@id='total'] </li></ul></ul><ul><li>Write HTML code allowing simple XPath expressions!!! </li></ul>
  12. 12. Apply the right testing paradigm <ul><li>Capture / replay </li></ul><ul><li>Model based testing </li></ul><ul><li>Data driven testing </li></ul><ul><li>Scripted automation </li></ul>
  13. 13. demo WebTestRecorder
  14. 14. Model based testing <ul><li>Specification: modelling expected behavior </li></ul><ul><li>Be tolerant to accidental behavior </li></ul><ul><ul><li>Example: </li></ul></ul><ul><ul><ul><li>bad XPath: /html/body/div[2]/center/table[3] </li></ul></ul></ul><ul><ul><ul><li>better: //*[@id='total'] </li></ul></ul></ul><ul><li>Fault model: modelling unexpected behavior </li></ul><ul><li>< not > </li></ul><ul><ul><ul><li>< verifyText text= &quot;NullPointerException&quot;/> </li></ul></ul></ul><ul><ul><ul><li></ not > </li></ul></ul></ul>
  15. 15. demo data driven
  16. 16. Scripted automation <ul><li>Maximum flexibility and power </li></ul><ul><li>Maximum responsibility </li></ul>
  17. 17. Extending WebTest
  18. 18. Why? <ul><li>Go beyond built-in features </li></ul><ul><li>Customize for special needs </li></ul><ul><li>Make your tests easier to write </li></ul><ul><li>Make your tests results easier to read </li></ul>
  19. 19. Building blocks WebTest Groovy Apache ANT Apache POI ... HtmlUnit Jakarta commons-httpclient NekoHTML Mozilla Rhino ...
  20. 20. WebTest uses HtmlUnit <ul><li>A “browser for Java programms” </li></ul><ul><li>100% pure java, headless </li></ul><ul><li>Able to simulate FF or IE </li></ul><ul><li>Very fast and leightweight </li></ul><ul><li>Used in WebTest but as well in JSFUnit, WebDriver, JWebUnit, Cactus, Celerity, Schnell, ... </li></ul><ul><li>Very good JS support... but not yet perfect </li></ul>
  21. 21. Go beyond built-in features <ul><li>... </li></ul><ul><li>< webtest > </li></ul><ul><li>< groovy description =&quot; configure NTLM proxy credentials &quot;> </li></ul><ul><ul><ul><li>step.context.webClient.credentialsProvider.addNTLMProxyCredentials( </li></ul></ul></ul><ul><ul><ul><li>&quot;michael&quot;, &quot;secret&quot;, </li></ul></ul></ul><ul><ul><ul><li>&quot;testserver&quot;, 12345, </li></ul></ul></ul><ul><ul><ul><li>&quot;BlueGene&quot;, &quot;my.windows.domain&quot;) </li></ul></ul></ul><ul><li></ groovy > </li></ul><ul><li>< invoke url =&quot; http://testserver/ &quot;/> </li></ul><ul><ul><li>..... </li></ul></ul><ul><li></ webtest > </li></ul><ul><li>... </li></ul>
  22. 22. Hack the web connection
  23. 23. Define custom steps <ul><li>Works but: </li></ul><ul><ul><li>too low level </li></ul></ul><ul><ul><li>not reusable </li></ul></ul><ul><li>... </li></ul><ul><li>< verifyXPath xpath= &quot;//*[@id='navPath1']” text= &quot;Home”/> </li></ul><ul><li>< verifyXPath xpath= &quot;//*[@id='navPath2']” text= &quot;Functional testing”/> </li></ul><ul><li>< verifyXPath xpath= &quot;//*[@id='navPath3']” text= &quot;WebTest”/> </li></ul><ul><li>... </li></ul>
  24. 24. Define custom steps <ul><li>< groovyScript name= &quot;verifyNavPath”><![CDATA[ </li></ul><ul><li>class VerifyNavigationPath extends com.canoo.webtest.steps.Step { </li></ul><ul><li>String level1, level2, level3, level4 </li></ul><ul><li>void doExecute() { </li></ul><ul><li>def ant = new AntBuilder(project) </li></ul><ul><li>def levels = [0, level1, level2, level3, level4] </li></ul><ul><li>for (i in 1..<levels.size()) { </li></ul><ul><li>if (levels[i]) </li></ul><ul><li>ant.verifyXPath(xpath: &quot;//a[@id='navPath${i}']/text()&quot;, text: levels[i], </li></ul><ul><ul><ul><li>description: &quot;Verify level ${i}&quot;) </li></ul></ul></ul><ul><li>} } } </li></ul><ul><li>project.addTaskDefinition('verifyNavPath', VerifyNavigationPath) </li></ul><ul><li>]]></ groovyScript > </li></ul><ul><li>... </li></ul><ul><li>< verifyNavPath level1= &quot;Home”/> </li></ul><ul><li>< verifyNavPath level1= &quot;Home” level2= &quot;Functional testing” level3= &quot;WebTest”/> </li></ul>
  25. 25. Myths & Best Practices
  26. 26. “ UI tests are brittle by nature”
  27. 27. Determine what you want to test! <ul><li>How to test addition into the basket? </li></ul><ul><ul><li>clickLink ”Einkaufskorb”? </li></ul></ul><ul><ul><li>clickLink “Ajax” ? </li></ul></ul><ul><ul><li>clickLink xpath=”//*[text() = 'Top Angebot']//a” ? </li></ul></ul><ul><ul><li>clickLink xpath=”//*[text() = '6.60']/following-sibling::a” ? </li></ul></ul><ul><li>=> it depends! These 4 examples don't test the same thing ! </li></ul>
  28. 28. wrong! “ UI tests are brittle by nature”
  29. 29. “ Tests ensure that the application works”
  30. 30. demo http://new.jugcologne.org
  31. 31. wrong! => tests allow to find errors, that's all “ Tests ensure that the application works”
  32. 32. “ UI tests must run in the browser to act like real users”
  33. 33. wrong! => a test pilot is not a standard pilot “ UI tests must run in the browser to act like real users”
  34. 34. “ UI tests are slow”
  35. 35. Speed matters! <ul><li>Execution speed </li></ul><ul><ul><li>WebTest very fast: no rendering, single process, HtmlUnit's API allows a better control </li></ul></ul><ul><ul><li>HtmlUnit vs Watir </li></ul></ul><ul><ul><ul><li>Celerity benchmarks: 69%-99% time reduction </li></ul></ul></ul><ul><ul><ul><li>Schnell benchmark: 97% time reduction </li></ul></ul></ul><ul><li>Report analyse speed </li></ul><ul><ul><li>Very rich reports </li></ul></ul><ul><ul><li>Debugging (mostly) useless </li></ul></ul>
  36. 36. demo wt.parallel.nbWorkers
  37. 37. wrong! “ UI tests are slow”
  38. 38. WebTest & AJAX
  39. 39. AJAX testing <ul><li>AJAX supported </li></ul><ul><ul><li>XMLHttpRequest supported since 2005 </li></ul></ul><ul><ul><li>But AJAX = complex JS libraries </li></ul></ul><ul><li>AJAX = testing challenge </li></ul><ul><li>WebTest JS support not yet perfect </li></ul><ul><li>Simulated browser offers testing facilities </li></ul>
  40. 40. AJAX = testing challenge <ul><li>Selenium & Co' solution: </li></ul><ul><li>WebTest's solution: </li></ul>BUT not yet perfect - works well with XHR and setTimeout - doesn't work well with setInterval ;-( nothing here! clickButton &quot;Update&quot; click &quot;Update&quot; waitForElementPresent &quot;some elt&quot; click
  41. 41. JS support not yet perfect <ul><li>HtmlUnit JS support continuously improved </li></ul><ul><li>HtmlUnit/WebTest already used in numerous projects to test AJAX applications </li></ul><ul><li>Currently supported libraries: GWT, Sarissa, ... </li></ul>
  42. 42. Testing Facility example: mashup unavailability other.server.com www.myserver.com ?
  43. 43. Testing Facility example: mashup unavailability
  44. 44. WebTest's future(s)
  45. 45. WebTest's future(s) <ul><li>Integration of WebDriver </li></ul><ul><li>Integration with JMeter for load testing </li></ul><ul><li>... </li></ul>
  46. 46. Integration with WebDriver N o w WebTest HtmlUnit HtmlUnit WebDriver Firefox IE ... C o m i n g WebTest
  47. 47. demo wt.withFirefox=1
  48. 48. demo WebTest Sampler
  49. 49. Conclusion
  50. 50. Lessons learned <ul><li>Use recorder only to give you a jump start </li></ul><ul><li>Write testable code </li></ul><ul><ul><li>write coherent html code </li></ul></ul><ul><ul><li>add nodes, ids, ... to make testing easier </li></ul></ul><ul><li>Test early </li></ul><ul><li>Modulize / Refactor / Reuse code </li></ul><ul><li>Customize your test tool </li></ul><ul><li>Distinguish guaranted from accidental behavior: </li></ul><ul><ul><li>which changes in the application should make a step fail </li></ul></ul><ul><ul><li>which changes in the application should NOT make it fail </li></ul></ul>
  51. 51. WebTest key properties <ul><li>simple </li></ul><ul><li>fast </li></ul><ul><li>excellent reporting </li></ul><ul><li>very low TCO </li></ul><ul><li>runs everywhere </li></ul><ul><li>no display needed </li></ul><ul><li>easy to extend </li></ul><ul><li>straightforward integration </li></ul><ul><li>js support not as good as in “normal” browser </li></ul><ul><li>doesn't accept (too) badly formatted html code </li></ul><ul><li>doesn't accept (too) badly formatted html code </li></ul>
  52. 52. References <ul><li>WebTest http://webtest.canoo.com </li></ul><ul><ul><li>WebTestRecorder http://webtestrecorder.canoo.com </li></ul></ul><ul><ul><li>Mailing list http://lists.canoo.com/mailman/listinfo/webtest </li></ul></ul><ul><li>WebTest screencasts </li></ul><ul><ul><li>Creating a first WebTest project http://opensource.basehaus.com/webtest/screencasts/creating-a-first-webtest-project.htm </li></ul></ul><ul><ul><li>Data driven WebTest http://opensource.basehaus.com/webtest/screencasts/data-driven-webtest.htm </li></ul></ul><ul><li>HtmlUnit http://htmlunit.sf.net </li></ul><ul><li>WebDriver http://code.google.com/p/webdriver/ </li></ul><ul><li>JMeter http://jakarta.apache.org/jmeter/ </li></ul><ul><li>Why Selenium sucks (Mark Striebeck, Selenium User Meetup) http://br.youtube.com/watch?v=EDb8yOM3Vpw </li></ul><ul><li>Groovy http://groovy.codehaus.org/ </li></ul><ul><ul><li>AntBuilder http://groovy.codehaus.org/Using+Ant+from+Groovy </li></ul></ul><ul><li>Watir http://wtr.rubyforge.org/ </li></ul><ul><li>Schnell http://code.google.com/p/schnell-jruby/ </li></ul><ul><li>Celerity http://celerity.rubyforge.org/ </li></ul><ul><li>JSFUnit http://www.jboss.org/jsfunit/ </li></ul>
  53. 53. <ul><ul><li>Happy testing! </li></ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×