WebTest - Efficient Functional Web Testing with HtmlUnit and Beyond
Upcoming SlideShare
Loading in...5
×
 

WebTest - Efficient Functional Web Testing with HtmlUnit and Beyond

on

  • 13,804 views

JUG Cologne presentation 25.08.08

JUG Cologne presentation 25.08.08

Statistics

Views

Total Views
13,804
Views on SlideShare
10,779
Embed Views
3,025

Actions

Likes
6
Downloads
156
Comments
1

12 Embeds 3,025

http://mguillem.wordpress.com 2937
http://www.slideshare.net 29
https://mguillem.wordpress.com 23
http://translate.googleusercontent.com 13
http://115.112.206.131 7
url_unknown 7
http://www-staging.bethel.jw.org 3
http://webcache.googleusercontent.com 2
http://health.medicbd.com 1
http://www.health.medicbd.com 1
https://s3.amazonaws.com 1
http://a0.twimg.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as OpenOffice

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

WebTest - Efficient Functional Web Testing with HtmlUnit and Beyond WebTest - Efficient Functional Web Testing with HtmlUnit and Beyond Presentation Transcript

  • WebTest Efficient Functional Web Testing with HtmlUnit and Beyond
      • Marc Guillemot
      • JUG Cologne
      • 25.08.2008
  • Your speaker
    • French developer exiled in Germany
    • Independent consultant since 2002
    • Focus on Java, Groovy, Grails and ... web testing
    • HtmlUnit & WebTest lead developer
    • Committer to Groovy & NekoHTML
    • Blog: http://mguillem.wordpress.com
    • Site (coming): http://www.efficient-webtesting.com
  • Agenda
    • Introduction to WebTest
    • Test automation: a software engineering activity
    • Extending WebTest
    • Myths and best practices
    • WebTest and AJAX
    • WebTest's future(s)
    • Conclusion
    • Q & A
  • What I won't say
    • Why you should automate your web application tests
    • How to install WebTest
    • How to integrate into your build
    • Why Selenium sucks ;-)
  • What is WebTest
    • Automated web functional testing tool
    • Open Source (Apache 2 license)
    • Founded in 2001
    • Currently 4 committers (Germany, Switzerland, USA, Australia)
    • Built on top of Apache Ant
    • http://webtest.canoo.com
  • Who uses WebTest?
    • Open Source project => no idea!
    • Let's look at the mailing list: 700 subscribers
      • [email_address]
      • *@apache.org
      • *@*.gov
      • Akamai, AMD, Axa, Bayer, Boeing, BT, Citigroup, Crédit-Suisse, CSC, DHL, EDS, IBM, Lexmark, Novartis, Oracle, Paypal, Sun, T-Mobile, Thalesgroup, Verisign, Zuehlke, ...
    • => everybody from small to huge
  • demo (on a strange site)
  • Over 100 WebTest Steps
    • General
      • <invoke .../>
      • <clickLink .../>
      • ...
    • Forms
      • <setInputField .../>
      • <setRadioButton .../>
      • <clickButton .../>
      • ...
    • Verification
      • <verifyTitle .../>
      • <verifyXPath .../>
      • <verifyInputField .../>
      • ...
    • PDF
      • <pdfDecryptDocument .../>
      • <pdfVerifyField .../>
      • ...
    • Excel documents
      • <excelFindRow .../>
      • <excelVerifyCellValue .../>
      • ...
    • Email, Applets, ...
    • Comprehensive online documentation http://webtest.canoo.com
  • A software engineering activity
  • Apply rules of software engineering
    • Remove duplications
    • Extract modules
    • Use source control
    • Consider modifications and extensions
      • Ant, Java, Groovy
  • XPath
    • Learn XPath
    • Learn XPath!!!
    • Apply it wisely
      • /html/body/div[2]/center/table[3]/tr/td[@id='total']
      • //*[@id='total']
    • Write HTML code allowing simple XPath expressions!!!
  • Apply the right testing paradigm
    • Capture / replay
    • Model based testing
    • Data driven testing
    • Scripted automation
  • demo WebTestRecorder
  • Model based testing
    • Specification: modelling expected behavior
    • Be tolerant to accidental behavior
      • Example:
        • bad XPath: /html/body/div[2]/center/table[3]
        • better: //*[@id='total']
    • Fault model: modelling unexpected behavior
    • < not >
        • < verifyText text= &quot;NullPointerException&quot;/>
        • </ not >
  • demo data driven
  • Scripted automation
    • Maximum flexibility and power
    • Maximum responsibility
  • Extending WebTest
  • Why?
    • Go beyond built-in features
    • Customize for special needs
    • Make your tests easier to write
    • Make your tests results easier to read
  • Building blocks WebTest Groovy Apache ANT Apache POI ... HtmlUnit Jakarta commons-httpclient NekoHTML Mozilla Rhino ...
  • WebTest uses HtmlUnit
    • A “browser for Java programms”
    • 100% pure java, headless
    • Able to simulate FF or IE
    • Very fast and leightweight
    • Used in WebTest but as well in JSFUnit, WebDriver, JWebUnit, Cactus, Celerity, Schnell, ...
    • Very good JS support... but not yet perfect
  • Go beyond built-in features
    • ...
    • < webtest >
    • < groovy description =&quot; configure NTLM proxy credentials &quot;>
        • step.context.webClient.credentialsProvider.addNTLMProxyCredentials(
        • &quot;michael&quot;, &quot;secret&quot;,
        • &quot;testserver&quot;, 12345,
        • &quot;BlueGene&quot;, &quot;my.windows.domain&quot;)
    • </ groovy >
    • < invoke url =&quot; http://testserver/ &quot;/>
      • .....
    • </ webtest >
    • ...
  • Hack the web connection
  • Define custom steps
    • Works but:
      • too low level
      • not reusable
    • ...
    • < verifyXPath xpath= &quot;//*[@id='navPath1']” text= &quot;Home”/>
    • < verifyXPath xpath= &quot;//*[@id='navPath2']” text= &quot;Functional testing”/>
    • < verifyXPath xpath= &quot;//*[@id='navPath3']” text= &quot;WebTest”/>
    • ...
  • Define custom steps
    • < groovyScript name= &quot;verifyNavPath”><![CDATA[
    • class VerifyNavigationPath extends com.canoo.webtest.steps.Step {
    • String level1, level2, level3, level4
    • void doExecute() {
    • def ant = new AntBuilder(project)
    • def levels = [0, level1, level2, level3, level4]
    • for (i in 1..<levels.size()) {
    • if (levels[i])
    • ant.verifyXPath(xpath: &quot;//a[@id='navPath${i}']/text()&quot;, text: levels[i],
        • description: &quot;Verify level ${i}&quot;)
    • } } }
    • project.addTaskDefinition('verifyNavPath', VerifyNavigationPath)
    • ]]></ groovyScript >
    • ...
    • < verifyNavPath level1= &quot;Home”/>
    • < verifyNavPath level1= &quot;Home” level2= &quot;Functional testing” level3= &quot;WebTest”/>
  • Myths & Best Practices
  • “ UI tests are brittle by nature”
  • Determine what you want to test!
    • How to test addition into the basket?
      • clickLink ”Einkaufskorb”?
      • clickLink “Ajax” ?
      • clickLink xpath=”//*[text() = 'Top Angebot']//a” ?
      • clickLink xpath=”//*[text() = '6.60']/following-sibling::a” ?
    • => it depends! These 4 examples don't test the same thing !
  • wrong! “ UI tests are brittle by nature”
  • “ Tests ensure that the application works”
  • demo http://new.jugcologne.org
  • wrong! => tests allow to find errors, that's all “ Tests ensure that the application works”
  • “ UI tests must run in the browser to act like real users”
  • wrong! => a test pilot is not a standard pilot “ UI tests must run in the browser to act like real users”
  • “ UI tests are slow”
  • Speed matters!
    • Execution speed
      • WebTest very fast: no rendering, single process, HtmlUnit's API allows a better control
      • HtmlUnit vs Watir
        • Celerity benchmarks: 69%-99% time reduction
        • Schnell benchmark: 97% time reduction
    • Report analyse speed
      • Very rich reports
      • Debugging (mostly) useless
  • demo wt.parallel.nbWorkers
  • wrong! “ UI tests are slow”
  • WebTest & AJAX
  • AJAX testing
    • AJAX supported
      • XMLHttpRequest supported since 2005
      • But AJAX = complex JS libraries
    • AJAX = testing challenge
    • WebTest JS support not yet perfect
    • Simulated browser offers testing facilities
  • AJAX = testing challenge
    • Selenium & Co' solution:
    • WebTest's solution:
    BUT not yet perfect - works well with XHR and setTimeout - doesn't work well with setInterval ;-( nothing here! clickButton &quot;Update&quot; click &quot;Update&quot; waitForElementPresent &quot;some elt&quot; click
  • JS support not yet perfect
    • HtmlUnit JS support continuously improved
    • HtmlUnit/WebTest already used in numerous projects to test AJAX applications
    • Currently supported libraries: GWT, Sarissa, ...
  • Testing Facility example: mashup unavailability other.server.com www.myserver.com ?
  • Testing Facility example: mashup unavailability
  • WebTest's future(s)
  • WebTest's future(s)
    • Integration of WebDriver
    • Integration with JMeter for load testing
    • ...
  • Integration with WebDriver N o w WebTest HtmlUnit HtmlUnit WebDriver Firefox IE ... C o m i n g WebTest
  • demo wt.withFirefox=1
  • demo WebTest Sampler
  • Conclusion
  • Lessons learned
    • Use recorder only to give you a jump start
    • Write testable code
      • write coherent html code
      • add nodes, ids, ... to make testing easier
    • Test early
    • Modulize / Refactor / Reuse code
    • Customize your test tool
    • Distinguish guaranted from accidental behavior:
      • which changes in the application should make a step fail
      • which changes in the application should NOT make it fail
  • WebTest key properties
    • simple
    • fast
    • excellent reporting
    • very low TCO
    • runs everywhere
    • no display needed
    • easy to extend
    • straightforward integration
    • js support not as good as in “normal” browser
    • doesn't accept (too) badly formatted html code
    • doesn't accept (too) badly formatted html code
  • References
    • WebTest http://webtest.canoo.com
      • WebTestRecorder http://webtestrecorder.canoo.com
      • Mailing list http://lists.canoo.com/mailman/listinfo/webtest
    • WebTest screencasts
      • Creating a first WebTest project http://opensource.basehaus.com/webtest/screencasts/creating-a-first-webtest-project.htm
      • Data driven WebTest http://opensource.basehaus.com/webtest/screencasts/data-driven-webtest.htm
    • HtmlUnit http://htmlunit.sf.net
    • WebDriver http://code.google.com/p/webdriver/
    • JMeter http://jakarta.apache.org/jmeter/
    • Why Selenium sucks (Mark Striebeck, Selenium User Meetup) http://br.youtube.com/watch?v=EDb8yOM3Vpw
    • Groovy http://groovy.codehaus.org/
      • AntBuilder http://groovy.codehaus.org/Using+Ant+from+Groovy
    • Watir http://wtr.rubyforge.org/
    • Schnell http://code.google.com/p/schnell-jruby/
    • Celerity http://celerity.rubyforge.org/
    • JSFUnit http://www.jboss.org/jsfunit/
      • Happy testing!