Writing an extensible web testing framework ready for the cloud   slide share
Upcoming SlideShare
Loading in...5
×
 

Writing an extensible web testing framework ready for the cloud slide share

on

  • 1,309 views

Presentation given at GR8Conf in Copenhagen, Denmark on May 22nd, 2013

Presentation given at GR8Conf in Copenhagen, Denmark on May 22nd, 2013

Statistics

Views

Total Views
1,309
Views on SlideShare
1,062
Embed Views
247

Actions

Likes
3
Downloads
16
Comments
0

4 Embeds 247

http://gr8conf.eu 183
http://2013.gr8conf.eu 49
https://twitter.com 12
http://m.gr8conf.eu 3

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Writing an extensible web testing framework ready for the cloud   slide share Writing an extensible web testing framework ready for the cloud slide share Presentation Transcript

  • Writing an Extensible Web TestingFramework ready for the CloudPresented by Mike Ensor May 22, 2013 – Presented at GR8Conf in Copenhagen Denmark
  • ©2013 Acquity Group, LLC. All rights reserved.Introductions2• What is this talk about?• Curious about automated browser testing• Testing a Reference application and/or SaaS• Expanding Geb’s inheritance model• Who is Mike!?• Developing software for 16+ years• Started working with Groovy and Grails in 2007• Attended GR8Conf last year• Acquity Group• Blending strategy, creative and technology• Currently a principal architect; Intershop and Demandware platform• Introduced Geb and Spock to Demandware practice
  • ©2013 Acquity Group, LLC. All rights reserved.Useful Links3Demandware testing framework• http://mikeensor.bitbucket.org/demandware-testing-frameworkSpock feature override extension• https://github.com/mike-ensor/spock-feature-override-extensionPresentation• http://www.slideshare.net/MikeEnsor/Twitter @mikeensor• http://www.twitter.com/mikeensorBlog• http://www.ensor.cc
  • Web Testing
  • ©2013 Acquity Group, LLC. All rights reserved.Why Test?5RaiseConfidenceSafety NetProveFunctionalityEnsureQualityLower Costs
  • ©2013 Acquity Group, LLC. All rights reserved.Cost Savings6$0$1,000$2,000$3,000$4,000$5,000$6,000Developer Unit Tests IntegrationTestsEnvironmentCost per DefectAvg CostGoogle Study presented at XP Days London 2009 – Mark Streibeck, Google Inc
  • ©2013 Acquity Group, LLC. All rights reserved.SaaS AccessAcceptanceFunctionalIntegrationUnit testsLevels of Testing7ComplexityQuantity
  • ©2013 Acquity Group, LLC. All rights reserved.History of Web TestingManualClickTesting•Timeconsuming•Error proneScreenScrape•Customscripts•MaintenancenightmareHTTPUnit•Does notwork withJavaScript•Tests HTMLstructureonlyCanooWebTest•Developingin XML*•DuplicatecodeSeleniumRC•Cut-Copy-Paste scripts•Duplicationof code•MousedrivenWebDriver•Very lowlevelframework•Easy tocreate brittletestsGeb•Abstractionon top ofWebDriver•Designed forcode reuse8
  • ©2013 Acquity Group, LLC. All rights reserved.What is Geb?9• Groovy based abstraction of Selenium Web Driver withjQuery-like syntax for content selection• Designed for robust yet simplistic syntax• Simplified DSL describing Pages and Modules• Words like• url• content• base• at• Created with DRY principal in mind• Use of highly configurable reusable modules• Abstraction of WebDriver’s Page Model
  • ©2013 Acquity Group, LLC. All rights reserved.Geb Core10PageObjectsModulesGeb
  • ©2013 Acquity Group, LLC. All rights reserved.Page Objects11• What is a “Page”?• Object representing a generic type ofwebpage on a site• Domain Specific Language• url - Appended to configurable ‘baseUrl’• Assists in navigating to page• at• closure used to determine if browseris ‘at’ the page• content• Variables representing page contentwith CSS selectorsclass HomePage extends Page {static url = "default/Home-Show"static at = {title.trim() == "Welcome"}static content = {pageTitle {$("header h1.title")}}}...<header><h1 class=“title”>Main Page Title</h1></header>...
  • ©2013 Acquity Group, LLC. All rights reserved.Building multiple Pages12class HomePage extends Page {static url = "default/Home-Show"static at = {title.trim() == "Welcome"}static content = {companyTitle {$("header h1.title")}footerCopywrite {$("footer span.copy")}}}class ProductPage extends Page {static url = "default/Product-Show"static at = {title.trim() == "Products"}static content = {companyTitle {$("header h1.title")}footerCopywrite {$("footer span.copy")}productTitle {$("#product h2.title")}}}
  • ©2013 Acquity Group, LLC. All rights reserved.Code Duplication13
  • ©2013 Acquity Group, LLC. All rights reserved.Dangers of Code Duplication14• Small change in underlying structure = many changes• Long sections of nearly identical code• Conceal purpose of code• File size• Code Maintenance
  • ©2013 Acquity Group, LLC. All rights reserved.Modules15• What is a ‘Module’?• Re-usable encapsulationconsisting of content references• Domain Specific Language• base• Optional content closure acting as abase to start content selection• content• Variables representing page contentwith CSS selectors, optionallyoriginating from base closureclass HeaderModule extends Module {static base = { $("header") }static content = {companyTitle {$("h1.title")}login { $("a.login") }}}...<header><h1 class=“title”>Page Title</h1><a href=“/login”>Login</a></header>...
  • ©2013 Acquity Group, LLC. All rights reserved.Geb: Pages + Modules16class HomePage extends Page {static url = "default/Home-Show"static at = { title.trim() == "Welcome" }static content = {footer { module FooterModule }header { module HeaderModule }}}class ProductPage extends Page {static url = "default/Product-Show"static at = { title.trim() == "Products" }static content = {productTitle { $("#product h2.title") }footer { module FooterModule }header { module HeaderModule }}}class HeaderModule extends Module {static base = { $("header") }static content = {companyTitle { $("h1.title") }login { $("a.login") }}}
  • ©2013 Acquity Group, LLC. All rights reserved.Now what?17Not Logged In Logged In
  • Extending Geb Objects
  • ©2013 Acquity Group, LLC. All rights reserved.ScenarioYour Mission:You have been asked to create a special Product page for when a customer has logged inThe page looks and functions exactly like the Product page, but has an extra header19
  • ©2013 Acquity Group, LLC. All rights reserved.Extending Page Objects20• Extend our page object• Re-define url, and at• Inherits content• Overwrite/add contentelementsclass PersonalProductPage extends ProductPage {static url = "default/MyProduct-Show"static at = { title.trim() == "Personalized" }static content = {// header inherited from HomePage// footer inherited from HomePageproductTitle {$("section#personal h2.title")}}}
  • ©2013 Acquity Group, LLC. All rights reserved.• Extend module object• Re-define base• Inherits content• Overwrite/add contentelementsExtending Modules Objects21class AuthHeaderModule extends HeaderModule {static base = { $("header section#personal") }static content = {// companyTitle inherited from HeaderModule// login inherited from HeaderModulelogout { $("a.logout") }}}
  • ©2013 Acquity Group, LLC. All rights reserved.Drawbacks & Best Practices• Watch out for• Overriding content segments happen at runtime• No name spacing• Possible to unintentionally overwrite content• Possible solutions• Document well (use JavaDoc style comments)• API Groovy Doc• Write a test to explicitly prove the inheritance• Tests should fail if the inheritance fails• Subscribe to user@geb.codehaus.org22
  • ©2013 Acquity Group, LLC. All rights reserved.With great powercomes great responsibility -- Uncle Ben23
  • Using Geb with Spock
  • ©2013 Acquity Group, LLC. All rights reserved.Spock• Specification based testing framework designed to describethe intention of the test in business friendly terms• Provides keywords for structure• when• Action• then• Expectation• Geb provides GebReportingSpec• Can be extended to change functionality• Provides extension model• Control specification lifecycle25
  • ©2013 Acquity Group, LLC. All rights reserved.Creating a Specification• Extend GebReportingSpec• Describe a feature• aka - “Test method”• Browse to• Verify at• Verify content• Defined in ProductPage• Verify module content• Defined inHeaderModule26class ProductPageSpec extends GebReportingSpec {def "user can view the product page"() {given:to ProductPageexpect:at ProductPageand:productTitle.text() == "My Cool Product"and:header.companyTitle.text() == "Company X"}}
  • Extensible Web Testing
  • ©2013 Acquity Group, LLC. All rights reserved.ScenarioYour Mission:You work on a SaaS platform where each implementation is very similar in features and structure.Design a testing framework where you can leverage common functionality inherent in allimplementations of the SaaS platform.28ReferenceApplication Implementation• Size & Color Filters• Breadcrumbs• Category Navigation• Mini-Cart• Wish List• 3-in-row Product Grid• Size & Color Filters• Breadcrumbs• Category Navigation• Mini-Cart• Wish List• 4-in-row Product Grid
  • ©2013 Acquity Group, LLC. All rights reserved.Common Functionality: Reference Application29Category FilterColor RefinementSize RefinementMini-CartWish ListBreadcrumbsPaginationPrice Refinement
  • ©2013 Acquity Group, LLC. All rights reserved.Common Functionality: Implementation Application30Category FilterPriceRefinementColorRefinementMini-CartWish ListBreadcrumbsPagination
  • ©2013 Acquity Group, LLC. All rights reserved.Inheriting Test Functionality• Add dependency onReference App’s Test JAR• Extend target specification• Inherits all features fromparent specification• Foresee problems?• Overwrite inheritedfeatures?• Use derived Page objects?• Investigate these one-by-one31class HomePgSpec extends GebReportingSpec {def "user can view the home page"() {// trimmed for claritygiven:to HomePage}def "user can login"() {// omitted for simplicity}}class ClientPageSpec extends HomePgSpec {// Inherit "user can view the home page“ & "user can login"def "user can open wishlist"() {// trimmed for claritygiven:to ClientHomePage}}
  • ©2013 Acquity Group, LLC. All rights reserved.Inheriting Test Functionality: First Run32class HomePgSpec extends GebReportingSpec {def "user can view the home page"() {given:to HomePagewhen:header.login.click()then:at MyAccountPage}def "user can login"() { /*obmitted for clarity*/ }}class ClientPageSpec extends HomePgSpec {def "user can open wishlist"() {given:to ClientHomePage}}• Run ClientPageSpec• Spock determines list of features• “user can view the home page”• “user can login”• “user can open wishlist”• Run “user can view the home page”• Navigate “to” HomePage ????
  • ©2013 Acquity Group, LLC. All rights reserved.Plan: overwrite inherited features• Look for a feature in the parentclass with the same name• Replace the inherited featurewith the provided feature• Utilize Spock Extension featuresto modify test execution33class ClientPageSpec extends HomePgSpec {def "user can login"() {given:to ClientHomePagewhen:header.login.click()then:at ClientMyAccountPage}def "user can open wishlist"() {// omitted for clarity}}
  • ©2013 Acquity Group, LLC. All rights reserved.Spock ExtensionsAdd runtimefunctionalityImplements VisitorPattern• Allow for modifications to theSpecification feature sequenceIgnore inheritedfeatures if implementedin ChildTriggered by Annotation• @OverrideFeatures34
  • ©2013 Acquity Group, LLC. All rights reserved.Overwrite Inherited Features• Look for a feature in the Parentclass with the same name• Create a marker @Annotation• Tell Spock to use this feature, notthe Parent by using a SpockExtension• spock-feature-override-extension• Open Sourced on github35@OverrideFeaturesclass ClientPageSpec extends HomePgSpec {def "user can login"() {given:to ClientHomePagewhen:header.login.click()then:at ClientMyAccountPage}def "user can open wishlist"() {// omitted for clarity}}
  • ©2013 Acquity Group, LLC. All rights reserved.Now what?• Able to override inherited features• Override every feature?• Why inherit?36class HomePage extends Page {static url = "default/Home-Show"static at = {title.trim() == "Welcome"}static content = {header { module HeaderModule }footer { module FooterModule }}}class ClientHomePage extends HomePage {static url = "default/Home-Show"static at = {title.trim() == "Company Home"}static content = {nav { module NavigationModule }}}• What do we want?• Dynamically swap sub-classed Pageobjects
  • ©2013 Acquity Group, LLC. All rights reserved.Dynamically Swap Page Objects• Page objects are managed by Browsers in Geb• Extend existing Browser• Create a “Browser” that knows about Page object relationships37geb.Browser---------------------------------------------Browser()Page at(Class<? extends Page>)Page to(Class<? extends Page>)PageSelectingBrowser extends Browser----------------------------------------------------------PageSelectingBrowser(Map pageMap)Page at(Class<? extends Page>)Page to(Class<? extends Page>)Page createPage(Class<? extends Page>)
  • ©2013 Acquity Group, LLC. All rights reserved.PageSelectingBrowser38class PageSelectingBrowser extends Browser {private final Map<Class<? extends Page>, Class<? extends Page>> overridePages/** Other methods omitted **/@OverridePage createPage(Class<? extends Page> possibleParent) {Class<? extends Page> type = possibleParentif (overridePages.containsKey(possibleParent)) {type = overridePages.get(possibleParent)if (!possibleParent.isAssignableFrom(type)) {throw new IllegalArgumentException("${type} not a ${possibleParent}")}}type}}
  • ©2013 Acquity Group, LLC. All rights reserved.Using PageSelectingBrowser39• Browsers are created/used in GebSpec or GebReportingSpec• Extend GebReportingSpec• Override createBrowser() and return new PageSelectingBrowser• Create a default empty Page object Map• Each specification can overwrite Page object Mapclass OverridableGebReportingSpec extends GebReportingSpec {@OverrideBrowser createBrowser() {new PageSelectingBrowser(createConf(), getChildPageMap())}Map<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {[:]}}
  • ©2013 Acquity Group, LLC. All rights reserved.OverridableGebReportingSpec40class HomePageSpecification extends OverridableGebReportingSpec {def "user can view the home page"() {given:to HomePageexpect:at HomePage}}class ClientHomePageSpecification extends HomePageSpecification {// inherit all features from HomePageSpecification@OverrideMap<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {[(HomePage.class): ClientHomePage.class]}}Dynamically replaceClientHomePage for HomePage
  • ©2013 Acquity Group, LLC. All rights reserved.Final Example41class HomePageSpec extends OverridableGebReportingSpec {def "user can view the home page"() { // omitted for clarity }def "user can login"() { // omitted for clarity }}@OverrideFeaturesclass ClientHomePageSpecification extends HomePageSpecification {def "user can login"() {when:to ClientHomePageheader.login.click()then:at ClientMyAccountPage}def "user can open wishlist"() { /* omitted for clarity */ }@OverrideMap<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {[(HomePage.class): ClientHomePage.class]}}Inherit FeaturesOverride Feature by NameCreate new featuresDynamically replace Page objects
  • ©2013 Acquity Group, LLC. All rights reserved.42
  • Testing in the Cloud
  • ©2013 Acquity Group, LLC. All rights reserved.Testing on Continuous Integration44
  • ©2013 Acquity Group, LLC. All rights reserved.Running without a Browser?45• What is RemoteWebDriver?• Remotely sends driver commands to Selenium Grid• Allows for same code to run on multiple browsers
  • ©2013 Acquity Group, LLC. All rights reserved.Sauce Labs46• SauceLabs – In-cloud based Remote WebDriver• Removes infrastructure requirements• Records video of remote run• Can operate behind firewalls via SSH tunnel
  • ©2013 Acquity Group, LLC. All rights reserved.Parallelization47• Initial approach• Modify GebSpec to create multiple browsers• Cons• Spock Extensions are not thread safe• Browsers run at different speeds• Remote WebDriver runners difficult to determine completion• Reports not easy to collate• Solution*• Poor man’s Parallel• Kick off multiple builds on CI with different configuration• Possible parallelization with Gradle• Fork per JVM
  • Questions?