• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Continuous Quality Assurance using Selenium WebDriver
 

Continuous Quality Assurance using Selenium WebDriver

on

  • 5,670 views

San Francisco Selenium Meetup, May 2013

San Francisco Selenium Meetup, May 2013

Statistics

Views

Total Views
5,670
Views on SlideShare
5,459
Embed Views
211

Actions

Likes
6
Downloads
95
Comments
0

7 Embeds 211

http://jaffamonkey.com 91
http://menta-tests.com 72
http://renttesters.com 24
https://twitter.com 18
http://jaffamonkey23.rssing.com 3
http://www.scoop.it 2
https://www.xing.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • track of singletons, allows rewrites.

Continuous Quality Assurance using Selenium WebDriver Continuous Quality Assurance using Selenium WebDriver Presentation Transcript

  • Quality AssuranceContinuous…using Selenium WebDriverSan Francisco Selenium Meetup Group - May 16, 2013
  • FabrizioBrancaLead System Developer at
  • San Francisco, CAJanineFionathat’s meIt’s a boy!
  • E-Commerce:MagentoCMS:TYPO3Portals:ZF, FLOW,…Mobile Searchperience:SOLR120 peoplein 7 officesworld-wideHighPerformance/ScaleGlobalEnterpriseProjects
  • ContinuousIntegrationDeploymentDevelopmentDeliveryImprovement
  • Quality AssuranceContinuousEverything sounds betterwith “continuous”
  • “Continuous”==“Doingstuffoverandoveragain”BuildsDeployment Tests InfrastructureSetupDevboxSetup== “Automation”
  • timespenttask sizedoes itmanuallydoes itmanuallygets annoyedwrites script toautomateruns scriptwinslosesGeeks vs. Non-GeeksMakes fun of geek’scomplicated method
  • — Ovidiu PlatonWe are not shipping your machine!if it works on your machine!I don’t care
  • PictureThe BIG
  • VagrantBoxes
  • AwesomestuffEvery singleline potentiallycan breakproduction!Bratwurst(Germans)Donut(Americans)Pizza(Italians)
  • Version ControlGIT, SVN,…PHP lint
  • Version Control Code Reviews“Gerrit”
  • Code Reviews BuildFilesDBInstallerSettingsTriggered bycommit, by timeor manuallyJenkinsminify js/css.tar.gz“virtual package”(revision)
  • Multiple ReleaseBranches?*Multiple BuildPipelines!………*http://nvie.com/posts/a-successful-git-branching-model/
  • Version Control Code Reviews BuildFilesDBInstallerSettings
  • BuildFilesBlerSettingsStatic CodeAnalysisCodeSnifferphp-pmdphp-cpdphp-depend
  • Static CodeAnalysisInstall on“Integration”First system wherecode from alldevelopers/teamsmeets each otherunstable system.Might be broken.Will be rebuildwithout warning
  • Install on“Integration”Unit TestsPHPUnitCodecoverage
  • Unit Tests Integration TestsAOE PMD(for Magento)broken links,HTTP headers,W3C validity,…Test features that rely on otherparts (e.g. framework, externalservices,…)External components(SSL, Redis, Varnish,…)Basically this is alsoPHPUnit, butaddressing other stuff
  • Integration Tests Acceptance TestsHere we go:Selenium!
  • Acceptance TestsInstall on“Staging”Behat(Cucumber)Same setup as production,but different machinesClient QA /approval
  • Install on“Staging”*http://www.slideshare.net/aoemedia/performance-measurement-and-tuningStressTestsJmeterSpin up EC2instances in thecloud to producetraffic*
  • Again:multiple servers orautoscaling setupStressTestsInstall on“Production”A/BDeploymentson differentscalesPlan B:rollbackZero Downtime
  • Varnish ArrayEC2Frontend ArrayBackend ArrayEC2 EC2EC2Worker ArrayEC2EC2Deployment xELBLoad BalancerRoute 53DNS-Serviceshop.angrybirds.comDeployment
  • Varnish ArrayFrontend Array Worker ArrayEC2Deployment xELBLoad BalancerVaEC2FroBackend ArrayEC2 EC2EC2EC2Deployment x+1Release YELBLoad BalancerRoute 53DNS-Serviceshop.angrybirds.comDeployment
  • StressTestsInstall on“Production”
  • Deployment !=Feature Release
  • Install on“Production”Enable Features
  • VersionControlCodeReviewsBuild Static CodeAnalysisInstall on“Integration”Unit Tests IntegrationTestsAcceptanceTestsInstall on“Staging”Install on“Production”EnableFeatures
  • Pipeline Visualization#284#283#282#281#280BuildStatic CodeAnalysisUnitTestsInstall on“Integration”IntegrationTestsAcceptanceTests#285for team dashboards
  • Acceptance Tests
  • test automationis a developer’s taskwe need to providethem some good tools…us
  • Selenium1vs.Selenium2
  • What’s out there?Selenium for PHPPHPUnit_Selenium(Sebastian Bergmann)WebDriver-PHP(Chibimagic)Php-webdriver-bindings(Lukasz Kolczynski)php-webdriverPHPUnit_Selenium(Giorgio Sironi)
  • php-webdriverSelenium 208/2011 Facebook/php-webdriver (Justin Bishop)Element-34/php-webdriver (Adam Goucher)instaclick/php-webdriver (Anthon Pang)
  • php-webdriver“A very thin wrapperof WebDriver”
  • So what?A thin layer is good!But not enough…no test logicno higher levelcomponentsdealing withSelenium specific tasksnot compatible tolegacy test casesNo abstraction /structure
  • ArchitectureHTTPAUTNo direct access(DB, classes,…)REST*(over HTTP)*http://code.google.com/p/selenium/wiki/JsonWireProtocol
  • Project test casesPHPUnitProject ComponentsComponents LibraryMentaphp-webdriverreusableprojectspecific
  • MentaSelenium 2 Testing Framework for PHPWho is“Menta”?What is“Menta”?
  • Why“Menta”?
  • FeaturesMentaThe framework.Not the cat…Components Session Management?Bonus Features :)Event / ObserverConfiguration ManagementPHPUnit integration
  • Why?✔✔✔✔✔✔✔✔✔✔✔✔Data Providers✔✔✔✔✔✔✔✔✔Dependent Tests✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔Test Infrastructure
  • Menta for PHPUnitAbstract test classPHPUnit_Framework_TestCaseMenta_PHPUnit_Testcase_Selenium2ProjectY_Abstract_TestCaseVery thin layer ontop of PHPUnitProjectY_TestCase_Registration ProjectY_TestCase_Checkoutthe “magic” happensin the components
  • Legacy Selenium 1 testsPHPUnit_Framework_TestCasePHPUnit_Extensions_SeleniumTestCaseProjectY_TestCase_Registration ProjectY_TestCase_Checkout
  • Legacy Selenium 1 testsPHPUnit_Framework_TestCaseMenta_PHPUnit_Testcase_Selenium2ProjectY_TestCase_Registration ProjectY_TestCase_CheckoutMenta_PHPUnit_Testcase_Selenium1Selenium1FacadeCompatible withPHPUnit_Extensions_SeleniumTestCase
  • HTML ReportScreenshots(on demandand on failure)test descriptionfrom phpdocadditionalmessageserror details
  • HTML Report“DurationHeatmap”shows longrunning tests
  • Menta for PHPUnitText Result PrinterPHPUnit 3.7.20 by Sebastian Bergmann.Configuration read from /var/www/Tests/Acceptance/conf/devfb.ff.vmhost.xml> SUITE:> SUITE:> SUITE: Acceptance_Tests_Account_Register> TEST: Acceptance_Tests_Account_Register::registerSUCCESS. (Duration: 8.18 sec)> TEST: Acceptance_Tests_Account_Register::loginSUCCESS. (Duration: 6.5 sec)> TEST: Acceptance_Tests_Account_Register::testRegistrationMailFAILURE: Searching for mail with subject Welcome, Test_705b077a962977e1a412696ea42249ae User_705b077a962977e1a412696ea42249ae timed out (Duration: 22< Duration: 36 sec< Duration: 36 sec> SUITE:> SUITE: Acceptance_Tests_Checkout_CheckoutMultiAddressTest> TEST: Acceptance_Tests_Checkout_CheckoutMultiAddressTest::putProductsInCartFAILURE: Element "//table[@id="shopping-cart-table"]//h3[contains(concat( , @class, ), attentionText )]" does not contain text "Bulb Vase" Fail< Duration: 9.87 sec> SUITE: Acceptance_Tests_Checkout_CheckoutTest> TEST: Acceptance_Tests_Checkout_CheckoutTest::categoryViewSUCCESS. (Duration: 1.79 sec)> TEST: Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCartFAILURE: Failed asserting that two strings are equal. (Duration: 3.55 sec)SKIPPED: This test depends on "Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart" to pass. (Duration: 5.34 sec)
  • Components?Components!
  • ManagerComponentMenta_ComponentManager::get(‘PageObject_Cart’) PageObject_Cart
  • RewritesComponentPageObject_CartProjectZ_PageObject_Cartextendschangelocatorsaddfunctionalitychangebehavioroverwritelabels
  • ManagerComponentMenta_ComponentManager::get(‘PageObject_Cart’) ProjectZ_PageObject_CartMenta_ComponentManager::addRewrite(‘PageObject_Cart’,‘ProjectZ_PageObject_Cart’);
  • ComponentsCommon Assert Wait Screenshot Selenium1FacadeImap WebService W3C Validator Curl …Some might not evenrequire a Selenium session
  • GalleryScreenshot
  • Components!PageObjects?PageObjectsComponents{ }}⊇{ PageObjects arecomponents
  • PageObjects?representsservices offered bya page or featureis the only thing witha deep knowledge ofthe HTMLclickCheckoutButton() isInCart($product) getTotals()✔✔✔
  • ComponentLibrariesMentaMagento(your library)Project-specificMagento(community-provided)General(your library)
  • SessionManagementLet Menta manageyour sessionsauto-closeauto-startincl. on exceptionsand ctrl+cReusing sessionsCleaning sessions
  • ConfigurationManagementFallback mechanismfor global settingsCreate multipleconfigurations fordifferent browsers,application instancesor Selenium ServersCompatible toPHPUnit (xml)
  • Event / ObserverMenta_Events::dispatchEvent(after_session_create);Menta_Events::addObserver(after_session_create, function($session) {$session->window(main)->postSize(array(width => 1024, height => 768));}
  • Bonus Features
  • Sauce Labs Support*configuration: http://username:accesskey@ondemand.saucelabs.com:80/wd/hubworks fine*update testresult (REST)
  • Sauce Labs SupportSpeaking name Test results
  • Perceptual diffsBuild XBuild X-1PerceptualDiff
  • github.com / AOEmedia / Mentagithub.com / AOEmedia / Menta_SampleProjectIt’s composer based andyou’ can get startedin a few minutes!
  • Thank you!Any questions?http://www.aoemedia.comhttp://www.fabrizio-branca.de@fbrncFollow me on twitter!My blog