Continuous Quality Assurance using Selenium WebDriver
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Continuous Quality Assurance using Selenium WebDriver

on

  • 6,487 views

San Francisco Selenium Meetup, May 2013

San Francisco Selenium Meetup, May 2013

Statistics

Views

Total Views
6,487
Views on SlideShare
6,144
Embed Views
343

Actions

Likes
7
Downloads
113
Comments
0

10 Embeds 343

http://menta-tests.com 157
http://jaffamonkey.com 94
http://www.multibyte.at 36
http://renttesters.com 24
https://twitter.com 18
http://www.scoop.it 5
http://plus.url.google.com 4
http://jaffamonkey23.rssing.com 3
https://www.xing.com 1
https://m.facebook.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • track of singletons, allows rewrites.

Continuous Quality Assurance using Selenium WebDriver Presentation Transcript

  • 1. Quality AssuranceContinuous…using Selenium WebDriverSan Francisco Selenium Meetup Group - May 16, 2013
  • 2. FabrizioBrancaLead System Developer at
  • 3. San Francisco, CAJanineFionathat’s meIt’s a boy!
  • 4. E-Commerce:MagentoCMS:TYPO3Portals:ZF, FLOW,…Mobile Searchperience:SOLR120 peoplein 7 officesworld-wideHighPerformance/ScaleGlobalEnterpriseProjects
  • 5. ContinuousIntegrationDeploymentDevelopmentDeliveryImprovement
  • 6. Quality AssuranceContinuousEverything sounds betterwith “continuous”
  • 7. “Continuous”==“Doingstuffoverandoveragain”BuildsDeployment Tests InfrastructureSetupDevboxSetup== “Automation”
  • 8. timespenttask sizedoes itmanuallydoes itmanuallygets annoyedwrites script toautomateruns scriptwinslosesGeeks vs. Non-GeeksMakes fun of geek’scomplicated method
  • 9. — Ovidiu PlatonWe are not shipping your machine!if it works on your machine!I don’t care
  • 10. PictureThe BIG
  • 11. VagrantBoxes
  • 12. AwesomestuffEvery singleline potentiallycan breakproduction!Bratwurst(Germans)Donut(Americans)Pizza(Italians)
  • 13. Version ControlGIT, SVN,…PHP lint
  • 14. Version Control Code Reviews“Gerrit”
  • 15. Code Reviews BuildFilesDBInstallerSettingsTriggered bycommit, by timeor manuallyJenkinsminify js/css.tar.gz“virtual package”(revision)
  • 16. Multiple ReleaseBranches?*Multiple BuildPipelines!………*http://nvie.com/posts/a-successful-git-branching-model/
  • 17. Version Control Code Reviews BuildFilesDBInstallerSettings
  • 18. BuildFilesBlerSettingsStatic CodeAnalysisCodeSnifferphp-pmdphp-cpdphp-depend
  • 19. Static CodeAnalysisInstall on“Integration”First system wherecode from alldevelopers/teamsmeets each otherunstable system.Might be broken.Will be rebuildwithout warning
  • 20. Install on“Integration”Unit TestsPHPUnitCodecoverage
  • 21. Unit Tests Integration TestsAOE PMD(for Magento)broken links,HTTP headers,W3C validity,…Test features that rely on otherparts (e.g. framework, externalservices,…)External components(SSL, Redis, Varnish,…)Basically this is alsoPHPUnit, butaddressing other stuff
  • 22. Integration Tests Acceptance TestsHere we go:Selenium!
  • 23. Acceptance TestsInstall on“Staging”Behat(Cucumber)Same setup as production,but different machinesClient QA /approval
  • 24. Install on“Staging”*http://www.slideshare.net/aoemedia/performance-measurement-and-tuningStressTestsJmeterSpin up EC2instances in thecloud to producetraffic*
  • 25. Again:multiple servers orautoscaling setupStressTestsInstall on“Production”A/BDeploymentson differentscalesPlan B:rollbackZero Downtime
  • 26. Varnish ArrayEC2Frontend ArrayBackend ArrayEC2 EC2EC2Worker ArrayEC2EC2Deployment xELBLoad BalancerRoute 53DNS-Serviceshop.angrybirds.comDeployment
  • 27. Varnish ArrayFrontend Array Worker ArrayEC2Deployment xELBLoad BalancerVaEC2FroBackend ArrayEC2 EC2EC2EC2Deployment x+1Release YELBLoad BalancerRoute 53DNS-Serviceshop.angrybirds.comDeployment
  • 28. StressTestsInstall on“Production”
  • 29. Deployment !=Feature Release
  • 30. Install on“Production”Enable Features
  • 31. VersionControlCodeReviewsBuild Static CodeAnalysisInstall on“Integration”Unit Tests IntegrationTestsAcceptanceTestsInstall on“Staging”Install on“Production”EnableFeatures
  • 32. Pipeline Visualization#284#283#282#281#280BuildStatic CodeAnalysisUnitTestsInstall on“Integration”IntegrationTestsAcceptanceTests#285for team dashboards
  • 33. Acceptance Tests
  • 34. test automationis a developer’s taskwe need to providethem some good tools…us
  • 35. Selenium1vs.Selenium2
  • 36. What’s out there?Selenium for PHPPHPUnit_Selenium(Sebastian Bergmann)WebDriver-PHP(Chibimagic)Php-webdriver-bindings(Lukasz Kolczynski)php-webdriverPHPUnit_Selenium(Giorgio Sironi)
  • 37. php-webdriverSelenium 208/2011 Facebook/php-webdriver (Justin Bishop)Element-34/php-webdriver (Adam Goucher)instaclick/php-webdriver (Anthon Pang)
  • 38. php-webdriver“A very thin wrapperof WebDriver”
  • 39. So what?A thin layer is good!But not enough…no test logicno higher levelcomponentsdealing withSelenium specific tasksnot compatible tolegacy test casesNo abstraction /structure
  • 40. ArchitectureHTTPAUTNo direct access(DB, classes,…)REST*(over HTTP)*http://code.google.com/p/selenium/wiki/JsonWireProtocol
  • 41. Project test casesPHPUnitProject ComponentsComponents LibraryMentaphp-webdriverreusableprojectspecific
  • 42. MentaSelenium 2 Testing Framework for PHPWho is“Menta”?What is“Menta”?
  • 43. Why“Menta”?
  • 44. FeaturesMentaThe framework.Not the cat…Components Session Management?Bonus Features :)Event / ObserverConfiguration ManagementPHPUnit integration
  • 45. Why?✔✔✔✔✔✔✔✔✔✔✔✔Data Providers✔✔✔✔✔✔✔✔✔Dependent Tests✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔✔Test Infrastructure
  • 46. Menta for PHPUnitAbstract test classPHPUnit_Framework_TestCaseMenta_PHPUnit_Testcase_Selenium2ProjectY_Abstract_TestCaseVery thin layer ontop of PHPUnitProjectY_TestCase_Registration ProjectY_TestCase_Checkoutthe “magic” happensin the components
  • 47. Legacy Selenium 1 testsPHPUnit_Framework_TestCasePHPUnit_Extensions_SeleniumTestCaseProjectY_TestCase_Registration ProjectY_TestCase_Checkout
  • 48. Legacy Selenium 1 testsPHPUnit_Framework_TestCaseMenta_PHPUnit_Testcase_Selenium2ProjectY_TestCase_Registration ProjectY_TestCase_CheckoutMenta_PHPUnit_Testcase_Selenium1Selenium1FacadeCompatible withPHPUnit_Extensions_SeleniumTestCase
  • 49. HTML ReportScreenshots(on demandand on failure)test descriptionfrom phpdocadditionalmessageserror details
  • 50. HTML Report“DurationHeatmap”shows longrunning tests
  • 51. Menta for PHPUnitText Result PrinterPHPUnit 3.7.20 by Sebastian Bergmann.Configuration read from /var/www/Tests/Acceptance/conf/devfb.ff.vmhost.xml> SUITE:> SUITE:> SUITE: Acceptance_Tests_Account_Register> TEST: Acceptance_Tests_Account_Register::registerSUCCESS. (Duration: 8.18 sec)> TEST: Acceptance_Tests_Account_Register::loginSUCCESS. (Duration: 6.5 sec)> TEST: Acceptance_Tests_Account_Register::testRegistrationMailFAILURE: Searching for mail with subject Welcome, Test_705b077a962977e1a412696ea42249ae User_705b077a962977e1a412696ea42249ae timed out (Duration: 22< Duration: 36 sec< Duration: 36 sec> SUITE:> SUITE: Acceptance_Tests_Checkout_CheckoutMultiAddressTest> TEST: Acceptance_Tests_Checkout_CheckoutMultiAddressTest::putProductsInCartFAILURE: Element "//table[@id="shopping-cart-table"]//h3[contains(concat( , @class, ), attentionText )]" does not contain text "Bulb Vase" Fail< Duration: 9.87 sec> SUITE: Acceptance_Tests_Checkout_CheckoutTest> TEST: Acceptance_Tests_Checkout_CheckoutTest::categoryViewSUCCESS. (Duration: 1.79 sec)> TEST: Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCartFAILURE: Failed asserting that two strings are equal. (Duration: 3.55 sec)SKIPPED: This test depends on "Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart" to pass. (Duration: 5.34 sec)
  • 52. Components?Components!
  • 53. ManagerComponentMenta_ComponentManager::get(‘PageObject_Cart’) PageObject_Cart
  • 54. RewritesComponentPageObject_CartProjectZ_PageObject_Cartextendschangelocatorsaddfunctionalitychangebehavioroverwritelabels
  • 55. ManagerComponentMenta_ComponentManager::get(‘PageObject_Cart’) ProjectZ_PageObject_CartMenta_ComponentManager::addRewrite(‘PageObject_Cart’,‘ProjectZ_PageObject_Cart’);
  • 56. ComponentsCommon Assert Wait Screenshot Selenium1FacadeImap WebService W3C Validator Curl …Some might not evenrequire a Selenium session
  • 57. GalleryScreenshot
  • 58. Components!PageObjects?PageObjectsComponents{ }}⊇{ PageObjects arecomponents
  • 59. PageObjects?representsservices offered bya page or featureis the only thing witha deep knowledge ofthe HTMLclickCheckoutButton() isInCart($product) getTotals()✔✔✔
  • 60. ComponentLibrariesMentaMagento(your library)Project-specificMagento(community-provided)General(your library)
  • 61. SessionManagementLet Menta manageyour sessionsauto-closeauto-startincl. on exceptionsand ctrl+cReusing sessionsCleaning sessions
  • 62. ConfigurationManagementFallback mechanismfor global settingsCreate multipleconfigurations fordifferent browsers,application instancesor Selenium ServersCompatible toPHPUnit (xml)
  • 63. Event / ObserverMenta_Events::dispatchEvent(after_session_create);Menta_Events::addObserver(after_session_create, function($session) {$session->window(main)->postSize(array(width => 1024, height => 768));}
  • 64. Bonus Features
  • 65. Sauce Labs Support*configuration: http://username:accesskey@ondemand.saucelabs.com:80/wd/hubworks fine*update testresult (REST)
  • 66. Sauce Labs SupportSpeaking name Test results
  • 67. Perceptual diffsBuild XBuild X-1PerceptualDiff
  • 68. github.com / AOEmedia / Mentagithub.com / AOEmedia / Menta_SampleProjectIt’s composer based andyou’ can get startedin a few minutes!
  • 69. Thank you!Any questions?http://www.aoemedia.comhttp://www.fabrizio-branca.de@fbrncFollow me on twitter!My blog