Your SlideShare is downloading. ×
Magento Testing on all fronts
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Magento Testing on all fronts

1,340
views

Published on

San Francisco Magento Meeting 2014-03-12

San Francisco Magento Meeting 2014-03-12

Published in: Technology

0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,340
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
29
Comments
0
Likes
8
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. On all Fronts Testing Magento San Francisco Magento Meetup Group - March 11, 2014
  • 2. Fabrizio BrancaLead System Developer at
  • 3. E-Commerce: Magento CMS: TYPO3 Portals: ZF, FLOW,… Mobile Searchperience: SOLR 120 people in 7 offices world-wide High Performance /Scale Global Enterprise Projects
  • 4. Unit Service UI Acceptance http://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid What the customer thinks the code should do What the developer thinks the code should do Load Test
  • 5. Continuous Integration Deployment Development Delivery Improvement
  • 6. Quality Assurance Continuous Everything sounds better with “continuous”
  • 7. “Continuous” ==“Doingstuffoverandoveragain” Builds Deployment Tests Infrastructure Setup Devbox Setup == “Automation”
  • 8. timespent task size does it manually does it manually gets annoyed writes script to automate runs script wins loses Geeks vs. Non-Geeks Makes fun of geek’s complicated method
  • 9. — Ovidiu Platon We are not shipping your machine! if it works on your machine! I don’t care
  • 10. Picture The BIG
  • 11. Vagrant Boxes
  • 12. Awesome stuff Every single line potentially can break production! Bratwurst (Germans) Donut (Americans) Pizza (Italians)
  • 13. Version Control GIT, SVN,… PHP lint
  • 14. Version Control Code Reviews Gerrit, Crucible, Phabricator,…
  • 15. Code Reviews Build Files DB Installer Settings Triggered by commit, by time or manually Jenkins minify js/css .tar.gz “virtual package” (revision)
  • 16. Version Control Code Reviews Build Files DB Installer Settings
  • 17. Build Files B ler Settings Static Code Analysis CodeSniffer php-pmd php-cpd php-depend
  • 18. Static Code Analysis Install on “Integration” First system where code from all developers/teams meets each other unstable system. Might be broken. Will be rebuild without warning
  • 19. Install on “Integration” Unit Tests PHPUnit Code coverage
  • 20. Unit Tests Integration Tests Test features that rely on other parts (e.g. framework, external services,…) External components (SSL, Redis, Varnish,…) Basically this is also PHPUnit, but addressing other stuff
  • 21. Integration Tests Acceptance Tests
  • 22. Acceptance Tests Install on “Staging” Behat (Cucumber) Same setup as production, but different machines Client QA / approval
  • 23. Install on “Staging” *http://www.slideshare.net/aoemedia/performance-measurement-and-tuning Stress Tests Jmeter Spin up EC2 instances in the cloud to produce traffic*
  • 24. Stress Tests Install on “Production”
  • 25. Install on “Production” Enable Features
  • 26. Version Control Code Reviews Build Static Code Analysis Install on “Integration” Unit Tests Integration Tests Acceptance Tests Install on “Staging” Install on “Production” Enable Features
  • 27. with EcomDev_PHPUnit Unit Tests
  • 28. https://github.com/EcomDev/ EcomDev_PHPUnit use the “dev” branch!
  • 29. Config Model Block Helper Controller fat model, skinny controller…
  • 30. Classes PHPUnit_Framework_TestCaseEcomDev_PHPUnit_Test_Case EcomDev_PHPUnit_Test_Case_Config EcomDev_PHPUnit_Test_Case_Controller
  • 31. Assertions
  • 32. Fixtures
  • 33. Mocks
  • 34. Data Provider
  • 35. Magento Integration Testsuite management Framework integration Create test environment (db+config)
  • 36. Manual http://www.ecomdev.org/2011/05/24/ecomdev_phpunit-manual-version-0-2-0-is-available-for- download.html Video tutorial http://www.ecomdev.org/2012/01/31/test-driven-magento-development-seminar-video-and- code.html Documentation More Information
  • 37. With Selenium Acceptance Testing
  • 38. Acceptance Tests
  • 39. What’s out there? Selenium for PHP PHPUnit_Selenium (Sebastian Bergmann) WebDriver-PHP (Chibimagic) Php-webdriver-bindings (Lukasz Kolczynski) php-webdriver PHPUnit_Selenium (Giorgio Sironi)
  • 40. php-webdriver Selenium 2 08/2011 Facebook/php-webdriver (Justin Bishop) Element-34/php-webdriver (Adam Goucher) instaclick/php-webdriver (Anthon Pang)
  • 41. php-webdriver “A very thin wrapper of WebDriver”
  • 42. So what? A thin layer good! But not enough… no test logic no higher level components dealing with Selenium specific tasks not compatible to legacy test cases No abstraction / structure
  • 43. Architecture HTTP AUT No direct access (DB, classes,…) REST* (over HTTP) *http://code.google.com/p/selenium/wiki/JsonWireProtocol
  • 44. Project test cases PHPUnit Project Components Components Library Menta php-webdriver reusable project specific
  • 45. Menta Selenium 2 Testing Framework for PHP Who is “Menta”? What is “Menta”?
  • 46. Features MentaThe framework. Not the cat… Components Session Management ?Bonus Features :)Event / ObserverConfiguration Management PHPUnit integration
  • 47. Why? ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Data Providers ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Dependent Tests ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Test Infrastructure
  • 48. Menta for PHPUnit Abstract test class PHPUnit_Framework_TestCaseMenta_PHPUnit_Testcase_Selenium2ProjectY_Abstract_TestCase Very thin layer on top of PHPUnit ProjectY_TestCase_Registration ProjectY_TestCase_Checkout the “magic” happens in the components
  • 49. HTML ReportScreenshots (on demand and on failure) test description from phpdoc additional messages error details
  • 50. HTML Report“Duration Heatmap” shows long running tests
  • 51. Menta for PHPUnit Text Result PrinterPHPUnit 3.7.20 by Sebastian Bergmann. Configuration read from /var/www/Tests/Acceptance/conf/devfb.ff.vmhost.xml > SUITE: > SUITE: > SUITE: Acceptance_Tests_Account_Register > TEST: Acceptance_Tests_Account_Register::register SUCCESS. (Duration: 8.18 sec) > TEST: Acceptance_Tests_Account_Register::login SUCCESS. (Duration: 6.5 sec) > TEST: Acceptance_Tests_Account_Register::testRegistrationMail FAILURE: Searching for mail with subject 'Welcome, Test_705b077a962977e1a412696ea42249ae User_705b077a962977e1a412696ea42249ae' timed out (Duration: 22 < Duration: 36 sec < Duration: 36 sec > SUITE: > SUITE: Acceptance_Tests_Checkout_CheckoutMultiAddressTest > TEST: Acceptance_Tests_Checkout_CheckoutMultiAddressTest::putProductsInCart FAILURE: Element "//table[@id="shopping-cart-table"]//h3[contains(concat(' ', @class, ' '), ' attentionText ')]" does not contain text "Bulb Vase" Fail < Duration: 9.87 sec > SUITE: Acceptance_Tests_Checkout_CheckoutTest > TEST: Acceptance_Tests_Checkout_CheckoutTest::categoryView SUCCESS. (Duration: 1.79 sec) > TEST: Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart FAILURE: Failed asserting that two strings are equal. (Duration: 3.55 sec) SKIPPED: This test depends on "Acceptance_Tests_Checkout_CheckoutTest::putPlatterInCart" to pass. (Duration: 5.34 sec)
  • 52. Components Common Assert Wait Screenshot Selenium1Facade Imap WebService W3C Validator Curl … Some might not even require a Selenium session
  • 53. Components! PageObjects? PageObjectsComponents { }}⊇{ PageObjects are components
  • 54. PageObjects? represents services offered by a page or feature is the only thing with a deep knowledge of the HTML clickCheckoutButton() isInCart($product) getTotals() ✔ ✔ ✔
  • 55. Component Libraries Menta Magento (your library) Project- specific Magento (community-provided) General (your library)
  • 56. Gallery Screenshot
  • 57. Perceptual diffs Build X Build X-1 Perceptual Diff
  • 58. github.com / AOEmedia / Menta github.com / AOEmedia / Menta_SampleProject It’s composer based and you’ can get started in a few minutes!
  • 59. With JMeter Load Testing
  • 60. JMeter jmeter.apache.org Current version: 2.11
  • 61. JMeter HTTP Simple Setup
  • 62. JMeter Slave JMeter Slave JMeter Slave JMeter Master Master/Slave Setup
  • 63. JMeter Slave JMeter Slave JMeter Slave JMeter Master EC2 Setup JMeter Slave JMeter Slave …
  • 64. Creating JMeter tests JMeter Test Manually Replaying Apache Logs Recording Browser interaction using a proxy“Convert” existing acceptance tests CSV Files (Selenium) HTT
  • 65. HTT Behat Selenium Firefox Proxy JMeter Test Converting Behat to JMeter
  • 66. Using a Proxy JMeter
  • 67. Using a Proxy Browser configuration Manual configuration Browser Extension
  • 68. Using a Proxy Browser configuration
  • 69. java -jar selenium-server-standalone-2.40.0.jar -trustAllSSLCertificates -Dwebdriver.firefox.profile=jmeter Using a Proxy Selenium configuration
  • 70. http://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.pdf https://wiki.apache.org/jmeter/TestRecording210 http://jmeter.apache.org/usermanual/component_reference.html#HTTP%28S%29_Test_Script _Recorder Using a Proxy
  • 71. https://jmeter.apache.org/usermanual/test_plan.html • ThreadGroups • Controllers • Samplers • Logic Controllers • Test Fragments • Timers • Assertions • Configuration Elements • Pre-Processor Elements • Post-Processor Elements Elements of a Test Plan
  • 72. http://www.slideshare.net/shubelal/abj-meterbrief-upladed/7 Elements of a Test Plan
  • 73. • Tree structure • Inheritance (Cookie Manager, Header Manager,…) • Timers • Variables • Assertions • Random values • Scripting • Extracting values • Following redirects • (check examples in bin/examples) Basic Concepts
  • 74. Workflows Search / Filters Static Pages 404 What to testConsider conversion rate / funnel!
  • 75. JMeter Plugins jmeter-plugins.org
  • 76. Distributed load testing General Master/Slave setup https://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf AOE presentations http://www.slideshare.net/axeljung01/jmeter-and-amazon-cloud-workshop-t3dd2012 http://www.slideshare.net/aoemedia/performance-measurement-and-tuning On EC2 https://github.com/oliverlloyd/jmeter-ec2 http://www.http503.com/2012/run-jmeter-on-amazon-ec2-cloud/
  • 77. Server Monitoring
  • 78. Thank you! Any questions? http://www.aoe.com http://www.fabrizio-branca.de @fbrnc Follow me on twitter! My blog