Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Boston meetup blaze_meter_feb2017

7,887 views

Published on

Eran Kinsbruner and Amir Rozenberg in a hosted boston meetup by BlazeMeter - Open source tools tips

Published in: Engineering
  • Be the first to comment

Boston meetup blaze_meter_feb2017

  1. 1. Boston Best Practices using open source for testing Eran Kinsbruner, @ek121268 Amir Rozenberg, @Amir_Rozenberg
  2. 2. SHIFTING TO OPEN SOURCE, BENEFITS MOBILE OSS MARKET LANDSCAPE, TOOLS, COMPARISON DEMO!!! OSS: BEYOND TESTING Q&A
  3. 3. What is automated Mobile Web Automation target Cannot be automated 60%-90%20% 30% 100% The gap we want to address Degree of automation Increased complexity and no timeMany teams with varying set of skills 2/8/2017 TestersDevTestersCoders
  4. 4. A Day in A Life –Real User Environments Is Essential Can Any OSS Framework Handle Such Requirements? Automation ought to cover the full user environment not just the Application Under Test!
  5. 5. Which Tool is the best fit for you? SELENIUM APPIUM CALABASH ESPRESSO XCTEST UI When looking at today’s open-source mobile test automation landscape, there are five highly-adopted test frameworks. Each tool has advantages for your mobile and web testing depending on your needs.
  6. 6. Making the Decision A Comparison Detailed analysis of each test automation tool: Sometimes More Than A Single Framework is The Right Answer
  7. 7. Selenium • The best choice for web test automation teams testing for responsive web design or stand-alone web sites • Less suitable for developing unit testing, making this framework less appealing for developers • On its own, Selenium core test reports are not highly informative and lack unique insights
  8. 8. Appium • Best suited for QA teams testing the functionality of native, mobile web and mobile hybrid apps across iOS and Android • Less suitable for developers who wish to develop and perform unit testing • Appium reports are a bit limited from a debugging and fast feedback loop perspective, and do not include videos, network logs and key vitals information
  9. 9. Calaba.sh • Designed for organizations that work in behavior-driven development workflows • Does not cover the entire mobile system level events • Offers an easy path to both develop and test features in parallel, all in an easy user-flow based language • Appealing for both dev and QA practitioners • Provides solid insights and reports to both dev and QA teams
  10. 10. Espresso & XCTest UI • Both requires a separate Test App executable • Espresso is for Android and XCTest is for iOS – not a single script • Both tools are fully integrated into development IDEs such as Android Studio/Xcode, and offer very easy-to-develop techniques, including test recorders • Fully maintained by Google and Apple, which assures they always support the latest OS features so developers can stay ahead of the market and test accordingly • Support both unit testing types and functional UI testing • Both are app-context only, which limits their abilities to test for user condition scenarios XCTEST UIESPRESSO
  11. 11. Things to Consider • Lab: Support for variety of frameworks is important for enterprises • Flexibility • Autonomy • Complete E2E coverage • Unattended reliable testing • Automation: support for multiple OSS frameworks becomes important • Different “persona’s” need different tools under 1 lab • Cross platform scripting • Robust framework to support high test coverage • Analysis: Digital reporting is necessary for teams to take action and fix issues faster & earlier When choosing a test automation tool
  12. 12. Area Components Quantum Management Reporting Execution Manager Test Development Test Authoring (IDE+Language) Object Repository Object Locator/Spy Appium/Selenium ✓OSS = ✓OSS ✓OSS http://projectQuantum.io
  13. 13. Digital – Responsive Web Application
  14. 14. DEMO – BDD OSS Framework
  15. 15. Few other OSS-made mobile testing toys Production Insight/ ”Testing in Production” Network features: HAR file & NETEM Some others: • Security scans: https://github.com/MobSF/Mobile- Security-Framework-MobSF • Deep visual analysis • Etc.
  16. 16. Q&A @Amir_Rozenberg@ek121268
  17. 17. @Amir_Rozenberg@ek121268
  18. 18. 22 Open & Integrated Xcode Selenium

×