Combining Automated Functional And Load Testing

1,846 views

Published on

In this free webinar you will learn about the benefits of combining functional and load tests.
Discover how to integrate Ranorex with NeoLoad to find functional issues that only occur under load conditions.

Published in: Technology
  • Be the first to comment

Combining Automated Functional And Load Testing

  1. 1. Integrating Ranorex and NeoLoad Time to make peace between functional and non-functional testing
  2. 2. Welcome to the Ranorex-Neotys Webinar Christian Breitwieser, Ranorex Innovation Manager Henrik Rexed, Neotys Technical Evangelist
  3. 3. Agenda • Quick Company and Tool Intro • Performance Matters in User Experience • Benefits of Automated Functional Testing • Benefits of Combining Automated Functional and Load Testing • Live Demonstration
  4. 4. • Functional test automation for desktop, web and mobile applications • Cross-technology end-to-end testing • Tool for testers and developers
  5. 5. • Functional test automation for desktop, web and mobile apps • Cross-technology end-to-end testing
  6. 6. Tool for testers and developers Code Module Action Based Modules Ranorex Studio
  7. 7. • Deliver faster applications faster through continuous performance validation - Who We Are
  8. 8. - Who We Are Mission: Deliver faster applications faster through continuous performance validation
  9. 9. - Who We Are Mission: Deliver faster applications faster through continuous performance validation
  10. 10. User Demands Process Revolutions We’re Driven by Your Challenges Technology Transitions
  11. 11. User Demands Process Revolutions We’re Driven by Your Challenges Technology Transitions
  12. 12. User Demands Process Revolutions We’re Driven by Your Challenges Technology Transitions
  13. 13. User Demands Process Revolutions We’re Driven by Your Challenges Technology Transitions Docker
  14. 14. Load and Performance Testing 24x7 Performance Monitoring Continuous Performance Validation Solution
  15. 15. Load and Performance Testing 24x7 Performance Monitoring Continuous Performance Validation Solution
  16. 16. Continuous Performance Validation Solution Development Continuous Integration & Component Testing
  17. 17. Continuous Performance Validation Solution Development Continuous Integration & Component Testing
  18. 18. Continuous Performance Validation Solution Pre-production System-wide Testing Development Continuous Integration & Component Testing
  19. 19. Continuous Performance Validation Solution Pre-production System-wide Testing Development Continuous Integration & Component Testing
  20. 20. Continuous Performance Validation Solution Pre-production System-wide Testing Development Continuous Integration & Component Testing
  21. 21. Continuous Performance Validation Solution Pre-production System-wide Testing Production Proactive Monitoring Development Continuous Integration & Component Testing
  22. 22. Continuous Performance Validation Solution Pre-production System-wide Testing Production Proactive Monitoring Development Continuous Integration & Component Testing
  23. 23. Continuous Performance Validation Solution Pre-production System-wide Testing Production Proactive Monitoring Development Continuous Integration & Component Testing
  24. 24. Agenda • Quick Company and Tool Intro • Performance Matters in User Experience • Benefits of Automated Functional Testing • Benefits of Combining Automated Functional and Load Testing • Live Demonstration
  25. 25. User Experience is Crucial • 57% of users use their mobile devices to access applications • 40% of users move towards a competitor after a bad user experience • 44-61% of users share their feelings on social networks
  26. 26. What is User Experience? Usability Ergonomics Performance Accessibility Functionality
  27. 27. User Experience Impacts the Business
  28. 28. Case Study Impact of the response time on the conversion rate:
  29. 29. Agenda • Quick Company and Tool Intro • Performance Matters in User Experience • Benefits of Automated Functional Testing • Benefits of Combining Automated Functional and Load Testing • Live Demonstration
  30. 30. Why Functionality is important Usability Ergonomics Performance Accessibility Functionality
  31. 31. Otherwise…
  32. 32. Test Automation Pyramid Unit Integration Functional Manual
  33. 33. Manual & Integration Testing Manual testing • Time-consuming • Error-prone • Cannot „run“ overnight Integration testing • No black box • Hard to cross technology borders • Good coding skill required
  34. 34. Benefits of Automated Functional Testing • Quick feedback • Parallel test execution • Scalable • Codeless functional testing • Increased test coverage
  35. 35. Numbers & Facts TomTom – Case Study: • Testing time: from 10 days down to 4h • Over 90% of time saved with Ranorex automated tests Siemens – Case Study: • Due to the use of C#, new test cases can be implemented more effectively • Only 60% of the time needed with Ranorex compared to previously used tool
  36. 36. Agenda • Quick Company and Tool Intro • Performance Matters in User Experience • Benefits of Automated Functional Testing • Benefits of Combining Automated Functional and Load Testing • Live Demonstration
  37. 37. Separated Functional & Performance Testing • Functional testing ensures functionality … but not when the AUT is under load • Load/Performance testing ensures a fluent user experience … but does not guarantee a functional UI
  38. 38. Ranorex Customer Experience Survey No Idea Yes No 54% 37% 9% Do you combine Ranorex test automation with non-functional tests? In addition to functional test automation you perform… 4% 19% 22% 27% 31% 33% Other Load Testing Database Testing Web-Service-Testing Performance Testing Unit Testing
  39. 39. Functional – Performance Tests Golden Rule: Automated performance testing should not take place if functional regression tests failed. Performance tests require that the application or part of the system is: • Operational: No systematic errors or exceptions • Representative : Enough data available in the test environment to determine the behavior under load conditions
  40. 40. How Can We Increase Risk Coverage? 100 test cases 40 test cases covering 80% of the risk not necessarily selected for performance testing We prioritize test cases based on risk. These test cases are validated through : - Automated regression testing - Exploratory testing
  41. 41. Validate Critical Business Case Under Load This guarantees: • The stability of the crucial use cases under real usage conditions • High user experience • The identification of potential bottle-necks, even across technology borders
  42. 42. The Initial Workflow… Automated functional testing Exploratory testing Load testing Exploratory testing under load …
  43. 43. Applying the Ranorex-NeoLoad Integration… 00:00:00 00:00:30 00:01:00 00:01:30 Load NeoLoad Application Server App Error Rate Virtual Users Load Automated functional tests User Experience
  44. 44. … You‘ll Find Bugs Earlier! …
  45. 45. Agenda • Quick Company and Tool Intro • Performance Matters in User Experience • Benefits of Automated Functional Testing • Benefits of Combining Automated Functional and Load Testing • Live Demonstration
  46. 46. Further Resources Combining Automated Functional and Load Testing http://www.ranorex.com/blog/combining-functional-and-load-testing How to Combine Ranorex and NeoLoad Tests http://www.ranorex.com/blog/combine-ranorex-and-neoload-tests
  47. 47. Questions? Henrik Rexed: henrik.rexed@neotys.com Download NeoLoad Free Edition Visit our neotys.com Follow me on Twitter @hrexed Christian Breitwieser: cbreitwieser@ranorex.com Download free Ranorex trial Visit www.ranorex.com Follow us on Twitter @Ranorex

×