Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Using Automation to Meet Demands for Performance and Quality
Similar to Using Automation to Meet Demands for Performance and Quality (20)
More from Neotys
More from Neotys (20)
Recently uploaded
Recently uploaded (20)
Using Automation to Meet Demands for Performance and Quality
- 1. Meet Demands for Performance and Quality using Automation By Henrik Rexed, Neotys
- 2. Housekeeping RULES 1. Type into the Question Panel of GoToWebinar 2. Questions will be answered at the end of the presentation All audience members are muted during the webinar The recordings of our webinars can be found here: www.Neotys.com
- 3. Who is this speaker? ● Henrik Rexed (khnightloader)* ● 12 years of Performance engineering ● Technical Evangelist @Neotys ● Lego Fan ● Focus on technologies for Performance engineers
- 4. User Experience is crucial 40% of users are moving towards the competition following a bad experience 57 % of users use their mobile 44-61 % of users share their feelings on social network
- 5. What is User Experience? Usability Ergonomy Performance Accessibility Functionality
- 6. User Experience impact the business Etam reduces it's average page load time from 1.2s to 500ms and increased conversions by 20%, time on site by 21%, and pages viewed per visit by 28%. Walmart saw up to a 2% increase in conversions for every 1 second of improvement in load time. Every 100ms improvment also resulted in up to a 1% increase in revenue. Shopzilla decreased load time by 5 seconds and saw a 12% increase in conversion rate, a 25% increase in page views and a 50% reduction in infrastructure required. Etsy saw a 12% increase in bounce rate when they added 160kb of images to their mobile page.
- 7. Web has Evolved….To increase the UX
- 8. Web Has involved….To increase the UX
- 9. Websites are becoming heavier 0 500 1000 1500 2000 2500 3000 15/11/2010 11/02/2011 16/05/2011 15/08/2011 15/11/2011 15/02/2012 15/05/2012 15/08/2012 15/11/2012 15/02/2013 15/05/2013 15/09/2013 01/01/2014 01/04/2014 01/07/2014 01/10/2014 01/01/2015 01/04/2015 01/07/2015 01/10/2015 01/01/2016 01/04/2016 15/07/2016 15/10/2016 01/02/2017 01/05/2017 01/08/2017 01/11/2017 01/02/2018 Page size in Kilobytes Total 0 20 40 60 80 100 120 2011 2012 2013 2014 2015 2016 2017 2018 Number of request per page Total Source httparchive
- 10. Classic web application flow Presentation Layer Application Layer Data Access layer Services Services Services Services Database Rendering client Server Request page Html, Css , Js
- 11. Classic web application flow with Ajax Presentation Layer Application Layer Data Access layer Services Services Services Services Database Rendering client Server Request page Html, Css , Js Presentation Layer Ajax Data Service Ajax Request Json
- 12. Single Page Application flow Data Access layer Services Services Services Services Database Data Service client Server Request page Javascript Presentation Layer Application logic Ajax Request Json
- 13. Javascript become more complex 0 50 100 150 200 250 300 350 400 450 500 2010 2011 2012 2013 2014 2015 2016 2017 2018 Size of the Javascript in kilo bytes Total
- 14. Requirements Design Implementation Testing Acceptance Deployment Inception Elaboration Construction Transition Development Operations DevOps Big Bang Waterfall Incremental Rational Unified Process Agile Scrum, Kanban Continuous DevOps Digital Singularity Agile Event Horizon 6+ 4 2 1 Change of Methodology
- 15. DevOps :New challenges for testers No Standard • For develepment, testing and build generation Maintenance of testing assets Testing solution integrated with the corporate toolchain Test more in less time
- 16. How can we resolve all those challenges?
- 17. Solution :Script Maintenance & End User Experience NeoLoad Functions Integration switched-off for Functional Testing Recording & Script Maintenance 1. Create your initial NeoLoad VU Path from Ranorex Script 2. User Path Update feature that smartly merges the new recording and the existing NeoLoad VU Path for a very fast script maintenance. + Runtime Modes Measure End-User Experience Transaction time measured on a real device. Sent to NeoLoad’s external data for a correlation with other test metrics. 1 Script = 3 uses
- 18. Pain 1 : DevOps - How can i reduce the maintenance time?
- 19. Every Architectures is based on components
- 20. Component Testing Any part of the system Not a standard practice Do not wait until the system is assembled Test cases are simpler, fewer variables • Test-Driven Development may be an answer Many systems are monolithic Third-party components
- 21. Work flow 3) Development life cycle Component Testing Business Test Case 1) Specification 2) Risk Analysis Project Lifecycle
- 22. PERFORMANCE TESTING PLATFORM Empowering You to Test Continuously Start testing earlier Automate and accelerate testing Let Agile and DevOps teams collaborate
- 23. Component Testing REST/SOAP APIs Microservices Continuous Integration Automation Identify Performance Bottlenecks Early
- 24. System-Wide Testing Advanced & Agile Testing Technology Deploy into production with confidence Ensure The Highest Level of Quality from the User Experience Handles complex, multi-tiered applications Eliminate system-wide testing bottlenecks – test at the speed of Agile
- 25. Script Maintenance Test Case Transfert Transaction Time Analysis
- 26. Fitting Performance Testing in the pipeline [JTest]
- 27. Pain 2 : How can I measure the UX?
- 28. Rendering workflow Client WebServer Send a requestUser click on the page Application code Browser receive HTML Dom Processing Page redering
- 29. Don’t measure only the response time from the protocol level
- 30. Detailed End User Experience – Navigation Timing API Detailled page Load Metrics from Browser’s Navigation Timing API DNS time, TCP time, Processing, OnLoad event…
- 31. Applying the Ranorex-NeoLoad Integration… 00:00:00 00:00:30 00:01:00 00:01:30 Load NeoLoad Application Server App Error Rate Virtual Users Load Automated functional tests User Experience
- 32. Pain 3 : How can we do relevant regression testing?
- 33. Functional Testing Under Load - Overview Automated Functional Test Under Realistic Conditions Launch 50VU Load test Launch Functional Test Case 1 Functional Test Case 1 With 50 VU condition Add 50VU Functional Test Case 1 With 100 VU condition Stop Load Test Re-Launch Functional Test Case 1 NeoLoad Load Test
- 34. Demo
- 35. Next Steps Henrik Rexed: hrexed@neotys.com Visit Neotys Community at answers.neotys.com Download NeoLoad Free Edition from www.neotys.com Questions? Visit our website neotys.com and www.kobiton.com
Editor's Notes
- Ah le fameux 100ms d’Amazon. Le 100 ms perdu sur une page qui entraine 1% de chiffre d’affaire en moins. Vous devez surement tous le connaître celui-là. Mais je ne suis pas sur qu’il vous parle … il a le mérite d’avoir existé mais il est tellement déconnecté de notre réalité et date de tellement longtemps … Heureusement depuis, il y a en a eu d’autres des études : régulièrement on interroge des utilisateurs, on fait des sondages, on mesure
- At Neotys, we believe that Speed and Quality can be achieved by “Testing Continuously” We design NeoLoad as a performance testing platform to empower you to test at every stage of the software development cycle in a continuous manner Test early and often. Identify performance bottlenecks early in the sprint, so they can be fixed rapidly – a far cheaper proposition than having to discover later during test or even production Run full system-wide testing to ensure your application’s reliability and speed under load. If a performance issue is detected, quickly provide a deep diagnosis to helps solve the problem faster Continuously collaborate on performance testing to support the performance engineering approach Key benefits of using Continuous Performance Testing include: Finding and Resolve Issues Earlier – Issues can be found at an earlier stage in the SDLC, then fixed quicker and at lower cost Faster Release Cycles – The automation built into leading-edge performance testing platforms save a huge amount of time. With this approach, testing becomes just another part of the build – not a manual step performed as an afterthought Deploy with confidence & lower risk – Improved code quality since everything is KNOW to perform as designed – under real-world loads
- Shift Left, identify performance bottlenecks early Why? Identification of performance issue early enables easier fix for Devs, cheaper investment for the entire organization Easier for Devs: Can be complex/time-consuming to have to go back one, or potentially several sprints prior to address performance issue(s) Cheaper for the entire organization: The later the detection, the more expensive the performance issue fix How to do it with NeoLoad: API/microservices testing before the app has a GUI Support REST/SOAP Component testing Test automation with CI integration – I.e. automated as part of the nightly build process Automatic warning when performance SLAs not achieved Real-time monitoring of live test(s) to accelerate reaction time, even before test is fully completed NeoLoad is easy to learn even for non-experts. Devs can use for testing during early stages NeoLoad load generators available as Docker containers to fit with your Dev factory Benefits: Automated and real-time visibility – components performance + visibility of performance trend to identify performance regression NeoLoad enables your QA team to match the pace of Agile with a platform that Is Agile and automated Integrates in the DevOps toolchain Relative Cost of Fixing Defects: A study was performed by the IBM System Science Institute in order determine the relative cost in order to fix defects within the SDLC. The study concluded that defects found in testing were 15 times more costly than if they were found during the design phase and 2 times more than if found during implementation. (Source: IBM System Science Institute) https://www.researchgate.net/publication/255965523_Integrating_Software_Assurance_into_the_Software_Development_Life_Cycle_SDLC It is commonly believed that the earlier a defect is found, the cheaper it is to fix it. For example, if a problem in the requirements is found only post-release, then it would cost 10–100 times more to fix than if it had already been found by the requirements review.” – (Source: Code Complete 2nd Edition, Steve McConnell)
- Why do you need system wide testing? To test “from the End User Perspective – end to end” making sure the app will work when all the robust bricks will have to work “together” One major idea is “Realistic” (external dependencies, fw, cdn, different devices, networks conditions): how do they impact the perf? They are our competitive advantages Complex application, multi-tier apps. Different devices (web, mobile, IoT) Component testing does mean fully assembled app will work
- Start Q&A session (While waiting for questions, read the next steps)