Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
1 of 14

Neotys PAC - Ian Molyneaux



Neotys organized its first Performance Advisory Council in Scotland, the 14th & 15th of November.
With 15 Load Testing experts from several countries (UK, France, New-Zeland, Germany, USA, Australia, India…) we explored several theme around Load Testing such as DevOps, Shift Right, AI etc.
By discussing around their experience, the methods they used, their data analysis and their interpretation, we created a lot of high-value added content that you can use to discover what will be the future of Load Testing.

You want to know more about this event ?

More Related Content

You Might Also Like

Related Books

Free with a 30 day trial from Scribd

See all

Neotys PAC - Ian Molyneaux

  1. 1. Historical Performance Testing Requirement Tooling Evolution in Response to Change
  2. 2. Tooling Evolution
  3. 3. HistoricalTooling Requirement.. • Protocol Support • Many and varied • Citrix – ICA • SAPGUI • Oracle NCA • ODBC • Native Database, MS SQL, SIEBEL, Oracle • Winsock, WAP, HTTP(s) • Tuxedo… • Client Support • Fat Client • Early Browser • Testing Client -> Server Model
  4. 4. Reasons for Change.. • Business • Digital Revolution and the WWW • Explosion of eCommerce • Cloud • Mobile • Technology • Demise of the client /server model • Move to distributed and increasingly modular architecture • SOA • Services (Web, RESTApi) • Micro-services • Containers • Move to web enablement and cloud deployment
  5. 5. Current Tooling Requirement.. • Far Fewer Protocols • HTTP / HTTPS • Websockets • Citrix – ICA • Some packaged applications (Remedy, SIEBEL) • Rarely a requirement for anything else • Legacy tooling vendors continue to deprecate protocol support • More Client Options • Browser • Mobile Device • Service / API • Client Choice and Expectation • SaaS, Self-Service Offerings • Opensource tooling resurgent – attempt to make it more of an Enterprise solution. • Licensing cost much more of an issue • Free Community versions of licensed tooling becoming the norm.
  6. 6. Impact on Tooling Functionality
  7. 7. Scripting • Change • Move from coding to DOM scripting • Support for multiple scripting languages • Improved auto-correlation • Easier to learn • Reduced scripting effort • Reduced dependencies – No compiler required • Real-World • Must still offer a scripting option • One well known vendor recently deprecates click and script support • Script in one tool, convert to another.
  8. 8. Load Injection • Change • On-premise and Cloud support • Windows and LINUX O/S support • Bandwidth simulation • Real World • Cloud support is mandatory • Cloud injector management sometimes clumsy • Much less requirement to test on-premise • Much easier to provision injectors
  9. 9. Test Management • Change • Increased sophistication in test design options • Options to integrate with test automation servers and frameworks • Jenkins, HP Quality / Performance Centre. HP ALM • Collaborative project options • Real World • Important to easily control VU weighting and injection profile • Reality is most tests are simple ramp to target load or break point • Little uptake of collaborative performance testing • Management of test assets still a mix of native and 3rd-party source control • GitHub
  10. 10. Analysis • Change • Increased sophistication of reporting. • Real-time graphical analysis the norm. • Greater selection of auto-generated graphs and tables. • Real World. • Test reporting still largely a manual exercise. • Manual extraction of graphs and tables into MS Word / pdf document. • MS Excel still a more flexible analysis option in many instances. • Many tools still don’t support X,Y and Z axis series in same graph. • Difficult to set / extract Nth percentiles (90th,95th). • Difficult to compare results between test runs • Native management of test results not as slick as it could be.
  11. 11. Integration • Change • Increased native integration with point monitoring solutions. • Integration with leading Application Performance Management (APM). • DynaTrace, Appdynamics, New Relic • Simplification of Application and Infrastructure metric correlation. • Integration with CI automation and build servers. • Jenkins, Cucumber • Real World • Monitoring Integration often troublesome to leverage effectively – • Pushback from SecOps • Relatively little uptake of single tool solution for NF automated CI integration as part of DevOps. • Clients prefer to build their own custom integrations as part of CI,TDD,BDD
  12. 12. Future Trends
  13. 13. FutureTrends? • Protocols • IOT support although use case still unclear • Continued deprecation of legacy protocols • Scripting • Maintain flexibility in language choice • DOM and code continue to co-exist • Load Injection • Better management of injector deployment • Test Management • Centralise test management across Dev (QA) Ops. • Ability to leverage non-native test assets • Analysis • Improved summary reporting • Auto comparison of KPI delta across test runs • Better integration with Big Data (ELK , Grafana) • Integration • Single vendor solution unlikely for CI • Maintain flexible API and leverage community plugin development
  14. 14. Thank you - Q & A