Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Neotys PAC 2018 - Jonathon Wright

135 views

Published on

The PAC aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing.

Since its beginning, the PAC is designed to connect performance experts during a single event. In June, during 24 hours, 20 participants convened exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.

Published in: Engineering
  • Be the first to comment

Neotys PAC 2018 - Jonathon Wright

  1. 1. Performance as Code (PAC) Jonathon Wright
  2. 2. PerformanceEngineeringCommunity https://www.linkedin.com/in/Automation/ https://github.com/PerfDriver https://twitter.com/Jonathon_Wright jlwright@iee.org https://www.slideshare.net/Jonathon_Wright
  3. 3. Performance as Code
  4. 4. Performanceas Code (PAC) • Background – World Digital Report 2018 • Performance as Code (PaC) • Minimum Viable Performance (MVPx) • Continuous Performance (CPx) • High Volume Performance (HVPx) - Microcontainerization (Unikernal + Kubernetes) • Digital Performance Lifecycle (DPL) • Test Data as Code (TDaC) - Model transactions with powerful data creation engine (GDPR) and databases look-up • Robotic Process Automation (RPA) - Expose all the parameters, attributes and options as parameters (CLI) • Lifecycle Virtualization
  5. 5. WORLD DIGITAL REPORT 2018
  6. 6. WORLD DIGITAL REPORT 2018 https://www.neotys.com/resources/world-digital-report-boxing-day-edition
  7. 7. WORLD DIGITAL REPORT 2018 RETAIL FOCUSED 200,000+ TOP RETAILERS 500 GLOBAL WEBSITES 1 Million SITES MONITORED 1,000+ REGIONS 4 http:///www.WorldDigitalReport.com
  8. 8. WORLD DIGITAL REPORT 2018 Ready to use, DOM interactive, visually and render complete times. Speed Digital assessment methodology for assessment of current digital solution with a focus for independently assessing opportunities for improvement within performance engineering. Digital Maturity Model index (DMMi) Monitored pre / post Boxing Day & January Sales (48hrs) Stability Measured average response times for both Desktop (WIFI) and Mobile (4G / 3G / 2G). Response Performance index report covering, W3C, R/R, DOM, CDN and DNS benchmarking. Index 30 INDEX RESPONS E STABILIT Y SPEED DIGITAL MATURITY MODEL INDEX (DMMi)
  9. 9. WORLD DIGITAL REPORT 2018 BENCHMARK Internet rankings came from Alexa.com and is a indication of global internet traffic to the site, this years weightings to derive the overall score was performance index (30%), response (15%), stability (20%) and speed (15%) which was real end user experience. MYLOADTEST.COM Monitored pre / post Christmas & Boxing Day (48hrs). (result A indicates between -1% & +1% change of response over time) STABILITY Measured average response times for both Desktop (WIFI) and Mobile (4G / 3G / 2G). (result of less than 5 (Desktop) / 10 (Mobile) seconds response time people tend to typically navigate away) RESPONSE Performance index report covering, W3C, R/R, DOM, CDN and DNS benchmarking. (result of less than the average of 70 really needs help with optimization) INDEX Company Twitter Index Traffic Response Stability Issues Overall Desktop Mobile Spike 1 Dominos.co.uk @Dominos_UK 98 8,113 0.54s 1.81s 5.81s A N/A A
  10. 10. Company Twitter Index Traffic Response Stability Issues Overall Desktop Mobile Spike 81 watchshop.com @watchshop 37 16,473 7.0 14.2 14.5 C+ D+ E 82 harrods.com @harrods 35 13,256 7.0 12.4 22.3 A F E 83 maplin.co.uk @maplintweet 35 15,362 4.5 24.0 42.6 E- E E 84 whsmith.co.uk @WHSmith 35 49,846 13.1 15.5 35.2 F D- E 85 topman.com @topman 34 11,733 5.2 10.6 11.3 A E E- 86 jessops.com @jessops 33 61,237 7.2 30.6 38.9 F D- E- 87 dorothyperkins.com @dorothy_perkins 33 27,140 5.3 10.6 19.9 A- D E- 88 dfs.co.uk @DFS 33 61,650 9.3 16.6 20.0 C- D- E- 89 waterstones.com @waterstones 31 26,529 10.2 20.4 24.4 A D- E- 90 lipsy.co.uk @LipsyLondon 30 12,133 9.8 14.8 30.0 A E F WORLD DIGITAL REPORT 2018
  11. 11. Performanceas Code (PAC) • PerfDriver.io (https://github.com/PerfDriver) execution: - scenario: perfdriver scenarios: perfdriver: script: VIP.pac
  12. 12. Performanceas Code (PAC) <?xml version="1.0" encoding="UTF-8"?> <pacTestPlan version="1.2" properties="2.3"> <hashTree> <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="/" enabled="true"> <boolProp name="HTTPSampler.postBodyRaw">true</boolProp> <elementProp name="HTTPsampler.Arguments" elementType="Arguments"> <collectionProp name="Arguments.arguments"> <elementProp name="" elementType="HTTPArgument"> <boolProp name="HTTPArgument.always_encode">false</boolProp> <stringProp name="Argument.value"></stringProp> <stringProp name="Argument.metadata">=</stringProp> </elementProp> </collectionProp> </elementProp> <stringProp name="HTTPSampler.domain">curiositysoftware.ie</stringProp> <stringProp name="HTTPSampler.port"></stringProp> <stringProp name="HTTPSampler.connect_timeout"></stringProp> <stringProp name="HTTPSampler.response_timeout"></stringProp> <stringProp name="HTTPSampler.protocol">http</stringProp> <stringProp name="HTTPSampler.contentEncoding"></stringProp> <stringProp name="HTTPSampler.path">/</stringProp> <stringProp name="HTTPSampler.method">GET</stringProp> <boolProp name="HTTPSampler.follow_redirects">true</boolProp> <boolProp name="HTTPSampler.auto_redirects">false</boolProp> <boolProp name="HTTPSampler.use_keepalive">true</boolProp> <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp> <boolProp name="HTTPSampler.monitor">false</boolProp> <stringProp name="HTTPSampler.embedded_url_re"></stringProp> </HTTPSamplerProxy> <hashTree /> https://github.com/PerfDriver/PerfDriver/tree/master/samples
  13. 13. Minimum ViablePerformance(MVPx)– Selenium.io sudo apt-get update sudo apt-get install • python • default-jre-headless • openjdk-8-jre-headless • python-tk • python-pip • python-dev • zlib1g-dev • net-tools • selenium • Appium-Python-Client • brew/apt§/yum • npm/node/js_tests/mocha • bzt
  14. 14. Minimum ViablePerformance(MVPx)- NeoLoad • uname –a (check build x86_64) • neoload_6_4_0_linux_x86.sh
  15. 15. Minimum ViablePerformance(MVPx)– PerfDriver.io Load generator agent • Geo-realistic location • Network Virtualization • High volume performance (HVPx) + mobile emulation • Low CPU / memory usage (1.4GHz Quad-Core / 1GB DDR2) • Network Function Virtualization (Gigabit Ethernet / 802.11ac) • Low Cost @ £32 GBP per agent
  16. 16. Minimum ViablePerformance - Microcontainerization • docker https://hub.docker.com/r/neotys/neoload-loadgenerator • unikernel https://hub.docker.com/r/unikernel/mirage • windows 2019 http://aka.ms/ws2019previewbuildsreleasenotes • openshift for red hat https://www.openshift.com • kubernetes https://kubernetes.io • suse CaaS
  17. 17. Continuous Performance - Microcontainerization- Kubernetes • https://portal.azure.com • devops project (preview) • docker docker pull neotys/neoload-loadgenerator docker pull perfdriver/perfdriver.io
  18. 18. Continuous Performance - Microcontainerization- HVPx • High vCPU (16 cores) • High Memory (432GB) • Nodes (5,000) • vUsers (2.5million) • £3.70 per hr
  19. 19. Continuous Performance - Microcontainerization- HVPx
  20. 20. Continuous Performance - Microcontainerization- HVPx
  21. 21. DigitalPerformanceLifecycle – Testinsight.io systemised ordered cognitive thinking represented in an algorithmic approach INSIGHT-DRIVEN – HOLISTIC MODELLING http://leanpub.com/digital
  22. 22. DigitalPerformanceLifecycle – Quantum Téléportation https://www.dynatrace.com/ platform/artificial- intelligencehttps://www.contrastsecurity.com/devops
  23. 23. DigitalPerformanceLifecycle – Shift Right (PAL) • Activating event • Encapsulate interactions • Cause & effect modelling • Learn behaviours • Automate workflows • Adaptive difficulty • Generate schematics • GAP Analysis • Process validation • Behavioural experiment • Visualize blueprint • Self monitoring
  24. 24. DigitalPerformanceLifecycle – GenerateDigitalBlueprint • Generate Business Process Model (BPMn v2.2) • Transactions (BPTx) • Swimlanes (ecosystems) • Generate Digital Blueprint (xPDL)
  25. 25. DigitalPerformanceLifecycle – Shift Right (PAL) • WebDriver (Headless) • PerfDriver (Headless)
  26. 26. DigitalPerformanceLifecycle – ReverseEngineering • Top Layer - End Users (Prod) / User Acceptance (UAT) - Manu’mation Tests via Browser / App - Automated Tests via Selenium (DOM) - Automated Tests via ArtOfTesting (.NET) • Middle Layer - Manu’mation Tests via Postman (API) - Automated Tests via Selenium (Headless) - Automated Tests via RESTSharp (Swagger) - Performance Tests via PerfDriver.io (R/R) - Load Tests via NeoLoad Agent (Raspberry) • Bottom Layer - IntelliTest Generated (VS2017) - Code Analyzers (SonarCube) - Unit Testing Frameworks (xUnit) fiddler2.com/r/?reverseproxy IIS or Apache (Port 81) WinINET System .NET .NET APP Internet Explorer Selenium (Headless) Firefox RESTSharp WinHTTP Proxy Fiddler (Port 80)
  27. 27. RoboticProcess Automation – CLI for TestingPerformance Sentiment Analysis
  28. 28. RoboticProcess Automation – MicrosoftCognitiveServices Trigger examples • Conversational AI (ChatOps) • Recurrence MS Flow control Action examples • Computer vision (OpenCV) • CustomVision.AI (Microsoft) https://southcentralus.api.cognitive.microsoft.com/customvision/v2.0/Prediction/b5cece8c-ea39-4b7d-bfe5-8ff4d3852b58/image?iterationId=bd8d8b99-559b-4c48-8372-3f3cb92f50f1
  29. 29. RoboticProcess Automation – Test Dataas Code
  30. 30. RoboticProcess Automation - jidoka.io Robot Type Business Value Driver Robot Activities Robot Interactions Robot Characteristics Robot Technology Basic Operational Efficiency • Do • Robot to Applications • Structured Data Processing • Semi-structured Data Processing • Rule-based RPA • Unattended RPA • UI, API Smart Operational Efficiency • Do • Think • Robot to Applications • Robot to Ops Employees • Structured Data Processing • Semi-structured Data Processing • Unstructured Data Processing • Advanced Exception Handling • Complex Unattended RPA • UI, API, OCR/Computer Vision • Event based Pause/Resume • Email/Text Alert Triggers Collaborative Speed of Business Transaction • Do • Think • Manage • Robot to Applications • Robot to Ops Employees • Robot to Robot • Structured Data Processing • Semi-structured Data Processing • Unstructured Data Processing • Interactive Exception Handling • Executing other robots • Unattended RPA • UI, API, OCR/Computer Vision • Event based Pause/Resume • Email/Text Alert Triggers • NLP, Virtual AI Assistant • API Triggered Robot Execution • IoT device triggered Robot Management Autonomous Digital Experience (DX) • Do • Think • Manage • Learn • Robot to Applications • Robot to Ops Employees • Robot to Robot • Robot to Customer • Structured Data Processing • Semi-structured Data Processing • Interactive Exception Handling • Unstructured Data Processing • Managing other robots • Decision Making • Unattended RPA • UI, API, OCR/Computer Vision • Event based Pause/Resume • Email/Text Alert Triggers • NLP, Virtual AI Assistant, Chatbot • API Triggered Robot Execution • Machine Learning, Data Analytics
  31. 31. Lifecycle Virtualization(AI Ready) – Dynamic MVC (R/R) var mdarray = []; for (i = 0; i < excounter; i++) { mdarray.push(context.variableManager.getValue("MaterialArray_"+ i)); } for (loop = 1; loop < excounter; loop++) { response += "<column><id>" + mdarray[loop] + "</id></column>"; } context.variableManager.setValue("MasterData", response); Controller Request Model Builds View RendersResponse
  32. 32. Lifecycle Virtualization(AI Ready) – (SV+ NV + NFV) BRINGS TRUE ARTIFICIAL INTELLIGENCE (AI) TO LIFECYCLE VIRTUALIZATION ▪ Virtualize services without requiring any knowledge or decoding of the service protocols ▪ Applies a genome sequence alignment algorithm, discovers byte-level patterns in message protocols ▪ Now virtualize a much wider range of protocols without requiring a new DPH HIGH ACCURACY: 99.6 – 100% ▪ Increased speed and accuracy with Entropy Weighting + Message Clustering ▪ The more data a service observes the data, more intelligent it becomes ▪ Perfect for performance testing where we deal with tons of data
  33. 33. DigitalAssurance – CognitiveAdoption Platforms • Cognitive Learning Services (CLS) • Virtual Personal Assistants (VPA) • Smart Advisors • Natural language processing • Situation Awareness • People-Literate Technology • Digital Experiences (DX) • Internet of Everything (IoE) • Human Augmentation • Ambient Experiences • Gesture Control • Brain Computer Interface (BCI) • Emotion Detection • Head-Mounted Displays • Virtual Worlds • Context Brokering Platforms • Digital Offers • City Data Exchange • Complex Event Processing • Atomic Level Mass Personalization • Connected Vehicle • Autonomous Vehicles (C2X) • Vehicles to Infrastructure (I2X) • Mood Recognition • Cognitive Adoption - The AI Imperative • Ultra-Intelligence (Singularity) • Artificial Intelligence Foundations • Deep Reinforcement Learning • Learning / Selfaware Software • Neural Networks / Conversational • Quantum / Fog / Edge Computing • Cognitive Reckoning / Abstract Thinking • Humanoid robots • Vision / Sensors / Knowledge Replication • Neuromorphic Hardware • Digital Twins / Hyperconverged • Cyber-physical systems • Cryptocurrency Lawful hacking • Cyber Threat Prediction / Zerotrust security • Location-based authentication • Growth / Performance hacking • Flexible displays / Screenless interface • Smart Dust / Workspace / Robots • Nanotube Electronics • Accumulated Reality • Self-healing & aware systems • Digital Out-of-Home (DOOH) • Additive manufacturing • Cognitive Adoption Platforms (GAP) • Virtual / Augmented / Mixed Reality • Immersive / Human Augmentation • Intelligent / Enterprise of Things (EoT) • Blockchain / Event Driven • Mobilegeddon / Intelligent Apps • Device Mesh / Sensorization • Data visualization / stewardship • Business / Digital transformation • Tri-Modal / Shadow IT • Interactive digital signage / Ambient UX • Omni channel engagements • Adaptive risk, trust & learning • Microcontainerization [Unikernels] • BI / Big Data [NoSQL] • Wearables / Haptics / Gesture tech • 4D / 3D printing • Uberfication of services • Object based storage • Smart building technologies • Cloud aggregator / broker • Chaos Monkey • Gamification • Commercial drones (UAVs)
  34. 34. DigitalEvolution- CognitiveLearning
  35. 35. DigitalAssured – Evolution,over Revolution

×