Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Stefano Doni - Achieve Superhuman Performance with Machine Learning

Since its beginning, the Performance Advisory Council aims to promote engagement between various experts from around the world, to create relevant, value-added content sharing between members. For Neotys, to strengthen our position as a thought leader in load & performance testing. During this event, 12 participants convened in Chamonix (France) exploring several topics on the minds of today’s performance tester such as DevOps, Shift Left/Right, Test Automation, Blockchain and Artificial Intelligence.

  • Login to see the comments

Stefano Doni - Achieve Superhuman Performance with Machine Learning

  1. 1. Achieve Superhuman Performance with Machine Learning Stefano Doni
  2. 2. The Problem of Untuned Systems
  3. 3. Configurationcomplexity is huge and on the rise MySQL configuration parameters grown 5x in the last 10 years Oracle Hotspot JVM versions has >700 configuration parameters How do configurations impact application performance and infrastructure efficiency? 757
  4. 4. Configurationssignificantlyimpact performanceand costs Source: Moviri Computer Measurement Group Best Paper, 2015 Transactions/sec Working days CPU Util Workload CPUUtilization% Before tuning: 60% CPU Utilization After tuning: CPU cut by 5x JBOSS JVM Performance Tuning
  5. 5. Hyper-configurationbeyond Human scale Hardware (Cloud) VM Instance Operating System Container Java Virtual Machine Middleware & Framework Application (Cloud) Network (Cloud) Storage # of Parameters 700 100 500 200 10 10 10 10 Looking for the optimal settings? It’s easy, just try 2100= 121,267,650,600,228,229,4 01,496,703,205,376 configurations…
  6. 6. No single optimalconfigurationexist
  7. 7. Enter the New Era: performance optimization
  8. 8. Key Capabilitiesfor AI-driven,AutomatedPerformance Optimization Powered by AI Automated Full-stack Goal-driven
  9. 9. A new visionfor AI-drivenand automatedperformance optimization CONFIGURE PERFORMANCE TEST MEASURE
  10. 10. Motivating Examples
  11. 11. Optimizinga core Banking platform • Goal • Increase the key business service metric: payments per second • … while keeping latency under SLAs • ... without additional infrastructure and license costs • Optimization scope • Java OpenJDK 8 • JBoss • RedHat DataGrid (InfiniSpan) • Linux
  12. 12. We outperformedexpertsand identifiedthebest configurationto increasepayments per second by 55% 1.55x performance achieved after 20h of automated tuning Manually tuned by experts (Baseline) → Score = 100%
  13. 13. Optimizationoutcomes:best configurationfor differentgoals What if you could run a series of automated performance tests for 24 hours where the outcome is the optimized configuration settings, across your stack, for • Throughput • Latency • Resource utilization • Cloud costs • …
  14. 14. MongoDB PerformanceOptimization • Goal • Increase database throughput (query/sec) • Decrease query latency • Save cloud costs • Optimization scope • MongoDB • Linux
  15. 15. Results Tuning MongoDB (~10 params.) +30% query/sec over vendor default Tuning MongoDB + Linux kernel (~40 params.) 2x query/sec over vendor default
  16. 16. AI can explainwhere does performancereally come from Q: Which parameters actually allowed to achieve 2x throughput? A: Out of 40 Linux kernel and MongoDB parameters, just three have a significant impact NO silver bullet! This is the result of a specific optimization. It is dependent on the application, workload, hardware, cloud options, optimization objectives, etc.
  17. 17. AI can efficientlysolve complex optimizationspaces Baseline (vendor default): MongoDB cache=15GB Linux read ahead=0 Optimized (2x query/sec): MongoDB cache=30GB Linux read ahead=8
  18. 18. Why full-stack optimization?The effects of Linux tuning Baseline Optimized Disk IOPS cut to 1/3 and disk latency doubled, apparently making things slower but… this resulted in 2x MongoDB throughput increase
  19. 19. AI can find counter-intuitivesettingsexpertsnever tried Baseline (vendor default): MongoDB cache=15GB MongoDB dirty target=5% Optimized (+20% query/sec): MongoDB cache=4.3GB MongoDB dirty target=58%
  20. 20. Conclusions
  21. 21. Driversfor adopting the new AI-drivenoptimizationapproach Costs CAPEX reduced due to increased performance / new investments deferral OPEX reduced with full automation of testing, analysis & tuning cycle Revenue Quality of service improved of customer facing services or batch processes Agility New app releases faster shrinking the optimization cycle from months to days Strategy Innovation programs accelerated thanks to automated performance optimization Risks Service outage or slowdown risks reduced thanks to optimized configurations
  22. 22. Conclusions • Todays’ software stack is far too complex for our human brains • Business impact: significant performance left on the table, lower agility • Machine learning can smartly navigate complex optimizations and find counter-intuitive, unexplored settings beating experts and yelding big gains • A new AI-driven approach to performance optimization is required to achieve the benefits in modern DevOps settings - #AIDevOps • Key capabilities include end-to-end automation of performance experiments, full-stack coverage, fast and robust AI optimization of user- driven goals
  23. 23. Thank you