Your SlideShare is downloading. ×
0
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Performance Testing Driving Capacity Management at easyJet
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Performance Testing Driving Capacity Management at easyJet

650

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
650
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Positionner le logo de la société du présentateur à la place de ce bloc texte Président de filière Orateur Sponsor Administrateur itSMF 2Titre de la présentation Performance Testing Driving Capacity Management at easyJet Andy Bolton Nathalie Grandvuillemin Andy Caddy Benoit de Savignac
  • 2. Performance Testing Driving Capacity Management at easyJet easyJet.com 3
  • 3. Performance Testing Driving Capacity Management at easyJet easyJet continues to grow easyJet 2009 £2.6bn revenue 181 aircraft 45m passengers 422 routes “Low cost with care and convenience” 4 easyJet 2011 £3bn+ revenue 204 aircraft 50m+ passengers 560routes “Passionate about making travel easy & affordable” easyJet 2014 £4bn+ revenue 230+ aircraft 60m+ passengers 650+ routes ““Europe’s preferred short haul airline”
  • 4. Performance Testing Driving Capacity Management at easyJet easyJet 1996 3 Airports 2 Countries 2 Aircraft 2 Routes 5 Base Airport Other Airport
  • 5. Performance Testing Driving Capacity Management at easyJet 6 easyJet 2011 150 Airports 34 Countries 210 Aircraft 560 Routes 1500+ Flights 60m+ Passengers Base Airport Other Airport
  • 6. Performance Testing Driving Capacity Management at easyJet easyJet a growth story In 15 years easyJet has become the UK’s #1 airline 7
  • 7. Performance Testing Driving Capacity Management at easyJet IT Budgets 8
  • 8. Performance Testing Driving Capacity Management at easyJet easyJet Revenue 9 Flight revenue (easyJet.com + B2B) Flight revenue (easyJet.com + B2B) Non Flight Revenue (Ancilary Partners) Non Flight Revenue (Ancilary Partners) 100% IT Reliance 100% IT Reliance
  • 9. Performance Testing Driving Capacity Management at easyJet How important is IT? 10
  • 10. Performance Testing Driving Capacity Management at easyJet 11 How important is IT?
  • 11. Performance Testing Driving Capacity Management at easyJet Agile Development Process 12
  • 12. Performance Testing Driving Capacity Management at easyJet Business Context Capacity planning service www.easyJet.com Business critical service 98% of revenue earned through the e-commerce service Highly competitive market Frequent innovation by adopting agile development Monthly software releases Legislative changes Consistent growth; typically 20% p.a. Low-cost operating model Requirement to manage risk 13
  • 13. Performance Testing Driving Capacity Management at easyJet easyJet’s Goals 14
  • 14. Performance Testing Driving Capacity Management at easyJet easyJet’s Goals 15
  • 15. Performance Testing Driving Capacity Management at easyJet Challenges Associated with Agile Frequent change introduces risk Window available for performance testing is typically 1-2 weeks Final production candidate code often not available at the start of testing Multiple code releases within the testing window Increased probability of functional and environmental defects Frequent rework of code necessitates modifications to test scripts Cumulative effect on capacity from 12 releases per annum Narrow time window to resolve performance defects 16
  • 16. Performance Testing Driving Capacity Management at easyJet Challenges Associated with Agile 17
  • 17. Performance Testing Driving Capacity Management at easyJet Typical Causes of Degraded Service Performance 18
  • 18. Performance Testing Driving Capacity Management at easyJet How Can We Reduce Risk? 19 Cause of degraded performance Mitigated by performance testing? Mitigated by modelling? Changes in user patterns No Yes Poor performance of 3rd parties May be limited by environment Yes Physical capacity bottlenecks May be limited by size of the environment Yes Logical bottlenecks Yes If known Software configuration changes Yes Yes, if tested Increases in demand May be limited by size of the environment Yes
  • 19. Performance Testing Driving Capacity Management at easyJet Why Test? Performance testing is used to understand the software’s performance characteristics Do unknown time and growth related defects exist? Memory leaks Poor database design/indexing Do unknown concurrency related defects exist? Database locking issues Memory locking issues Race conditions Obtain model inputs 20
  • 20. Performance Testing Driving Capacity Management at easyJet Why Model? Modelling uses our understanding of the software to predict end to end service performance under different scenarios 21
  • 21. Performance Testing Driving Capacity Management at easyJet Scenario Modelling 22
  • 22. Performance Testing Driving Capacity Management at easyJet Coverage of Testing and Modelling 23 ModellingTesting Application Infrastructure Business Scenarios Technical Scenarios
  • 23. Performance Testing Driving Capacity Management at easyJet High Level Process 24
  • 24. Performance Testing Driving Capacity Management at easyJet Risk Assessment 25
  • 25. Performance Testing Driving Capacity Management at easyJet Risk Assessment Assess the potential performance impact on different elements of infrastructure Plan tests to help mitigate that impact Focus on areas of highest risk due to limited timeframes 26 Web DB Other Project X --- Medium Medium Low Test 1 Project Y --- Low High Medium Tests 2 & 3 Project Z --- Low Low Low None Description ID Risks Risk Mitigation
  • 26. Performance Testing Driving Capacity Management at easyJet Performance Tests Employed 27
  • 27. Performance Testing Driving Capacity Management at easyJet Baseline Tests Used to determine the impact of the planned release Comparative performance between releases May be Load, Stress or Soak Tests Service times and response times are compared to each other 28 Database Server Service Time Web Server Service Time Test 1.1 Release X 752 1240 Test 1.1 Release Y 813 1192 % Difference 8% -4% 0 200 400 600 800 1000 1200 1400 ServiceTime(ms) Example service time comparison between two fictional baseline tests
  • 28. Performance Testing Driving Capacity Management at easyJet Single Transaction Type Tests 29 To determine the impact on the infrastructure of individual user transactions Every business process which is to be modelled will have a calculated service time for each hardware element 132 233 156 122 221 328 Milliseconds Web CPU per page (fictional data) Home Search Select Options Login Purchase
  • 29. Performance Testing Driving Capacity Management at easyJet Modelling 30 Performance Test Results
  • 30. Performance Testing Driving Capacity Management at easyJet Performance Modelling 31 Business Service Component Virtual Infrastructure
  • 31. Performance Testing Driving Capacity Management at easyJet Performance Modelling 32 Business Service Component Virtual Infrastructure Aircraft Home Page Search Booking HTTP1 APP1 APP2 DB1
  • 32. Performance Testing Driving Capacity Management at easyJet Performance Modelling 33 Business Service Component Virtual Infrastructure 600 100 80 5 160 70 10 80 3.5 transactions/s
  • 33. Performance Testing Driving Capacity Management at easyJet Demand Model Validation The model should be validated on an ongoing basis against actual values The first stage is to validate the demand part of the model to ensure that arrival rates are being calculated correctly Following this the true demand can be used to validate the utilisation calculations 34 600 100 80 5 610 80 80 6
  • 34. Performance Testing Driving Capacity Management at easyJet Capacity and Performance Model Validation 35 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Comparison of Actual Database Load to Predicted Database Load Actual Database CPU Predicted Database CPU
  • 35. Performance Testing Driving Capacity Management at easyJet What you need – People Capacity and performance management skills: Demand forecasting Modelling Load testing A strong understanding of risk management A good understanding of the business model Flexibility Project management skills 36
  • 36. Performance Testing Driving Capacity Management at easyJet What you need – Process 37 Interface with the release team Participation in the agile development `scrums` Access to developers to understand application functionality Interface with the test environment management team for test planning Interface to the business to access business forecasts Interface to operations to access production measurements
  • 37. Performance Testing Driving Capacity Management at easyJet What you need – Tool & Environment An appropriately sized test environment Production-like environment is not required Consider the ratio of capacity between presentation, business and data tiers Toolset Performance monitoring tools (production and test) Load generation tool Modelling tool 38
  • 38. Performance Testing Driving Capacity Management at easyJet What you need – Data Software/hardware configuration (production and test) Performance measurements from production and test environments Benchmark data (Spec, TPC, etc.) Service times derived from performance test results SLAs Internal SLAs with third-parties Historical and forecast business demand Historical and forecast service demand Measured error of previous forecasts 39
  • 39. Performance Testing Driving Capacity Management at easyJet Historical Business Demand 40 y = 0.0461x - 1673.1 R² = 0.9668 0 20 40 60 80 100 120 140 160 180 Mar-04 Jun-04 Sep-04 Dec-04 Mar-05 Jun-05 Sep-05 Dec-05 Mar-06 Jun-06 Sep-06 Dec-06 Mar-07 Jun-07 Sep-07 Dec-07 Mar-08 Jun-08 Sep-08 Dec-08 Mar-09 NumberofOwnedAircraft Delivery Date Fleet Plan to April 2009 GB Airways acquisition
  • 40. Performance Testing Driving Capacity Management at easyJet Historical Service Demand 41
  • 41. Performance Testing Driving Capacity Management at easyJet Costs 42
  • 42. Performance Testing Driving Capacity Management at easyJet Benefits Testing and modelling complement one another The combined approach provides effective risk-mitigation in the context of: Frequent change Narrow performance test window Business growth The approach enables us to rapidly answer business and technical ‘what if’ scenarios Costs are limited through the use of a scaled-down performance test environment and low-cost modelling tools Provides performance assurance in a highly competitive market place “Capacitas give us the confidence in making the correct release decision” James Mackay, Test Manager, easyJet 43
  • 43. Performance Testing Driving Capacity Management at easyJet Questions Presentation will be available for download at www.capacitas.fr Please visit us at our stand for any further questions 44

×