Your SlideShare is downloading. ×
0
London Web Meetup – Dec 2011Continuous Integration – A Performance Engineers Tale     CONFIDENTIAL and not for reproductio...
“WE WILL DO    CONTINUOUS    INTEGRATION commandment - The              1128 TESTING”              Management             ...
INITIAL THOUGHTS•Automate all of our Performance Tests• Add tests to the Build process                                    ...
WHY IT WONT WORK•Performance testing needs a functional  application• The length of time needed to gather  significant amo...
MORE INFORMATION5   CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
GOALS FOR PERFORMANCE•Measure Performance as part of every build• Compare between builds (regression)• Alert if performanc...
Delivery                                                                             Team                                 ...
Delivery                                                                             Team                                 ...
Delivery                                                                             Team                                 ...
Delivery           Alert               Deviation                                                                          ...
Delivery           Alert               Deviation                                                                          ...
Delivery           Alert               Deviation                                                                          ...
Delivery           Alert                 Deviation                                                                        ...
“THE POINT OF TESTING IS TO FLUSH OUT, UNDERCONTROLLED CONDITIONS, ANY PROBLEM THAT THESYSTEM WILL HAVE IN PRODUCTION. A S...
Delivery           Alert                 Deviation                                                                        ...
Delivery           Alert                 Deviation                                                                        ...
Delivery           Alert                 Deviation                                                                        ...
Delivery           Alert                 Deviation                                                                        ...
Delivery           Alert                 Deviation                                                                        ...
ADVANTAGES•Allows standard performance regression testing to be fully automated, freeing up time for more complex performa...
NOT IN SCOPE•Capacity Testing• Soak Testing• Failure Testing•Stress Testing                                           Stil...
THINGS I LEARNT•Keep an open mind to new stuff 22   CONFIDENTIAL and not for reproduction without prior written consent. ©...
THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought 23   CONFIDENTIAL and not for repro...
THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work...
THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work...
THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work...
THANK YOUCONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
Upcoming SlideShare
Loading in...5
×

Continuous Integration - A Performance Engineer's Tale

1,968

Published on

Andrew Harding from Betfair's presentation on web performance testing in a continuous integration environment. Covers some good reasons why and why not to do perf testing during continuous integration.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,968
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
34
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Continuous Integration - A Performance Engineer's Tale"

  1. 1. London Web Meetup – Dec 2011Continuous Integration – A Performance Engineers Tale CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  2. 2. “WE WILL DO CONTINUOUS INTEGRATION commandment - The 1128 TESTING” Management th2 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  3. 3. INITIAL THOUGHTS•Automate all of our Performance Tests• Add tests to the Build process CI  3 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  4. 4. WHY IT WONT WORK•Performance testing needs a functional application• The length of time needed to gather significant amount of data points• Performance testing requires analysis of results to find meaningful conclusions 4 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  5. 5. MORE INFORMATION5 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  6. 6. GOALS FOR PERFORMANCE•Measure Performance as part of every build• Compare between builds (regression)• Alert if performance deviates from normal•Fail the build if performance does not meet SLA’s• Automatic process 6 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  7. 7. Delivery Team Build7 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  8. 8. Delivery Team Ready for Build Test? Deploy Environment8 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  9. 9. Delivery Team Ready for Build Test? Deploy Environment Performance Perf Trends Measurements9 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  10. 10. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Environment Performance Perf Trends Measurements10 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  11. 11. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Environment Continuous Performance Perf Trends Load Measurements Injection Usage Profile Production logs11 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  12. 12. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Environment Continuous End User Perf Trends Load Response Injection Service/API Usage response Profile Production logs System monitors12 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  13. 13. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Environment Continuous End User Perf Trends Load Response Injection For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors13 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  14. 14. “THE POINT OF TESTING IS TO FLUSH OUT, UNDERCONTROLLED CONDITIONS, ANY PROBLEM THAT THESYSTEM WILL HAVE IN PRODUCTION. A SIGNIFICANTPART OF THIS IS THE ENVIRONMENT WITHIN WHICHTHE PRODUCTION SYSTEM WILL RUN. IF YOU TEST INA DIFFERENT ENVIRONMENT, EVERY DIFFERENCERESULTS IN A RISK THAT WHAT HAPPENS UNDER TESTWON’T HAPPEN IN PRODUCTION” MARTIN FOWLER14 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  15. 15. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Production Measurements Environment End User Perf Trends Response For Each Data Service/API Point response System Perf Details monitors15 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  16. 16. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Environment Continuous End User Perf Trends Load Response Injection For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors16 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  17. 17. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Environment Continuous End User Perf Trends Load Response Injection LoadRunner For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors17 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  18. 18. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Keynote Environment Continuous End User Perf Trends Load Response Injection For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors18 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  19. 19. Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Dynatrace Ajax Premium Environment Continuous End User Perf Trends Load Response Injection For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors19 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  20. 20. ADVANTAGES•Allows standard performance regression testing to be fully automated, freeing up time for more complex performance work•Separating load injection from performance measurements enables performance to be included in build cycle•Provides immediate feedback to delivery teams 20 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  21. 21. NOT IN SCOPE•Capacity Testing• Soak Testing• Failure Testing•Stress Testing Still Plenty of Traditional work to do 21 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  22. 22. THINGS I LEARNT•Keep an open mind to new stuff 22 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  23. 23. THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought 23 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  24. 24. THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work 24 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  25. 25. THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work•Use Google 25 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  26. 26. THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work•Use Google• Design for the ideal, compromise later 26 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  27. 27. THANK YOUCONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×