Continuous Integration - A Performance Engineer's Tale

London Web Meetup – Dec 2011Continuous Integration – A Performance Engineers Tale CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

“WE WILL DO CONTINUOUS INTEGRATION commandment - The 1128 TESTING” Management th2 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

INITIAL THOUGHTS•Automate all of our Performance Tests• Add tests to the Build process CI  3 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

WHY IT WONT WORK•Performance testing needs a functional application• The length of time needed to gather significant amount of data points• Performance testing requires analysis of results to find meaningful conclusions 4 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

MORE INFORMATION5 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

GOALS FOR PERFORMANCE•Measure Performance as part of every build• Compare between builds (regression)• Alert if performance deviates from normal•Fail the build if performance does not meet SLA’s• Automatic process 6 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Team Build7 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Team Ready for Build Test? Deploy Environment8 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Team Ready for Build Test? Deploy Environment Performance Perf Trends Measurements9 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Environment Performance Perf Trends Measurements10 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Environment Continuous Performance Perf Trends Load Measurements Injection Usage Profile Production logs11 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Environment Continuous End User Perf Trends Load Response Injection Service/API Usage response Profile Production logs System monitors12 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Environment Continuous End User Perf Trends Load Response Injection For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors13 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

“THE POINT OF TESTING IS TO FLUSH OUT, UNDERCONTROLLED CONDITIONS, ANY PROBLEM THAT THESYSTEM WILL HAVE IN PRODUCTION. A SIGNIFICANTPART OF THIS IS THE ENVIRONMENT WITHIN WHICHTHE PRODUCTION SYSTEM WILL RUN. IF YOU TEST INA DIFFERENT ENVIRONMENT, EVERY DIFFERENCERESULTS IN A RISK THAT WHAT HAPPENS UNDER TESTWON’T HAPPEN IN PRODUCTION” MARTIN FOWLER14 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Production Measurements Environment End User Perf Trends Response For Each Data Service/API Point response System Perf Details monitors15 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Environment Continuous End User Perf Trends Load Response Injection For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors16 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Environment Continuous End User Perf Trends Load Response Injection LoadRunner For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors17 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Keynote Environment Continuous End User Perf Trends Load Response Injection For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors18 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

Delivery Alert Deviation from Team normal? Ready Fail Fail to for Build meet Test? SLA’s? Deploy Measurements Dynatrace Ajax Premium Environment Continuous End User Perf Trends Load Response Injection For Each Data Service/API Point Usage response Profile Production logs System Perf Details monitors19 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

ADVANTAGES•Allows standard performance regression testing to be fully automated, freeing up time for more complex performance work•Separating load injection from performance measurements enables performance to be included in build cycle•Provides immediate feedback to delivery teams 20 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

NOT IN SCOPE•Capacity Testing• Soak Testing• Failure Testing•Stress Testing Still Plenty of Traditional work to do 21 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

THINGS I LEARNT•Keep an open mind to new stuff 22 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought 23 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work 24 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work•Use Google 25 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

THINGS I LEARNT•Keep an open mind to new stuff• Give it more than 2 minutes of thought• Get past the reasons it won’t work•Use Google• Design for the ideal, compromise later 26 CONFIDENTIAL and not for reproduction without prior written consent. © of the Sporting Exchange Limited.

http://www.seriticonsulting.com	396
https://mail.google.com	1
https://twitter.com	1

Continuous Integration - A Performance Engineer's Tale

by Stephen Thair on Dec 13, 2011

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 398

Statistics

Continuous Integration - A Performance Engineer’s Tale Presentation Transcript