Monitoring Performance from Development through ProductionAgile Development of HIGH Performance APPLICATIONS
Fabian LangeHead of Competence Center PerformanceJava since its beginningAgile since its establishmentPerformance since waiting got boringMeandcodecentriccodecentric AG
Specialized in
Perfomance Services
Agile Software Factory
AlwayslookingfornewtalentTable of ContentsChapter One"The curse of non-functional Requirements”Chapter Two“Ensuring Great Performance”Chapter Three“A Real World Example”Chapter Four“The DevOps Revolution"Epilogue
The curse of non-functional RequirementsChapter One
Who measures performance …in production?before production?during development?Who does development …the waterfall way?the agile way?Let‘s Do a Poll
We finally can test functional requirements!Many modern practicesTDDATDDBDDTesting Requirements
Executable Specifications makes functional testing a breeze!RequirementsaretheTEst*** Settings ***Resource         ${RESOURCES}/BDD.txtTest Template    Branch Manager Change Should not affectEmployee*** Keyword ***Branch Manager Change Should not affectEmployee    [Arguments]  ${periodClosed}  ${periodOpenAndModified}  ${importDay}  ${oldManagerValidUntil}  ${newManagerValidFrom}GiveninitializedcriteriaforbonuscommercialAnd a branch B withbranchmanager M_OLD andemployee E1Andevaluationfor E1 forperiod ${periodClosed} whichisclosedAndevaluationfor E1 forperiod ${periodOpenAndModified} whichis open andmodifiedWhen M_NEW becomesnewmanagerofbranch BAndimportserviceiscalled on ${importDay}Thenthenewbranchmanagerofbranch B is M_NEW valid from ${newManagerValidFrom}Andbranchmanager M_OLD managesemployee E until ${oldManagerValidUntil}Andbranchmanager M_NEW managesemployee E from ${newManagerValidFrom}And Evaluations for E1 still havethe same content| *Test* | *ClosedPeriod*        | *Open Period*          | *Run Import On* | *Old Manager Stops* | *New Manager Starts* || 1      | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 | 11.11.2009      | 30.11.2009          |  1.12.2009           || 2      | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 |  1.11.2009      | 31.10.2009          |  1.11.2009           || 3      | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 |  1.12.2009      | 30.11.2009          |  1.12.2009           |
Performance is Non-Functional!All Non Functional Requirements are not very agileThey cannot be added later onSo you need to know about them!They form the Definition of Done
Testing Non-Functional is hard!There are no absolute measuresNo production infrastructure
How Do You Measure Performance?Relevant Measures are hard to findResponse TimeFor usersSystem LoadFor planningTrafficFor moneyRealistic Measures are hard to obtain2 seconds?Load avg < 2.8 ?Less than 2TB per month?
How Do You Test Performance?Who ...... has a process for performance tests?... does loadtests?... plans for scalability?... uses a profiler?... uses a server monitor?... uses an application monitor?
„ProductionisFaster“
Ensuring great PerformanceChapter Two:
A Typical Performance Analysis ProcessTom, the boss calls:“We loose customers because  of bad performance”Lynn from QA does a load test:“Application is slow as a snail”Task force is set upJohn tries to learn performance toolsSarah does a microbenchmark and gains 5 ms Application still slow Everybody unhappy
How about…Developers care about performanceGood tools are understood and usedPerformance is tested regularlyAnomalies are taken care of Application is running smoothly Everybody is happy
Care About PerformanceCaring is fundamentally importantDevelopment teams need to extend their scopeIn Scrum teams need to be able to do all the work to get done
Good ToolsTool paralysis does not helpChoose 1 or 2 good tools and learn themTools should work everywhereflickr.com/photos/pmtorrone/2381346935
Continuously Test PerformanceFind a good balanceAutomated ChecksManual TestsFunctional tests already provide dataHow about a load test every iteration?
Investigate Suspicious DataBecause you careAnd you have the toolsAnd you have the dataYou should investigate findings“When you have eliminated the impossible,whatever remains, however improbable,must be the truth”	- Sherlock Holmesflickr.com/photos/cayusa/2159980025
A real World ExampleChapter Three
Team Cares About PerformanceDefinition of Done includes a lotspec, design, unit test, code, acceptance tests, documentation, usability review, code review, stability tests, compatibility tests, interoperability test, load tests, security tests, performance tests…Get it right from the beginningDo not pile up technical debtTo go well you have to go slow
20% Slowdown due to debt
Sprint 1
15 Points delivered
Sprint 2
12 Points delivered
Sprint 3
9 Points delivered
Sprint 4
7 Points deliveredTechnical Debt / Undone WorkWrong Definition of DoneBetter Definition of DoneSprint 1
12 Points delivered
Sprint 2
12 Points delivered
Sprint 3
12 Points delivered
Sprint 4

Agile Development of High Performance Applications

  • 1.
    Monitoring Performance fromDevelopment through ProductionAgile Development of HIGH Performance APPLICATIONS
  • 2.
    Fabian LangeHead ofCompetence Center PerformanceJava since its beginningAgile since its establishmentPerformance since waiting got boringMeandcodecentriccodecentric AG
  • 3.
  • 4.
  • 5.
  • 6.
    AlwayslookingfornewtalentTable of ContentsChapterOne"The curse of non-functional Requirements”Chapter Two“Ensuring Great Performance”Chapter Three“A Real World Example”Chapter Four“The DevOps Revolution"Epilogue
  • 7.
    The curse ofnon-functional RequirementsChapter One
  • 8.
    Who measures performance…in production?before production?during development?Who does development …the waterfall way?the agile way?Let‘s Do a Poll
  • 9.
    We finally cantest functional requirements!Many modern practicesTDDATDDBDDTesting Requirements
  • 10.
    Executable Specifications makesfunctional testing a breeze!RequirementsaretheTEst*** Settings ***Resource ${RESOURCES}/BDD.txtTest Template Branch Manager Change Should not affectEmployee*** Keyword ***Branch Manager Change Should not affectEmployee [Arguments] ${periodClosed} ${periodOpenAndModified} ${importDay} ${oldManagerValidUntil} ${newManagerValidFrom}GiveninitializedcriteriaforbonuscommercialAnd a branch B withbranchmanager M_OLD andemployee E1Andevaluationfor E1 forperiod ${periodClosed} whichisclosedAndevaluationfor E1 forperiod ${periodOpenAndModified} whichis open andmodifiedWhen M_NEW becomesnewmanagerofbranch BAndimportserviceiscalled on ${importDay}Thenthenewbranchmanagerofbranch B is M_NEW valid from ${newManagerValidFrom}Andbranchmanager M_OLD managesemployee E until ${oldManagerValidUntil}Andbranchmanager M_NEW managesemployee E from ${newManagerValidFrom}And Evaluations for E1 still havethe same content| *Test* | *ClosedPeriod* | *Open Period* | *Run Import On* | *Old Manager Stops* | *New Manager Starts* || 1 | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 | 11.11.2009 | 30.11.2009 | 1.12.2009 || 2 | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 | 1.11.2009 | 31.10.2009 | 1.11.2009 || 3 | 1.11.2009 - 30.11.2009 | 1.12.2009 - 31.12.2009 | 1.12.2009 | 30.11.2009 | 1.12.2009 |
  • 11.
    Performance is Non-Functional!AllNon Functional Requirements are not very agileThey cannot be added later onSo you need to know about them!They form the Definition of Done
  • 12.
    Testing Non-Functional ishard!There are no absolute measuresNo production infrastructure
  • 13.
    How Do YouMeasure Performance?Relevant Measures are hard to findResponse TimeFor usersSystem LoadFor planningTrafficFor moneyRealistic Measures are hard to obtain2 seconds?Load avg < 2.8 ?Less than 2TB per month?
  • 14.
    How Do YouTest Performance?Who ...... has a process for performance tests?... does loadtests?... plans for scalability?... uses a profiler?... uses a server monitor?... uses an application monitor?
  • 15.
  • 16.
  • 17.
    A Typical PerformanceAnalysis ProcessTom, the boss calls:“We loose customers because of bad performance”Lynn from QA does a load test:“Application is slow as a snail”Task force is set upJohn tries to learn performance toolsSarah does a microbenchmark and gains 5 ms Application still slow Everybody unhappy
  • 18.
    How about…Developers careabout performanceGood tools are understood and usedPerformance is tested regularlyAnomalies are taken care of Application is running smoothly Everybody is happy
  • 19.
    Care About PerformanceCaringis fundamentally importantDevelopment teams need to extend their scopeIn Scrum teams need to be able to do all the work to get done
  • 20.
    Good ToolsTool paralysisdoes not helpChoose 1 or 2 good tools and learn themTools should work everywhereflickr.com/photos/pmtorrone/2381346935
  • 21.
    Continuously Test PerformanceFinda good balanceAutomated ChecksManual TestsFunctional tests already provide dataHow about a load test every iteration?
  • 22.
    Investigate Suspicious DataBecauseyou careAnd you have the toolsAnd you have the dataYou should investigate findings“When you have eliminated the impossible,whatever remains, however improbable,must be the truth” - Sherlock Holmesflickr.com/photos/cayusa/2159980025
  • 23.
    A real WorldExampleChapter Three
  • 24.
    Team Cares AboutPerformanceDefinition of Done includes a lotspec, design, unit test, code, acceptance tests, documentation, usability review, code review, stability tests, compatibility tests, interoperability test, load tests, security tests, performance tests…Get it right from the beginningDo not pile up technical debtTo go well you have to go slow
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
  • 33.
    7 Points deliveredTechnicalDebt / Undone WorkWrong Definition of DoneBetter Definition of DoneSprint 1
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.