SlideShare a Scribd company logo
4/23/15	
  
1	
  
© Utopia Solutions
Performance	
  Tes3ng	
  in	
  the	
  Agile	
  Lifecycle	
  
Lee	
  Barnes,	
  CTO	
  
Utopia	
  Solu3ons	
  
@USI_LeeBarnes	
  
© Utopia Solutions2
Key Questions
What	
  needs	
  to	
  change?	
  
How	
  do	
  we	
  make	
  those	
  changes?	
  
Why	
  don’t	
  tradi3onal	
  performance	
  
tes3ng	
  models	
  fit	
  nicely	
  with	
  agile?	
  
4/23/15	
  
2	
  
© Utopia Solutions3
Case Study
Back	
  story…	
  
•  Large	
  global	
  manufacturer	
  
•  Performance	
  Tes3ng	
  CoE	
  in	
  opera3on	
  for	
  ~10	
  yrs	
  
•  Seen	
  as	
  a	
  huge	
  value	
  to	
  the	
  enterprise	
  
•  Began	
  moving	
  toward	
  agile	
  a	
  few	
  years	
  ago…	
  
and	
  the	
  luster	
  of	
  the	
  CoE	
  began	
  to	
  dull…	
  
© Utopia Solutions4
Traditional Performance Testing Model
Center	
  of	
  
Excellence	
  
Project	
  1	
  
Project	
  2	
   Project	
  
N	
  
Provides	
  Consistency	
  in…	
  
•  Resources	
  /	
  Skills	
  
•  Processes	
  
•  Deliverable	
  Quality	
  
•  Technology	
  
Request	
  Test	
  
Project	
  
Team	
  
Center	
  of	
  
Excellence	
  
Perform	
  Discovery	
  
Develop	
  &	
  Execute	
  
Deliver	
  Results	
  Evaluate	
  Results	
  
Info	
  /	
  Environment	
  
4/23/15	
  
3	
  
© Utopia Solutions5
CoE in the Waterfall Years
Plan	
   Design	
   Build	
   Test	
  	
   Deploy	
  
•  One	
  size	
  fits	
  all	
  
•  Strict	
  entry	
  criteria	
  
•  Performed	
  late	
  in	
  SDLC	
  
•  One-­‐way	
  communica3on	
  of	
  
results	
  
	
  
Service	
  Characteris?cs	
  
Performance	
  Tes3ng	
  
Too	
  late!	
  
© Utopia Solutions6
Agile Development	
  
Analyze	
  
Develop	
  
Integrate	
  
Acceptance	
  
Test	
  
Deploy	
  
Where	
  does	
  performance	
  
tes3ng	
  fit?	
  
4/23/15	
  
4	
  
© Utopia Solutions7
Early Attempts on Agile Projects
Sprint	
  1	
   Sprint	
  2	
   Sprint	
  3	
   …	
  	
   Deploy	
  
Agile	
  
Performance	
  Tes3ng	
  
Same	
  Process…	
  
Same	
  Risk!	
  
© Utopia Solutions8
Early Attempts on Agile Projects
Sprint	
  1	
   Sprint	
  2	
   Sprint	
  3	
   …	
  	
   Deploy	
  
Agile	
  
Performance	
  
Tes3ng	
  
Performance	
  
Tes3ng	
  
Performance	
  
Tes3ng	
  
Performance	
  
Tes3ng	
  
CoE	
  Approach	
  
•  Rigid	
  
•  4+	
  week	
  test	
  cycle	
  
Agile	
  Approach	
  
•  Flexible	
  
•  2	
  week	
  sprints	
  
+ =
4/23/15	
  
5	
  
© Utopia Solutions
Back	
  to	
  the	
  drawing	
  
board…	
  
© Utopia Solutions10
Early	
  Cycle	
  /	
  
Ongoing	
  
Performance	
  
Analysis	
  
Establish Objectives
Ul?mate	
  Goal…	
  
Late	
  Cycle	
  
Performance	
  
Tes?ng	
  	
  
ShiG	
  LeG	
  
4/23/15	
  
6	
  
© Utopia Solutions11
Identify Gaps
What’s	
  stopping	
  us	
  from	
  analyzing	
  performance	
  
con?nuously?	
  
•  Performance	
  objec3ves	
  weren’t	
  part	
  of	
  
user	
  stories	
  
•  Agile	
  teams	
  lacked	
  performance	
  
engineering	
  exper3se	
  
•  Performance	
  tes3ng	
  knowledge	
  and	
  tools	
  
were	
  locked	
  in	
  the	
  CoE	
  
•  APM	
  tools	
  /	
  skills	
  were	
  non-­‐existent	
  
© Utopia Solutions12
Breakdown the Barrier Between CoE and Agile Teams
4/23/15	
  
7	
  
© Utopia Solutions13
Disclaimers
It	
  was	
  not	
  easy.	
  
The	
  changes	
  discussed	
  on	
  the	
  next	
  few	
  
slides	
  were	
  implemented	
  over	
  a	
  two	
  
year	
  dura3on.	
  
It	
  was	
  not	
  painless.	
  
© Utopia Solutions14
Performance Testing in an Agile Environment
Build	
  performance	
  objec3ves	
  in	
  to	
  user	
  
stories	
  
Make	
  performance	
  everyone’s	
  job	
  
Create	
  a	
  role	
  to	
  coordinate	
  
performance	
  ac3vi3es	
  
CoE	
  performance	
  test	
  architects	
  
become	
  mentors	
  
Begin	
  developing	
  a	
  
performance	
  focused	
  
culture	
  
4/23/15	
  
8	
  
© Utopia Solutions15
Incorporating Performance Requirements…
…	
  as	
  acceptance	
  criteria	
  for	
  exis3ng	
  user	
  stories	
  
As	
  a	
  dealer	
  I	
  want	
  to	
  create	
  a	
  
product	
  configura3on	
  for	
  an	
  order	
  so	
  
that	
  my	
  customer	
  can	
  purchase	
  the	
  
equipment	
  they	
  need	
  
•  All	
  steps	
  in	
  the	
  product	
  config.	
  
process	
  will	
  have	
  a	
  response	
  3me	
  
of	
  less	
  than	
  2	
  seconds	
  
•  1,500	
  dealers	
  are	
  concurrently	
  
using	
  the	
  system	
  to	
  configure	
  	
  
products	
  for	
  customer	
  orders	
  
…	
  as	
  technical	
  stories	
  
As	
  a	
  dealer,	
  I	
  want	
  the	
  response	
  3me	
  of	
  any	
  step	
  in	
  the	
  product	
  
configura3on	
  process	
  to	
  be	
  less	
  than	
  2	
  seconds	
  
…	
  as	
  constraints	
  
The	
  system	
  will	
  respond	
  in	
  less	
  than	
  2	
  seconds	
  for	
  all	
  steps	
  in	
  the	
  
product	
  configura3on	
  process	
  for	
  a	
  maximum	
  of	
  1,500	
  concurrent	
  users	
  
configuring	
  orders	
  
© Utopia Solutions16
Performance Testing in an Agile Environment
Unit	
  performance	
  tes3ng	
  
Component	
  /	
  system	
  level	
  performance	
  
tes3ng	
  
Include	
  performance	
  tes3ng	
  in	
  the	
  CI	
  
chain	
  
Incorporate	
  performance	
  
tes3ng	
  /	
  analysis	
  into	
  
each	
  sprint	
  
4/23/15	
  
9	
  
© Utopia Solutions17
Performance Unit Testing
Start	
  with	
  exis3ng	
  unit	
  tests	
  
“Decorate”	
  them	
  with	
  
3mers	
  and	
  load	
  
Monitor	
  results	
  across	
  
builds	
  
Build	
  1	
   Build	
  2	
   Build	
  3	
   Build	
  4	
  
© Utopia Solutions18
Beyond Unit Testing
Map	
  user	
  stories	
  onto	
  system	
  
components	
  
Benchmark	
  in	
  development	
  /	
  QA	
  
environment	
  
Look	
  for	
  trends	
  in	
  measurements	
  vs.	
  
absolutes	
  
Use	
  service	
  virtualiza3on	
  for	
  3rd	
  party	
  
or	
  yet	
  to	
  be	
  developed	
  components	
  
Incorporate	
  as	
  part	
  of	
  the	
  CI	
  chain	
  
Harden	
  scripts	
  against	
  applica3on	
  
changes	
  
Component	
  and	
  
system	
  level	
  tes3ng	
  
4/23/15	
  
10	
  
© Utopia Solutions19
Adding Performance Tests to CI
Custom	
  Maven	
  plug-­‐ins	
  exist	
  to	
  run	
  
JMeter	
  (and	
  other	
  tools)	
  performance	
  
tests	
  –	
  very	
  useful	
  
People	
  get	
  mad	
  when	
  you	
  
unnecessarily	
  break	
  the	
  build	
  
Set	
  your	
  response	
  3me	
  thresholds	
  
and	
  client	
  3meouts	
  accordingly	
  	
  
They	
  get	
  really	
  mad	
  when	
  you	
  can’t	
  
figure	
  out	
  why	
  you	
  broke	
  the	
  build	
  
Add	
  error	
  handling	
  and	
  logging	
  to	
  
scripts	
  to	
  enable	
  quick	
  failure	
  
inves3ga3on	
  
Lessons	
  Learned	
  
© Utopia Solutions20
Waterfall	
   Agile	
  
Responsibility	
  for	
  
Performance	
  
Performance	
  CoE	
  
Role	
  of	
  CoE	
   End	
  of	
  cycle	
  PT	
   	
  
	
  
Role	
  of	
  
development	
  
Throw	
  system	
  over	
  wall	
  
Wait	
  for	
  results	
  
Tools	
   Tradi3onal	
  load	
  test	
  tools	
   	
  
	
  
	
  
	
  
Environment	
   Dedicated	
  /	
  produc3on-­‐like	
  
Deliverable	
   Boiler	
  plate	
  results	
  delivered	
  
just	
  prior	
  to	
  release	
  
Waterfall	
   Agile	
  
Responsibility	
  
for	
  Performance	
  
Performance	
  CoE	
   Everyone	
  
Role	
  of	
  CoE	
   End	
  of	
  cycle	
  PT	
   Provide	
  mentorship	
  to	
  agile	
  team	
  
Pre-­‐deployment	
  system	
  PT	
  
Role	
  of	
  
development	
  
Throw	
  system	
  over	
  wall	
  
Wait	
  for	
  results	
  
Con3nuous	
  focus	
  on	
  performance	
  
Tools	
   Tradi3onal	
  load	
  test	
  tools	
   APM	
  tools	
  
Service	
  virtualiza3on	
  
Con3nuous	
  integra3on	
  
Tradi3onal	
  load	
  test	
  tools	
  
Environment	
   Dedicated	
  /	
  produc3on-­‐like	
   Development	
  /	
  desktop	
  
Deliverable	
   Boiler	
  plate	
  results	
  delivered	
  
just	
  prior	
  to	
  release	
  
Con3nuous	
  feed	
  of	
  ac3onable	
  
data	
  
Performance Testing in an Agile Environment
Transforma?on	
  Summary	
  
4/23/15	
  
11	
  
© Utopia Solutions21
Summary
Key	
  takeaways…	
  
•  Agile	
  performance	
  tes3ng	
  is	
  not	
  
more	
  /	
  quicker	
  performance	
  tests	
  	
  	
  	
  
it’s	
  a	
  con3nuous	
  focus	
  on	
  
performance	
  
•  Performance	
  is	
  part	
  of	
  everyone’s	
  job	
  
•  Success	
  requires	
  a	
  shio	
  in	
  people,	
  
process	
  and	
  technology	
  
© Utopia Solutions22
Questions…
…. and Answers!
Direct future questions to:
Lee Barnes
Founder and CTO
Utopia Solutions, Inc.
Email: lee.barnes@utopiasolutions.com
Twitter: twitter.com/USI_LeeBarnes
LinkedIn: linkedin.com/in/leebarnes
Blog: www.RaiseYourMQ.com
?
4/23/15	
  
12	
  
© Utopia Solutions
Links to Unit Performance Testing Frameworks
•  JUnitPerf	
  –	
  hpp://www.clarkware.com/sooware/JUnitPerf.html	
  
•  P-­‐unit	
  –	
  hpp://p-­‐unit.sourceforge.net/	
  

More Related Content

What's hot

A Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps MetamorphosisA Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps Metamorphosis
XebiaLabs
 
Pooja shift left 1.0
Pooja shift left 1.0Pooja shift left 1.0
Pooja shift left 1.0
Xebia India
 
Rapid Strategic SRE Assessments
Rapid Strategic SRE AssessmentsRapid Strategic SRE Assessments
Rapid Strategic SRE Assessments
Marc Hornbeek
 
DevOps Maturity Curve v5
DevOps Maturity Curve v5DevOps Maturity Curve v5
DevOps Maturity Curve v5
Paul Peissner
 
10 Steps to Shift Left - Guiding Principles in QA
10 Steps to Shift Left - Guiding Principles in QA10 Steps to Shift Left - Guiding Principles in QA
10 Steps to Shift Left - Guiding Principles in QA
Jim Spillson
 
Addressing Performance Testing Challenges in Agile- Impetus Webinar
Addressing Performance Testing Challenges in Agile- Impetus WebinarAddressing Performance Testing Challenges in Agile- Impetus Webinar
Addressing Performance Testing Challenges in Agile- Impetus Webinar
Impetus Technologies
 
Agile Aspects of Performance Testing
Agile Aspects of Performance TestingAgile Aspects of Performance Testing
Agile Aspects of Performance Testing
Alexander Podelko
 
DevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation SlidesDevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation Slides
SlideTeam
 
Dev ops != Dev+Ops
Dev ops != Dev+OpsDev ops != Dev+Ops
Dev ops != Dev+Ops
Shalu Ahuja
 
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
ITSM Academy, Inc.
 
Exploratory testing and Dev Ops - best friends?
Exploratory testing and Dev Ops - best friends?Exploratory testing and Dev Ops - best friends?
Exploratory testing and Dev Ops - best friends?
Sven Schirmer
 
Relieveing the Testing Bottle Neck - Webinar
Relieveing the Testing Bottle Neck - WebinarRelieveing the Testing Bottle Neck - Webinar
Relieveing the Testing Bottle Neck - Webinar
Cprime
 
Continuous Quality: What DevOps Means for QA
Continuous Quality: What DevOps Means for QAContinuous Quality: What DevOps Means for QA
Continuous Quality: What DevOps Means for QAJeff Sussna
 
Contributors for Delivering a Successful Testing Project Seminar
Contributors for Delivering a Successful Testing Project SeminarContributors for Delivering a Successful Testing Project Seminar
Contributors for Delivering a Successful Testing Project Seminar
KMS Technology
 
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
Gene Kim
 
What business benefits from DevOps 2014
What business benefits from DevOps 2014What business benefits from DevOps 2014
What business benefits from DevOps 2014
Erno Aapa
 
Agile Test Automation
Agile Test AutomationAgile Test Automation
Agile Test Automation
Werner Keil
 
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | EdurekaDevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
Edureka!
 
Test Automation is for Everyone
Test Automation is for EveryoneTest Automation is for Everyone
Test Automation is for Everyone
Worksoft
 

What's hot (20)

A Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps MetamorphosisA Blueprint for a Successful DevOps Metamorphosis
A Blueprint for a Successful DevOps Metamorphosis
 
Pooja shift left 1.0
Pooja shift left 1.0Pooja shift left 1.0
Pooja shift left 1.0
 
Rapid Strategic SRE Assessments
Rapid Strategic SRE AssessmentsRapid Strategic SRE Assessments
Rapid Strategic SRE Assessments
 
DevOps Maturity Curve v5
DevOps Maturity Curve v5DevOps Maturity Curve v5
DevOps Maturity Curve v5
 
10 Steps to Shift Left - Guiding Principles in QA
10 Steps to Shift Left - Guiding Principles in QA10 Steps to Shift Left - Guiding Principles in QA
10 Steps to Shift Left - Guiding Principles in QA
 
Addressing Performance Testing Challenges in Agile- Impetus Webinar
Addressing Performance Testing Challenges in Agile- Impetus WebinarAddressing Performance Testing Challenges in Agile- Impetus Webinar
Addressing Performance Testing Challenges in Agile- Impetus Webinar
 
Agile Aspects of Performance Testing
Agile Aspects of Performance TestingAgile Aspects of Performance Testing
Agile Aspects of Performance Testing
 
DevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation SlidesDevOps Powerpoint Presentation Slides
DevOps Powerpoint Presentation Slides
 
DevOps-CoE
DevOps-CoEDevOps-CoE
DevOps-CoE
 
Dev ops != Dev+Ops
Dev ops != Dev+OpsDev ops != Dev+Ops
Dev ops != Dev+Ops
 
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
DevOps Test Engineering: Putting the ‘Continuous’ in Testing, an ITSM Academy...
 
Exploratory testing and Dev Ops - best friends?
Exploratory testing and Dev Ops - best friends?Exploratory testing and Dev Ops - best friends?
Exploratory testing and Dev Ops - best friends?
 
Relieveing the Testing Bottle Neck - Webinar
Relieveing the Testing Bottle Neck - WebinarRelieveing the Testing Bottle Neck - Webinar
Relieveing the Testing Bottle Neck - Webinar
 
Continuous Quality: What DevOps Means for QA
Continuous Quality: What DevOps Means for QAContinuous Quality: What DevOps Means for QA
Continuous Quality: What DevOps Means for QA
 
Contributors for Delivering a Successful Testing Project Seminar
Contributors for Delivering a Successful Testing Project SeminarContributors for Delivering a Successful Testing Project Seminar
Contributors for Delivering a Successful Testing Project Seminar
 
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
DOES15 - Sherry Chang - Intel’s Journey to Large Scale DevOps Transformation
 
What business benefits from DevOps 2014
What business benefits from DevOps 2014What business benefits from DevOps 2014
What business benefits from DevOps 2014
 
Agile Test Automation
Agile Test AutomationAgile Test Automation
Agile Test Automation
 
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | EdurekaDevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
DevOps vs Agile | DevOps Tutorial For Beginners | DevOps Training | Edureka
 
Test Automation is for Everyone
Test Automation is for EveryoneTest Automation is for Everyone
Test Automation is for Everyone
 

Similar to Performance Testing in the Agile Lifecycle

Performance Testing in the Agile Lifecycle
Performance Testing in the Agile LifecyclePerformance Testing in the Agile Lifecycle
Performance Testing in the Agile Lifecycle
Lee Barnes
 
Agile Load Testing in the Real World
Agile Load Testing in the Real WorldAgile Load Testing in the Real World
Agile Load Testing in the Real World
Lee Barnes
 
Agile Load Testing In The Real World
Agile Load Testing In The Real WorldAgile Load Testing In The Real World
Agile Load Testing In The Real World
SOASTA
 
Agile Load Testing In The Real World
Agile Load Testing In The Real WorldAgile Load Testing In The Real World
Agile Load Testing In The Real World
SOASTA
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
Fabricio Epaminondas
 
Continuous Performance Testing in DevOps - Lee Barnes
Continuous Performance Testing in DevOps - Lee BarnesContinuous Performance Testing in DevOps - Lee Barnes
Continuous Performance Testing in DevOps - Lee Barnes
QA or the Highway
 
Performance Testing in Agile Process
Performance Testing in Agile ProcessPerformance Testing in Agile Process
Performance Testing in Agile Process
Idexcel Technologies
 
How To Transform the Manual Testing Process to Incorporate Test Automation
How To Transform the Manual Testing Process to Incorporate Test AutomationHow To Transform the Manual Testing Process to Incorporate Test Automation
How To Transform the Manual Testing Process to Incorporate Test Automation
Ranorex
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOps
Perfecto Mobile
 
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
Roberto Pérez Alcolea
 
Incorporating Performance Testing in Agile Development Process
Incorporating Performance Testing in Agile Development ProcessIncorporating Performance Testing in Agile Development Process
Incorporating Performance Testing in Agile Development Process
Michael Vax
 
Experimentation at Blue Apron (webinar)
Experimentation at Blue Apron (webinar)Experimentation at Blue Apron (webinar)
Experimentation at Blue Apron (webinar)
Optimizely
 
Test Automation on Large Agile Projects: It's Not a Cakewalk
Test Automation on Large Agile Projects: It's Not a CakewalkTest Automation on Large Agile Projects: It's Not a Cakewalk
Test Automation on Large Agile Projects: It's Not a Cakewalk
TechWell
 
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Emerasoft, solutions to collaborate
 
Agile methodologiesvswaterfall
Agile methodologiesvswaterfallAgile methodologiesvswaterfall
Agile methodologiesvswaterfall
Muthu Natarajan
 
How to build confidence in your release cycle
How to build confidence in your release cycleHow to build confidence in your release cycle
How to build confidence in your release cycle
DiUS
 
How to Deliver Winning Mobile Apps
How to Deliver Winning Mobile AppsHow to Deliver Winning Mobile Apps
How to Deliver Winning Mobile Apps
TechWell
 
Test Process in Agile vs Waterfall
Test Process in Agile vs WaterfallTest Process in Agile vs Waterfall
Test Process in Agile vs Waterfall
Return on Intelligence
 
CoreyCPayneOctober2016
CoreyCPayneOctober2016CoreyCPayneOctober2016
CoreyCPayneOctober2016Corey Payne
 
[Customizable Template] How to Get Stakeholder Buy-In for a Toolchain Integra...
[Customizable Template] How to Get Stakeholder Buy-In for a Toolchain Integra...[Customizable Template] How to Get Stakeholder Buy-In for a Toolchain Integra...
[Customizable Template] How to Get Stakeholder Buy-In for a Toolchain Integra...
Planview
 

Similar to Performance Testing in the Agile Lifecycle (20)

Performance Testing in the Agile Lifecycle
Performance Testing in the Agile LifecyclePerformance Testing in the Agile Lifecycle
Performance Testing in the Agile Lifecycle
 
Agile Load Testing in the Real World
Agile Load Testing in the Real WorldAgile Load Testing in the Real World
Agile Load Testing in the Real World
 
Agile Load Testing In The Real World
Agile Load Testing In The Real WorldAgile Load Testing In The Real World
Agile Load Testing In The Real World
 
Agile Load Testing In The Real World
Agile Load Testing In The Real WorldAgile Load Testing In The Real World
Agile Load Testing In The Real World
 
Continuous testing in agile projects 2015
Continuous testing in agile projects 2015Continuous testing in agile projects 2015
Continuous testing in agile projects 2015
 
Continuous Performance Testing in DevOps - Lee Barnes
Continuous Performance Testing in DevOps - Lee BarnesContinuous Performance Testing in DevOps - Lee Barnes
Continuous Performance Testing in DevOps - Lee Barnes
 
Performance Testing in Agile Process
Performance Testing in Agile ProcessPerformance Testing in Agile Process
Performance Testing in Agile Process
 
How To Transform the Manual Testing Process to Incorporate Test Automation
How To Transform the Manual Testing Process to Incorporate Test AutomationHow To Transform the Manual Testing Process to Incorporate Test Automation
How To Transform the Manual Testing Process to Incorporate Test Automation
 
Shorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOpsShorten Business Life Cycle Using DevOps
Shorten Business Life Cycle Using DevOps
 
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
[DPE Summit] How Improving the Testing Experience Goes Beyond Quality: A Deve...
 
Incorporating Performance Testing in Agile Development Process
Incorporating Performance Testing in Agile Development ProcessIncorporating Performance Testing in Agile Development Process
Incorporating Performance Testing in Agile Development Process
 
Experimentation at Blue Apron (webinar)
Experimentation at Blue Apron (webinar)Experimentation at Blue Apron (webinar)
Experimentation at Blue Apron (webinar)
 
Test Automation on Large Agile Projects: It's Not a Cakewalk
Test Automation on Large Agile Projects: It's Not a CakewalkTest Automation on Large Agile Projects: It's Not a Cakewalk
Test Automation on Large Agile Projects: It's Not a Cakewalk
 
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
Webinar: "5 semplici passi per migliorare la Quality e i processi di Test".
 
Agile methodologiesvswaterfall
Agile methodologiesvswaterfallAgile methodologiesvswaterfall
Agile methodologiesvswaterfall
 
How to build confidence in your release cycle
How to build confidence in your release cycleHow to build confidence in your release cycle
How to build confidence in your release cycle
 
How to Deliver Winning Mobile Apps
How to Deliver Winning Mobile AppsHow to Deliver Winning Mobile Apps
How to Deliver Winning Mobile Apps
 
Test Process in Agile vs Waterfall
Test Process in Agile vs WaterfallTest Process in Agile vs Waterfall
Test Process in Agile vs Waterfall
 
CoreyCPayneOctober2016
CoreyCPayneOctober2016CoreyCPayneOctober2016
CoreyCPayneOctober2016
 
[Customizable Template] How to Get Stakeholder Buy-In for a Toolchain Integra...
[Customizable Template] How to Get Stakeholder Buy-In for a Toolchain Integra...[Customizable Template] How to Get Stakeholder Buy-In for a Toolchain Integra...
[Customizable Template] How to Get Stakeholder Buy-In for a Toolchain Integra...
 

More from TechWell

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
TechWell
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
TechWell
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
TechWell
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
TechWell
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
TechWell
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
TechWell
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
TechWell
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
TechWell
 
Ma 15
Ma 15Ma 15
Ma 15
TechWell
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
TechWell
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
TechWell
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
TechWell
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
TechWell
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
TechWell
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
TechWell
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
TechWell
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
TechWell
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
TechWell
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
TechWell
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
TechWell
 

More from TechWell (20)

Failing and Recovering
Failing and RecoveringFailing and Recovering
Failing and Recovering
 
Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization Instill a DevOps Testing Culture in Your Team and Organization
Instill a DevOps Testing Culture in Your Team and Organization
 
Test Design for Fully Automated Build Architecture
Test Design for Fully Automated Build ArchitectureTest Design for Fully Automated Build Architecture
Test Design for Fully Automated Build Architecture
 
System-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good StartSystem-Level Test Automation: Ensuring a Good Start
System-Level Test Automation: Ensuring a Good Start
 
Build Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test StrategyBuild Your Mobile App Quality and Test Strategy
Build Your Mobile App Quality and Test Strategy
 
Testing Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for SuccessTesting Transformation: The Art and Science for Success
Testing Transformation: The Art and Science for Success
 
Implement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlowImplement BDD with Cucumber and SpecFlow
Implement BDD with Cucumber and SpecFlow
 
Develop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your SanityDevelop WebDriver Automated Tests—and Keep Your Sanity
Develop WebDriver Automated Tests—and Keep Your Sanity
 
Ma 15
Ma 15Ma 15
Ma 15
 
Eliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps StrategyEliminate Cloud Waste with a Holistic DevOps Strategy
Eliminate Cloud Waste with a Holistic DevOps Strategy
 
Transform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOpsTransform Test Organizations for the New World of DevOps
Transform Test Organizations for the New World of DevOps
 
The Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—LeadershipThe Fourth Constraint in Project Delivery—Leadership
The Fourth Constraint in Project Delivery—Leadership
 
Resolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile TeamsResolve the Contradiction of Specialists within Agile Teams
Resolve the Contradiction of Specialists within Agile Teams
 
Pin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile GamePin the Tail on the Metric: A Field-Tested Agile Game
Pin the Tail on the Metric: A Field-Tested Agile Game
 
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile TeamsAgile Performance Holarchy (APH)—A Model for Scaling Agile Teams
Agile Performance Holarchy (APH)—A Model for Scaling Agile Teams
 
A Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps ImplementationA Business-First Approach to DevOps Implementation
A Business-First Approach to DevOps Implementation
 
Databases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery ProcessDatabases in a Continuous Integration/Delivery Process
Databases in a Continuous Integration/Delivery Process
 
Mobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to AutomateMobile Testing: What—and What Not—to Automate
Mobile Testing: What—and What Not—to Automate
 
Cultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for SuccessCultural Intelligence: A Key Skill for Success
Cultural Intelligence: A Key Skill for Success
 
Turn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile TransformationTurn the Lights On: A Power Utility Company's Agile Transformation
Turn the Lights On: A Power Utility Company's Agile Transformation
 

Recently uploaded

Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
AMB-Review
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Natan Silnitsky
 
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
Hivelance Technology
 
Lecture 1 Introduction to games development
Lecture 1 Introduction to games developmentLecture 1 Introduction to games development
Lecture 1 Introduction to games development
abdulrafaychaudhry
 
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
informapgpstrackings
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web Services
KrzysztofKkol1
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
Globus
 
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
XfilesPro
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
Georgi Kodinov
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
Globus
 
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamOpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
takuyayamamoto1800
 
Why React Native as a Strategic Advantage for Startup Innovation.pdf
Why React Native as a Strategic Advantage for Startup Innovation.pdfWhy React Native as a Strategic Advantage for Startup Innovation.pdf
Why React Native as a Strategic Advantage for Startup Innovation.pdf
ayushiqss
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2
 
Advanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should KnowAdvanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should Know
Peter Caitens
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
IES VE
 
Vitthal Shirke Microservices Resume Montevideo
Vitthal Shirke Microservices Resume MontevideoVitthal Shirke Microservices Resume Montevideo
Vitthal Shirke Microservices Resume Montevideo
Vitthal Shirke
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Globus
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
Max Andersen
 
Software Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdfSoftware Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdf
MayankTawar1
 

Recently uploaded (20)

Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
 
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
Multiple Your Crypto Portfolio with the Innovative Features of Advanced Crypt...
 
Lecture 1 Introduction to games development
Lecture 1 Introduction to games developmentLecture 1 Introduction to games development
Lecture 1 Introduction to games development
 
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web Services
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
 
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
How Does XfilesPro Ensure Security While Sharing Documents in Salesforce?
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
 
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamOpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoam
 
Why React Native as a Strategic Advantage for Startup Innovation.pdf
Why React Native as a Strategic Advantage for Startup Innovation.pdfWhy React Native as a Strategic Advantage for Startup Innovation.pdf
Why React Native as a Strategic Advantage for Startup Innovation.pdf
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
Advanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should KnowAdvanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should Know
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
 
Vitthal Shirke Microservices Resume Montevideo
Vitthal Shirke Microservices Resume MontevideoVitthal Shirke Microservices Resume Montevideo
Vitthal Shirke Microservices Resume Montevideo
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
 
Software Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdfSoftware Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdf
 

Performance Testing in the Agile Lifecycle

  • 1. 4/23/15   1   © Utopia Solutions Performance  Tes3ng  in  the  Agile  Lifecycle   Lee  Barnes,  CTO   Utopia  Solu3ons   @USI_LeeBarnes   © Utopia Solutions2 Key Questions What  needs  to  change?   How  do  we  make  those  changes?   Why  don’t  tradi3onal  performance   tes3ng  models  fit  nicely  with  agile?  
  • 2. 4/23/15   2   © Utopia Solutions3 Case Study Back  story…   •  Large  global  manufacturer   •  Performance  Tes3ng  CoE  in  opera3on  for  ~10  yrs   •  Seen  as  a  huge  value  to  the  enterprise   •  Began  moving  toward  agile  a  few  years  ago…   and  the  luster  of  the  CoE  began  to  dull…   © Utopia Solutions4 Traditional Performance Testing Model Center  of   Excellence   Project  1   Project  2   Project   N   Provides  Consistency  in…   •  Resources  /  Skills   •  Processes   •  Deliverable  Quality   •  Technology   Request  Test   Project   Team   Center  of   Excellence   Perform  Discovery   Develop  &  Execute   Deliver  Results  Evaluate  Results   Info  /  Environment  
  • 3. 4/23/15   3   © Utopia Solutions5 CoE in the Waterfall Years Plan   Design   Build   Test     Deploy   •  One  size  fits  all   •  Strict  entry  criteria   •  Performed  late  in  SDLC   •  One-­‐way  communica3on  of   results     Service  Characteris?cs   Performance  Tes3ng   Too  late!   © Utopia Solutions6 Agile Development   Analyze   Develop   Integrate   Acceptance   Test   Deploy   Where  does  performance   tes3ng  fit?  
  • 4. 4/23/15   4   © Utopia Solutions7 Early Attempts on Agile Projects Sprint  1   Sprint  2   Sprint  3   …     Deploy   Agile   Performance  Tes3ng   Same  Process…   Same  Risk!   © Utopia Solutions8 Early Attempts on Agile Projects Sprint  1   Sprint  2   Sprint  3   …     Deploy   Agile   Performance   Tes3ng   Performance   Tes3ng   Performance   Tes3ng   Performance   Tes3ng   CoE  Approach   •  Rigid   •  4+  week  test  cycle   Agile  Approach   •  Flexible   •  2  week  sprints   + =
  • 5. 4/23/15   5   © Utopia Solutions Back  to  the  drawing   board…   © Utopia Solutions10 Early  Cycle  /   Ongoing   Performance   Analysis   Establish Objectives Ul?mate  Goal…   Late  Cycle   Performance   Tes?ng     ShiG  LeG  
  • 6. 4/23/15   6   © Utopia Solutions11 Identify Gaps What’s  stopping  us  from  analyzing  performance   con?nuously?   •  Performance  objec3ves  weren’t  part  of   user  stories   •  Agile  teams  lacked  performance   engineering  exper3se   •  Performance  tes3ng  knowledge  and  tools   were  locked  in  the  CoE   •  APM  tools  /  skills  were  non-­‐existent   © Utopia Solutions12 Breakdown the Barrier Between CoE and Agile Teams
  • 7. 4/23/15   7   © Utopia Solutions13 Disclaimers It  was  not  easy.   The  changes  discussed  on  the  next  few   slides  were  implemented  over  a  two   year  dura3on.   It  was  not  painless.   © Utopia Solutions14 Performance Testing in an Agile Environment Build  performance  objec3ves  in  to  user   stories   Make  performance  everyone’s  job   Create  a  role  to  coordinate   performance  ac3vi3es   CoE  performance  test  architects   become  mentors   Begin  developing  a   performance  focused   culture  
  • 8. 4/23/15   8   © Utopia Solutions15 Incorporating Performance Requirements… …  as  acceptance  criteria  for  exis3ng  user  stories   As  a  dealer  I  want  to  create  a   product  configura3on  for  an  order  so   that  my  customer  can  purchase  the   equipment  they  need   •  All  steps  in  the  product  config.   process  will  have  a  response  3me   of  less  than  2  seconds   •  1,500  dealers  are  concurrently   using  the  system  to  configure     products  for  customer  orders   …  as  technical  stories   As  a  dealer,  I  want  the  response  3me  of  any  step  in  the  product   configura3on  process  to  be  less  than  2  seconds   …  as  constraints   The  system  will  respond  in  less  than  2  seconds  for  all  steps  in  the   product  configura3on  process  for  a  maximum  of  1,500  concurrent  users   configuring  orders   © Utopia Solutions16 Performance Testing in an Agile Environment Unit  performance  tes3ng   Component  /  system  level  performance   tes3ng   Include  performance  tes3ng  in  the  CI   chain   Incorporate  performance   tes3ng  /  analysis  into   each  sprint  
  • 9. 4/23/15   9   © Utopia Solutions17 Performance Unit Testing Start  with  exis3ng  unit  tests   “Decorate”  them  with   3mers  and  load   Monitor  results  across   builds   Build  1   Build  2   Build  3   Build  4   © Utopia Solutions18 Beyond Unit Testing Map  user  stories  onto  system   components   Benchmark  in  development  /  QA   environment   Look  for  trends  in  measurements  vs.   absolutes   Use  service  virtualiza3on  for  3rd  party   or  yet  to  be  developed  components   Incorporate  as  part  of  the  CI  chain   Harden  scripts  against  applica3on   changes   Component  and   system  level  tes3ng  
  • 10. 4/23/15   10   © Utopia Solutions19 Adding Performance Tests to CI Custom  Maven  plug-­‐ins  exist  to  run   JMeter  (and  other  tools)  performance   tests  –  very  useful   People  get  mad  when  you   unnecessarily  break  the  build   Set  your  response  3me  thresholds   and  client  3meouts  accordingly     They  get  really  mad  when  you  can’t   figure  out  why  you  broke  the  build   Add  error  handling  and  logging  to   scripts  to  enable  quick  failure   inves3ga3on   Lessons  Learned   © Utopia Solutions20 Waterfall   Agile   Responsibility  for   Performance   Performance  CoE   Role  of  CoE   End  of  cycle  PT       Role  of   development   Throw  system  over  wall   Wait  for  results   Tools   Tradi3onal  load  test  tools           Environment   Dedicated  /  produc3on-­‐like   Deliverable   Boiler  plate  results  delivered   just  prior  to  release   Waterfall   Agile   Responsibility   for  Performance   Performance  CoE   Everyone   Role  of  CoE   End  of  cycle  PT   Provide  mentorship  to  agile  team   Pre-­‐deployment  system  PT   Role  of   development   Throw  system  over  wall   Wait  for  results   Con3nuous  focus  on  performance   Tools   Tradi3onal  load  test  tools   APM  tools   Service  virtualiza3on   Con3nuous  integra3on   Tradi3onal  load  test  tools   Environment   Dedicated  /  produc3on-­‐like   Development  /  desktop   Deliverable   Boiler  plate  results  delivered   just  prior  to  release   Con3nuous  feed  of  ac3onable   data   Performance Testing in an Agile Environment Transforma?on  Summary  
  • 11. 4/23/15   11   © Utopia Solutions21 Summary Key  takeaways…   •  Agile  performance  tes3ng  is  not   more  /  quicker  performance  tests         it’s  a  con3nuous  focus  on   performance   •  Performance  is  part  of  everyone’s  job   •  Success  requires  a  shio  in  people,   process  and  technology   © Utopia Solutions22 Questions… …. and Answers! Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: lee.barnes@utopiasolutions.com Twitter: twitter.com/USI_LeeBarnes LinkedIn: linkedin.com/in/leebarnes Blog: www.RaiseYourMQ.com ?
  • 12. 4/23/15   12   © Utopia Solutions Links to Unit Performance Testing Frameworks •  JUnitPerf  –  hpp://www.clarkware.com/sooware/JUnitPerf.html   •  P-­‐unit  –  hpp://p-­‐unit.sourceforge.net/