Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Pitfalls in Performance Testing AxxesCC 06/2015


Published on

Zijn performantietesten noodzakelijk voor je project? Hoe begin je eraan? Welke kennis, tools en ervaring heb je nodig of moet je inhuren? Verschilt de aanpak voor een online applicatie van een backoffice applicatie? Wat gedaan als de performantie onvoldoende is? Wouter Soubry, Software Quality & Application Performance Consultant bij Axxes, maakt in deze presentatie komaf met al deze pitfalls bij Performance Testing.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Pitfalls in Performance Testing AxxesCC 06/2015

  1. 1. Click  to  edit  Master  /tle  style   Pi2alls  in  Performance  Tes/ng   June  2015   Wouter  Soubry   1  #AxxesCC  
  2. 2. Why  are  we  here?   2  #AxxesCC  
  3. 3. 3   Load  Tes/ng  Failures  –   #AxxesCC  
  4. 4. 4   Load  Tes/ng  Failures  –   #AxxesCC  
  5. 5.  Technical  point  of  view:    Single  user  performance   Acceptable  slowdown  under  load/stress   Not  only  websites!   Every  project  should  consider  performance   explicitly   5   What  is  (applica/on)  performance?   #AxxesCC  
  6. 6. hWp://     hWp://­‐US/app/search/speed_survey     6   Experiment   #AxxesCC  
  7. 7.  “The  site  needs  to  be  sufficiently  fast”    “Page  must  load  in  4s”    PDF  genera/on  should  not  take  >  30s    The  site  must  be  able  to  handle  1000  users   Trading  system  must  be  able  to  do  1000  TPS   7   Performance  Requirements   #AxxesCC  
  8. 8. Pi2all  1   Wrong,  ambiguous  or  non-­‐existent   performance  requirements   8  #AxxesCC  
  9. 9. Specific:  200  concurrent  users,  scenario   walkthrough,  4s  @  95  percen/le  response  all   pages  upon  first  /me  visit,  pc  on  dsl   Measurable:  at  the  client  side,  with  preload   Achievable:  ‘heavy’  page  in  200ms?    Relevant   Above  the  fold  load  /me    “Onload”  event  versus  speed  index    Time-­‐bound:  golive,  gradual  adop/on,   campaign  start   9   “SMART”  Performance  Requirements   #AxxesCC  
  10. 10.  Throughput,  load  envelope    Response  /me    User  behaviour  (think  /me,  flow)    Device,  connec/on    Context:  /cket  sale,  tax  deadline   10   Aspects  of  Performance  Reqs   #AxxesCC  
  11. 11. Staging   Produc/on   11  #AxxesCC  
  12. 12. Pi2all  2   Linear  extrapola/on   12  #AxxesCC  
  13. 13. Scalability  needs  to  be  built-­‐in  (and  tested)   What  is  the  uncertainty  of  your  expected   load?    Is  scalability  important?  How  fast  can  you   provision?    Business  impact  of  out  of  spec  performance?     13   Avoiding  Surprises   #AxxesCC  
  14. 14. 14   Invite  your  Friends?   #AxxesCC  
  15. 15. 15   Tooling   #AxxesCC  
  16. 16. 16   L’Embarras  du  Choix   ApacheBench   ?#AxxesCC  
  17. 17. 17   Tooling:  Cost  /  ROI   #AxxesCC   /  
  18. 18. Pi2all  3   A  fool  with  a  tool  is  s/ll  a  fool   (Grady  Booch)   18  #AxxesCC  
  19. 19.  Simple  >  complex    Understand  what  the  tool  is  doing   What  do  the  metrics  mean   What  is  measured  exactly    Tools  are  no  subs/tute  for  thinking,  but…   Good  repor/ng  helps   Intui/ve  interface++   Find  similar  use  cases  –  how  did  they  do  it?   19   Choose  Your  Weapon  Wisely   #AxxesCC  
  20. 20.  Simple  >  complex    Demo  with  ApacheBench  +  nmon   20   Choose  Your  Weapon  Wisely   #AxxesCC  
  21. 21. 21   Load  Test  Recording   #AxxesCC   Proxy   Interceptor   JMeter  recording  demo  
  22. 22. Generate  load   Measure  end-­‐user  perf    “RUM”/synthe/c    Transac/on  breakdown    Code-­‐level  profiling   Infrastructure  monitoring    Business  analy/cs   22   Tool  Categories   }  APM   #AxxesCC  
  23. 23. SaaS  ‘cloud’  load  genera/on   Blazemeter,  loadstorm,…    APM  suites      New  Relic   AppDynamics   Dynatrace    …    Big  (machine)  data   Splunk,  Loggly,  logentries,  Sumologic,  Logstash/elk    Monitoring  infrastructure   23   Tool  Convergence   #AxxesCC  
  24. 24.  Domain  knowledge    System  administra/on    Networking  exper/se    Database  adminstra/on    Applica/on  stack  knowledge:  frameworks    Tools  knowledge    Performance  tes/ng  knowhow   (+  some  sta/s/cs)   24   Needed  Skills   #AxxesCC  
  25. 25. 25  #AxxesCC  
  26. 26. Pi2all  4   Not  leveraging  in-­‐house  exper/se   26  #AxxesCC  
  27. 27.  Large  bank  online  security  overhaul    Compliance  log  subsystem   27   A  Tale  from  the  Trenches   #AxxesCC  
  28. 28. Pi2all  5   Premature  or  late  op/miza/on   28  #AxxesCC  
  29. 29. End-­‐user   performance   sufficient?   Transac/on   drilldown   Iden/fy  &   eliminate   boWleneck   29   Chasing  the  BoWleneck   #AxxesCC  
  30. 30.  Week  before  golive    Performance  tes/ng  completed,  green  light   Func/onal  freeze    Cri/cal  bugfixes  only   Launch  goes   30   Recipe  for  Disaster   #AxxesCC  
  31. 31. Pi2all  6   No  con/nuous  profiling   31  #AxxesCC  
  32. 32. Regression  tes/ng  performance    CI  tools:  Jenkins,  Atlassian  Bamboo,  TFS,  …    Webpagetest/    Single-­‐user  performance    Performance  budget    Speed  index    Demo:  Jmeter/Jenkins   32  #AxxesCC  
  33. 33.  Integra/on  environment   Not  produc/on  representa/ve   Regression  detec/on   Rule  out  startup  effects   Iden/fy  cri/cal  measurements    HTTP  APIs    DB  transac/ons    XML  processing   33   Jenkins  and  JMeter   #AxxesCC  
  34. 34. Pi2all  7   No  proper  configura/on   management   34  #AxxesCC  
  35. 35. Annota/on  overlays  on  metrics   hWp://     What  to  annotate:   Deploys   Config  changes   Anything  with  poten/al  performance  impact   Newrelic  deployments  demo   35   What  introduced  the  problem?   #AxxesCC  
  36. 36.  It’s  about  the  user.  The  User.    Tools:  use  what  you  know  best,   simpler  =  beWer    Monitoring:  develop  a  (healthy)  data  addic/on   Config  management  hygiene    Knowledge:  stand  on  the  shoulders  of  giants   36   Key  Takeaways   #AxxesCC   Thank  You