SlideShare a Scribd company logo
Performance testing and reporting with JMeter Introduced by: Nghi Nguyen Van ~ March, 1 st  2012 ~
Agenda Introduction What is performance testing, why is it important Performance testing and monitoring tools Performance testing with JMeter What to focus in a performance test report Tips and tricks Q & A
Purpose of this presentation: Introduce about performance testing activity in eXo SEA Share experience in do performance testing Share experience in working with Jmeter This presentation serves to: ●   Leaders to understand about performance testing activity, reports ●   Who is new in using Jmeter to gain knowledge about using Jmeter ●   Who experienced in Jmeter want to gain, share & discuss about using Jmeter ●   Testers who want to understand about building, using performance testing system Introduction
What is performance testing, why is it important?
Definition: Performance testing is a type of testing intended to determine the responsiveness throughput reliability and/or scalability of a system under a given workload What is performance testing
Performance testing is commonly conducted to accomplish the following: Assess production readiness Evaluate against performance criteria Compare performance characteristics of multiple systems or system configurations Find the source of performance problems Support system tuning Find throughput levels What is performance testing...
Performance test Load test Stress test Capacity test Types of Performance Testing
Application performance impacted by various of factors Last minute surprises are always bad Applications made up from hundred/thousand of components Components change day by day Performance issues are hard to fix Why is it so important?
Performance testing and monitoring tools
Imagine about a performance testing system This system must provide information about: Availability/Stability Response time Throughput Utilization
Can provide data: Availability/Stability Response time Throughput Commercial tools: LoadRunner (HP) IBM Rational Performance Tester (IBM) Visual Studio Load Test (Microsoft) Opensource tools: Apache JMeter (Apache Jakarta) The grinder OpenSTA (Open System Testing Architecture)  Needed tools - Virtual users simulators
Data can provide: Percent of CPU utilization Amount of free memory Page file utilization Disk time Amount of free disk space Tools: OpenNMS Hyperic HQ Zabbix Perfmon (Windows) Needed tools – System monitoring
Can provide data: Detailed information about the application resource used Amount of used memory A mount of threads Openning files Monitoring tools: JvisualVM/VisualVM Jmxtrans + graphite Jprofiler HypericHQ Jmanage Javamelody Needed tools – application monitoring
A real performance testing system Application monitoring tool: jmxtrans, graphite Application measuring tool: JMeter System monitoring tool: OpenNMS
Performance testing with JMeter
The Apache JMeter™ desktop application is: -  open source software -  100% pure Java application  designed to load test functional behavior and measure performance -  designed for testing Web Applicatio n JMeter features: -  Can load and performance test many  different server types -  Complete portability  and 100% Java purity -  Full  multithreading  framework  -  Careful GUI design  -  Caching and offline analysis/replaying of  test results. -  Highly Extensible Apache JMeter introduction
+) Easy to use for both developer and tester record/modify  with GUI understand & modify under text mode install  the tool and  distribute  test scripts +) Good enough Rich elements that help  doing performance test easily  (especially web apps) Stable Good for both  simple and advance  test configuration Good with  CI/automation   test  model Extendable via plugins +) Opensource +-) JMeter is a great tool but poor document Why JMeter is chosen/still be in used
Response time 90% line of response time  (90 th  percentile of response time)  90% of all the response time values less or equal than this value) Average response time Simple average of all values=(sum of all values from 1 st   to Nth)/N Min and Max response time Error rate Throughput Performance testing notices – Basic parameters
Performance testing notices – Most reliable response time Most reliable response time = 90% line of response time  (90 th  percentile of response time)!
Performance testing tips - Do stability test Purpose Make sure that the test is reliable Gain better knowledge about application/test Give/confirm later tests' settings: Test duration Test cycles …
Performance testing tips - Do stability test How Choose a simple test Repeat the test for many times (e.g: >=6 times) Calculate difference from result of each test to their average Apply the performance test statements
JMeter demo & notes
Demo notices – Jenkins flow Jenkins flows: JMeter benchmark process Perf-suite Loop until end of setting file Various of test settings Easy to maintain/extend
Demo notices – JMeter script Introduce JMeter script How is a script used in a benchmark flow with jenkins Scripts: PLF_PERF_05_Login.jmx PLF_PERF_04_SocialRead-3.0.x.jmx PLF_PERF_03_ECMS-Public.jmx
JMeter notices – recording/modifying script Use selenium script -> this is for QA who have to repeat the scenario  oftenly Save and  backup  the first recorded version, backup after every importance change Find and replace all the  changeable values  by Jmeter variables such as host address port number date and time account used in the scenario Id of components HTTP response assertions are  needed to avoid something wrong
JMeter notices – script organization.p1
JMeter notices – script organization.p2 Test plan interface Jmeter properties usage can be used to make up an interface for a testscript Properties' value should be assigned to runtime variables at user defined variable table and use these variables instead This will help to make a script clear, easy to understand and maintain, especially with scripts need to be run with different configurations Ram-up, Loop duration, Think time, Thread count Rampup value helps a heavy test can be loaded smoother Loop duration: a test need to be ended with a suitable duration/loops count Think time (timer) should be used to simulate a real user Threads count will impact to test results, it is number of virtual users that you need to simulate Cookies, Session, Iteration Cookies manager is needed in most of the cases to use the same http session for each VU (iteration) Each round of a VU is an iteration All of each VU iterations can use the same http session via cookies manager setting
JMeter notices – script usage Test process (startup, warm-up, measure) Automate test rounds (jenkins & perf-suite) Confirm the test script with single VU (as a result of preheat) Perf test short duration & long duration Jmeter GUI mode vs Non-gui mode: Non-gui mode is preferred JTL file/Listeners: jtl file is needed/enough via the command  jmeter -n -l output.jtl -t …  listeners are not needed
Data collecting methods  JMeter plugins and command line mode with JTL file Use suitable listeners in Jmeter GUI mode Merge results from multiple jtl files Monitoring tools (jmxtrans graphite, opennms/zabbix): collect via browsers
Data collecting method – JMeter-plugins command line  JMeter plugins and command line mode with JTL file java -jar CMDRunner.jar --tool Reporter --input-jtl results.jtl  --plugin-type <Plugin Type Classes>  --generate-png/--generate-csv <file_name>  [… other options...] Plugin Type Classes: AggregateReport = JMeter's native Aggregate Report , can be saved only as CSV ThreadsStateOverTime = Active Threads Over Time HitsPerSecond ResponseTimesDistribution ResponseTimesOverTime ResponseTimesPercentiles ThroughputOverTime TransactionsPerSecond
Data analysis and reporting Performance test statements Stability of cycles % of difference in response time Throughput distribution Application cycles Memory leak Session leak Resource utilization (see bottle necks) Error rate Throughput, 90% line of response time
[90% line of response time]  is  more reliable  than  [average response time] Variation  in range of [-5%,5%]  should be  treated  as normal for  [response time] Variation  out of range [-10%,10%]  should be  treated  as an improvement/decline of  [response time] Variation  in range of [-10%,-5%)  and ( 5%,10%]  should be  considered   as a decline/improvement of  [response time] Variation  in range of [-3%,3%]  should be  treated  as normal for  [Throughput] Variation  out of range [-5%,5%]  should be  treated  as an improvement/decline of  [Throughput] Variation in range of  [-5%,-3%)  and ( 3%,5%]  should be  considered   as a decline/improvement of  [Throughput] Data analysis and reporting- Performance test statements
As a result of the statements: Tests cycles on the same package and same test configuration  should not be vary out of range [-5%, 5%]  in comparison to their aggregation results The aggregation of cycles, itself, makes the test result more accurate, reduces error but  unstable results  still need to be checked. The stability of test rounds help us to give more accurate reports Data analysis and reporting – stability of cycles
% of difference in response time Throughput distribution Application cycles Data analysis and reporting – detail
Memory leak Session leak Data analysis and reporting – detail
Data analysis and reporting – resource utilization
What to focus in a performance test report
TC-PLF-3.5.1 PLF_PERF_04_SocialRead explanation https://wiki-int.exoplatform.org/display/exoReleases/TC-PLF-3.5.1+PLF_PERF_04_SocialRead A real performance test report explanation
Tips & tricks - A clear name for each performance test - non-GUI mode is more stable than GUI mode - Do not use listeners if not needed - Rampup is needed for heavy load - Assertion is needed to simulate a virtual user - Shutdown.sh -> save your effort while doing a long test - Unstable tests: think of data while an user run its scenario - If one user can not login, its later steps should not be counted - Backup after every important step you made to your script easily by cloning the jmx file - Speedup jmeter script modifying with text editors which support regex, e.g: kate
Links Http://code.google.com/p/jmeter-plugins/ http://jmeter.apache.org/ http://code.google.com/p/jmxtrans/wiki/IntroductionToJmxTrans?tm=6 https://wiki-int.exoplatform.org/display/QAF/Graphite https://wiki-int.exoplatform.org/display/TQA/Performance+test+suites+and+utilities https://wiki-int.exoplatform.org/display/TQA/PLF_PERF_04_SocialRead-3.0.x+-+PLF-3.0.7+-+DS2+stability+test https://wiki-int.exoplatform.org/display/TQA/PLF_PERF_04_SocialRead-3.5.x+-+PLF-3.5.0GA+-+DS2+stability+test https://wiki-int.exoplatform.org/display/exoReleases/TC-PLF-3.5.1_TOMCAT+Performance+Reports https://wiki-int.exoplatform.org/display/exoReleases/TC-PLF-3.5.1+PLF_PERF_04_SocialRead
Q & A
Thank you!

More Related Content

What's hot

Mykola Kovsh - Functional API automation with Jmeter
Mykola Kovsh - Functional API automation with JmeterMykola Kovsh - Functional API automation with Jmeter
Mykola Kovsh - Functional API automation with Jmeter
Ievgenii Katsan
 
Performance Testing using Jmeter and Capacity Testing
Performance Testing using Jmeter and Capacity TestingPerformance Testing using Jmeter and Capacity Testing
Performance Testing using Jmeter and Capacity Testing
Akshay Patole
 
Performance testing using jmeter
Performance testing using jmeterPerformance testing using jmeter
Performance testing using jmeter
Rachappa Bandi
 
Introduction to jmeter & how to view jmeter Test Result in Real-Time
Introduction to jmeter & how to view jmeter Test Result in Real-TimeIntroduction to jmeter & how to view jmeter Test Result in Real-Time
Introduction to jmeter & how to view jmeter Test Result in Real-Time
BugRaptors
 
Performance Testing from Scratch + JMeter intro
Performance Testing from Scratch + JMeter introPerformance Testing from Scratch + JMeter intro
Performance Testing from Scratch + JMeter intro
Mykola Kovsh
 
Jmeter interviewquestions
Jmeter interviewquestionsJmeter interviewquestions
Jmeter interviewquestions
girichinna27
 
Perofrmance testing and apache jmeter
Perofrmance testing and apache jmeterPerofrmance testing and apache jmeter
Perofrmance testing and apache jmeter
lethibichhoa
 
"Introduction to JMeter" @ CPTM 3rd Session
"Introduction to JMeter" @ CPTM 3rd Session"Introduction to JMeter" @ CPTM 3rd Session
"Introduction to JMeter" @ CPTM 3rd Session
Tharinda Liyanage
 
JMeter
JMeterJMeter
JMeter - Performance testing your webapp
JMeter - Performance testing your webappJMeter - Performance testing your webapp
JMeter - Performance testing your webapp
Amit Solanki
 
Introduction to JMeter
Introduction to JMeterIntroduction to JMeter
Introduction to JMeter
Galih Lasahido
 
How to Simplify Load Testing: JMeter and Beyond
How to Simplify Load Testing: JMeter and BeyondHow to Simplify Load Testing: JMeter and Beyond
How to Simplify Load Testing: JMeter and Beyond
Andrey Pokhilko
 
Jmeter Performance Testing
Jmeter Performance TestingJmeter Performance Testing
Jmeter Performance Testing
Atul Pant
 
Interpreting Performance Test Results
Interpreting Performance Test ResultsInterpreting Performance Test Results
Interpreting Performance Test Results
Eric Proegler
 
Performance testing with jmeter
Performance testing with jmeter Performance testing with jmeter
Performance testing with jmeter
Knoldus Inc.
 
Performance testing with Jmeter
Performance testing with JmeterPerformance testing with Jmeter
Performance testing with Jmeter
Prashanth Kumar
 
Jmeter Tester Certification
Jmeter Tester CertificationJmeter Tester Certification
Jmeter Tester Certification
Vskills
 
Load Test Drupal Site Using JMeter and Amazon AWS
Load Test Drupal Site Using JMeter and Amazon AWSLoad Test Drupal Site Using JMeter and Amazon AWS
Load Test Drupal Site Using JMeter and Amazon AWS
Vladimir Ilic
 
JMeter_ Cubet Seminar ppt
JMeter_ Cubet Seminar pptJMeter_ Cubet Seminar ppt
JMeter_ Cubet Seminar ppt
Cubet Techno Labs
 
JMeter Post-Processors
JMeter Post-ProcessorsJMeter Post-Processors
JMeter Post-Processors
Loadium
 

What's hot (20)

Mykola Kovsh - Functional API automation with Jmeter
Mykola Kovsh - Functional API automation with JmeterMykola Kovsh - Functional API automation with Jmeter
Mykola Kovsh - Functional API automation with Jmeter
 
Performance Testing using Jmeter and Capacity Testing
Performance Testing using Jmeter and Capacity TestingPerformance Testing using Jmeter and Capacity Testing
Performance Testing using Jmeter and Capacity Testing
 
Performance testing using jmeter
Performance testing using jmeterPerformance testing using jmeter
Performance testing using jmeter
 
Introduction to jmeter & how to view jmeter Test Result in Real-Time
Introduction to jmeter & how to view jmeter Test Result in Real-TimeIntroduction to jmeter & how to view jmeter Test Result in Real-Time
Introduction to jmeter & how to view jmeter Test Result in Real-Time
 
Performance Testing from Scratch + JMeter intro
Performance Testing from Scratch + JMeter introPerformance Testing from Scratch + JMeter intro
Performance Testing from Scratch + JMeter intro
 
Jmeter interviewquestions
Jmeter interviewquestionsJmeter interviewquestions
Jmeter interviewquestions
 
Perofrmance testing and apache jmeter
Perofrmance testing and apache jmeterPerofrmance testing and apache jmeter
Perofrmance testing and apache jmeter
 
"Introduction to JMeter" @ CPTM 3rd Session
"Introduction to JMeter" @ CPTM 3rd Session"Introduction to JMeter" @ CPTM 3rd Session
"Introduction to JMeter" @ CPTM 3rd Session
 
JMeter
JMeterJMeter
JMeter
 
JMeter - Performance testing your webapp
JMeter - Performance testing your webappJMeter - Performance testing your webapp
JMeter - Performance testing your webapp
 
Introduction to JMeter
Introduction to JMeterIntroduction to JMeter
Introduction to JMeter
 
How to Simplify Load Testing: JMeter and Beyond
How to Simplify Load Testing: JMeter and BeyondHow to Simplify Load Testing: JMeter and Beyond
How to Simplify Load Testing: JMeter and Beyond
 
Jmeter Performance Testing
Jmeter Performance TestingJmeter Performance Testing
Jmeter Performance Testing
 
Interpreting Performance Test Results
Interpreting Performance Test ResultsInterpreting Performance Test Results
Interpreting Performance Test Results
 
Performance testing with jmeter
Performance testing with jmeter Performance testing with jmeter
Performance testing with jmeter
 
Performance testing with Jmeter
Performance testing with JmeterPerformance testing with Jmeter
Performance testing with Jmeter
 
Jmeter Tester Certification
Jmeter Tester CertificationJmeter Tester Certification
Jmeter Tester Certification
 
Load Test Drupal Site Using JMeter and Amazon AWS
Load Test Drupal Site Using JMeter and Amazon AWSLoad Test Drupal Site Using JMeter and Amazon AWS
Load Test Drupal Site Using JMeter and Amazon AWS
 
JMeter_ Cubet Seminar ppt
JMeter_ Cubet Seminar pptJMeter_ Cubet Seminar ppt
JMeter_ Cubet Seminar ppt
 
JMeter Post-Processors
JMeter Post-ProcessorsJMeter Post-Processors
JMeter Post-Processors
 

Similar to Performance testing and_reporting_with_j_meter by Le Van Nghi

Performance testing and j meter
Performance testing and j meterPerformance testing and j meter
Performance testing and j meter
Purna Chandar
 
Performancetestingjmeter 131210111657-phpapp02
Performancetestingjmeter 131210111657-phpapp02Performancetestingjmeter 131210111657-phpapp02
Performancetestingjmeter 131210111657-phpapp02
Nitish Bhardwaj
 
Performancetestingjmeter 121109061704-phpapp02
Performancetestingjmeter 121109061704-phpapp02Performancetestingjmeter 121109061704-phpapp02
Performancetestingjmeter 121109061704-phpapp02
Shivakumara .
 
Continuous Performance Testing
Continuous Performance TestingContinuous Performance Testing
Continuous Performance Testing
Mark Price
 
Soft Test Ireland - Introduction to Jakarta Jmeter - Philip Bannon
Soft Test Ireland - Introduction to Jakarta Jmeter - Philip BannonSoft Test Ireland - Introduction to Jakarta Jmeter - Philip Bannon
Soft Test Ireland - Introduction to Jakarta Jmeter - Philip Bannon
David O'Dowd
 
Test Automation for QTP
Test Automation for QTPTest Automation for QTP
Test Automation for QTP
Kiran Rayachoti
 
Test Automation
Test AutomationTest Automation
Test Automation
Kiran Rayachoti
 
Performance testing jmeter
Performance testing jmeterPerformance testing jmeter
Performance testing jmeter
Bhojan Rajan
 
JMETER-SKILLWISE
JMETER-SKILLWISEJMETER-SKILLWISE
JMETER-SKILLWISE
Skillwise Consulting
 
Qtp Training Deepti 1 Of 4187
Qtp Training Deepti 1 Of 4187Qtp Training Deepti 1 Of 4187
Qtp Training Deepti 1 Of 4187
Azhar Satti
 
Basic of jMeter
Basic of jMeter Basic of jMeter
Basic of jMeter
Shub
 
Software testing
Software testingSoftware testing
Software testing
nil65
 
Verification & Validation
Verification & ValidationVerification & Validation
Verification & Validation
Dhanasekaran Nagarajan
 
Testing Types And Models
Testing Types And ModelsTesting Types And Models
Testing Types And Models
nazeer pasha
 
Performance testing and j meter overview
Performance testing and j meter overviewPerformance testing and j meter overview
Performance testing and j meter overview
krishna chaitanya
 
Performance Testing Training in Hyderabad
Performance Testing Training in HyderabadPerformance Testing Training in Hyderabad
Performance Testing Training in Hyderabad
rajasrichalamala3zen
 
performance testing training in hyderabad
performance testing training in hyderabadperformance testing training in hyderabad
performance testing training in hyderabad
aparna3zen
 
Performance Testing.3zen.pdf
Performance Testing.3zen.pdfPerformance Testing.3zen.pdf
Performance Testing.3zen.pdf
swathi3zen
 
performance testing training in hyderabad
performance testing training in hyderabadperformance testing training in hyderabad
performance testing training in hyderabad
madhupriya3zen
 
Fundamentals Performance Testing
Fundamentals Performance TestingFundamentals Performance Testing
Fundamentals Performance Testing
Bhuvaneswari Subramani
 

Similar to Performance testing and_reporting_with_j_meter by Le Van Nghi (20)

Performance testing and j meter
Performance testing and j meterPerformance testing and j meter
Performance testing and j meter
 
Performancetestingjmeter 131210111657-phpapp02
Performancetestingjmeter 131210111657-phpapp02Performancetestingjmeter 131210111657-phpapp02
Performancetestingjmeter 131210111657-phpapp02
 
Performancetestingjmeter 121109061704-phpapp02
Performancetestingjmeter 121109061704-phpapp02Performancetestingjmeter 121109061704-phpapp02
Performancetestingjmeter 121109061704-phpapp02
 
Continuous Performance Testing
Continuous Performance TestingContinuous Performance Testing
Continuous Performance Testing
 
Soft Test Ireland - Introduction to Jakarta Jmeter - Philip Bannon
Soft Test Ireland - Introduction to Jakarta Jmeter - Philip BannonSoft Test Ireland - Introduction to Jakarta Jmeter - Philip Bannon
Soft Test Ireland - Introduction to Jakarta Jmeter - Philip Bannon
 
Test Automation for QTP
Test Automation for QTPTest Automation for QTP
Test Automation for QTP
 
Test Automation
Test AutomationTest Automation
Test Automation
 
Performance testing jmeter
Performance testing jmeterPerformance testing jmeter
Performance testing jmeter
 
JMETER-SKILLWISE
JMETER-SKILLWISEJMETER-SKILLWISE
JMETER-SKILLWISE
 
Qtp Training Deepti 1 Of 4187
Qtp Training Deepti 1 Of 4187Qtp Training Deepti 1 Of 4187
Qtp Training Deepti 1 Of 4187
 
Basic of jMeter
Basic of jMeter Basic of jMeter
Basic of jMeter
 
Software testing
Software testingSoftware testing
Software testing
 
Verification & Validation
Verification & ValidationVerification & Validation
Verification & Validation
 
Testing Types And Models
Testing Types And ModelsTesting Types And Models
Testing Types And Models
 
Performance testing and j meter overview
Performance testing and j meter overviewPerformance testing and j meter overview
Performance testing and j meter overview
 
Performance Testing Training in Hyderabad
Performance Testing Training in HyderabadPerformance Testing Training in Hyderabad
Performance Testing Training in Hyderabad
 
performance testing training in hyderabad
performance testing training in hyderabadperformance testing training in hyderabad
performance testing training in hyderabad
 
Performance Testing.3zen.pdf
Performance Testing.3zen.pdfPerformance Testing.3zen.pdf
Performance Testing.3zen.pdf
 
performance testing training in hyderabad
performance testing training in hyderabadperformance testing training in hyderabad
performance testing training in hyderabad
 
Fundamentals Performance Testing
Fundamentals Performance TestingFundamentals Performance Testing
Fundamentals Performance Testing
 

More from eXo Platform

Workshop blockchain au service de l engagement - 20-3-2019
Workshop blockchain au service de l engagement - 20-3-2019Workshop blockchain au service de l engagement - 20-3-2019
Workshop blockchain au service de l engagement - 20-3-2019
eXo Platform
 
Digitalisation RH –Avec ou sans les DRH
Digitalisation RH –Avec ou sans les DRHDigitalisation RH –Avec ou sans les DRH
Digitalisation RH –Avec ou sans les DRH
eXo Platform
 
De l'Intranet institutionnel à la Digital Workplace : la démarche du groupe d...
De l'Intranet institutionnel à la Digital Workplace : la démarche du groupe d...De l'Intranet institutionnel à la Digital Workplace : la démarche du groupe d...
De l'Intranet institutionnel à la Digital Workplace : la démarche du groupe d...
eXo Platform
 
L’Intranet, socle de l’entreprise digitale !
L’Intranet, socle de l’entreprise digitale !L’Intranet, socle de l’entreprise digitale !
L’Intranet, socle de l’entreprise digitale !
eXo Platform
 
Un hub collaboratif unique et intégré : les pour et les contre
Un hub collaboratif unique et intégré : les pour et les contreUn hub collaboratif unique et intégré : les pour et les contre
Un hub collaboratif unique et intégré : les pour et les contre
eXo Platform
 
Intranet Project: Roll-out Strategy & Pain Points to consider
Intranet Project: Roll-out Strategy & Pain Points to considerIntranet Project: Roll-out Strategy & Pain Points to consider
Intranet Project: Roll-out Strategy & Pain Points to consider
eXo Platform
 
How we migrate 350+ Maven CI jobs to Pipeline as Code with Jenkins 2 and Docker
How we migrate 350+ Maven CI jobs to Pipeline as Code with Jenkins 2 and DockerHow we migrate 350+ Maven CI jobs to Pipeline as Code with Jenkins 2 and Docker
How we migrate 350+ Maven CI jobs to Pipeline as Code with Jenkins 2 and Docker
eXo Platform
 
Peut-on décider un changement de culture d'entreprise?
Peut-on décider un changement de culture d'entreprise? Peut-on décider un changement de culture d'entreprise?
Peut-on décider un changement de culture d'entreprise?
eXo Platform
 
Conférence sur la Digital Workplace au Salon Intranet & Collaboratif
Conférence sur la Digital Workplace au Salon Intranet & CollaboratifConférence sur la Digital Workplace au Salon Intranet & Collaboratif
Conférence sur la Digital Workplace au Salon Intranet & Collaboratif
eXo Platform
 
eXo Platform - Votre plateforme de travail collaboratif
eXo Platform - Votre plateforme de travail collaboratifeXo Platform - Votre plateforme de travail collaboratif
eXo Platform - Votre plateforme de travail collaboratif
eXo Platform
 
eXo Platform 4.4 Released: Work Better with More Context!
eXo Platform 4.4 Released: Work Better with More Context!eXo Platform 4.4 Released: Work Better with More Context!
eXo Platform 4.4 Released: Work Better with More Context!
eXo Platform
 
When Collaboration Drives Your Digital Transformation
When Collaboration Drives Your Digital TransformationWhen Collaboration Drives Your Digital Transformation
When Collaboration Drives Your Digital Transformation
eXo Platform
 
Solutions to your employee disengagement
Solutions to your employee disengagementSolutions to your employee disengagement
Solutions to your employee disengagement
eXo Platform
 
eXoer on the grill: eXo Add-ons factory using Docker and Codenvy
eXoer on the grill: eXo Add-ons factory using Docker and CodenvyeXoer on the grill: eXo Add-ons factory using Docker and Codenvy
eXoer on the grill: eXo Add-ons factory using Docker and Codenvy
eXo Platform
 
Introduce Bootstrap 3 to Develop Responsive Design Application
Introduce Bootstrap 3 to Develop Responsive Design ApplicationIntroduce Bootstrap 3 to Develop Responsive Design Application
Introduce Bootstrap 3 to Develop Responsive Design Application
eXo Platform
 
CRaSH the shell for the Java Virtual Machine
CRaSH the shell for the Java Virtual MachineCRaSH the shell for the Java Virtual Machine
CRaSH the shell for the Java Virtual Machine
eXo Platform
 
Hands on iOS developments with Jenkins
Hands on iOS developments with JenkinsHands on iOS developments with Jenkins
Hands on iOS developments with Jenkins
eXo Platform
 
CRaSH @ JUGSummerCamp 2012 - Quickie
CRaSH @ JUGSummerCamp 2012 - QuickieCRaSH @ JUGSummerCamp 2012 - Quickie
CRaSH @ JUGSummerCamp 2012 - Quickie
eXo Platform
 
Modèles Business Open Source - Success story eXo
Modèles Business Open Source - Success story eXoModèles Business Open Source - Success story eXo
Modèles Business Open Source - Success story eXo
eXo Platform
 

More from eXo Platform (19)

Workshop blockchain au service de l engagement - 20-3-2019
Workshop blockchain au service de l engagement - 20-3-2019Workshop blockchain au service de l engagement - 20-3-2019
Workshop blockchain au service de l engagement - 20-3-2019
 
Digitalisation RH –Avec ou sans les DRH
Digitalisation RH –Avec ou sans les DRHDigitalisation RH –Avec ou sans les DRH
Digitalisation RH –Avec ou sans les DRH
 
De l'Intranet institutionnel à la Digital Workplace : la démarche du groupe d...
De l'Intranet institutionnel à la Digital Workplace : la démarche du groupe d...De l'Intranet institutionnel à la Digital Workplace : la démarche du groupe d...
De l'Intranet institutionnel à la Digital Workplace : la démarche du groupe d...
 
L’Intranet, socle de l’entreprise digitale !
L’Intranet, socle de l’entreprise digitale !L’Intranet, socle de l’entreprise digitale !
L’Intranet, socle de l’entreprise digitale !
 
Un hub collaboratif unique et intégré : les pour et les contre
Un hub collaboratif unique et intégré : les pour et les contreUn hub collaboratif unique et intégré : les pour et les contre
Un hub collaboratif unique et intégré : les pour et les contre
 
Intranet Project: Roll-out Strategy & Pain Points to consider
Intranet Project: Roll-out Strategy & Pain Points to considerIntranet Project: Roll-out Strategy & Pain Points to consider
Intranet Project: Roll-out Strategy & Pain Points to consider
 
How we migrate 350+ Maven CI jobs to Pipeline as Code with Jenkins 2 and Docker
How we migrate 350+ Maven CI jobs to Pipeline as Code with Jenkins 2 and DockerHow we migrate 350+ Maven CI jobs to Pipeline as Code with Jenkins 2 and Docker
How we migrate 350+ Maven CI jobs to Pipeline as Code with Jenkins 2 and Docker
 
Peut-on décider un changement de culture d'entreprise?
Peut-on décider un changement de culture d'entreprise? Peut-on décider un changement de culture d'entreprise?
Peut-on décider un changement de culture d'entreprise?
 
Conférence sur la Digital Workplace au Salon Intranet & Collaboratif
Conférence sur la Digital Workplace au Salon Intranet & CollaboratifConférence sur la Digital Workplace au Salon Intranet & Collaboratif
Conférence sur la Digital Workplace au Salon Intranet & Collaboratif
 
eXo Platform - Votre plateforme de travail collaboratif
eXo Platform - Votre plateforme de travail collaboratifeXo Platform - Votre plateforme de travail collaboratif
eXo Platform - Votre plateforme de travail collaboratif
 
eXo Platform 4.4 Released: Work Better with More Context!
eXo Platform 4.4 Released: Work Better with More Context!eXo Platform 4.4 Released: Work Better with More Context!
eXo Platform 4.4 Released: Work Better with More Context!
 
When Collaboration Drives Your Digital Transformation
When Collaboration Drives Your Digital TransformationWhen Collaboration Drives Your Digital Transformation
When Collaboration Drives Your Digital Transformation
 
Solutions to your employee disengagement
Solutions to your employee disengagementSolutions to your employee disengagement
Solutions to your employee disengagement
 
eXoer on the grill: eXo Add-ons factory using Docker and Codenvy
eXoer on the grill: eXo Add-ons factory using Docker and CodenvyeXoer on the grill: eXo Add-ons factory using Docker and Codenvy
eXoer on the grill: eXo Add-ons factory using Docker and Codenvy
 
Introduce Bootstrap 3 to Develop Responsive Design Application
Introduce Bootstrap 3 to Develop Responsive Design ApplicationIntroduce Bootstrap 3 to Develop Responsive Design Application
Introduce Bootstrap 3 to Develop Responsive Design Application
 
CRaSH the shell for the Java Virtual Machine
CRaSH the shell for the Java Virtual MachineCRaSH the shell for the Java Virtual Machine
CRaSH the shell for the Java Virtual Machine
 
Hands on iOS developments with Jenkins
Hands on iOS developments with JenkinsHands on iOS developments with Jenkins
Hands on iOS developments with Jenkins
 
CRaSH @ JUGSummerCamp 2012 - Quickie
CRaSH @ JUGSummerCamp 2012 - QuickieCRaSH @ JUGSummerCamp 2012 - Quickie
CRaSH @ JUGSummerCamp 2012 - Quickie
 
Modèles Business Open Source - Success story eXo
Modèles Business Open Source - Success story eXoModèles Business Open Source - Success story eXo
Modèles Business Open Source - Success story eXo
 

Recently uploaded

High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
Anant Gupta
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
WhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring AppsWhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring Apps
HackersList
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
DianaGray10
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
aslasdfmkhan4750
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
Axel Rennoch
 
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
Priyanka Aash
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
SubhamMandal40
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
aakash malhotra
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ssuser1915fe1
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
maigasapphire
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
Priyanka Aash
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
313mohammedarshad
 

Recently uploaded (20)

High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
WhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring AppsWhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring Apps
 
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision MakingConnector Corner: Leveraging Snowflake Integration for Smarter Decision Making
Connector Corner: Leveraging Snowflake Integration for Smarter Decision Making
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
 
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
 
Sonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdfSonkoloniya documentation - ONEprojukti.pdf
Sonkoloniya documentation - ONEprojukti.pdf
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
 
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
Girls Call Churchgate 9910780858 Provide Best And Top Girl Service And No1 in...
 
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
 

Performance testing and_reporting_with_j_meter by Le Van Nghi

  • 1. Performance testing and reporting with JMeter Introduced by: Nghi Nguyen Van ~ March, 1 st 2012 ~
  • 2. Agenda Introduction What is performance testing, why is it important Performance testing and monitoring tools Performance testing with JMeter What to focus in a performance test report Tips and tricks Q & A
  • 3. Purpose of this presentation: Introduce about performance testing activity in eXo SEA Share experience in do performance testing Share experience in working with Jmeter This presentation serves to: ● Leaders to understand about performance testing activity, reports ● Who is new in using Jmeter to gain knowledge about using Jmeter ● Who experienced in Jmeter want to gain, share & discuss about using Jmeter ● Testers who want to understand about building, using performance testing system Introduction
  • 4. What is performance testing, why is it important?
  • 5. Definition: Performance testing is a type of testing intended to determine the responsiveness throughput reliability and/or scalability of a system under a given workload What is performance testing
  • 6. Performance testing is commonly conducted to accomplish the following: Assess production readiness Evaluate against performance criteria Compare performance characteristics of multiple systems or system configurations Find the source of performance problems Support system tuning Find throughput levels What is performance testing...
  • 7. Performance test Load test Stress test Capacity test Types of Performance Testing
  • 8. Application performance impacted by various of factors Last minute surprises are always bad Applications made up from hundred/thousand of components Components change day by day Performance issues are hard to fix Why is it so important?
  • 9. Performance testing and monitoring tools
  • 10. Imagine about a performance testing system This system must provide information about: Availability/Stability Response time Throughput Utilization
  • 11. Can provide data: Availability/Stability Response time Throughput Commercial tools: LoadRunner (HP) IBM Rational Performance Tester (IBM) Visual Studio Load Test (Microsoft) Opensource tools: Apache JMeter (Apache Jakarta) The grinder OpenSTA (Open System Testing Architecture) Needed tools - Virtual users simulators
  • 12. Data can provide: Percent of CPU utilization Amount of free memory Page file utilization Disk time Amount of free disk space Tools: OpenNMS Hyperic HQ Zabbix Perfmon (Windows) Needed tools – System monitoring
  • 13. Can provide data: Detailed information about the application resource used Amount of used memory A mount of threads Openning files Monitoring tools: JvisualVM/VisualVM Jmxtrans + graphite Jprofiler HypericHQ Jmanage Javamelody Needed tools – application monitoring
  • 14. A real performance testing system Application monitoring tool: jmxtrans, graphite Application measuring tool: JMeter System monitoring tool: OpenNMS
  • 16. The Apache JMeter™ desktop application is: - open source software - 100% pure Java application designed to load test functional behavior and measure performance - designed for testing Web Applicatio n JMeter features: - Can load and performance test many different server types - Complete portability and 100% Java purity - Full multithreading framework - Careful GUI design - Caching and offline analysis/replaying of test results. - Highly Extensible Apache JMeter introduction
  • 17. +) Easy to use for both developer and tester record/modify with GUI understand & modify under text mode install the tool and distribute test scripts +) Good enough Rich elements that help doing performance test easily (especially web apps) Stable Good for both simple and advance test configuration Good with CI/automation test model Extendable via plugins +) Opensource +-) JMeter is a great tool but poor document Why JMeter is chosen/still be in used
  • 18. Response time 90% line of response time (90 th percentile of response time) 90% of all the response time values less or equal than this value) Average response time Simple average of all values=(sum of all values from 1 st to Nth)/N Min and Max response time Error rate Throughput Performance testing notices – Basic parameters
  • 19. Performance testing notices – Most reliable response time Most reliable response time = 90% line of response time (90 th percentile of response time)!
  • 20. Performance testing tips - Do stability test Purpose Make sure that the test is reliable Gain better knowledge about application/test Give/confirm later tests' settings: Test duration Test cycles …
  • 21. Performance testing tips - Do stability test How Choose a simple test Repeat the test for many times (e.g: >=6 times) Calculate difference from result of each test to their average Apply the performance test statements
  • 22. JMeter demo & notes
  • 23. Demo notices – Jenkins flow Jenkins flows: JMeter benchmark process Perf-suite Loop until end of setting file Various of test settings Easy to maintain/extend
  • 24. Demo notices – JMeter script Introduce JMeter script How is a script used in a benchmark flow with jenkins Scripts: PLF_PERF_05_Login.jmx PLF_PERF_04_SocialRead-3.0.x.jmx PLF_PERF_03_ECMS-Public.jmx
  • 25. JMeter notices – recording/modifying script Use selenium script -> this is for QA who have to repeat the scenario oftenly Save and backup the first recorded version, backup after every importance change Find and replace all the changeable values by Jmeter variables such as host address port number date and time account used in the scenario Id of components HTTP response assertions are needed to avoid something wrong
  • 26. JMeter notices – script organization.p1
  • 27. JMeter notices – script organization.p2 Test plan interface Jmeter properties usage can be used to make up an interface for a testscript Properties' value should be assigned to runtime variables at user defined variable table and use these variables instead This will help to make a script clear, easy to understand and maintain, especially with scripts need to be run with different configurations Ram-up, Loop duration, Think time, Thread count Rampup value helps a heavy test can be loaded smoother Loop duration: a test need to be ended with a suitable duration/loops count Think time (timer) should be used to simulate a real user Threads count will impact to test results, it is number of virtual users that you need to simulate Cookies, Session, Iteration Cookies manager is needed in most of the cases to use the same http session for each VU (iteration) Each round of a VU is an iteration All of each VU iterations can use the same http session via cookies manager setting
  • 28. JMeter notices – script usage Test process (startup, warm-up, measure) Automate test rounds (jenkins & perf-suite) Confirm the test script with single VU (as a result of preheat) Perf test short duration & long duration Jmeter GUI mode vs Non-gui mode: Non-gui mode is preferred JTL file/Listeners: jtl file is needed/enough via the command jmeter -n -l output.jtl -t … listeners are not needed
  • 29. Data collecting methods JMeter plugins and command line mode with JTL file Use suitable listeners in Jmeter GUI mode Merge results from multiple jtl files Monitoring tools (jmxtrans graphite, opennms/zabbix): collect via browsers
  • 30. Data collecting method – JMeter-plugins command line JMeter plugins and command line mode with JTL file java -jar CMDRunner.jar --tool Reporter --input-jtl results.jtl --plugin-type <Plugin Type Classes> --generate-png/--generate-csv <file_name> [… other options...] Plugin Type Classes: AggregateReport = JMeter's native Aggregate Report , can be saved only as CSV ThreadsStateOverTime = Active Threads Over Time HitsPerSecond ResponseTimesDistribution ResponseTimesOverTime ResponseTimesPercentiles ThroughputOverTime TransactionsPerSecond
  • 31. Data analysis and reporting Performance test statements Stability of cycles % of difference in response time Throughput distribution Application cycles Memory leak Session leak Resource utilization (see bottle necks) Error rate Throughput, 90% line of response time
  • 32. [90% line of response time] is more reliable than [average response time] Variation in range of [-5%,5%] should be treated as normal for [response time] Variation out of range [-10%,10%] should be treated as an improvement/decline of [response time] Variation in range of [-10%,-5%) and ( 5%,10%] should be considered as a decline/improvement of [response time] Variation in range of [-3%,3%] should be treated as normal for [Throughput] Variation out of range [-5%,5%] should be treated as an improvement/decline of [Throughput] Variation in range of [-5%,-3%) and ( 3%,5%] should be considered as a decline/improvement of [Throughput] Data analysis and reporting- Performance test statements
  • 33. As a result of the statements: Tests cycles on the same package and same test configuration should not be vary out of range [-5%, 5%] in comparison to their aggregation results The aggregation of cycles, itself, makes the test result more accurate, reduces error but unstable results still need to be checked. The stability of test rounds help us to give more accurate reports Data analysis and reporting – stability of cycles
  • 34. % of difference in response time Throughput distribution Application cycles Data analysis and reporting – detail
  • 35. Memory leak Session leak Data analysis and reporting – detail
  • 36. Data analysis and reporting – resource utilization
  • 37. What to focus in a performance test report
  • 38. TC-PLF-3.5.1 PLF_PERF_04_SocialRead explanation https://wiki-int.exoplatform.org/display/exoReleases/TC-PLF-3.5.1+PLF_PERF_04_SocialRead A real performance test report explanation
  • 39. Tips & tricks - A clear name for each performance test - non-GUI mode is more stable than GUI mode - Do not use listeners if not needed - Rampup is needed for heavy load - Assertion is needed to simulate a virtual user - Shutdown.sh -> save your effort while doing a long test - Unstable tests: think of data while an user run its scenario - If one user can not login, its later steps should not be counted - Backup after every important step you made to your script easily by cloning the jmx file - Speedup jmeter script modifying with text editors which support regex, e.g: kate
  • 40. Links Http://code.google.com/p/jmeter-plugins/ http://jmeter.apache.org/ http://code.google.com/p/jmxtrans/wiki/IntroductionToJmxTrans?tm=6 https://wiki-int.exoplatform.org/display/QAF/Graphite https://wiki-int.exoplatform.org/display/TQA/Performance+test+suites+and+utilities https://wiki-int.exoplatform.org/display/TQA/PLF_PERF_04_SocialRead-3.0.x+-+PLF-3.0.7+-+DS2+stability+test https://wiki-int.exoplatform.org/display/TQA/PLF_PERF_04_SocialRead-3.5.x+-+PLF-3.5.0GA+-+DS2+stability+test https://wiki-int.exoplatform.org/display/exoReleases/TC-PLF-3.5.1_TOMCAT+Performance+Reports https://wiki-int.exoplatform.org/display/exoReleases/TC-PLF-3.5.1+PLF_PERF_04_SocialRead
  • 41. Q & A

Editor's Notes

  1. This is a very simple image about a testing system. A developer usually test with this system.
  2. The Apache JMeter™ desktop application is: -open source software, -100% pure Java application designed to load test functional behavior and measure performance - originally designed for testing Web Applications JMeter features: -Can load and performance test many different server types: Web - HTTP, HTTPS SOAP, Database via JDBC, LDAP, JMS, Mail - POP3(S) and IMAP(S) -Complete portability and 100% Java purity. -Full multithreading framework -Careful GUI design -Caching and offline analysis/replaying of test results. -Highly Extensible
  3. (JMeter-plugins for example: that enrich JMeter features can provide information about: Availability, Response time, Throughput in many views
  4. http://svn.exoplatform.org/projects/platform/branches/3.0.x/testsuite/perf/src/test/jmeter/PLF_PERF_05_Login.jmx http://svn.exoplatform.org/projects/platform/branches/3.0.x/testsuite/perf/src/test/jmeter/PLF_PERF_04_SocialRead-3.0.x.jmx http://svn.exoplatform.org/projects/platform/branches/3.0.x/testsuite/perf/src/test/jmeter/PLF_PERF_03_ECMS-Public.jmx
  5. http://http://code.google.com/p/jmeter-plugins/wiki/JMeterPluginsCMD
  6. java -jar CMDRunner.jar --tool Reporter --input-jtl results.jtl --plugin-type ResponseTimesOverTime --generate-png test.png --width 800 --height 600 java -jar CMDRunner.jar --tool Reporter --input-jtl results.jtl --plugin-type ResponseTimesOverTime --generate-csv test.csv