Scalable eCommerce Platform SolutionsJagger: Continuous PerformanceTesting of Mission-Critical ApplicationsIlya Katsov, Gr...
Scalable eCommerce Platform Solutions4/29/2013 2PerformanceTestingToolSystemUnderTestReportWhat’s wrong?
Scalable eCommerce Platform Solutions4/29/2013 3What we do• Develop backend services for 3 of Top-30 US Retailers• Product...
Scalable eCommerce Platform Solutions4/29/2013 4Antipatterns of Performance Testing• Acceptance performance testing• Indep...
Scalable eCommerce Platform Solutions4/29/2013 5PerformanceTestingToolSystemUnderTestReportWhat’s wrong?
Scalable eCommerce Platform Solutions4/29/2013 6Performance Testing EcosystemDevelopmentEnvironmentPerformanceEnvironmentQ...
Scalable eCommerce Platform Solutions4/29/2013 7Slave Slave Slave SlaveSlaveMasterSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTA...
Scalable eCommerce Platform Solutions4/29/2013 8Lessons learned: Automate. Automate to scale.• Completely automated flows•...
Scalable eCommerce Platform Solutions4/29/2013 9Lessons learned: Automate. Automate to scale.• Decision makers that can ac...
Scalable eCommerce Platform Solutions4/29/2013 10Lessons learned: Monitoring is everything.• OS-level and JVM-level metric...
Scalable eCommerce Platform Solutions4/29/2013 11Lessons learned: Profile system under test• Performance issues investigat...
Scalable eCommerce Platform Solutions4/29/2013 12Lessons learned: Profile system under test• Sampling profiler for JVM (st...
Scalable eCommerce Platform Solutions4/29/2013 13Lessons learned: Validate responses• Custom validators• Record results of...
Scalable eCommerce Platform Solutions4/29/2013 14Lessons learned: Manage your data well.• Statistics from one of our accou...
Scalable eCommerce Platform Solutions4/29/2013 15Lessons learned: Manage your data well.• Reports - PDF and web UI
Scalable eCommerce Platform Solutions4/29/2013 16Results and RoadmapSuccessful:• Continuous testing• Monitoring• Validatio...
Scalable eCommerce Platform Solutions4/29/2013 17jagger@griddynamics.comContact Ushttps://jagger.griddynamics.netDistribut...
Upcoming SlideShare
Loading in...5
×

Jagger: Сервер непрерывного тестирования производительности

550

Published on

Доклад Ильи Кацова, SQA Days-13, 26-27 апреля 2013, www.sqadays.com

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
550
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Jagger: Сервер непрерывного тестирования производительности

  1. 1. Scalable eCommerce Platform SolutionsJagger: Continuous PerformanceTesting of Mission-Critical ApplicationsIlya Katsov, Grid Dynamics
  2. 2. Scalable eCommerce Platform Solutions4/29/2013 2PerformanceTestingToolSystemUnderTestReportWhat’s wrong?
  3. 3. Scalable eCommerce Platform Solutions4/29/2013 3What we do• Develop backend services for 3 of Top-30 US Retailers• Production clusters of 40 nodes and more• Load up to 50K TPS• Heavy computationsin navigational queries:
  4. 4. Scalable eCommerce Platform Solutions4/29/2013 4Antipatterns of Performance Testing• Acceptance performance testing• Independent functional and performance testing• Poor tracking of testing results• Insufficient introspection of the system under test• Failover is tested in production
  5. 5. Scalable eCommerce Platform Solutions4/29/2013 5PerformanceTestingToolSystemUnderTestReportWhat’s wrong?
  6. 6. Scalable eCommerce Platform Solutions4/29/2013 6Performance Testing EcosystemDevelopmentEnvironmentPerformanceEnvironmentQAEnvironmentProductionEnvironmentFunctionalTestsProfiling PerformanceTestsMonitoringReleaseManagerPerformanceTest LeadPerformanceArchitectTechnicalLead
  7. 7. Scalable eCommerce Platform Solutions4/29/2013 7Slave Slave Slave SlaveSlaveMasterSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentJagger Technical ArchitectureWeb UIHDFSMySQLZookeeper
  8. 8. Scalable eCommerce Platform Solutions4/29/2013 8Lessons learned: Automate. Automate to scale.• Completely automated flows• Jenkins plugin for automatic deployment• Results warehousing in the database
  9. 9. Scalable eCommerce Platform Solutions4/29/2013 9Lessons learned: Automate. Automate to scale.• Decision makers that can access all collected data• Session comparison• VCS-friendly configuration in plain text files
  10. 10. Scalable eCommerce Platform Solutions4/29/2013 10Lessons learned: Monitoring is everything.• OS-level and JVM-level metrics• Ability to load external metrics• All metrics can be involved into automatic decisionmaking• Jagger can be used to monitor an external process:ActivityJaggerAgentJagger ClusterCI Server
  11. 11. Scalable eCommerce Platform Solutions4/29/2013 11Lessons learned: Profile system under test• Performance issues investigation – a typical flow:Problem ResolvedFixPerformanceTestingPerformanceProblem Detected
  12. 12. Scalable eCommerce Platform Solutions4/29/2013 12Lessons learned: Profile system under test• Sampling profiler for JVM (stacktraces via JMX)• Profiler contains intelligence to detect hot spotsPerformanceProblem DetectedProblem ResolvedFixProfilingresultsPerformanceTesting
  13. 13. Scalable eCommerce Platform Solutions4/29/2013 13Lessons learned: Validate responses• Custom validators• Record results of single-user test and use for validation
  14. 14. Scalable eCommerce Platform Solutions4/29/2013 14Lessons learned: Manage your data well.• Statistics from one of our accounts:• About 100 test cases• 20 scheduled test sessions weekly + ad hoc testing• 30GB of test results, more than 1500 retained sessions
  15. 15. Scalable eCommerce Platform Solutions4/29/2013 15Lessons learned: Manage your data well.• Reports - PDF and web UI
  16. 16. Scalable eCommerce Platform Solutions4/29/2013 16Results and RoadmapSuccessful:• Continuous testing• Monitoring• Validation• Data managementNot very successful:• Continuous robustnesstesting• Integration with productionmonitoring
  17. 17. Scalable eCommerce Platform Solutions4/29/2013 17jagger@griddynamics.comContact Ushttps://jagger.griddynamics.netDistribution and Documentation
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×