Jagger: Сервер непрерывного тестирования производительности

  • 454 views
Uploaded on

Доклад Ильи Кацова, SQA Days-13, 26-27 апреля 2013, www.sqadays.com

Доклад Ильи Кацова, SQA Days-13, 26-27 апреля 2013, www.sqadays.com

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
454
On Slideshare
0
From Embeds
0
Number of Embeds
3

Actions

Shares
Downloads
6
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Scalable eCommerce Platform SolutionsJagger: Continuous PerformanceTesting of Mission-Critical ApplicationsIlya Katsov, Grid Dynamics
  • 2. Scalable eCommerce Platform Solutions4/29/2013 2PerformanceTestingToolSystemUnderTestReportWhat’s wrong?
  • 3. Scalable eCommerce Platform Solutions4/29/2013 3What we do• Develop backend services for 3 of Top-30 US Retailers• Production clusters of 40 nodes and more• Load up to 50K TPS• Heavy computationsin navigational queries:
  • 4. Scalable eCommerce Platform Solutions4/29/2013 4Antipatterns of Performance Testing• Acceptance performance testing• Independent functional and performance testing• Poor tracking of testing results• Insufficient introspection of the system under test• Failover is tested in production
  • 5. Scalable eCommerce Platform Solutions4/29/2013 5PerformanceTestingToolSystemUnderTestReportWhat’s wrong?
  • 6. Scalable eCommerce Platform Solutions4/29/2013 6Performance Testing EcosystemDevelopmentEnvironmentPerformanceEnvironmentQAEnvironmentProductionEnvironmentFunctionalTestsProfiling PerformanceTestsMonitoringReleaseManagerPerformanceTest LeadPerformanceArchitectTechnicalLead
  • 7. Scalable eCommerce Platform Solutions4/29/2013 7Slave Slave Slave SlaveSlaveMasterSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentJagger Technical ArchitectureWeb UIHDFSMySQLZookeeper
  • 8. Scalable eCommerce Platform Solutions4/29/2013 8Lessons learned: Automate. Automate to scale.• Completely automated flows• Jenkins plugin for automatic deployment• Results warehousing in the database
  • 9. Scalable eCommerce Platform Solutions4/29/2013 9Lessons learned: Automate. Automate to scale.• Decision makers that can access all collected data• Session comparison• VCS-friendly configuration in plain text files
  • 10. Scalable eCommerce Platform Solutions4/29/2013 10Lessons learned: Monitoring is everything.• OS-level and JVM-level metrics• Ability to load external metrics• All metrics can be involved into automatic decisionmaking• Jagger can be used to monitor an external process:ActivityJaggerAgentJagger ClusterCI Server
  • 11. Scalable eCommerce Platform Solutions4/29/2013 11Lessons learned: Profile system under test• Performance issues investigation – a typical flow:Problem ResolvedFixPerformanceTestingPerformanceProblem Detected
  • 12. Scalable eCommerce Platform Solutions4/29/2013 12Lessons learned: Profile system under test• Sampling profiler for JVM (stacktraces via JMX)• Profiler contains intelligence to detect hot spotsPerformanceProblem DetectedProblem ResolvedFixProfilingresultsPerformanceTesting
  • 13. Scalable eCommerce Platform Solutions4/29/2013 13Lessons learned: Validate responses• Custom validators• Record results of single-user test and use for validation
  • 14. Scalable eCommerce Platform Solutions4/29/2013 14Lessons learned: Manage your data well.• Statistics from one of our accounts:• About 100 test cases• 20 scheduled test sessions weekly + ad hoc testing• 30GB of test results, more than 1500 retained sessions
  • 15. Scalable eCommerce Platform Solutions4/29/2013 15Lessons learned: Manage your data well.• Reports - PDF and web UI
  • 16. Scalable eCommerce Platform Solutions4/29/2013 16Results and RoadmapSuccessful:• Continuous testing• Monitoring• Validation• Data managementNot very successful:• Continuous robustnesstesting• Integration with productionmonitoring
  • 17. Scalable eCommerce Platform Solutions4/29/2013 17jagger@griddynamics.comContact Ushttps://jagger.griddynamics.netDistribution and Documentation