Jagger: Сервер непрерывного тестирования производительности
Upcoming SlideShare
Loading in...5
×
 

Jagger: Сервер непрерывного тестирования производительности

on

  • 698 views

Доклад Ильи Кацова, SQA Days-13, 26-27 апреля 2013, www.sqadays.com

Доклад Ильи Кацова, SQA Days-13, 26-27 апреля 2013, www.sqadays.com

Statistics

Views

Total Views
698
Views on SlideShare
431
Embed Views
267

Actions

Likes
0
Downloads
5
Comments
0

5 Embeds 267

http://sqadays.com 199
http://www.sqadays.com 65
http://sqadays.uhus.ru 1
http://itconf.uhus.ru 1
http://sqadays-test.byte-force.yar.ru 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Jagger: Сервер непрерывного тестирования производительности Jagger: Сервер непрерывного тестирования производительности Presentation Transcript

    • Scalable eCommerce Platform SolutionsJagger: Continuous PerformanceTesting of Mission-Critical ApplicationsIlya Katsov, Grid Dynamics
    • Scalable eCommerce Platform Solutions4/29/2013 2PerformanceTestingToolSystemUnderTestReportWhat’s wrong?
    • Scalable eCommerce Platform Solutions4/29/2013 3What we do• Develop backend services for 3 of Top-30 US Retailers• Production clusters of 40 nodes and more• Load up to 50K TPS• Heavy computationsin navigational queries:
    • Scalable eCommerce Platform Solutions4/29/2013 4Antipatterns of Performance Testing• Acceptance performance testing• Independent functional and performance testing• Poor tracking of testing results• Insufficient introspection of the system under test• Failover is tested in production
    • Scalable eCommerce Platform Solutions4/29/2013 5PerformanceTestingToolSystemUnderTestReportWhat’s wrong?
    • Scalable eCommerce Platform Solutions4/29/2013 6Performance Testing EcosystemDevelopmentEnvironmentPerformanceEnvironmentQAEnvironmentProductionEnvironmentFunctionalTestsProfiling PerformanceTestsMonitoringReleaseManagerPerformanceTest LeadPerformanceArchitectTechnicalLead
    • Scalable eCommerce Platform Solutions4/29/2013 7Slave Slave Slave SlaveSlaveMasterSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentSuTSuTAgentJagger Technical ArchitectureWeb UIHDFSMySQLZookeeper
    • Scalable eCommerce Platform Solutions4/29/2013 8Lessons learned: Automate. Automate to scale.• Completely automated flows• Jenkins plugin for automatic deployment• Results warehousing in the database
    • Scalable eCommerce Platform Solutions4/29/2013 9Lessons learned: Automate. Automate to scale.• Decision makers that can access all collected data• Session comparison• VCS-friendly configuration in plain text files
    • Scalable eCommerce Platform Solutions4/29/2013 10Lessons learned: Monitoring is everything.• OS-level and JVM-level metrics• Ability to load external metrics• All metrics can be involved into automatic decisionmaking• Jagger can be used to monitor an external process:ActivityJaggerAgentJagger ClusterCI Server
    • Scalable eCommerce Platform Solutions4/29/2013 11Lessons learned: Profile system under test• Performance issues investigation – a typical flow:Problem ResolvedFixPerformanceTestingPerformanceProblem Detected
    • Scalable eCommerce Platform Solutions4/29/2013 12Lessons learned: Profile system under test• Sampling profiler for JVM (stacktraces via JMX)• Profiler contains intelligence to detect hot spotsPerformanceProblem DetectedProblem ResolvedFixProfilingresultsPerformanceTesting
    • Scalable eCommerce Platform Solutions4/29/2013 13Lessons learned: Validate responses• Custom validators• Record results of single-user test and use for validation
    • Scalable eCommerce Platform Solutions4/29/2013 14Lessons learned: Manage your data well.• Statistics from one of our accounts:• About 100 test cases• 20 scheduled test sessions weekly + ad hoc testing• 30GB of test results, more than 1500 retained sessions
    • Scalable eCommerce Platform Solutions4/29/2013 15Lessons learned: Manage your data well.• Reports - PDF and web UI
    • Scalable eCommerce Platform Solutions4/29/2013 16Results and RoadmapSuccessful:• Continuous testing• Monitoring• Validation• Data managementNot very successful:• Continuous robustnesstesting• Integration with productionmonitoring
    • Scalable eCommerce Platform Solutions4/29/2013 17jagger@griddynamics.comContact Ushttps://jagger.griddynamics.netDistribution and Documentation