The Container Store uses AppDynamics in their development lifecycle to: 1) Install AppDynamics in test environments and build automated functional and performance test suites to baseline metrics and alert on deviations. 2) Empower their development and QA teams by giving them insights into the test environment and custom reports on any metrics. 3) Establish best practices like monitoring all applications, testing continuously, and performance testing early to detect issues before production and expedite remediation.

1. How The Container Store uses
AppDynamics in their development life cycle
August Azzarello, The Container Store

2. Our Presenter
August Azzarello
Software Quality Engineer
The Container Store

3. Agenda
• The Container Store
• Challenges
• Solutions
• Best Practices
• Benefits
• Q&A

4. • Leading retailer of storage and organization products
• Founded in 1978 – 77 stores nationwide
• FORTUNE magazine’s 100 Best Companies To Work For®
www.containerstore.com www.whatwestandfor.com

5. Challenges
• APM solution used as reactive monitoring tool
• Customer impacting issues already introduced
• No insight into test environment, lack of reporting
on automated testing

6. Solutions
• Install AppDynamics in test environments
• Build automated functional and performance test suites
• Baseline performance and functional metrics, alert on
deviations
• Build BT and alert policies in test

7. Transaction Score
Easily identify trends

8. Application Dashboard – Flow Map
Perform targeted testing by identifying which applications talk to each other

9. Custom Dashboards
Complete visibility into application environment

10. Compare Releases
Compare important metrics between deployments

11. Metric Browser and Custom Reports
Any metric easily becomes a custom report

12. • Selenium (Ruby) for
Functional Testing
• Locust.io (Python) for
Performance Testing
Tools Used for Automated Testing

13. BEST PRACTICES

14. Best Practices
• Monitor everything
– If you can’t see it, you can’t test or monitor it
– Make health rules in test, promote to production
– BT Maintenance
• Test continuously
– Continuous tests will help build baselines, better the ability to realize deviations on
performance and functional errors
– Dedicated automation environment
• Performance test early in development life cycle
– The earlier performance problems are detected, the less impact they will have on the release
cycle
• Empower development & QA team members
– AppDynamics can give the ability for QA members to give more in depth bug reports, forging
strong team relationships and increasing efficiency

15. Key Metrics to Monitor in Test
• Performance Testing
– Infrastructure
• CPU
• Disk
• Memory
• Network
• Functional Testing
– Application Metrics
• Errors/Exceptions
• HTTP Error Codes
• Number of Calls
– Calls to DB(s)
– Calls to backend services
– Calls to remote services
Copyright © 2014 AppDynamics. All rights reserved. 15

16. Expedite Remediation of Performance Issues
Copyright © 2014 AppDynamics. All rights reserved. 16
• Auto-discover & monitor end-to-end
business transaction performance
within minutes, with no overhead
• Self-learning platform dynamically
computes optimal app performance
• Resolve application issues quickly for
exceptional end-user experience

17. Production Readiness in Pre-Production
• Test for desirable performance in pre-
production using same tool
• Deep transaction traces and
snapshots to find code issues
• Set-up automated notification rules
for performance testing
Copyright © 2014 AppDynamics. All rights reserved. 17

18. Reduce Cost and Risk with Automated Change
Analysis
• Compare performance snapshots before
and after a major code release (or
individual bug fixes)
• Understand the performance impact of
application, server/database changes, and
with infrastructure changes
• Identify and resolve unplanned changes
proactively
Copyright © 2014 AppDynamics. All rights reserved. 18

19. Benefits
• Set performance expectations before production
deployments – ~40% improvement since we
started using AppDynamics in test
• Fine tune alert and BT policies prior to
production deployments
• Identify testing requirements, and testing gaps
• Decrease performance test result reporting from
5 hours to 20 minutes
Copyright © 2015 AppDynamics. All rights reserved. 19

20. Q&A

21. Thank You