Barry Jaspan, Senior Architect at Acquia presented at MassTLC's automated testing event on June 12, 2013 at Brightcove, Boston, MA

1. TestOps:
The testing side of DevOps
The testing side of DevOps
Barry Jaspan
Senior Architect
Acquia

2. Server configuration is software
• The days of manual configuration by sysadmins are over
• Puppet, Chef, and similar tools turn server config into
software
• Software needs testing
• Automated
• Fast
• Clone of the production environment
• "If it isn't tested, it doesn't work."

3. Unit vs. System tests
• Unit tests isolate individual program modules
• Injection mocks out external systems
• Problem:You can't mock out the real world and get accurate results
• System tests are end-to-end
• Exercise real servers as the app(s) will
• Problem: Reality is very messy!
• Race conditions and retries
• EC2 launch failure
• Cosmic rays
• For infrastructure, system tests are essential

4. • Basic server builds
• Infrastructure moving parts
• Backup and restore
• Message queues & worker auto-scaling
• Monitoring & alerting
• Application interaction with the infrastructure
• Force app to operate under degraded infrastructure conditions
• Server reboot and relaunch
• Release process for upgrades
• Security compliance