Chaos testing involves intentionally introducing failures into a system to test its resiliency. The document discusses using tools like Vagrant, Docker, and integration tests to simulate failures like slowness, database failovers, and network issues. It introduces Chaos Gopher, a set of simple Unix/Go tools for chaos testing, and recommends improving integration test suites to better handle intermittent failures. The author is available to answer any other questions about chaos testing approaches.

1. Chaos Monkey
on my Laptop

2. What is chaos testing?

3. Matt Campbell

4. Eikon Messenger

6. Scale
300k financial traders
Every major bank around the world

7. Our testing setup

8. –Said by no one
“Sweet lets just start killing
production databases”

9. Lets learn how to crawl first

10. Vagrant
Docker
Integration Tests

11. Continuous
Integration

12. Failure integration
tests

13. Less Mocking

14. Database failovers

15. Microservice Problems

16. Performance
Downtime
Discovery

17. TheSlowNess

18. Problems
• Connection pooling in go
• Dead tcp connections
• Hung connections
• Network splits / Congestion

19. Production
integration tests

20. Chaos Gopher

21. • Unix/GO Style tools
• Simple utilities , Slowness, Failover, Assassin

22. Where Next?
Better integration test suites
Allowing for flappy tests
Statistically analysis of flapping tests

23. Other cool projects
• Chaos Monkey (the mac daddy, Netflix)
• Chaos Lemur (from Pivotal)

24. Questions?
Matthew Campbell
hyper@hyperworks.nu
@kanwisher
github.com/mattkanwisher