Download to read offline
Pytest testmon - executing large test suites quickly
- 1. Executing large test suitesquickly Tibor Arpas tibor@infinit.sk
- 2. Your experience withtests
- 3. Users hate waiting ●100 milliseconds ● 1 second ● 10 seconds
- 4. So computing adapts ●big improvements in recent years ● 10 seconds
- 6. Vision ● instant testresults ● not feasible yet
- 7. But… ● how aboutrunning just affected tests?
- 8. So it’s uselessto call those tests ● slow ● inefficient
- 9. Transforming this ideainto a tool ● it’s pytest plugin called testmon
- 10. Example “project” witha test suite in one file
- 11.
- 12.
- 14. Here you comein ● try ● talk ● share ● fork github.com/tarpas/testm on
- 15. Contacts Tibor Arpas • pypi.python.org/pypi/testmon •github.com/tarpas/testmon • tibor@infinit.sk
Editor's Notes
- #3 how many of you did write at least one automated test? how many use nose how many use py.test (py.test test runner) how many of you have spent minutes waiting for a test suite to execute? ------ About you
- #4 This are 3 limits for user interactions. 100 ms response, that’s quick. 1 second is slow, 10 seconds, it’s broken. And the most important thing is that after 10 seconds you start doing something else. ------- Users hate waiting
- #5 computers and apps have done a good job in getting quicker in recent years There is very few activities in today’s computing where you have to wait for more than 10 seconds how about executing test suite ------ So computing adapts big improvement in recent years
- #6 It takes MINUTES 50 times slower than most of the other computer tasks intensive load on the computer (delays with other tasks, fans screaming) ------- S M R T K A
- #7 The vision Do you think about running a spell checker in a word processor? No, it’s ran automatically after each written word. Do you think about executing syntax checker in your IDE? No it’s executed after each written word automatically and instantly. I think test suite should be the same. Requires orders of magnitude improvement in efficiency Not feasible yet ----- Vision instant test results not feasible yet
- #8 if your change is inside a method, chances are the method is not executed by all tests ----- Two main ones are “incremental” run reshuffle
- #9 so it’s a waste of resources to run the whole test suite each time given the dependencies most of the test methods are not calling modified code so their result will be the same as last time -------- Majority of code changes are local waste to run the whole test suite each time
- #10 it’s called testmon it’s a pytest plugin the dependency tracking is based on coverage ------ Transforming these ideas into a tool
- #11 Example projet with a test suite in one file
- #12 Here we have grid of the dependencies.
- #15 I think this is going to be a basic feature of any IDE ----- We need your help talk share contribute on IndieGoGo igg.me/at/testmon
- #16 -------- Questions? (if time left) Contacts again Tibor tibor@infinit.sk