This document summarizes the testing approach and strategies used for an application built with Django and Django Rest Framework. It discusses the use of both unit and integration tests, along with different testing frameworks like Django's built-in test tools, pytest, and libraries like pytest-django and pytest-xdist. It also covers strategies like database fixtures, parameterized testing, and mocking HTTP requests. The document concludes by discussing plans to improve testing in the future such as enabling parallel test runs and adding performance regression testing.

2. About me
● Professional Software
Engineer since June 2010
● Django user since
February 2018
● Software Engineering
Manager at
● SyDjango co-organiser
since August 2019
@tyomo4ka

3. XCore Project

4. XCore
myTrip.com
GoToGate
Trip.ru
XClaim
XCMS
XCover.com
Xtranet
BrightWrite
Partners

5. Dates
● Development started in Feb 2018
● Live since June 2019
Platform
● Python 3.6-3.7
● Django 2.0–2.2
● DRF 3.9-3.10
● 49,520 LOC
● 826 tests run in ~52 sec on Bitbucket
pipelines
● 86.9% code coverage
XCore Facts

6. v1.0
early days

7. Django
● Extends unittest
● Django helpers: client, settings, db fixtures, etc
● Extended and Django-specific assertions

8. Django
● django.test.SimpleTestCase – assertions, client, Django settings
● django.test.TransactionTestCase – reset db, fixtures, more
assertions
● django.test.TestCase – wraps each test in atomic block
● django.test.LiveServerTestCase – starts a real server

9. Django Rest Framework (DRF)
● Replicates Django’s test cases
● Replaces default client with APIClient
● RequestsAPIClient testing against an external server

10. ● Unit tests are used to test the smallest bits of the application logic in
isolation from the rest of the application
● Integration testing is used to test larger parts of the application
which requires application communication with external services
(db, files, queues, etc)
Unit vs integration

11. Unit vs integration
● The same testing frameworks is usually used for both
● In most projects there is no way to run unit tests in separation from
integration tests

12. Unit
Integration
API
Cost
Number of tests on the project

13. Typical unit test scenario
1. Prepare SUT (System Under the Test)
○ could be skipped in many cases in others it’s as simple as manipulating with
objects in memory
○ achieve test isolation using mocking framework
○ no real IO calls (DB, queues, HTTP)
2. Run SUT logic
3. Perform assertions
○ typically using data returned back to the test

14. Typical integration test scenario
1. Prepare SUT
○ Reset DB, queues
○ Load fixtures – DB, files, configs, etc
○ Best practice is to avoid calling external (from the app perspective) systems – use
mock servers or mock Python IO calls
2. Run SUT logic
3. Perform assertions
○ typically checking the state of the system (DB, files, etc)

15. Unit vs integration in Django
● Always base your TestCase class of SimpleTestCase when writing
unit tests
● For integration and API tests use TestCase unless you really need to
test transaction.on_commit logic in which case you need to use
TransactionTestCase
● For UI tests use LiveServerTestCase

16. DB Fixtures (Django)
● Use dumpdata cli command to generate fixtures using data stored
in the DB
● Import in TestCases fixtures class property

17. DB Fixtures (factory_boy)
● Simple inheritance (inc. traits)
● Generated values: sequences, lazy attributes, faker, fuzzy attributes
● Native integration with Django models
● Non-db objects (JSON structures, POPO objects, etc)

18. v2.0
pre-production and early days in
production

19. pytest
● Run tests in parallel
● More flexibility around running tests
● Nicer output

20. pytest
● pytest --lf or pytest --ff
● pytest -k
● pytest --pdb
● markers, hooks, fixtures, etc
● plugins

21. assert magic

22. pytest-django
● --reuse-db
● set of useful django helpers

23. pytest-xdist
● run tests in parallel
● pytest -n auto or pytest -n ${NUM}
● pytest --looponfail
(run tests on remote servers via ssh)

24. Tests run > 50%
faster with
pytest-xdist

25. pytest-xdist

26. pytest-sugar
● instafail
● nice progressbar formatting

27. Parameterized tests (pytest)

28. Parameterized tests (ddt)

29. Parameterized tests (ddt)

30. HTTP requests mocking (responses)

31. pyvcr

32. pyvcr

33. freezegun

34. V3.0
scaling (future)

35. Plans for the future
● Small cleanups: replace responses with vcr everywhere, small test
refactoring
● Split test suite so we can run unit tests independently
● Smoke suite: migrate from Postman to pytest (or behave?)
● Add Hypothesis framework in some critical places
● Performance optimisation project: performance regression testing

36. Before we wrap up

37. Quality is everyone’s responsibility

38. Tests MUST run automatically in CI pipeline

39. End-to-end tests are important

40. Writing tests before code sometimes is helpful

41. Testing is the way not a way

42. But sometimes skipping testing is ok

43. What is not covered in this talk?
● Postman tests (monitors, smoke suite)
● Performance tests with Locust
● XFT building another Django app for testing insurance price
● UI testing with Selenium
● Behavioral testing with behave

44. Thanks for listening!
Feedback is welcome
Please reach me on Twitter @tyomo4ka