Django Testing

15,799 views

Published on

Talk about where Testing is in Django, and how to get started doing it.

Published in: Business, Technology
2 Comments
50 Likes
Statistics
Notes
No Downloads
Views
Total views
15,799
On SlideShare
0
From Embeds
0
Number of Embeds
807
Actions
Shares
0
Downloads
409
Comments
2
Likes
50
Embeds 0
No embeds

No notes for slide

Django Testing

  1. Django Testing Eric Holscher http://ericholscher.com Tuesday, May 5, 2009 1
  2. How do you know? • First 4 months of my job was porting and testing Ellington • Going from Django r1290 to Django 1.0. • Suite from 0 to 400 tests. Tuesday, May 5, 2009 2
  3. 30,000 Ft View • State of testing in Django • Why you should be testing • How you start testing • Useful tools • Eventual Goals Tuesday, May 5, 2009 3
  4. State of Django Testing Tuesday, May 5, 2009 4
  5. assertTrue('Hello World', community.testing.status) Tuesday, May 5, 2009 5
  6. Django 1.1 Making Testing Possible since 2009 Tuesday, May 5, 2009 6
  7. manage.py startapp creates a tests.py Tuesday, May 5, 2009 7
  8. from django.test import TestCase class SimpleTest(TestCase): def test_basic_addition(self): quot;quot;quot; Tests that 1 + 1 always equals 2. quot;quot;quot; self.failUnlessEqual(1 + 1, 2) __test__ = {quot;doctestquot;: quot;quot;quot; Another way to test that 1 + 1 is equal to 2. >>> 1 + 1 == 2 True quot;quot;quot;} Tuesday, May 5, 2009 8
  9. Fast Tests (Transactions) Tuesday, May 5, 2009 9
  10. Minutes (Lower is better) Ellington Test Speedup 60 45 30 15 0 Django 1.0 Django 1.1 Tuesday, May 5, 2009 10
  11. You now have no excuse. Tuesday, May 5, 2009 11
  12. Why to test Tuesday, May 5, 2009 12
  13. Scary Tuesday, May 5, 2009 13
  14. Less Scary Tuesday, May 5, 2009 14
  15. Not Scary Tuesday, May 5, 2009 15
  16. Peace of Mind Tuesday, May 5, 2009 16
  17. Code must adapt Tuesday, May 5, 2009 17
  18. “It is not the strongest of the species that survives, nor the most intelligent, but the one most responsive to change.” - Charles Darwin Tuesday, May 5, 2009 18
  19. Won’t somebody please think of the users?! Tuesday, May 5, 2009 19
  20. Tests as Documentation Tuesday, May 5, 2009 20
  21. Tests as Documentation Test driven development + Document driven development = Test Driven Documentation Tuesday, May 5, 2009 20
  22. “Code without tests is broken as designed” - Jacob Kaplan-Moss Tuesday, May 5, 2009 21
  23. Dizzying Array of Testing Options Tuesday, May 5, 2009 22
  24. What kind of test? • doctest • unittest Tuesday, May 5, 2009 23
  25. Doctests • Inline documentation • <Copy from terminal to test file> • Can’t use PDB • Hide real failures • “Easy” Tuesday, May 5, 2009 24
  26. def parse_ttag(token, required_tags): quot;quot;quot; A function to parse a template tag. It sets the name of the tag to 'tag_name' in the hash returned. >>> from test_utils.templatetags.utils import parse_ttag >>> parse_ttag('super_cool_tag for my_object as obj', ['as']) {'tag_name': u'super_cool_tag', u'as': u'obj'} >>> parse_ttag('super_cool_tag for my_object as obj', ['as', 'for']) {'tag_name': u'super_cool_tag', u'as': u'obj', u'for': u'my_object'} quot;quot;quot; bits = token.split(' ') tags = {'tag_name': bits.pop(0)} for index, bit in enumerate(bits): bit = bit.strip() if bit in required_tags: if len(bits) != index-1: tags[bit] = bits[index+1] return tags Tuesday, May 5, 2009 25
  27. Unit Tests • Use for everything else • More rubust • setUp and tearDown • Standard (XUnit) Tuesday, May 5, 2009 26
  28. import random import unittest class TestRandom(unittest.TestCase): def setUp(self): self.seq = range(10) def testshuffle(self): # make sure the shuffled sequence does not lose any elements random.shuffle(self.seq) self.seq.sort() self.assertEqual(self.seq, range(10)) if __name__ == '__main__': unittest.main() Tuesday, May 5, 2009 27
  29. Django TestCase • Subclasses unittest • Fixtures • Assertions • Mail • URLs Tuesday, May 5, 2009 28
  30. Test Client • Test HTTP Requests without server • Test Views, Templates, and Context Tuesday, May 5, 2009 29
  31. from django.contrib.auth.models import User from django.test import TestCase from django.core import mail class PasswordResetTest(TestCase): fixtures = ['authtestdata.json'] urls = 'django.contrib.auth.urls' def test_email_not_found(self): quot;Error is raised if the provided email address isn't currently registeredquot; response = self.client.get('/password_reset/') self.assertEquals(response.status_code, 200) response = self.client.post('/password_reset/', {'email': 'not_a_real_email@email.com'}) self.assertContains(response, quot;That e-mail address doesn't have an associated user accountquot;) self.assertEquals(len(mail.outbox), 0) Tuesday, May 5, 2009 30
  32. What flavor of test? • Unit • Functional • Browser Tuesday, May 5, 2009 31
  33. Unit test • Low level tests • Small, focused, exercising one bit of functionality Tuesday, May 5, 2009 32
  34. Regression test • Written when you find a bug • Proves bug was fixed • Django Tickets Tuesday, May 5, 2009 33
  35. Functional • “Black Box Testing” • Check High Level Functionality Tuesday, May 5, 2009 34
  36. Functional Testing Tools • Twill • Django Test Client • ... Tuesday, May 5, 2009 35
  37. Ellington • Lots of different clients • Need to test deployment (Functional) Tuesday, May 5, 2009 36
  38. go {{ site.url }}/marketplace/search/ formvalue 1 q pizza submit code 200 Tuesday, May 5, 2009 37
  39. Tuesday, May 5, 2009 38
  40. Pretty Functional Tests Tuesday, May 5, 2009 39
  41. Can test • All sites on a server • All sites of a certain type • A single problemed client • A test across all sites Tuesday, May 5, 2009 40
  42. Browser tests • Run tests in a web browser • Check compatibility of design • Basically an IE sanity check • Only real way to test JS, AJAX, CSS Tuesday, May 5, 2009 41
  43. Browser Testing Tools • Windmill • Selenium Tuesday, May 5, 2009 42
  44. Tuesday, May 5, 2009 43
  45. Other kinds of testing • Spiders • Fuzz testing • Load testing • Prayer Tuesday, May 5, 2009 44
  46. Where do I start? Tuesday, May 5, 2009 45
  47. Fixed a bug Tuesday, May 5, 2009 46
  48. Poking at code on the command line Tuesday, May 5, 2009 47
  49. Pony turned Horse Tuesday, May 5, 2009 48
  50. Now what? Tuesday, May 5, 2009 49
  51. Start with a regression test or a functional test Tuesday, May 5, 2009 50
  52. Use unittest unless you have a reason not to! Tuesday, May 5, 2009 51
  53. Use the data, Luke • Use data as a pivot • Fixtures means you use Unit Tests • Creation on the command line, Doctests Tuesday, May 5, 2009 52
  54. Creating Fixtures • ./manage.py dumpdata <app> • ./manage.py makefixture Model[x:y] • Follows relations • By Hand Tuesday, May 5, 2009 53
  55. TestShell ./manage.py testshell <fixture> Tuesday, May 5, 2009 54
  56. Making Functional tests • Usually a relatively annoying process • Testmaker makes it easy. • ./manage.py testmaker <app> • Simply browse and your session is recorded. Tuesday, May 5, 2009 55
  57. Testing your views generally gets you the most coverage. Tuesday, May 5, 2009 56
  58. 80% Case Tuesday, May 5, 2009 57
  59. When > Where Tuesday, May 5, 2009 58
  60. Tools Tuesday, May 5, 2009 59
  61. Coverage Tuesday, May 5, 2009 60
  62. Tuesday, May 5, 2009 61
  63. Mock Objects • http://www.voidspace.org.uk/python/mock/ Tuesday, May 5, 2009 62
  64. import unittest from mock import Mock from templatetags.cms_tags import if_link_is_active, IsActiveNode class TestIsActiveTag(unittest.TestCase): def test_returns_correct_node_type(self): token = Mock(methods=['split_contents']) token.split_contents.return_value = ('if_link_is_active', 'bar') self.assertEqual(type(if_link_is_active(Mock(), token)), IsActiveNode) Tuesday, May 5, 2009 63
  65. Custom Test Runners Tuesday, May 5, 2009 64
  66. Django Test Extensions • Gareth Rushgrove • Extra Assertions • Coverage and XML Test Runners • http://github.com/garethr/django-test-extensions Tuesday, May 5, 2009 65
  67. Django Sane Testing • Ella Folk, Lucas (Hi!) • Based on nosetests • Selenium • Live server • http://devel.almad.net/trac/django-sane-testing/ Tuesday, May 5, 2009 66
  68. Django Test Utils • Mine! • Testmaker • Crawler • Random fanciness • http://github.com/ericholscher/django-test-utils/tree/ master Tuesday, May 5, 2009 67
  69. Recording tests is generally seen as bad. Tuesday, May 5, 2009 68
  70. My philosophy • Write tests. • Notice patterns and best practices • Automate recording of tests with those patterns • If you can’t automate, use tools to make it easier. Tuesday, May 5, 2009 69
  71. Process for testmaker • Most view tests check status_code and response context • Write middleware that catches this info • Records it to a file • Err on the side of more data. Tuesday, May 5, 2009 70
  72. Goals (That perfect world) Tuesday, May 5, 2009 71
  73. Some form of TDD • Write tests as you write code • Makes your code easy to test Tuesday, May 5, 2009 72
  74. Follow Django’s Model • Tests with every commit • Docs with every commit • Run tests before commiting Tuesday, May 5, 2009 73
  75. Use a DVCS • At work we have a central SVN repo • Git feature branches • Code is staged for documentation and testing • Committed to SVN once it is “done” Tuesday, May 5, 2009 74
  76. Continuous Integration Tuesday, May 5, 2009 75
  77. NEVER LEAVE THE BUILD BROKEN Tuesday, May 5, 2009 76
  78. Love Green Tuesday, May 5, 2009 77
  79. Fast(er) Tests Tuesday, May 5, 2009 78
  80. JSON Tuesday, May 5, 2009 79
  81. Profiling python -m cProfile manage.py test Tuesday, May 5, 2009 80
  82. Mock Objects http://www.voidspace.org.uk/python/mock/ Tuesday, May 5, 2009 81
  83. Future and Ponies Tuesday, May 5, 2009 82
  84. Summer of Code • Test-Only Models • Coverage • Windmill tests of the admin Tuesday, May 5, 2009 83
  85. Test Suites in Django Tuesday, May 5, 2009 84
  86. Central Testing Repository Tuesday, May 5, 2009 85
  87. Nose Plugin Integration Tuesday, May 5, 2009 86
  88. Things to remember • Testing is not hard, you just have to get started. • If your code doesn’t have tests, it will be hard/ impossible to refactor • Once you have tests, you need to run them! Tuesday, May 5, 2009 87
  89. Credits • http://www.flickr.com/photos/tym/192416981/ • http://www.flickr.com/photos/seandreilinger/2459266781/ • http://www.homebabysafety.com/images/baby_crawl_suit.jpg • http://www.flickr.com/photos/pinkypigs/960572985/ Tuesday, May 5, 2009 88

×