Django Testing
                              Eric Holscher
                         http://ericholscher.com




Tuesday, M...
How do you know?

                • First 4 months of my job was porting and testing
                       Ellington

   ...
30,000 Ft View

                • State of testing in Django
                • Why you should be testing
                •...
State of Django Testing



Tuesday, May 5, 2009                             4
assertTrue('Hello World', community.testing.status)


Tuesday, May 5, 2009                                                ...
Django 1.1

                       Making Testing Possible since 2009




Tuesday, May 5, 2009                            ...
manage.py startapp
                        creates a tests.py



Tuesday, May 5, 2009                         7
from django.test import TestCase

   class SimpleTest(TestCase):
       def test_basic_addition(self):
           quot;quo...
Fast Tests
                       (Transactions)



Tuesday, May 5, 2009                    9
Minutes (Lower is better)

                                          Ellington Test Speedup
                       60




...
You now have no excuse.



Tuesday, May 5, 2009                       11
Why to test



Tuesday, May 5, 2009                 12
Scary
Tuesday, May 5, 2009           13
Less Scary
Tuesday, May 5, 2009                14
Not Scary
Tuesday, May 5, 2009               15
Peace of Mind
Tuesday, May 5, 2009                   16
Code must adapt



Tuesday, May 5, 2009                     17
“It is not the strongest of
                       the species that survives, nor
                        the most intelli...
Won’t somebody please
                        think of the users?!



Tuesday, May 5, 2009                           19
Tests as Documentation




Tuesday, May 5, 2009                            20
Tests as Documentation

                             Test driven development
                                       +
    ...
“Code without tests is
                        broken as designed”
                        - Jacob Kaplan-Moss


Tuesday, ...
Dizzying Array of Testing
                               Options



Tuesday, May 5, 2009                               22
What kind of test?


                • doctest
                • unittest




Tuesday, May 5, 2009                        ...
Doctests

                • Inline documentation
                • <Copy from terminal to test file>
                • Can’...
def parse_ttag(token, required_tags):
       quot;quot;quot;
       A function to parse a template tag.

         It sets ...
Unit Tests

                • Use for everything else
                • More rubust
                • setUp and tearDown
 ...
import random
   import unittest

   class TestRandom(unittest.TestCase):

         def setUp(self):
             self.seq...
Django TestCase

                • Subclasses unittest
                • Fixtures
                • Assertions
           ...
Test Client


                • Test HTTP Requests without server
                • Test Views, Templates, and Context



...
from django.contrib.auth.models import User
   from django.test import TestCase
   from django.core import mail

   class ...
What flavor of test?


                • Unit
                • Functional
                • Browser




Tuesday, May 5, 20...
Unit test


                • Low level tests
                • Small, focused, exercising one bit of functionality




Tu...
Regression test


                • Written when you find a bug
                • Proves bug was fixed
                • Dja...
Functional


                • “Black Box Testing”
                • Check High Level Functionality




Tuesday, May 5, 20...
Functional Testing Tools


                • Twill
                • Django Test Client
                • ...




Tuesday,...
Ellington


                • Lots of different clients
                • Need to test deployment (Functional)




Tuesday...
go {{ site.url }}/marketplace/search/
   formvalue 1 q pizza
   submit
   code 200




Tuesday, May 5, 2009               ...
Tuesday, May 5, 2009   38
Pretty Functional Tests
Tuesday, May 5, 2009                             39
Can test

                • All sites on a server
                • All sites of a certain type
                • A single...
Browser tests

                • Run tests in a web browser
                • Check compatibility of design
              ...
Browser Testing Tools


                • Windmill
                • Selenium




Tuesday, May 5, 2009                    ...
Tuesday, May 5, 2009   43
Other kinds of testing

                • Spiders
                • Fuzz testing
                • Load testing
          ...
Where do I start?



Tuesday, May 5, 2009                       45
Fixed a bug



Tuesday, May 5, 2009                 46
Poking at code on the
                          command line



Tuesday, May 5, 2009                           47
Pony turned Horse



Tuesday, May 5, 2009                       48
Now what?




Tuesday, May 5, 2009               49
Start with a regression
                       test or a functional test



Tuesday, May 5, 2009                          ...
Use unittest unless you
                       have a reason not to!



Tuesday, May 5, 2009                             51
Use the data, Luke


                • Use data as a pivot
                • Fixtures means you use Unit Tests
           ...
Creating Fixtures

                • ./manage.py dumpdata <app>
                • ./manage.py makefixture Model[x:y]
      ...
TestShell



                       ./manage.py testshell <fixture>




Tuesday, May 5, 2009                               ...
Making Functional tests

                • Usually a relatively annoying process
                • Testmaker makes it easy...
Testing your views
                       generally gets you the
                          most coverage.


Tuesday, May 5...
80% Case



Tuesday, May 5, 2009              57
When > Where



Tuesday, May 5, 2009                  58
Tools



Tuesday, May 5, 2009           59
Coverage



Tuesday, May 5, 2009              60
Tuesday, May 5, 2009   61
Mock Objects



                • http://www.voidspace.org.uk/python/mock/




Tuesday, May 5, 2009                       ...
import unittest
   from mock import Mock

   from templatetags.cms_tags import if_link_is_active, IsActiveNode

   class T...
Custom Test Runners



Tuesday, May 5, 2009                         64
Django Test Extensions

                • Gareth Rushgrove
                • Extra Assertions
                • Coverage a...
Django Sane Testing

                • Ella Folk, Lucas (Hi!)
                • Based on nosetests
                • Selen...
Django Test Utils

                • Mine!
                • Testmaker
                • Crawler
                • Random ...
Recording tests is
                       generally seen as bad.



Tuesday, May 5, 2009                            68
My philosophy

                • Write tests.
                • Notice patterns and best practices
                • Autom...
Process for testmaker

                • Most view tests check status_code and response
                       context

  ...
Goals
                       (That perfect world)



Tuesday, May 5, 2009                          71
Some form of TDD


                • Write tests as you write code
                • Makes your code easy to test




Tues...
Follow Django’s Model


                • Tests with every commit
                • Docs with every commit
               ...
Use a DVCS

                • At work we have a central SVN repo
                • Git feature branches
                • ...
Continuous Integration



Tuesday, May 5, 2009                            75
NEVER LEAVE THE
                        BUILD BROKEN



Tuesday, May 5, 2009                     76
Love Green
Tuesday, May 5, 2009                77
Fast(er) Tests



Tuesday, May 5, 2009                    78
JSON



Tuesday, May 5, 2009          79
Profiling



                       python -m cProfile manage.py test




Tuesday, May 5, 2009                              ...
Mock Objects



                       http://www.voidspace.org.uk/python/mock/




Tuesday, May 5, 2009                  ...
Future and Ponies



Tuesday, May 5, 2009                       82
Summer of Code


                • Test-Only Models
                • Coverage
                • Windmill tests of the adm...
Test Suites in Django



Tuesday, May 5, 2009                           84
Central Testing
                        Repository



Tuesday, May 5, 2009                     85
Nose Plugin Integration



Tuesday, May 5, 2009                             86
Things to remember

                • Testing is not hard, you just have to get started.
                • If your code do...
Credits

       • http://www.flickr.com/photos/tym/192416981/
       • http://www.flickr.com/photos/seandreilinger/245926678...
Upcoming SlideShare
Loading in...5
×

Django Testing

11,764

Published on

Talk about where Testing is in Django, and how to get started doing it.

Published in: Business, Technology
2 Comments
50 Likes
Statistics
Notes
No Downloads
Views
Total Views
11,764
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
399
Comments
2
Likes
50
Embeds 0
No embeds

No notes for slide

Django Testing

  1. 1. Django Testing Eric Holscher http://ericholscher.com Tuesday, May 5, 2009 1
  2. 2. How do you know? • First 4 months of my job was porting and testing Ellington • Going from Django r1290 to Django 1.0. • Suite from 0 to 400 tests. Tuesday, May 5, 2009 2
  3. 3. 30,000 Ft View • State of testing in Django • Why you should be testing • How you start testing • Useful tools • Eventual Goals Tuesday, May 5, 2009 3
  4. 4. State of Django Testing Tuesday, May 5, 2009 4
  5. 5. assertTrue('Hello World', community.testing.status) Tuesday, May 5, 2009 5
  6. 6. Django 1.1 Making Testing Possible since 2009 Tuesday, May 5, 2009 6
  7. 7. manage.py startapp creates a tests.py Tuesday, May 5, 2009 7
  8. 8. from django.test import TestCase class SimpleTest(TestCase): def test_basic_addition(self): quot;quot;quot; Tests that 1 + 1 always equals 2. quot;quot;quot; self.failUnlessEqual(1 + 1, 2) __test__ = {quot;doctestquot;: quot;quot;quot; Another way to test that 1 + 1 is equal to 2. >>> 1 + 1 == 2 True quot;quot;quot;} Tuesday, May 5, 2009 8
  9. 9. Fast Tests (Transactions) Tuesday, May 5, 2009 9
  10. 10. Minutes (Lower is better) Ellington Test Speedup 60 45 30 15 0 Django 1.0 Django 1.1 Tuesday, May 5, 2009 10
  11. 11. You now have no excuse. Tuesday, May 5, 2009 11
  12. 12. Why to test Tuesday, May 5, 2009 12
  13. 13. Scary Tuesday, May 5, 2009 13
  14. 14. Less Scary Tuesday, May 5, 2009 14
  15. 15. Not Scary Tuesday, May 5, 2009 15
  16. 16. Peace of Mind Tuesday, May 5, 2009 16
  17. 17. Code must adapt Tuesday, May 5, 2009 17
  18. 18. “It is not the strongest of the species that survives, nor the most intelligent, but the one most responsive to change.” - Charles Darwin Tuesday, May 5, 2009 18
  19. 19. Won’t somebody please think of the users?! Tuesday, May 5, 2009 19
  20. 20. Tests as Documentation Tuesday, May 5, 2009 20
  21. 21. Tests as Documentation Test driven development + Document driven development = Test Driven Documentation Tuesday, May 5, 2009 20
  22. 22. “Code without tests is broken as designed” - Jacob Kaplan-Moss Tuesday, May 5, 2009 21
  23. 23. Dizzying Array of Testing Options Tuesday, May 5, 2009 22
  24. 24. What kind of test? • doctest • unittest Tuesday, May 5, 2009 23
  25. 25. Doctests • Inline documentation • <Copy from terminal to test file> • Can’t use PDB • Hide real failures • “Easy” Tuesday, May 5, 2009 24
  26. 26. def parse_ttag(token, required_tags): quot;quot;quot; A function to parse a template tag. It sets the name of the tag to 'tag_name' in the hash returned. >>> from test_utils.templatetags.utils import parse_ttag >>> parse_ttag('super_cool_tag for my_object as obj', ['as']) {'tag_name': u'super_cool_tag', u'as': u'obj'} >>> parse_ttag('super_cool_tag for my_object as obj', ['as', 'for']) {'tag_name': u'super_cool_tag', u'as': u'obj', u'for': u'my_object'} quot;quot;quot; bits = token.split(' ') tags = {'tag_name': bits.pop(0)} for index, bit in enumerate(bits): bit = bit.strip() if bit in required_tags: if len(bits) != index-1: tags[bit] = bits[index+1] return tags Tuesday, May 5, 2009 25
  27. 27. Unit Tests • Use for everything else • More rubust • setUp and tearDown • Standard (XUnit) Tuesday, May 5, 2009 26
  28. 28. import random import unittest class TestRandom(unittest.TestCase): def setUp(self): self.seq = range(10) def testshuffle(self): # make sure the shuffled sequence does not lose any elements random.shuffle(self.seq) self.seq.sort() self.assertEqual(self.seq, range(10)) if __name__ == '__main__': unittest.main() Tuesday, May 5, 2009 27
  29. 29. Django TestCase • Subclasses unittest • Fixtures • Assertions • Mail • URLs Tuesday, May 5, 2009 28
  30. 30. Test Client • Test HTTP Requests without server • Test Views, Templates, and Context Tuesday, May 5, 2009 29
  31. 31. from django.contrib.auth.models import User from django.test import TestCase from django.core import mail class PasswordResetTest(TestCase): fixtures = ['authtestdata.json'] urls = 'django.contrib.auth.urls' def test_email_not_found(self): quot;Error is raised if the provided email address isn't currently registeredquot; response = self.client.get('/password_reset/') self.assertEquals(response.status_code, 200) response = self.client.post('/password_reset/', {'email': 'not_a_real_email@email.com'}) self.assertContains(response, quot;That e-mail address doesn't have an associated user accountquot;) self.assertEquals(len(mail.outbox), 0) Tuesday, May 5, 2009 30
  32. 32. What flavor of test? • Unit • Functional • Browser Tuesday, May 5, 2009 31
  33. 33. Unit test • Low level tests • Small, focused, exercising one bit of functionality Tuesday, May 5, 2009 32
  34. 34. Regression test • Written when you find a bug • Proves bug was fixed • Django Tickets Tuesday, May 5, 2009 33
  35. 35. Functional • “Black Box Testing” • Check High Level Functionality Tuesday, May 5, 2009 34
  36. 36. Functional Testing Tools • Twill • Django Test Client • ... Tuesday, May 5, 2009 35
  37. 37. Ellington • Lots of different clients • Need to test deployment (Functional) Tuesday, May 5, 2009 36
  38. 38. go {{ site.url }}/marketplace/search/ formvalue 1 q pizza submit code 200 Tuesday, May 5, 2009 37
  39. 39. Tuesday, May 5, 2009 38
  40. 40. Pretty Functional Tests Tuesday, May 5, 2009 39
  41. 41. Can test • All sites on a server • All sites of a certain type • A single problemed client • A test across all sites Tuesday, May 5, 2009 40
  42. 42. Browser tests • Run tests in a web browser • Check compatibility of design • Basically an IE sanity check • Only real way to test JS, AJAX, CSS Tuesday, May 5, 2009 41
  43. 43. Browser Testing Tools • Windmill • Selenium Tuesday, May 5, 2009 42
  44. 44. Tuesday, May 5, 2009 43
  45. 45. Other kinds of testing • Spiders • Fuzz testing • Load testing • Prayer Tuesday, May 5, 2009 44
  46. 46. Where do I start? Tuesday, May 5, 2009 45
  47. 47. Fixed a bug Tuesday, May 5, 2009 46
  48. 48. Poking at code on the command line Tuesday, May 5, 2009 47
  49. 49. Pony turned Horse Tuesday, May 5, 2009 48
  50. 50. Now what? Tuesday, May 5, 2009 49
  51. 51. Start with a regression test or a functional test Tuesday, May 5, 2009 50
  52. 52. Use unittest unless you have a reason not to! Tuesday, May 5, 2009 51
  53. 53. Use the data, Luke • Use data as a pivot • Fixtures means you use Unit Tests • Creation on the command line, Doctests Tuesday, May 5, 2009 52
  54. 54. Creating Fixtures • ./manage.py dumpdata <app> • ./manage.py makefixture Model[x:y] • Follows relations • By Hand Tuesday, May 5, 2009 53
  55. 55. TestShell ./manage.py testshell <fixture> Tuesday, May 5, 2009 54
  56. 56. Making Functional tests • Usually a relatively annoying process • Testmaker makes it easy. • ./manage.py testmaker <app> • Simply browse and your session is recorded. Tuesday, May 5, 2009 55
  57. 57. Testing your views generally gets you the most coverage. Tuesday, May 5, 2009 56
  58. 58. 80% Case Tuesday, May 5, 2009 57
  59. 59. When > Where Tuesday, May 5, 2009 58
  60. 60. Tools Tuesday, May 5, 2009 59
  61. 61. Coverage Tuesday, May 5, 2009 60
  62. 62. Tuesday, May 5, 2009 61
  63. 63. Mock Objects • http://www.voidspace.org.uk/python/mock/ Tuesday, May 5, 2009 62
  64. 64. import unittest from mock import Mock from templatetags.cms_tags import if_link_is_active, IsActiveNode class TestIsActiveTag(unittest.TestCase): def test_returns_correct_node_type(self): token = Mock(methods=['split_contents']) token.split_contents.return_value = ('if_link_is_active', 'bar') self.assertEqual(type(if_link_is_active(Mock(), token)), IsActiveNode) Tuesday, May 5, 2009 63
  65. 65. Custom Test Runners Tuesday, May 5, 2009 64
  66. 66. Django Test Extensions • Gareth Rushgrove • Extra Assertions • Coverage and XML Test Runners • http://github.com/garethr/django-test-extensions Tuesday, May 5, 2009 65
  67. 67. Django Sane Testing • Ella Folk, Lucas (Hi!) • Based on nosetests • Selenium • Live server • http://devel.almad.net/trac/django-sane-testing/ Tuesday, May 5, 2009 66
  68. 68. Django Test Utils • Mine! • Testmaker • Crawler • Random fanciness • http://github.com/ericholscher/django-test-utils/tree/ master Tuesday, May 5, 2009 67
  69. 69. Recording tests is generally seen as bad. Tuesday, May 5, 2009 68
  70. 70. My philosophy • Write tests. • Notice patterns and best practices • Automate recording of tests with those patterns • If you can’t automate, use tools to make it easier. Tuesday, May 5, 2009 69
  71. 71. Process for testmaker • Most view tests check status_code and response context • Write middleware that catches this info • Records it to a file • Err on the side of more data. Tuesday, May 5, 2009 70
  72. 72. Goals (That perfect world) Tuesday, May 5, 2009 71
  73. 73. Some form of TDD • Write tests as you write code • Makes your code easy to test Tuesday, May 5, 2009 72
  74. 74. Follow Django’s Model • Tests with every commit • Docs with every commit • Run tests before commiting Tuesday, May 5, 2009 73
  75. 75. Use a DVCS • At work we have a central SVN repo • Git feature branches • Code is staged for documentation and testing • Committed to SVN once it is “done” Tuesday, May 5, 2009 74
  76. 76. Continuous Integration Tuesday, May 5, 2009 75
  77. 77. NEVER LEAVE THE BUILD BROKEN Tuesday, May 5, 2009 76
  78. 78. Love Green Tuesday, May 5, 2009 77
  79. 79. Fast(er) Tests Tuesday, May 5, 2009 78
  80. 80. JSON Tuesday, May 5, 2009 79
  81. 81. Profiling python -m cProfile manage.py test Tuesday, May 5, 2009 80
  82. 82. Mock Objects http://www.voidspace.org.uk/python/mock/ Tuesday, May 5, 2009 81
  83. 83. Future and Ponies Tuesday, May 5, 2009 82
  84. 84. Summer of Code • Test-Only Models • Coverage • Windmill tests of the admin Tuesday, May 5, 2009 83
  85. 85. Test Suites in Django Tuesday, May 5, 2009 84
  86. 86. Central Testing Repository Tuesday, May 5, 2009 85
  87. 87. Nose Plugin Integration Tuesday, May 5, 2009 86
  88. 88. Things to remember • Testing is not hard, you just have to get started. • If your code doesn’t have tests, it will be hard/ impossible to refactor • Once you have tests, you need to run them! Tuesday, May 5, 2009 87
  89. 89. Credits • http://www.flickr.com/photos/tym/192416981/ • http://www.flickr.com/photos/seandreilinger/2459266781/ • http://www.homebabysafety.com/images/baby_crawl_suit.jpg • http://www.flickr.com/photos/pinkypigs/960572985/ Tuesday, May 5, 2009 88
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×