0
Token Testing Talk
                  Eric Holscher
       http://ericholscher.com
               Djangocon 2009
Testing Talks per Conference
10.0


 7.5


 5.0


 2.5


  0
         Pycon    RailsConf   DjangoCon
How do you know?


» First 4 months of my job was porting and
  testing Ellington
» Going from Django r1290 to Django 1.0....
30,000 Ft View


» State of testing in Django
» Why you should be testing
» How you start testing
» Useful tools
» Eventua...
State of Django Testing
assertTrue('Hello World',
community.testing.status)
Django 1.1




Making Testing Possible since 2009
manage.py startapp creates a
         tests.py
Basic Test
from django.test import TestCase

class SimpleTest(TestCase):
    def test_basic_addition(self):
        """
  ...
Fast Tests
(Transactions)
Minutes (Lower is better)



             Ellington Test Speedup
60


45


30


15


0
     Django 1.0                    ...
You now have no
    excuse.
Why to test
Scary
Less Scary
Not Scary
Peace of Mind
Code must adapt
“It is not the strongest of
the species that survives, nor
 the most intelligent, but the
   one most responsive to
      ...
Won’t somebody please think of
         the users?!
Tests as Documentation
Tests as Documentation


   Test driven development
              +
Document driven development
              =
 Test Driv...
“Code without tests is
  broken as designed
 - Jacob Kaplan-Moss
Dizzying Array of Testing
        Options
What kind of test?




» doctest

» unittest
Doctests



» Inline documentation

» <Copy from terminal to test file>

» Easy
Awesome Documentation

def parse_ttag(token, required_tags):
    """
    A function to parse a template tag.

   It sets t...
Awesome Documentation



>>> from test_utils.templatetags.utils import parse_ttag
>>> parse_ttag('super_cool_tag for my_ob...
Doctest problems




» Can’t use PDB

» Hide real failures
Unit Tests



» Standard (XUnit)

» More robust

» setUp and tearDown
Basic Unit Test
import random
import unittest

class TestRandom(unittest.TestCase):

    def setUp(self):
        self.seq...
Django TestCase


» Subclasses unittest

» Fixtures

» Assertions

» Mail

» URLs
Test Client




» Test HTTP Requests without server

» Test Views, Templates, and Context
Django’s TestCase

from django.contrib.auth.models import User
from django.test import TestCase
from django.core import ma...
What flavor of test?



» Unit

» Functional

» Browser
Unit test


» Low level tests

» Small, focused, exercising one bit of
  functionality
» Great for libraries
Regression test



» Written when you find a bug

» Proves bug was fixed

» Django Tickets
Functional




» ‘Black Box Testing’

» Check High Level Functionality
Functional Testing Tools




» Twill

» Django Test Client
Browser tests


» Run tests in a web browser

» Check compatibility of design

» Basically an IE sanity check

» Only real...
Browser Testing Tools




» Windmill

» Selenium
Other kinds of testing



» Spiders

» Fuzz testing

» Load testing
Where do I start?
Use unittest unless you have a
        reason not to!
Start with a regression test or a
         functional test
Fixed a bug
Poking at code on the command
              line
Pony turned Horse
Use the data, Luke



» Use data as a pivot

» Fixtures means you use Unit Tests

» Creation on the command line, Doctests
Creating Fixtures


»./manage.py dumpdata <app>
»./manage.py makefixture Model[x:y]
  » Follows relations
  » Slicing
»By ...
Getting in Context




» ./manage.py testshell <fixture>

» (i)pdb
Making Functional tests


» Usually a relatively annoying process

» Testmaker makes it easy.

» ./manage.py testmaker [ap...
Testing your views gets you the
        most coverage.
80% Case
When > Where
Enough to be useful
Tools
Summer of Code


» Test-Only Models

» Skipping tests

» Coverage

» Windmill tests of the admin
Test-Only Models



» Allow for models available only during
  tests
» Don’t need test_project now!
Test-Only Models

#test_models.py

from django.db import models

class TestModel(models.Model):
    name = models.CharFiel...
Skipping Tests


» Views Required

» Models Required

» Specific database type

» Conditional on a Function
Skipping Tests

from django.tests.decorators import conditional_skip
import datetime

class TestUnderCondition(TestCase):
...
Skipping Tests


======================================================================
SKIPPED: test_email_found (django....
Coverage



» Basically the hotness

» django-admin.py test --coverage

» django-admin.py test --coverage --report
Basic Coverage
Pretty Coverage
Pretty Coverage
Windmill Tests


» Didn’t get finished

» Allow for functional tests of the admin

» Allow for custom Windmill tests of yo...
Custom Test Runners



» Allow you to do anything you want

» Load custom fixtures

» nose or py.test runners
Django Test Extensions


» Gareth Rushgrove

» Extra Assertions

» Coverage and XML Test Runners

» http://github.com/gare...
Django Sane Testing

» Ella Folks

» Based on nosetests

» Selenium

» Live server

» http://devel.almad.net/trac/django-
...
Django Test Utils

» Mine!

» Testmaker

» Crawler

» Random fanciness

» http://github.com/ericholscher/django-
  test-ut...
Goals
Some form of TDD




» Write tests as you write code

» Makes your code easy to test
Follow Django’s Model



» Tests with every commit

» Docs with every commit

» Run tests before commiting
Continuous Integration
NEVER LEAVE THE BUILD
       BROKEN
Love Green
Profiling



» python -m cProfile manage.py test

» Allows you to profile a repeatable case

» Faster tests = Faster produ...
Things to remember
Testing is not hard, you just
    have to get started.
If your code doesn’t have tests, it
    will be hard/impossible to
              refactor
Once you have tests, you need to
          run them!
Teh Pretty




» Thanks to Idan Gazit for design help
Credits


» http://www.flickr.com/photos/tym/
  192416981/
» http://www.flickr.com/photos/
  seandreilinger/2459266781/
» ...
Questions?



» twitter.com/ericholscher

» http://ericholscher.com

» eric@ericholscher.com
Token  Testing Slides
Upcoming SlideShare
Loading in...5
×

Token Testing Slides

6,727

Published on

A rehash of my previous Django testing talk with a bit of content added.

Published in: Technology, Business
0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
6,727
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
112
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide

Transcript of "Token Testing Slides"

  1. 1. Token Testing Talk Eric Holscher http://ericholscher.com Djangocon 2009
  2. 2. Testing Talks per Conference 10.0 7.5 5.0 2.5 0 Pycon RailsConf DjangoCon
  3. 3. How do you know? » First 4 months of my job was porting and testing Ellington » Going from Django r1290 to Django 1.0. » Suite from 0 to 400 tests. (Now has 1715)
  4. 4. 30,000 Ft View » State of testing in Django » Why you should be testing » How you start testing » Useful tools » Eventual Goals
  5. 5. State of Django Testing
  6. 6. assertTrue('Hello World', community.testing.status)
  7. 7. Django 1.1 Making Testing Possible since 2009
  8. 8. manage.py startapp creates a tests.py
  9. 9. Basic Test from django.test import TestCase class SimpleTest(TestCase): def test_basic_addition(self): """ Tests that 1 + 1 always equals 2. """ self.failUnlessEqual(1 + 1, 2) __test__ = {"doctest": """ Another way to test that 1 + 1 is equal to 2. >>> 1 + 1 == 2 True """}
  10. 10. Fast Tests (Transactions)
  11. 11. Minutes (Lower is better) Ellington Test Speedup 60 45 30 15 0 Django 1.0 Django 1.1
  12. 12. You now have no excuse.
  13. 13. Why to test
  14. 14. Scary
  15. 15. Less Scary
  16. 16. Not Scary
  17. 17. Peace of Mind
  18. 18. Code must adapt
  19. 19. “It is not the strongest of the species that survives, nor the most intelligent, but the one most responsive to change. - Charles Darwin
  20. 20. Won’t somebody please think of the users?!
  21. 21. Tests as Documentation
  22. 22. Tests as Documentation Test driven development + Document driven development = Test Driven Documentation
  23. 23. “Code without tests is broken as designed - Jacob Kaplan-Moss
  24. 24. Dizzying Array of Testing Options
  25. 25. What kind of test? » doctest » unittest
  26. 26. Doctests » Inline documentation » <Copy from terminal to test file> » Easy
  27. 27. Awesome Documentation def parse_ttag(token, required_tags): """ A function to parse a template tag. It sets the name of the tag to 'tag_name' in the hash returned. >>> from test_utils.templatetags.utils import parse_ttag >>> parse_ttag('super_cool_tag for my_object as obj', ['as']) {'tag_name': u'super_cool_tag', u'as': u'obj'} >>> parse_ttag('super_cool_tag for my_object as obj', ['as', 'for']) {'tag_name': u'super_cool_tag', u'as': u'obj', u'for': u'my_object'} """ bits = token.split(' ') tags = {'tag_name': bits.pop(0)} for index, bit in enumerate(bits): bit = bit.strip() if bit in required_tags: if len(bits) != index-1: tags[bit] = bits[index+1] return tags
  28. 28. Awesome Documentation >>> from test_utils.templatetags.utils import parse_ttag >>> parse_ttag('super_cool_tag for my_object as obj', ['as']) {'tag_name': u'super_cool_tag', u'as': u'obj'} >>> parse_ttag('super_sweet for object as obj', ['as', 'for']) {'tag_name': u'super_sweet', u'as': u'obj', u'for': u'object'}
  29. 29. Doctest problems » Can’t use PDB » Hide real failures
  30. 30. Unit Tests » Standard (XUnit) » More robust » setUp and tearDown
  31. 31. Basic Unit Test import random import unittest class TestRandom(unittest.TestCase): def setUp(self): self.seq = range(10) def testshuffle(self): # shuffled sequence does not lose any elements random.shuffle(self.seq) self.seq.sort() self.assertEqual(self.seq, range(10)) if __name__ == '__main__': unittest.main()
  32. 32. Django TestCase » Subclasses unittest » Fixtures » Assertions » Mail » URLs
  33. 33. Test Client » Test HTTP Requests without server » Test Views, Templates, and Context
  34. 34. Django’s TestCase from django.contrib.auth.models import User from django.test import TestCase from django.core import mail class PasswordResetTest(TestCase): fixtures = ['authtestdata.json'] urls = 'django.contrib.auth.urls' def test_email_not_found(self): "Error is raised if the provided email address isn't currently registered" response = self.client.post('/password_reset/', {'email': 'not_a_real_email@email.com'}) self.assertEquals(len(mail.outbox), 0)
  35. 35. What flavor of test? » Unit » Functional » Browser
  36. 36. Unit test » Low level tests » Small, focused, exercising one bit of functionality » Great for libraries
  37. 37. Regression test » Written when you find a bug » Proves bug was fixed » Django Tickets
  38. 38. Functional » ‘Black Box Testing’ » Check High Level Functionality
  39. 39. Functional Testing Tools » Twill » Django Test Client
  40. 40. Browser tests » Run tests in a web browser » Check compatibility of design » Basically an IE sanity check » Only real way to test JS, AJAX, CSS » Slow
  41. 41. Browser Testing Tools » Windmill » Selenium
  42. 42. Other kinds of testing » Spiders » Fuzz testing » Load testing
  43. 43. Where do I start?
  44. 44. Use unittest unless you have a reason not to!
  45. 45. Start with a regression test or a functional test
  46. 46. Fixed a bug
  47. 47. Poking at code on the command line
  48. 48. Pony turned Horse
  49. 49. Use the data, Luke » Use data as a pivot » Fixtures means you use Unit Tests » Creation on the command line, Doctests
  50. 50. Creating Fixtures »./manage.py dumpdata <app> »./manage.py makefixture Model[x:y] » Follows relations » Slicing »By Hand
  51. 51. Getting in Context » ./manage.py testshell <fixture> » (i)pdb
  52. 52. Making Functional tests » Usually a relatively annoying process » Testmaker makes it easy. » ./manage.py testmaker [app] » Simply browse and your session is recorded.
  53. 53. Testing your views gets you the most coverage.
  54. 54. 80% Case
  55. 55. When > Where
  56. 56. Enough to be useful
  57. 57. Tools
  58. 58. Summer of Code » Test-Only Models » Skipping tests » Coverage » Windmill tests of the admin
  59. 59. Test-Only Models » Allow for models available only during tests » Don’t need test_project now!
  60. 60. Test-Only Models #test_models.py from django.db import models class TestModel(models.Model): name = models.CharField() #tests.py from django.test import TestCase class TestMyViews(TestCase): test_models = ['test_models'] def testIndexPageView(self): from myapp.models import TestModel TestModel.objects.get(name='daniellindsleyrocksdahouse')
  61. 61. Skipping Tests » Views Required » Models Required » Specific database type » Conditional on a Function
  62. 62. Skipping Tests from django.tests.decorators import conditional_skip import datetime class TestUnderCondition(TestCase): def _check_2009(): # Condition returning True if test should be run and False if it # should be skipped. if datetime.datetime.now() > datetime.datetime(2009, 01, 01): return True @conditional_skip(_check_2009, reason='This test only runs in 2009') def testOnlyIn2009(self): # Test to run if _my_condition evaluates to True
  63. 63. Skipping Tests ====================================================================== SKIPPED: test_email_found (django.contrib.auth.tests.basic.PasswordResetTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/dnaquin/Dropbox/Sandbox/django/django/test/decorators.py", line 43, in _skip raise SkippedTest(reason=reason) SkippedTest: Required view for this test not found: django.contrib.auth.views.password_reset ---------------------------------------------------------------------- Ran 408 tests in 339.663s FAILED (failures=1, skipped=2)
  64. 64. Coverage » Basically the hotness » django-admin.py test --coverage » django-admin.py test --coverage --report
  65. 65. Basic Coverage
  66. 66. Pretty Coverage
  67. 67. Pretty Coverage
  68. 68. Windmill Tests » Didn’t get finished » Allow for functional tests of the admin » Allow for custom Windmill tests of your apps
  69. 69. Custom Test Runners » Allow you to do anything you want » Load custom fixtures » nose or py.test runners
  70. 70. Django Test Extensions » Gareth Rushgrove » Extra Assertions » Coverage and XML Test Runners » http://github.com/garethr/django-test- extensions
  71. 71. Django Sane Testing » Ella Folks » Based on nosetests » Selenium » Live server » http://devel.almad.net/trac/django- sane-testing/
  72. 72. Django Test Utils » Mine! » Testmaker » Crawler » Random fanciness » http://github.com/ericholscher/django- test-utils/tree/master
  73. 73. Goals
  74. 74. Some form of TDD » Write tests as you write code » Makes your code easy to test
  75. 75. Follow Django’s Model » Tests with every commit » Docs with every commit » Run tests before commiting
  76. 76. Continuous Integration
  77. 77. NEVER LEAVE THE BUILD BROKEN
  78. 78. Love Green
  79. 79. Profiling » python -m cProfile manage.py test » Allows you to profile a repeatable case » Faster tests = Faster production
  80. 80. Things to remember
  81. 81. Testing is not hard, you just have to get started.
  82. 82. If your code doesn’t have tests, it will be hard/impossible to refactor
  83. 83. Once you have tests, you need to run them!
  84. 84. Teh Pretty » Thanks to Idan Gazit for design help
  85. 85. Credits » http://www.flickr.com/photos/tym/ 192416981/ » http://www.flickr.com/photos/ seandreilinger/2459266781/ » http://www.homebabysafety.com/images/ baby_crawl_suit.jpg
  86. 86. Questions? » twitter.com/ericholscher » http://ericholscher.com » eric@ericholscher.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×