Making the most of your Test Suite

3,895 views

Published on

Published in: Technology
1 Comment
11 Likes
Statistics
Notes
  • Where can I download this slides?
    Thanks!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
3,895
On SlideShare
0
From Embeds
0
Number of Embeds
169
Actions
Shares
0
Downloads
0
Comments
1
Likes
11
Embeds 0
No embeds

No notes for slide

Making the most of your Test Suite

  1. 1. Making the most out of your test suite Eric Holscher http://ericholscher.com Djangocon.eu 2010
  2. 2. Overview » Defining your tests » Running your tests on every commit » Monitoring and deployment » Playing with test data
  3. 3. Defining your tests
  4. 4. I download your code...
  5. 5. Now I want to run your tests
  6. 6. How do I do that?
  7. 7. Ways to run Django tests » ./manage.py test <app> » ./manage.py test testapp » nosetests » setup.py test » run_tests.py » ...
  8. 8. Python has even more
  9. 9. We need a standard way to run Python tests
  10. 10. setup.py test is the answer
  11. 11. ...but it’s not ready
  12. 12. setup.py test » Part of setuptools/distribute » Will be a part of distutils2 » Django has no support for this yet » We can lead standardization efforts
  13. 13. Pony Build
  14. 14. Pony Build » Started by Titus Brown » Super simple Continuous Integration Server » Interesting part is the client » github.com/ericholscher/pony_barn
  15. 15. Define how to run tests in Python
  16. 16. Full Example class PonyBuild(BaseBuild): def __init__(self): super(PonyBuild, self).__init__() self.name = "sphinx" self.required = ['nose'] self.repo_url = 'http://bitbucket.org/birkenfeld/sphinx/' def define_commands(self): self.commands = [ pony.HgClone(self.repo_url, egg=self.get_name()), pony.BuildCommand([self.context.python, 'setup.py', 'install']), pony.TestCommand([self.context.python, 'tests/run.py']), ]
  17. 17. Supports standard ways of running tests out of the box
  18. 18. Ships with support for » ./manage.py test <app> » ./manage.py test <testapp> » nose » setup.py test
  19. 19. Simple Pony Build class PonyBuild(DjangoGitBuild): def __init__(self): super(PonyBuild, self).__init__() self.repo_url = "git://github.com/alex/django-filter.git" self.name = "django-filter" self.package_name = 'django_filters' self.installed_apps = ['django_filters', 'django_filters.tests']
  20. 20. Lets you run your build in different environments
  21. 21. Different Environments » Local Environment » Temporary Directory » Virtualenv
  22. 22. At least ship with a shell script
  23. 23. Document how to run tests
  24. 24. Continuous Integration
  25. 25. “Run your tests on every commit”
  26. 26. A fancy wrapper around a shell script
  27. 27. Without this, your tests are useless
  28. 28. Make sure you setup.py install your package!
  29. 29. --noinput is your friend
  30. 30. Use Virtualenv
  31. 31. Watch Titus’s Pycon Talk
  32. 32. Three main options » Buildbot » Hudson » Devmason
  33. 33. Hudson
  34. 34. Hudson stats » Written in Java :x » Great plugin community » Super simple to setup » Becoming Defacto » Worthwhile if you have resources
  35. 35. We use this at work and it kicks ass
  36. 36. hudson.djangoproject.com
  37. 37. Setting up your own
  38. 38. Useful Capabilities » Coverage » XML Output » PyLint » Profiling
  39. 39. Plugins » Hudson Cobertura Plugin » Hudson Violations Plugin » IRC Plugin
  40. 40. pip install coverage (Compiles a C extension)
  41. 41. Coverage.py coverage run manage.py test or coverage run 1.1.X/tests/runtests.py coverage xml -o 1.1.X/coverage.xml
  42. 42. XML Output » Gareth Rushgrove’s django-test- extensions » Requires a subclassed TestRunner » github.com/garethr/django-test- extensions » github.com/danielfm/unittest-xml- reporting
  43. 43. Example Build Bits from my_test_settings import * TEST_RUNNER = ‘test_extensions.test_runners.xmloutput’
  44. 44. Nightly Tasks » pylint » profiling
  45. 45. django-nose makes a lot of this easier
  46. 46. Devmason
  47. 47. Language Agnostic Build Reporting API
  48. 48. <3 Piston
  49. 49. devmason.com/docs
  50. 50. Integrates with Github and Bitbucket
  51. 51. Have simple API support for Python unittest and Pony Build
  52. 52. Keep your builds in the Pony Barn
  53. 53. “Free” for Python community
  54. 54. Will hopefully help standardize tests
  55. 55. github.com/ericholscher/ devmason-server
  56. 56. github.com/ericholscher/ devmason-utils
  57. 57. Monitoring & Deployment
  58. 58. “Django Kong” » “Functional Testing” » “Devops” » Uses Twill: twill.idyll.org » github.com/ericholscher/django-kong
  59. 59. Simple Example go {{ site.url }}/ code 200 find “Awesome” follow “Events” code 200 find “Today’s Events”
  60. 60. Check to see if your code works on live sites
  61. 61. Easily run your tests across all types of a site
  62. 62. Catch errors users are having but never reporting
  63. 63. Gives you a neat “feel” for your sites
  64. 64. We check all of our sites every 3 minutes
  65. 65. A neat hack allows you to test your site locally too!
  66. 66. Stuff to do » Nagios integration » “Run levels” » Better aggregates timing data » Have tests live on disk/in VCS
  67. 67. Run nightly automated tools to check your sites
  68. 68. Patu
  69. 69. Test Utils Crawler
  70. 70. Get a Dashboard!
  71. 71. Test Data
  72. 72. My Pony
  73. 73. We all run tests a lot.
  74. 74. There is valuable data about our software there.
  75. 75. CPANTS » Perl’s Effort of aggregating test data » Lots of pretty graphs
  76. 76. Cpants pivots » Operating System » Architecture » Language Versions
  77. 77. We need this for Python
  78. 78. Devmason can serve as the reporting platform
  79. 79. We have all the tech we need, just need to tie it together
  80. 80. Things to Remember
  81. 81. Document how to run your tests (in executable form)
  82. 82. Get a CI server setup
  83. 83. What you don’t measure you don’t improve
  84. 84. Get some basic monitoring for your sites functionality
  85. 85. URLS » github.com/ericholscher/django-kong » hudson-ci.org » github.com/ctb/pony-build » pypants.org » cpantesters.org » devmason.com » github.com/ericholscher/devmason-server » github.com/ericholscher/devmason-utils
  86. 86. Questions? » eric@ericholscher.com » twitter.com/ericholscher

×