Your SlideShare is downloading. ×
0
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Making the most of your Test Suite
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Making the most of your Test Suite

3,392

Published on

Published in: Technology
1 Comment
11 Likes
Statistics
Notes
  • Where can I download this slides?
    Thanks!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
3,392
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
1
Likes
11
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Making the most out of your test suite Eric Holscher http://ericholscher.com Djangocon.eu 2010
  • 2. Overview » Defining your tests » Running your tests on every commit » Monitoring and deployment » Playing with test data
  • 3. Defining your tests
  • 4. I download your code...
  • 5. Now I want to run your tests
  • 6. How do I do that?
  • 7. Ways to run Django tests » ./manage.py test <app> » ./manage.py test testapp » nosetests » setup.py test » run_tests.py » ...
  • 8. Python has even more
  • 9. We need a standard way to run Python tests
  • 10. setup.py test is the answer
  • 11. ...but it’s not ready
  • 12. setup.py test » Part of setuptools/distribute » Will be a part of distutils2 » Django has no support for this yet » We can lead standardization efforts
  • 13. Pony Build
  • 14. Pony Build » Started by Titus Brown » Super simple Continuous Integration Server » Interesting part is the client » github.com/ericholscher/pony_barn
  • 15. Define how to run tests in Python
  • 16. Full Example class PonyBuild(BaseBuild): def __init__(self): super(PonyBuild, self).__init__() self.name = "sphinx" self.required = ['nose'] self.repo_url = 'http://bitbucket.org/birkenfeld/sphinx/' def define_commands(self): self.commands = [ pony.HgClone(self.repo_url, egg=self.get_name()), pony.BuildCommand([self.context.python, 'setup.py', 'install']), pony.TestCommand([self.context.python, 'tests/run.py']), ]
  • 17. Supports standard ways of running tests out of the box
  • 18. Ships with support for » ./manage.py test <app> » ./manage.py test <testapp> » nose » setup.py test
  • 19. Simple Pony Build class PonyBuild(DjangoGitBuild): def __init__(self): super(PonyBuild, self).__init__() self.repo_url = "git://github.com/alex/django-filter.git" self.name = "django-filter" self.package_name = 'django_filters' self.installed_apps = ['django_filters', 'django_filters.tests']
  • 20. Lets you run your build in different environments
  • 21. Different Environments » Local Environment » Temporary Directory » Virtualenv
  • 22. At least ship with a shell script
  • 23. Document how to run tests
  • 24. Continuous Integration
  • 25. “Run your tests on every commit”
  • 26. A fancy wrapper around a shell script
  • 27. Without this, your tests are useless
  • 28. Make sure you setup.py install your package!
  • 29. --noinput is your friend
  • 30. Use Virtualenv
  • 31. Watch Titus’s Pycon Talk
  • 32. Three main options » Buildbot » Hudson » Devmason
  • 33. Hudson
  • 34. Hudson stats » Written in Java :x » Great plugin community » Super simple to setup » Becoming Defacto » Worthwhile if you have resources
  • 35. We use this at work and it kicks ass
  • 36. hudson.djangoproject.com
  • 37. Setting up your own
  • 38. Useful Capabilities » Coverage » XML Output » PyLint » Profiling
  • 39. Plugins » Hudson Cobertura Plugin » Hudson Violations Plugin » IRC Plugin
  • 40. pip install coverage (Compiles a C extension)
  • 41. Coverage.py coverage run manage.py test or coverage run 1.1.X/tests/runtests.py coverage xml -o 1.1.X/coverage.xml
  • 42. XML Output » Gareth Rushgrove’s django-test- extensions » Requires a subclassed TestRunner » github.com/garethr/django-test- extensions » github.com/danielfm/unittest-xml- reporting
  • 43. Example Build Bits from my_test_settings import * TEST_RUNNER = ‘test_extensions.test_runners.xmloutput’
  • 44. Nightly Tasks » pylint » profiling
  • 45. django-nose makes a lot of this easier
  • 46. Devmason
  • 47. Language Agnostic Build Reporting API
  • 48. <3 Piston
  • 49. devmason.com/docs
  • 50. Integrates with Github and Bitbucket
  • 51. Have simple API support for Python unittest and Pony Build
  • 52. Keep your builds in the Pony Barn
  • 53. “Free” for Python community
  • 54. Will hopefully help standardize tests
  • 55. github.com/ericholscher/ devmason-server
  • 56. github.com/ericholscher/ devmason-utils
  • 57. Monitoring & Deployment
  • 58. “Django Kong” » “Functional Testing” » “Devops” » Uses Twill: twill.idyll.org » github.com/ericholscher/django-kong
  • 59. Simple Example go {{ site.url }}/ code 200 find “Awesome” follow “Events” code 200 find “Today’s Events”
  • 60. Check to see if your code works on live sites
  • 61. Easily run your tests across all types of a site
  • 62. Catch errors users are having but never reporting
  • 63. Gives you a neat “feel” for your sites
  • 64. We check all of our sites every 3 minutes
  • 65. A neat hack allows you to test your site locally too!
  • 66. Stuff to do » Nagios integration » “Run levels” » Better aggregates timing data » Have tests live on disk/in VCS
  • 67. Run nightly automated tools to check your sites
  • 68. Patu
  • 69. Test Utils Crawler
  • 70. Get a Dashboard!
  • 71. Test Data
  • 72. My Pony
  • 73. We all run tests a lot.
  • 74. There is valuable data about our software there.
  • 75. CPANTS » Perl’s Effort of aggregating test data » Lots of pretty graphs
  • 76. Cpants pivots » Operating System » Architecture » Language Versions
  • 77. We need this for Python
  • 78. Devmason can serve as the reporting platform
  • 79. We have all the tech we need, just need to tie it together
  • 80. Things to Remember
  • 81. Document how to run your tests (in executable form)
  • 82. Get a CI server setup
  • 83. What you don’t measure you don’t improve
  • 84. Get some basic monitoring for your sites functionality
  • 85. URLS » github.com/ericholscher/django-kong » hudson-ci.org » github.com/ctb/pony-build » pypants.org » cpantesters.org » devmason.com » github.com/ericholscher/devmason-server » github.com/ericholscher/devmason-utils
  • 86. Questions? » eric@ericholscher.com » twitter.com/ericholscher

×