This document discusses testing practices for pragmatic developers. It argues that while testing is important, it should be done pragmatically by focusing on value over completeness. Unit, integration, mock and other types of tests are described along with test frameworks and best practices like test-driven development. The document emphasizes that testing needs to balance costs and value.

1. Testing for
Pragmatic People
Michael Davis
Development Leader
Plone Conference
5th November 2011

2. Theory
• Code without tests is broken code
• 100% test coverage is a minimum
• God kills a kitten every time someone checks in code
without a test
• Testing will make you more attractive to the opposite
sex
• There's a special place in hell for people who code
without tests

3. Reality
• Testing is hard
• Expensive
• Slow to run
• Maintaining
• Test framework keeps changing
• Refactoring
• Tests wrong

4. Confused Much?
http://www.f ickr.com/photos/39585662@N00/5331407243
l

5. Realism
• Tests are not mandatory
• However, they are ...
• Advocated
• Best practice
• Helpful
• Eminently sensible

6. Testing is a
tool

7. Cost/Value
Value of Test
Cost of Test
Testing Experience

8. What do we mean
by Tests
• Unit tests
• Integration tests
• Mock tests
• Doc tests
• Regression tests

9. Unit tests
• Much faster than integration tests
• For testing a small element of the code base
• Testing individual methods
• getToolByName is always a problem
• Get complex quickly
• Expensive to maintain

10. Doc tests
• Primarily for describing functionality using python
calls
• Fails at first failure
• Difficult to write
• Good for documentation

11. Regression Tests
• Not actually a test
• About when and why

12. Mock Testing
• Using mock objects to run tests
• Useful in unit testing to mock required objects
• Can mock getToolByName to return the right tool
• Used to test the untestable

13. Integration Tests
• Often called unit tests
• Full Plone instance setup

14. What about Test
Driven
Development

15. Mindsets
• Writing code - how
• Writing test – what
• Effects versus Mechanics

16. Keep It Simple
• Minimise effort
• Maximise reuse

17. Test Framework
class TestCase(PloneSandboxLayer):
defaultBases = (PLONE_FIXTURE,)
def setUpZope(self, app, confgurationContext):
import my.product
self.loadZCML(package=my.product)
z2.installProduct(app, PROJECTNAME)
def setUpPloneSite(self, portal):
self.applyProfle(portal, '%s:default' % PROJECTNAME)
def tearDownZope(self, app):
z2.uninstallProduct(app, PROJECTNAME)

18. Test Framework
from plone.app.testing import PloneSandboxLayer
from plone.app.testing import PLONE_FIXTURE
from plone.app.testing import IntegrationTesting
from plone.testing import z2
from my.product.confg import PROJECTNAME
class TestCase(PloneSandboxLayer):
....
FIXTURE = TestCase()
INTEGRATION_TESTING = IntegrationTesting(bases=(FIXTURE,),
name="fxture:Integration")

19. Simple Test
class TestReinstall(unittest.TestCase):
"""Ensure product can be reinstalled safely"""
layer = INTEGRATION_TESTING
def setUp(self):
self.portal = self.layer['portal']
def testReinstall(self):
portal_setup = getToolByName(self.portal, 'portal_setup')
portal_setup.runAllImportStepsFromProfle('profle-%s:default' %
PROJECTNAME)

20. Running Tests
bin/test -s my.product
Ran 2 tests with 0 failures and 0 errors in 1.050 seconds.
Tearing down left over layers:
Tear down collective.wfform.tests.base.fixture:Integration in 0.000 seconds.
Tear down collective.wfform.tests.base.TestCase in 0.013 seconds.
Tear down plone.app.testing.layers.PloneFixture in 0.380 seconds.
Tear down plone.testing.z2.Startup in 0.024 seconds.
Tear down plone.testing.zca.LayerCleanup in 0.006 seconds.

21. Browser Layer
class TestInstallation(unittest.TestCase):
"""Ensure product is properly installed"""
layer = INTEGRATION_TESTING
def setUp(self):
self.portal = self.layer['portal']
def testBrowserLayerRegistered(self):
sm = getSiteManager(self.portal)
layers = [o.__name__ for o in registered_layers()]
assert 'IMyProduct' in layers

22. Skin Layer
class TestInstallation(unittest.TestCase):
"""Ensure product is properly installed"""
layer = INTEGRATION_TESTING
def setUp(self):
self.portal = self.layer['portal']
def testSkinLayersInstalled(self):
assert 'my_skin' in self.portal.portal_skins.objectIds()
assert 'my_template' in self.portal.portal_skins.my_skin.objectIds()

23. Portal Type
class TestInstallation(unittest.TestCase):
"""Ensure product is properly installed"""
layer = INTEGRATION_TESTING
def setUp(self):
self.portal = self.layer['portal']
def testTypesInstalled(self):
portal_types = getToolByName(self.portal, 'portal_types')
assert 'MyType' in portal_types.objectIds(), portal_types.objectIds()

24. Portal Types
class TestContentType(unittest.TestCase):
"""Test content type"""
layer = INTEGRATION_TESTING
def setUp(self):
self.portal = self.layer['portal']
def testAddType(self):
setRoles(self.portal, TEST_USER_ID, ['Manager'])
self.portal.invokeFactory('MyType', 'mt1')
mt1 = getattr(self.portal, 'mt1')
mt1.setTitle('The Title of My Object')
Assert mt1.Title() == 'The Title of My Object', mt1.Title()

25. Schema
class TestContentSchema(unittest.TestCase):
"""Test content type schema"""
layer = INTEGRATION_TESTING
def setUp(self):
self.portal = self.layer['portal']
setRoles(self.portal, TEST_USER_ID, ['Manager'])
self.portal.invokeFactory('MyType', 'mt1')
self.mt1 = getattr(self.portal, 'mt1')
def testSchema(self):
schema = self.mt1.schema
feld_ids = schema.keys()
assert 'referenceNumber' in feld_ids

26. Form Validation
class TestValidation(unittest.TestCase):
"""Test validation"""
layer = INTEGRATION_TESTING
def setUp(self):
self.portal = self.layer['portal']
setRoles(self.portal, TEST_USER_ID, ['Manager'])
self.portal.invokeFactory('MyType', 'mt1')
self.mt1 = getattr(self.portal, 'mt1')
app = makerequest(self.app)
app.REQUEST.form['name'] = 'Michael Davis'

27. Validation Test
def testValidates(self):
dummy_controller_state = ControllerState(
id='my_form',
context=s1,
button='submit',
status='success',
errors={},
next_action=None,)
controller = self.portal.portal_form_controller object
controller_state = controller.validate(dummy_controller_state,
app.REQUEST, ['my_validate',])
assert controller_state.getErrors() == {}, controller_state.getErrors()

28. Test Patterns
def testValidatePrisoner(self):
app.REQUEST.form['name'] = 6
…
assert len(controller_state.getErrors()) == 1
assert controller_state.getErrors().has_key('name')
assert controller_state.getErrors()['name'] == 'You are not a number'

29. Test Patterns
def testValidateClint(self):
app.REQUEST.form['name'] = ''
…
assert len(controller_state.getErrors()) == 1
assert controller_state.getErrors().has_key('name')
assert controller_state.getErrors()['name'] == 'You are not the man with no
name'

30. CI Server

31. Use your tools
responsibly
http://www.f ickr.com/photos/12803689@N02/4402962654
l