Intro to Testing in Zope, Plone

Intro to Testing in Zope, Plone Andriy Mylenkyy © Quintagroup, 2008

What we'll talk about

Testing intro

unittest

doctest

zope.testing

Zope Test Cases

Plone Test Cases

Testing intro

Problem

Types of test

Unit tests

Integration tests

Functional tests

System tests

Testing and documentation

unittest

Author – Steve Purcell

Actually is PyUnit, which is part of the Python 2.1 standard library

based on Erich Gamma's JUnit and Kent Beck's Smalltalk testing framework.

unittest.py TestSuite TestCase TC TC TC TestSuite(TS)

unittest: TestCase

testMethodName

result

fail*/assert*

(failUnless|assert)Raises

run/debug

__call__

run(result) result.startTest() result.stopTest() testMethod() setUp() tearDown() debug

unittest: module members

TestSuite

_test (addTest, addTests)

run/debug (__call__)

TestLoader

define test method prefix [' test ']

sort test method function [ cmp ]

Methods :

loadTestsFrom* [TestCase, Module, Name, Names]

getTestCaseNames -> use test method prefix for find test methods -> used for build TestCase instances

unittest: module members (cont.)

TextTestRunner

TestProgram

FunctionTestCase

TestResult

Help functions :

makeSuite ,

findTestCase ,

getTestCaseNames

unittest: TestSuite / TestCase import unittest as ut class WidgetTC(ut.TestCase): def setUp(self): ... def tearDown(self): ... def testDefaultSize(self): assertEqual(...) def testResize(self): self.failIf(...) def otherTest(self): self.assertRaises(...) if __name__ == '__main__': # 1 # ut.main() #1b# ut.main(defaultTest='WidgetTC') #2,3# suite = ut.TestSuite() # 2 # suite.addTest(ut.makeSuite(WidgetTC)) # 3 # suite.addTest('WidgetTC('otherTest')) #2,3# runner = ut.TextTestRunner() #2,3# runner.run(sute) TestSuite _tests WidgetTC("testDefaultSize") WidgetTC("testResize") unittest.makeSuite(WidgetTC)

unittest: command line mytest.py ---------------------------------------------- import unittest class WidgetTC(unittest.TestCase): ... if __name__ == '__main__': unittest.main() ----------------------------------------------- $ python mytest.py .. OK ------------------------------------------------- $ python mytest.py WidgetTC.testDefaultSize WidgetTC.testResize .. OK ------------------------------------------------- $ python mytest.py -v testDefaultSize (__main__.WidgetTC) ... ok testResize (__main__.WidgetTC) ... ok ... OK

doctest

Authors: Jim Fulton, Edward Loper

part of the Python standard library since version 2.1.

import types, mymodule, doctest def hasInt(*args): """ Test is integer value present in args. >>> hasInt("the string") False >>> hasInt("the string", 1) True """ args = list(args) while args: if type(args.pop()) == types.IntType: return True return False __test__ = {'My module test' : mymodule, 'My string test' : “””>>>1==1 True“””} if __name__ == '__main__': doctest.testmod() doctest: bases

Python shell with explanations

Docstrings

Separate file

__test__ in module

doctest: testmod/testfile

testmod ()

m= None

globs= None

optionflags= 0

verbose=None, report=True, raise_on_error=False

testfile (filename)

module_relative=True, package=None

parser=DocTestParser()

same to testmod

run_docstring_examples (f, globs)

compileflags=None

doctest: Option flags

COMPARISON_FLAGS

DONT_ACCEPT_TRUE_FOR_1, DONT_ACCEPT_BLANKLINE, NORMALIZE_WHITESPACE, ELLIPSIS, IGNORE_EXCEPTION_DETAIL

REPORTING_FLAGS

REPORT_ONLY_FIRST_FAILURE, REPORT_*DIFF

`want` strings markers

BLANKLINE_MARKER='<BLANKLINE>', ELLIPSIS_MARKER = '...'

Test ELLIPSIS option >>> range(100) #doctest:+ELLIPSIS [0, ..., 99] >>> range(100) [0, ..., 99] import doctest “”” >>> range(100) [0, ..., 99] “”” doctest.testmod( optionflags=doctest.ELLIPSIS)

doctest: Unittest Support

DocTestSuite

Common options: module, globs, extraglobs, test_finder

**options: setUp, tearDown, globs, optionflags

DocFileTest(path)

module_relative=True, package=None

set_unittest_reportflags. Only reporting

“”” >>> range(100) [0, ..., 99] “”” import doctest, unittest if __name__ == “__main__”: suite = unittest.TestSuite() testrunner = unittest.TextTestRunner() testrunner.run(doctest.DocTestSuite(optionflags=doctest.ELLIPSIS)) # testrunner.run(doctest.DocTestSuite())

doctest: Internal structure object DocTes t Example Example Example result DocTestFinder DocTestRunner

zope.testing

testrunner for run automated tests

primary feature - it finds tests in directory trees

highly customizable

Test filtering

Organization of tests into levels and layers

Reporting

Analysis of test execution

zope.testing: Lyaers

Minimize the number of invocations

TestCase/TestSuite associate with layer

Classes with setUp and tearDown class methods

Nested Layers

testSetUp/testTearDown

-l --layer=<layer name>

“Unit” layer (-u) / not unit (-f)

setUp tearDown setUp tearDown TestCase TestCase TestCase

zope.testing: Lyaers (cont) class BaseLayer : @classmethod def setUp (cls): “ Layer setup ” @classmethod def tearDown (cls): “ Layer teardown ” @classmethod def testSetUp (cls): “ Before each test setup ” @classmethod def testTearDown (cls): “ After each test teardown ” class MyTestCase(TestCase): layer = NestedLayer def setUp(self): pass def tearDown(self): pass class NestedLayer( BaseLayer) : @classmethod def setUp (cls): “ Layer setup ” @classmethod def tearDown (cls): “ Layer teardown ” testrunner.py --path', directory_with_tests --layer 112 --layer Unit'.split

zope.testing: Levels

Additional filter – alow run tests for specific level

TestCase/TestInteger attribute 'level' integer >0

Test runner can be configured to only run tests at a specific level or below by default.

Additional filter – alow run tests for specific level

TestCase/TestInteger attribute 'level' integer >0

Test runner can be configured to only run tests at a specific level or below by default.

If no level -> assumed level 1

by default, only tests at level 1

--at-level (-a) / --all

TestCase TestCase TestCase TestCase TestCase TestCase Level 1 Level 2 TestCase TestCase TestCase Level - 3

zope.testing: Test selection

Search for

Package (-s, --package, --dir)

Module (-m, --module)

Test within the module (-t, --test )

layer (-l, --level) & level (-a, --at-level / --all )

--tests-pattern (--tests-pattern ^tests$ )

--test-suite ( ' test_suite ' by default)

Module & Test can be re.expr, positional args

zope.testing: options (cont)

Where search for tests

--path

--test-path / --package-path --> not add to sys.path

--progress (-p). -v (with test descriptions)

--coverage=<coverage_dir> - where put coverage reports

--profile

--repeat

--verbose (-v) / --quiet (-q)

zope.testing: test.py script

Wrapper around zope.testing.testrunner

Extends testrunner with options:

--config-file

--nowarnings

Prepare default options list for call with testrunner:

--path = INSTANCE_HOME/python/lib

--package-path products Products

call testrunner.run

zope.testing: examples

$ZH/bin/test.py --config-file=$IH/etc/zope.conf

--path=~/Plone/lib/python

--package-path ~/Plone/Products Products

-s Products.CMFQuickInstallerTool

$INSTANCE_HOME/bin/zopectl test

-s Products.CMFQuickInstallerTool

ZopeTestCase(ZTC)

Author : Stefan H. Holek

ZTC Goals

Simplify distribute tests with Zope products.

Be as close to “real” Zope as possible.

Be as simple to use as unittest.

Decisions

Base ZTC on Testing and improve on it

Provide a “batteries included“ Zope test sandbox

Support for testing Zope security.

ZTC: framework.py vs test.py

framework.py now depricated

bin/test.py –config-file=.../zope.conf ...

BUT:

Supports custom_zodb.py in the tests directory

Allows to connect to a running ZEO server

import os from ZODB.DemoStorage import DemoStorage from ZODB.FileStorage import FileStorage db = os.path.join('..', '..', '..', 'var', 'Data.fs') db = os.path.abspath(db) Storage = DemoStorage(base=FileStorage(db, read_only=1))

ZTC: ZopeTestCase.py

ZopeTestCase class

ZopeTestCase sets up a default fixture

Handles opening and closing of ZODB con.

Handles transactions

Provides a REQUEST object

API to modify the test user’s credentials

ZTC: Entrypoints setUp(self) try: self. beforeSetUp() self.app = self._app() self._setup() self. afterSetUp() except: self._clear() raise tearDown(self) try: self. beforeTearDown() self._clear(1) except: self._clear() raise Before ALL ZTC Folder, UserFolder, User, login After Fixture created ConnectionRegistry Before delete folder from self.app After folder delete Before close connection to ZODB beforeClose() afterClear() After ALL Zope.APP Opens a ZODB connection Request.__of__(app)

ZTC: Class diagram

ZTC: Writing Tests

Create a test case class derived from ZopeTestCase.ZopeTestCase

Override afterSetUp() / tearDown() to add the objects you want to test, clearing data

Write one or more tests exercising these objects

Define test_suite object, which return unittest.TestSuite object

ZTC: Example

import unittest

from Testing import ZopeTestCase

ZopeTestCase.installProduct(’SomeProduct’)

class MyTest(ZopeTestCase.ZopeTestCase):

def afterSetUp(self):

self.folder.addDTMLMethod(’foo’)

def testFoo(self):

assert hasattr(self.folder, ’foo’)

def test_suite():

return unittest.TestSuite(

[unittest.makeSuite(MyTest),] )

PortalTestCase

Almost ZopeTestCase

portal object (self.portal)

user folder (UF) inside the portal

portal object must create in Subclasses

getPortal() returns a usable portal object

Defaults:

user with role 'Member' inside the UF

default user's memberarea (self.folder)

default user is logged in

PortalTestCase (cont)

portal = 'portal'

setUp(self) try: self. beforeSetUp() self.app = self._app() self.portal = self._portal() self._setup() self. afterSetUp() except: self._clear() raise _portal(self) return self.getPortal() getPortal(self) return getattr(self.app, poratl)

PloneTestCase(PTC)

PloneTestCase class

Usefull methods:

hasProduct. installProduct

hasPackage, installPackage

setupPloneSite(products, extension_profiles)

Constants:

portal_*, default_*

PLONE2. x , PLONE3. x , PLONE4. x

PTC: PloneTestCase (cont)

Plone installs in setup.py module

ZCMLLayer, PloneSiteLayer(ZCML) layers

onsetup, onteardown functions.

Extend PortalTestCase security with methods:

setGroups, loginAsPortalOwner

PTC: PloneTestCase example import os, sys if __name__ == '__main__': execfile(os.path.join(sys.path[0], 'framework.py')) from Products.PloneTestCase import PloneTestCase PloneTestCase.setupPloneSite() class TestDocument(PloneTestCase.PloneTestCase): def afterSetUp(self): ... def testAddDocument(self): self.failUnless(...) def test_suite(): ... return suite if __name__ == '__main__': framework()

Resources