Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

2

Share

Download to read offline

Modern Python Testing

Download to read offline

Presentation of lessons learned on Porting RestrictedPython to Python 3 compatibility using better testing environments for Python.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Modern Python Testing

  1. 1. Modern Testing Alexander Loechel PloneConf 2017 - Barcelona
  2. 2. The Goal of Testing → Produce high quality / correct software “Program testing can be used to show the presence of bugs, but never show their absence!” Edsger Dijkstra
  3. 3. Testing is about responsibility & sustainability → Path for Plone on Python 3
  4. 4. What to test ● Requirements ● Design ● Interfaces ● Code / Implementation ● Documentation ● Conventions
  5. 5. Test Design / Types V-ModelUser Requirements Acceptance Test Requirements Verification Functional Test System Design Validation System Integration Test Detailed Design Subsystem Integration Tests Software Architecture (Interfaces) Unit-Tests Coding / Implementation Classical Software Development Process / Governmental Standard
  6. 6. Open Source Development The development process in Open Source Projects work differently, but a lot of the concepts of the V-Model are reflected in other ways. Documentation → Reflects Requirements, Intentions and Design Tests reflects the requirements ⇒ TTD - Test Driven Development
  7. 7. “Testing leads to failure, and failure leads to understanding.” Burt Rutan Writing tests is mandatory for good software
  8. 8. Definitions
  9. 9. Test A test is a specific set of assertions
  10. 10. Test Case A test case is the smallest unit of testing. It checks for a specific response to a particular set of inputs.
  11. 11. Test Fixture A test fixture represents the preparation needed to perform one or more tests, and any associate cleanup actions
  12. 12. Test Suite A test suite is a collection of test cases, test suites, or both. It is used to aggregate tests that should be executed together.
  13. 13. Test Layer (plone.testing / plone.app.testing) A test layer represents the baseline for a specific test → Reflects Test-Level in V-Model ● Unit tests ● Integration tests ● Functional tests ● Acceptance tests
  14. 14. Test Runner A test runner is a component which orchestrates the execution of tests and provides the outcome to the user.
  15. 15. Test Invocation Tool A test invocation tool is a component which provides the required infrastructure to the test runner to execute the test sets.
  16. 16. Mock mock is a library for testing in Python. It allows you to replace parts of your system under test with mock objects and make assertions about how they have been used. → Test Isolation
  17. 17. Writing test / What to test ● Requirements → Acceptance tests ● Design → Functional tests ● Interfaces → Integration tests ● Code / Implementation → Unit tests ● Documentation → Test if your code examples actually works ● Conventions → Test if the convention of the project is followed (e.g. Coding Conventions)
  18. 18. Test Frameworks
  19. 19. Unittest The Python unit testing framework, sometimes referred to as “PyUnit,” is a Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in turn, a Java version of Kent’s Smalltalk testing framework. Each is the de facto standard unit testing framework for its respective language. unittest supports test automation, sharing of setup and shutdown code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework. The unittest module provides classes that make it easy to support these qualities for a set of tests. ● Python Standard Library module ● Used in Plone for testing ● Specific BaseClasses, and assert methods necessary, setup and teardown methods
  20. 20. Unitest example (Plone context - Plone Training Documentation) import unittest class TalkIntegrationTest(unittest.TestCase): layer = PLONECONF_SITE_INTEGRATION_TESTING def setUp(self): self.portal = self.layer['portal'] setRoles(self.portal, TEST_USER_ID, ['Manager']) def test_fti(self): fti = queryUtility(IDexterityFTI, name='talk') self.assertTrue(fti) def test_adding(self): self.portal.invokeFactory('talk', 'talk') self.assertTrue(self.portal.talk) self.assertTrue(ITalk.providedBy(self.portal.talk)) … suite = unittest.TestLoader().loadTestsFromTestCase(TalkIntegrationTest) Assert Methods Method Checks that assertEqual(a, b) a == b assertNotEqual(a, b) a != b assertTrue(x) bool(x) is True assertFalse(x) bool(x) is False assertIs(a, b) a is b assertIsNot(a, b) a is not b assertIsNone(x) x is None7 assertIsNotNone(x) x is not None assertIn(a, b) a in b assertNotIn(a, b) a not in b assertIsInstance(a, b) isinstance(a, b) assertNotIsInstance(a, b) not isinstance(a, b)
  21. 21. “The pytest framework makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.” ● De Facto Standard in the Python world ● Some magic to make writing tests simpler ● just assert ● Implicit test loader ● Plugable addon system
  22. 22. pytest example (Plone context - RestrictedPython) from tests import e_eval import pytest @pytest.mark.parametrize(*e_eval) def test_Eq(e_eval): assert e_eval('1 == 1') is True @pytest.mark.parametrize(*e_eval) def test_NotEq(e_eval): assert e_eval('1 != 2') is True @pytest.mark.parametrize(*e_eval) def test_Gt(e_eval): assert e_eval('2 > 1') is True @pytest.mark.parametrize(*e_eval) def test_Lt(e_eval): assert e_eval('1 < 2')
  23. 23. unittest2 / nose / nose2 Forks of unitest that either enhance or backport functionality Mostly outdated and not recommended anymore.
  24. 24. Robot Framework Robot Framework is a generic test automation framework for acceptance testing and acceptance test-driven development (ATDD). ● Focus: Web Applications ● Wrapper around Selenium
  25. 25. Robot tests example (Plone context - Plone Training Documentation) *** Settings *********************************************** Resource plone/app/robotframework/selenium.robot Resource plone/app/robotframework/keywords.robot Library Remote ${PLONE_URL}/RobotRemote Test Setup Open test browser Test Teardown Close all browsers *** Test Cases ********************************************* Scenario: As a site administrator I can add a Talk Given a logged-in site administrator and an add talk form When I type 'My Talk' into the title field and I type 'Awesome talk' into the details field and I type 'Team Banzai' into the speakers field and I type 'banzai@example.com' into the email field and I submit the form Then a talk with the title 'My Talk' has been created Scenario: As a site administrator I can view a Talk Given a logged-in site administrator and a talk 'My Talk' When I go to the talk view Then I can see the talk title 'My Talk' Scenario: As a visitor I can view the new talk list When I go to the talk list view Then I can see a talk about 'Diazo designs are great' *** Keywords **************************************** # --- Given ------------------------------------------ a logged-in site administrator Enable autologin as Site Administrator an add talk form Go To ${PLONE_URL}/++add++talk a talk 'My Talk' Create content type=talk id=my-talk title=My Talk # --- WHEN -------------------------------------------- I type '${title}' into the title field Input Text name=form.widgets.IDublinCore.title ${title} I type '${details}' into the details field Select frame form-widgets-details_ifr Input text tinymce ${details} Unselect Frame I type '${speaker}' into the speakers field Input Text name=form.widgets.speaker ${speaker} I type '${email}' into the email field Input Text name=form.widgets.email ${email} I submit the form Click Button Save I go to the talk view Go To ${PLONE_URL}/my-talk Wait until page contains Site Map I go to the talk list view Go To ${PLONE_URL}/demoview Wait until page contains Site Map # --- THEN ---------------------- a talk with the title '${title}' has been created Wait until page contains Site Map Page should contain ${title} Page should contain Item created I can see the talk title '${title}' Wait until page contains Site Map Page should contain ${title} I can see a talk about '${topic}' Wait until page contains Site Map Page should contain ${topic}
  26. 26. “The first principle is that you must not fool yourself - and you are the easiest person to fool.” Richard Feynman
  27. 27. Test runners ● unittest testrunner ● zope.testrunner ● pytest-testrunner ● gocept.pytestlayer A test runners is a component which orchestrates the execution of tests and provides the outcome to the user. ● collects tests ● presents outcome ● interact with other tools (e.g. coverage)
  28. 28. Test invocation tools ● Command line ● Continuous Integration Servers ● tox
  29. 29. Test invocation tools ● Command line ○ python setup.py tests ○ python -m unittest test_module.TestClass ○ python -m pytest tests ○ bin/test (zc.buildout script generated by zope.recipe.testrunner)
  30. 30. ● Continuous Integration Servers (Linux/MacOS X - Machines) perfect for pure Python tests (Docker containers, Linux/MacOS X) (Windows) Test invocation tools
  31. 31. travis-ci example (Plone context - RestrictedPython) language: python sudo: false matrix: include: - python: "2.7" env: TOXENV=docs,lint-py27 - python: "3.6" env: TOXENV=docs,lint-py36 - python: "2.7" env: TOXENV=py27 - python: "2.7" env: TOXENV=py27-datetime - python: "3.4" env: TOXENV=py34 - python: "3.5" env: TOXENV=py35 - python: "3.6" env: TOXENV=py36 - python: "3.6" env: TOXENV=py36-datetime - python: "3.7-dev" env: TOXENV=py37 - python: "pypy" env: TOXENV=pypy - python: "pypy3" env: TOXENV=pypy allow_failures: - python: "3.7-dev" env: TOXENV=py37 install: - travis_retry pip install -U pip setuptools - travis_retry pip install -U -c constraints.txt tox coveralls coverage script: - travis_retry tox after_success: - coverage combine - coveralls notifications: email: false
  32. 32. Test invocation tools ● tox - Translate the concept of continuous Integration to local development ○ De facto standard as a local test invocation tool ○ Groups environments, allow to test different Python versions support Fantastic if you use additional helpers: ● pyenv - having multiple Python version ● git-hooks - run scripts on git commands → pre-commit hook
  33. 33. tox example (Plone context - RestrictedPython) [tox] envlist = py{27,34,35,36}, docs, lint-py27, lint-py36, coverage, skip_missing_interpreters = False [testenv] usedevelop = True extras = develop commands = pytest --cov=src --cov-report=xml --html=reports/pytest/report-{envname}.html --doctest-glob=*.rst --self-contained-html {posargs} pytest --doctest-modules src/RestrictedPython/compile.py {posargs} setenv = COVERAGE_FILE=.coverage.{envname} deps = -cconstraints.txt pytest-cov pytest-remove-stale-bytecode pytest-html [testenv:coverage] basepython = python2.7 skip_install = true deps = -cconstraints.txt coverage setenv = COVERAGE_FILE=.coverage commands = coverage erase coverage combine coverage html coverage xml coverage report [testenv:docs] deps = -cconstraints.txt Sphinx commands = python -V sphinx-build -b html -d build/docs/doctrees docs build/docs/html sphinx-build -b doctest docs build/docs/doctrees
  34. 34. tox example - Linter - enforce Coding Conventions (Plone context - RestrictedPython) [lint] skip_install = true deps = -cconstraints.txt isort flake8 # helper to generate HTML reports: flake8-html # Useful flake8 plugins that are Python and Plone specific: flake8-coding flake8-debugger flake8-deprecated Flake8-print flake8-pytest flake8-todo flake8-isort mccabe # Potential flake8 plugins that should be used: Flake8-blind-except flake8-commas,flake8-docstrings Flake8-pep3101 flake8-plone-hasattr, flake8-string-format Flake8_strict flake8-quotes commands = mkdir -p {toxinidir}/reports/flake8 isort --check-only --recursive {toxinidir}/src {toxinidir}/tests setup.py - flake8 --format=html --htmldir={toxinidir}/reports/flake8 --doctests src tests setup.py flake8 src tests setup.py --doctests whitelist_externals = mkdir [testenv:lint-py27] basepython = python2.7 skip_install = true deps = {[lint]deps} commands = {[lint]commands} whitelist_externals = {[lint]whitelist_externals} [testenv:lint-py36] basepython = python3.6 skip_install = true deps = {[lint]deps} commands = {[lint]commands} whitelist_externals = {[lint]whitelist_externals}
  35. 35. The Zen of Python - PEP20 Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. ... Lessons learned from Zope and Plone → we should embrace each tool that helps us to provide a fantastic products
  36. 36. My Wishes to better “Best Practices” for Plone ● Adopt tox on all packages ● Switch to a different package structure and enforce that → bobtemplates.plone ○ docs ○ src ○ tests ● detailed and enforced settings for conventions ○ .editorconf ○ setup.cfg → https://github.com/plone/plone_best_practices_discussion
  • MonishaChitral

    Jan. 21, 2021
  • KevinNguyen100

    Jun. 19, 2018

Presentation of lessons learned on Porting RestrictedPython to Python 3 compatibility using better testing environments for Python.

Views

Total views

1,944

On Slideshare

0

From embeds

0

Number of embeds

52

Actions

Downloads

74

Shares

0

Comments

0

Likes

2

×