Acceptance tests are high-level tests to verify the completeness of user stories (end user features) for a developed product. It is said that while lower level unit and integration tests confirm that the product has been made right, only the acceptance tests confirm that the product actually does the right thing. Robot Framework is a generic test automation framework for acceptance testing and acceptance test-driven development. The key feature of Robot Framework is its extensible test syntax, which allows to write tests in a business-readable domain-specific language, including BDD (behavior driven development) style for executable acceptance criteria. Selenium is a browser automation framework. It allows to use real browsers in acceptance testing so that all effects of CSS and JavaScript taken into account, including possible differences between different browsers. This talk walks through alternative ways for using Robot Framework and Selenium to perform acceptance testing, including * testing live sites (readonly) * testing sandboxed Plone sites with live data (beforestorage + demostorage) * testing sandboxed Plone add-ons (plone.app.testing + plone.app.robotframework) * testing Sphinx documentation (while generating screenshots)

1. Acceptance testing!
Plone sites and add-ons
with Robot Framework!
and Selenium
Asko Soukka, asko.soukka@iki.fi
University of Jyväskylä, Finland

2. Hello!
+ Asko Soukka
Twitter: @datakurre
IRC: datakurre
Email: asko.soukka@iki.fi
Web: datakurre.pandala.org
+ Software developer at
University of Jyväskylä
Finland
+ Plone Foundation Member

3. Menu
Terminology!
+ Acceptance testing
+ Robot framework
+ Selenium
!
Acceptance testing Plone!
+ Just do it with a live instance
+ Do it safely with a dedicated instance
+ Do it correctly in a p.a.testing sandbox
+ Do it with while writing your
documentation with screenshots
!

4. Acceptance testing
Functional testing of user stories!
!
!
Scenario: As an editor I can translate a document
Given a site owner
and a document in English
When I translate the document into Catalan
Then I can view the document in Catalan

5. Acceptance tests
!
Integration tests
!
!
!
Unit tests
!
!

6. Robot Framework
!
Keyword driven testing in your own
domain specific language!
!
Extend with Python, Java or XML-RPC
$ pip install robotframework
$ pybot —-help
$ pybot my_test_suite.robot

7. Robot Framework
*** Settings ***
!
Library string WITH NAME string
!
*** Test Cases ***
!
Convert string to uppercase
${value} = string.upper twist and shout!
Should be equal ${value} TWIST AND SHOUT!

8. !
*** Settings ***
!
Library ...
Resource ...
!
*** Variables ***
${NAME} value
!
*** Test Cases ***
Scenario title
Some keyword argument ${NAME}
!
*** Keywords ***
!
Some keyword
[Arguments] ${arg1} ${arg2}
Another keyword ${arg1} ${arg2}
!
!
test_suite.robot!

11. Selenium
Browser automation framework!
!
All browser and service logos and trademarks on this slide are property of their respective owners.

12. Selenium2Library
Selenium2 (WebDriver) keywords!
for Robot Framework!
$ pip install robotframework-selenium2library
$ pybot —-help
$ pybot my_selenium_test_suite.robot

13. Selenium2Library
*** Settings ***
!
Library Selenium2Library
!
*** Test Cases ***
!
Front-page should contain search box
Open browser https://www.jyu.fi/
Element should be visible
... css=form[name='searchform'] input
[Teardown] Close all browsers
!

15. Testing Plone with live data
[buildout]
parts += testinstance
!
[testinstance]
recipe = plone.recipe.zope2instance
...
eggs =
...
zc.beforestorage
Products.PrintingMailHost
...
before-storage = now
demo-storage = true
shared-blob = false

16. Testing Plone add-ons
*** Settings ***
!
Resource plone/app/robotframework/selenium.robot
Library Remote ${PLONE_URL}/RobotRemote
!
*** Test Cases ***
!
Scenario: Powered by Plone remains visible
Open test browser
Go to ${PLONE_URL}
Page should contain Powered by Plone
[Teardown] Close all browsers

17. Testing Plone add-ons
import unittest
!
import robotsuite
from myproduct.testing import MYPRODUCT_ROBOT_TESTING
from plone.testing import layered
!
!
def test_suite():
suite = unittest.TestSuite()
suite.addTests([layered(
robotsuite.RobotTestSuite(’test_hello.robot'),
layer=MYPRODUCT_ROBOT_TESTING)])
return suite

18. Testing Plone add-ons
from plone.testing import z2
from plone.app.testing import FunctionalTesting
from plone.app.robotframework.testing import
REMOTE_LIBRARY_BUNDLE_FIXTURE
!
MYPRODUCT_ROBOT_TESTING = FunctionalTesting(
bases=(MYPRODUCT_FIXTURE,
REMOTE_LIBRARY_BUNDLE_FIXTURE,
z2.ZSERVER_FIXTURE),
name="MyProduct:Robot")

19. Testing Plone add-ons
from plone.testing import z2
from plone.app.testing import FunctionalTesting
from plone.app.robotframework.testing import
REMOTE_LIBRARY_BUNDLE_FIXTURE
!
MYPRODUCT_ROBOT_TESTING = FunctionalTesting(
bases=(MYPRODUCT_FIXTURE,
REMOTE_LIBRARY_BUNDLE_FIXTURE,
z2.ZSERVER_FIXTURE),
name="MyProduct:Robot")

20. Testing Plone add-ons
http://docs.plone.org/search.html?q=robotframework
extras_require={
'test': [
...
'plone.app.robotframework',
],
},

21. To BDD or not to…
...
Scenario: As an editor I can translate a document
Given a site owner
and a document in English
When I translate the document into Catalan
Then I can view the document in Catalan
!
*** Keywords ***
!
A site owner
Enable autologin as Site Administrator
...

22. Testing Plone with live data
!
eggs =
...
plone.app.robotframework
collective.monkeypatcher
...
zcml-additional =
<configure
xmlns="http://namespaces.zope.org/zope"
xmlns:monkey="http://namespaces.plone.org/monkey">
<include package="collective.monkeypatcher" />
<monkey:patch
description="Enable Robot Framework remote library"
class="Products.CMFPlone.Portal.PloneSite"
original="RobotRemote"
replacement="plone.app.robotframework.testing.RobotRemote"
ignoreOriginal="true"
/>
</configure>
REVISITED!

23. Sphinx integration
extensions = ['sphinxcontrib_robotframework']
.. code:: robotframework
!
*** Settings ***
!
Library string WITH NAME string
!
*** Test Cases ***
!
Convert string to uppercase
...

25. $ pip install Sphinx
$ pip install sphinxcontrib-robotframework
$ pip install robotframework-selenium2library
.. figure:: ploneconf.png
.. code:: robotframework
!
*** Settings ***
!
Library Selenium2Library
!
*** Test Cases ***
!
Show Plone Conference 2014 website
Open browser http://2014.ploneconf.org/
Capture page screenshot ploneconf.png
[Teardown] Close all browsers

27. !
.. figure:: portal-footer.png
.. code:: robotframework
!
*** Settings ***
!
Resource plone/app/robotframework/server.robot
Resource Selenium2Screenshots/keywords.robot
!
*** Test Cases ***
!
Scenario: Powered by Plone remains visible
Setup Plone site
... plone.app.testing.PLONE_ZSERVER
Go to ${PLONE_URL}
Page should contain Powered by Plone
Capture and crop page screenshot
... portal-footer.png id=portal-footer
[Teardown] Run keywords
... Tear down Plone site
... Close all browsers
!

30. More
+ Robot Framework
http://robotframework.org/
+ Selenium2Library (for Robot Framework)
https://github.com/rtomac/robotframework-selenium2library
+ Overview of Plone-related Robot ecosystem
http://datakurre.pandala.org/2013/09/meet-robot-family-
for-plone-developers.html