Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Integration Group - Robot Framework

Robot Framework is a keyword driven automated testing framework.

  • Login to see the comments

Integration Group - Robot Framework

  1. 1. Created by Jan Medved www.opendaylight.org Getting Started with Robot Framework Carol Sanders, Brocade
  2. 2. Created by Jan Medved www.opendaylight.org  Robot Framework is a Keyword Driven automation framework using Python or Java as it's backend.  Test cases are automated by writing steps using Robot framework keywords.  Provides a simple library API for creating customized test libraries.  Provides a command line interface and XML based outputs for integration into existing build infrastructure (continuous integration systems).  Report files are created as a result of every pybot execution: report.html, log.html, output.xml.  Implemented with Python – Runs also on Jython (JVM) and IronPython (.NET) – Can be extended natively using Python or Java – Other languages supported via a remote interface  Open source – Apache 2.0 license – Active development and growing community What is Robot Framework 2
  3. 3. Created by Jan Medved www.opendaylight.org Test Suite Organization 3
  4. 4. Created by Jan Medved www.opendaylight.org Integration CSIT Organization 4 git clone https://git.opendaylight.org/gerrit/integration cd- ./integration/test/ Two directories CSIT – contains the test suites, resource files containing keyword definitions particular to your project tests, and testplans which are all needed to successfully run your test suite. Tools – test scripts specifically written that test features CSIT Directories (you will always put your files here) Libraries Suites Testplans Variables
  5. 5. Created by Jan Medved www.opendaylight.org Viewing Test Suites in Robot Framework Ride 5
  6. 6. Created by Jan Medved www.opendaylight.org Collections of test cases are called test suites. Test suites are made of files and directories. 1. Now that you have cloned the integration repository, you have the integration test infrastructure needed to begin creating your test suite. 2. Open ride on the command line – ride.py& 3. Open directory to Integration test suites directory -- ./integration/tests/csit/suites 4. Create a new directory. This where your test suite and associated files will be located. Use the name of your project as the directory name. 5. Create a new test suite within the directory just created. 6. Include all elements described in Test Suite Contents: Settings, Keywords, Resource file(s), Variables, Documentation, Tags, Log, Setup, Teardown 7. Follow best practices when writing test cases Creating Robot Framework Tests Suites 6
  7. 7. Created by Jan Medved www.opendaylight.org Integration Sample Test Suite 7
  8. 8. Created by Jan Medved www.opendaylight.org Sample Test Case Format 8 *** Settings *** Documentation Test suite for RESTCONF inventory Suite Setup Create Session session http://${CONTROLLER}:${RESTCONFPORT} auth=${AUTH} headers=${HEADERS_XML} Suite Teardown Delete All Sessions Library Collections Library ../../../libraries/RequestsLibrary.py Library ../../../libraries/Common.py Variables ../../../variables/Variables.py Resource ../../../libraries/Utils.txt *** Variables *** ${REST_CONTEXT} /restconf/operational/opendaylight-inventory:nodes @{node_list} openflow:1 openflow:2 openflow:3 *** Test Cases *** Get list of nodes [Documentation] Get the inventory [Tags] restconf Log ${start} Wait Until Keyword Succeeds 30s 2s Ensure All Nodes Are In Response ${REST_CONTEXT} ${node_list} Get nodeconnector for a node 1 [Documentation] Get the inventory for a node [Tags] restconf Log ${start} ${resp} Get session ${REST_CONTEXT}/node/openflow:1 Should Be Equal As Strings ${resp.status_code} 200 Should Contain ${resp.content} openflow:1:1 Should Contain ${resp.content} openflow:1:2 A test case file, for our purposes, contains the following default sections: Settings, Variables, Test Cases, Setup and Teardown. Setup and Teardown usually defined in resources file or initialization file.
  9. 9. Created by Jan Medved www.opendaylight.org Initialization File Format 9 *** Settings *** Documentation Flow test suite for the OpenDaylight karaf-compatible feature set Suite Setup Start Suite Suite Teardown Stop Suite Resource ../../../libraries/Utils.txt *** Variables *** *** Keywords *** [Teardown] This section is used to shutdown systems, close connections, and cleanup configurations.
  10. 10. Created by Jan Medved www.opendaylight.org Test Suite Contents 10 Recommended Test Suite Contents  Setup  Settings  Resource file  Keywords  Documentation  Log  Tags  Test case step(s)  Tear Down
  11. 11. Created by Jan Medved www.opendaylight.org Test Suite Contents 11  A test case file can contain other configuration information to ease the processing and speed results such as specifying variables, path to external keywords or python functions or other configuration data.  When a test directory is executed, the files and directories it contains are processed recursively as follows:  Files and directories with names starting with a dot (.) or an underscore (_) are ignored.  Directories with the name CVS are ignored (case-sensitive).  Files not having one of the recognized extensions (.html, .xhtml, .htm, .tsv, .txt, .rst, or .rest) are ignored (case-insensitive).  Other files and directories are processed.  If a file or directory that is processed does not contain any test cases, it is silently ignored (a message is written to the syslog) and the processing continues.
  12. 12. Created by Jan Medved www.opendaylight.org 12 Test execution is normally started using pybot, jybot or ipybot runner script. These scripts are otherwise identical, but the first one executes tests using Python, the second using Jython, and the last one using IronPython. Regardless of execution approach, the path (or paths) to the test data to be executed is given as an argument after the command. Additionally, different command line options can be used to alter the test execution or generated outputs in some way. Specifying test data to be executed Robot Framework test cases are created in files and directories, and they are executed by giving the path to the file or directory in question to the selected runner script. The path can be absolute or, more commonly, relative to the directory where tests are executed from. The given file or directory creates the top-level test suite, which gets its name, unless overridden with the --name option, from the file or directory name. Example cli command to execute test cases / suites pybot test_cases.html | pybot path/to/my_tests/ | pybot c:robottests.txt It is also possible to give paths to several test case files or directories at once, separated with spaces. In this case, Robot Framework creates the top-level test suite automatically, and the specified files and directories become its child test suites. The name of the created test suite is derived from child suite names by concatenating them together with an ampersand (&) and spaces. These automatically created names are often quite long and complicated. In most cases, it is thus better to use the --name option for overriding it: pybot my_tests.html your_tests.html pybot --name Example path/to/tests/pattern_*.html Using options Robot Framework provides a number of command line options that can be used to control how test cases are executed and what outputs are generated. When options are used, they must always be given between the runner script and the data sources. For example: pybot -L debug my_tests.txt pybot --include smoke --variable HOST:10.0.0.42 path/to/tests/ Test Suite Execution
  13. 13. Created by Jan Medved www.opendaylight.org Short and long options Options always have a long name, such as --name, and the most frequently needed options also have a short name, such as -N. In addition to that, long options can be shortened as long as they are unique. For example, --logle DEBUG works, while -- lo log.html does not, because the former matches only --loglevel, but the latter matches several options. Short and shortened options are practical when executing test cases manually, but long options are recommended in start-up scripts, because they are easier to understand. The long option format is case-insensitive, which facilitates writing option names in an easy-to-read format. For example, -- SuiteStatLevel is equivalent to, but easier to read than --suitestatlevel. Setting option values Most of the options require a value, which is given after the option name. Both short and long options accept the value separated from the option name with a space, as in --include tag or -i tag. With long options, the separator can also be the equals sign, for example --include=tag, and with short options the separator can be omitted, as in -itag. Some options can be specified several times. For example, --variable VAR1:value --variable VAR2:another sets two variables. If the options that take only one value are used several times, the value given last is effective. Simple patterns Many command line options take arguments as simple patterns. These glob-like patterns are matched according to the following rules: •* is a wildcard matching any string, even an empty string. •? is a wildcard matching any single character. •Unless noted otherwise, pattern matching is case, space, and underscore insensitive. Examples: --test Example* # Matches tests with name starting 'Example', case insensitively. --include f?? # Matches tests with a tag that starts with 'f' or 'F' and is three characters long. Test Suite Execution 13
  14. 14. Created by Jan Medved www.opendaylight.org Test Suite Reporting 14 Robot outputs three files detailing the results of the test case / suite run ● Output.xml Output files contain all the test execution results in machine readable XML format. Log, report and xUnit files are typically generated based on them, and they can also be combined and otherwise post-processed with Rebot. ● Report.html Report files contain an overview of the test execution results in HTML format. They have statistics based on tags and executed test suites, as well as a list of all executed test cases. When both reports and logs are generated, the report has links to the log file for easy navigation to more detailed information. It is easy to see the overall test execution status from report, because its background color is green, if all critical tests pass, and bright red otherwise. The command line option --report (-r) determines where report files are created. Similarly as log files, reports are always created unless NONE is used as a value, and their default name is report.html. ● Log.html Log files contain details about the executed test cases in HTML format. They have a hierarchical structure showing test suite, test case and keyword details. Log files are needed nearly every time when test results are to be investigated in detail. Even though log files also have statistics, reports are better for getting an higher-level overview. The command line option --log (-l) determines where log files are created. Unless the special value NONE is used, log files are always created and their default name is log.html.
  15. 15. Created by Jan Medved www.opendaylight.org Robot Test Suite Log File Example Test Suite Report 15
  16. 16. Created by Jan Medved www.opendaylight.org Summary 16 Test Suite and Test Case Automation Best Practices ● Document each test case. Explain the purpose of the test. ● Use the Log function to return useful information about keyword usage and test case runs. ● Only test one attribute of the feature in a test case instead of many attributes. ● Every test case should have a tag to enable/disable test case run. ● Use ‘wait until’ instead of sleep when waiting for a process to complete. ● Use Resource files if your keywords span more than one test case and when using user created keywords. ● Use an initialization file ‘__init_.txt to start services or systems needed for the tests. ● It is better to use variables instead of static assignment. Make you test suite more portable and dynamic. ● Keep your test case Clear and concise. ● Clean up the test environment after each test suite run. ● Setup and Teardown sections in test case can be used for this purpose. ● Define global variables in a variables file.
  17. 17. Created by Jan Medved www.opendaylight.org Open Daylight Integration Reference https://wiki.opendaylight.org/view/CrossProject:Integration_Group:Test_Tools Robot Integrate Dev Environment (RIDE) https://code.google.com/p/robotframework-ride/ https://github.com/robotframework/RIDE/wiki https://code.google.com/p/robotframework-ride/wiki/InstallationInstructions Robot Guides http://robotframework.org/ https://code.google.com/p/robotframework/ http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#executing- test-cases Extending Robot http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#extending- robot-framework http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.4#test-data- syntax How to write good test cases: https://code.google.com/p/robotframework/wiki/HowToWriteGoodTestCases References 17

×