Tellurium.A.New.Approach.For.Web.Testing

Tellurium: A New Approach for Web Testing Jian Fang (John.Jian.Fang@gmail.com) Vivek Mongolu (vivekmongolu@gmail.com) Matt Senter (matt.senter@gmail.com) Haroon Rasheed (haroonzone@gmail.com) Quan Bui (Quan.Bui@gmail.com) Mikhail Koryak (koryak@gmail.com)

Outline Motivations  Overview Introduction Examples and Demo Future Directions Resources

What’s Wrong with Selenium • Take Tellurium project web site as an example. – The above UI includes one basic search module and one advanced search module – Use Selenium IDE and it generates the following Selenium test case

Example Selenium Test case selenium.open(quot;/p/aost/issues/advsearchquot;); selenium.select(quot;//div[@id='maincol']/form/table/tbody/tr[1]/td[3]/selectquot;, quot;label=regexp:sAll Issuesquot;); selenium.type(quot;wordsquot;, quot;Telluriumquot;); selenium.type(quot;withoutquot;, quot;corequot;); selenium.click(quot;btnquot;); selenium.waitForPageToLoad(quot;30000quot;); ... Assertions ... selenium.open(quot;/p/aost/issues/advsearchquot;); selenium.select(quot;canquot;, quot;label=regexp:sAll Issuesquot;); selenium.type(quot;qquot;, quot;Telluriumquot;); selenium.click(quot;//input[@value='Search']quot;); selenium.waitForPageToLoad(quot;30000quot;); ... Assertions ...

What do you observe? • Verbose – “Selenium.” everywhere – Locators everywhere • Not expressive – Can you see what the UI looks like? – Can you tell what the UI element that the locator is associated with? • Coupling – UI locators, actions, and assertions are coupled together • Fragile – Look at the xpath //div[@id='maincol']/form/table/tbody/tr[1]/td[3]/select • – What if the div or the table portions are changed? •

What do you observe? (Cont’d) • Record and Play – Need to cover all testing scenarios – How about data dependency? – What if the UI is changed? • Refactor – Seems to be difficult, especially if you are not the person who wrote the tests. • Reusable – Less likely • Any way to handle dynamic factors on the web? – Dynamic data such as data grid and list of options – Javascript events

How about Canoo WebTest • Canoo WebTest is an Open Source tool for automated testing of web import com.canoo.webtest.WebtestCase applications class SimpleTest extends WebtestCase { • Test cases in XML or void testWebtestOnGoogle() { Groovy webtest(quot;check that WebTest is Google's top 'WebTest' resultquot;) { • Use HtmlUnit as testing invoke quot;http://www.google.comquot;, description: quot;Go to Googlequot; driving engine verifyTitle quot;Googlequot; • Same problems as Selenium setInputField name: quot;qquot;, value: quot;WebTestquot; clickButton quot;I'm Feeling Luckyquot; Individual UI – verifyTitle quot;Canoo WebTestquot; Not robust – } Not reusable – } Test code couple with UI – } •

What Else? • Twill is a simple language that allows users to browse the Web from a command-line interface – open source and written in Python – use link, forms, cookies, and most standard Web features – has a simple Python interface • Disadvantages Deal with individual UI – Need to parse html interactively, not convenient to use – Only support Python – Simple, not powerful – setlocal query quot;twill Pythonquot; go http://www.google.com/ fv 1 q $query submit btnI # use the quot;I'm feeling luckyquot; button show

Motivations  Robust to changes  Easy to refactor Reusable Expressive Easy to use Address dynamic factors – Dynamic data – Dynamic web context – Javascript events – Propose: Tellurium Automated Testing Framework  (Tellurium)

What is Tellurium? • Tellurium is a UI module based web testing framework • Tellurium is built on top of Selenium at the current stage – May have our own tellurium test driving engine in the future • Tellurium is implemented in Groovy and Java • Tellurium includes a set of Domain Specific Languages (DSLs) to define UI, actions, and tests • Tellurium handles the mapping from Java object to UI • Tellurium provides you the capability to write tests in – Java (JUnit 4 or TestNG) – Groovy – Pure DSL script • Tellurium supports Data Driven Testing

What Tellurium looks like? • (I) Basic Search Module ui.Form(uid: quot;issueSearchquot;, clocator: [action: quot;listquot;, method: quot;getquot;], group: quot;truequot;) { TextBox(uid: quot;searchLabelquot;, clocator: [tag: quot;spanquot;]) Selector(uid: quot;issueTypequot;, clocator: [name: quot;canquot;, id: quot;canquot;]) InputBox(uid: quot;searchBoxquot;, clocator: [name: quot;qquot;]) SubmitButton(uid: quot;searchButtonquot;, clocator: [value: quot;Searchquot;]) }

What Tellurium looks like? (Cont’d) • Advanced Search Module ui.Form(uid: quot;issueAdvancedSearchquot;, clocator: [action: quot;advsearch.doquot;, method: quot;postquot;], group: quot;truequot;) { Table(uid: quot;searchTablequot;, clocator: [class: quot;advqueryquot;]){ Selector(uid: quot;row:1, column: 3quot;, clocator: [name: quot;canquot;]) SubmitButton(uid: quot;row:1, column:4quot;, clocator: [value: quot;Searchquot;, name: quot;btnquot;]) InputBox(uid: quot;row:2, column:3quot;, clocator:[name: quot;wordsquot;]) InputBox(uid: quot;row:3, column:3quot;, clocator:[name: quot;withoutquot;]) } }

What Tellurium looks like? (Cont’d) public void selectIssueType(String type){ selectByLabel quot;issueSearch.issueTypequot;, type } public void searchIssue(String issue){ keyType quot;issueSearch.searchBoxquot;, issue click quot;issueSearch.searchButtonquot; waitForPageToLoad 30000 } public void advancedSearchIssue(String issueType, String words, String without){ if(issueType != null){ selectByLabel quot;issueAdvancedSearch.searchTable[1][3]quot;, issueType } if(words != null){ keyType quot;issueAdvancedSearch.searchTable[2][3]quot;, words } if(without != null){ keyType quot;issueAdvancedSearch.searchTable[3][3]quot;, without } click quot;issueAdvancedSearch.searchTable[1][4]quot; waitForPageToLoad 30000 } Methods defined for UI modules

How challenges are addressed in Tellurium • No longer “record and play” style, but UI module oriented – defining nested UI objects and methods as a UI module class, write your own test cases as many as you like based on the UI module class. • Decoupling – UI and tests are separated – UI is defined in a Groovy UI module class – Write your own test cases in Java, Groovy, or DSL scripts • Robust to changes – Composite locator (UI element attributes) is used to define UI and the actual xpath will be generated at runtime, i.e., different attributes -> different runtime xpath. UI module internal changes are addressed. – Group locating is used to remove the dependency of the UI objects from external UI elements, i.e., external UI changes will not affect the current UI module for most cases.

How challenges are addressed in Tellurium • Easy to refactor – Refactor the individual UI module class • Reusable – UI module class could be re-used for the same application – Tellurium Widgets can be re-used for different applications • Expressive – UI module is straightforward – Groovy syntax – DSLs for actions and tests • Dynamic factors on web – UI templates for data grid or List of dynamic data – “respond” attribute in UI object for Javascript events – “Option” UI object for multiple possible UI patterns

How challenges are addressed in Tellurium • Easy to use – Tellurium provides a configuration file for you to wire in your own new UI objects/widgets and customize the framework – Test can be written in • Java • Groovy • Pure DSL script – Support JUnit 4 and TestNG – Reference projects are provided to guide users on how to use Tellurium – Support ant and Maven build • Data Driven Testing

Why Tellurium is A NEW Approach? • Tellurium comes with a set of new concepts that make Tellurium a different and new approach for Web Testing – UI Module – UID – Composite Locator – Group Locating – UI Template

New Concepts: UI Module Tellurium Automated Testing Framework is UI  module based – UI module is a collection of UI elements you group them together – UI module represents a composite UI object in the format of nested basic UI elements – The UI elements inside the composite UI object have relationship to each other, i.e., logically or to be on the same subtree on the DOM – For each UI modules, you need to define a set of methods to act on the UI

New Concepts: UID Form: “A” “A” Container: “C” “A.C” InputBox: “B” “A.B” Selector: “D” List: “E” “A.C.D” “A.C.E” UrlLink: “1” InputButton: “n” “A.C.E[1]” A.C.E[n] Each UI object has a UID and is referenced by cascading UIDs  Elements in List and Table are referenced by indexes  UID makes Tellurium less verbose and more expressive 

New Concepts: Composite Locator Composite Locator (clocator): A set of attributes used to define  the object and the actual locator will be derived automatically at run time. For example, SubmitButton(uid: quot;googlesearchquot;, clocator: [name: quot;btnGquot;, value: quot;Google Searchquot;])  Which is equal to the following runtime xpath:  /input[@name=”btnG” and @value=”Google Search”]  The class definition is as follows,  class CompositeLocator {  –String header –String tag –String text –String trailer –def position –Map<String, String> attributes = [:] }  Each clocator will generate a run time relative xpath and it  makes Tellurium robust to changes inside UI module.

New Concepts: Group Locating Usually XPath only includes one path to the target node, for example,  “/div/div/form/input”, that is to say, only information along this path is used. We can use information in the nested object such as the whole form in  Fig (B), i.e., “A Form whose children are input (attributes), selector (attributes), and input(attributes)” The composite object has a reference point and once the reference point  is determined, the other UI elements can be located by their relations to the reference point.

New Concepts: UI Template UI template can be used for  – Unknown number of UI elements, but the types are known – Many and similar UI elements Tellurium List and Table objects use UI templates to define the UI elements they include inside. List(uid: quot;subcategoryquot;, clocator: [tag: quot;divquot;, class:quot;sub_cat_sectionquot;]){ TextBox(uid: quot;2quot;, clocator: [tag: quot;divquot;, class: quot;sub_cat_titlequot;]) Container(uid: quot;allquot;, clocator: [tag: quot;pquot;]){ UrlLink(uid: quot;linkquot;, clocator: [:]) } } Table(uid: quot;searchTablequot;, clocator: [class: quot;advqueryquot;]){ Selector(uid: quot;row:1, column: 3quot;, clocator: [name: quot;canquot;]) SubmitButton(uid: quot;row:1, column:4quot;, clocator: [value: quot;Searchquot;, name: quot;btnquot;]) InputBox(uid: quot;row:2, column:3quot;, clocator:[name: quot;wordsquot;]) InputBox(uid: quot;row:3, column:3quot;, clocator:[name: quot;withoutquot;]) }

Do you need to know Groovy ? Groovy is a dynamic language for Java  Compatible with Java syntax – Object oriented – More expressive – MetaClass for meta programming – Gstring: “label=${target}” – Closure – Optional type – Tellurium Core is implemented in Groovy and your UI  module class must be a Groovy class. But Test cases can be written in Java, Groovy, or pure DSL scripts. You do not really need to know Groovy when use Tellurium – UI definition is descriptive using DSL – Methods for UI module can be written in Java syntax – But using Groovy features will make your code more concise Groovy IDE support (Eclipse, Netbeans, IntelliJ) 

Tellurium Architecture

UI Object UI Object includes  – uid: should be unique inside a UI module and the top level uids should be different for different UI modules. – namespace: for future extension – locator: the locator of the UI object, could be • A base locator (relative xpath) • A composite locator (object attributes) – group: this option only applies to a collection type of UI object such as Container, Table, List, and Form. If quot;groupquot; attribute to be true, group locating feature will turn on – respond: define what JavaScript events the UI object could respond to. •UI objects are held on an object registry, which consists of multiple object trees.

Simple UI Objects Button  SubmitButton CheckBox Image Icon RadioButton TextBox InputBox UrlLink Selector

Container Container is an abstract object that can hold a collection of  UI objects. “group” attribute: Group Locating, sub-tree on the DOM  Container Type Object  – Container – Form – List: UI template – Table: UI template, Data Grid – Frame: IFrame – Window: popup window – Option: multiple UI patterns at run time

Widgets Tellurium provides you the capability to composite UI  objects into a widget object and then you can use the widget directly just like using a tellurium UI object. – Reusable – Name space to avoid name collision – Compiled as a jar file – Load from Tellurium configuration file  Tellurium provides an onWidget method – onWidget(String uid, String method, Object[] args) • Example: DOJO Date Picker widget •ui.Form(uid: quot;dropdownquot;, clocator: [:], group: quot;truequot;){ ...... • DOJO_DatePicker(uid: quot;datePickerquot;, clocator: [tag: quot;divquot;, dojoattachpoint: • quot;subWidgetContainerNodequot;]) •} • onWidget quot;dropdown.datePickerquot;, selectPrevYear

DSL Context Domain Specific Language is a small language and can  only live inside its domain context. The DSL context is the heart of the Tellurium and all UI modules must extend the DslContext class Thanks to Groovy’s syntax, all the methods defined in  the DSL context can be written in an expressive way, i.e., DSL expression. For example, the following method defines the DSL: “click id” • def click(String id){} DSL context includes UI object definitions, event  actions, and data access. DslContext is extended to support Data Driving Testing,  which includes input file layout, test definition, and testing flow.

UI Definition Sequence • Sequence diagram for UI definition

Object-Locator Mapping Object-Locator Mapping is used to map UI uids  to their runtime locators – XPath Builder: generate runtime xpath based on given attributes – Group Locating: generate runtime xpath for the reference point if the “group” attribute is true – Locator Processing: call different locate stragies based on the locator type of the UI object to generate relative xpath (call XPath Builder and Group Locating if apply) – UI ID mapping: When reference a UI object by UID, for example, “issueSearch.searchBox”, walk through individual UI objects, i.e., “issueSearch” and “searchBox” in this case, and combine the generated relative xpaths to form the actual runtime xpath. Note: UI object can be found by its UID from the object  registry.

Event Handler Event Handler deals with all UI events such as “type”,  “click”, “select”, ... By default, the event handler will add default events  such as “mouseOver”, “mouseOut”, “focus”, and “blur” to event actions so that your test code can trigger Javascript events. You can turn this off in Tellurium configuration. Event Handler will look at the UI object’s respond  attribute to include appropriate events for Javascript. For example, id: quot;searchboxquot;, clocator: [title: quot;Google Searchquot;], respond: [quot;clickquot;, quot;focusquot;, quot;mouseOverquot;, quot;mouseOu

Data Accessor Data accessor is used to get data or attributes  from web, or check the status/availability of the UI element. For example, – getText() – getValue() – isElementPresent() – isChecked() – waitForElementPresent() – getXpathCount()

Dispatcher The dispatcher will forward the events from the  event handler and method calls from the data accessor to the selenium client. The dispatcher is designed for future extension so that we can switch to a different UI testing engine similar to Selenium and keep the upper layers intact.

How Tellurium works • Sequence diagram for the “click” action

Tellurium JUnit Test Case TelluriumJavaTestCase is designed for JUnit 4.4 Test Case  and it supports the following annotations: @BeforeClass  @AfterClass  @Before  @After  @Test  @Ignore  It comes with predefined @BeforeClass and @AfterClass  methods to set up and tear down the connection to the Selenium server before tests begin and after all tests are done, respectively.

Tellurium TestNG Test Case TelluriumTestNGTestCase is similar to  TelluriumJavaTestCase and the difference is that it is for TestNG For TestNG, you can use the following annotations:  @BeforeSuite, @AfterSuite  @BeforeTest, @AfterTest  @BeforeGroups, @AfterGroups  @BeforeClass, @AfterClass  @BeforeMethod, @AfterMethod  @DataProvider  @Parameters  @Test 

Tellurium Groovy Test Case Tellurium is implemented in Groovy and Java.  You can use Groovy to write your test case as well. TelluriumGroovyTestCase is for a single test and TelluriumSuiteGroovyTestCase is for a test suite.

DSL Script Tellurium supports Domain Specific Language (DSL) for Web  testing and you can even write all your tests in pure DSL scripts. Because of the Groovy Syntax, each method in the DslContext  can be written using DSL syntax. For example, mouseOver(String id) -> mouseOver id  DSLs to support Tellurium tests:  – openUrl url – connectUrl url – assertEquals expected, actual • Tellurium supports pure dsl tests. The dsl script is actually a Groovy class and it will be executed by DslScriptExecutor – rundsl.sh – rundsl.bat

Tellurium Configuration Tellurium comes with a global configuration file,  TelluriumConfig.groovy, which sits in the project root directory and you can change all the settings in this file. If this file is missing, the framework will use the default settings specified in Groovy classes. The configure file makes it very easy for users to customize  Tellurium. The configuration file includes the following sections:  Embedded Server – Event Handler – Data Accessor – Connector – Data Driven – Test – UI Object – Widget –

Data Driven Testing Data Driven Testing is a different  way to write tests – separate test data from the test scripts – the test flow is controlled by the input file. Data Provider  loadData() – bind() – cacheVariable() – – getCachedVariable() TelluriumDataDrivenModule  – UI Module, FieldSet, Tests TelluriumDataDrivenTest  – includeModule() – stepToEnd()

Tellurium Tools Tellurium sub-project TrUMP creates a Firefox plugin to  automatically generate UI modules for users. The user just needs to click on the UI elements on the web and UI modules will be automatically generated to include all the selected UI elements. The plugin only generates the UI modules, not the test itself, which is the difference between TrUMP and Selenium IDE. You need to define methods for the UI modules and then write test cases based on the UI modules. For the time being, you need to use other tools to check the runtime html and create UI module manually. The useful tools include: – DOM Inspector – Xpather – Firebug

Tellurium Projects Tellurium Engine: Similar to Selenium • Core and will be Tellurium test driving engine. More efficient and supports nested UI object. JUnit TestNG Reference Project Reference Project Tellurium Core: UI objects, locator • mapping, and test support. Tellurium widget: DOJO widget • Tellurium Widget extension Tellurium Reference Projects: JUnit 4 • and TestNG projects to demonstrate how to create real world Tellurium Tellurium Core test cases using Tellurium project TrUMP web site as a reference. TrUMP: Firefox Plugin to automatically • generate UI modules Tellurium Engine Tellurium projects support ant and Maven  build Tellurium currently supports Groovy 1.6.0  and Selenium RC 1.0

Tellurium UI Model Plugin (TrUMP) • TrUMP IDE is a Firefox plugin to automatically create UI modules for users • Download TrUMP xpi file from Tellurium project site to install it • Different xpi files for Firefox 3 and Firefox 2. • They are the same except the way to show the view is different • Logging and preference support

TrUMP: How it works Array UI Module Web page User Clicks 1 Validate Generate 2 Validator Customize Validate 3 Internal Tree

TrUMP: Recording • Click on any element you want to choose • Click one more time to unselect • Click on the selected element and TrUMP will show you where it is on the web

TrUMP: Generate • The “Generate” button will generate an internal tree first • Then TrUMP generates the default UI module – Selected attributes from a White list – Event handlers are converted to the respond attribute – Group option is off by default

TrUMP: Customize • Group option is available to check • Optional attributes are available to select – Position, header, and may others

TrUMP: How to Customize You can change UIDs and UI types • Select or remove attributes • Turn on group option • TrUMP automatically validates the UI module, make sure you do • not see any red “X” mark, or you do not care or act directly on the UI elements with red “X” mark • Avoid attributes that may change such as position and header

TrUMP: Export UI Module package tellurium.ui import org.tellurium.dsl.DslContext /** * This UI module file is automatically generated by TrUMP 0.1.0. * */ class NewUiModule extends DslContext{ public void defineUi() { ui.Container(uid: quot;Telluriumquot;, clocator: [tag: quot;bodyquot;, class: quot;t2quot;]){ Form(uid: quot;Formquot;, clocator: [tag: quot;formquot;, method: quot;getquot;, action: quot;listquot;], group: quot;truequot;){ Selector(uid: quot;DownloadTypequot;, clocator: [tag: quot;selectquot;, name: quot;canquot;, id: quot;canquot;]) InputBox(uid: quot;SearchBoxquot;, clocator: [tag: quot;inputquot;, type: quot;textquot;, name: quot;qquot;, id: quot;qquot;]) SubmitButton(uid: quot;Searchquot;, clocator: [tag: quot;inputquot;, type: quot;submitquot;, value: quot;Searchquot;]) } Container(uid: quot;Titlequot;, clocator: [tag: quot;tablequot;, id: quot;mtquot;]){ UrlLink(uid: quot;Issuesquot;, clocator: [tag: quot;aquot;, text: quot;Issuesquot;], respond: [quot;clickquot;]) UrlLink(uid: quot;Wikiquot;, clocator: [tag: quot;aquot;, text: quot;Wikiquot;], respond: [quot;clickquot;]) UrlLink(uid: quot;Downloadsquot;, clocator: [tag: quot;aquot;, text: quot;Downloadsquot;], respond: [quot;clickquot;]) } } } //Add your methods here } • Export UI module to a groovy file with everything setting up for you • You only need to fill up your methods to act on the UI based on your application • Create JUnit or TestNG Java test cases based on the UI module

How to use Tellurium? Use Tellurium jar and configuration file Maven Use Reference Project as Manually create Project Maven Archetype a base Trump Create UI modules Groovy Class extends DslContext Create methods for UI modules JUnit/TestNG Create Test Cases In Java, Groovy, or DSL Run Tests

Examples: Google Start Page Create UI Module  – Manual process now to look the runtime html – HTML Tool: DOM Inspector, XPather, Firebug – Tellurium Firefox Plug-in (TrUMP) class NewGoogleStartPage extends DslContext{ public void defineUi() { ui.Container(uid: quot;google_start_pagequot;, clocator: [tag: quot;tdquot;], group: quot;truequot;){ InputBox(uid: quot;searchboxquot;, clocator: [title: quot;Google Searchquot;]) SubmitButton(uid: quot;googlesearchquot;, clocator: [name: quot;btnGquot;, value: quot;Google Searchquot;]) SubmitButton(uid: quot;Imfeelingluckyquot;, clocator: [value: quot;I'm Feeling Luckyquot;]) } } def doGoogleSearch(String input){ type quot;searchboxquot;, input pause 500 click quot;googlesearchquot; waitForPageToLoad 30000 } def doImFeelingLucky(String input){ type quot;searchboxquot;, input pause 500 click quot;Imfeelingluckyquot; waitForPageToLoad 30000 } }

Examples: Google Start Page (Cont’d) Create JUnit test case  public class GoogleStartPageJavaTestCase extends TelluriumJavaTestCase { protected static NewGoogleStartPage ngsp; @BeforeClass public static void initUi() { ngsp = new NewGoogleStartPage(); ngsp.defineUi(); } @Test public void testGoogleSearch(){ connectUrl(quot;http://www.google.comquot;); ngsp.doGoogleSearch(quot;tellurium selenium Groovy Testquot;); } @Test public void testGoogleSearchFeelingLucky(){ connectUrl(quot;http://www.google.comquot;); ngsp.doImFeelingLucky(quot;tellurium selenium DSL Testingquot;); } }

Future Directions Implement Locator Query Language (LQL) using  JQuery in Tellurium Core and Tellurium Engine. Add UI template support for the TrUMP project. Complete the DOJO widget project to add more DOJO widgets. Prototype Tellurium Engine project as Tellurium test driving engine to support nested UI objects and LQL.

Resources Tellurium Project website: http://code.google.com/p/aost/  Tellurium User Group: http://groups.google.com/group/tellurium-users  Groovy: http://groovy.codehaus.org/  JQuery: http://jquery.com/  JUnit: http://www.junit.org/  TestNG: http://testng.org/doc/documentation-main.html  HtmlUnit: http://htmlunit.sourceforge.net  Canoo WebTest: http://webtest.canoo.com  Selenium: http://seleniumhq.org/  Twill: http://twill.idyll.org/ 

Tellurium.A.New.Approach.For.Web.Testing

by John.Jian.Fang

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 40

Statistics

Tellurium.A.New.Approach.For.Web.Testing Presentation Transcript

http://www.slideshare.net	36
http://www.linkedin.com	4