1. EPAM SYSTEMS
UI Automation/White/CodedUI.
common problems & tricks
Dmitry Hes, Tsimafei Avilin, EPAM Sys.
4/6/2015
Summary: The page describes some of the problems in QA Automation that occur
when using UIAutomation (CodedUI, White) to automated desktop applications
and methods for their solution on the example of the project developed with
DevExpress library.

2. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 1
 At the beginning was … UI Automation
 Using Automation UI
 Using White framework.
 Using Visual Studio 2010 Coded UI.
o Working with White
 Some tricks to automate unsupported controls.
o UI Automation properties
o Using patterns, on SelectionPatternIdentifiers example
o DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled
property
o When simple way ends…
 Use LegacyIAccessible
 Using COM-wrapper
 Use Win32 API
 Call Invoke()
 Use BoundingRectangle
 Data caching
Summary: The page describes some of the problems in QA Automation that occur
when using UIAutomation (CodedUI, White) to automated desktop applications
and methods for their solution on the example of the project developed with
DevExpress library.
At the beginning was … UI Automation
Using Automation UI
This framework provides access to user interface elements Win32, Windows
Forms or WPF applications with the possibility to get or set the element property,
emulate user inputs (the official documentation is available here). Note that it is
base for some popular frameworks such as CodedUI and White.
In the UI Automation UI elements are represented as a tree structure of
AutomationElement objects with desktop as a root. The searching of element are
carried out through the tree by conditions specify in PropertyCondition. Some
controls implement patterns (e.g., SelectionItemPattern, SelectionPatternIdentifiers
the pattern for ListBox, CheckBox, GroupBox).

3. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 2
Control patterns provide a way to categorize and expose a control's functionality
independent of the control type or the appearance of the control.
UI Automation uses control patterns to represent common control behaviors. For
example, the Invoke control pattern uses for controls that can be invoked (such as
buttons) and the Scroll control pattern for controls that have scroll bars (such as list
boxes, list views, or combo boxes). Because each control pattern represents a
separate functionality, their combination describes the full set of functionality
supported by a particular control.
Some advantages of UI Automation:
- free;
- support Win32, WinForms, WPF;
- included in .Net framework.
Drawback:
- the framework works well standard UI controls, but to support custom and
complex controls it’s necessary to make additional implementations;
- a lot of code for controls searching and processing.
Using White framework.
Based on the UI Automation this framework has convenient access to the controls
and their properties. He has the same features as UI Automation. The framework
can be used to test the Win32, WPF, WinForm, SWT and Silverlight applications.
White framework allows you to write tests simpler and more readable.
You can use a ready BDDfy framework (read here), which allows to easily
implement BDD testing with White.

4. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 3
Using Visual Studio 2010 Coded UI.
Coded UI — Microsoft’s solution based of the same UI Automation, appeared in
Visual Studio 2010 (you can read here). Under this link you can find useful article
how to use AutomationPeer to automate WPF/Silverlight custom controls.
Profits:
- recorder is available, recording user actions for autogenerating tests;
- Visual Studio in-box;
- support from Microsoft, integration in TFS;
- frequently recurring test steps can easily be re-used. It’s up to you how fine-
grained you want to record your test steps. Calling a step (consisting of one or
more actions) is just a matter of a single function call;
- generates C# and XML code by default;
- features fuzzy matching of UI elements. This seems to make the tests more
robust when updating the user interface;
- you can test many different kinds of user interfaces, not just the web.
Shortcomings:
- the same as that UI Automation;
- proprietary software: Coded UI available in more expensive Visual studio
editions: Ultimate, Premium;
- test steps are stored in an XML file (called UIMap) which in turn compiles to
C# code. The files is big and clunky and there currently is no editor or
documentation for it. So if you want to make some changes, things can get
complicated unless you want to re-record an entire test step.
- creating very simple assertions (such as “look for the string “foo” on this web
page”) is a bit clumsy and requires too many mouse clicks. An IE accelerator
would be great!

5. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 4
- while fuzzy matching works great in most cases, it can get in the way in
others.
Based on the above choice was made in favor of the White framework, since it is
free and makes writing tests easier and makes them look intuitive.
Working with White
It is easy to start working with the White for people who has not did. Even it does
not have such a good guide to use as MSDN, but for beginning White has enough
resources here http://teststack.azurewebsites.net/white/index.html.
To find the required control by AutomatedId property it’s enough to execute Get
method parameterized of control type, and as a function parameter to specify the
desired AutomatedId. For example, for Edit field with AutomatedId = "txtCity":
TextBox button = window.Get< TextBox>("txtCity");

6. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 5
To set other search criteria, such as text, you can use the SearchCriteria class.
They can be combined with the methods And…(). So to find the button by the text
"OK":
Button button = window.Get<Button>(SearchCriteria.ByText("OK"));
To find the controls in the application, as well as getting its properties are useful
Inspectors like as Inspect (download - Inspect.exe) supplied with MS win8 SDK,
or ACorns.Hawkeye (https://hawkeye.codeplex.com/). Moreover, Hawkeye shows
the actual type of the object, as well as an object implementation library.
Some tricks to automate unsupported controls.
It was noted that White is a wrapper of UI Automation, and thus it implements the
same basic controls. However, the application under test uses set of custom
controls implemented by e.g. DevExpress library. Some troubles in processing
such controls and their solutions were found out.
UI Automation properties
The first trouble is that White does not process custom DateEdit field. The
framework does not treat it neither like any DateTimePicker, nor Edit. As the
DataGrid contains MaskBox field, to get the control values you can get the value
of control through UI Automation properties:

7. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 6
//Get value from whiteElement.AutomationElement
private string GetDate( AutomationElement element)
{
AutomationElement dateEdit = element.FindFirst(
TreeScope.Children, new PropertyCondition(
AutomationElement.ControlTypeProperty, ControlType.Edit));
object valuePattern = null;
dateEdit.TryGetCurrentPattern( ValuePattern.Pattern, out valuePattern);
return ((ValuePattern)valuePattern).Current.Value;
}
To set values use typing emulation:
//Type text @dateString into @element
private void SetDataToField( AutomationElement element, string dateString)
{
AutomationElement dateEdit
= element.FindFirst(TreeScope.Children, new
PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Edit));
dateEdit.SetFocus();
System.Windows.Forms.SendKeys.SendWait(dateString);
}

8. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 7
Using patterns, on SelectionPatternIdentifiers example
The next problem is the processing RadioGroup - getting the value of the selected
item. Applying White isSelected property for element RadioButton, always return
False value.
So UI Automation is useful. The implementation of custom function helps us. This
function refers to the parental control (RadioGroup) and takes the value of the
selected item, using the pattern SelectionPatternIdentifiers.
public bool isSelected()
{
string selectedElement =
GetSelectedElementName( _element.AutomationElement);
return selectedElement.ToLower().Equals(_element.Name.ToLower());
}
//Gets selected element name
private string GetSelectedElementName(AutomationElement element)
{
AutomationElement parentElement = Utils.Util.GetParentElement(element);
var selection = parentElement.GetCurrentPropertyValue(
SelectionPatternIdentifiers.SelectionProperty);
AutomationElement[] selectedElements = selection as AutomationElement[];
if (selectedElements.Length == 0) { return ""; }
return selectedElements[0].Current.Name;
}

9. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 8
DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled property
ComboBox controls refer to DevExpress XtraEditors.LookUpEdit class library and
represent as an Edit field combined with a Button, clicking on which opens a drop-
down list. However, the list is a UI Automation Window element, is a direct
descendant (child) of the main form.
In addition, when working with a library DevExpress a problem with dynamically
created controls or dynamically fills tables was spotted. UI Automation does not
see and does not find such controls and child elements of the table even searching

10. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 9
methods in TreeScope. The solution to this problem is setting to disable static
cleaning event queue property in the application under test:
DevExpress.Xpf.Core.ClearAutomationEventsHelper.IsEnabled = false
However, this may reduce the performance of the application (more info here).
When simple way ends…
White is not so quite good for processing Tables. Especially when tables using a
complex structure as a set of data in the collapsing combined group, nested tables,
any controls instead of text in cells.
After reading variety of websites and forums, solutions for the processing of
custom controls were found out.

11. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 10
Use LegacyIAccessible
To get cell values use UI Automation. To identify the type of line (collapsing
group element / data line) read the value LegacyIAccessible.State. To do this, you
should use IUIAutomationLegacyIAccessiblePattern, which provides the interface
ILegacyIAccessibleProvider. However, LegacyIAccessible pattern is not available
for control based on UI Automation, because it is not implemented neither UI
Automation nor White
if ((
bool) child.GetCurrentPropertyValue(AutomationElementIdentifiers.IsLegacyIAccessiblePatternAvailableProperty))
{ var pattern = ((LegacyIAccessiblePattern) child.GetCurrentPattern(LegacyIAccessiblePattern.Pattern));
var state = pattern.GetIAccessible().accState;
// Do something with state...
Using COM-wrapper
Using COM-wrapper library UIAComWrapper.dll distributed with Windows Kits
is one of the way to solution. It have implementation for Inspect.exe of
ILegacyIAccessibleProvider for getting LegacyIAccessible properties.
Use Win32 API
In addition, you can use Win32 API to send window messages and thus get
control properties or invoke events.
Here is an example to get DevExpress panel Caption.

12. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 11
using System.Runtime.InteropServices;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Automation;
using System.Windows.Automation.Text;
[System.Runtime.InteropServices.DllImport("user32.dll", EntryPoint =
"SendMessage", CharSet = System.Runtime.InteropServices.CharSet.Auto)]
private static extern bool SendMessage(IntPtr hWnd, uint Msg, int wParam, StringBuilder
lParam);
[System.Runtime.InteropServices.DllImport("user32.dll", SetLastError = true)]
private static extern IntPtr SendMessage(int hWnd, int Msg, int wparam, int lparam);
//Get Text property of UI element using Win32 API
public static string GetText(AutomationElement element)
{
const int WM_GETTEXT = 0x000D;
const int WM_GETTEXTLENGTH = 0x000E;
//Get handle property of the control
int wndHandle =
(int)element.GetCurrentPropertyValue(AutomationElement.NativeWindowHandleProperty);
StringBuilder title = new StringBuilder();
// Get the size of the string required to hold the window title.
Int32 size = SendMessage(wndHandle, WM_GETTEXTLENGTH, 0, 0).ToInt32();
// If the return is 0, there is no title.
if (size > 0)
{
title = new StringBuilder(size + 1);
SendMessage( new IntPtr(wndHandle), ( int)WM_GETTEXT, title.Capacity, title);
}
return title.ToString();
}

13. Dmitry Hes, Tsimafei Avilin, EPAM Sys. 12
Call Invoke()
Furthermore you can call Invoke() method to get control properties. This method
implements InvokePattern, which provides support operation and invoking
unambiguous action of controls that do not maintain state by activation.
Use BoundingRectangle
Worst case to use BoundingRectangle property to get control layout. Then use
mouse clicks via SendInput expanding/collapsing/ the table.
Data caching
For processing controls which consist volume data (lists, tables, trees), as well as
getting properties and control patterns in one operation it is necessary to use data
caching. It allows improving performance and, in some cases, avoiding prolonged
searching or exceeding time-out interval through the frequent accessing to
elements. It may lead to test failures.
In UI Automation, caching means pre-fetching of data. Caching occurs when the
application activates a CacheRequest object and then uses any method or property
that returns an AutomationElement, for example, FindFirst, FindAll. The methods
of the TreeWalker class are an exception; caching is only done if a CacheRequest
is specified as a parameter (for example,
TreeWalker.GetFirstChild(AutomationElement, CacheRequest)). Caching data
helps to avoid direct access to date. Any changes made to the CacheRequest after
you subscribe to the event have no effect. More information see here.