UIA is an API that allows automation of desktop applications. It represents UI elements in a tree structure and supports properties, methods, events to define functionality. UIA is available since Windows Vista and supports Win32, WinForms, WPF and UWP applications. Helper libraries like TestStack.White and FlaUI make UIA accessible from .NET by handling P/Invoke and COM interactions. The UIA client uses Find and Pattern methods to discover elements, get properties and invoke interactions while the server implements providers to return values and handle method calls.

1. www.luxoft.com
Desktop Application Automation:
.NET Developer’s perspective
Ivan Danilov

2. www.luxoft.com
Automation definition and goals
• Microsoft UI Automation (UIA) is an application programming interface (API) that allows one to
access, identify, and manipulate the user interface (UI) elements of another application
[Wikipedia]
• Application testing
• Accessibility scenarios

3. www.luxoft.com
Availability
• Available since Windows Vista / Windows Server 2008
• Part of Windows 7 and above right after OS install
• Automates Win32, WinForms, WPF, UWP and other applications
• Ability to automate depends on UIA support that can be implemented for any UI in theory
• Frameworks and technologies listed above have UIA support out of the box

4. www.luxoft.com
UIA terminology
• Server – application that is being automated
• Client – automation driver, e.g. automated UI test
• AutomationElement – basic building block; every piece of UI that is exposed to automation is
represented with AutomationElement. They are contained in tree structure with desktop as the
root
• There’s no one-to-one relationship between automation elements and controls, but often it is the
case
• Control Patterns support the methods, properties, events, and relationships needed to define a
discrete piece of functionality available in a control

5. www.luxoft.com
History and UIA versions
• UIA is a successor to Microsoft Active Accessibility (MSAA)
• MSAA functionality is preserved in UIA in form of LegacyIAccessible pattern
• UIAv2
• Initial implementation
• Set of methods consumed with simple P/Invoke
• UIAv3
• Exists side-by-side with UIAv2 in the same dll
• COM object implementation

6. www.luxoft.com
UIAv2 vs UIAv3
UIAv2
• Better support for old applications, like WinForms or Win32
• Some stability issues
• Has managed .NET UIA wrapper
UIAv3
• More stable, faster
• Allows custom user-defined patterns
• Due to different implementation, control tree for some standard controls differ from UIAv2
• No Microsoft supported means to consume from .NET application

7. www.luxoft.com
Helper libraries
Since both P/Invoke and COM are inconvenient in .NET, several helper libraries emerged:
• TestStack.White
• Based on Managed UIA wrapper
• Has separate branch using UIAv3, mostly preserving same API layout
• FlaUI
• Most recent, supports both UIAv2 and UIAv3 out of the box
• More extensible

8. www.luxoft.com
Core features
• UIA elements discovering and identification
• Endpoint on UIA server to communicate with
• Communication protocol
• Getting elements’ properties:
• What is the text on this button I have?
• Invoking elements’ interactions:
• Click on the button
• Getting notifications on UIA events
• Call delegate every time new window is opened

9. www.luxoft.com
Tools
UIAv2
• UISpy
UIAv3
• Inspect
• VisualUIVerify
• FlaInspect

10. www.luxoft.com
Architecture picture

11. www.luxoft.com
UIA client specifics
• Initialize UIA
• Get root AutomationElement
• Descend via automationElement.Find*() family of methods to find necessary element
• Get its pattern implementation you need with GetPattern method
• Get properties and/or invoke pattern methods as necessary

12. www.luxoft.com
UIA server specifics
• Entry point is WM_GETOBJECT windows message
• Server needs to return pointer to raw element provider (IRawElementProviderSimple
implementation in most cases)
• That implementation is called by core UIA for patterns, methods and properties
• For example, to get value of some property – its ID is provided by core UIA and value is expected
in return
• To invoke pattern method – ID of the method is provided along with object[] with invocation
arguments
• Very tedious to implement directly
• Good thing is, all standard MS controls implement this out of the box

13. www.luxoft.com
WPF/UWP specifics
• Since WPF/UWP don’t have HWND for each control – they can’t receive WM_GETOBJECT
directly at control
• Instead, it provides WM_GETOBJECT functionality on the window-level, and everything else is
done inside WPF
• This functionality relies on AutomationPeer class (or descendant) exposed from
UIElement.OnCreateAutomationPeer method, and visual tree to discover child elements
• Thus, if you have custom control in WPF that does not have UIA implemented (and mostly it is
the case with third-party controls like Telerik or DevExpress) and you need to automate it – you
have to provide AutomationPeer for it

14. www.luxoft.com
Resources
• MSDN page on UIA
https://msdn.microsoft.com/en-us/library/ms747327(v=vs.110).aspx
• TestStack.White
https://github.com/TestStack/White
• FlaUI
https://github.com/Roemer/FlaUI
• Custom UIA patterns usage in .NET
https://github.com/TestStack/uia-custom-pattern-managed
• Series of blog posts on custom patterns in UIA by M. Bernstein
https://blogs.msdn.microsoft.com/winuiautomation/2010/12/07/uia-custom-patterns-part-1/
https://blogs.msdn.microsoft.com/winuiautomation/2010/12/10/uia-custom-patterns-part-2/
https://blogs.msdn.microsoft.com/winuiautomation/2010/12/27/uia-custom-patterns-part-3/

15. www.luxoft.com
THANK YOU
Q&A