The document discusses UI automation in UiPath, including how it works by simulating mouse and keyboard inputs and outputs, and the various activities used for inputting data to and extracting data from applications. It covers selectors for identifying UI elements, the UI Explorer tool, and different input and output methods like hardware events, send window message, and simulate. The goal of the session is to introduce attendees to UI automation concepts and have them complete an assignment automating a password generator app.

1. RPA Kickstarter Day 3
Ui Automation & Selectors
Lovely Sinha

2. 2
1. Introduction to User Interface
Automation
 What is UI Automation?
 How does UI Automation Work?
2. Selectors and Ui Explorer
3. Input Activities and Methods
4. Output Activities and Methods
Agenda

3. 3
User Interface (UI) Elements :
All graphical user interface pieces that construct an application, be they windows, checkboxes, text fields or drop-
down lists, and so on.
UI Automation:
The process of interacting with graphical elements within applications by simulating mouse and keyboard controls.
There are majorly two interactions we can observe in UI Automation
 Input: Inserting data into an application
 Output: Fetching data from an application or Ui Elements

4. 4
UI Automation Activities
Latest Package Available in
Manage Package Section
Activities can be split into:
•Containers - These are activities that
identify the browsers or apps the
process needs to interact with. All
activities included within a Container
will execute on the same
application. Some examples are Open
Browser, Attach Browser, Open
Application, or Use Application/Browser
•Input Activities - These activities
send input to UI elements. They can be
used to click, check, type into, send
hotkeys, and so on.
•Output Activities - get structured data
or get UiElements containing images.
•Synchronization Activities - They
help you create triggers based on UI
behavior, thus enabling the Robot to
execute certain actions when specific
events occur on a machine.

5. 5
Activity Properties

6. 6
Modern Vs Classic Design Experience

7. 7
Example 1: UI Demo Application
Open Demo UI
Application
Enter Data
Click Accept
Get Transaction Number
Input
Actions
Output
Actions

8. 8
Selectors
Unique identifiers of UI elements. These store UI
element attributes which will help identify the UI
element uniquely during automation.
Wild cards
There are two types of wildcards available in UiPath
 * - Replaces any number of characters.
 ? - Replaces exactly one character.

9. 9
UI Explorer
It is the wizard which is used to indicate and edit selectors. If we are unable to locate the individual
element then we can switch to other Ui Frameworks for better results.

10. 10
Example 2: ACME Add Vendor
Open Browser
https://acme-
test.uipath.com/login
Enter Credentials
Click Vendors -> Add
Vendor
Display added vendor
details
Input
Actions
Output
Actions

11. 11
Input and Output Methods
Input Methods
•Hardware Events
Clicks: the mouse cursor moves across the screen. Typing: the keyboard driver is used to type individual
characters. Does not work in the background. The attended User cannot touch the mouse or keyboard during
the automation.
•Send Window Message
Replays the window messages that the target application receives when the mouse/keyboard is used.
Works in the background. Users can work on other activities during the execution of the automated
processes.
•Simulate
Uses the technology of the target application (the API level) to send instructions. Works in the background.
Users can work on other activities during the execution of the automated processes

12. 12
Output Methods
Output Methods
•Full Text
the default method and good enough in most cases. It is the fastest, it can extract hidden text, it has 100%
accuracy, and can work in the background.
•Native
compatible with applications that use Graphics Design Interface (GDI), the Microsoft API used for representing
graphical objects. It doesn’t extract hidden text and it cannot work in the background; and just like Full Text, it
doesn’t support virtual environments.
•OCR
OCR (or Optical Character Recognition) is the only output method that works with virtual environments and
with “reading” text from images. Its technology relies on recognizing each character and its position. On the
other hand, it cannot work in the background, it cannot extract hidden text, and its speed is by far the lowest.

13. 13
UI Automation is the process
of automating application
user interfaces by
simulating human
input and output actions
through specific UI Activities.
The first step in automating UIs
is understanding the logical
sequence of steps that a
human user would take. The
second step consists in
translating these steps into
UiPath Studio Activities and
configuring them.
The complexity of the UI or
of the application is
irrelevant. All desktop/web
applications can be
automated by using universal
or application-tailored
activities.
Takeaways

14. 14
Assignment: Password Generator
Open Browser and Navigate
to URL
https://www.rpasamples.com/
passwordgenerator
Type Password Length
Generate Password
Get Password
Input
Actions
Output
Actions
Post your Unique 10-digit
password with the following
hashtags :
#UiPathCommunity
#RPAKickStarter
“My Buddy Bot generated a
password – [your password]”

15. 15
Q&A

16. 16
Learning References:
Login to https://academy.uipath.com/
Course Name:
https://academy.uipath.com/courses/ui-automation-with-studio
https://academy.uipath.com/courses/selectors-in-studio
Documentation Links:
https://docs.uipath.com/studio/docs/about-ui-elements
https://docs.uipath.com/studio/docs/about-selectors

17. 17
.
Next Session
https://community.uipath.c
om/events/details/uipath-
learners-presents-rpa-
developer-kickstarter-day-
4-recorders-in-uipath/