📕 Agenda: Intro to Ui Automation What is Ui Automation, Web automation How to use the recording tool Handling selectors in Studio Data scraping Screen scraping Introduction to Excel automation Data tables and data manipulation Launch browser, attach browser Demo - follow-along Your trainers: 👨🏼‍💻 Dillan Hackett, UiPath MVP 2022, Partner & Director @Tquila Automation 👨🏻‍💻 Sean Jerome Llanto, UiPath MVP 2022, RPA Solutions Architect @Ingram Micro 👨🏻‍💻 Stefano Negro, UiPath MVP 2022, RPA Tech Lead @BSP Consultant

1. Introduction to Ui Automation

2. 2
User interface Automation is the process of interacting with visual elements on
the Desktop.
This is done through simulating mouse movement, mouse click and keyboard
inputs.
What is Ui Automation?
Mouse movements and
clicks
Keyboard inputs Reading computer screen

3. 3
With Ui Automation in Robotic Process Automation it’s
possible to simulate the human interaction with
applications with UiPath Studio.
UiPath Studio uses interaction with most of the GUI
Frameworks (Desktop Applications, Web, Mainframe)
and recognizes different objects.
For every object a set of specific interaction is
available, for example:
• Write into TextBox
• Select Item from Dropdown
• Check/Uncheck Box
• Toggle RadioButton
Why use Ui Automation?

4. 4
The following are the most used Activities related to Ui Automation:
• Click: the mouse interacts with an element (button, field, box..) to click on it. It’s
possible to use left/right button and also double click or click down/up.
• Type Into: compiles a text field with an input given. It could be a default input or
taken from a variable. Moreover it’s possible to use special keys during the Type
• Get Text: Reads a field and returns the text value inside. It’s possible to use
different “Scraping” method, to deal with particular objects that don’t retrieve the text
automatically.
• Select Item: Interacts with a dropdown and select the Item specified as input. When
spying the element, the list of items will be given.
Ui Activities

5. The Recording Tool

6. 6
Recording tool
With the recording you can save a lot of time when creating an automation.
Using this functionality you can capture every interaction with the desktop and
automatically turn it to an activity inside your sequence.
You can record Click, Text typing, Checkboxes, Drop-downs, Mouse Hover and
Keyboard Shortcuts.

7. 7
Modern Experience
Using the Modern Experience
automation, the recording tool is now
unique, so it’s not necessary to select the
App type in advance.
You can change the Automation
type from the Project Settings of
your automation.
If you select Modern Automation,
you can still use Classic Automation
as well.

8. 8
Web Recording
• Used for recording in web apps and browsers
• Generates containers and uses Simulate Type/Click input method by
default
Desktop Recording
• Used when multiple actions are performed in many windows
• Suitable for all types of desktop apps
• Generates a container (with the selector of the top level) in which
activities are enclosed and partial selectors for each activity
• Faster than Basic recording
Basic Recording
• Used when multiple actions are performed in a single window and
generates a full selector for each activity
• Slow than the one that uses containers and is suitable for single
activities.
• Slower compared to Desktop recording
Basic, Desktop & Web Recording

9. 9
Computer Vision Recording
• Used to visually identify all elements, rather than replying on selectors
or images & is suitable for automatically generating workflows that use
CV activities
Native Citrix Recording
• Used in Native Citrix automation projects & is the equivalent of the
Desktop recording for Citrix environments
Image Recording
• Used to record virtualized environments such as VNC, virtual
machines, and Citrix for image, text & keyboard automation
• Require explicit positioning
Image, Native Citrix & Computer Vision
Recording

10. 10
More About Recording
All user interface elements are
highlighted while you record, as you can
see in the following screenshot, so that
you can be sure the correct buttons,
fields or menus are selected.
Regardless of the type
of recording selected,
some actions are
recordable and some
are not.
Recordable Non-Recordable
● Left-click on
buttons, check
boxes, drop-
down lists and
other GUI
elements
● Text typing
● Modifier keys
● Mouse hover
Note:
If you make changes to your display settings without restarting the
computer, elements are not going to be properly identified.

11. 11
Text typing
Left clicks
Click on Check boxes
Click on Drop-downs
▪ Following keyboard shortcuts can be used while recording:
▪ F2 – pauses the recording for 3 seconds. A countdown timer is displayed in the bottom left corner of
the screen. Can be useful with menus that automatically hide
▪ Esc – exists the automatic or manual recording. If pressed again, recording is saved as a sequence,
and you return to the main view
During Recording, you can do….

12. 12
More About Recording
After the Recording tool has been used, a sequence is created with all the
activities automatically generated.
HINT:
It’s always useful to check every selector for default values that could change from
time to time.
For example the Browser Url, a text-box that has dynamic values and so-on.
We will see in a few moments how to change selectors
Accordingly and follow best practice to make our automation
as dynamic and robust as possible.

13. 13
Recording tool Example
https://www.rpasamples.com/unicornname

14. Selectors

15. 15
Selectors
The Selectors are specific attributes related to an element that let the automation
uniquely identify it.
Every element has a specific selector.
When using recording or spy an element, a selector is automatically created and
most of the times it’s not necessary to change its attributes.
There are some attributes however that are dynamic, so it’s necessary to “tweak”
the selector values in order to make the automation stable.
You can see the selector of an element by clicking on the
Options menu and select Edit Target

16. 16
By default a «Fuzzy Selector» is chosen, and
it is populated with some html tags.
Each row contains a «Node».
Node1 usually is the root node, that
represents the top window of the app.
You can check if there are some tags that
could change during runtime, for example:
• Text field related to input (eg. Dates)
• Browser Url
• Layout fields (eg. HTML paths)
Selectors

17. 17
Selector Editor Window
Classic Design Modern Design
▪ Enables the user to see the automatically generated selectors and edit their
attributes

18. 18
Ui Explorer
An advanced tool that enables you to
create a custom selector for a specific
UI element.
It is available as a standalone tool you
can download from the Resource
Center in your Automation Cloud
instance, or from Studio only if the
UiPath.UIAutomation.Activities
package is installed as a dependency
to the project.

19. 19
Validate
• Shows the status of
the selector by
checking the validity of
the selector definition
and the visibility of the
target element on
screen.
• Has the following
status:
• O (buffer): Selector is
being validated
• √: Valid selector
• ×: Invalid selector
• ?: Modified selector,
revalidate
Indicate Element
• Indicates a new UI
element to replace the
previous one
Repair
• Enables the user to re-
indicate the same
target UI element and
repair the selector.
• The button is available
only when the selector
is invalid
Highlight
• Brings the target
element in the
foreground.
• The highlight stays on
until its switched off
• The button is enabled
only if the selector is
valid.
Indicate Anchor
• Indicates a new anchor
relative to the indicated
Ui element
Properties of Ui Explorer

20. 20
Full selectors:
● Contain all the elements needed to identify a UI element, including the top-level
window
● Generated by the Basic recorder
● Recommended when switching between multiple windows
Partial selectors:
● Generated by the Desktop recorder
● Do not contain information about the top-level window
● Activities containing partial selectors are enclosed in a container (Attach Browser or
Attach Window) that contains a full selector of the top-level window
● Recommended when performing multiple actions in the same window
Types of Selectors

21. 21
Full Selectors
• Contain all the elements needed to identify a UI element,
including the top-level window
• Starts with a window or an HTML identifier
• Generated by the Basic Recorder
• Best suited when the actions performed require switching
between multiple windows
Partial Selectors
• Don’t contain the information of the top-level window, thus
the activities with partial selectors must be enclosed in
containers
• Generated by Desktop Recorder
• Best suited for performing multiple actions in the same
window
Full & Partial Selectors Sample

22. 22
Types of Selectors
▪ Used to find the address of a UI element dynamically and identify the attributes of the
element across windows:
• Use a variable or an argument as a property for the attribute of
the target tag
• Allows easy identification of a target element based on the value
of the variable or argument
• Best suited for situations in which the targeted element
constantly changes its value
Format
<tag attribute=‘{{Value}}’ />
• Tag: the target tag, such as <ctrl/>
• Attribute: the target attribute, such as name=‘menuItem’
• {{Value}}: the name of the variable or argument which holds the
property of the element the user wants to interact with
Dynamic Selectors

23. 23
A wildcard is a special character that can replace the dynamic part of the selector.
There are two types of wildcards:
When a wildcard is used or a variable is added in between selectors, it is known as customizing selectors.
Asterisk
{*}
• Replaces zero or more
characters
Question
mark {?}
• Replaces a single character
Wildcards in Selectors

24. 24
Selectors Demo

25. Data Scraping

26. 26
Data Scraping/Table Extraction
Enables you to extract structured data from your browser, application or
document to a database, .csv file or even Excel spreadsheet.
Structured data is a specific kind of information that is highly organized and is presented in
a predictable pattern.
Classic Design
Modern Design

27. 27
Data Scraping Demo
Open Web Browser Navigate to URL
https://acme-test.uipath.com/login
Enter User Credentials
and Login to the portal
Navigate to Work Items List
Extract Table Data
from multiple pages
Save captured data into an excel/csv file

28. 28
Data Scraping/Table Extraction Exercise
▪ Login to https://acme-test.uipath.com/login
▪ Steps:
▪ Open browser
▪ Navigate to the above URL
▪ Enter username and password
▪ Sign in
▪ Click Work Items
▪ Use Data Scraping or Table Extraction
▪ Click the table to be extracted
▪ Follow the prompt
▪ Add a write csv activity
▪ run the .xaml file

29. Excel and Data Manipulation

30. 30
Excel Automation - What can you automate?
• Create report:
• Read and write data
• Sort data
• Data Filters
• Pivot tables
• Excel formatting
• Work with ranges – append, auto fill, add/delete rows/columns, sheet
• Check reports for errors:
• Delete duplicate data
• Data reconciliation
• Fill in forms with data from Excel spreadsheets
• Extract data from emails and move to desktop/web application
• Data extraction and migration
• Data Table to Excel, Excel to Data Table
• Run macro

31. 31
How do I use it?
• Package: UiPath.Excel.Activities
Note: You can enable Studio X activities also inside Studio
for more excel activities

32. 32
Data Tables
One of the most important
activities in Studio – DataTable
• Maintain Big Data in Data Table
variables
• Read from excel
• Merge data tables
• Output to excel
• Data processing in loop
• Data lookup
• Filter
• Data Sorting
• Remove duplicates

33. 33
Data Manipulation
▪ Data manipulation is the process through which the data is altered using
various operations in order to facilitate its usage
▪ Operations:
▪ Modifying
▪ Structuring
▪ Formatting
▪ Sorting
▪ Allows the use to change data from one form to another by
concatenating, converting, extracting, splitting or segregating to make it
ready to be used by RPA in workflows
Data Manipulation allows us to customize the data in order to be able to
generate the desired outcome based on the process needs

34. 34
Variables
▪ Data of all kinds can be stored in variables, and there are several variable types in
Studio that can be classified into three types
Variable
Types Scalar
Characters
Booleans
Numbers
Collections
Arrays
Lists
Strings (a collection of characters)
Dictionary
Tables
Two dimensional structures that hold
data indexed by rows and columns

35. 35
Common operations for data manipulation are:
Retrieving
• Specify
conditions to
extract, filter
or
summarize
data
Adding
• Add new
data in a
data
structure
Deleting
• Remove
entries from
a data
structure
Modifying
• Changes the
existing
entries in a
data
structure
Operations for Data Manipulation

36. 36
Data conversion is the process of converting one type of data to another type.
The methods for data conversion include:
Simply assigning the
data value to the
desired data type.
In UiPath, explicit
assignment is
generally used for data
conversion
• Eg:
Integer.parse(“1243”)
to convert string to
number
Assign
Convert any datatype
to string using the
.ToString method.
•Eg: the user can convert
DateTime datatype to string
datatype or number to string
and vice versa
.ToString Method
Stores any kind of
data, including text,
numbers, dates, and
arrays.
Automatically
converted to other
types to perform
certain actions.
The first element in the
expression is used as
a guideline for what
operation Studio
performs
GenericValue
Variable
Data Conversion

37. 37
String is a data type corresponding to text that contains a sequence of text.
It is used anytime a text needs to be captured, processed, sent between applications,
or displayed
Strings and String Manipulation

38. 38
Concat
• Concatenates the string representations of two specified objects
• String.Concat(Var1, Var2)
Contains
• Checks whether a specified substring occurs within a string.
• Returns true or false
• VarName.Contains(“Test”)
Format
• Converts an entire expression into a string (and Inserts them into another text)
• Reduces complexity and increases readability
• String.Format(“{0} is {1}”, VarName1, Varname2)
IndexOf
• Returns the zero-based index of the first occurrence of a character in a string
• Varname.Indexof(“A”)
Strings and String Manipulation
● Some of the operations that can be performed on strings are:

39. 39
Methods for String Manipulations
Join
• Concatenates the elements in a collection and displays them as string
• String.Join(“|”, CollectionVariable)
Replace
• Replaces all the occurrences of a substring in a string
• VariableName.Replace(“original”, “replaced”)
Split
• Splits a string into substrings using a given separator
• VariableName.Split(“|”c)(index)
Substring
• Extracts a substring from a string using the starting index and the length
• VariableName.Substring(StartIndex, Length)
Strings and String Manipulation

40. 40
Excel Example
• Create a Sample Excel File (or download
our sample)
• Use Excel Application Scope to point to our
file
• Use “Get Worksheet” to retrieve the name
of the first sheet (index:0)
• Read all DT
• Write a new Header with “Write Cell”
• For each data in DT:
• Check if the Country is USA: Write
“North America”
• If not: Write “Europe”

41. 41
Excel Demo Run-through 1
Open Excel File Read Columns A to D
for the current row
Concatenate Column Values
Into Column E
Read next Available row
Row
Available?
Yes
No
End Process

42. 42
Excel Demo Run-through 2
Open Excel File Read Columns A to D
for the current row
Read next Available row
Row
Available?
Yes
No
End Process
Open Web Browser Navigate to survey URL:
https://www.surveymonkey.com/r/G2PPSVV
Enter details requested
In the survey
(data from excel) then submit
Survey response

43. 43
Thank You
Any questions?

44. Appendix

45. 45
Open a Notepad file
• Save two files with different dates as the file name
• Create a new Sequence, give it a proper name and add an annotation
• Use an ‘Attach Window’ and indicate the Notepad file that is opened. Inside the container:
• Use a ‘Send hotkey’ to send Ctrl + h in order to replace 2 spaces with one
• Use 2 ‘Type into’ activities – one for the first field (with 2 spaces), and another for the second field (with 1 space)
• Use 2 ‘Click’ activities, one for the ‘Replace all’ button and the other for ‘Close’
• Open the second Notepad file and run the workflow. It will return the ‘Cannot find element’ error
• Open the selector for the ‘Attach window’ container. Point out that the title attribute contains the name of the
first file
• Use the repair function in the Selector Editor and point out that the part of the date that was different has been
replaced by “*”
• Open a third file and re-run the workflow. If it returns another error, replace the entire date by “*”
https://docs.uipath.com/studio/docs/selectors-with-wildcards
Selector Demo

46. 46
Exercise
• Go To https://www.rpasamples.com/opportunities
• Use Scrape Data to retrieve the Account list into a DataTable
• Use “Write Range” within Excel Application Scope
• Add a new column to the table with “Add Column Data”
• Loop through the DT and check if qty is over 40 -> add a note
• Use “Write Range” again (or write cell if it’s not too long) to update the Excel