An automation uses prompt engineering to extract key information from invoices by asking an LLM a series of questions about invoice fields, amounts, dates, etc. The LLM responses are then used to populate accounting systems. A second automation engages in a conversational prompt to diagnose and resolve IT issues, with the LLM providing recommendations that an RPA bot then executes. A third automation prompts an LLM on legal documents to identify key clauses, which an RPA bot then checks against compliance databases to flag any issues.

1. How to use Generative AI with the UiPath
Platform
Moonshot Subjects for the Hyper
Hackathon
June – July 2023
George Roth
Technology Evangelist UiPath
george.roth@uipath.com
https://www.linkedin.com/in/geroth/

2. 2
Explore Novel Use Cases
Build Better Automations Faster
UiPath Uses Gen AI in 3 Ways
Generative AI Powered
Automations
Incorporating Gen AI directly into
Business Automation Use Cases
Supercharge Developer
Productivity
“Co-Pilot” Like experience bringing a user-
friendly interface to building automations
Product & Model
Augmentation
Add LLM capabilities to improve
design time
1 2 3

3. 3
Advantages of using UiPath with Gen AI for Automation
More than calling a Generative AI Model
Context
Gathering
Specialized
Models
Robots Do Work Human In The Loop
Confidence
in governance
UiPath also has Specialized
models to complement Gen
AI interactions
Gen AI needs context,
UiPath can gather the
context from all sources
Gen AI is just a brain,
Automation is the muscle
that does the work
Gen AI “hallucinates”.
There are times you can
not get things wrong
Your business is governed
with audit logs and
controls while using Gen
AI with UiPath

4. 4
3 Patterns for Gen AI Powered Automations
A use case may use 1 or multiple patterns
Pattern
1
Reader / Writer
Pattern
2
Pattern
3
Analyst / Doer
Assistant
UiPath can execute
processes as a result of
LLM calls
UiPath can gather context
from multiple sources to
generate and distribute
personalized messages
UiPath can add context and
action to conversational
assistants
UiPath
Advantage
Source 1
Source 2
Source 3
Source 4
LLM
Ingest Analyze
Data Next Best Action
System 1
System 2
System 3
System 4
Do
LLM
System 1
System 2
System 3
System 4
Do
Conversational
Interface
Human
Knowledge
Response
Action
OR
Source 1
Source 2
Source 3
Source 4
LLM
Read Write
Emails Summaries Content

5. Appendix 1
3 Pattern Deep Dive

6. 6
Pattern 1 - Reader / Writer
Description:
LLMs are great at taking context and generating personalized text, whether longer or shorter than the original context. UiPath is collecting all the
relevant context and prompting the LLM for text. UiPath can keep a human in the loop to fine tune the output.
Example Use Cases
Cold Call Emails
Gather context about your audience and
generate email
Customer Feedback Response
Gather sentiment and customer history
to generate tailored response
Proposal Writer
Combine multiple answers from your KB
with additional context for a tailored
answer to a proposal question
Applicant Communications
Combine feedback from interviewers,
JD details, and applicant resume for
tailored communications
Customer Summary
Summarize customer history, support
ticket history, etc. for faster consumption
by customer facing agents
Email/PMO Summarizer
Summarize information from PM tools,
emails, other sources for faster
executive overviews
KYC Summarizer
Gather and summarize materials from
multiple sources for faster KYC review
Compliance and ESG Reporting
Monitor data and reports from multiple
systems/source and generate consistent
reporting
Product Documentation
Create and maintain product
documentation summarizing information
from feature tickets and marketing
Fraud Communications
Generate correspondence with
customers collating information multiple
systems
Insurance Claims Communication
Communicate to Customers around
their claims request synthesizing
information from multiple systems
Healthcare Appeals Communication
Tailor communications to customers
about using information about the
customer and the circumstance
Human Input
and Validation
Human in
the loop
Source 1
Source 2
Source 3
Source 4
LLM
Read Write
Emails Summaries Content
System 1
System 2
System 3
System 4
Distribute

7. 7
Pattern 2 – Analyst / Doer
Description:
LLMs can generate structured output (data tables, code, XML/JSON) from multiple unstructured sources when prompted well. UiPath is
collecting the relevant sources and prompting the LLM to generate structured outputs. Critically, UiPath can validate the contents of the
structured outputs vs. systems of record or humans. UiPath can further execute processes based on the output.
Source 1
Source 2
Source 3
Source 4
LLM
Ingest Analyze
Data Next Best Action
System 1
System 2
System 3
System 4
Do/Validate
Human Input
and Validation
Human in
the loop
Example Use Cases
Multi-Source Report Creating
Gather reporting from different systems,
documents, and emails and combine
them into one set
Structuring and Normalizing Data
Normalize data from different sources
into a common schema
2 Way Match (Generic Reconciliation)
Normalize data from multiple systems
and further reconcile the two noting
differences for humans to validate
Contact Center Next Best Action
Gather context from sources,
recommend an action from a list and
execute the action
After Call Work (Action Item Doer)
Extract actions from call scripts for
follow-up. Execute those automatable.
Upsell / Cross-sell Assistant
Gather customer history and needs,
generate a recommendation for what to
sell, and a script to sell it
Contract Extractor
Extract structured data out of contracts /
amendments, validate against sources,
and input into systems
Company Filing Extractor
Extract key figures out of company
filings, validate with human, and use in
processes
Test Data Creator
Generate test data for application
testing and insert it into the system
using UiPath
Generic Classifier
Take in unstructured sources and
classify against a list of defined options.
Use this data as input for processes
Competition Analysis
Monitor pricing, news, reviews of
competitors and extract structured
findings
Vendor Selection
Analyze proposals from multiple
vendors, extract key differentiations, and
recommend a vendor

8. 8
Pattern 3 - Assistant
Description:
The most common use case enterprises are building is a custom ChatGPT on their own knowledge sources. UiPath can augment and enrich
these chat interfaces with more knowledge sources, either directly in the LLM or in the prompt. For assistant interactions that result in an action,
UiPath be used for last-mile process execution.
LLM
System 1
System 2
System 3
System 4
Do
Conversational
Interface
Human
Knowledge
Response
Action
OR
Human in
the loop
Example Use Cases
Knowledge Base Assistant
Use UiPath to augment the knowledge
accessible to the LLM. Vectorize
databases or embed in the prompt.
Support Escalation Assistant
Variation of a Knowledge Base
Assistant focused on customer support
escalations.
Learning Assistant
UiPath can help find courses from
different learning platforms and
recommend one for a user
Self Service Helpdesk
LLMs will create enhanced assistant
interfaces. Execute common tasks
behind the scenes with UiPath.
Employee Benefits Assistant
Answer employee questions about
benefits and automate benefit selection
/ changing
Employee Travel Concierge
UiPath can bring in context on travel
policy and flight/hotel data to allow
users to book compliant trips faster
Legacy System Augmenter
Legacy systems likely won’t have LLMs
integrated. UiPath can be used to bring
LLM experiences to old systems.
Localization Assistant
LLMs have become decent at localizing
to different languages. UiPath can help
validate the output against other tools
Supply Chain Buyer Assistant
A conversational interface help speed
up inventory management and ordering
across suppliers for buyers
Personalized Assistant (JARVIS)
Use a conversational interface to
perform more Reader/Writer and
Analyzer/Doer actions
Guided Form Entry
A conversational interface collects &
validates input from users. UiPath can
input those answers into a system.
Ask GPT (Document)
Gather a specific document with
automation and then allow users to ask
questions about it

9. Appendix 2
The Importance of Context for Gen AI

10. 10
I’m an SDR at UiPath. Write a nice email to an
automation developer that should get their
interest and make them want to book a demo
call with me
A Prompt With Little Context Leads to Generic Results
Subject: Supercharge Your Automotive Automation with UiPath
Dear [Automation Developer's Name],
I hope you're doing well. My name is [Your Name], and I'm a Sales Development Representative at
UiPath, the industry-leading Robotic Process Automation (RPA) platform.
As an Automation Developer at [Company Name], I believe our solution can greatly enhance your
automation efforts, driving efficiencies in areas like manufacturing, supply chain, finance, and customer
service. With UiPath, you can rapidly build, deploy and scale automations, integrate with your existing
systems and boost your team's productivity.
Here are some key benefits of our platform:
Agility and Scalability: Quickly design and scale automations across your organization with our intuitive,
low-code platform.
Enhanced Accuracy: Reduce manual errors and improve decision-making with reliable, accurate data.
Increased Productivity: Free up your team for higher-value tasks by automating repetitive processes.
Seamless Integration: Connect with your existing systems, like ERP and CRM, for smooth coordination.
I would love to offer you a personalized demo so you can see first-hand how UiPath can revitalize your
business processes. To schedule a demo call, please use this [link] to select a suitable date and time.
I look forward to showcasing the powerful capabilities of UiPath and discussing its potential impact on
[Company Name]'s automation journey.
Best regards,
[Your Name]
Sales Development Representative
[Your Email Address]
[Your Phone Number]
UiPath Inc.
The Output
The Task The Context

11. 11
My name is {Ed Challis}.
My job role is {Sales Development Representative} at
UiPath.
I’m emailing {Tom Joseph} who works at {BMW}
{Tom Joseph}’s job title is {Automation Program Manager}
Write an email to {Tom Joseph}
That links to our top 3 use-cases in Tom’s industry:
• {use-case 1: Automotive}
• {use-case 2: Automotive}
• {use-case 3: Automotive}
The goal of the email is get {Tom Joseph} to request a
demo with me.
Adopt the UiPath style guide:
• {uipath style guide bullet 1}
• {uipath style guide bullet 2}
• {uipath style guide bullet 3}
A Prompt with Context Creates a Much Better Outcomes
Subject: Empower BMW's Automation Journey with UiPath's Top 3 RPA Use-Cases
Hi Tom,
I hope you're doing well! As BMW's Automation Program Manager, you have a profound impact on
driving innovation and optimizing operations. That's why I'm excited to share our top 3 RPA use-
cases that are tailored for the unique needs of auto-manufacturers like BMW:
1.Accounts Payable: Explore how auto-manufacturers leverage RPA to streamline invoice
processing and increase accuracy. Learn more: www.uipath.com/auto-ap
2.Order Management: Discover how RPA saves millions by boosting efficiency and reducing errors
in order management. Dive in: www.uipath.com/order-management
3.Customer Service: Uncover our approach to revolutionizing customer experiences within the auto
industry using automation. Check it out: www.uipath.com/auto-cs
Tom, I'd love to provide a personalized demo of our UiPath platform, focused on transforming BMW's
automation capabilities. Simply reply to this email or call me at 07793070297 to schedule a demo
call at your convenience.
Looking forward to showcasing how UiPath's RPA solutions can elevate BMW's operations and
customer satisfaction to new heights.
Best regards,
Ed Challis
Enterprise Account Exec
ed.challis@uipath.com
07793070297
UiPath Inc.
The Task The Output
The Context

12. 12
My name is {Ed Challis}.
My job role is {Sales Development Representative} at UiPath.
I’m emailing {Tom Joseph} who works at {BMW}
{Tom Joseph}’s job title is {Automation Program Manager}
Write an email to {Tom Joseph}
That links to our top 3 use-cases in Tom’s industry:
• {use-case 1: Automotive}
• {use-case 2: Automotive}
• {use-case 3: Automotive}
The goal of the email is get {Tom Joseph} to request a
personalized demo.
Adopt the UiPath style guide:
• {uipath style guide bullet 1}
• {uipath style guide bullet 2}
• {uipath style guide bullet 3}
UiPath is Uniquely Qualified to Gather Context From
Multiple Systems to Engineer a Prompt
HRIS
CRM
CMS
KBs
LLM

13. 13
Subject:
Create a personal intelligent assistant that can be specific for a certain role in an enterprise. Examples: Accountant
Personal Assistant, Payable Personal Assistant, etc.
Definition:
LangChain is a software development framework designed to simplify the creation of applications using large language
models (LLMs). As a language model integration framework, LangChain's use-cases largely overlap with those of language
models in general, including document analysis and summarization, chatbots, and code analysis.[
Examples:
LangChain: Introduction and Getting Started | Pinecone
LangChain In Action: Real-World Use Case With Step-by-Step Tutorial – YouTube
(68) I build an autonomous researcher via GPT | Langchain ⛓️ Tutorial – YouTube
(68) Hugging Face + Langchain in 5 mins | Access 200k+ FREE AI models for your AI apps -
YouTube
Using LangChain and Hugging Face in end to end AI
Apps with multiple LLMs

14. 14
Subject:
Create some automations that use multimodal AI, for examples text to voice, voice to text, text to image, image to text.
Also it can be an automation for example that extracts images from a document, and extracts texts from images.
Definition:
Multi-modal AI is a new AI paradigm, in which various data types like image, text, speech and numerical data are combined
with multiple intelligence processing algorithms to achieve higher performances. Multi-modal AI often outperforms single-
modal AI in many real-world problems. Multimodal AI engages a variety of data modalities, leading to a better
understanding and analysis of the information. The Multimodal AI framework provides complicated data fusion algorithms
and machine learning technologies.
Reads and Examples:
Multi-Modal AI Is the New Frontier in Processing Big Data (analyticsinsight.net)
(68) Multimodal AI and GPT – 4
(68) Aimesoft - Multimodal AI – YouTube
Multi Modal AI

15. 15
Subject:
Create a personal intelligent assistant that can be specific for a certain role in an enterprise. Examples: Accountant Personal
Assistant, Payable Personal Assistant, etc.
Definition:
An intelligent personal assistant (IPA) is a software agent that can perform tasks or services for an individual using natural
language. These tasks or services may include information search, location services, remote activation, appointment
management, health data monitoring, online purchase, and entertainment. IPAs may also integrate with Internet of Things
(IoT) devices. IPAs are usually available as smartphone or other mobile device applications.
It could be implemented as an App or as an Attended Bot.
Reads and Examples:
'This is just the beginning of a revolution': DeepMind cofounder says personal AI assistants will make us better at basically
everything (msn.com)
(68) BMW’s Intelligent Personal Assistant – YouTube
(68) How to Create Personal AI Assistant | Like Iron Man - YouTube
Personal Intelligent Assistant

16. 16
Subject:
Create automations and use collection of prompts to solve tasks by accessing LLMs. This can be combined with other AI are
techniques and models.
Definition:
Prompt engineering is a concept in artificial intelligence, particularly natural language processing. In prompt engineering,
the description of the task that the AI is supposed to accomplish is embedded in the input, e.g. as a question, instead of it
being explicitly given. Prompt engineering typically works by converting one or more tasks to a prompt-based dataset and
training a language model with what has been called "prompt-based learning" or just "prompt learning".
Reads and Examples:
Prompt engineering for reusable activities
(68) ChatGPT: 5 Prompt Engineering Secrets For Beginners – YouTube
Promptbase.com
Prompt Engineering