SlideShare a Scribd company logo
Chatbots in HR
Improving the
Employee Experience
Kong Yean Hwei, Amy | Advanced Work Based Project Report
Crew 4 | Hyper Island Singapore | Jan 2017
2
Table of Contents
EXECUTIVE SUMMARY ……………………………………………………………. 2
WHAT ARE WE EXPLORING? …………………………………………………….
Changing Landscape in Advertising ……………….…………….……………
Digital is Part of the Process ……………………………………….…………….
Why Chatbots, and Why HR? ……………………………………………………
4
4
4
5
EXPLORING THE LANDSCAPE – A LITERATURE REVIEW ………………
Origin and Development of Chatbots …………………………………………
Why are Chatbots of Such Interest Now? …………………………………….
Applications andBenefits ofHRChatbots ……………………………………
Chatbots in PubComms SG for HR Information Services ……………….
Designing and Building Chatbots ………………………………………………
Some Considerations When Implementing Chatbots ……………………
Learnings and Conclusions for an Experiment ……………………………..
6
6
7
9
11
11
14
15
THE HR CHATBOT EXPERIMENT ……………………………………………….
Experiment Considerations ………………………………………………………
Experiment Design ……………………………………………………………….
Experiment Round 1 with Twine – New Insights …………………………...
Experiment Round 2 with Chatfuel – Validation and More …………….
Overall Findings and Conclusion ………………………………………………..
Limitations of the Experiment …………………………………………………..
16
16
16
18
24
28
30
REVIEW AND RECOMMENDATIONS ……………………………………………
Yay or Nay? ……………………………………………………………………………
How Much Does It Cost? ………………………………………………………….
It Could Save SGD 125,000 Annually …………………………………………..
30
30
30
30
Suggested Timeline and Next Steps …………………………………………..
Other Recommendations and Conclusions ………………………………….
The Time is Now …………………………………………………………………….
30
32
32
REFLECTIONS ………………………………………………………………………….. 32
REFERENCES ..……………………………………………………………………..….. 35
APPENDICES ..………………………………………………………………..………..
Appendix 1: Key Questions to Ask Before Investing in a Chatbot ……
Appendix 2: Proposal to PubComms SG …….…………..……………….…
Appendix 3: Qualitative Interviewing Framework …..…..…………....…
Appendix 4: Twine Documentation on ERIS Prototype 1 .......………..
Appendix 5: Experiment Data and Analysis ………………………….……..
Appendix 6: Chatfuel Documentation on ERIS chatbot ………….……..
41
42
ABOUT THE AUTHOR
The author was most recently the HR Director in Publicis Communications
Singapore. She held dual roles, leading HR operational responsibilities for
Publicis Worldwide, MSLGROUP and Nurun, and the Learning & Development
function for all brands within Publicis Communications Singapore, including the
brands mentioned above plus Saatchi & Saatchi, Leo Burnett and Prodigious.
This Advanced Work Based Project is based on Publicis Communications
Singapore.
REDACTED
REDACTED
3
EXECUTIVE SUMMARY
Technology has disrupted the advertising and communications industry.
Agencies are seeing a chronic decline in industry prices of 4.5% to 5% per
year, compounded. Companies are trying to capitalize on this digital wave,
including Publicis Groupe, the holding company of Publicis
Communications,which has set this as a strategicpriority for the company.
It is logical to assume that internal company processes should also reflect
these digital priorities, and this Advanced Work Based Project aims to be
part of the solution by exploring how technology applied to processes
within Publicis Communications Singapore can provide value.
Specifically, this report sets out to explore the following problem
statement: “How might we explore the potential and applications of
chatbots in HR, specifically in Publicis Communications Singapore, to
improve the employee experience?”
This report reviews the origin and development of chatbots, and the
confluence of recent developments that have triggered the surge of
interest in chatbots. It also reviews the potential applications of chatbots
in HR and its benefits, and considerations needed for the design,
development, and implementation of chatbots.
An experiment using the Build-Measure-Learn feedback loop of the Lean
Startup methodology was conducted in Publicis Communications
Singapore to gain employee feedback on the problem statement. Two
prototypes were tested with a small group of employees; one was a low-
fidelity prototype using the Twine program, and the other a chatbot
Minimum Viable Product (MVP) written using Chatfuel, a Do-It-Yourself
chatbot builder.
To access the ERIS MVP on Chatfuel, please email the author at
amy.kong@hyperisland.sg for an access link.
Based on the learnings of the literature review, the chatbot was developed
as a text-based, rules- and retrieval-based, closed domain chatbot with
some level of artificial intelligence. The topic of the chatbot was focused on
providing information on a specific HR policy.
Interviewees were asked to interact with the prototype, then semi-
structured interviews were conducted to get their feedback on the
prototype and their views on the usefulness and potential applications of
chatbots in the HR function in PubComms SG.
Overall results show that employees like chatbots’ ability to provide instant
access to information, the ability for them to gain answers independently,
and friendly tone. Each mentioned at least two other areas of applications
of chatbots in HR, including providing information on company policies,
processes, onboarding, and training.
Additional insights that emerged included that interviewees felt chatbots
would help them to avoid embarrassment when they needed to ask certain
types of questions, and that chatbots could provide anonymity when they
need to read up on sensitive company policies.
Interviewees were clear that they would go to a chatbot for functional
queries, while they would go to a HR team member for counsel on non-
routine requests or emotional issues. This opens the possibility that
chatbots could help to transition HR away from a transactional role towards
an expertise role in the company.
Based on the results of the experiment, there is significant potential and
value in investing in HR chatbots to improve the employee experience and
employee productivity. Calculations show that Publicis Communications
Singapore could save about SGD 125,000 annually if just 5 minutes were
shaved off each HR request from employees.
4
WHAT ARE WE EXPLORING?
Changing Landscape in Advertising
Technology has disrupted most industries, and it is no
different in advertising. Where advertisements
previously lay predominantly in the domain and
control of advertising agencies, the advent of
technology has changed that.
The industry has shifted dramatically and advertising
agencies now face challenges on multiple fronts:
• Emergence of new forms of advertising such as
online, mobile and search engine marketing (SEM)
that has seen a redistribution of budgets from
traditional forms of advertising (Pandey, 2016)
• Competition from companies that were never
considered competitors before, such as
o Management consultancies (Dan, 2016;
Roxburgh, 2016; Vranica, 2016)
o Publishers (Marshall and Alpert, 2016)
o Digital tech companies (Johnson, 2015)
o In-house agencies (Schaefer, 2015; Stiglin,
2015; Morrison, 2016)
• Commoditization of the industry as technology
enables anyone to become creative content
creators (Fromowitz, 2016; Oetting, 2015)
As a result, the industry is seeing huge pressure on
prices and margins. According to Farmer (2015),
declining fees (at 2-3% annually) and growing
workloads (2-3%annually)overthepast 20years have
resulted in a chronic decline in industry prices of 4.5%
to 5% per year, compounded.
Digital is Part of the Process
Many companies, and industries, are trying to
capitalize on this digital wave, including Publicis
Groupe. Publicis Groupe is the holding company that
owns Publicis Communications (PubComms), the
Client brand that this Advanced Work Based Project
(AWBP) is based on.
In fact, it is of such a strategic priority to Publicis
Groupe that it separately reports its digital revenues
in its quarterly earnings reports, and current digital
revenues stand at 54% of total Groupe revenue in the
third quarter of 2016 (Publicis Groupe, 2016b).
It is also investing in training and development efforts
for its employees. In its 2015 Annual Report
Registration Document (Publicis Groupe, 2016c), it
states, “…the goal is to ensure that each employee is
able to acquire basic know-how, whether in
rudimentary coding or better understanding the
latest generation applications. The boom in mobile
device usage and the new challenges of
interconnectivity (connected objects) have led
agencies to hire and train talented individuals with
multiple skills, given the speed of industry change… It
is essential to support our future managers.”
If employees are expected to be digitally-savvy and
able to recommend leading edge digital
communications solutions for clients, then it is logical
to assume that internal company processes should
reflect these digital priorities as part of the digital
immersion for company employees.
According to Edgar Schein (cited in Christensen et al,
2016),processes are a criticalpart ofan organization’s
unspoken culture. They tell people inside the
company, “This is what matters most to us.”
However, there are some that say that despite all the
rhetoric, things are not changing:
“What’s broken is not our people. It’s our process. It’s a process
that has been endlessly debated but not reinvented,
and it has not adapted to the changing world around us.”
Grayson and North, 2016
5
“What’s broken is not our people. It’s our
process. It’s a process that has been endlessly
debated but not reinvented, and it has not
adapted to the changing world around us... We
traffic in risky ideas. But we don’t make any
bold moves to change how we operate
internally.” (Grayson and North, 2016)
This Advanced Work Based Project (AWBP) aims to be
part of the solution, by exploring how technology
applied to processes in a company – Publicis
Communications Singapore (PubComms SG) - can
provide value. Specifically, the use of chatbots in HR.
Why Chatbots, and Why HR?
Chatbots are a hot topic now, but most of the current
discussion seems to focus on the use of chatbots to
engage with customers. One of the functions of
Human Resources (HR), as a parallel, is like the
employee customer service function of the company.
So, would it be a far stretch to say that chatbots could
also have significant potential in HR?
Currently, most of the software or technology used in
the PubComms HR function are used for non-
employee facing activities (backend functions) to
ensure operational continuity or as a management
reporting tool, e.g. payroll systems, PTalent (Publicis
Groupe’s HR Information System). There are
currently few technologies used that facilitate or
assist the interaction between the HR function and
employees, and all interactions are manually
managed, e.g. HR queries, notifications to employees
on HR updates etc.
Improving the employee experience cannot be
underestimated. Advertising agencies are expertise-
led companies, i.e. talent is a strategic differentiator,
and so talent attraction and retention should be key
activities for the company. This is even more
important now, as new competitors like tech firms
and startups are aggressively wooing talent away
(Tadena, 2015; Ember, 2016).
This represents an area of opportunity for exploration,
to use technology to improve processes and help
create a company culture that is aligned with its
strategic priorities, that in turn improves the overall
employee experience.
As such, the defined problem statement that we will
use in this paper is:
“How might we explore the potential and
applications of chatbots in HR, specifically in
PubComms SG, to improve the employee
experience?”
ABOUT PUBLICIS COMMUNICATIONS
BACKGROUND
PubComms is one of 4 Solutions hubs within Publicis
Groupe and comprises all of Publicis Groupe’s creative
brand networks, including Publicis Worldwide,
MSLGROUP, Nurun, Saatchi & Saatchi, Leo Burnett,
BBH, Marcel and Prodigious (Publicis Groupe, 2016a).
Publicis Groupe is one of the largest marketing and
communications company in the world, alongside
WPP, Interpublic, and Omnicom (Elliott, 2002).
In early 2016, a Chief Executive Officer was appointed
to lead Publicis Communications in Singapore
(Mumbrella Asia, 2016), and subsequently a Chief
Talent Officer and a Chief Finance Officer. The finance
and HR functions across the various brands were
consolidated to align with the new management
structure, overseeing a headcount of about 250
people.
As each creative brand previously had its own HR
policies, the HR team (author included), was tasked to
align and improve employee and HR operations and
policies across the various brands under PubComms
SG.
CLIENT STAKEHOLDERS
The key client contact for this project was Dan
Spencer, Chief Talent Officer, Singapore and
Australia/New Zealand, Publicis Communications.
Other key stakeholders included the HR leaders of
PubComms SG, Sharon Ooi, HR Director, and Adele
Sam, Talent Manager, who provided the information
and liaison support in the execution of the
experiment.
REDACTED
REDACTED
REDACTEDREDACTEDREDACTED
6
EXPLORING THE LANDSCAPE – A LITERATURE REVIEW
Origin and Development of Chatbots
What is a Chatbot?
Different terms have been used for chatbots (or
chatterbots), including conversational agents,
dialog systems and virtual assistants. This is due to
the proliferation of a variety of similar systems built
with different technical architecture (Perez-Marin
and Pascual-Nieto, 2011), for different purposes.
For this report, we define a chatbot as an automated
online software program that tries to mimic a
human conversation in its interaction with a human
user to answer questions or perform tasks (Shawar
and Atwell, 2007; Knowledge@Wharton, 2016).
Specifically, we will focus this report on text-based
chatbots.
Chatbots can be built for websites or apps and
accessed through smartphones, tablets, desktops
and tablets. In recent years, most of the major
messaging platforms have launched tools to help
developers integrate or deploy chatbots on their
platforms.
These include Facebook Messenger, Slack,
Telegram, Google and Microsoft, Kik, among others
(Richman, 2016).
What makes a Chatbot?
For a chatbot to conduct human-like conversation,
ArtificialIntelligence (AI) is a criticalcomponent.Nils
J. Nilsson, as cited by Stone et al. (2016), defines AI
as the “activity devoted to making machines
intelligent, and intelligence is that quality that
enables an entity to function appropriately and with
foresight in its environment”.
AI has many applications (Stone et al., 2016), but in
this report, we will narrow the topics to those
relevant to text-based chatbots.
Chatbots functions in two ways (Schlicht, 2016):
1. Rule-based: These types of chatbots respond to
very specific commands (rules), and are unable to
process if information is not input in the right way.
It is only as smart as you program it to be
2. Machine Learning: Machine Learning is a form of
AI, and machine-learning chatbots continuously
get smarter as they learn, via Natural Language
Processing (NLP), from the conversations it has
with people
The recent boom in chatbots has been fuelled by
advancements in AI research, combined with
technological improvements such as the availability
of cloud computing resources and wide-spread,
web-based data gathering (Stone et al., 2016).
These advancements in AI include the areas of:
• Natural language Processing (NLP) and Natural
Language Understanding (NLU)
NLP explores how computers can understand
and manipulate natural language text or speech
to do useful things (Chowdhury, 2005). NLU is a
subset of NLP that deals with machine reading
comprehension (Ovchinnikova, 2012), i.e. the
ability of the computer to understand human
language. In short, NLP applications try to
understand natural human communication and
communicate in return (Marr, 2016).
• Machine Learning
With machine learning,computers areenabledto
continuously learn from data on its own, using
algorithms (set of rules or steps), to find insights
without being explicitly programmed where to
look (Sas.com, n.d.). In chatbots, machine
learning is used to help machines learn and
understand the vast nuances in human language,
and learn to respond in a manner that the
audience is likely to comprehend (Marr, 2016).
While this report will not go into the details of these
technologies, these are key terms to understand as
they will be mentioned in any discussion around
chatbots.
7
Key Developments in AI and Chatbots
Turing Test and Loebner Prize Competition
One of the most significant milestones was created
when Alan Turing created “The Imitation Game”,
now known as the Turing Test, in 1950.
Starting with the question “Can machines think?”
(Turing, 1950), Turing created a test that would
determine if a computer is capable of thinking like a
human. The Turing Test has since been used as a
benchmark that any AI must pass en route to true
intelligence (Ball, 2015).
However, it was not until 1991 that there was an
implementation of the Turing Test (Loebner.net,
n.d.b) – the annual Loebner Prize Competition –
created by Dr. Hugh Loebner to advance the field of
AI (Loebner, n.d.a). As long as there is an entry for
that year, the prize will be awarded, and so while
there have been multiple Loebner Prize winners, no
program passed the Turing Test until 2014 (Griffin,
2014).
The First Chatbot, ELIZA
Then in 1966, the first chatbot ever coded, ELIZA,
was documented by Joseph Weizenbaum. ELIZA
was written to simulate a psychotherapist
conducting a psychiatric interview (Weizenbaum,
1966). ELIZA worked by analysing the words users
inputandmatching themto a list ofpossible scripted
responses (Newman, 2016).
More critically, ELIZA seemed so believable to users
that Weizenbaum perceived his program as a threat,
and in his book Computer Power and Human Reason,
he attacked AI (specifically ELIZA) and computer
science research, thereby slowing the pace of
research into AI (Epstein, Roberts and Beber, 2008).
A.L.I.C.E
In 1995, A.L.I.C.E. (Artificial Linguistic Internet
Computer Entity), the first AIML-based (Artificial
Intelligence Markup Language) personality program,
was created.
AIML is a computer language designed for creating
natural language-based chatbots that is still used
today.
A.L.I.C.E.won theLoebnerPrize three times,in 2000,
2001 and 2004 (Epstein, Roberts and Beber, 2008),
and is one of the strongest AI programs of its time.
It was programmed to give the illusion that it was
intelligent and self-aware, but was driven by
“supervised learning”, where a botmaster monitors
the conversations and creates content to make
responses more appropriate, accurate, “human”, etc
(Epstein, Roberts and Beber, 2008).
Bots for Messenger
In April 2016, Facebook launched the bots for
Messenger Platform (Marcus, 2016). While
Facebook is not the first company to launch a bot
platform, it is significant because it now provided, at
scale, the opportunity for businesses to automate
their one-to-one engagement with their customers
to drive business and e-commerce (Rosenberg,
2016), thus driving the current interest and boom in
chatbots.
Figure 1 shows a more detailed timeline on the
history of chatbots.
Why Are Chatbots of Such Interest Now?
A confluence of several developments has triggered
the surge of interest in chatbots:
Advancements in AI Research and Computing
As mentioned, advancements in AI research,
particularly in machine learning and NLP, are
enabling chatbots to provide more accurate and
intelligent responses.
Improvements in computing and hardware
technologies have also enabled integration of
various apps into a chatbot that allow chatbots to
complete tasks with little or no operator
intervention.
In fact, research firm Gartner predicts that in 2017,
only 33% of all customer service interactions will
need a human intermediary, as compared to 60% in
2014. (Gartner.com, 2015)
8
Figure 1.: The History of Chatbots (Futurism, n.d.)
9
Rise of Mobile Messaging and Fall of Apps
According to Business Insider (2016), monthly active
users of messaging apps (where chatbots can reside)
surpassed social networks in 2015 in terms of
monthly active users (see Figure 2).
Companies also face increasing friction in getting
consumers to download and use a mobile
application. The average mobile user globally has 33
apps installed on his or her device, of which 12 apps
are used daily. Of that, 80% of the average global
mobile user’s time is spent only on 3 apps (Meeker,
2016).
This means that companies and brands are finding it
increasingly difficult to engage one-on-one with
customers.
App Integrations Attract Users to Stay Within
Platform Eco-system
The ability to integrate various apps into a chatbot
also means that users are attracted to stay within
the messaging app (Bayerque, 2016).
In the context ofbrands,it means that customers are
likely to stay within the brand eco-system, thereby
increasing duration of interaction with the brand.
Bots Can “PiggyBack” Existing Massive Platforms
Bots are the beginning of micro apps on the backs of
massive platforms which will lead to more focus and
reach for start-ups and more delighted users
(Batalion, 2016) because brands can:
1. Spend less time in development, e.g. building a
chatbot on Facebook Messenger allows you to
build one experience (without needing to write
for different mobile phone sizes, or operating
systems) and access Facebook user rich profiles
instantly
2. Focus more time on creating positive user
experiences because they can leverage richer
user interfaces and software plug-ins (that
platforms develop or offer) to process input
What all this means is that companies and
organisations can now increase quality one-to-one,
personalised engagements with customers at scale,
at potentially reduced costs and time.
Applications and Benefits of HR chatbots
Current Applications of Chatbots in HR
General HR Support
Information Services (Search and Retrieval)
A typical challenge that employees face is knowing
where and how to find information on HR policies
and processes, or even whom to approach within the
HR department for specific queries.
Benefits of using chatbots in information services
include (McNeale and Newyear, 2013):
• Tackling the problem of users not familiar with
specific terminology and jargon
• Unlike humans, chatbots are unruffled by rude
customers, high traffic, or repeated requests for
the same information, and remain consistently
patient and polite
This also shifts the responsibility of locating the
needed information from the user to the designer of
the chatbot where the designer leads the user
through a question and answer dialogue to discover
the information needed and provide it (McNeale and
Newyear, 2013).
An example of such a chatbot is “Ask Ivy”, a “virtual
HR agent” launched by Intel in 2013 that uses a
combinationofNLP,AIandoptimizedsearchhosted
on their Intranet to help answer employees'
questions about their pay, stock, benefits, or other
HR programs (Pearce, 2013).
Figure 2.: Messaging Apps Surpass Social
Networks (Business Insider, 2016)
10
HR Task Automation or Assistance
Virtual personal assistants are now a big focus due
to the ability to now integrate several apps (and
actions) into chatbots.
The most current and well known examples are
iPhone’s Siri, Amazon’s Alexa, Microsoft’s Cortana
and the Google Assistant (Dunn, 2016).
In the domain of virtual chatbot assistants in HR,
there are some products now in development in the
market, including:
• HR task automation chatbot from ADP
Innovation labs that is still in development
(Boulton, 2016)
• Mila, overstock.com’s chatbot created to assist
with employee sick leave reporting and team
rescheduling at their call center (Greenfield, 2016;
Aspect.com, n.d.)
Recruitment
Streamlining Job Applications
Chatbots can be used to vet potential candidates
(Bhaduri, 2016). One recent example is Mya
(http://trymya.io/), a chatbot that vets and
interviews job candidates, using textual analytics to
filter irrelevant resumes and interviews selected
candidates on their work experience and
qualifications. Mya also administers tests, provides
application status updates and tips to candidates
(Dishman, 2016; Deutscher, 2016; Trymya.io, n.d.).
Q&A Tool for Job Applicants
Chatbots can also serve as a Q&A resource for
potential candidates on the position they are hiring
for, or the company’s hiring policies (Sullivan, 2016).
Benefits of using chatbots this way include: Higher
quality of applicants (through candidates’ self-
selection), improved candidate experience, reduce
time spent by recruiters (and costs) in answering
repeated FAQs from candidates.
One of the most established examples is Sgt. Star,
launched in 2007 by the United States Army to help
potential recruits to learn about a career in the Army.
First residing on the Army recruitment webpage,Sgt.
Star then made its debut as a mobile app in 2014.
(Maass, 2014; Nextit.com, n.d.).
Onboarding
Chatbots can also beusedto contactnew employees
(or allow them to self-serve) with onboarding
information, such as organisational structure and
history, how to get enrolled in programs or benefits,
suggestions on colleagues to meet, and links to the
required HR forms, in place of a HR generalist
(Boulton, 2016; Zwier, 2016).
Training (Bhaduri, 2016)
There is already ongoing discussion in academic and
education sectors on the value of chatbots in
training and learning that can be applied to the
corporate sector (Perez-Marin and Pascual-Nieto,
2011; Olson, 2016). Chatbots can be used to deliver
interactive and personalised training, monitoring
and nudging, and instant help (Newton, 2016; Riel,
2016).
Benefits of Using Chatbots in HR
Based on the above, the potential benefits of using
chatbots in HR include improving:
• Employee productivity and employee
experience by:
o Providing on-demand employee
engagement with 24/7, instant responses
o Simplifying information access
o Supplying just-in-time and fast access to
information
o Personalisation of the employee interaction
• HR team’s performance (value-add, productivity
and motivation) by
o Handling repetitive tasks (e.g. answering
FAQs) thereby
- Freeing up HR resources to focus on
more value-added work or reduce costs
- Reduce frustration and/or monotony of
the HR team
o Automating standardised tasks
o Identifying potential issues using data
analytics and addressing them before they
escalate
11
Chatbots in PubComms SG for HR
Information Services
The experiment conducted to explore the problem
statement of this report focused on the use of
chatbots to provide HR information services in
PubComms SG as it strikes the best balance
between feasibility and desirability.
Why Information Services?
Firstly, the HR function in PubComms SG already
has ready content that can be re-purposed to
populate any chatbot prototype.
Secondly, as a first investment in chatbots, the
easiest chatbots to build are rules-based chatbots
that rely on specific topics (closed domain) with
predefined responses (retrieval-based) (Figure 3.).
Having machine learning capabilities is a bonus and
can be incorporated into the chatbot with
subsequent iterations (Varagnat, 2016).
Jumping into a chatbot that requires extensive
machine learning capabilities from the start will
create barriers of entry as it can be time-consuming
(due to lack of expertise) and potentially costly.
What About the Other Applications of
Chatbots?
While recruitment is a constant needfor PubComms
SG, it is not feasible because:
• In the communications agency environment,
the company information that candidates
would be interested in (e.g. client roster) is
considered confidential. It is not available on
public forums and usually only communicated
face-to-face to the candidate verbally during
the interview. The highly customised nature of
remuneration in the advertising industry also
does not suit the use of chatbots for recruitment
Q&A at this point in time
• Using chatbots to vet potential employees
require sophisticated filtering technology as the
interviewing process also involves interviewing
for soft skills. The technology is still very new,
and to demonstrate the potential of chatbots in
PubComms SG, it is better to choose an area
that is more established. Additionally, the
various hiring managers within PubComms SG
have different interviewing and selection
criteria, and it would be difficult to standardise
this quickly to build the prototype. Finally, most
off-the-shelf products in the market are US-
based and there may be cultural differences in
the way resumes are written, or how candidates
respond in an interview.
In the area of HR task automation, companies
openly admit that it is very early days and all these
assistants are far from polished (Dunn, 2016).
As for onboarding and training, there are currently
no consolidated company information materials
(due to the recent re-organisation) that can be used
to populate the chatbot. Use cases of the value of
chatbots in corporate training in the industry are
also still few.
Designing and Building Chatbots
This section is split into the following subsections,
to explore best approaches for designing and
building chatbots:
1. What Consumers Want
2. Three Approaches to Creating Chatbots
3. User Experience Design Consideration
REDACTED
12
Appendix 1 also provides a list of key questions to
consider prior to investing resources in a chatbot.
What Consumers Want
Prior to designing a chatbot,it is worth exploring the
functions and experiences consumers want. As this
is a relatively new area of interest commercially,
there are no any large scale, benchmark global
consumer surveys that have been conducted, but
some surveys have been conducted in specific
markets. These provide starting insights that could
be validated through Proofof Concept experiments.
Based on results of three recent consumer surveys,
one in the US and two in the UK with sample sizes of
1,000 respondents each (Aspect, 2016; Mindshare,
Goldsmiths University of London, 2016; myclever
Agency, 2016), these are some of the findings
grouped around certain central themes:
“Humanness” of a Chatbot
Consumers prioritize efficiency in understanding
their queries over the personality or friendliness of
the chatbot. This was raised in the two UK surveys.
However, the US survey indicated that consumers
expect chatbots to perform better in “friendliness”
and “ease of use” as compared to “accuracy” and
“interaction success”. These could be inter-related,
i.e. consumers don’t have a good experience with
the efficiency of chatbots, therefore prioritising
them higher than the tone of interaction of chatbots.
Expected Benefits
• Getting 24-hour service
• Quick answers to simple to moderate questions
• Getting an instant response
• Being able to self-serve without having to talk to
a customer service agent
Predicted / Expected Functions of Chatbots
• Quick emergency answers
• Forwarding to appropriate human when
requested by user
• Buy basic items (clothes, food)
• Retain information on context and history of all
previous interactions for a more personalised
experience
• Get basic information about a company, product,
or service
• Information on product availability
Three Approaches to Creating Chatbots
There are three ways to create chatbots (Vorobiov
and Kotsurenko, n.d.):
1. Buy a ready solution: there are vendors who
offer off-the-shelf solutions. Examples of HR-
specific chatbot solutions include Mya,
mentioned earlier, Talla for onboarding
employees within Slack (https://talla.com/). This
allows businesses to speed up time-to-market,
but the downside is cost and not all ready-made
solutions are customisable.
2. Use DIY chatbot builders: there are a variety of
Do-It-Yourself (DIY) bot builders, some requiring
technical knowledge, and others not requiring
any. A higher degree of customisation is possible,
and it is relatively cheap and quick. Functions
may be limited but new features are constantly
being added.However,it can be hardto compare
functionalities and prices across chatbot builders
as business models vary widely.
3. Build a chatbot from scratch: While technically
still the most challenging, there are many
developer tools available in the market that
make chatbot building much easier now.
Naturally, this option requires technical
expertise and more cost, but results in a solution
that is tailor-made for, and proprietary to, the
company.
Figure 4 shows an infographic ofthe current chatbot
landscape. This infographic does not include
chatbot development vendors as they are industry-
specific, but it gives a good overview of the market.
As the objective of this report is to explore the
potential and applications of chatbots in HR to
improve the employee experience, we will focus on
using DIY chatbot builders (referred to as chatbot
builders from here) as it is the cheapest and fastest
way to build a prototype, and evaluating technical
considerations is not an area of discussion here.
13
User Experience Design Considerations
Given that part of this report’s problem statement is
concerned with improving the PubComms SG
employee experience, we need to examine how to
create good User Experience (UX) in a chatbot.
Thisis especially whenchatbotsnowenableone-on-
one interactions between the user and the company,
the quality of that interaction becomes more
important.
Many articles use the terms UX and User
Interface (UI) interchangeably. For clarity, we
attempt to make a distinction between the two. The
industry perspective of UX focuses on the
“optimisation of a product for effective and
enjoyable use”, including ensuring the product
logically flows from step to step (Lo, 2014;
Lamprecht, 2016).
UI, on the other hand, is concerned with the “look
and feel, the presentation and interactivity of the
product”that guides theuser intoa goodinteraction
with the product.
To simplify it even further, the comparison between
UX and UI:
“Something that looks great but is difficult to
use is exemplary of great UI and poor UX.
While something very usable that looks
terrible is exemplary of great UX and poor UI.”
Helga Moreno, as cited by Lamprecht (2016)
In this context, a large part of creating a good UI is
already handled by chatbot builders as their
products come with pre-set user interfaces. Some
industry best practices to designing a good UX in
chatbots (Ady, 2016; Howard, 2016; Jain, 2016; Yao,
2016):
Smallest Number of Steps
Plan the conversation flowto be efficient,so that the
user reaches his goal in a few steps as possible.
Always Provide a Path to the End Goal
Give users cues to reach next steps that lead to the
end point, so that they are not left hanging mid-way
in a conversation without knowing what to do.
Figure 4.: Chatbots Landscape (Medium, 2016)
14
Decide Parameters of Information to be Stored
If one of the aims of the chatbot is to provide a
personalised experience, then information about
the user needs to be retained / stored. Establishing
parameters (or categories) of information to be
stored helps to provide targeted or relevant
information for users. For example, if there are
different sets of employee benefits associated with
different job titles in the company, establishing job
titles as a parameter allows the chatbot to retrieve
the different sets of employee benefits by the user’s
job title.
Set the Right Expectations
• Explain the purpose of the chatbot
• Explain what the chatbot can do
• Explain how to interact with the chatbot
• Set a first action for the user to take
• Provide hints or prompters on the possible paths
of exploration at the end of the onboarding
Allow Flexibility of Exploration
Allow users the flexibility to skip around options.
Use Buttons to Supplement Free Text Input
Particularly for closed domain chatbots, buttons
provide more accuracy in interpreting users’
requests since the options are pre-defined. Clicking
a button is also faster than typing text.
Layer AI to Interpret User’s Requests
Use AI to learn the various ways users phrase
requests differently so that the chatbot can trigger
the right response or action, e.g. “Hi”, “Hello”,
“Wassup”, “Yo” should trigger a return greeting or
opening message from the chatbot.
Use Images or Graphics to Supplement Text
Images or graphics are easier to understand than
text explanations.
Provide a Null Response
If the chatbot is unable to understand what the user
wants,a message shouldbe triggeredto indicate so.
Non-responses from the chatbot where the user is
left not knowing what to do next should be avoided.
Tone Should Reflect the Brand Personality
For example, the chatbot may want to use more
casual, trendy words if it is targeted at millennials.
Also consider the use of emoji or gifs or graphics
appropriate to the chosen tone of voice.
Help Users Get Help
Provide options for users to get help outside of the
chatbot as needed. This can include directing the
user to documentation, or a human point of contact
for further inquiries.
Integrate with Existing Business Systems
Enable chatbots to access CRM databases to allow a
richer experience with users and avoid having the
user repeat their history or user details every time
they interact with the chatbot.
Some Considerations When
Implementing Chatbots
Hype
There is a lot hype currently around chatbots, such
that the general public's expectations of what
chatbots can do will exceed the reality of what they
can actually do (Hobson, 2016).
This means that in launching any chatbot,
companies should take care to communicate clearly
to users what they should expect from any
interactions with that specific chatbot.
The reality also is that not every bot needs to be
sophisticated, and it will depend on the objective or
outcome that the chatbot is built for.
Ifit is meant as an informationalservice,there is very
little point in building a bot to conduct a
conversation with a user, when the bot is meant to
be transaction-based (Newman, 2016).
Companies that are considering implementing
chatbots need to be clear about the objective or goal
of the chatbot and evaluate the technologies or
functions needed to achieve that, rather than
include unnecessary user features that result in
additional cost for the company.
15
Benchmark for a Minimum Viable Product (MVP)
Software companies are used to developing a MVP,
or a minimum standard version of the product that
consumers would be willing to buy, to test in the
market.
However, for a good user experience, the product
will likely need to have more accurate NLP and
information before a MVP can be developed, which
may mean that chatbots could require more capital
than a traditional web or mobile app, where good
frameworks are more commonly available (May,
2016).
As above, clarity in the goal of the chatbot is
essential,to be able to strip down to a MVPfor users.
Alternatively, companies can first pilot a chatbot in a
very confined and specific area to minimise capital
outlay, while not compromising on the UX of the
chatbot. This is the approach that has been adopted
in developing the chatbot prototype for PubComms
SG, as seen in the next sections of this paper.
Learnings and Conclusions for an
Experiment
Chatbots are not new, but advancements in AI and
computing technology have enabled the
development of chatbots that can understand users
better, have more meaningful conversations and
perform tasks more effectively and accurately. This
has opened chatbots for commercial possibilities at
scale,particularly chatbots on messaging platforms.
There are multiple possibilities of applying chatbots
in HR; specific to PubComms SG, it is recommended
that the focus of any chatbot pilot be focused on
providing information services on specific topics
that allow a MVP to be developed quickly and cost-
effectively for piloting with employees.
Also, the best approach in creating one seems to be
the use of chatbot builders to minimize cost,
increase the speed of prototype development and
remove the constraints of needing technical
expertise. It also allows more effort to be focused on
providing a quality user experience to help
employees better visualise the value and
possibilities of using chatbots in the HR function of
PubComms SG.
16
THE HR CHATBOT EXPERIMENT
Experiment Considerations
Demonstrate Proof of Concept (POC)
PubComms SG has never implemented similar
technology in its support functions, so it was
important to gain enough feedback to demonstrate
POC to the PubComms SG HR team for them to
evaluate if resources should be invested in such a
tool, and provide justification to higher
management on its value.
Lean Resources to Build the Experiment
As there was no budget, the experiment would have
to be built on free platforms.
Platforms That Do Not Require Much (or Any)
Coding Knowledge
As the author has no background in coding, the
chatbot would have to be built on tools that require
little to no coding knowledge. As the objective of
this experiment was to gain employees’ feedback,
and not evaluate technical capabilities of chatbots,
this would not pose a significant problem in the
experiment.
Topic where a Chatbot would Provide Noticeable
Value
To demonstrate the value of a chatbot to users, the
topic selected would need to be one where it is not
common knowledge, or of a complexity level that
would require employees to seek information or
help.
Ease of Testing and Confidentiality
As PubComms SG currently has no intranet, the
experiment would have to be designed and hosted
on external (and possibly public) platforms. As such,
the topic selected for the experiment should have
no confidentiality impact on PubComms SG.
Speed
The chatbot had to be created quickly as the
experiment neededtobe builtandconductedwithin
one month.
Experiment Design
Parts of the Lean Startup methodology (Ries, 2011)
was used to design the experiment. Advantages of
this methodology vs traditional research or product
development methods include developing products
that customers (employees) want, more quickly and
cheaply, and making it less risky (Blank, 2013).
If we were to view the proposed HR chatbot as a
startup, there are three distinct stages (Figure 5.)
and two periods of focus (Maurya, 2012):
Stage 1: Problem/Solution Fit – Is there a problem
worth solving?
Stage2:Product/Market Fit – HaveI built something
people want?
Stage 3: Scale – How do I accelerate growth?
The experiment was designed to explore and
answer Stage 1 and 2; i.e. show validated learning
that will help with presenting recommendations to
the client.
PROBLEM / SOLUTION
FIT
•
Stage 1
PRODUCT/MARKET
FIT
••
Stage 2
SCALE
•••
Stage 3
Focus: Validated Learning
Experiments: Pivots
Focus: Growth
Experiments: Optimizations
Figure 5.: Three Stages of a Startup (Maurya, 2012)
17
Build-Measure-Learn Feedback Loop
A key component of Lean Startup is the Build-
Measure-Learn Feedback Loop (see Figure 6.) The
aim of this is to go through the Feedback Loop in as
little time and as few resources as possible to
maximize learning through incremental iterative
engineering and gain insights (Blank, 2015).
It would allow the requiredminimum feedback to be
gathered to demonstrate POC, enable the
experiment to be conducted quickly, enable pivots
on the idea, and help sharpen the final
recommendations made to the client. It was also
made clear to the client that this is was an
exploratory pilot project, as evidenced by the
Research Questions listed in the project proposal to
the client (Appendix 2) where the expected
outcome of the project was “A set of
recommendations on the value and use of chatbots
in HR to improve the employee experience…”.
According to Ries (2011), planning for the Build-
Measure-Learn (BML) Feedback Loop needs to be
done in reverse, i.e. “we figure out what we need to
learn … figure outwhatweneedto measure toknow
if we are gaining validated learning, and then figure
out what product we need to build to run that
experiment and get that measurement”.
As such, the subsequent sections explaining the
experiment are elaborated in reverse of the BML
Feedback Loop to demonstrate how this
experiment was planned.
Rounds of Experiments
The original goal was to build a Minimum Viable
Product (MVP) (Ries, 2011) for testing, but as this is
a very new topic and the first time the author was
exploring it, it was difficult to estimate the
timeframe needed to build the MVP and the quality
of the MVP. This made it challenging to manage
client expectations on the output to expect.
As a solution, it was then decided to set a minimum
goal of producing a low-fidelity prototype, and a
stretch goalofproducing an ERIS chatbot MVP.This
was also in line with the Lean Startup methodology
of gaining learnings and feedback as early in the
process as possible.
The client was provided with interim updates at the
end of each experiment.
Data Collection and Interviewee Selection
Qualitative Interviews
The prototype testing was conducted via qualitative
interviews for the following reasons:
1. Target audience universe is small. Headcount
for PubComms SG is approximately 250, and
therefore any quantitative research would need
relatively large numbers which was not feasible
due to the time constraints
2. Main objective is learning. The purpose of the
experiments was not just to validate or
invalidate hypotheses, it was also to gain other
insights that may help with any pivots (Ries,
2011, p. 178) which cannot be achieved with
closed-ended, quantitative research
Face-to-face, semi-structured interviews, i.e.
interviews conducted following an interview guide
using open-ended questions, were conducted
among employees.Figure 6.: Build-Measure-Learn Feedback
Loop (Ries 2011, p. 75)
Ideas
BUILD
Product
MEASURE
Data
LEARN
Minimize total time
through the loop
REDACTED
18
Employees were first given the prototype to try, and
then asked for their reactions to the prototype and
thoughts on the potential usefulness and
applications of chatbots in the HR function of
PubComms SG. Responses from interviewees were
anonymised, coded and categorised for analysis.
Using qualitative interviews as a research method
would not only help to gain additional insights, but
it would also ensure that there was no confirmation
bias – defined as “seeking or interpreting evidence
in ways that are partial to existing beliefs,
expectations, or a hypothesis in hand” (Nickerson,
1998) – as the responses would be non-prompted.
As with allqualitative data analysis,there is a certain
levelofsubjectivity as interviewee responses are not
standardized, and there is a level of judgement and
interpretation required to code the data.
Interviewee Selection
A small sample of employees, 6 employees for each
round of experiment (12 individuals across two
rounds of experiments) was chosen.
To ensure a fair representation of employees and
reduce sampling bias (Uwex.edu, n.d.), the client
selected the employee interviewees for each round
based on a provided profile matrix (Table 1).
The client was also asked to choose employees from
a mix of job roles and brands. Eventually, 11 of the
12 selected interviewees responded within the
response timeframe and were interviewed.
Experiment Round 1 with Twine – New
Insights
Learning Objective: Value Over Growth
According to Ries (2011, p. 81), the riskiest elements
of any plan, are “leap-of-faith” assumptions,
because the success of the venture rests on them.
Two of the most important are the value hypothesis
and the growth hypothesis.
The value hypothesis tests whether a product or
service really delivers value to customers once they
are using it, and growth hypothesis tests how new
customers will discover a product or service (Ries
2011, p. 61).
For this experiment, the value hypothesis was
chosen over the growth hypothesis for testing as this
is the problem statement we were trying to answer,
andemployees wouldhaveno problems discovering
the chatbot as there are internal communication
tools to publicize the availability of this tool.
Going back to the problem statement, “How might
we explore the potential and applications of
chatbots in HR, specifically in PubComms SG, to
improve the employee experience?”, the
experiment sought to find interviewees’ feedback
on two aspects:
1. Potential of HR chatbots in PubComms SG to
improve the employee experience
2. Applications of HR chatbots in PubComms SG
to improve the employee experience
If these two statements yielded positive results,
then it would support a client recommendation to
invest in one.
Measure: Using Literature Review as a
Starting Point
Using the learnings from the Literature Review as a
starting point, the metrics of the experiment were
defined as non-prompted responses from
interviewees on the following:
1. Potential of HR chatbots in PubComms SG to
improve the employee experience
o 24-hour service
o Quick answers
o Getting an instant response
o Ability to self-serve without having to talk
to a customer service agent
o Get basic information on the policy
19
2. Applications of HR chatbots in PubComms SG to
improve the employee experience
o Ability of interviewees to envision 2 or more
uses of chatbots in the HR function
Two assumptions were made:
• Non-prompted feedback from interviewees
would reflect their priorities and desired benefits
from chatbots
• If interviewees perceived that HR chatbots
provided value, they would be able to envision
multiple possibilities for its application
Other Measurements and Data
Other questions were also developed to mine
information that would help in exploring other areas
of opportunities (see Appendix 3 for the Qualitative
Interviewing Framework that was used as a guide to
conduct the interviews).
This included seeking information from employees
on:
• Product features anddesign thatemployees like,
that would help with subsequent iterations of
the chatbot
• Other employee needs that could be solved,
that would help the HR team to improve the
overall employee experience
• Other technology tools that employees may
desire for use in PubComms SG that may help
identify other areas of opportunity for
technology investment in the company
• Their perceptions of the role of chatbots within
the HR team, to better understand how best to
deploy technology and human resources, from
an end-user perspective
There were also other questions embedded into the
interview for the benefit of the client, particularly
two questions:
1. How would you describe the quality of your
interactions with the HR team?
2. How would you rate the HR team’s use of
technology to provide HR services to employees?
These questions had no significant impact on the
experiment, but were included to help the client
better understand employees’ context and
perspective of employees’ views of the HR team.
Build: Focusing on the Conversational Flow
Topic
The selected topic for the chatbot was the
Employee Referral Incentive Scheme (ERIS), where
employees stand to receive a monetary incentive if
they successfully refer their friends for open job
positions in Singapore. ERIS was chosen because it
is:
o A defined area of inquiry that would provide a
clear scope for the chatbot
o A relatively new policy that employees still
require assistance in understanding
o Not too simple that employees find it easier to
refer to policy documents
o Complex enough for the chatbot to provide
noticeable value-add to employees, but not
exceeding the author’s technical capabilities and
resources in creating the chatbot
o Of a low level of confidentiality that would not
impact PubComms SG should the prototype
need to be hosted on public platforms, e.g.
Facebook Messenger
Software
The low fidelity prototype was a html version
simulating a chatbot conversation, built using the
Twine program (https://twinery.org/). Originally
created to write online choose-your-own-adventure
games, it is a useful tool in designing and planning
the conversational flows in chatbots because it
allows for interactive, non-linear flows, like a
conversation, without requiring the creator to have
technical coding or programming expertise (Init.ai
Decoded, 2016; WillowTree, Inc., 2016; Winstead,
2016; Vaneseltine, 2014).
Twine was chosen as it provides an overview of the
conversation flow (Figure 7), to allow the author to
evaluate the effectiveness of the conversation, e.g.
help users to get to the end point as fast as possible.
Refer to Appendix 4 to access and view the ERIS
conversational flow written on Twine (requires
download of the free program to view). Video
demonstrations of the front-end and back-end view
are also included in the same appendix and shown in
Figure 8 and 9.
20
Figure 7.: ERIS – Twine Conversational Flow
REDACTED
21
Insights surfaced in the literature review section on
good chatbot UX were incorporated. This includes
ensuring that the prototype:
• Set the right expectations
• Allow flexibility of exploration
• Create the tone to reflect the brand personality
• Help users get help
However, as Twine does not have certain
functionalities of chatbots, the following best
practices were not incorporated:
• Use buttons to supplement free text input
• Layer AI to interpret user’s requests
• Use images or graphics to supplement text
• Introduce product features gradually
• Provide a null response
• Integration with existing business systems
• Securing information security and privacy of
information
This prototype was shown to interviewees on a
desktop browser, rather than through a messaging
platform.
Results: New Insights
Potential of HR Chatbots in PubComms SG
Interview results show that the hypothesised
benefits of chatbots to improve the employee
experience were validated.
Interviewees said that they thought that the
chatbot provided good information on the policy
and was personable. They also said that a chatbot
would provide instant answers and enable
employees to find information independently.
Employees also rated the usefulness of a chatbot in
helping to improve the employee experience 7.6 out
of a possible 10, where 10 is the most useful, and 1
the least useful.
Some interesting insights emerged from the
interviews:
Chatbot is easy to use
ERIS is a relatively more complex HR policy within
PubComms SG due to differing incentive amounts
(depends on the type of job position employees are
making referrals for), and incentive payment
processes (depends on the nature of the
employment contract of the referring employee).
The conversational flow in the prototype was
designed based on conditional logic, i.e. users were
taken through a series of questions, and based on
their answers, the chatbot would provide
information that was specificto their situation (refer
to Appendix 4 for documentation on the Twine
program).
Therefore, this feedback from interviewees,
especially when it was the most mentioned
feedback, is surprising. This indicates that a chatbot
has the potential for simplifying complex
information for users, if planned well.
Chatbots help employees to avoid embarrassment
An unexpected feedback from employees was that
chatbots would help employees to avoid
embarrassment when they needed to as HR
questions that they felt were trivial.
Chatbots help to provide anonymity
This was raised specific to situations when
colleagues are trying to understand policies on
sensitive issues: “In an Asian culture, people are
more non-confrontational, and may not want to
raise alarm bells until they are sure. Making such
information readily available without having to ask
a HR team member [for access] allows them to do
their initial research”.
“In an Asian culture, people are more non-confrontational, and
may not want to raise alarm bells until they are sure. Making such
information readily available without having to ask a HR team
member [f0r access] allows them to do their initial research.”
REDACTED
22
Figure 8: ERIS – Demonstration of Twine Front End User View
VIDEO REDACTED
23
Figure 9: ERIS – Demonstration of Twine Back End View
VIDEO REDACTED
24
Chatbots help to relieve the HR team
Beyond seeing the benefits for themselves, most
saw the benefit of using chatbots to help relieve the
HR team’s workload so that they can redirect their
time to focus on more important issues.
Applications of HR Chatbots in PubComms SG
All interviewees mentioned at least two other areas
of applications, including:
• Company policies
• Company processes
• Company information
• Employee contract details
• New employee induction / orientation
• Training
• IT requests
Interestingly, the most of them related to
information services. Refer to Appendix 5 for the full
results and analysis of the interviews.
UX Feedback
Some areas of improvement that interviewees
suggested to improve the user experience:
• Reduce the wordiness of the chatbot
• Have a “home” button (there was a “back”
option in the prototype, but not a “home”
option)
Role of Chatbots vs HR team
Interviewees were also clear on the role of the
chatbot within the HR team. They would use a
chatbot for functional or transactional queries
and/or generic FAQs, and approach a HR member
when they need counsel on non-routine requests,
emotional issues, or exceptions to the regular
policies and processes. This seems to indicate that
chatbots can help transition HR away from a
transaction perception in employees towards an
expertise role.
Experiment Round 2 with Chatfuel –
Validation and More
Learn: Would Feedback be the Same?
Based on the learnings from Experiment Round 1,
the following decisions were made on the desired
learning from Round 2:
• Potential of HR Chatbots in PubComms SG
- To reconfirm the benefits even after the
prototype is changed from a html, desktop
version to a chatbot MVP hosted on a mobile
messaging platform
- Continue to listen and monitor if the new
insights would continue to be mentioned in
Round 2, thereby validating them
• Applications of HR Chatbots in PubComms SG
- Reconfirm the desired applications of
chatbots in HR
Build: Creating a chatbot MVP
The aim of the Round 2 Experiment was to be able
to test an ERIS chatbot MVP. The MVP was written
using Chatfuel (http://chatfuel.com/), a chatbot
builder.
Two other bot builders were considered – Botsify
and Wit.ai – but Chatfuel was eventually chosen as
it had the best combination of elements that met
the design considerations and required features of
the experiment (Table 2).
25
Figure 10: ERIS Chatbot Demonstration on Facebook Messenger
VIDEO REDACTED
26
Figure 11: ERIS Chatbot – Chatfuel Demonstration
VIDEO REDACTED
27
A key factor in deciding which chatbot builder to use
was the messaging platform through which users
could access and interact with the chatbot. Offering
a chatbot through an owned PubComms SG asset
was not possible as PubComms does not have any
SG-specific intranet or website.
A quick review of the chatbot builders available in
the market show that most of them design their
services to be hosted on a few key messaging
platforms (Levinson, 2016; BotStory, 2016):
Facebook Messenger, Slack, Skype, Telegram and
Kik. As Facebook Messenger was the platformlikely
to be most used among PubComms SG employees
on their mobile phones, it was one of the criteria
used to narrow the list of chatbot builders.
Given the above, an additional question was added
to the interview guide on interviewees’ thoughts on
accessing a HR chatbot via Facebook Messenger.
In adapting the Twine prototype to Chatfuel, these
additional features were added to the MVP:
• Include a ‘home’ button or its equivalent
• Provide a null response
• Allow users to type in their own queries / open-
ended questions
• More efficient user / conversational flows, i.e. less
steps to get to the required information
• Reduce wordiness and use more graphics
• Layer AI to interpret users’ requests
• Introduce product features gradually (Dropbox)
Features recommended in the literature review that
were not included:
• Integration with existing business systems
• Securing information security and privacy of
information
This was not required as the MVP was designed to
be a standalone prototype (so it could be built
quickly) and thus not needing additional
information security measures.
Appendix 6 includes source files for the two video
demonstrations of the ERIS chatbot MVP you see in
Figure 10 and 11. One shows the user view when
interacting with the ERIS chatbot via Facebook
Messenger, and the other is a demonstration of the
architecture of the ERIS chatbot on Chatfuel.
*To access the ERIS chatbot MVP on Chatfuel
(backend view), please email the author at
amy.kong@hyperisland.sg for an access link (valid
for 24 hours). To interact with the ERIS chatbot,
please use the Facebook Messenger app and search
for @PubCommsERIS (case-sensitive) or scan the
Messenger Code (Figure 12.)
Results: Validation and More
Interview results continued to validate the
hypothesised benefits of chatbots to improve the
employee experience were validated.
In fact, the usefulness rating of chatbots in
improving the employee experience rose from an
average of 7.6 in Round 1 to 7.9 in Round 2.
As this group of interviewees did not interact with
the Twine prototype, there was no opportunity to
investigate if this increase in rating was due to the
switch in interaction from a desktop web browser to
Facebook Messenger.
Figure 12: ERIS Chatbot Messenger Code
To interact with ERIS:
1. Launch the Facebook Messenger App
2. From Home, tap the icon
3. Tap your picture at the top of the page
4. Tap Scan Code
5. Scan the image on the left
REDACTED
REDACTED
28
Interviewees (except for one) were also able to
envision at least two other areas of applications of
chatbots in HR, and were still able to differentiate
when they would use a chatbot and when to
approach a HR team member.
A more nuanced view emerged on the feedback on
avoiding embarrassment. Interviewees said that
chatbots wouldhelp when employeesneededto ask:
o Trivial questions (same feedback in Round 1)
o Questions the employee is supposedto know the
answers to
o Questions the employee perceives may impact
their image/reputation among co-workers
Three additional areas of applications of chatbots in
HR raised by interviewees in Round 2 include:
• Making company announcements
• Collecting employee suggestions
• FAQ facility for hiring managers, e.g. if the
company is still within the ‘S pass’ (a type of
employment pass in Singapore) government
quota for them to be able to hire foreigners
The wordiness of the chatbot continued to be a
consistent feedback for improvement in Round 2.
Balancing between wordiness and omission of
critical information will need to be considered,
particularly in HR, where there could be legal or
employment implications if important information
is not communicated to the employee.
Interviewees agreed that a HR chatbot needs to be
hosted outside of Facebook, as it is a personal social
media platform and HR chatbots are for
professional purposes.
One employee pointed out that the chatbot should
be hosted on a platform that most employees use,
and another pointed out that most employees use
mobile phones for work, which makes a messenger-
based chatbot appropriate.
Overall, interviewees in Round 2 seemed to show
more enthusiasm over the prototype. There were
more expressions of excitement over the ERIS
chatbot MVP. Some quotes from the interviewees:
“Good that we are using technology in HR. We do it in
our business. The fact that we are taking HR online
makes it exciting. Now that I know that PubComms SG
is considering it, I really want it and am excited for it. I
can’t see how it would be a disadvantage.”
“This is good. It primes employees for more future
forward communications. That’s how it will be in the
next few years. It will help people to learn how to apply
it in their own (client) brands.”
“We need to do things faster now, and something like
this helps to get things done faster, especially when
teams are so lean now.”
“It is overwhelming how smart it is…I like the language
used. It is friendly and approachable, and it makes me
feel like I can ask any question.”
“We don’t have to wait for a response from HR or be
dependent on the HR person being around.
Currentlywe pass a lot ofmessages around… so that
HR doesn’t receive repeated questions. This allows
you to find the answer directly yourself.”
Overall Findings and Conclusion
Problem Statement Exploration and
Validation
After two rounds of experiments, results indicate
that employees think that chatbots have the
“This is good. It primes employees for more future forward
communications. That’s how it will be in the next few years. It will
help people to learn how to apply it in their own (client) brands.”
29
potential to improve the employee experience, and
can be applied in various areas in HR within
PubComms SG.
Specifically, the benefits to employees include:
• Instant access and answers
• Ability to find information independently
• Provide good information
• Personable
Interviewees found chatbots useful, and were able
to envision how chatbots could be used in the HR
function, including in:
• Information services: company information,
policies, processes, and status updates
• Training
• Information collection and dissemination:
broadcasting company announcements and
collecting employee suggestions and feedback
Interviewees even suggested that chatbots could be
used to log IT service requests.
Finally, PubComms SG HR chatbots should be
hosted on appropriate messaging platforms
suitable for work use.
Additional Insights
• Chatbots seem to be able to play a role in
simplifying complex information for employees,
when the conversational flow of the chatbot is
planned well
• Chatbots help employees to avoid feeling
embarrassed when asking certain types of
questions
• Chatbots can provide employees with a sense of
anonymity on sensitive issues
• In the longer term, chatbots could provide the
opportunity to shift perceptions of the HR team
from an administrative / transactional role to
more of an expert role
Limitations of the Experiment
Small Sample
As the experiment was designed to generate
qualitative feedback, it also meant that the sample
size is small and may not be representative of
PubComms SG, although some of the feedback was
validated in Round 2 of the experiment.
Lack of Quantitative Metrics
As data collected was qualitative, there were no
quantitative metrics that were collected. One
quantitative metric that could provide valuable
insight (and justification to management) if future
experiments are conducted, is the time saved in
using a chatbot to complete HR tasks, versus the
current approach.
Confined Area of Enquiry
The chatbot was built based on ERIS, which is a very
specific HR policy within PubComms SG. In
implementing any HR chatbot, the areas of enquiry
would need to be broader than one specific policy,
and the chatbot was not tested on its ability to filter
or direct questions down different topic paths. This
may impact the “ease of use” feedback that was
seen in this experiment.
“We need to do things faster
now, and something like this
helps to get things done
faster, especially when teams
are so lean now.”
“Good that we are using
technology in HR. We do it in
our business. The fact that we
are taking HR online makes it
exciting. Now that I know
that PubComms SG is
considering it, I really want it
and am excited for it. I can’t
see how it would be a
disadvantage.”
30
Review and Recommendations
Yay or Nay?
Based on the Literature Review and Experiment,
there is significant potentialandvalue in investing in
HR chatbots to improve the employee experience
and improve employee productivity by reducing
time spent on completing HR tasks.
The availability of chatbot builders and vendor
solutions in the market now also lower the barriers
of entry for any investment in building chatbots.
How Much Does It Cost?
It depends on the complexity of the chatbot. There
are also still no industry standards for pricing, and
various vendors and platforms have different
business models which make it hard to do a cross-
vendor or solution comparison of costs.
However, to help with understanding the cost
implications, Table 3 provides a list of budget items
to be considered.
It Could Save SGD 125,000 Annually
Based on conservative estimates (see Table 4), a
chatbot could save the company SGD 125,000
annually, excluding the cost of building the chatbot.
This is a conservative estimate as a chatbot’s instant
response capability also allows employees to reduce
multi-tasking as a result of waiting for responses
from the HR team. According to the American
Psychological Association (2006), “even brief
mental blocks created by shifting between tasks can
cost as much as 40 percent of someone's productive
time.”
If you add in task automation capabilities into the
chatbot, cost saved would also increase.
Suggested Timeline and Next Steps
It may be more appropriate to look at a timeline of 6
months to launch an “official” HR chatbot for the
following reasons:
Allow Time for Market to Mature
The market is still at a relatively early stage of
development.
This should settle down somewhat in 6 months for
PubComms SG to get a clearer perspective of the
real possibilities of chatbots.
Allow Time for More Vendor Solutions Emerge
and Stabilize
PubComms SG should also consider off-the shelf
solutions, but many of these solutions are US-based,
or still in beta versions. Waiting 6 months will allow
more vendor solutions to emerge and become more
stable.
Also, given the huge messaging usage in Asia, Asia-
specific or global solutions should emerge in the
market soon.
Other Decisions Around the Implementation of
the Chatbot
There are some areas that need to be reviewed and
decided, which will influence how the HR chatbot in
PubComms SG will be implemented. This includes:
31
Topic/s of the Chatbot
While the prototypes were built based on ERIS for
specific reasons, ERIS is still a policy that is relevant
only to a subset of employees, used infrequently. It
may be worth considering a different set of policies
to build the chatbot around, for example, top 10
FAQs, leave entitlements, medical benefits, list of
panel of doctors etc.
Choice of Messaging Platform
Due to the confidentiality of HR policies and
information, an appropriate messaging platform
needs to be selected. At the same time, the choice
of platform needs to be easily accessible by
employees via their personal mobile devices. Some
options that can be considered:
• Skype for Business (Lync), the official corporate
messaging tool for PubComms SG. This assumes
that this feature will be made available to all
employees on their mobile phones
• Create a mobile-friendly intranet to house
company information common to all brands, and
also use it to host a chatbot
Integration with Publicis Groupe Enterprise HR
Software, Data Protection and Security
A decision needs to be made on how personalised
the chatbot should be. Personalising interactions
requires integrating the chatbot with PubComm
SG’s existing enterprise HR software to access
employee data, and as such issues of data
protection and security need to be addressed.
On the upside, SAP, the enterprise HR software
vendor used by Publicis Groupe, is partnering Kore
to integrate chatbot products into its software
(Besser, 2016), which resolves the security issues.
On the downside, the speed at which chatbots can
be deployed in PubComms SG may be slowed, as
any use of the enterprise HR software will need to
be authorised by the Pubicis Groupe global team,
and vetted and handled by ReSources IT (Publicis
Groupe IT department).
Analytics
There is an opportunity for PubComms SG to collect
data to monitor and improve the performance of
the chatbot, understand employees concerns better
and eventually shape the Talent strategy of the
company. The type of data to be collected should be
considered during the design of the chatbot. This
was not a focus for the experiment as the sample
size was too small to collect any meaningful data.
Employee Privacy and Anonymity
Given that some employees have said that they feel
that chatbots can provide anonymity, there needs
to be discussion on how much,andwhat data should
be collected, and what level of anonymity will be
provided.
Accuracy and Completeness of Information
Scenarios where information provided could have a
financial impact to the employee or company, or if
an employee’s critical employment decision is
dependent on a complete understanding of a
specific HR policy, need to be discussed to ensure
that proper check mechanisms are put in place. No
doubt, these situations happens now without the
use of chatbots, but they need to be discussed and
the handling approach agreed on to ensure that
there is a proper resolution mechanism in place.
REDACTED
REDACTED
REDACTED
32
HR Talent
HR employees with a new skill set may be needed to
drive a technology project such as this. The selected
employee may not need to be a technical specialist,
but needs to be able to understand the project
parameters and work with vendors. In the longer
term, a review will be needed of the composition
andskillsetoftheHRteamasmoretechnologytools
are used in the HR function.
Other Recommendations and
Considerations
While there are many Do-It-Yourself chatbot
builders in the market, it may be best for
PubComms SG to hire a vendor to develop one if a
customised solution is desired. Integrating the
chatbot with the enterprise HR software will need
considerable technical skill. Additionally, new
chatbot features are constantly being added, and
unless an employee is dedicatedto focus on building
the chatbot, it would be difficult to keep up with the
constant upgraded capabilities of the chatbot
builders.
The Time is Now
Even as the suggested timeline is in 6 months’ time,
the preparatory work needs to start now, as some of
the considerations listed will involve careful
consideration and multiple stakeholders that need
time to resolve. MVPs can still be tested with a
curated employee group of early adopters so that
learnings are accumulated and can be baked into
the “official” chatbot when ready.
Acknowledgements
My journey at Hyper Island would not have been possible without the advice and constant moral
support from friends and family. I would like to specially thank the following who have
contributed directly to my AWBP journey:
Dan Spencer
Chief Talent Officer, Publicis Communications, Singapore and Australia / New Zealand
Leong Yeng Wai
Student, Hyper Island Singapore
Benjamin Koe
Product Lead, Dentsu Aegis Global Data Innovation Centre
Narendra Nag
Chief Digital Officer, Xynteo
Lim Chwen Yiing and Lim Han Boon
Best Buddies
Jonathan Briggs
Co-Founder and Singapore Academic Director, Hyper Island
Jarrod Howe
Operations Director, Hyper Island Singapore
33
Reflections
This journey leading up to the completion of this Advanced Work Based
Project (AWBP) has been quite an interesting and enlightening one. These
are some of the learnings and experiences that have shaped this AWBP:
Complexity of Chatbots and The Landscape
Much of the literature talks about how easy it is to create chatbots. It is
true, but it is also false. It is true that many of the tools now in the market,
particularly the chatbot builders, are relatively easy to use. However,
before getting to the pointofbuilding a chatbot,company decision makers
need to first understand if they are making the right choice in investments,
which means understanding the landscape, and that is where it gets
complex.
When I first started this project, I had no idea where to start, as I have no
coding or IT specialist experience and my experience with technology is
purely from an end-user perspective. As I started the research, it seemed
like falling down a never-ending rabbit hole. Just to understand chatbot
literature that is regularly peppered with jargon, you must know how it fits
vis-à-vis AI, machine learning, NLP, NLU, Natural Language Generation.
What’s the difference between a chatbot and a conversational agent?
What is the difference between a conversational agent and a dialog
system?
That’s just understanding chatbots andits possible functions.To create the
chatbot, you need to understand the different approaches to building one,
then understand the comparative differences across the plethora of tools
to choose one to use. And the list goes on. I first worked on wit.ai to start
building the chatbot as recommended by chatbot-related articles as being
easy to use. But it turned out to be an easy tool for developers to use, and
not for non-coders like me. So, I had to abandon the work I started in wit.ai
and re-start the search for a chatbot builder.
For a HR teamthat may not have a large IT departmentto rely on for advice,
or even a for CEO who is interested in investing in one, the landscape can
be hard to decipher.
Jargon Everywhere
The proliferate use of jargon is only part of the problem. The other part of
the problem is that at this stage of market development, terminology is not
clearly defined or differentiated, or writers have a poor understanding of
chatbot technology. Sometimes the same terms mean different things, or
different terms mean the same thing. Some articles refer to AI as different
from machine learning. Some articles talk about chatbot services while
others talk about chatbot platforms when they mean the same thing.
As such, I tried to be as clear as possible in this report on the different terms
and definitions, and explain the chatbot landscape as simply as I could,
based on my understanding. As this report is meant to be shared with
PubComms SG, hopefully this will help provide clarity for the client.
Easy for Whom?
It is easy to use a chatbot builder, but it may not be simple to build a
chatbot, or at least build a good one. As I was using wit.ai, I was not able to
understand how the conversation with the user would flow (that would help
me evaluate its effectiveness), as the tool does not help you to do so. This
is the same with other chatbot builders that I explored. Adding on to that
stress was the lack of time to conduct the experiment,which was how I then
researched and found Twine. Even as I was using Twine, I realised that you
need an understanding of conditional logic. Upon consulting my ex-
colleague who has developer experience, I also realised that you need to
plan the conversational flow not in sequential steps, but in blocks of
information, and then create paths or flows to the informational blocks.
REDACTED
34
Reflections (cont’d)
In short, anybody can build a chatbot, if that anybody understands how to
plan conversational flows, understands conditional logic etc. Along the
way, you’ll encounter other terms like ‘rails’, ‘conversational trees’, ‘dialog
trees’ and so on, and down the rabbit hole you go again. Nevertheless, it
is true that the barrier to entry for building chatbots are now far lower
than before.
Drinking From a Firehose
Chatbots are such a hot topic now, that news articles and blog posts are
generated almost every day. I found it difficult to keep track of the
emerging articles and evaluate them for inclusion in this paper. It was also
a challenge to select a balance of commercial vs academic sources.
Firstly, academic journals do not necessarily reflect the current state of
business discussion around the topic, and secondly, they tend to be very
technical. There was significant temptation to use data primarily from
commercial articles as they provide quick and short summaries. I have
tried to refer to academic sources in this paper for key definitions, terms,
concepts and key milestones and discussions, and kept the references to
commercial articles confined to areas where the academic journals may
not touch upon.
Iteration vs Documentation
When using Chatfuel, changes to any part of the chatbot is reflected ‘live’.
There is also no option for version downloads or control. As such, during
the experimentation phase, I found it difficult to keep track of the
changes that I made to the chatbot. In the longer term, it also raises
questions on how to restore chatbots should there be any changes that
need to be reversed.
When is Customer Feedback Valid?
Part of the iteration process is seeking qualitative feedback from
customers. One of the questions that I kept asking myself was, “at what
point is customer feedback considered valid or an outlier?”. Ultimately, I
made the decision based on what I felt made sense and was in line with
the overall objective of the chatbot, but it seemed very subjective to me. If
given the opportunity, I would ask current Lean Startup practitioners how
they make such decisions, particularly with qualitative testing with small
sample sizes
Challenge in Integrating with Enterprise HR Software
I anticipate that there will be challenges in getting this chatbot project
running in PubComms SG if backend integration with the enterprise HR
software is needed. Such requests will need to be approved by the Global
Publicis Groupe team, and actioned by ReSources IT, which is the internal
IT shared services function. This affects the speed at which backend
functions in PubComms SG can capitalise on emerging opportunities.
PubComms SG may be better off creating a generic chatbot now, then
present the case study to the Global team for endorsement, to increase
the speed of implementation.
REDACTED
35
References
Ady, M. (2016). How to Build a Chatbot Your Users Will Love. [online] VentureBeat. Available at: http://venturebeat.com/2016/08/17/how-to-build-a-chatbot-your-users-will-love/
[Accessed 3 Jan. 2017].
American Psychological Association. (2006). Multitasking: Switching costs. [online] Available at: http://www.apa.org/research/action/multitask.aspx [Accessed 5 Jan. 2017].
Aspect, (2016). 2016 Aspect Consumer Experience Index. [online] Phoenix. Available at: https://www.aspect.com/globalassets/2016-aspect-consumer-experience-index-
survey_index-results-final.pdf [Accessed 1 Jan. 2017].
Aspect.com. (n.d.). Aspect Mila: The Agent’s Personal Assistant. [online] Available at: https://www.aspect.com/solutions/workforce-optimization/aspect-mila [Accessed 2 Jan.
2017].
Ball, P. (2015). The truth about the Turing Test. [online] Bbc.com. Available at: http://www.bbc.com/future/story/20150724-the-problem-with-the-turing-test [Accessed 31 Dec.
2016].
Batalion, A. (2016). “Bot” is the Wrong Name.. and Why People Who Think They are Silly are Wrong. [online] Medium. Available at: https://medium.com/lightspeed-venture-
partners/bot-is-the-wrong-name-and-why-people-who-think-they-are-silly-are-wrong-dc0c0b76ae18#.y4x0znitc [Accessed 1 Jan. 2017].
Bayerque, N. (2016). A Short History of Chatbots and Artificial Intelligence. [online] VentureBeat. Available at: http://venturebeat.com/2016/08/15/a-short-history-of-chatbots-and-
artificial-intelligence/ [Accessed 1 Jan. 2017].
Besser, L. (2016). Kore bots for SAP empower every employee to delegate like a boss. [online] Ideas.sap.com. Available at: https://ideas.sap.com/D32372 [Accessed 4 Jan. 2017].
Bhaduri, A. (2016). The Digital Tsunami: HR. [online] Abhijit Bhaduri's Official Website. Available at: http://www.abhijitbhaduri.com/index.php/2016/08/the-digital-tsunami-hr/
[Accessed 2 Jan. 2017].
Blank, S. (2013). Why the Lean Start-Up Changes Everything. [online] Harvard Business Review. Available at: https://hbr.org/2013/05/why-the-lean-start-up-changes-everything
[Accessed 22 Dec. 2016].
Blank, S. (2015). Why Build, Measure, Learn – isn’t just throwing things against the wall to see if they work – the Minimal Viable Product. [online] Steve Blank. Available at:
https://steveblank.com/2015/05/06/build-measure-learn-throw-things-against-the-wall-and-see-if-they-work/ [Accessed 25 Dec. 2016].
BotStory. (2016). Top messaging platforms supporting bots. [online] Available at: http://botstory.co/top-platforms-support-bots-chatbots/ [Accessed 28 Dec. 2016].
Boulton, C. (2016). From tacos to HR, chatbots make it personal. [online] CIO. Available at: http://www.cio.com/article/3063051/instant-messaging/from-tacos-to-hr-chatbots-
make-it-personal.html [Accessed 2 Jan. 2017].
Business Insider. (2016). Messaging Apps are Now Bigger than Social Networks. [online] Available at: http://www.businessinsider.com/the-messaging-app-report-2015-
11?IR=T&r=US&IR=T [Accessed 1 Jan. 2017].
36
Chowdhury, G. (2005). Natural language processing. Annual Review of Information Science and Technology, 37(1), pp.51-89.
Christensen, C., Hall, T., Dillon, K. and Duncan, D. (2016). Know Your Customers’ “Jobs to Be Done”. [online] Harvard Business Review. Available at: https://hbr.org/2016/09/know-
your-customers-jobs-to-be-done [Accessed 26 Dec. 2016].
Dan, A. (2016). Consultants Are Eating The Agencies' Three-Martini Lunch. [online] Forbes.com. Available at: http://www.forbes.com/sites/avidan/2016/04/25/consultants-are-
eating-the-agencies-three-martini-lunch/#5ba472b63aba [Accessed 30 Dec. 2016].
Deutscher, M. (2016). FirstJob’s Mya is the latest chatbot that aims to automate recruiting. [online] SiliconANGLE. Available at: http://siliconangle.com/blog/2016/07/11/firstjobs-
mya-is-the-latest-chatbot-that-aims-to-automate-recruiting/ [Accessed 2 Jan. 2017].
Dishman, L. (2016). This Chatbot Can Make Sure Your Resume Won't End Up In A Black Hole. [online] Fast Company. Available at: https://www.fastcompany.com/3061677/the-
future-of-work/the-chatbot-who-can-make-sure-youll-never-get-radio-silence-after-applyin [Accessed 2 Jan. 2017].
Dunn, J. (2016). We put Siri, Alexa, Google Assistant, and Cortana through a marathon of tests to see who’s winning the virtual assistant race — here’s what we found - Business
Insider. [online] Business Insider. Available at: http://www.businessinsider.sg/siri-vs-google-assistant-cortana-alexa-2016-11/?r=US&IR=T#rfBHqO3D8eSOZMdp.97
[Accessed 2 Jan. 2017].
Elliott, S. (2002). Advertising's Big Four: It's Their World Now. [online] Nytimes.com. Available at: http://www.nytimes.com/2002/03/31/business/advertising-s-big-four-it-s-their-
world-now.html?pagewanted=all&src=pm [Accessed 20 Dec. 2016].
Ember, S. (2016). Ad Agencies Need Young Talent. Cue the Beanbag Chairs.. [online] Nytimes.com. Available at: http://www.nytimes.com/2016/04/19/business/media/ad-agencies-
need-young-talent-cue-the-bean-bag-chairs.html?_r=0 [Accessed 31 Dec. 2016].
Epstein, R., Roberts, G. and Beber, G. (2008). Parsing the turing test. 1st ed. New York: Springer, pp.181-210.
Farmer, M. (2015). Madison Avenue Manslaughter. 1st ed. New York: LID Publishing Ltd.
Fromowitz, M. (2016). Commoditization: The biggest threat facing ad agencies today. [online] Campaign Asia. Available at: http://www.campaignasia.com/article/commoditization-
the-biggest-threat-facing-ad-agencies-today/425997 [Accessed 30 Dec. 2016].
Futurism. (n.d.). The History of Chatbots [INFOGRAPHIC]. [online] Available at: http://futurism.com/images/the-history-of-chatbots-infographic/ [Accessed 1 Jan. 2017].
Gartner.com. (2015). Gartner Says Weak Mobile Customer Service Is Harming Customer Engagement. [online] Available at: http://www.gartner.com/newsroom/id/2956618
[Accessed 1 Jan. 2017].
Grayson, A. and North, G. (2016). How the Advertising Industry is Wasting Talent and What We Can Do About It. [online] Medium. Available at:
https://medium.com/@andygrayson/how-the-advertising-industry-is-wasting-talent-and-what-we-can-do-about-it-4824411b9006#.8h9lmf6e3 [Accessed 31 Dec. 2016].
Greenfield, R. (2016). Chatbots Are Your Newest, Dumbest Co-Workers. [online] Bloomberg.com. Available at: https://www.bloomberg.com/news/articles/2016-05-05/chatbots-are-
your-newest-dumbest-co-workers [Accessed 2 Jan. 2017].
37
Griffin, A. (2014). Computer becomes first to pass Turing Test in artificial intelligence. [online] The Independent. Available at: http://www.independent.co.uk/life-style/gadgets-and-
tech/computer-becomes-first-to-pass-turing-test-in-artificial-intelligence-milestone-but-academics-warn-9508370.html [Accessed 31 Dec. 2016].
Hobson, N. (2016). Are chatbots in the workplace the entry point to cognitive personal assistants? | NevilleHobson.com. [online] NevilleHobson.com. Available at:
http://www.nevillehobson.com/2016/05/10/chatbots-workplace-cognitive/ [Accessed 1 Jan. 2017].
Howard, T. (2016). The Guide To Designing A Magical Chatbot Experience. [online] Chatbots Magazine. Available at: https://chatbotsmagazine.com/the-guide-to-designing-a-
magical-chatbot-experience-part-1-efbf32444448#.wyex0oqer [Accessed 3 Jan. 2017].
Init.ai Decoded. (2016). The Product Designer’s Guide to Conversational Commerce. [online] Available at: https://blog.init.ai/the-product-designers-guide-to-conversational-
commerce-cbe466753add#.gn5sjy8bk [Accessed 27 Dec. 2016].
Jain, A. (2016). The Flow Framework — How to build a kickass UX for your Chat-bot? (Part — II). [online] ChatterOn. Available at: https://blog.chatteron.io/how-to-build-a-kickass-ux-
for-your-chat-bot-part-ii-the-flow-framework-811355905249#.6dlq6d3qw [Accessed 3 Jan. 2017].
Johnson, B. (2015). State of the Agency Market: What You Need to Know. [online] Adage.com. Available at: http://adage.com/article/digital/datacenter-agency-report-2015-
charts/298214/ [Accessed 30 Dec. 2016].
Knowledge@Wharton. (2016). The Rise of the Chatbots: Is It Time to Embrace Them? - Knowledge@Wharton. [online] Available at:
http://knowledge.wharton.upenn.edu/article/rise-chatbots-time-embrace/ [Accessed 31 Dec. 2016].
Kojouharov, S. (2016). Ultimate Guide to Leveraging NLP & Machine Learning for your Chatbot. [online] Chatbot’s Life. Available at: https://chatbotslife.com/ultimate-guide-to-
leveraging-nlp-machine-learning-for-you-chatbot-531ff2dd870c#.zaqk0o35t [Accessed 2 Jan. 2017].
Lamprecht, E. (2016). The Difference Between UX and UI Design-A Layman’s Guide. [online] Blog.careerfoundry.com. Available at: http://blog.careerfoundry.com/ui-design/the-
difference-between-ux-and-ui-design-a-laymans-guide/ [Accessed 4 Jan. 2017].
Levinson, R. (2016). The tools every bot creator must know. [online] Chatbots Magazine. Available at: https://chatbotsmagazine.com/the-tools-every-bot-creator-must-know-
c0e9dd685094#.v6h0lbonf [Accessed 28 Dec. 2016].
Lo, M. (2014). UI, UX: Who Does What? A Designer's Guide To The Tech Industry. [online] Co.Design. Available at: https://www.fastcodesign.com/3032719/ui-ux-who-does-what-a-
designers-guide-to-the-tech-industry [Accessed 4 Jan. 2017].
Loebner, H. (n.d.a). In Response. [online] Loebner.net. Available at: http://loebner.net/Prizef/In-response.html [Accessed 31 Dec. 2016].
Loebner.net. (n.d.b). Home Page of the Loebner Prize in Artificial Intelligence. [online] Available at: http://www.loebner.net/Prizef/loebner-prize.html [Accessed 31 Dec. 2016].
Maass, D. (2014). Answers and Questions About Military, Law Enforcement, and Intelligence Agency Chatbots. [online] Electronic Frontier Foundation. Available at:
https://www.eff.org/deeplinks/2014/04/answers-questions-about-military-law-enforcement-and-intelligence-agency-chatbots [Accessed 2 Jan. 2017].
Marcus, D. (2016). Messenger Platform at F8. [online] Newsroom.fb.com. Available at: http://newsroom.fb.com/news/2016/04/messenger-platform-at-f8/ [Accessed 1 Jan. 2017].
38
Marr, B. (2016). What Is The Difference Between Artificial Intelligence And Machine Learning?. [online] Forbes.com. Available at:
http://www.forbes.com/sites/bernardmarr/2016/12/06/what-is-the-difference-between-artificial-intelligence-and-machine-learning/3/#657340f11a30 [Accessed 1 Jan. 2017].
Marshall, J. and Alpert, L. (2016). Publishers Take On Ad-Agency Roles With Branded Content. [online] WSJ. Available at: http://www.wsj.com/articles/publishers-take-on-ad-
agency-roles-with-branded-content-1481457605 [Accessed 30 Dec. 2016].
Maurya, A. (2012). Running Lean: Iterate from a Plan A to a Plan that Works. 2nd ed. Sebastopol, CA: O'Reilly.
McNeale, M. and Newyear, D. (2013). Introducing Chatbots in Libraries. Library Technology Reports, 49(8), pp.5 - 10.
Medium. (2016). 聊天機器人市場版圖 — Chatbots Landscape. [online] Available at:
https://medium.com/@compose.ai/%E8%81%8A%E5%A4%A9%E6%A9%9F%E5%99%A8%E4%BA%BA%E5%B8%82%E5%A0%B4%E7%89%88%E5%9C%96-chatbots-
landscape-f89302206875#.wk2de23dy [Accessed 2 Jan. 2017].
Meeker, M. (2016). 2016 Internet Trends Report. [online] Kpcb.com. Available at: http://www.kpcb.com/internet-trends [Accessed 1 Jan. 2017].
Mindshare, Goldsmiths University of London, (2016). Humanity in the Machine. [online] Available at:
http://www.mindshareworld.com/sites/default/files/MINDSHARE_HUDDLE_HUMANITY_MACHINE_2016_0.pdf [Accessed 1 Jan. 2017].
Morrison, M. (2016). Sprint Names Exec Creative Director to Run New In-House Agency. [online] Adage.com. Available at: http://adage.com/article/agency-news/sprint-staffs-house-
agency/304045/ [Accessed 30 Dec. 2016].
Mumbrella Asia. (2016). Lou Dela Pena handed broader role leading Publicis Communications in Singapore - Mumbrella Asia. [online] Available at:
http://www.mumbrella.asia/2016/01/lou-dela-pena-handed-broader-role-leading-publicis-communications-in-singapore/ [Accessed 20 Dec. 2016].
myclever Agency, (2016). Chatbots: A Consumer Research Study. [online] London. Available at:
http://www.slideshare.net/mycleveragency?utm_campaign=profiletracking&utm_medium=sssite&utm_source=ssslideview [Accessed 1 Jan. 2017].
Newman, J. (2016). How The New, Improved Chatbots Rewrite 50 Years Of Bot History. [online] Fast Company. Available at: https://www.fastcompany.com/3059439/why-the-new-
chatbot-invasion-is-so-different-from-its-predecessors [Accessed 31 Dec. 2016].
Newton, C. (2016). Can AI fix education? We asked Bill Gates. [online] The Verge. Available at: http://www.theverge.com/2016/4/25/11492102/bill-gates-interview-education-
software-artificial-intelligence [Accessed 2 Jan. 2017].
Nextit.com. (n.d.). SGT STAR helps potential recruits learn about Army life. [online] Available at: http://www.nextit.com/work/army.php [Accessed 2 Jan. 2017].
Nickerson, R. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), pp.175-220.
Oetting, J. (2015). The Biggest Threats to the Agency Model. [online] Blog.hubspot.com. Available at: https://blog.hubspot.com/agency/threats-agency-
model#sm.000q8a2qc1192eo6ypt159w7mslve [Accessed 30 Dec. 2016].
39
Olson, P. (2016). Duolingo's New Chat Bots Can Teach You French, German And Spanish. [online] Forbes.com. Available at:
http://www.forbes.com/sites/parmyolson/2016/10/07/duolingo-chat-bots-artificial-intelligence/#2d80c93766e5 [Accessed 2 Jan. 2017].
Ovchinnikova, E. (2012). Integration of world knowledge for natural language understanding. 1st ed. Amsterdam: Atlantis Press, p.15.
Pandey, R. (2016). Global ad spend: Trends for 2016 and 2017. [online] Marketing Interactive. Available at: http://www.marketing-interactive.com/global-ad-spend-trends-2016-
2017/ [Accessed 30 Dec. 2016].
Pearce, K. (2013). Ask Ivy. [online] Jobs@Intel Blog. Available at: http://blogs.intel.com/jobs/2013/05/16/ask-ivy/ [Accessed 2 Jan. 2017].
Perez-Marin, D. and Pascual-Nieto, I. (2011). Conversational Agents and Natural Language Interaction. 1st ed. Hershey, PA: Information Science Reference, pp.1-22, 107-127, 358-
378.
Publicis Groupe, (2016a). Publicis Groupe Announces Important Nominations and its Transformation Plan. [online] Available at:
http://livredor.publicis.com/notesInternes/2015/021215_EN_PublicisGroupe_Transformation_DEF.pdf [Accessed 20 Dec. 2016].
Publicis Groupe, (2016b). Third Quarter 2016 Revenue. [online] Paris: Publicis Groupe. Available at: http://www.publicisgroupe.com/media/display/id/8072.pdf [Accessed 31 Dec.
2016].
Publicis Groupe, (2016c). 2015 Registration Document - Annual Financial Report. [online] Paris: Publicis Groupe, pp.90 - 95. Available at:
http://www.publicisgroupe.com/documents/document-de-reference-2015-en.pdf [Accessed 31 Dec. 2016].
Richman, D. (2016). Microsoft Azure offers ‘bots as a service’ as demand rises for automated online interactions. [online] GeekWire. Available at:
http://www.geekwire.com/2016/microsoft-azure-offers-bots-service-demand-rises-automated-online-interactions/ [Accessed 5 Jan. 2017].
Riel, J. (2016). How Chatbots Can Help With Learning. [online] College of Education | University of Illinois Chicago. Available at: https://education.uic.edu/academics-
admissions/student-life/how-chatbots-can-help-learning [Accessed 2 Jan. 2017].
Ries, E. (2011). The Lean Startup. 1st ed. London: Penguin Group, pp.56-78.
Rosenberg, S. (2016). New Messenger Platform Features: Link Ads to Messenger, Enhanced Mobile Websites, Payments and More. [online] Facebook for Developers. Available at:
https://developers.facebook.com/blog/post/2016/09/12/new-messenger-features-payments-ads-enhanced-mobile-websites/ [Accessed 1 Jan. 2017].
Roxburgh, H. (2016). Why big consultancies buy design agencies. [online] Campaign Asia. Available at: http://www.campaignasia.com/article/why-big-consultancies-buy-design-
agencies/407973 [Accessed 30 Dec. 2016].
Sas.com. (n.d.). Machine Learning: What it is and why it matters. [online] Available at: http://www.sas.com/en_us/insights/analytics/machine-learning.html [Accessed 1 Jan. 2017].
Schaefer, M. (2015). 6 Reasons Marketing Is Moving In-House. [online] Harvard Business Review. Available at: https://hbr.org/2015/07/6-reasons-marketing-is-moving-in-house
[Accessed 30 Dec. 2016].
Schlicht, M. (2016). The Complete Beginner’s Guide To Chatbots. [online] Chatbots Magazine. Available at: https://chatbotsmagazine.com/the-complete-beginner-s-guide-to-
chatbots-8280b7b906ca#.dxwvm1po5 [Accessed 31 Dec. 2016].
Chatbots in HR: Improving the Employee Experience
Chatbots in HR: Improving the Employee Experience
Chatbots in HR: Improving the Employee Experience
Chatbots in HR: Improving the Employee Experience

More Related Content

What's hot

Digital transformation
Digital transformationDigital transformation
Digital transformation
Anushya D
 
Business Analyst Training
Business  Analyst  TrainingBusiness  Analyst  Training
Business Analyst Training
Craig Brown
 
Bridging enterprise-architecture and systems-thinking
Bridging enterprise-architecture and systems-thinkingBridging enterprise-architecture and systems-thinking
Bridging enterprise-architecture and systems-thinking
Tetradian Consulting
 
Target Operating Model Definition
Target Operating Model DefinitionTarget Operating Model Definition
Target Operating Model Definition
stuart1403
 
IT Consulting Services
IT Consulting ServicesIT Consulting Services
IT Consulting Services
IMCS Group
 
Introduction to BOT:Build, Operate and Transfer Offshoring
Introduction to BOT:Build, Operate and Transfer OffshoringIntroduction to BOT:Build, Operate and Transfer Offshoring
Introduction to BOT:Build, Operate and Transfer Offshoring
ewinterbottom
 
Call Center Best Pratices Overview
Call Center Best Pratices OverviewCall Center Best Pratices Overview
Call Center Best Pratices Overview
Brian Hughes
 
Service Delivery Management (Lucia Eversley)
Service Delivery Management (Lucia Eversley)Service Delivery Management (Lucia Eversley)
Service Delivery Management (Lucia Eversley)
BDPA Education and Technology Foundation
 
Using ITIL 4 and IT4IT together
Using ITIL 4 and IT4IT togetherUsing ITIL 4 and IT4IT together
Using ITIL 4 and IT4IT together
Rob Akershoek
 
Cooper Parlor: Service Blueprinting
Cooper Parlor: Service Blueprinting Cooper Parlor: Service Blueprinting
Cooper Parlor: Service Blueprinting
Izac Ross
 
Digital Operating Model & IT4IT
Digital Operating Model & IT4ITDigital Operating Model & IT4IT
Digital Operating Model & IT4IT
David Favelle
 
The Transformation Office - A new organisational capability for the digital e...
The Transformation Office - A new organisational capability for the digital e...The Transformation Office - A new organisational capability for the digital e...
The Transformation Office - A new organisational capability for the digital e...
Rafael Lemaitre
 
50 of the best chatbot use cases [Case studies Included]
50 of the best chatbot use cases [Case studies Included]50 of the best chatbot use cases [Case studies Included]
50 of the best chatbot use cases [Case studies Included]
Sam Prasanth
 
It Consulting Slides
It Consulting SlidesIt Consulting Slides
It Consulting Slides
Joe O'Mahoney
 
Business Case Summary
Business Case SummaryBusiness Case Summary
Business Case Summary
JArangote_Consulting
 
Virtual internships consulting
Virtual internships   consultingVirtual internships   consulting
Virtual internships consulting
Nithin Kumar Peratla
 
Strategic Operating Model
Strategic Operating Model Strategic Operating Model
Telco 4.0 Business Operating Model Value Proposition Overview
Telco 4.0 Business Operating Model Value Proposition   OverviewTelco 4.0 Business Operating Model Value Proposition   Overview
Telco 4.0 Business Operating Model Value Proposition Overview
Nigel Tebbutt
 
Business by Design
Business by DesignBusiness by Design
Business by Design
Craig Martin
 
Capability Webinar January 2022
Capability Webinar January 2022Capability Webinar January 2022
Capability Webinar January 2022
Intersection Group
 

What's hot (20)

Digital transformation
Digital transformationDigital transformation
Digital transformation
 
Business Analyst Training
Business  Analyst  TrainingBusiness  Analyst  Training
Business Analyst Training
 
Bridging enterprise-architecture and systems-thinking
Bridging enterprise-architecture and systems-thinkingBridging enterprise-architecture and systems-thinking
Bridging enterprise-architecture and systems-thinking
 
Target Operating Model Definition
Target Operating Model DefinitionTarget Operating Model Definition
Target Operating Model Definition
 
IT Consulting Services
IT Consulting ServicesIT Consulting Services
IT Consulting Services
 
Introduction to BOT:Build, Operate and Transfer Offshoring
Introduction to BOT:Build, Operate and Transfer OffshoringIntroduction to BOT:Build, Operate and Transfer Offshoring
Introduction to BOT:Build, Operate and Transfer Offshoring
 
Call Center Best Pratices Overview
Call Center Best Pratices OverviewCall Center Best Pratices Overview
Call Center Best Pratices Overview
 
Service Delivery Management (Lucia Eversley)
Service Delivery Management (Lucia Eversley)Service Delivery Management (Lucia Eversley)
Service Delivery Management (Lucia Eversley)
 
Using ITIL 4 and IT4IT together
Using ITIL 4 and IT4IT togetherUsing ITIL 4 and IT4IT together
Using ITIL 4 and IT4IT together
 
Cooper Parlor: Service Blueprinting
Cooper Parlor: Service Blueprinting Cooper Parlor: Service Blueprinting
Cooper Parlor: Service Blueprinting
 
Digital Operating Model & IT4IT
Digital Operating Model & IT4ITDigital Operating Model & IT4IT
Digital Operating Model & IT4IT
 
The Transformation Office - A new organisational capability for the digital e...
The Transformation Office - A new organisational capability for the digital e...The Transformation Office - A new organisational capability for the digital e...
The Transformation Office - A new organisational capability for the digital e...
 
50 of the best chatbot use cases [Case studies Included]
50 of the best chatbot use cases [Case studies Included]50 of the best chatbot use cases [Case studies Included]
50 of the best chatbot use cases [Case studies Included]
 
It Consulting Slides
It Consulting SlidesIt Consulting Slides
It Consulting Slides
 
Business Case Summary
Business Case SummaryBusiness Case Summary
Business Case Summary
 
Virtual internships consulting
Virtual internships   consultingVirtual internships   consulting
Virtual internships consulting
 
Strategic Operating Model
Strategic Operating Model Strategic Operating Model
Strategic Operating Model
 
Telco 4.0 Business Operating Model Value Proposition Overview
Telco 4.0 Business Operating Model Value Proposition   OverviewTelco 4.0 Business Operating Model Value Proposition   Overview
Telco 4.0 Business Operating Model Value Proposition Overview
 
Business by Design
Business by DesignBusiness by Design
Business by Design
 
Capability Webinar January 2022
Capability Webinar January 2022Capability Webinar January 2022
Capability Webinar January 2022
 

Similar to Chatbots in HR: Improving the Employee Experience

What next for Recruitment in a digitally exploding world
What next for Recruitment in a digitally exploding worldWhat next for Recruitment in a digitally exploding world
What next for Recruitment in a digitally exploding world
Gary Fay
 
HRO handbook V 2.0
HRO handbook V 2.0HRO handbook V 2.0
HRO handbook V 2.0
PeopleStrong
 
How is HR addressing dual challenge of being digital and doing digital?
How is HR addressing dual challenge of being digital and doing digital?How is HR addressing dual challenge of being digital and doing digital?
How is HR addressing dual challenge of being digital and doing digital?
Abhinav Singhal
 
Optimizing Digital Business Outcomes
Optimizing Digital Business OutcomesOptimizing Digital Business Outcomes
Optimizing Digital Business Outcomes
Bob Kantor
 
Space Matters: Shaping the Workplace to Get the Right Work Done
Space Matters: Shaping the Workplace to Get the Right Work DoneSpace Matters: Shaping the Workplace to Get the Right Work Done
Space Matters: Shaping the Workplace to Get the Right Work Done
Cognizant
 
Product Management in London - A Recruiters view...
Product Management in London - A Recruiters view...Product Management in London - A Recruiters view...
Product Management in London - A Recruiters view...
Will Boulton
 
Intelligent Automation In Financial Services
Intelligent Automation In Financial ServicesIntelligent Automation In Financial Services
Intelligent Automation In Financial Services
Accenture Insurance
 
Cognizant Jobs of the Future Index
Cognizant Jobs of the Future IndexCognizant Jobs of the Future Index
Cognizant Jobs of the Future Index
Cognizant
 
Insights success The 10 Most Valuable Outsourcing Companies 2016
 Insights success The 10 Most Valuable Outsourcing Companies 2016 Insights success The 10 Most Valuable Outsourcing Companies 2016
Insights success The 10 Most Valuable Outsourcing Companies 2016
Insights success media and technology pvt ltd
 
Insights success the 10 most valuable outsourcing companies august 2016 (1)
Insights success the 10 most valuable outsourcing companies august 2016 (1)Insights success the 10 most valuable outsourcing companies august 2016 (1)
Insights success the 10 most valuable outsourcing companies august 2016 (1)
Merry D'souza
 
the-way-we-work-hr-today_pwc-en_2017.pdf
the-way-we-work-hr-today_pwc-en_2017.pdfthe-way-we-work-hr-today_pwc-en_2017.pdf
the-way-we-work-hr-today_pwc-en_2017.pdf
ssuserdf3f8a
 
#Human Resource Management Challenges in the Digital Age# by Mr. Samantha Rat...
#Human Resource Management Challenges in the Digital Age# by Mr. Samantha Rat...#Human Resource Management Challenges in the Digital Age# by Mr. Samantha Rat...
#Human Resource Management Challenges in the Digital Age# by Mr. Samantha Rat...
CPM Sri Lanka
 
Outsourcing 2019
Outsourcing 2019Outsourcing 2019
Digital Strategy Research Essay: Effects of difficult capabilities
Digital Strategy Research Essay: Effects of difficult capabilitiesDigital Strategy Research Essay: Effects of difficult capabilities
Digital Strategy Research Essay: Effects of difficult capabilities
Total Assignment Help
 
Case Study : Aligning people capabilities to a company's digital strategy
Case Study : Aligning people capabilities to a company's digital strategyCase Study : Aligning people capabilities to a company's digital strategy
Case Study : Aligning people capabilities to a company's digital strategy
Youlab
 
Futureproofing In House PR Teams - 3 Monkeys & PR Moment
Futureproofing In House PR Teams - 3 Monkeys & PR MomentFutureproofing In House PR Teams - 3 Monkeys & PR Moment
Futureproofing In House PR Teams - 3 Monkeys & PR Moment
3 Monkeys Communications
 
The New Drivers of Recruiting and How Technology Should Enable Them
The New Drivers of Recruiting and How Technology Should Enable ThemThe New Drivers of Recruiting and How Technology Should Enable Them
The New Drivers of Recruiting and How Technology Should Enable Them
Human Capital Media
 
Mba project1
Mba project1Mba project1
Mba project1
SHRUTI SAGAR
 
BCG NASSCOM Future of Work Report_07JUNE2022_vFLR.pdf
BCG NASSCOM Future of Work Report_07JUNE2022_vFLR.pdfBCG NASSCOM Future of Work Report_07JUNE2022_vFLR.pdf
BCG NASSCOM Future of Work Report_07JUNE2022_vFLR.pdf
AdarshSL5
 
Enterprise Fusion: Your Pathway To A Better Customer Experience
Enterprise Fusion: Your Pathway To A Better Customer ExperienceEnterprise Fusion: Your Pathway To A Better Customer Experience
Enterprise Fusion: Your Pathway To A Better Customer Experience
Cognizant
 

Similar to Chatbots in HR: Improving the Employee Experience (20)

What next for Recruitment in a digitally exploding world
What next for Recruitment in a digitally exploding worldWhat next for Recruitment in a digitally exploding world
What next for Recruitment in a digitally exploding world
 
HRO handbook V 2.0
HRO handbook V 2.0HRO handbook V 2.0
HRO handbook V 2.0
 
How is HR addressing dual challenge of being digital and doing digital?
How is HR addressing dual challenge of being digital and doing digital?How is HR addressing dual challenge of being digital and doing digital?
How is HR addressing dual challenge of being digital and doing digital?
 
Optimizing Digital Business Outcomes
Optimizing Digital Business OutcomesOptimizing Digital Business Outcomes
Optimizing Digital Business Outcomes
 
Space Matters: Shaping the Workplace to Get the Right Work Done
Space Matters: Shaping the Workplace to Get the Right Work DoneSpace Matters: Shaping the Workplace to Get the Right Work Done
Space Matters: Shaping the Workplace to Get the Right Work Done
 
Product Management in London - A Recruiters view...
Product Management in London - A Recruiters view...Product Management in London - A Recruiters view...
Product Management in London - A Recruiters view...
 
Intelligent Automation In Financial Services
Intelligent Automation In Financial ServicesIntelligent Automation In Financial Services
Intelligent Automation In Financial Services
 
Cognizant Jobs of the Future Index
Cognizant Jobs of the Future IndexCognizant Jobs of the Future Index
Cognizant Jobs of the Future Index
 
Insights success The 10 Most Valuable Outsourcing Companies 2016
 Insights success The 10 Most Valuable Outsourcing Companies 2016 Insights success The 10 Most Valuable Outsourcing Companies 2016
Insights success The 10 Most Valuable Outsourcing Companies 2016
 
Insights success the 10 most valuable outsourcing companies august 2016 (1)
Insights success the 10 most valuable outsourcing companies august 2016 (1)Insights success the 10 most valuable outsourcing companies august 2016 (1)
Insights success the 10 most valuable outsourcing companies august 2016 (1)
 
the-way-we-work-hr-today_pwc-en_2017.pdf
the-way-we-work-hr-today_pwc-en_2017.pdfthe-way-we-work-hr-today_pwc-en_2017.pdf
the-way-we-work-hr-today_pwc-en_2017.pdf
 
#Human Resource Management Challenges in the Digital Age# by Mr. Samantha Rat...
#Human Resource Management Challenges in the Digital Age# by Mr. Samantha Rat...#Human Resource Management Challenges in the Digital Age# by Mr. Samantha Rat...
#Human Resource Management Challenges in the Digital Age# by Mr. Samantha Rat...
 
Outsourcing 2019
Outsourcing 2019Outsourcing 2019
Outsourcing 2019
 
Digital Strategy Research Essay: Effects of difficult capabilities
Digital Strategy Research Essay: Effects of difficult capabilitiesDigital Strategy Research Essay: Effects of difficult capabilities
Digital Strategy Research Essay: Effects of difficult capabilities
 
Case Study : Aligning people capabilities to a company's digital strategy
Case Study : Aligning people capabilities to a company's digital strategyCase Study : Aligning people capabilities to a company's digital strategy
Case Study : Aligning people capabilities to a company's digital strategy
 
Futureproofing In House PR Teams - 3 Monkeys & PR Moment
Futureproofing In House PR Teams - 3 Monkeys & PR MomentFutureproofing In House PR Teams - 3 Monkeys & PR Moment
Futureproofing In House PR Teams - 3 Monkeys & PR Moment
 
The New Drivers of Recruiting and How Technology Should Enable Them
The New Drivers of Recruiting and How Technology Should Enable ThemThe New Drivers of Recruiting and How Technology Should Enable Them
The New Drivers of Recruiting and How Technology Should Enable Them
 
Mba project1
Mba project1Mba project1
Mba project1
 
BCG NASSCOM Future of Work Report_07JUNE2022_vFLR.pdf
BCG NASSCOM Future of Work Report_07JUNE2022_vFLR.pdfBCG NASSCOM Future of Work Report_07JUNE2022_vFLR.pdf
BCG NASSCOM Future of Work Report_07JUNE2022_vFLR.pdf
 
Enterprise Fusion: Your Pathway To A Better Customer Experience
Enterprise Fusion: Your Pathway To A Better Customer ExperienceEnterprise Fusion: Your Pathway To A Better Customer Experience
Enterprise Fusion: Your Pathway To A Better Customer Experience
 

Recently uploaded

Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neo4j
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
Kumud Singh
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
DianaGray10
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 

Recently uploaded (20)

Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AIMind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
 
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 

Chatbots in HR: Improving the Employee Experience

  • 1. Chatbots in HR Improving the Employee Experience Kong Yean Hwei, Amy | Advanced Work Based Project Report Crew 4 | Hyper Island Singapore | Jan 2017
  • 2. 2 Table of Contents EXECUTIVE SUMMARY ……………………………………………………………. 2 WHAT ARE WE EXPLORING? ……………………………………………………. Changing Landscape in Advertising ……………….…………….…………… Digital is Part of the Process ……………………………………….……………. Why Chatbots, and Why HR? …………………………………………………… 4 4 4 5 EXPLORING THE LANDSCAPE – A LITERATURE REVIEW ……………… Origin and Development of Chatbots ………………………………………… Why are Chatbots of Such Interest Now? ……………………………………. Applications andBenefits ofHRChatbots …………………………………… Chatbots in PubComms SG for HR Information Services ………………. Designing and Building Chatbots ……………………………………………… Some Considerations When Implementing Chatbots …………………… Learnings and Conclusions for an Experiment …………………………….. 6 6 7 9 11 11 14 15 THE HR CHATBOT EXPERIMENT ………………………………………………. Experiment Considerations ……………………………………………………… Experiment Design ………………………………………………………………. Experiment Round 1 with Twine – New Insights …………………………... Experiment Round 2 with Chatfuel – Validation and More ……………. Overall Findings and Conclusion ……………………………………………….. Limitations of the Experiment ………………………………………………….. 16 16 16 18 24 28 30 REVIEW AND RECOMMENDATIONS …………………………………………… Yay or Nay? …………………………………………………………………………… How Much Does It Cost? …………………………………………………………. It Could Save SGD 125,000 Annually ………………………………………….. 30 30 30 30 Suggested Timeline and Next Steps ………………………………………….. Other Recommendations and Conclusions …………………………………. The Time is Now ……………………………………………………………………. 30 32 32 REFLECTIONS ………………………………………………………………………….. 32 REFERENCES ..……………………………………………………………………..….. 35 APPENDICES ..………………………………………………………………..……….. Appendix 1: Key Questions to Ask Before Investing in a Chatbot …… Appendix 2: Proposal to PubComms SG …….…………..……………….… Appendix 3: Qualitative Interviewing Framework …..…..…………....… Appendix 4: Twine Documentation on ERIS Prototype 1 .......……….. Appendix 5: Experiment Data and Analysis ………………………….…….. Appendix 6: Chatfuel Documentation on ERIS chatbot ………….…….. 41 42 ABOUT THE AUTHOR The author was most recently the HR Director in Publicis Communications Singapore. She held dual roles, leading HR operational responsibilities for Publicis Worldwide, MSLGROUP and Nurun, and the Learning & Development function for all brands within Publicis Communications Singapore, including the brands mentioned above plus Saatchi & Saatchi, Leo Burnett and Prodigious. This Advanced Work Based Project is based on Publicis Communications Singapore. REDACTED REDACTED
  • 3. 3 EXECUTIVE SUMMARY Technology has disrupted the advertising and communications industry. Agencies are seeing a chronic decline in industry prices of 4.5% to 5% per year, compounded. Companies are trying to capitalize on this digital wave, including Publicis Groupe, the holding company of Publicis Communications,which has set this as a strategicpriority for the company. It is logical to assume that internal company processes should also reflect these digital priorities, and this Advanced Work Based Project aims to be part of the solution by exploring how technology applied to processes within Publicis Communications Singapore can provide value. Specifically, this report sets out to explore the following problem statement: “How might we explore the potential and applications of chatbots in HR, specifically in Publicis Communications Singapore, to improve the employee experience?” This report reviews the origin and development of chatbots, and the confluence of recent developments that have triggered the surge of interest in chatbots. It also reviews the potential applications of chatbots in HR and its benefits, and considerations needed for the design, development, and implementation of chatbots. An experiment using the Build-Measure-Learn feedback loop of the Lean Startup methodology was conducted in Publicis Communications Singapore to gain employee feedback on the problem statement. Two prototypes were tested with a small group of employees; one was a low- fidelity prototype using the Twine program, and the other a chatbot Minimum Viable Product (MVP) written using Chatfuel, a Do-It-Yourself chatbot builder. To access the ERIS MVP on Chatfuel, please email the author at amy.kong@hyperisland.sg for an access link. Based on the learnings of the literature review, the chatbot was developed as a text-based, rules- and retrieval-based, closed domain chatbot with some level of artificial intelligence. The topic of the chatbot was focused on providing information on a specific HR policy. Interviewees were asked to interact with the prototype, then semi- structured interviews were conducted to get their feedback on the prototype and their views on the usefulness and potential applications of chatbots in the HR function in PubComms SG. Overall results show that employees like chatbots’ ability to provide instant access to information, the ability for them to gain answers independently, and friendly tone. Each mentioned at least two other areas of applications of chatbots in HR, including providing information on company policies, processes, onboarding, and training. Additional insights that emerged included that interviewees felt chatbots would help them to avoid embarrassment when they needed to ask certain types of questions, and that chatbots could provide anonymity when they need to read up on sensitive company policies. Interviewees were clear that they would go to a chatbot for functional queries, while they would go to a HR team member for counsel on non- routine requests or emotional issues. This opens the possibility that chatbots could help to transition HR away from a transactional role towards an expertise role in the company. Based on the results of the experiment, there is significant potential and value in investing in HR chatbots to improve the employee experience and employee productivity. Calculations show that Publicis Communications Singapore could save about SGD 125,000 annually if just 5 minutes were shaved off each HR request from employees.
  • 4. 4 WHAT ARE WE EXPLORING? Changing Landscape in Advertising Technology has disrupted most industries, and it is no different in advertising. Where advertisements previously lay predominantly in the domain and control of advertising agencies, the advent of technology has changed that. The industry has shifted dramatically and advertising agencies now face challenges on multiple fronts: • Emergence of new forms of advertising such as online, mobile and search engine marketing (SEM) that has seen a redistribution of budgets from traditional forms of advertising (Pandey, 2016) • Competition from companies that were never considered competitors before, such as o Management consultancies (Dan, 2016; Roxburgh, 2016; Vranica, 2016) o Publishers (Marshall and Alpert, 2016) o Digital tech companies (Johnson, 2015) o In-house agencies (Schaefer, 2015; Stiglin, 2015; Morrison, 2016) • Commoditization of the industry as technology enables anyone to become creative content creators (Fromowitz, 2016; Oetting, 2015) As a result, the industry is seeing huge pressure on prices and margins. According to Farmer (2015), declining fees (at 2-3% annually) and growing workloads (2-3%annually)overthepast 20years have resulted in a chronic decline in industry prices of 4.5% to 5% per year, compounded. Digital is Part of the Process Many companies, and industries, are trying to capitalize on this digital wave, including Publicis Groupe. Publicis Groupe is the holding company that owns Publicis Communications (PubComms), the Client brand that this Advanced Work Based Project (AWBP) is based on. In fact, it is of such a strategic priority to Publicis Groupe that it separately reports its digital revenues in its quarterly earnings reports, and current digital revenues stand at 54% of total Groupe revenue in the third quarter of 2016 (Publicis Groupe, 2016b). It is also investing in training and development efforts for its employees. In its 2015 Annual Report Registration Document (Publicis Groupe, 2016c), it states, “…the goal is to ensure that each employee is able to acquire basic know-how, whether in rudimentary coding or better understanding the latest generation applications. The boom in mobile device usage and the new challenges of interconnectivity (connected objects) have led agencies to hire and train talented individuals with multiple skills, given the speed of industry change… It is essential to support our future managers.” If employees are expected to be digitally-savvy and able to recommend leading edge digital communications solutions for clients, then it is logical to assume that internal company processes should reflect these digital priorities as part of the digital immersion for company employees. According to Edgar Schein (cited in Christensen et al, 2016),processes are a criticalpart ofan organization’s unspoken culture. They tell people inside the company, “This is what matters most to us.” However, there are some that say that despite all the rhetoric, things are not changing: “What’s broken is not our people. It’s our process. It’s a process that has been endlessly debated but not reinvented, and it has not adapted to the changing world around us.” Grayson and North, 2016
  • 5. 5 “What’s broken is not our people. It’s our process. It’s a process that has been endlessly debated but not reinvented, and it has not adapted to the changing world around us... We traffic in risky ideas. But we don’t make any bold moves to change how we operate internally.” (Grayson and North, 2016) This Advanced Work Based Project (AWBP) aims to be part of the solution, by exploring how technology applied to processes in a company – Publicis Communications Singapore (PubComms SG) - can provide value. Specifically, the use of chatbots in HR. Why Chatbots, and Why HR? Chatbots are a hot topic now, but most of the current discussion seems to focus on the use of chatbots to engage with customers. One of the functions of Human Resources (HR), as a parallel, is like the employee customer service function of the company. So, would it be a far stretch to say that chatbots could also have significant potential in HR? Currently, most of the software or technology used in the PubComms HR function are used for non- employee facing activities (backend functions) to ensure operational continuity or as a management reporting tool, e.g. payroll systems, PTalent (Publicis Groupe’s HR Information System). There are currently few technologies used that facilitate or assist the interaction between the HR function and employees, and all interactions are manually managed, e.g. HR queries, notifications to employees on HR updates etc. Improving the employee experience cannot be underestimated. Advertising agencies are expertise- led companies, i.e. talent is a strategic differentiator, and so talent attraction and retention should be key activities for the company. This is even more important now, as new competitors like tech firms and startups are aggressively wooing talent away (Tadena, 2015; Ember, 2016). This represents an area of opportunity for exploration, to use technology to improve processes and help create a company culture that is aligned with its strategic priorities, that in turn improves the overall employee experience. As such, the defined problem statement that we will use in this paper is: “How might we explore the potential and applications of chatbots in HR, specifically in PubComms SG, to improve the employee experience?” ABOUT PUBLICIS COMMUNICATIONS BACKGROUND PubComms is one of 4 Solutions hubs within Publicis Groupe and comprises all of Publicis Groupe’s creative brand networks, including Publicis Worldwide, MSLGROUP, Nurun, Saatchi & Saatchi, Leo Burnett, BBH, Marcel and Prodigious (Publicis Groupe, 2016a). Publicis Groupe is one of the largest marketing and communications company in the world, alongside WPP, Interpublic, and Omnicom (Elliott, 2002). In early 2016, a Chief Executive Officer was appointed to lead Publicis Communications in Singapore (Mumbrella Asia, 2016), and subsequently a Chief Talent Officer and a Chief Finance Officer. The finance and HR functions across the various brands were consolidated to align with the new management structure, overseeing a headcount of about 250 people. As each creative brand previously had its own HR policies, the HR team (author included), was tasked to align and improve employee and HR operations and policies across the various brands under PubComms SG. CLIENT STAKEHOLDERS The key client contact for this project was Dan Spencer, Chief Talent Officer, Singapore and Australia/New Zealand, Publicis Communications. Other key stakeholders included the HR leaders of PubComms SG, Sharon Ooi, HR Director, and Adele Sam, Talent Manager, who provided the information and liaison support in the execution of the experiment. REDACTED REDACTED REDACTEDREDACTEDREDACTED
  • 6. 6 EXPLORING THE LANDSCAPE – A LITERATURE REVIEW Origin and Development of Chatbots What is a Chatbot? Different terms have been used for chatbots (or chatterbots), including conversational agents, dialog systems and virtual assistants. This is due to the proliferation of a variety of similar systems built with different technical architecture (Perez-Marin and Pascual-Nieto, 2011), for different purposes. For this report, we define a chatbot as an automated online software program that tries to mimic a human conversation in its interaction with a human user to answer questions or perform tasks (Shawar and Atwell, 2007; Knowledge@Wharton, 2016). Specifically, we will focus this report on text-based chatbots. Chatbots can be built for websites or apps and accessed through smartphones, tablets, desktops and tablets. In recent years, most of the major messaging platforms have launched tools to help developers integrate or deploy chatbots on their platforms. These include Facebook Messenger, Slack, Telegram, Google and Microsoft, Kik, among others (Richman, 2016). What makes a Chatbot? For a chatbot to conduct human-like conversation, ArtificialIntelligence (AI) is a criticalcomponent.Nils J. Nilsson, as cited by Stone et al. (2016), defines AI as the “activity devoted to making machines intelligent, and intelligence is that quality that enables an entity to function appropriately and with foresight in its environment”. AI has many applications (Stone et al., 2016), but in this report, we will narrow the topics to those relevant to text-based chatbots. Chatbots functions in two ways (Schlicht, 2016): 1. Rule-based: These types of chatbots respond to very specific commands (rules), and are unable to process if information is not input in the right way. It is only as smart as you program it to be 2. Machine Learning: Machine Learning is a form of AI, and machine-learning chatbots continuously get smarter as they learn, via Natural Language Processing (NLP), from the conversations it has with people The recent boom in chatbots has been fuelled by advancements in AI research, combined with technological improvements such as the availability of cloud computing resources and wide-spread, web-based data gathering (Stone et al., 2016). These advancements in AI include the areas of: • Natural language Processing (NLP) and Natural Language Understanding (NLU) NLP explores how computers can understand and manipulate natural language text or speech to do useful things (Chowdhury, 2005). NLU is a subset of NLP that deals with machine reading comprehension (Ovchinnikova, 2012), i.e. the ability of the computer to understand human language. In short, NLP applications try to understand natural human communication and communicate in return (Marr, 2016). • Machine Learning With machine learning,computers areenabledto continuously learn from data on its own, using algorithms (set of rules or steps), to find insights without being explicitly programmed where to look (Sas.com, n.d.). In chatbots, machine learning is used to help machines learn and understand the vast nuances in human language, and learn to respond in a manner that the audience is likely to comprehend (Marr, 2016). While this report will not go into the details of these technologies, these are key terms to understand as they will be mentioned in any discussion around chatbots.
  • 7. 7 Key Developments in AI and Chatbots Turing Test and Loebner Prize Competition One of the most significant milestones was created when Alan Turing created “The Imitation Game”, now known as the Turing Test, in 1950. Starting with the question “Can machines think?” (Turing, 1950), Turing created a test that would determine if a computer is capable of thinking like a human. The Turing Test has since been used as a benchmark that any AI must pass en route to true intelligence (Ball, 2015). However, it was not until 1991 that there was an implementation of the Turing Test (Loebner.net, n.d.b) – the annual Loebner Prize Competition – created by Dr. Hugh Loebner to advance the field of AI (Loebner, n.d.a). As long as there is an entry for that year, the prize will be awarded, and so while there have been multiple Loebner Prize winners, no program passed the Turing Test until 2014 (Griffin, 2014). The First Chatbot, ELIZA Then in 1966, the first chatbot ever coded, ELIZA, was documented by Joseph Weizenbaum. ELIZA was written to simulate a psychotherapist conducting a psychiatric interview (Weizenbaum, 1966). ELIZA worked by analysing the words users inputandmatching themto a list ofpossible scripted responses (Newman, 2016). More critically, ELIZA seemed so believable to users that Weizenbaum perceived his program as a threat, and in his book Computer Power and Human Reason, he attacked AI (specifically ELIZA) and computer science research, thereby slowing the pace of research into AI (Epstein, Roberts and Beber, 2008). A.L.I.C.E In 1995, A.L.I.C.E. (Artificial Linguistic Internet Computer Entity), the first AIML-based (Artificial Intelligence Markup Language) personality program, was created. AIML is a computer language designed for creating natural language-based chatbots that is still used today. A.L.I.C.E.won theLoebnerPrize three times,in 2000, 2001 and 2004 (Epstein, Roberts and Beber, 2008), and is one of the strongest AI programs of its time. It was programmed to give the illusion that it was intelligent and self-aware, but was driven by “supervised learning”, where a botmaster monitors the conversations and creates content to make responses more appropriate, accurate, “human”, etc (Epstein, Roberts and Beber, 2008). Bots for Messenger In April 2016, Facebook launched the bots for Messenger Platform (Marcus, 2016). While Facebook is not the first company to launch a bot platform, it is significant because it now provided, at scale, the opportunity for businesses to automate their one-to-one engagement with their customers to drive business and e-commerce (Rosenberg, 2016), thus driving the current interest and boom in chatbots. Figure 1 shows a more detailed timeline on the history of chatbots. Why Are Chatbots of Such Interest Now? A confluence of several developments has triggered the surge of interest in chatbots: Advancements in AI Research and Computing As mentioned, advancements in AI research, particularly in machine learning and NLP, are enabling chatbots to provide more accurate and intelligent responses. Improvements in computing and hardware technologies have also enabled integration of various apps into a chatbot that allow chatbots to complete tasks with little or no operator intervention. In fact, research firm Gartner predicts that in 2017, only 33% of all customer service interactions will need a human intermediary, as compared to 60% in 2014. (Gartner.com, 2015)
  • 8. 8 Figure 1.: The History of Chatbots (Futurism, n.d.)
  • 9. 9 Rise of Mobile Messaging and Fall of Apps According to Business Insider (2016), monthly active users of messaging apps (where chatbots can reside) surpassed social networks in 2015 in terms of monthly active users (see Figure 2). Companies also face increasing friction in getting consumers to download and use a mobile application. The average mobile user globally has 33 apps installed on his or her device, of which 12 apps are used daily. Of that, 80% of the average global mobile user’s time is spent only on 3 apps (Meeker, 2016). This means that companies and brands are finding it increasingly difficult to engage one-on-one with customers. App Integrations Attract Users to Stay Within Platform Eco-system The ability to integrate various apps into a chatbot also means that users are attracted to stay within the messaging app (Bayerque, 2016). In the context ofbrands,it means that customers are likely to stay within the brand eco-system, thereby increasing duration of interaction with the brand. Bots Can “PiggyBack” Existing Massive Platforms Bots are the beginning of micro apps on the backs of massive platforms which will lead to more focus and reach for start-ups and more delighted users (Batalion, 2016) because brands can: 1. Spend less time in development, e.g. building a chatbot on Facebook Messenger allows you to build one experience (without needing to write for different mobile phone sizes, or operating systems) and access Facebook user rich profiles instantly 2. Focus more time on creating positive user experiences because they can leverage richer user interfaces and software plug-ins (that platforms develop or offer) to process input What all this means is that companies and organisations can now increase quality one-to-one, personalised engagements with customers at scale, at potentially reduced costs and time. Applications and Benefits of HR chatbots Current Applications of Chatbots in HR General HR Support Information Services (Search and Retrieval) A typical challenge that employees face is knowing where and how to find information on HR policies and processes, or even whom to approach within the HR department for specific queries. Benefits of using chatbots in information services include (McNeale and Newyear, 2013): • Tackling the problem of users not familiar with specific terminology and jargon • Unlike humans, chatbots are unruffled by rude customers, high traffic, or repeated requests for the same information, and remain consistently patient and polite This also shifts the responsibility of locating the needed information from the user to the designer of the chatbot where the designer leads the user through a question and answer dialogue to discover the information needed and provide it (McNeale and Newyear, 2013). An example of such a chatbot is “Ask Ivy”, a “virtual HR agent” launched by Intel in 2013 that uses a combinationofNLP,AIandoptimizedsearchhosted on their Intranet to help answer employees' questions about their pay, stock, benefits, or other HR programs (Pearce, 2013). Figure 2.: Messaging Apps Surpass Social Networks (Business Insider, 2016)
  • 10. 10 HR Task Automation or Assistance Virtual personal assistants are now a big focus due to the ability to now integrate several apps (and actions) into chatbots. The most current and well known examples are iPhone’s Siri, Amazon’s Alexa, Microsoft’s Cortana and the Google Assistant (Dunn, 2016). In the domain of virtual chatbot assistants in HR, there are some products now in development in the market, including: • HR task automation chatbot from ADP Innovation labs that is still in development (Boulton, 2016) • Mila, overstock.com’s chatbot created to assist with employee sick leave reporting and team rescheduling at their call center (Greenfield, 2016; Aspect.com, n.d.) Recruitment Streamlining Job Applications Chatbots can be used to vet potential candidates (Bhaduri, 2016). One recent example is Mya (http://trymya.io/), a chatbot that vets and interviews job candidates, using textual analytics to filter irrelevant resumes and interviews selected candidates on their work experience and qualifications. Mya also administers tests, provides application status updates and tips to candidates (Dishman, 2016; Deutscher, 2016; Trymya.io, n.d.). Q&A Tool for Job Applicants Chatbots can also serve as a Q&A resource for potential candidates on the position they are hiring for, or the company’s hiring policies (Sullivan, 2016). Benefits of using chatbots this way include: Higher quality of applicants (through candidates’ self- selection), improved candidate experience, reduce time spent by recruiters (and costs) in answering repeated FAQs from candidates. One of the most established examples is Sgt. Star, launched in 2007 by the United States Army to help potential recruits to learn about a career in the Army. First residing on the Army recruitment webpage,Sgt. Star then made its debut as a mobile app in 2014. (Maass, 2014; Nextit.com, n.d.). Onboarding Chatbots can also beusedto contactnew employees (or allow them to self-serve) with onboarding information, such as organisational structure and history, how to get enrolled in programs or benefits, suggestions on colleagues to meet, and links to the required HR forms, in place of a HR generalist (Boulton, 2016; Zwier, 2016). Training (Bhaduri, 2016) There is already ongoing discussion in academic and education sectors on the value of chatbots in training and learning that can be applied to the corporate sector (Perez-Marin and Pascual-Nieto, 2011; Olson, 2016). Chatbots can be used to deliver interactive and personalised training, monitoring and nudging, and instant help (Newton, 2016; Riel, 2016). Benefits of Using Chatbots in HR Based on the above, the potential benefits of using chatbots in HR include improving: • Employee productivity and employee experience by: o Providing on-demand employee engagement with 24/7, instant responses o Simplifying information access o Supplying just-in-time and fast access to information o Personalisation of the employee interaction • HR team’s performance (value-add, productivity and motivation) by o Handling repetitive tasks (e.g. answering FAQs) thereby - Freeing up HR resources to focus on more value-added work or reduce costs - Reduce frustration and/or monotony of the HR team o Automating standardised tasks o Identifying potential issues using data analytics and addressing them before they escalate
  • 11. 11 Chatbots in PubComms SG for HR Information Services The experiment conducted to explore the problem statement of this report focused on the use of chatbots to provide HR information services in PubComms SG as it strikes the best balance between feasibility and desirability. Why Information Services? Firstly, the HR function in PubComms SG already has ready content that can be re-purposed to populate any chatbot prototype. Secondly, as a first investment in chatbots, the easiest chatbots to build are rules-based chatbots that rely on specific topics (closed domain) with predefined responses (retrieval-based) (Figure 3.). Having machine learning capabilities is a bonus and can be incorporated into the chatbot with subsequent iterations (Varagnat, 2016). Jumping into a chatbot that requires extensive machine learning capabilities from the start will create barriers of entry as it can be time-consuming (due to lack of expertise) and potentially costly. What About the Other Applications of Chatbots? While recruitment is a constant needfor PubComms SG, it is not feasible because: • In the communications agency environment, the company information that candidates would be interested in (e.g. client roster) is considered confidential. It is not available on public forums and usually only communicated face-to-face to the candidate verbally during the interview. The highly customised nature of remuneration in the advertising industry also does not suit the use of chatbots for recruitment Q&A at this point in time • Using chatbots to vet potential employees require sophisticated filtering technology as the interviewing process also involves interviewing for soft skills. The technology is still very new, and to demonstrate the potential of chatbots in PubComms SG, it is better to choose an area that is more established. Additionally, the various hiring managers within PubComms SG have different interviewing and selection criteria, and it would be difficult to standardise this quickly to build the prototype. Finally, most off-the-shelf products in the market are US- based and there may be cultural differences in the way resumes are written, or how candidates respond in an interview. In the area of HR task automation, companies openly admit that it is very early days and all these assistants are far from polished (Dunn, 2016). As for onboarding and training, there are currently no consolidated company information materials (due to the recent re-organisation) that can be used to populate the chatbot. Use cases of the value of chatbots in corporate training in the industry are also still few. Designing and Building Chatbots This section is split into the following subsections, to explore best approaches for designing and building chatbots: 1. What Consumers Want 2. Three Approaches to Creating Chatbots 3. User Experience Design Consideration REDACTED
  • 12. 12 Appendix 1 also provides a list of key questions to consider prior to investing resources in a chatbot. What Consumers Want Prior to designing a chatbot,it is worth exploring the functions and experiences consumers want. As this is a relatively new area of interest commercially, there are no any large scale, benchmark global consumer surveys that have been conducted, but some surveys have been conducted in specific markets. These provide starting insights that could be validated through Proofof Concept experiments. Based on results of three recent consumer surveys, one in the US and two in the UK with sample sizes of 1,000 respondents each (Aspect, 2016; Mindshare, Goldsmiths University of London, 2016; myclever Agency, 2016), these are some of the findings grouped around certain central themes: “Humanness” of a Chatbot Consumers prioritize efficiency in understanding their queries over the personality or friendliness of the chatbot. This was raised in the two UK surveys. However, the US survey indicated that consumers expect chatbots to perform better in “friendliness” and “ease of use” as compared to “accuracy” and “interaction success”. These could be inter-related, i.e. consumers don’t have a good experience with the efficiency of chatbots, therefore prioritising them higher than the tone of interaction of chatbots. Expected Benefits • Getting 24-hour service • Quick answers to simple to moderate questions • Getting an instant response • Being able to self-serve without having to talk to a customer service agent Predicted / Expected Functions of Chatbots • Quick emergency answers • Forwarding to appropriate human when requested by user • Buy basic items (clothes, food) • Retain information on context and history of all previous interactions for a more personalised experience • Get basic information about a company, product, or service • Information on product availability Three Approaches to Creating Chatbots There are three ways to create chatbots (Vorobiov and Kotsurenko, n.d.): 1. Buy a ready solution: there are vendors who offer off-the-shelf solutions. Examples of HR- specific chatbot solutions include Mya, mentioned earlier, Talla for onboarding employees within Slack (https://talla.com/). This allows businesses to speed up time-to-market, but the downside is cost and not all ready-made solutions are customisable. 2. Use DIY chatbot builders: there are a variety of Do-It-Yourself (DIY) bot builders, some requiring technical knowledge, and others not requiring any. A higher degree of customisation is possible, and it is relatively cheap and quick. Functions may be limited but new features are constantly being added.However,it can be hardto compare functionalities and prices across chatbot builders as business models vary widely. 3. Build a chatbot from scratch: While technically still the most challenging, there are many developer tools available in the market that make chatbot building much easier now. Naturally, this option requires technical expertise and more cost, but results in a solution that is tailor-made for, and proprietary to, the company. Figure 4 shows an infographic ofthe current chatbot landscape. This infographic does not include chatbot development vendors as they are industry- specific, but it gives a good overview of the market. As the objective of this report is to explore the potential and applications of chatbots in HR to improve the employee experience, we will focus on using DIY chatbot builders (referred to as chatbot builders from here) as it is the cheapest and fastest way to build a prototype, and evaluating technical considerations is not an area of discussion here.
  • 13. 13 User Experience Design Considerations Given that part of this report’s problem statement is concerned with improving the PubComms SG employee experience, we need to examine how to create good User Experience (UX) in a chatbot. Thisis especially whenchatbotsnowenableone-on- one interactions between the user and the company, the quality of that interaction becomes more important. Many articles use the terms UX and User Interface (UI) interchangeably. For clarity, we attempt to make a distinction between the two. The industry perspective of UX focuses on the “optimisation of a product for effective and enjoyable use”, including ensuring the product logically flows from step to step (Lo, 2014; Lamprecht, 2016). UI, on the other hand, is concerned with the “look and feel, the presentation and interactivity of the product”that guides theuser intoa goodinteraction with the product. To simplify it even further, the comparison between UX and UI: “Something that looks great but is difficult to use is exemplary of great UI and poor UX. While something very usable that looks terrible is exemplary of great UX and poor UI.” Helga Moreno, as cited by Lamprecht (2016) In this context, a large part of creating a good UI is already handled by chatbot builders as their products come with pre-set user interfaces. Some industry best practices to designing a good UX in chatbots (Ady, 2016; Howard, 2016; Jain, 2016; Yao, 2016): Smallest Number of Steps Plan the conversation flowto be efficient,so that the user reaches his goal in a few steps as possible. Always Provide a Path to the End Goal Give users cues to reach next steps that lead to the end point, so that they are not left hanging mid-way in a conversation without knowing what to do. Figure 4.: Chatbots Landscape (Medium, 2016)
  • 14. 14 Decide Parameters of Information to be Stored If one of the aims of the chatbot is to provide a personalised experience, then information about the user needs to be retained / stored. Establishing parameters (or categories) of information to be stored helps to provide targeted or relevant information for users. For example, if there are different sets of employee benefits associated with different job titles in the company, establishing job titles as a parameter allows the chatbot to retrieve the different sets of employee benefits by the user’s job title. Set the Right Expectations • Explain the purpose of the chatbot • Explain what the chatbot can do • Explain how to interact with the chatbot • Set a first action for the user to take • Provide hints or prompters on the possible paths of exploration at the end of the onboarding Allow Flexibility of Exploration Allow users the flexibility to skip around options. Use Buttons to Supplement Free Text Input Particularly for closed domain chatbots, buttons provide more accuracy in interpreting users’ requests since the options are pre-defined. Clicking a button is also faster than typing text. Layer AI to Interpret User’s Requests Use AI to learn the various ways users phrase requests differently so that the chatbot can trigger the right response or action, e.g. “Hi”, “Hello”, “Wassup”, “Yo” should trigger a return greeting or opening message from the chatbot. Use Images or Graphics to Supplement Text Images or graphics are easier to understand than text explanations. Provide a Null Response If the chatbot is unable to understand what the user wants,a message shouldbe triggeredto indicate so. Non-responses from the chatbot where the user is left not knowing what to do next should be avoided. Tone Should Reflect the Brand Personality For example, the chatbot may want to use more casual, trendy words if it is targeted at millennials. Also consider the use of emoji or gifs or graphics appropriate to the chosen tone of voice. Help Users Get Help Provide options for users to get help outside of the chatbot as needed. This can include directing the user to documentation, or a human point of contact for further inquiries. Integrate with Existing Business Systems Enable chatbots to access CRM databases to allow a richer experience with users and avoid having the user repeat their history or user details every time they interact with the chatbot. Some Considerations When Implementing Chatbots Hype There is a lot hype currently around chatbots, such that the general public's expectations of what chatbots can do will exceed the reality of what they can actually do (Hobson, 2016). This means that in launching any chatbot, companies should take care to communicate clearly to users what they should expect from any interactions with that specific chatbot. The reality also is that not every bot needs to be sophisticated, and it will depend on the objective or outcome that the chatbot is built for. Ifit is meant as an informationalservice,there is very little point in building a bot to conduct a conversation with a user, when the bot is meant to be transaction-based (Newman, 2016). Companies that are considering implementing chatbots need to be clear about the objective or goal of the chatbot and evaluate the technologies or functions needed to achieve that, rather than include unnecessary user features that result in additional cost for the company.
  • 15. 15 Benchmark for a Minimum Viable Product (MVP) Software companies are used to developing a MVP, or a minimum standard version of the product that consumers would be willing to buy, to test in the market. However, for a good user experience, the product will likely need to have more accurate NLP and information before a MVP can be developed, which may mean that chatbots could require more capital than a traditional web or mobile app, where good frameworks are more commonly available (May, 2016). As above, clarity in the goal of the chatbot is essential,to be able to strip down to a MVPfor users. Alternatively, companies can first pilot a chatbot in a very confined and specific area to minimise capital outlay, while not compromising on the UX of the chatbot. This is the approach that has been adopted in developing the chatbot prototype for PubComms SG, as seen in the next sections of this paper. Learnings and Conclusions for an Experiment Chatbots are not new, but advancements in AI and computing technology have enabled the development of chatbots that can understand users better, have more meaningful conversations and perform tasks more effectively and accurately. This has opened chatbots for commercial possibilities at scale,particularly chatbots on messaging platforms. There are multiple possibilities of applying chatbots in HR; specific to PubComms SG, it is recommended that the focus of any chatbot pilot be focused on providing information services on specific topics that allow a MVP to be developed quickly and cost- effectively for piloting with employees. Also, the best approach in creating one seems to be the use of chatbot builders to minimize cost, increase the speed of prototype development and remove the constraints of needing technical expertise. It also allows more effort to be focused on providing a quality user experience to help employees better visualise the value and possibilities of using chatbots in the HR function of PubComms SG.
  • 16. 16 THE HR CHATBOT EXPERIMENT Experiment Considerations Demonstrate Proof of Concept (POC) PubComms SG has never implemented similar technology in its support functions, so it was important to gain enough feedback to demonstrate POC to the PubComms SG HR team for them to evaluate if resources should be invested in such a tool, and provide justification to higher management on its value. Lean Resources to Build the Experiment As there was no budget, the experiment would have to be built on free platforms. Platforms That Do Not Require Much (or Any) Coding Knowledge As the author has no background in coding, the chatbot would have to be built on tools that require little to no coding knowledge. As the objective of this experiment was to gain employees’ feedback, and not evaluate technical capabilities of chatbots, this would not pose a significant problem in the experiment. Topic where a Chatbot would Provide Noticeable Value To demonstrate the value of a chatbot to users, the topic selected would need to be one where it is not common knowledge, or of a complexity level that would require employees to seek information or help. Ease of Testing and Confidentiality As PubComms SG currently has no intranet, the experiment would have to be designed and hosted on external (and possibly public) platforms. As such, the topic selected for the experiment should have no confidentiality impact on PubComms SG. Speed The chatbot had to be created quickly as the experiment neededtobe builtandconductedwithin one month. Experiment Design Parts of the Lean Startup methodology (Ries, 2011) was used to design the experiment. Advantages of this methodology vs traditional research or product development methods include developing products that customers (employees) want, more quickly and cheaply, and making it less risky (Blank, 2013). If we were to view the proposed HR chatbot as a startup, there are three distinct stages (Figure 5.) and two periods of focus (Maurya, 2012): Stage 1: Problem/Solution Fit – Is there a problem worth solving? Stage2:Product/Market Fit – HaveI built something people want? Stage 3: Scale – How do I accelerate growth? The experiment was designed to explore and answer Stage 1 and 2; i.e. show validated learning that will help with presenting recommendations to the client. PROBLEM / SOLUTION FIT • Stage 1 PRODUCT/MARKET FIT •• Stage 2 SCALE ••• Stage 3 Focus: Validated Learning Experiments: Pivots Focus: Growth Experiments: Optimizations Figure 5.: Three Stages of a Startup (Maurya, 2012)
  • 17. 17 Build-Measure-Learn Feedback Loop A key component of Lean Startup is the Build- Measure-Learn Feedback Loop (see Figure 6.) The aim of this is to go through the Feedback Loop in as little time and as few resources as possible to maximize learning through incremental iterative engineering and gain insights (Blank, 2015). It would allow the requiredminimum feedback to be gathered to demonstrate POC, enable the experiment to be conducted quickly, enable pivots on the idea, and help sharpen the final recommendations made to the client. It was also made clear to the client that this is was an exploratory pilot project, as evidenced by the Research Questions listed in the project proposal to the client (Appendix 2) where the expected outcome of the project was “A set of recommendations on the value and use of chatbots in HR to improve the employee experience…”. According to Ries (2011), planning for the Build- Measure-Learn (BML) Feedback Loop needs to be done in reverse, i.e. “we figure out what we need to learn … figure outwhatweneedto measure toknow if we are gaining validated learning, and then figure out what product we need to build to run that experiment and get that measurement”. As such, the subsequent sections explaining the experiment are elaborated in reverse of the BML Feedback Loop to demonstrate how this experiment was planned. Rounds of Experiments The original goal was to build a Minimum Viable Product (MVP) (Ries, 2011) for testing, but as this is a very new topic and the first time the author was exploring it, it was difficult to estimate the timeframe needed to build the MVP and the quality of the MVP. This made it challenging to manage client expectations on the output to expect. As a solution, it was then decided to set a minimum goal of producing a low-fidelity prototype, and a stretch goalofproducing an ERIS chatbot MVP.This was also in line with the Lean Startup methodology of gaining learnings and feedback as early in the process as possible. The client was provided with interim updates at the end of each experiment. Data Collection and Interviewee Selection Qualitative Interviews The prototype testing was conducted via qualitative interviews for the following reasons: 1. Target audience universe is small. Headcount for PubComms SG is approximately 250, and therefore any quantitative research would need relatively large numbers which was not feasible due to the time constraints 2. Main objective is learning. The purpose of the experiments was not just to validate or invalidate hypotheses, it was also to gain other insights that may help with any pivots (Ries, 2011, p. 178) which cannot be achieved with closed-ended, quantitative research Face-to-face, semi-structured interviews, i.e. interviews conducted following an interview guide using open-ended questions, were conducted among employees.Figure 6.: Build-Measure-Learn Feedback Loop (Ries 2011, p. 75) Ideas BUILD Product MEASURE Data LEARN Minimize total time through the loop REDACTED
  • 18. 18 Employees were first given the prototype to try, and then asked for their reactions to the prototype and thoughts on the potential usefulness and applications of chatbots in the HR function of PubComms SG. Responses from interviewees were anonymised, coded and categorised for analysis. Using qualitative interviews as a research method would not only help to gain additional insights, but it would also ensure that there was no confirmation bias – defined as “seeking or interpreting evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand” (Nickerson, 1998) – as the responses would be non-prompted. As with allqualitative data analysis,there is a certain levelofsubjectivity as interviewee responses are not standardized, and there is a level of judgement and interpretation required to code the data. Interviewee Selection A small sample of employees, 6 employees for each round of experiment (12 individuals across two rounds of experiments) was chosen. To ensure a fair representation of employees and reduce sampling bias (Uwex.edu, n.d.), the client selected the employee interviewees for each round based on a provided profile matrix (Table 1). The client was also asked to choose employees from a mix of job roles and brands. Eventually, 11 of the 12 selected interviewees responded within the response timeframe and were interviewed. Experiment Round 1 with Twine – New Insights Learning Objective: Value Over Growth According to Ries (2011, p. 81), the riskiest elements of any plan, are “leap-of-faith” assumptions, because the success of the venture rests on them. Two of the most important are the value hypothesis and the growth hypothesis. The value hypothesis tests whether a product or service really delivers value to customers once they are using it, and growth hypothesis tests how new customers will discover a product or service (Ries 2011, p. 61). For this experiment, the value hypothesis was chosen over the growth hypothesis for testing as this is the problem statement we were trying to answer, andemployees wouldhaveno problems discovering the chatbot as there are internal communication tools to publicize the availability of this tool. Going back to the problem statement, “How might we explore the potential and applications of chatbots in HR, specifically in PubComms SG, to improve the employee experience?”, the experiment sought to find interviewees’ feedback on two aspects: 1. Potential of HR chatbots in PubComms SG to improve the employee experience 2. Applications of HR chatbots in PubComms SG to improve the employee experience If these two statements yielded positive results, then it would support a client recommendation to invest in one. Measure: Using Literature Review as a Starting Point Using the learnings from the Literature Review as a starting point, the metrics of the experiment were defined as non-prompted responses from interviewees on the following: 1. Potential of HR chatbots in PubComms SG to improve the employee experience o 24-hour service o Quick answers o Getting an instant response o Ability to self-serve without having to talk to a customer service agent o Get basic information on the policy
  • 19. 19 2. Applications of HR chatbots in PubComms SG to improve the employee experience o Ability of interviewees to envision 2 or more uses of chatbots in the HR function Two assumptions were made: • Non-prompted feedback from interviewees would reflect their priorities and desired benefits from chatbots • If interviewees perceived that HR chatbots provided value, they would be able to envision multiple possibilities for its application Other Measurements and Data Other questions were also developed to mine information that would help in exploring other areas of opportunities (see Appendix 3 for the Qualitative Interviewing Framework that was used as a guide to conduct the interviews). This included seeking information from employees on: • Product features anddesign thatemployees like, that would help with subsequent iterations of the chatbot • Other employee needs that could be solved, that would help the HR team to improve the overall employee experience • Other technology tools that employees may desire for use in PubComms SG that may help identify other areas of opportunity for technology investment in the company • Their perceptions of the role of chatbots within the HR team, to better understand how best to deploy technology and human resources, from an end-user perspective There were also other questions embedded into the interview for the benefit of the client, particularly two questions: 1. How would you describe the quality of your interactions with the HR team? 2. How would you rate the HR team’s use of technology to provide HR services to employees? These questions had no significant impact on the experiment, but were included to help the client better understand employees’ context and perspective of employees’ views of the HR team. Build: Focusing on the Conversational Flow Topic The selected topic for the chatbot was the Employee Referral Incentive Scheme (ERIS), where employees stand to receive a monetary incentive if they successfully refer their friends for open job positions in Singapore. ERIS was chosen because it is: o A defined area of inquiry that would provide a clear scope for the chatbot o A relatively new policy that employees still require assistance in understanding o Not too simple that employees find it easier to refer to policy documents o Complex enough for the chatbot to provide noticeable value-add to employees, but not exceeding the author’s technical capabilities and resources in creating the chatbot o Of a low level of confidentiality that would not impact PubComms SG should the prototype need to be hosted on public platforms, e.g. Facebook Messenger Software The low fidelity prototype was a html version simulating a chatbot conversation, built using the Twine program (https://twinery.org/). Originally created to write online choose-your-own-adventure games, it is a useful tool in designing and planning the conversational flows in chatbots because it allows for interactive, non-linear flows, like a conversation, without requiring the creator to have technical coding or programming expertise (Init.ai Decoded, 2016; WillowTree, Inc., 2016; Winstead, 2016; Vaneseltine, 2014). Twine was chosen as it provides an overview of the conversation flow (Figure 7), to allow the author to evaluate the effectiveness of the conversation, e.g. help users to get to the end point as fast as possible. Refer to Appendix 4 to access and view the ERIS conversational flow written on Twine (requires download of the free program to view). Video demonstrations of the front-end and back-end view are also included in the same appendix and shown in Figure 8 and 9.
  • 20. 20 Figure 7.: ERIS – Twine Conversational Flow REDACTED
  • 21. 21 Insights surfaced in the literature review section on good chatbot UX were incorporated. This includes ensuring that the prototype: • Set the right expectations • Allow flexibility of exploration • Create the tone to reflect the brand personality • Help users get help However, as Twine does not have certain functionalities of chatbots, the following best practices were not incorporated: • Use buttons to supplement free text input • Layer AI to interpret user’s requests • Use images or graphics to supplement text • Introduce product features gradually • Provide a null response • Integration with existing business systems • Securing information security and privacy of information This prototype was shown to interviewees on a desktop browser, rather than through a messaging platform. Results: New Insights Potential of HR Chatbots in PubComms SG Interview results show that the hypothesised benefits of chatbots to improve the employee experience were validated. Interviewees said that they thought that the chatbot provided good information on the policy and was personable. They also said that a chatbot would provide instant answers and enable employees to find information independently. Employees also rated the usefulness of a chatbot in helping to improve the employee experience 7.6 out of a possible 10, where 10 is the most useful, and 1 the least useful. Some interesting insights emerged from the interviews: Chatbot is easy to use ERIS is a relatively more complex HR policy within PubComms SG due to differing incentive amounts (depends on the type of job position employees are making referrals for), and incentive payment processes (depends on the nature of the employment contract of the referring employee). The conversational flow in the prototype was designed based on conditional logic, i.e. users were taken through a series of questions, and based on their answers, the chatbot would provide information that was specificto their situation (refer to Appendix 4 for documentation on the Twine program). Therefore, this feedback from interviewees, especially when it was the most mentioned feedback, is surprising. This indicates that a chatbot has the potential for simplifying complex information for users, if planned well. Chatbots help employees to avoid embarrassment An unexpected feedback from employees was that chatbots would help employees to avoid embarrassment when they needed to as HR questions that they felt were trivial. Chatbots help to provide anonymity This was raised specific to situations when colleagues are trying to understand policies on sensitive issues: “In an Asian culture, people are more non-confrontational, and may not want to raise alarm bells until they are sure. Making such information readily available without having to ask a HR team member [for access] allows them to do their initial research”. “In an Asian culture, people are more non-confrontational, and may not want to raise alarm bells until they are sure. Making such information readily available without having to ask a HR team member [f0r access] allows them to do their initial research.” REDACTED
  • 22. 22 Figure 8: ERIS – Demonstration of Twine Front End User View VIDEO REDACTED
  • 23. 23 Figure 9: ERIS – Demonstration of Twine Back End View VIDEO REDACTED
  • 24. 24 Chatbots help to relieve the HR team Beyond seeing the benefits for themselves, most saw the benefit of using chatbots to help relieve the HR team’s workload so that they can redirect their time to focus on more important issues. Applications of HR Chatbots in PubComms SG All interviewees mentioned at least two other areas of applications, including: • Company policies • Company processes • Company information • Employee contract details • New employee induction / orientation • Training • IT requests Interestingly, the most of them related to information services. Refer to Appendix 5 for the full results and analysis of the interviews. UX Feedback Some areas of improvement that interviewees suggested to improve the user experience: • Reduce the wordiness of the chatbot • Have a “home” button (there was a “back” option in the prototype, but not a “home” option) Role of Chatbots vs HR team Interviewees were also clear on the role of the chatbot within the HR team. They would use a chatbot for functional or transactional queries and/or generic FAQs, and approach a HR member when they need counsel on non-routine requests, emotional issues, or exceptions to the regular policies and processes. This seems to indicate that chatbots can help transition HR away from a transaction perception in employees towards an expertise role. Experiment Round 2 with Chatfuel – Validation and More Learn: Would Feedback be the Same? Based on the learnings from Experiment Round 1, the following decisions were made on the desired learning from Round 2: • Potential of HR Chatbots in PubComms SG - To reconfirm the benefits even after the prototype is changed from a html, desktop version to a chatbot MVP hosted on a mobile messaging platform - Continue to listen and monitor if the new insights would continue to be mentioned in Round 2, thereby validating them • Applications of HR Chatbots in PubComms SG - Reconfirm the desired applications of chatbots in HR Build: Creating a chatbot MVP The aim of the Round 2 Experiment was to be able to test an ERIS chatbot MVP. The MVP was written using Chatfuel (http://chatfuel.com/), a chatbot builder. Two other bot builders were considered – Botsify and Wit.ai – but Chatfuel was eventually chosen as it had the best combination of elements that met the design considerations and required features of the experiment (Table 2).
  • 25. 25 Figure 10: ERIS Chatbot Demonstration on Facebook Messenger VIDEO REDACTED
  • 26. 26 Figure 11: ERIS Chatbot – Chatfuel Demonstration VIDEO REDACTED
  • 27. 27 A key factor in deciding which chatbot builder to use was the messaging platform through which users could access and interact with the chatbot. Offering a chatbot through an owned PubComms SG asset was not possible as PubComms does not have any SG-specific intranet or website. A quick review of the chatbot builders available in the market show that most of them design their services to be hosted on a few key messaging platforms (Levinson, 2016; BotStory, 2016): Facebook Messenger, Slack, Skype, Telegram and Kik. As Facebook Messenger was the platformlikely to be most used among PubComms SG employees on their mobile phones, it was one of the criteria used to narrow the list of chatbot builders. Given the above, an additional question was added to the interview guide on interviewees’ thoughts on accessing a HR chatbot via Facebook Messenger. In adapting the Twine prototype to Chatfuel, these additional features were added to the MVP: • Include a ‘home’ button or its equivalent • Provide a null response • Allow users to type in their own queries / open- ended questions • More efficient user / conversational flows, i.e. less steps to get to the required information • Reduce wordiness and use more graphics • Layer AI to interpret users’ requests • Introduce product features gradually (Dropbox) Features recommended in the literature review that were not included: • Integration with existing business systems • Securing information security and privacy of information This was not required as the MVP was designed to be a standalone prototype (so it could be built quickly) and thus not needing additional information security measures. Appendix 6 includes source files for the two video demonstrations of the ERIS chatbot MVP you see in Figure 10 and 11. One shows the user view when interacting with the ERIS chatbot via Facebook Messenger, and the other is a demonstration of the architecture of the ERIS chatbot on Chatfuel. *To access the ERIS chatbot MVP on Chatfuel (backend view), please email the author at amy.kong@hyperisland.sg for an access link (valid for 24 hours). To interact with the ERIS chatbot, please use the Facebook Messenger app and search for @PubCommsERIS (case-sensitive) or scan the Messenger Code (Figure 12.) Results: Validation and More Interview results continued to validate the hypothesised benefits of chatbots to improve the employee experience were validated. In fact, the usefulness rating of chatbots in improving the employee experience rose from an average of 7.6 in Round 1 to 7.9 in Round 2. As this group of interviewees did not interact with the Twine prototype, there was no opportunity to investigate if this increase in rating was due to the switch in interaction from a desktop web browser to Facebook Messenger. Figure 12: ERIS Chatbot Messenger Code To interact with ERIS: 1. Launch the Facebook Messenger App 2. From Home, tap the icon 3. Tap your picture at the top of the page 4. Tap Scan Code 5. Scan the image on the left REDACTED REDACTED
  • 28. 28 Interviewees (except for one) were also able to envision at least two other areas of applications of chatbots in HR, and were still able to differentiate when they would use a chatbot and when to approach a HR team member. A more nuanced view emerged on the feedback on avoiding embarrassment. Interviewees said that chatbots wouldhelp when employeesneededto ask: o Trivial questions (same feedback in Round 1) o Questions the employee is supposedto know the answers to o Questions the employee perceives may impact their image/reputation among co-workers Three additional areas of applications of chatbots in HR raised by interviewees in Round 2 include: • Making company announcements • Collecting employee suggestions • FAQ facility for hiring managers, e.g. if the company is still within the ‘S pass’ (a type of employment pass in Singapore) government quota for them to be able to hire foreigners The wordiness of the chatbot continued to be a consistent feedback for improvement in Round 2. Balancing between wordiness and omission of critical information will need to be considered, particularly in HR, where there could be legal or employment implications if important information is not communicated to the employee. Interviewees agreed that a HR chatbot needs to be hosted outside of Facebook, as it is a personal social media platform and HR chatbots are for professional purposes. One employee pointed out that the chatbot should be hosted on a platform that most employees use, and another pointed out that most employees use mobile phones for work, which makes a messenger- based chatbot appropriate. Overall, interviewees in Round 2 seemed to show more enthusiasm over the prototype. There were more expressions of excitement over the ERIS chatbot MVP. Some quotes from the interviewees: “Good that we are using technology in HR. We do it in our business. The fact that we are taking HR online makes it exciting. Now that I know that PubComms SG is considering it, I really want it and am excited for it. I can’t see how it would be a disadvantage.” “This is good. It primes employees for more future forward communications. That’s how it will be in the next few years. It will help people to learn how to apply it in their own (client) brands.” “We need to do things faster now, and something like this helps to get things done faster, especially when teams are so lean now.” “It is overwhelming how smart it is…I like the language used. It is friendly and approachable, and it makes me feel like I can ask any question.” “We don’t have to wait for a response from HR or be dependent on the HR person being around. Currentlywe pass a lot ofmessages around… so that HR doesn’t receive repeated questions. This allows you to find the answer directly yourself.” Overall Findings and Conclusion Problem Statement Exploration and Validation After two rounds of experiments, results indicate that employees think that chatbots have the “This is good. It primes employees for more future forward communications. That’s how it will be in the next few years. It will help people to learn how to apply it in their own (client) brands.”
  • 29. 29 potential to improve the employee experience, and can be applied in various areas in HR within PubComms SG. Specifically, the benefits to employees include: • Instant access and answers • Ability to find information independently • Provide good information • Personable Interviewees found chatbots useful, and were able to envision how chatbots could be used in the HR function, including in: • Information services: company information, policies, processes, and status updates • Training • Information collection and dissemination: broadcasting company announcements and collecting employee suggestions and feedback Interviewees even suggested that chatbots could be used to log IT service requests. Finally, PubComms SG HR chatbots should be hosted on appropriate messaging platforms suitable for work use. Additional Insights • Chatbots seem to be able to play a role in simplifying complex information for employees, when the conversational flow of the chatbot is planned well • Chatbots help employees to avoid feeling embarrassed when asking certain types of questions • Chatbots can provide employees with a sense of anonymity on sensitive issues • In the longer term, chatbots could provide the opportunity to shift perceptions of the HR team from an administrative / transactional role to more of an expert role Limitations of the Experiment Small Sample As the experiment was designed to generate qualitative feedback, it also meant that the sample size is small and may not be representative of PubComms SG, although some of the feedback was validated in Round 2 of the experiment. Lack of Quantitative Metrics As data collected was qualitative, there were no quantitative metrics that were collected. One quantitative metric that could provide valuable insight (and justification to management) if future experiments are conducted, is the time saved in using a chatbot to complete HR tasks, versus the current approach. Confined Area of Enquiry The chatbot was built based on ERIS, which is a very specific HR policy within PubComms SG. In implementing any HR chatbot, the areas of enquiry would need to be broader than one specific policy, and the chatbot was not tested on its ability to filter or direct questions down different topic paths. This may impact the “ease of use” feedback that was seen in this experiment. “We need to do things faster now, and something like this helps to get things done faster, especially when teams are so lean now.” “Good that we are using technology in HR. We do it in our business. The fact that we are taking HR online makes it exciting. Now that I know that PubComms SG is considering it, I really want it and am excited for it. I can’t see how it would be a disadvantage.”
  • 30. 30 Review and Recommendations Yay or Nay? Based on the Literature Review and Experiment, there is significant potentialandvalue in investing in HR chatbots to improve the employee experience and improve employee productivity by reducing time spent on completing HR tasks. The availability of chatbot builders and vendor solutions in the market now also lower the barriers of entry for any investment in building chatbots. How Much Does It Cost? It depends on the complexity of the chatbot. There are also still no industry standards for pricing, and various vendors and platforms have different business models which make it hard to do a cross- vendor or solution comparison of costs. However, to help with understanding the cost implications, Table 3 provides a list of budget items to be considered. It Could Save SGD 125,000 Annually Based on conservative estimates (see Table 4), a chatbot could save the company SGD 125,000 annually, excluding the cost of building the chatbot. This is a conservative estimate as a chatbot’s instant response capability also allows employees to reduce multi-tasking as a result of waiting for responses from the HR team. According to the American Psychological Association (2006), “even brief mental blocks created by shifting between tasks can cost as much as 40 percent of someone's productive time.” If you add in task automation capabilities into the chatbot, cost saved would also increase. Suggested Timeline and Next Steps It may be more appropriate to look at a timeline of 6 months to launch an “official” HR chatbot for the following reasons: Allow Time for Market to Mature The market is still at a relatively early stage of development. This should settle down somewhat in 6 months for PubComms SG to get a clearer perspective of the real possibilities of chatbots. Allow Time for More Vendor Solutions Emerge and Stabilize PubComms SG should also consider off-the shelf solutions, but many of these solutions are US-based, or still in beta versions. Waiting 6 months will allow more vendor solutions to emerge and become more stable. Also, given the huge messaging usage in Asia, Asia- specific or global solutions should emerge in the market soon. Other Decisions Around the Implementation of the Chatbot There are some areas that need to be reviewed and decided, which will influence how the HR chatbot in PubComms SG will be implemented. This includes:
  • 31. 31 Topic/s of the Chatbot While the prototypes were built based on ERIS for specific reasons, ERIS is still a policy that is relevant only to a subset of employees, used infrequently. It may be worth considering a different set of policies to build the chatbot around, for example, top 10 FAQs, leave entitlements, medical benefits, list of panel of doctors etc. Choice of Messaging Platform Due to the confidentiality of HR policies and information, an appropriate messaging platform needs to be selected. At the same time, the choice of platform needs to be easily accessible by employees via their personal mobile devices. Some options that can be considered: • Skype for Business (Lync), the official corporate messaging tool for PubComms SG. This assumes that this feature will be made available to all employees on their mobile phones • Create a mobile-friendly intranet to house company information common to all brands, and also use it to host a chatbot Integration with Publicis Groupe Enterprise HR Software, Data Protection and Security A decision needs to be made on how personalised the chatbot should be. Personalising interactions requires integrating the chatbot with PubComm SG’s existing enterprise HR software to access employee data, and as such issues of data protection and security need to be addressed. On the upside, SAP, the enterprise HR software vendor used by Publicis Groupe, is partnering Kore to integrate chatbot products into its software (Besser, 2016), which resolves the security issues. On the downside, the speed at which chatbots can be deployed in PubComms SG may be slowed, as any use of the enterprise HR software will need to be authorised by the Pubicis Groupe global team, and vetted and handled by ReSources IT (Publicis Groupe IT department). Analytics There is an opportunity for PubComms SG to collect data to monitor and improve the performance of the chatbot, understand employees concerns better and eventually shape the Talent strategy of the company. The type of data to be collected should be considered during the design of the chatbot. This was not a focus for the experiment as the sample size was too small to collect any meaningful data. Employee Privacy and Anonymity Given that some employees have said that they feel that chatbots can provide anonymity, there needs to be discussion on how much,andwhat data should be collected, and what level of anonymity will be provided. Accuracy and Completeness of Information Scenarios where information provided could have a financial impact to the employee or company, or if an employee’s critical employment decision is dependent on a complete understanding of a specific HR policy, need to be discussed to ensure that proper check mechanisms are put in place. No doubt, these situations happens now without the use of chatbots, but they need to be discussed and the handling approach agreed on to ensure that there is a proper resolution mechanism in place. REDACTED REDACTED REDACTED
  • 32. 32 HR Talent HR employees with a new skill set may be needed to drive a technology project such as this. The selected employee may not need to be a technical specialist, but needs to be able to understand the project parameters and work with vendors. In the longer term, a review will be needed of the composition andskillsetoftheHRteamasmoretechnologytools are used in the HR function. Other Recommendations and Considerations While there are many Do-It-Yourself chatbot builders in the market, it may be best for PubComms SG to hire a vendor to develop one if a customised solution is desired. Integrating the chatbot with the enterprise HR software will need considerable technical skill. Additionally, new chatbot features are constantly being added, and unless an employee is dedicatedto focus on building the chatbot, it would be difficult to keep up with the constant upgraded capabilities of the chatbot builders. The Time is Now Even as the suggested timeline is in 6 months’ time, the preparatory work needs to start now, as some of the considerations listed will involve careful consideration and multiple stakeholders that need time to resolve. MVPs can still be tested with a curated employee group of early adopters so that learnings are accumulated and can be baked into the “official” chatbot when ready. Acknowledgements My journey at Hyper Island would not have been possible without the advice and constant moral support from friends and family. I would like to specially thank the following who have contributed directly to my AWBP journey: Dan Spencer Chief Talent Officer, Publicis Communications, Singapore and Australia / New Zealand Leong Yeng Wai Student, Hyper Island Singapore Benjamin Koe Product Lead, Dentsu Aegis Global Data Innovation Centre Narendra Nag Chief Digital Officer, Xynteo Lim Chwen Yiing and Lim Han Boon Best Buddies Jonathan Briggs Co-Founder and Singapore Academic Director, Hyper Island Jarrod Howe Operations Director, Hyper Island Singapore
  • 33. 33 Reflections This journey leading up to the completion of this Advanced Work Based Project (AWBP) has been quite an interesting and enlightening one. These are some of the learnings and experiences that have shaped this AWBP: Complexity of Chatbots and The Landscape Much of the literature talks about how easy it is to create chatbots. It is true, but it is also false. It is true that many of the tools now in the market, particularly the chatbot builders, are relatively easy to use. However, before getting to the pointofbuilding a chatbot,company decision makers need to first understand if they are making the right choice in investments, which means understanding the landscape, and that is where it gets complex. When I first started this project, I had no idea where to start, as I have no coding or IT specialist experience and my experience with technology is purely from an end-user perspective. As I started the research, it seemed like falling down a never-ending rabbit hole. Just to understand chatbot literature that is regularly peppered with jargon, you must know how it fits vis-à-vis AI, machine learning, NLP, NLU, Natural Language Generation. What’s the difference between a chatbot and a conversational agent? What is the difference between a conversational agent and a dialog system? That’s just understanding chatbots andits possible functions.To create the chatbot, you need to understand the different approaches to building one, then understand the comparative differences across the plethora of tools to choose one to use. And the list goes on. I first worked on wit.ai to start building the chatbot as recommended by chatbot-related articles as being easy to use. But it turned out to be an easy tool for developers to use, and not for non-coders like me. So, I had to abandon the work I started in wit.ai and re-start the search for a chatbot builder. For a HR teamthat may not have a large IT departmentto rely on for advice, or even a for CEO who is interested in investing in one, the landscape can be hard to decipher. Jargon Everywhere The proliferate use of jargon is only part of the problem. The other part of the problem is that at this stage of market development, terminology is not clearly defined or differentiated, or writers have a poor understanding of chatbot technology. Sometimes the same terms mean different things, or different terms mean the same thing. Some articles refer to AI as different from machine learning. Some articles talk about chatbot services while others talk about chatbot platforms when they mean the same thing. As such, I tried to be as clear as possible in this report on the different terms and definitions, and explain the chatbot landscape as simply as I could, based on my understanding. As this report is meant to be shared with PubComms SG, hopefully this will help provide clarity for the client. Easy for Whom? It is easy to use a chatbot builder, but it may not be simple to build a chatbot, or at least build a good one. As I was using wit.ai, I was not able to understand how the conversation with the user would flow (that would help me evaluate its effectiveness), as the tool does not help you to do so. This is the same with other chatbot builders that I explored. Adding on to that stress was the lack of time to conduct the experiment,which was how I then researched and found Twine. Even as I was using Twine, I realised that you need an understanding of conditional logic. Upon consulting my ex- colleague who has developer experience, I also realised that you need to plan the conversational flow not in sequential steps, but in blocks of information, and then create paths or flows to the informational blocks. REDACTED
  • 34. 34 Reflections (cont’d) In short, anybody can build a chatbot, if that anybody understands how to plan conversational flows, understands conditional logic etc. Along the way, you’ll encounter other terms like ‘rails’, ‘conversational trees’, ‘dialog trees’ and so on, and down the rabbit hole you go again. Nevertheless, it is true that the barrier to entry for building chatbots are now far lower than before. Drinking From a Firehose Chatbots are such a hot topic now, that news articles and blog posts are generated almost every day. I found it difficult to keep track of the emerging articles and evaluate them for inclusion in this paper. It was also a challenge to select a balance of commercial vs academic sources. Firstly, academic journals do not necessarily reflect the current state of business discussion around the topic, and secondly, they tend to be very technical. There was significant temptation to use data primarily from commercial articles as they provide quick and short summaries. I have tried to refer to academic sources in this paper for key definitions, terms, concepts and key milestones and discussions, and kept the references to commercial articles confined to areas where the academic journals may not touch upon. Iteration vs Documentation When using Chatfuel, changes to any part of the chatbot is reflected ‘live’. There is also no option for version downloads or control. As such, during the experimentation phase, I found it difficult to keep track of the changes that I made to the chatbot. In the longer term, it also raises questions on how to restore chatbots should there be any changes that need to be reversed. When is Customer Feedback Valid? Part of the iteration process is seeking qualitative feedback from customers. One of the questions that I kept asking myself was, “at what point is customer feedback considered valid or an outlier?”. Ultimately, I made the decision based on what I felt made sense and was in line with the overall objective of the chatbot, but it seemed very subjective to me. If given the opportunity, I would ask current Lean Startup practitioners how they make such decisions, particularly with qualitative testing with small sample sizes Challenge in Integrating with Enterprise HR Software I anticipate that there will be challenges in getting this chatbot project running in PubComms SG if backend integration with the enterprise HR software is needed. Such requests will need to be approved by the Global Publicis Groupe team, and actioned by ReSources IT, which is the internal IT shared services function. This affects the speed at which backend functions in PubComms SG can capitalise on emerging opportunities. PubComms SG may be better off creating a generic chatbot now, then present the case study to the Global team for endorsement, to increase the speed of implementation. REDACTED
  • 35. 35 References Ady, M. (2016). How to Build a Chatbot Your Users Will Love. [online] VentureBeat. Available at: http://venturebeat.com/2016/08/17/how-to-build-a-chatbot-your-users-will-love/ [Accessed 3 Jan. 2017]. American Psychological Association. (2006). Multitasking: Switching costs. [online] Available at: http://www.apa.org/research/action/multitask.aspx [Accessed 5 Jan. 2017]. Aspect, (2016). 2016 Aspect Consumer Experience Index. [online] Phoenix. Available at: https://www.aspect.com/globalassets/2016-aspect-consumer-experience-index- survey_index-results-final.pdf [Accessed 1 Jan. 2017]. Aspect.com. (n.d.). Aspect Mila: The Agent’s Personal Assistant. [online] Available at: https://www.aspect.com/solutions/workforce-optimization/aspect-mila [Accessed 2 Jan. 2017]. Ball, P. (2015). The truth about the Turing Test. [online] Bbc.com. Available at: http://www.bbc.com/future/story/20150724-the-problem-with-the-turing-test [Accessed 31 Dec. 2016]. Batalion, A. (2016). “Bot” is the Wrong Name.. and Why People Who Think They are Silly are Wrong. [online] Medium. Available at: https://medium.com/lightspeed-venture- partners/bot-is-the-wrong-name-and-why-people-who-think-they-are-silly-are-wrong-dc0c0b76ae18#.y4x0znitc [Accessed 1 Jan. 2017]. Bayerque, N. (2016). A Short History of Chatbots and Artificial Intelligence. [online] VentureBeat. Available at: http://venturebeat.com/2016/08/15/a-short-history-of-chatbots-and- artificial-intelligence/ [Accessed 1 Jan. 2017]. Besser, L. (2016). Kore bots for SAP empower every employee to delegate like a boss. [online] Ideas.sap.com. Available at: https://ideas.sap.com/D32372 [Accessed 4 Jan. 2017]. Bhaduri, A. (2016). The Digital Tsunami: HR. [online] Abhijit Bhaduri's Official Website. Available at: http://www.abhijitbhaduri.com/index.php/2016/08/the-digital-tsunami-hr/ [Accessed 2 Jan. 2017]. Blank, S. (2013). Why the Lean Start-Up Changes Everything. [online] Harvard Business Review. Available at: https://hbr.org/2013/05/why-the-lean-start-up-changes-everything [Accessed 22 Dec. 2016]. Blank, S. (2015). Why Build, Measure, Learn – isn’t just throwing things against the wall to see if they work – the Minimal Viable Product. [online] Steve Blank. Available at: https://steveblank.com/2015/05/06/build-measure-learn-throw-things-against-the-wall-and-see-if-they-work/ [Accessed 25 Dec. 2016]. BotStory. (2016). Top messaging platforms supporting bots. [online] Available at: http://botstory.co/top-platforms-support-bots-chatbots/ [Accessed 28 Dec. 2016]. Boulton, C. (2016). From tacos to HR, chatbots make it personal. [online] CIO. Available at: http://www.cio.com/article/3063051/instant-messaging/from-tacos-to-hr-chatbots- make-it-personal.html [Accessed 2 Jan. 2017]. Business Insider. (2016). Messaging Apps are Now Bigger than Social Networks. [online] Available at: http://www.businessinsider.com/the-messaging-app-report-2015- 11?IR=T&r=US&IR=T [Accessed 1 Jan. 2017].
  • 36. 36 Chowdhury, G. (2005). Natural language processing. Annual Review of Information Science and Technology, 37(1), pp.51-89. Christensen, C., Hall, T., Dillon, K. and Duncan, D. (2016). Know Your Customers’ “Jobs to Be Done”. [online] Harvard Business Review. Available at: https://hbr.org/2016/09/know- your-customers-jobs-to-be-done [Accessed 26 Dec. 2016]. Dan, A. (2016). Consultants Are Eating The Agencies' Three-Martini Lunch. [online] Forbes.com. Available at: http://www.forbes.com/sites/avidan/2016/04/25/consultants-are- eating-the-agencies-three-martini-lunch/#5ba472b63aba [Accessed 30 Dec. 2016]. Deutscher, M. (2016). FirstJob’s Mya is the latest chatbot that aims to automate recruiting. [online] SiliconANGLE. Available at: http://siliconangle.com/blog/2016/07/11/firstjobs- mya-is-the-latest-chatbot-that-aims-to-automate-recruiting/ [Accessed 2 Jan. 2017]. Dishman, L. (2016). This Chatbot Can Make Sure Your Resume Won't End Up In A Black Hole. [online] Fast Company. Available at: https://www.fastcompany.com/3061677/the- future-of-work/the-chatbot-who-can-make-sure-youll-never-get-radio-silence-after-applyin [Accessed 2 Jan. 2017]. Dunn, J. (2016). We put Siri, Alexa, Google Assistant, and Cortana through a marathon of tests to see who’s winning the virtual assistant race — here’s what we found - Business Insider. [online] Business Insider. Available at: http://www.businessinsider.sg/siri-vs-google-assistant-cortana-alexa-2016-11/?r=US&IR=T#rfBHqO3D8eSOZMdp.97 [Accessed 2 Jan. 2017]. Elliott, S. (2002). Advertising's Big Four: It's Their World Now. [online] Nytimes.com. Available at: http://www.nytimes.com/2002/03/31/business/advertising-s-big-four-it-s-their- world-now.html?pagewanted=all&src=pm [Accessed 20 Dec. 2016]. Ember, S. (2016). Ad Agencies Need Young Talent. Cue the Beanbag Chairs.. [online] Nytimes.com. Available at: http://www.nytimes.com/2016/04/19/business/media/ad-agencies- need-young-talent-cue-the-bean-bag-chairs.html?_r=0 [Accessed 31 Dec. 2016]. Epstein, R., Roberts, G. and Beber, G. (2008). Parsing the turing test. 1st ed. New York: Springer, pp.181-210. Farmer, M. (2015). Madison Avenue Manslaughter. 1st ed. New York: LID Publishing Ltd. Fromowitz, M. (2016). Commoditization: The biggest threat facing ad agencies today. [online] Campaign Asia. Available at: http://www.campaignasia.com/article/commoditization- the-biggest-threat-facing-ad-agencies-today/425997 [Accessed 30 Dec. 2016]. Futurism. (n.d.). The History of Chatbots [INFOGRAPHIC]. [online] Available at: http://futurism.com/images/the-history-of-chatbots-infographic/ [Accessed 1 Jan. 2017]. Gartner.com. (2015). Gartner Says Weak Mobile Customer Service Is Harming Customer Engagement. [online] Available at: http://www.gartner.com/newsroom/id/2956618 [Accessed 1 Jan. 2017]. Grayson, A. and North, G. (2016). How the Advertising Industry is Wasting Talent and What We Can Do About It. [online] Medium. Available at: https://medium.com/@andygrayson/how-the-advertising-industry-is-wasting-talent-and-what-we-can-do-about-it-4824411b9006#.8h9lmf6e3 [Accessed 31 Dec. 2016]. Greenfield, R. (2016). Chatbots Are Your Newest, Dumbest Co-Workers. [online] Bloomberg.com. Available at: https://www.bloomberg.com/news/articles/2016-05-05/chatbots-are- your-newest-dumbest-co-workers [Accessed 2 Jan. 2017].
  • 37. 37 Griffin, A. (2014). Computer becomes first to pass Turing Test in artificial intelligence. [online] The Independent. Available at: http://www.independent.co.uk/life-style/gadgets-and- tech/computer-becomes-first-to-pass-turing-test-in-artificial-intelligence-milestone-but-academics-warn-9508370.html [Accessed 31 Dec. 2016]. Hobson, N. (2016). Are chatbots in the workplace the entry point to cognitive personal assistants? | NevilleHobson.com. [online] NevilleHobson.com. Available at: http://www.nevillehobson.com/2016/05/10/chatbots-workplace-cognitive/ [Accessed 1 Jan. 2017]. Howard, T. (2016). The Guide To Designing A Magical Chatbot Experience. [online] Chatbots Magazine. Available at: https://chatbotsmagazine.com/the-guide-to-designing-a- magical-chatbot-experience-part-1-efbf32444448#.wyex0oqer [Accessed 3 Jan. 2017]. Init.ai Decoded. (2016). The Product Designer’s Guide to Conversational Commerce. [online] Available at: https://blog.init.ai/the-product-designers-guide-to-conversational- commerce-cbe466753add#.gn5sjy8bk [Accessed 27 Dec. 2016]. Jain, A. (2016). The Flow Framework — How to build a kickass UX for your Chat-bot? (Part — II). [online] ChatterOn. Available at: https://blog.chatteron.io/how-to-build-a-kickass-ux- for-your-chat-bot-part-ii-the-flow-framework-811355905249#.6dlq6d3qw [Accessed 3 Jan. 2017]. Johnson, B. (2015). State of the Agency Market: What You Need to Know. [online] Adage.com. Available at: http://adage.com/article/digital/datacenter-agency-report-2015- charts/298214/ [Accessed 30 Dec. 2016]. Knowledge@Wharton. (2016). The Rise of the Chatbots: Is It Time to Embrace Them? - Knowledge@Wharton. [online] Available at: http://knowledge.wharton.upenn.edu/article/rise-chatbots-time-embrace/ [Accessed 31 Dec. 2016]. Kojouharov, S. (2016). Ultimate Guide to Leveraging NLP & Machine Learning for your Chatbot. [online] Chatbot’s Life. Available at: https://chatbotslife.com/ultimate-guide-to- leveraging-nlp-machine-learning-for-you-chatbot-531ff2dd870c#.zaqk0o35t [Accessed 2 Jan. 2017]. Lamprecht, E. (2016). The Difference Between UX and UI Design-A Layman’s Guide. [online] Blog.careerfoundry.com. Available at: http://blog.careerfoundry.com/ui-design/the- difference-between-ux-and-ui-design-a-laymans-guide/ [Accessed 4 Jan. 2017]. Levinson, R. (2016). The tools every bot creator must know. [online] Chatbots Magazine. Available at: https://chatbotsmagazine.com/the-tools-every-bot-creator-must-know- c0e9dd685094#.v6h0lbonf [Accessed 28 Dec. 2016]. Lo, M. (2014). UI, UX: Who Does What? A Designer's Guide To The Tech Industry. [online] Co.Design. Available at: https://www.fastcodesign.com/3032719/ui-ux-who-does-what-a- designers-guide-to-the-tech-industry [Accessed 4 Jan. 2017]. Loebner, H. (n.d.a). In Response. [online] Loebner.net. Available at: http://loebner.net/Prizef/In-response.html [Accessed 31 Dec. 2016]. Loebner.net. (n.d.b). Home Page of the Loebner Prize in Artificial Intelligence. [online] Available at: http://www.loebner.net/Prizef/loebner-prize.html [Accessed 31 Dec. 2016]. Maass, D. (2014). Answers and Questions About Military, Law Enforcement, and Intelligence Agency Chatbots. [online] Electronic Frontier Foundation. Available at: https://www.eff.org/deeplinks/2014/04/answers-questions-about-military-law-enforcement-and-intelligence-agency-chatbots [Accessed 2 Jan. 2017]. Marcus, D. (2016). Messenger Platform at F8. [online] Newsroom.fb.com. Available at: http://newsroom.fb.com/news/2016/04/messenger-platform-at-f8/ [Accessed 1 Jan. 2017].
  • 38. 38 Marr, B. (2016). What Is The Difference Between Artificial Intelligence And Machine Learning?. [online] Forbes.com. Available at: http://www.forbes.com/sites/bernardmarr/2016/12/06/what-is-the-difference-between-artificial-intelligence-and-machine-learning/3/#657340f11a30 [Accessed 1 Jan. 2017]. Marshall, J. and Alpert, L. (2016). Publishers Take On Ad-Agency Roles With Branded Content. [online] WSJ. Available at: http://www.wsj.com/articles/publishers-take-on-ad- agency-roles-with-branded-content-1481457605 [Accessed 30 Dec. 2016]. Maurya, A. (2012). Running Lean: Iterate from a Plan A to a Plan that Works. 2nd ed. Sebastopol, CA: O'Reilly. McNeale, M. and Newyear, D. (2013). Introducing Chatbots in Libraries. Library Technology Reports, 49(8), pp.5 - 10. Medium. (2016). 聊天機器人市場版圖 — Chatbots Landscape. [online] Available at: https://medium.com/@compose.ai/%E8%81%8A%E5%A4%A9%E6%A9%9F%E5%99%A8%E4%BA%BA%E5%B8%82%E5%A0%B4%E7%89%88%E5%9C%96-chatbots- landscape-f89302206875#.wk2de23dy [Accessed 2 Jan. 2017]. Meeker, M. (2016). 2016 Internet Trends Report. [online] Kpcb.com. Available at: http://www.kpcb.com/internet-trends [Accessed 1 Jan. 2017]. Mindshare, Goldsmiths University of London, (2016). Humanity in the Machine. [online] Available at: http://www.mindshareworld.com/sites/default/files/MINDSHARE_HUDDLE_HUMANITY_MACHINE_2016_0.pdf [Accessed 1 Jan. 2017]. Morrison, M. (2016). Sprint Names Exec Creative Director to Run New In-House Agency. [online] Adage.com. Available at: http://adage.com/article/agency-news/sprint-staffs-house- agency/304045/ [Accessed 30 Dec. 2016]. Mumbrella Asia. (2016). Lou Dela Pena handed broader role leading Publicis Communications in Singapore - Mumbrella Asia. [online] Available at: http://www.mumbrella.asia/2016/01/lou-dela-pena-handed-broader-role-leading-publicis-communications-in-singapore/ [Accessed 20 Dec. 2016]. myclever Agency, (2016). Chatbots: A Consumer Research Study. [online] London. Available at: http://www.slideshare.net/mycleveragency?utm_campaign=profiletracking&utm_medium=sssite&utm_source=ssslideview [Accessed 1 Jan. 2017]. Newman, J. (2016). How The New, Improved Chatbots Rewrite 50 Years Of Bot History. [online] Fast Company. Available at: https://www.fastcompany.com/3059439/why-the-new- chatbot-invasion-is-so-different-from-its-predecessors [Accessed 31 Dec. 2016]. Newton, C. (2016). Can AI fix education? We asked Bill Gates. [online] The Verge. Available at: http://www.theverge.com/2016/4/25/11492102/bill-gates-interview-education- software-artificial-intelligence [Accessed 2 Jan. 2017]. Nextit.com. (n.d.). SGT STAR helps potential recruits learn about Army life. [online] Available at: http://www.nextit.com/work/army.php [Accessed 2 Jan. 2017]. Nickerson, R. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), pp.175-220. Oetting, J. (2015). The Biggest Threats to the Agency Model. [online] Blog.hubspot.com. Available at: https://blog.hubspot.com/agency/threats-agency- model#sm.000q8a2qc1192eo6ypt159w7mslve [Accessed 30 Dec. 2016].
  • 39. 39 Olson, P. (2016). Duolingo's New Chat Bots Can Teach You French, German And Spanish. [online] Forbes.com. Available at: http://www.forbes.com/sites/parmyolson/2016/10/07/duolingo-chat-bots-artificial-intelligence/#2d80c93766e5 [Accessed 2 Jan. 2017]. Ovchinnikova, E. (2012). Integration of world knowledge for natural language understanding. 1st ed. Amsterdam: Atlantis Press, p.15. Pandey, R. (2016). Global ad spend: Trends for 2016 and 2017. [online] Marketing Interactive. Available at: http://www.marketing-interactive.com/global-ad-spend-trends-2016- 2017/ [Accessed 30 Dec. 2016]. Pearce, K. (2013). Ask Ivy. [online] Jobs@Intel Blog. Available at: http://blogs.intel.com/jobs/2013/05/16/ask-ivy/ [Accessed 2 Jan. 2017]. Perez-Marin, D. and Pascual-Nieto, I. (2011). Conversational Agents and Natural Language Interaction. 1st ed. Hershey, PA: Information Science Reference, pp.1-22, 107-127, 358- 378. Publicis Groupe, (2016a). Publicis Groupe Announces Important Nominations and its Transformation Plan. [online] Available at: http://livredor.publicis.com/notesInternes/2015/021215_EN_PublicisGroupe_Transformation_DEF.pdf [Accessed 20 Dec. 2016]. Publicis Groupe, (2016b). Third Quarter 2016 Revenue. [online] Paris: Publicis Groupe. Available at: http://www.publicisgroupe.com/media/display/id/8072.pdf [Accessed 31 Dec. 2016]. Publicis Groupe, (2016c). 2015 Registration Document - Annual Financial Report. [online] Paris: Publicis Groupe, pp.90 - 95. Available at: http://www.publicisgroupe.com/documents/document-de-reference-2015-en.pdf [Accessed 31 Dec. 2016]. Richman, D. (2016). Microsoft Azure offers ‘bots as a service’ as demand rises for automated online interactions. [online] GeekWire. Available at: http://www.geekwire.com/2016/microsoft-azure-offers-bots-service-demand-rises-automated-online-interactions/ [Accessed 5 Jan. 2017]. Riel, J. (2016). How Chatbots Can Help With Learning. [online] College of Education | University of Illinois Chicago. Available at: https://education.uic.edu/academics- admissions/student-life/how-chatbots-can-help-learning [Accessed 2 Jan. 2017]. Ries, E. (2011). The Lean Startup. 1st ed. London: Penguin Group, pp.56-78. Rosenberg, S. (2016). New Messenger Platform Features: Link Ads to Messenger, Enhanced Mobile Websites, Payments and More. [online] Facebook for Developers. Available at: https://developers.facebook.com/blog/post/2016/09/12/new-messenger-features-payments-ads-enhanced-mobile-websites/ [Accessed 1 Jan. 2017]. Roxburgh, H. (2016). Why big consultancies buy design agencies. [online] Campaign Asia. Available at: http://www.campaignasia.com/article/why-big-consultancies-buy-design- agencies/407973 [Accessed 30 Dec. 2016]. Sas.com. (n.d.). Machine Learning: What it is and why it matters. [online] Available at: http://www.sas.com/en_us/insights/analytics/machine-learning.html [Accessed 1 Jan. 2017]. Schaefer, M. (2015). 6 Reasons Marketing Is Moving In-House. [online] Harvard Business Review. Available at: https://hbr.org/2015/07/6-reasons-marketing-is-moving-in-house [Accessed 30 Dec. 2016]. Schlicht, M. (2016). The Complete Beginner’s Guide To Chatbots. [online] Chatbots Magazine. Available at: https://chatbotsmagazine.com/the-complete-beginner-s-guide-to- chatbots-8280b7b906ca#.dxwvm1po5 [Accessed 31 Dec. 2016].