EPANDING THE CONTENT OF AN OUTLINE using notes.pptx
Chatbots for Brand Representation in Comparison with Traditional Websites
1. Chatbots for Brand Representation in
Comparison with Traditional Websites
Johannes Kühnel, BSc
May 28, 2020
Institute of Interactive Systems and Data Science
Graz University of Technology, Austria
2. Table of contents
1. Introduction
2. Proof of Concept
3. User Study
4. Conclusion
1
4. What are Chatbots?
Chatbots are computer programs that interact with users us-
ing natural languages. (Shawar and Atwell 2007)
Usually chatbots ...
• provide a service (e.g. bookings)
• use conversational interfaces
• feature simple or more complex AI
2
5. “Bots are the new apps”
Satya Nadella, Microso t CEO
2
6. Motivation
Messaging Platform Monthly Active Users
WhatsApp 1,500
Facebook Messenger 1,300
Weixin / Wechat 1,083
Table 1: Monthly active users of the top 3 messaging platforms in millions.
Sources: Hootsuite and We Are Social (2019) and Nadella (2016) and https://telegram.org/blog/200-million (visited on
2019-02-13).
3
7. Related Work
Exploratory study by Beriault-Poirier, Tep, and Sénécal (2018)
• comparison of websites and chatbots of 3 brands
• participants perform 1 task per brand and platform
• Keypoints: websites offered better user experience, positive
emotions with chatbot
4
8. Related Work
Exploratory study by Beriault-Poirier, Tep, and Sénécal (2018)
• comparison of websites and chatbots of 3 brands
• participants perform 1 task per brand and platform
• Keypoints: websites offered better user experience, positive
emotions with chatbot
Case study by Shawar and Atwell (2015)
• Frequently Asked Questions (FAQ)
• chatbot vs search engine
• Keypoints: more relevant answers, higher preference
4
10. Theodore, a Company Chatbot
A chatbot — named Theodore — to represent Graz based so tware
developer CodeFlügel was created.
5
11. Theodore, a Company Chatbot
A chatbot — named Theodore — to represent Graz based so tware
developer CodeFlügel was created.
Goal: chatbot capable of representing and informing about the
company (like the existing website)
5
12. Theodore, a Company Chatbot
A chatbot — named Theodore — to represent Graz based so tware
developer CodeFlügel was created.
Goal: chatbot capable of representing and informing about the
company (like the existing website)
Theodore should
• reproduce most of the website’s features
• run on Facebook Messenger and a custom webchat
• re-use existing Application Programming Interfaces (APIs)
5
13. Dialog Design
Chatbot has to provide information about
• the company itself
• products & services
• contact information
• vacancies
• newsletter subscription
• social media & blog posts
6
14. Dialog Design
Chatbot has to provide information about
• the company itself
• products & services
• contact information
• vacancies
• newsletter subscription
• social media & blog posts
Additional requirements include
• help function & fallback mechanism
• informal language
• aware of being a bot
6
15. Technologies
Backend
Languages & Frameworks: Node.js with Express.js and Socket.IO
APIs: Facebook, Dialogflow, Mailchimp, Wordpress 1
Webchat
Languages & Frameworks: Angular (using TypeScript) with Socket.IO
APIs: Backend 2
1connection via REST API / HTTP requests
2connection via WebSockets
7
16. Message Format
Facebook Messenger’s message format (JSON) used for both
Messenger and webchat.
Support for various components:
• Generic Template
• List
• Button
• Media
• Quick Replies
8
17. Natural Language Understanding
Natural Language Understanding (NLU): process of “understanding”
natural language in computer science
Intents: the meaning or purpose of the user input
Parameters (entities): terms tied to the intents (e.g. context, amount,
time etc.)
9
18. Natural Language Understanding
Natural Language Understanding (NLU): process of “understanding”
natural language in computer science
Intents: the meaning or purpose of the user input
Parameters (entities): terms tied to the intents (e.g. context, amount,
time etc.)
Figure 1: Dialogflow Basics — Intents
Source: https://cloud.google.com/dialogflow/docs/basics
9
19. Natural Language Understanding cont.
• initial design was framework agnostic
• looked at several services/frameworks for NLU
• Dialogflow (Google), Wit.ai (Facebook), LUIS (Microso t), Watson
Assistant (IBM), Amazon Lex
10
20. Natural Language Understanding cont.
• initial design was framework agnostic
• looked at several services/frameworks for NLU
• Dialogflow (Google), Wit.ai (Facebook), LUIS (Microso t), Watson
Assistant (IBM), Amazon Lex
• final implementation only for Dialogflow
• scope of thesis / PoC
• differences in framework/service concepts
• pricing
10
22. Bot Frameworks
Evaluated bot development frameworks:
• Microso t Bot Framework
• Botkit
• Botmaster.ai
• Botpress
• BotMan
Disadvantages include:
• reliance on third party services, higher latencies
• only one platform per instance
• slow development
• limited content handling
• poor platform support
11
23. Bot Frameworks
Evaluated bot development frameworks:
• Microso t Bot Framework
• Botkit
• Botmaster.ai
• Botpress
• BotMan
Disadvantages include:
• reliance on third party services, higher latencies
• only one platform per instance
• slow development
• limited content handling
• poor platform support
Due to these shortcomings, a custom implementation was chosen.
11
29. User Study
• comparison of chatbot and website
• identify advantages and weaknesses
• selection based on relevance to the company
• 3 groups
• (potential) clients
• potential employees
• blog readers and everyone else
• 20 participants (at least 5 per group)
16
30. User Study cont.
• pre- & post-test questionnaires
• users perform 9 typical tasks
• e.g. “Find out where CodeFlügel’s office is located and how to call
them.”
• each with the chatbot and website
• interview a terwards
Figure 6: Test Setup
17
37. Conclusion
• positive feedback about the chatbot
• train for whole sentences and keywords
• chatbot more entertaining
• high acceptance rate
• chatbot was faster (specific information)
23
38. Conclusion
• positive feedback about the chatbot
• train for whole sentences and keywords
• chatbot more entertaining
• high acceptance rate
• chatbot was faster (specific information)
• menus offer quick navigation
• minor issues with intent matching
• exploration better with website
23
39. Future Work
• improve intent matching
• broader audience / more participants
• different chatbots and personas
• brand perception
• conversion rates and other Key Performance Indicators (KPIs)
24
46. Task List
1. Find out what CodeFlügel does or which services they provide.
2. Find out where CodeFlügel’s office is located and their phone
number.
3. Find out which companies CodeFlügel has already implemented
projects for.
4. Find and open the latest blog entry.
5. Sign up for the newsletter with the e-mail address
<firstname>.<surname>@codefluegel.com.
6. Find out if and which jobs are currently available.
7. Find a way to try Augmented Reality (AR) for yourself.
8. Find at least one Augmented Reality (AR) project created by
CodeFlügel.
9. Find out what Augmented Reality (AR) actually is.
47. Participant Details
• age from 20–33, averaging 27.7
• 60% with bachelor degree
• 45% studying (bachelor or master’s program)
• 70% with technical background (mostly IT)
• mostly male
52. References i
Beriault-Poirier, Amélie, Sandrine Prom Tep, and Sylvain Sénécal (Oct.
2018). “Putting Chatbots to the Test: Does the User Experience
Score Higher with Chatbots Than Websites?” In: Human Systems
Engineering and Design. Springer International Publishing,
pp. 204–212. isbn: 978-3-030-02053-8. doi:
10.1007/978-3-030-02053-8_32.
Hootsuite and We Are Social (2019). Digital 2019. Global Digital
Overview. Tech. rep. Mindbowser. url:
https://datareportal.com/reports/digital-2019-
global-digital-overview (visited on 02/04/2019).
Nadella, Satya (Mar. 25, 2016). Build 2016. Keynote Presentation.
Microso t. url: https://channel9.msdn.com/Events/
Build/2016/KEY01#time=1h41m11s (visited on 02/13/2019).
53. References ii
Shawar, Bayan Abu and Eric Atwell (2007). “Chatbots: Are They Really
Useful?” In: LDV-Forum 22.1, pp. 29–49. url:
https://jlcl.org/content/2-allissues/20-Heft1-
2007/Bayan_Abu-Shawar_and_Eric_Atwell.pdf (visited on
01/24/2019).
– (Sept. 2015). “A chatbot as a Question Answering Tool”. In: 2015
International Conference on Advances in So tware, Control and
Mechanical Engineering. 2015 International Conference on
Advances in So tware, Control and Mechanical Engineering, pp. 1–6.
isbn: 978-93-84422-37-0. doi: 10.17758/UR.U0915120.