SlideShare a Scribd company logo
1 of 53
Download to read offline
Chatbots for Brand Representation in
Comparison with Traditional Websites
Johannes Kühnel, BSc
May 28, 2020
Institute of Interactive Systems and Data Science
Graz University of Technology, Austria
Table of contents
1. Introduction
2. Proof of Concept
3. User Study
4. Conclusion
1
Intro
What are Chatbots?
Chatbots are computer programs that interact with users us-
ing natural languages. (Shawar and Atwell 2007)
Usually chatbots ...
• provide a service (e.g. bookings)
• use conversational interfaces
• feature simple or more complex AI
2
“Bots are the new apps”
Satya Nadella, Microso t CEO
2
Motivation
Messaging Platform Monthly Active Users
WhatsApp 1,500
Facebook Messenger 1,300
Weixin / Wechat 1,083
Table 1: Monthly active users of the top 3 messaging platforms in millions.
Sources: Hootsuite and We Are Social (2019) and Nadella (2016) and https://telegram.org/blog/200-million (visited on
2019-02-13).
3
Related Work
Exploratory study by Beriault-Poirier, Tep, and Sénécal (2018)
• comparison of websites and chatbots of 3 brands
• participants perform 1 task per brand and platform
• Keypoints: websites offered better user experience, positive
emotions with chatbot
4
Related Work
Exploratory study by Beriault-Poirier, Tep, and Sénécal (2018)
• comparison of websites and chatbots of 3 brands
• participants perform 1 task per brand and platform
• Keypoints: websites offered better user experience, positive
emotions with chatbot
Case study by Shawar and Atwell (2015)
• Frequently Asked Questions (FAQ)
• chatbot vs search engine
• Keypoints: more relevant answers, higher preference
4
Proof of Concept
Theodore, a Company Chatbot
A chatbot — named Theodore — to represent Graz based so tware
developer CodeFlügel was created.
5
Theodore, a Company Chatbot
A chatbot — named Theodore — to represent Graz based so tware
developer CodeFlügel was created.
Goal: chatbot capable of representing and informing about the
company (like the existing website)
5
Theodore, a Company Chatbot
A chatbot — named Theodore — to represent Graz based so tware
developer CodeFlügel was created.
Goal: chatbot capable of representing and informing about the
company (like the existing website)
Theodore should
• reproduce most of the website’s features
• run on Facebook Messenger and a custom webchat
• re-use existing Application Programming Interfaces (APIs)
5
Dialog Design
Chatbot has to provide information about
• the company itself
• products & services
• contact information
• vacancies
• newsletter subscription
• social media & blog posts
6
Dialog Design
Chatbot has to provide information about
• the company itself
• products & services
• contact information
• vacancies
• newsletter subscription
• social media & blog posts
Additional requirements include
• help function & fallback mechanism
• informal language
• aware of being a bot
6
Technologies
Backend
Languages & Frameworks: Node.js with Express.js and Socket.IO
APIs: Facebook, Dialogflow, Mailchimp, Wordpress 1
Webchat
Languages & Frameworks: Angular (using TypeScript) with Socket.IO
APIs: Backend 2
1connection via REST API / HTTP requests
2connection via WebSockets
7
Message Format
Facebook Messenger’s message format (JSON) used for both
Messenger and webchat.
Support for various components:
• Generic Template
• List
• Button
• Media
• Quick Replies
8
Natural Language Understanding
Natural Language Understanding (NLU): process of “understanding”
natural language in computer science
Intents: the meaning or purpose of the user input
Parameters (entities): terms tied to the intents (e.g. context, amount,
time etc.)
9
Natural Language Understanding
Natural Language Understanding (NLU): process of “understanding”
natural language in computer science
Intents: the meaning or purpose of the user input
Parameters (entities): terms tied to the intents (e.g. context, amount,
time etc.)
Figure 1: Dialogflow Basics — Intents
Source: https://cloud.google.com/dialogflow/docs/basics
9
Natural Language Understanding cont.
• initial design was framework agnostic
• looked at several services/frameworks for NLU
• Dialogflow (Google), Wit.ai (Facebook), LUIS (Microso t), Watson
Assistant (IBM), Amazon Lex
10
Natural Language Understanding cont.
• initial design was framework agnostic
• looked at several services/frameworks for NLU
• Dialogflow (Google), Wit.ai (Facebook), LUIS (Microso t), Watson
Assistant (IBM), Amazon Lex
• final implementation only for Dialogflow
• scope of thesis / PoC
• differences in framework/service concepts
• pricing
10
Bot Frameworks
Evaluated bot development frameworks:
• Microso t Bot Framework
• Botkit
• Botmaster.ai
• Botpress
• BotMan
11
Bot Frameworks
Evaluated bot development frameworks:
• Microso t Bot Framework
• Botkit
• Botmaster.ai
• Botpress
• BotMan
Disadvantages include:
• reliance on third party services, higher latencies
• only one platform per instance
• slow development
• limited content handling
• poor platform support
11
Bot Frameworks
Evaluated bot development frameworks:
• Microso t Bot Framework
• Botkit
• Botmaster.ai
• Botpress
• BotMan
Disadvantages include:
• reliance on third party services, higher latencies
• only one platform per instance
• slow development
• limited content handling
• poor platform support
Due to these shortcomings, a custom implementation was chosen.
11
Architecture
Figure 2: Request – Response Architecture
12
UI Components
Figure 3: Blog post components in the webchat and Facebook Messenger.
13
UI Components cont.
Figure 4: Quick Replies in the webchat and Facebook Messenger.
14
UI Components cont.
Figure 5: Generic Template components showing products in the webchat
and Facebook Messenger.
15
User Study
User Study
• comparison of chatbot and website
• identify advantages and weaknesses
• selection based on relevance to the company
• 3 groups
• (potential) clients
• potential employees
• blog readers and everyone else
• 20 participants (at least 5 per group)
16
User Study cont.
• pre- & post-test questionnaires
• users perform 9 typical tasks
• e.g. “Find out where CodeFlügel’s office is located and how to call
them.”
• each with the chatbot and website
• interview a terwards
Figure 6: Test Setup
17
Results cont.
Figure 7: Average time per task
18
Results
Figure 8: Average total time and average time for tasks
19
Results cont.
Figure 9: What users say about the company’s chatbot and website
20
Results cont.
Figure 10: Which platform was more appealing to the users
21
Results cont.
Figure 11: 95% would use more chatbots
22
Conclusion
Conclusion
• positive feedback about the chatbot
• train for whole sentences and keywords
• chatbot more entertaining
• high acceptance rate
• chatbot was faster (specific information)
23
Conclusion
• positive feedback about the chatbot
• train for whole sentences and keywords
• chatbot more entertaining
• high acceptance rate
• chatbot was faster (specific information)
• menus offer quick navigation
• minor issues with intent matching
• exploration better with website
23
Future Work
• improve intent matching
• broader audience / more participants
• different chatbots and personas
• brand perception
• conversion rates and other Key Performance Indicators (KPIs)
24
Live Demo
URL: https://theodore.kuehnel.co.at
24
Questions?
24
Website
Figure 12: Screenshot of the company homepage, taken on 2018-11-04.
Webchat cont.
Figure 13: List, contact and typing indicator components in the webchat.
Facebook Messenger cont.
Figure 14: List, contact and typing indicator components in Facebook
Messenger.
Message Format - Details
{
"message" : {
"attachment" : {
"type" : "template" ,
"payload" : {
"template_type" : "generic" ,
"elements" : [
{
"title" : "<TITLE_TEXT>" ,
"image_url" : "<IMAGE_URL_TO_DISPLAY>" ,
"subtitle" : "<SUBTITLE_TEXT>" ,
"default_action" : {
"type" : "web_url" ,
"url" : "<DEFAULT_URL_TO_OPEN>" ,
"messenger_extensions" : <TRUE | FALSE > ,
"webview_height_ratio" : "<COMPACT | TALL | FULL>"
} ,
"buttons" : [ < BUTTON_OBJECT > , . . . ]
} ,
. . .
]
}
}
}
}
Listing 1: A generic template JSON message in Facebook Messenger’s
format.
Task List
1. Find out what CodeFlügel does or which services they provide.
2. Find out where CodeFlügel’s office is located and their phone
number.
3. Find out which companies CodeFlügel has already implemented
projects for.
4. Find and open the latest blog entry.
5. Sign up for the newsletter with the e-mail address
<firstname>.<surname>@codefluegel.com.
6. Find out if and which jobs are currently available.
7. Find a way to try Augmented Reality (AR) for yourself.
8. Find at least one Augmented Reality (AR) project created by
CodeFlügel.
9. Find out what Augmented Reality (AR) actually is.
Participant Details
• age from 20–33, averaging 27.7
• 60% with bachelor degree
• 45% studying (bachelor or master’s program)
• 70% with technical background (mostly IT)
• mostly male
Results cont.
Figure 15: What users expect of a company website and chatbot
Results cont.
Figure 16: The chart shows how the users communicated with the chatbot
Results cont.
Figure 17: Preferred type of speech of the chatbot
Results cont.
Figure 18: Total time needed to complete the tasks (per User)
References i
Beriault-Poirier, Amélie, Sandrine Prom Tep, and Sylvain Sénécal (Oct.
2018). “Putting Chatbots to the Test: Does the User Experience
Score Higher with Chatbots Than Websites?” In: Human Systems
Engineering and Design. Springer International Publishing,
pp. 204–212. isbn: 978-3-030-02053-8. doi:
10.1007/978-3-030-02053-8_32.
Hootsuite and We Are Social (2019). Digital 2019. Global Digital
Overview. Tech. rep. Mindbowser. url:
https://datareportal.com/reports/digital-2019-
global-digital-overview (visited on 02/04/2019).
Nadella, Satya (Mar. 25, 2016). Build 2016. Keynote Presentation.
Microso t. url: https://channel9.msdn.com/Events/
Build/2016/KEY01#time=1h41m11s (visited on 02/13/2019).
References ii
Shawar, Bayan Abu and Eric Atwell (2007). “Chatbots: Are They Really
Useful?” In: LDV-Forum 22.1, pp. 29–49. url:
https://jlcl.org/content/2-allissues/20-Heft1-
2007/Bayan_Abu-Shawar_and_Eric_Atwell.pdf (visited on
01/24/2019).
– (Sept. 2015). “A chatbot as a Question Answering Tool”. In: 2015
International Conference on Advances in So tware, Control and
Mechanical Engineering. 2015 International Conference on
Advances in So tware, Control and Mechanical Engineering, pp. 1–6.
isbn: 978-93-84422-37-0. doi: 10.17758/UR.U0915120.

More Related Content

Similar to Chatbots for Brand Representation in Comparison with Traditional Websites

To Bot or Not: How Bots can Support Collaboration in Software Engineering (I...
To Bot or Not:  How Bots can Support Collaboration in Software Engineering (I...To Bot or Not:  How Bots can Support Collaboration in Software Engineering (I...
To Bot or Not: How Bots can Support Collaboration in Software Engineering (I...
Margaret-Anne Storey
 

Similar to Chatbots for Brand Representation in Comparison with Traditional Websites (20)

Dynamics 365 Saturday Amsterdam 02/2018 - Dynamics 365 and chatbots
Dynamics 365 Saturday Amsterdam 02/2018 - Dynamics 365 and chatbotsDynamics 365 Saturday Amsterdam 02/2018 - Dynamics 365 and chatbots
Dynamics 365 Saturday Amsterdam 02/2018 - Dynamics 365 and chatbots
 
IRJET - A Study on Building a Web based Chatbot from Scratch
IRJET - A Study on Building a Web based Chatbot from ScratchIRJET - A Study on Building a Web based Chatbot from Scratch
IRJET - A Study on Building a Web based Chatbot from Scratch
 
Student information chatbot final report
Student information chatbot  final report Student information chatbot  final report
Student information chatbot final report
 
Using Chatbots in Extension Programming
Using Chatbots in Extension ProgrammingUsing Chatbots in Extension Programming
Using Chatbots in Extension Programming
 
Build an Intelligent Bot (Node.js)
Build an Intelligent Bot (Node.js)Build an Intelligent Bot (Node.js)
Build an Intelligent Bot (Node.js)
 
Behind the buzzwords: using chatbots & AI for everyday wins!
Behind the buzzwords: using chatbots & AI for everyday wins!Behind the buzzwords: using chatbots & AI for everyday wins!
Behind the buzzwords: using chatbots & AI for everyday wins!
 
Chatbot development workshop with the Microsoft Bot Framework
Chatbot development workshop with the Microsoft Bot FrameworkChatbot development workshop with the Microsoft Bot Framework
Chatbot development workshop with the Microsoft Bot Framework
 
IRJET - A Web-based College Enquiry Chatbot using .Net and Dataset
IRJET - A Web-based College Enquiry Chatbot using .Net and DatasetIRJET - A Web-based College Enquiry Chatbot using .Net and Dataset
IRJET - A Web-based College Enquiry Chatbot using .Net and Dataset
 
Lessons learned from building a commercial bot development platform
Lessons learned from building a commercial bot development platformLessons learned from building a commercial bot development platform
Lessons learned from building a commercial bot development platform
 
Chatbot Technology
Chatbot TechnologyChatbot Technology
Chatbot Technology
 
Furniture Online Ordering using Chatbot
Furniture Online Ordering using ChatbotFurniture Online Ordering using Chatbot
Furniture Online Ordering using Chatbot
 
Global Azure2021 Verona.pptx
Global Azure2021 Verona.pptxGlobal Azure2021 Verona.pptx
Global Azure2021 Verona.pptx
 
Practical Microsoft Bot Framework for Office 365 developers
Practical Microsoft Bot Framework for Office 365 developersPractical Microsoft Bot Framework for Office 365 developers
Practical Microsoft Bot Framework for Office 365 developers
 
14 Most Powerful Platforms to Build a Chatbot.pdf
14 Most Powerful Platforms to Build a Chatbot.pdf14 Most Powerful Platforms to Build a Chatbot.pdf
14 Most Powerful Platforms to Build a Chatbot.pdf
 
14 Most Powerful Platforms to Build a Chatbot.pdf
14 Most Powerful Platforms to Build a Chatbot.pdf14 Most Powerful Platforms to Build a Chatbot.pdf
14 Most Powerful Platforms to Build a Chatbot.pdf
 
CHATBOT PPT-2.pptx
CHATBOT PPT-2.pptxCHATBOT PPT-2.pptx
CHATBOT PPT-2.pptx
 
Chatbots
ChatbotsChatbots
Chatbots
 
To Bot or Not: How Bots can Support Collaboration in Software Engineering (I...
To Bot or Not:  How Bots can Support Collaboration in Software Engineering (I...To Bot or Not:  How Bots can Support Collaboration in Software Engineering (I...
To Bot or Not: How Bots can Support Collaboration in Software Engineering (I...
 
Serverless chatbot: from idea to production at blazing speed
Serverless chatbot: from idea to production at blazing speedServerless chatbot: from idea to production at blazing speed
Serverless chatbot: from idea to production at blazing speed
 
Tweet Tracking App Design Document
Tweet Tracking App Design DocumentTweet Tracking App Design Document
Tweet Tracking App Design Document
 

More from Educational Technology

More from Educational Technology (20)

The use of programming tasks in interactive videos to increase learning effec...
The use of programming tasks in interactive videos to increase learning effec...The use of programming tasks in interactive videos to increase learning effec...
The use of programming tasks in interactive videos to increase learning effec...
 
Analysis of students' behavior watching iMooX courses with interactive elements
Analysis of students' behavior watching iMooX courses with interactive elementsAnalysis of students' behavior watching iMooX courses with interactive elements
Analysis of students' behavior watching iMooX courses with interactive elements
 
Portability of Mobile Applications
Portability of Mobile ApplicationsPortability of Mobile Applications
Portability of Mobile Applications
 
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...
Erhebung von Lernaktivitäten in einem Pop-Up-Makerspace mit einer technischen...
 
Mixed Reality im Distance Learning in der Hochschullehre
Mixed Reality im Distance Learning in der HochschullehreMixed Reality im Distance Learning in der Hochschullehre
Mixed Reality im Distance Learning in der Hochschullehre
 
Development of a WCAG theme for a learning management system
Development of a WCAG theme for a learning management systemDevelopment of a WCAG theme for a learning management system
Development of a WCAG theme for a learning management system
 
Math trainer as a chatbot via system(push) messages for Android
Math trainer as a chatbot via system(push) messages for AndroidMath trainer as a chatbot via system(push) messages for Android
Math trainer as a chatbot via system(push) messages for Android
 
Empirical Analysis of Automated Editing of Raw Learning Video Footage
Empirical Analysis of Automated Editing of Raw Learning Video FootageEmpirical Analysis of Automated Editing of Raw Learning Video Footage
Empirical Analysis of Automated Editing of Raw Learning Video Footage
 
Fächerintegrativer Unterricht am Beispiel der Leichtathletik
Fächerintegrativer Unterricht am Beispiel der LeichtathletikFächerintegrativer Unterricht am Beispiel der Leichtathletik
Fächerintegrativer Unterricht am Beispiel der Leichtathletik
 
DENKEN UND TECHNIK Über manipulative Auswirkungen von Internettechnologien
DENKEN UND TECHNIK Über manipulative Auswirkungen von InternettechnologienDENKEN UND TECHNIK Über manipulative Auswirkungen von Internettechnologien
DENKEN UND TECHNIK Über manipulative Auswirkungen von Internettechnologien
 
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...
Empfehlungen für den Unterricht im Fach Informatik für Menschen mit Autismus-...
 
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...
Entwicklung eines Online-Kurses für digitale Kompetenzen für Studienanfänger:...
 
School Start Screening Tool
School Start Screening ToolSchool Start Screening Tool
School Start Screening Tool
 
Development of a mobile French language learning platform
Development of a mobile French language learning platformDevelopment of a mobile French language learning platform
Development of a mobile French language learning platform
 
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...
Learning Analytics and Spelling Acquisition in German - the Path to Indivdual...
 
Learning Analytics and MOOCs
Learning Analytics and MOOCsLearning Analytics and MOOCs
Learning Analytics and MOOCs
 
Fächerintegrativer Unterricht am Beispiel des Lernroboters Thymio
Fächerintegrativer Unterricht am Beispiel des Lernroboters ThymioFächerintegrativer Unterricht am Beispiel des Lernroboters Thymio
Fächerintegrativer Unterricht am Beispiel des Lernroboters Thymio
 
Einsatz von Mixed Reality im Klassenzimmer
Einsatz von Mixed Reality im KlassenzimmerEinsatz von Mixed Reality im Klassenzimmer
Einsatz von Mixed Reality im Klassenzimmer
 
Development of a learning diary for a MOOC platform
Development of a learning diary for a MOOC platformDevelopment of a learning diary for a MOOC platform
Development of a learning diary for a MOOC platform
 
Potential of Bots for Encylclopedias
Potential of Bots for EncylclopediasPotential of Bots for Encylclopedias
Potential of Bots for Encylclopedias
 

Recently uploaded

The basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxThe basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptx
heathfieldcps1
 
會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文
會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文
會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文
中 央社
 

Recently uploaded (20)

Matatag-Curriculum and the 21st Century Skills Presentation.pptx
Matatag-Curriculum and the 21st Century Skills Presentation.pptxMatatag-Curriculum and the 21st Century Skills Presentation.pptx
Matatag-Curriculum and the 21st Century Skills Presentation.pptx
 
Application of Matrices in real life. Presentation on application of matrices
Application of Matrices in real life. Presentation on application of matricesApplication of Matrices in real life. Presentation on application of matrices
Application of Matrices in real life. Presentation on application of matrices
 
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdf
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdfTelling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdf
Telling Your Story_ Simple Steps to Build Your Nonprofit's Brand Webinar.pdf
 
philosophy and it's principles based on the life
philosophy and it's principles based on the lifephilosophy and it's principles based on the life
philosophy and it's principles based on the life
 
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General QuizPragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
 
Behavioral-sciences-dr-mowadat rana (1).pdf
Behavioral-sciences-dr-mowadat rana (1).pdfBehavioral-sciences-dr-mowadat rana (1).pdf
Behavioral-sciences-dr-mowadat rana (1).pdf
 
Incoming and Outgoing Shipments in 2 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 2 STEPS Using Odoo 17Incoming and Outgoing Shipments in 2 STEPS Using Odoo 17
Incoming and Outgoing Shipments in 2 STEPS Using Odoo 17
 
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
 
An Overview of the Odoo 17 Discuss App.pptx
An Overview of the Odoo 17 Discuss App.pptxAn Overview of the Odoo 17 Discuss App.pptx
An Overview of the Odoo 17 Discuss App.pptx
 
The basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxThe basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptx
 
Removal Strategy _ FEFO _ Working with Perishable Products in Odoo 17
Removal Strategy _ FEFO _ Working with Perishable Products in Odoo 17Removal Strategy _ FEFO _ Working with Perishable Products in Odoo 17
Removal Strategy _ FEFO _ Working with Perishable Products in Odoo 17
 
Exploring Gemini AI and Integration with MuleSoft | MuleSoft Mysore Meetup #45
Exploring Gemini AI and Integration with MuleSoft | MuleSoft Mysore Meetup #45Exploring Gemini AI and Integration with MuleSoft | MuleSoft Mysore Meetup #45
Exploring Gemini AI and Integration with MuleSoft | MuleSoft Mysore Meetup #45
 
“O BEIJO” EM ARTE .
“O BEIJO” EM ARTE                       .“O BEIJO” EM ARTE                       .
“O BEIJO” EM ARTE .
 
Essential Safety precautions during monsoon season
Essential Safety precautions during monsoon seasonEssential Safety precautions during monsoon season
Essential Safety precautions during monsoon season
 
Operations Management - Book1.p - Dr. Abdulfatah A. Salem
Operations Management - Book1.p  - Dr. Abdulfatah A. SalemOperations Management - Book1.p  - Dr. Abdulfatah A. Salem
Operations Management - Book1.p - Dr. Abdulfatah A. Salem
 
Morse OER Some Benefits and Challenges.pptx
Morse OER Some Benefits and Challenges.pptxMorse OER Some Benefits and Challenges.pptx
Morse OER Some Benefits and Challenges.pptx
 
size separation d pharm 1st year pharmaceutics
size separation d pharm 1st year pharmaceuticssize separation d pharm 1st year pharmaceutics
size separation d pharm 1st year pharmaceutics
 
會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文
會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文
會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文會考英文
 
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...
TỔNG HỢP HƠN 100 ĐỀ THI THỬ TỐT NGHIỆP THPT VẬT LÝ 2024 - TỪ CÁC TRƯỜNG, TRƯ...
 
[GDSC YCCE] Build with AI Online Presentation
[GDSC YCCE] Build with AI Online Presentation[GDSC YCCE] Build with AI Online Presentation
[GDSC YCCE] Build with AI Online Presentation
 

Chatbots for Brand Representation in Comparison with Traditional Websites

  • 1. Chatbots for Brand Representation in Comparison with Traditional Websites Johannes Kühnel, BSc May 28, 2020 Institute of Interactive Systems and Data Science Graz University of Technology, Austria
  • 2. Table of contents 1. Introduction 2. Proof of Concept 3. User Study 4. Conclusion 1
  • 4. What are Chatbots? Chatbots are computer programs that interact with users us- ing natural languages. (Shawar and Atwell 2007) Usually chatbots ... • provide a service (e.g. bookings) • use conversational interfaces • feature simple or more complex AI 2
  • 5. “Bots are the new apps” Satya Nadella, Microso t CEO 2
  • 6. Motivation Messaging Platform Monthly Active Users WhatsApp 1,500 Facebook Messenger 1,300 Weixin / Wechat 1,083 Table 1: Monthly active users of the top 3 messaging platforms in millions. Sources: Hootsuite and We Are Social (2019) and Nadella (2016) and https://telegram.org/blog/200-million (visited on 2019-02-13). 3
  • 7. Related Work Exploratory study by Beriault-Poirier, Tep, and Sénécal (2018) • comparison of websites and chatbots of 3 brands • participants perform 1 task per brand and platform • Keypoints: websites offered better user experience, positive emotions with chatbot 4
  • 8. Related Work Exploratory study by Beriault-Poirier, Tep, and Sénécal (2018) • comparison of websites and chatbots of 3 brands • participants perform 1 task per brand and platform • Keypoints: websites offered better user experience, positive emotions with chatbot Case study by Shawar and Atwell (2015) • Frequently Asked Questions (FAQ) • chatbot vs search engine • Keypoints: more relevant answers, higher preference 4
  • 10. Theodore, a Company Chatbot A chatbot — named Theodore — to represent Graz based so tware developer CodeFlügel was created. 5
  • 11. Theodore, a Company Chatbot A chatbot — named Theodore — to represent Graz based so tware developer CodeFlügel was created. Goal: chatbot capable of representing and informing about the company (like the existing website) 5
  • 12. Theodore, a Company Chatbot A chatbot — named Theodore — to represent Graz based so tware developer CodeFlügel was created. Goal: chatbot capable of representing and informing about the company (like the existing website) Theodore should • reproduce most of the website’s features • run on Facebook Messenger and a custom webchat • re-use existing Application Programming Interfaces (APIs) 5
  • 13. Dialog Design Chatbot has to provide information about • the company itself • products & services • contact information • vacancies • newsletter subscription • social media & blog posts 6
  • 14. Dialog Design Chatbot has to provide information about • the company itself • products & services • contact information • vacancies • newsletter subscription • social media & blog posts Additional requirements include • help function & fallback mechanism • informal language • aware of being a bot 6
  • 15. Technologies Backend Languages & Frameworks: Node.js with Express.js and Socket.IO APIs: Facebook, Dialogflow, Mailchimp, Wordpress 1 Webchat Languages & Frameworks: Angular (using TypeScript) with Socket.IO APIs: Backend 2 1connection via REST API / HTTP requests 2connection via WebSockets 7
  • 16. Message Format Facebook Messenger’s message format (JSON) used for both Messenger and webchat. Support for various components: • Generic Template • List • Button • Media • Quick Replies 8
  • 17. Natural Language Understanding Natural Language Understanding (NLU): process of “understanding” natural language in computer science Intents: the meaning or purpose of the user input Parameters (entities): terms tied to the intents (e.g. context, amount, time etc.) 9
  • 18. Natural Language Understanding Natural Language Understanding (NLU): process of “understanding” natural language in computer science Intents: the meaning or purpose of the user input Parameters (entities): terms tied to the intents (e.g. context, amount, time etc.) Figure 1: Dialogflow Basics — Intents Source: https://cloud.google.com/dialogflow/docs/basics 9
  • 19. Natural Language Understanding cont. • initial design was framework agnostic • looked at several services/frameworks for NLU • Dialogflow (Google), Wit.ai (Facebook), LUIS (Microso t), Watson Assistant (IBM), Amazon Lex 10
  • 20. Natural Language Understanding cont. • initial design was framework agnostic • looked at several services/frameworks for NLU • Dialogflow (Google), Wit.ai (Facebook), LUIS (Microso t), Watson Assistant (IBM), Amazon Lex • final implementation only for Dialogflow • scope of thesis / PoC • differences in framework/service concepts • pricing 10
  • 21. Bot Frameworks Evaluated bot development frameworks: • Microso t Bot Framework • Botkit • Botmaster.ai • Botpress • BotMan 11
  • 22. Bot Frameworks Evaluated bot development frameworks: • Microso t Bot Framework • Botkit • Botmaster.ai • Botpress • BotMan Disadvantages include: • reliance on third party services, higher latencies • only one platform per instance • slow development • limited content handling • poor platform support 11
  • 23. Bot Frameworks Evaluated bot development frameworks: • Microso t Bot Framework • Botkit • Botmaster.ai • Botpress • BotMan Disadvantages include: • reliance on third party services, higher latencies • only one platform per instance • slow development • limited content handling • poor platform support Due to these shortcomings, a custom implementation was chosen. 11
  • 24. Architecture Figure 2: Request – Response Architecture 12
  • 25. UI Components Figure 3: Blog post components in the webchat and Facebook Messenger. 13
  • 26. UI Components cont. Figure 4: Quick Replies in the webchat and Facebook Messenger. 14
  • 27. UI Components cont. Figure 5: Generic Template components showing products in the webchat and Facebook Messenger. 15
  • 29. User Study • comparison of chatbot and website • identify advantages and weaknesses • selection based on relevance to the company • 3 groups • (potential) clients • potential employees • blog readers and everyone else • 20 participants (at least 5 per group) 16
  • 30. User Study cont. • pre- & post-test questionnaires • users perform 9 typical tasks • e.g. “Find out where CodeFlügel’s office is located and how to call them.” • each with the chatbot and website • interview a terwards Figure 6: Test Setup 17
  • 31. Results cont. Figure 7: Average time per task 18
  • 32. Results Figure 8: Average total time and average time for tasks 19
  • 33. Results cont. Figure 9: What users say about the company’s chatbot and website 20
  • 34. Results cont. Figure 10: Which platform was more appealing to the users 21
  • 35. Results cont. Figure 11: 95% would use more chatbots 22
  • 37. Conclusion • positive feedback about the chatbot • train for whole sentences and keywords • chatbot more entertaining • high acceptance rate • chatbot was faster (specific information) 23
  • 38. Conclusion • positive feedback about the chatbot • train for whole sentences and keywords • chatbot more entertaining • high acceptance rate • chatbot was faster (specific information) • menus offer quick navigation • minor issues with intent matching • exploration better with website 23
  • 39. Future Work • improve intent matching • broader audience / more participants • different chatbots and personas • brand perception • conversion rates and other Key Performance Indicators (KPIs) 24
  • 42. Website Figure 12: Screenshot of the company homepage, taken on 2018-11-04.
  • 43. Webchat cont. Figure 13: List, contact and typing indicator components in the webchat.
  • 44. Facebook Messenger cont. Figure 14: List, contact and typing indicator components in Facebook Messenger.
  • 45. Message Format - Details { "message" : { "attachment" : { "type" : "template" , "payload" : { "template_type" : "generic" , "elements" : [ { "title" : "<TITLE_TEXT>" , "image_url" : "<IMAGE_URL_TO_DISPLAY>" , "subtitle" : "<SUBTITLE_TEXT>" , "default_action" : { "type" : "web_url" , "url" : "<DEFAULT_URL_TO_OPEN>" , "messenger_extensions" : <TRUE | FALSE > , "webview_height_ratio" : "<COMPACT | TALL | FULL>" } , "buttons" : [ < BUTTON_OBJECT > , . . . ] } , . . . ] } } } } Listing 1: A generic template JSON message in Facebook Messenger’s format.
  • 46. Task List 1. Find out what CodeFlügel does or which services they provide. 2. Find out where CodeFlügel’s office is located and their phone number. 3. Find out which companies CodeFlügel has already implemented projects for. 4. Find and open the latest blog entry. 5. Sign up for the newsletter with the e-mail address <firstname>.<surname>@codefluegel.com. 6. Find out if and which jobs are currently available. 7. Find a way to try Augmented Reality (AR) for yourself. 8. Find at least one Augmented Reality (AR) project created by CodeFlügel. 9. Find out what Augmented Reality (AR) actually is.
  • 47. Participant Details • age from 20–33, averaging 27.7 • 60% with bachelor degree • 45% studying (bachelor or master’s program) • 70% with technical background (mostly IT) • mostly male
  • 48. Results cont. Figure 15: What users expect of a company website and chatbot
  • 49. Results cont. Figure 16: The chart shows how the users communicated with the chatbot
  • 50. Results cont. Figure 17: Preferred type of speech of the chatbot
  • 51. Results cont. Figure 18: Total time needed to complete the tasks (per User)
  • 52. References i Beriault-Poirier, Amélie, Sandrine Prom Tep, and Sylvain Sénécal (Oct. 2018). “Putting Chatbots to the Test: Does the User Experience Score Higher with Chatbots Than Websites?” In: Human Systems Engineering and Design. Springer International Publishing, pp. 204–212. isbn: 978-3-030-02053-8. doi: 10.1007/978-3-030-02053-8_32. Hootsuite and We Are Social (2019). Digital 2019. Global Digital Overview. Tech. rep. Mindbowser. url: https://datareportal.com/reports/digital-2019- global-digital-overview (visited on 02/04/2019). Nadella, Satya (Mar. 25, 2016). Build 2016. Keynote Presentation. Microso t. url: https://channel9.msdn.com/Events/ Build/2016/KEY01#time=1h41m11s (visited on 02/13/2019).
  • 53. References ii Shawar, Bayan Abu and Eric Atwell (2007). “Chatbots: Are They Really Useful?” In: LDV-Forum 22.1, pp. 29–49. url: https://jlcl.org/content/2-allissues/20-Heft1- 2007/Bayan_Abu-Shawar_and_Eric_Atwell.pdf (visited on 01/24/2019). – (Sept. 2015). “A chatbot as a Question Answering Tool”. In: 2015 International Conference on Advances in So tware, Control and Mechanical Engineering. 2015 International Conference on Advances in So tware, Control and Mechanical Engineering, pp. 1–6. isbn: 978-93-84422-37-0. doi: 10.17758/UR.U0915120.