The document discusses using chatbots for learning and training. It outlines several potential uses of chatbots including answering learner questions, providing personalized learning recommendations, reinforcing learning through repetition, facilitating learning dialogues, assessing learner progress, and providing feedback and motivation as a learning coach. It also provides examples of chatbot implementations for answering questions in learning communities, individualized learning recommendations, and reinforcement of learning on mobile devices. Evaluation results show the chatbot was able to answer around 70% of questions with high confidence compared to 42% originally, and user acceptance of the chatbot was generally positive.

1. PUBLIC
Dr. Lars Satow, SAP (@LarsSatow)
Januar, 2019
Learning & Training mit Chatbots
Erfahrungen mit Chatbots in der Weiterbildung

2. 2INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
1. Learner-Support: Beantworten von Fragen; Unterstützung beim Lernen; Erinnern
2. Personalisierung: Lernempfehlungen zugeschnitten auf den persönlichen Bedarf
3. Lern-Verstärkung: Wiederholung von Inhalten; Erinnern an Lernzeiten
4. Trainings-Partner: Lernförderliche Dialoge
5. Assessments: Beurteilung von Lernergebnissen und Lernfortschritten
6. Lern-Coach: Feedback, Bewertung, Motivation
Einsatzszenarien für Digitale Lern-Assistenten

3. Ed the Bot @ Recast.AI: Machine Learning Overview
Intent Matching
Output
Reaction
Lerner
Input
Machine Learning

4. Ed the Bot @ Recast.AI – Architecture Overview
SAP Learning Hub
Learning
Room A
Learning
Room B
Learning
Room C
SAP Cloud Platform
Java
module
• Mapping
• Tracking
SAP Leonardo Recast.AI
Recast
Bot A
Recast
Bot B
Recast Bot
C
Other Apps
and Web
pages
• Intent matching
• Complex skills
• Answer generation
• Machine Learning
• Reporting
• Authoring

5. 5INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Scenario 1: Answer Learner Questions in Learning Communities

6. 6INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Scenario 2: Individualized Learning Recommendations

7. 7INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
1. Learner-Support: Beantworten von Fragen; Unterstützung beim Lernen; Erinnern an Terminen
2. Personalisierung: Lernempfehlungen zugeschnitten auf den persönlichen Bedarf
3. Lern-Verstärkung: Wiederholung von Inhalten; Erinnern an Lernzeiten
4. Trainings-Partner: Lernförderliche Dialoge
5. Assessments: Beurteilung von Lernergebnissen und Lernfortschritten
6. Lern-Coach: Feedback, Bewertung, Motivation
Einsatzszenarien für Digitale Lern-Assistenten

8. 8INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Learning (Course / E-Learning)
Reinforce with app
SAP Certification Exam
Scenario 3: Learning Reinforcement on Mobile Device

9. 9INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Start Screen
Progress
Tracking
Content

10. 10INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Learn

11. 11INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Test

12. 12INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Ask
Beantworten von Fragen
Erinnern an Lernzeiten
Lernförderliche Dialoge

13. Evaluation

14. 14INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Bot Evaluation Jan 2018
Correct; 78%;
Answer provided; 42%
210 User Questions Jan 2018
Source: Brandl, R. (2018). Deployment of Machine Learning to support Tutoring in collaborative Learning Communities. Master thesis PH Heidelberg.

15. 15INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Bot Evaluation: Effect of Machine Learning
Dec 2018 Dec 2017
Questions n = 1000 (last 1000) n = 210 (first 210)
Answers provided n = 698 ~70% n = 88 ~ 42%
Confidence M = 0.97 M = .78
Marked as best answer by
user
n = 36 ~ 9% n = 7 ~ 8%

16. 16INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
Bot Evaluation: Acceptance
1 2 3 4 5 6 7 8 9 10
Interact with Bot
1 2 3 4 5 6 7 8 9 10
Usefulness of bot
Interact with Bot
1 2 3 4 5 6 7 8 9 10
Access Learning Room from Program
1 2 3 4 5 6 7 8 9 10
Ask question in Learning Rooms
1 2 3 4 5 6 7 8 9 10
Go to page, where you can find all content
Interact with Moderators
1 2 3 4 5 6 7 8 9 10
Search and find ABAP Course
Search

17. 17INTERNAL© 2019 SAP SE or an SAP affiliate company. All rights reserved. ǀ
 Free paper (German)
 PPT & Videos
@LarsSatow

18. PUBLIC
Dr. Lars Satow, SAP (@LarsSatow)
Januar, 2019
Learning & Training mit Chatbots
Erfahrungen mit Chatbots in der Weiterbildung