Behaviour-based language-interactive speaking systems (BLISS) - Helmer Strik

1. Radboud University Nijmegen
NWO Data2Person project BLISS:
Behaviour-based Language-Interactive
Speaking Systems
Helmer Strik
NovoLanguage BV
Radboud Univ. Nijmegen
CLS : Centre for Language Studies
CLST : Centre for Language and Speech Technology
AI – Big Data

2. Radboud University Nijmegen
NWO Data2Person project: BLISS
BLISS: Behaviour-based Language-Interactive Speaking Systems
Applicants:
Helmer Strik, Catia Cucchiarini
Consortium:
 CLS, CLST (RU):
Louis ten Bosch, Iris Hendrickx
 HMI (Twente Univ.) Mariët Theune
 ReadSpeaker
Staffan Meij, Esther Judd-Klabbers
 Games for Health Projects
Jurriaan van Rijswijk, Roland Goetgeluk, Rob Tieben
• Health care organisation & clients
http://hstrik.ruhosting.nl/bliss/
CLS all: BLISS - Reading Tutor 2
AI – Big Data

3. Radboud University Nijmegen
Now
Increasing:
 Elderly, with (healthcare) problems, e.g. dementia
• remain at home, self-management
 Big data: text, speech
Games for Health Projects
GfH_EU - BLISS 3
QuestionR and HealthyR – text InterviewR – speech / spoken data

4. Radboud University Nijmegen
NLG
TTS
ASR
NLU
Dial.
Manager
Speech SDS
Speech Client
ClientStandard SDSKnowledge Bases
Domain
Ontology
Info
NLG: Natural Language Generation
TTS: Text-To-Speech
NLU: Natural Language Understanding
ASR: Autom. Speech Recognition
SDS: Spoken Dialog System
GfH_EU - BLISS

5. Radboud University Nijmegen
NLG
TTS
Dialogue
History
Big Data:
Text - Sp.
ASR
NLU
Dial.
Manager
Speech BLISS
Speech Client
Client-- BLISS --Knowledge Bases
Domain
Ontology
IE
Framing
IE: Info. Extraction
NLG: Natural Language Generation
TTS: Text-To-Speech
NLU: Natural Language Understanding
ASR: Autom. Speech Recognition
Goal: Big Data + IE => personalized SDS: BLISS
GfH_EU - BLISS
Happiness model

6. Radboud University Nijmegen
Innovative aspects - Research Questions
 IE on speech (interviews, dialogs, ...) using Deep Neural Nets (DNNs);
combined with IE on texts.
Speech contains much more information than just the words, verbal info.
RQ1. How to extract relevant knowledge about the client’s health,
wellbeing and happiness?
 Big Data  personalized SDS that communicates with clients ...
... to facilitate their self-management of health, wellbeing & happiness.
Happiness and health are related - Happiness Model (Veenhuizen, 2008)
RQ2. How to incorporate all knowledge in a ‘Model of Happiness’?
RQ3. ... and use it to personalize the SDS?
 Large scale use by clients.
Integrated in applications on agenda management that use gamification.
Happiness ~ The ideal schedule - sequence of daily activities.
RQ4. To what extent does the SDS help increase the client’s health,
wellbeing and happiness?
CLS all: BLISS - Reading Tutor 6

7. Radboud University Nijmegen
Now & Future
Increasing:
 Elderly, with problems, e.g. dementia
• remain at home, self-management
 Big data: text, speech & video, wearables
• Speech & video - not only words => monitoring
Games for Health Projects
GfH_EU - BLISS 7
CalendR –ideal day schedule InterviewR – speech / spoken data

8. Radboud University Nijmegen
Thanks for your attention.
Questions ?
For more information see:
 http://hstrik.ruhosting.nl/
 http://hstrik.ruhosting.nl/contact/
 http://hstrik.ruhosting.nl/projects/
• http://hstrik.ruhosting.nl/bliss/
GfH_EU - BLISS 8