This document discusses the role of natural language processing (NLP), conversational AI, and virtual voice assistants in pediatrics. It begins with an introduction to NLP and how it allows computers to understand spoken and written human language. It then discusses several use cases for clinical NLP, including automation of workflows, analytics, prediction, and conversational agents. Examples of chatbots and virtual assistants currently used in healthcare are provided. The document outlines the current state of conversational AI and envisions future directions such as multimodal data fusion to create contextual AI, integration of CAI into physician workflows, and use of hybrid technologies combining CAI with augmented reality and robotics. It concludes that NLP can unlock insights from unstructured data, CAI provides

1. Role of NLP, Conversational AI
&
Virtual Voice Assistants in Pediatrics
JAI NAHAR, MD, MBA
Associate Professor of Pediatrics
George Washington University School of Medicine
Attending, Division of Cardiology
Children’s National Hospital, Washington DC
AIMed Pediatrics
Nov 11th, 2020

2. Disclosures
• None

3. 1. Introductory concepts: NLP
2. Clinical NLP: Application
3. Conversational AI & Virtual Voice Assistants
4. Future Directions
5. Conclusion
Agenda

4. Natural Language Processing (NLP)
• ThisAI methodology allows the
computer to understand spoken as
well as written human language
• NLP = NL Understanding (NLU)+ NL
Generation (NLG)
Intelligence-Based Medicine
1st Edition. August 2020
Artificial Intelligence and Human Cognition in Clinical Medicine and Healthcare
Author: Anthony Chang
Chang A. Analytics and Algorithms, Big Data, Cognitive Computing, and Deep
Learning in Medicine and Health Care. AI Med Ebook; 2017
Language
and
Speech:
Broca’s
and
Wernicke’s
areas

5. NLP: Functions
• Automation: Data Extraction, Entry
• Analytics
• Prediction
• Classification
• Risk Stratification
• Assistance & Coaching (Virtual Assistant)
• Sensing (ASR)
• Expression (NLG)
Source: Stanford NLP Group

6. Clinical NLP: Use cases
MD Work Flow
Augmentation
Care Delivery
Revenue
Optimization
Research
Patient Portal
UX
Conversational
AI
• Automatic Speech recognition
• EHR documentation
• Chart Review
• Data Mining
• Predictive analytics
• Phenotypic Classification
• Risk Scores/stratification
• Clinical decision support
• Targeted Intervention
• Adverse event prevention
• Quality Improvement
• Computer assisted coding
• Automatic preauthorization
• No show prediction
• Clinical trial matching
• Automated Registry reporting
• Analytics
Understanding Portal’s
Medical Terminology
• Conversational agents
(Virtual Assistants, Chabots)
• Ambient Clinical Intelligence
• Voice Biomarker analysis

7. Clinical NLP and Care Delivery : Data Mining
Input Data
Unstructured
Data
EMR/Other
sources
Use of NLP
Structured
Data (Machine
Interpretable)
ML
Analytics
Targeted Early Intervention
Quality Improvement
Data Discovery Advanced Analytics
Phenotypic
Classification
Risk stratification
Clinical
decision
support

8. NLP Application: Patient Safety
https://discoveries.childrenshospital.org/nlp-patient-safety/

9. Voice,Voice Intelligence, Conversational AI
WhyTalk aboutVoice?
History: Human Computer Interaction
GUI
WEB
www
Mobile
VUI
1980s 1990s 2000s Present
Voice:
• Natural mode of
communication
• Convenient
• Scalable
• Rich Content
• Biomarkers
• Metadata

10. Conversational AI
Technology which allows Human Machine interaction through the use of
Natural conversation, utilizing voice user interface and Machine Intelligence
Human Machine
Conversational
AI
Voice
Technology
NLP
ML,
Deep
Learning
Synergistic
Convergence
Conversation
VUI

11. Current State
Conversational AI Ecosystem
Smart
Speakers
Smart
Displays
Smart voice
enabled
devices
Chatbots
Virtual
Assistants
Ambient
Clinical
Intelligence
Conversational
AI
Voice
VUI
IOT
Ambient
Sensors

12. DigitalTriage: Chatbots
Espinoza J, Crown K, Kulkarni O
A Guide to Chatbots for COVID-19 Screening at Pediatric Health Care Facilities
JMIR Public Health Surveill 2020;6(2):e18808
http://publichealth.jmir.org/2020/2/e18808/
Sample chatbot process map. *: Institutional discretion, follow public health
agency guidelines.
Sample Chatbot process map

13. Virtual Assistants
My Children’s Enhanced Recovery after surgery (ERAS)
Boston Children's Hospital
• Patient facing
• Post operative recovery update
• Post Op appointment information
https://www.voice.health/news/my-childrens-enhanced-recovery-after-
surgery-eras-alexa-skill
Task: Information
exchange
Technology: Alexa
Environment:
Patient’s home

14. Embodied Conversational Agents (ECA)
Embodied Conversational Agents
Edited by Justine Cassell, Joseph Sullivan, Scott Prevost and Elizabeth F. Churchill
March 2000
https://mitpress.mit.edu/books/embodied-conversational-agents
Embodied conversational agents are
computer-generated cartoonlike characters
that demonstrate many of the same
properties as humans in face-to-face
conversation, including the ability to
produce and respond to verbal and
nonverbal communication.
Examples of embodied conversational agent embodiments
Embodied Conversational Agents in Clinical Psychology: A Scoping Review
Provoost S, Lau HM, Ruwaard J, Riper H
J Med Internet Res 2017;19(5):e151
URL: https://www.jmir.org/2017/5/e151

15. Future Directions
• Multimodal data fusion Contextual AI
• Smart Physician Friendly EHRs (NLP and CAI)
• Advanced HCI: Children Friendly HumanoidVirtual Assistants
• HybridTechnology: CAI+AR; CAI+Robotics

16. Conclusion: KeyTakeaways
1. NLP: Unlocking Actionable Insights from Unstructured data
2. CAI: Intelligent User Interface
3. UX, Smooth Work Flow integration
4. Data Privacy, Security, is integral to successful implementation

17. Thanks!
Thanks
for your
time