The document outlines a proposed solution to enhance emergency service operations by converting audio calls to text transcripts, identifying key entities, and matching recent similar calls. The solution aims to assist emergency service agents (ESA) both when they are engaged and when they are busy, thereby improving response times and information gathering. The document also details the architecture and phases of implementation, emphasizing future enhancements for accuracy and user interface improvements.

1. Watson Dispatch Manager
Call for Code 2018
Team 1, Dublin, Ireland

2. Solution Outline
 Proposed solution augments Emergency Service Agent (ESA) by
 1--Converting Emergency call's audio stream to a transcript ready for use by ESA
 2--Identify all entities ( nouns of relevance ) in the transcript for populating any fields in
Emergency Dispatch System
 3--Search and provide closely matching emergency calls of recent past with similar entity values
for consideration as duplicates or supplementary information
 4--In case of waiting queue at Emergency Response number, engage user and collect
information for action by ESA.
 Note: Not all of the above is targeted for 31st August, 2018 milestone.

3. Use Cases and User research
 There are two possible use cases for this solution
 Functionality while an ESA is engaged with a caller (Passive case)
 Functionality to interact with caller while no ESA is available to engage. (Active case)
 Examples of why Active case would be relevant
 Example from latest natural disaster: Floods in Kerala India - https://indianexpress.com/article/india/kerala-floods-
rains-rescue-inside-kochis-sos-centre-5312671/
 https://www.sundaypost.com/fp/999-callers-on-hold-for-20-minutes-as-pressure-on-nhs-rises999-emergencylives-at-
risk-as-desperate-callers-are-put-on-hold-for-20-minutesnhs-winter-crisis-ae-wards-under-siege-as-virulent-flu/
 https://globalnews.ca/news/2376143/911-on-hold-how-other-provinces-answer-emergency-calls/
 The team engaged with Emergency dispatchers in Ireland’s dispatch service (999) to understand the value
proposition and the outcome is that target ESA population would be interested in such a solution.

4. Architecture of Solution
IBM CloudApplication Layer
MH
Orchestration
Application
Call Platform
Text To Speech
Emergency Caller
Web UI
Watson Assistant
WatsonOrchestration
Db Storage
HTTPS
(JSON)
ESA
.WAV /
.AIFF
Speech To Text
Natural Language Understanding
Watson Discovery Service
Phase 2

5. Information Flow Phase 1
 For active use case, Agent platform invokes Watson Assistant service using api when all ESA are busy.
 Watson Assistant (WA) initiates conversation and sends initiation message from WA to (Text to Speech)
TTS service and relays audio output to Agent platform.
 Audio input is captured from the Agent platform using broadband sampling and passed on as .wav /
.aiff packet to ‘Speech To Text’ which then provides transcript output.
 Transcript output is used by Watson Assistant to identify intent and entities associated with the incident.
 WA provides closing message to caller (through TTS) with summary of entity values and user interface
shows a match with an existing incident.

6. User Interface Phase 1
Entity values
Location on
map
Transcription

7. Information Flow Phase 2
 For active use case, Agent platform invokes Watson Orchestrator using its api when all ESA are busy.
 WO invokes Watson Assistant (WA) to initiate conversation and sends initiation message from WA to
TTS service and relays audio output to Agent platform.
 Audio input is captured from the Agent platform using broadband sampling and passed on as .wav /
.aiff packet to WO which then relays the output to ‘Speech To Text’ service and obtains transcript
output.
 WO calls Natural Language Understanding service to obtain vital entity values.
 WO provides closing message to caller (through TTS) with summary of entity values.
 WO updates the Watson Discovery Service collection with the latest incident and queries for similar
reported incidents in last ‘n’ minutes using entity values in last callers narration.
 WO compares search result set to query parameters and presents matches and differences on the web
UI for the ESA to review and action.
 WO pushes enriched data from WDS collection to db storage.
 For passive use case, Watson Assistant would not be involved.

8. Phase 1 Goals
 Transcription of Audio
 Basic conversational interaction (welcome and closure)
 Limited entity detection
 Basic comparison
 UI with Transcription, Entity and comparison display

9. Refinement and possibilities (Phase 2 post 31/08)
 Improvement in transcription accuracy with language model and acoustic model.
 Conversational improvement to ask for missing entities and disambiguation
 Improvement in entity detection and relationship detection through Watson Knowledge
Studio
 Improvement in User Interface and comparison of incidents.
 Analytics in UI example: based on location entity by interface to google maps.