Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How well do Voice Assistants answer Consumer Health Questions?

Presented at the International Clinical Librarians' Conference in October 2019.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

How well do Voice Assistants answer Consumer Health Questions?

  1. 1. ‘Alexa, are vaccines safe?’ Evaluating voice assistant responses to consumer health questions Emily Alagha, MSIS, AHIP - Clinical Information Specialist & Data Management Librarian - Georgetown University Medical Center Rachel Helbing, MLIS, MS, AHIP - Director of Library Services for Health Sciences - University of Houston
  2. 2. Objectives 1. Background 2. Purpose 3. Data collection 4. Results 5. Clinical implications
  3. 3. Voice Assistants
  4. 4. Siri Google Assistant Alexa
  5. 5. Online Misinformation
  6. 6. Vaccination rates
  7. 7. Outbreaks
  8. 8. Online Misinformation Vaccination rates Outbreaks
  9. 9. Quality & accuracy Siri Google Assistant Alexa vaccine safety & use
  10. 10. Smart speaker brand owned ©2019MintelGroupLtd.AllRightsReserved. ConfidentialtoMintel. Read on mintel.com Source: Base: Fieldwork: 554 internet users aged 18+ who own a smart speaker March 2019 Lightspeed/Mintel What brands of smart speakers do you own? Please select all that apply. 72% 30% 14% 6% 5% 1% 1% 0% 25% 50% 75% 100% Amazon Google Apple JBL Sonos I don't know Other, please specify 14
  11. 11. Selecting Test Questions The Centers for Disease Control (CDC) immunization FAQ page and the CDC infant immunization FAQ page
  12. 12. AnswerThePublic.com search for ‘vaccines’
  13. 13. Test Questions 54 total
  14. 14. Data Collection 1. Set up iPads 2. Asked & recorded answers 3. Scored with rubric 4. Calculated results
  15. 15. Siri 5.16 5.10 0.98 Google Assistant Alexa Mean Scores
  16. 16. Voice Assistant Alexa Google Assistant Siri VA provided same response to both researchers 62.96% 70.37% 68.52% VA understood question and provided answer 25% 94% 94% Average spoken words per answer 29 21 13 Most cited source Wikipedia.org CDC.gov CDC.gov
  17. 17. Limitations • Data collected in 2018; may not reflect recent algorithm updates • Scoring rubric has not been assessed for reliability and may need future refinements • May not reflect real use cases that incorporate tailored search results • Overall scores for all assistants may have been lower if prioritizing audio only output
  18. 18. Implications: Web Search
  19. 19. Implications: Third Party Apps
  20. 20. Implications: Third Party Apps
  21. 21. Conclusions • Siri and Google provided evidence-based links; little audio output • Alexa was worst at natural language comprehension; high audio output • Many information providers are entering voice information provision via traditional websites and third-party apps • Information professionals should push for better usability and increased transparency about health information partnerships
  22. 22. References 1. https://about.ads.microsoft.com/en-us/insights/2019-voice-report 2. https://pixabay.com/photos/voice-control-voice-commands-2598422/ 3. Brand icons from https://www.flaticon.com • https://www.flaticon.com/authors/simpleicon (Google) • https://www.freepik.com (Apple & Amazon) 4. https://www.pewresearch.org/fact-tank/2013/02/01/majority-of-adults-look-online-for-health- information/ 5. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0194811 6. https://www.ncbi.nlm.nih.gov/pubmed/26974260 7. https://www.nature.com/articles/d41586-018-07034-4 8. https://www.washingtonpost.com/national/health-science/percentage-of-young-us-children- who-dont-receive-any-vaccines-has-quadrupled-since-2001/2018/10/11/4a9cca98-cd0d-11e8- 920f-dd52e1ae4570_story.html?utm_term=.f3d142106107
  23. 23. References 9. https://www.cdc.gov/measles/cases-outbreaks.html 10. https://9to5google.com/2019/04/29/google-assistant-siri-market-share/ 11. https://www.healthcareitnews.com/news/uchealth-launches-alexa-based-virtual-assistant- voice-enabled-patient-engagement 12. https://www.theguardian.com/media/2019/aug/27/bbc-to-launch-alexa-rival-that-will-grasp- regional-accents 13. https://www.theguardian.com/society/2019/jul/10/nhs-teams-up-with-amazon-to-bring-alexa- to-patients 14. https://www.wired.com/story/amazon-alexa-search-for-the-one-perfect-answer/ 15. https://earthtoplanet.com/the-future-of-medicine/

    Be the first to comment

Presented at the International Clinical Librarians' Conference in October 2019.

Views

Total views

156

On Slideshare

0

From embeds

0

Number of embeds

15

Actions

Downloads

0

Shares

0

Comments

0

Likes

0

×