The study compared the user experience of Siri and Alexa across 4 tasks: setting timers/alarms, playing music, asking travel/commute questions, and random questions. Overall, participants were slightly more satisfied with Alexa (3.71 rating vs 3.17 for Siri), but both received the same ease-of-use rating of 3.83. Alexa performed better at playing music but Siri was better at answering travel questions. For random questions, Siri provided more informative answers but both assistants had inconsistencies.

1. 11www.metrostarsystems.com/digital
SIRI VS ALEXAWHICH ONE HAS THE SUPERIOR USER EXPERIENCE?

2. 22www.metrostarsystems.com/digital
INTRODUCTION
Our Human Factors team conducted a
usability study of the user interface (voice
and graphical) of Apple’s Siri and
Amazon’s Alexa.

3. 33www.metrostarsystems.com/digital
Participants
50%
50%

4. 44www.metrostarsystems.com/digital
Previous Experience with Siri & Alexa
83% 17% 33% 67%
Percentage of users who
had used Siri “for many
years” prior to the test
Percentage of users who
had “played around” with
Siri prior to the test
Percentage of users who
had “played around” with
Alexa prior to the test
Percentage of users who
had “never used” Alexa
prior to the test

5. 55www.metrostarsystems.com/digital
When conducting our usability
test, we chose tasks that would
cover different aspects of a virtual
assistant’s functionality. These are
also tasks that the average users
would be likely to ask a virtual
assistant to complete. After each
task, our participants were asked
to rate their experience on a scale
of 1-5, 1 being total failure, 5
being perfect.
TASKS

6. 66www.metrostarsystems.com/digital
Task 1: Alarm & Timer
Ask Siri and Alexa to complete a timer or clock related task
100% 100%
Distribution of participants
rating their experience on
a scale of 1-5 (5 being
perfect)
Percentage of users who
were able to complete the
task without any major
difficulties
Distribution of participants
rating their experience on
a scale of 1-5 (5 being
perfect)
Percentage of users who
were able to complete the
task without any major
difficulties
SIRI ALEXA
0
1
2
3
4
1 2 3 4 5
Experience Satisfaction
0
2
4
6
8
1 2 3 4 5
Experience Satisfaction

7. 77www.metrostarsystems.com/digital
Task 1
Summary
Although all participants were able to complete a timer- or
clock-related task without any major difficulty, Alexa (with
an average rating of 5) performed better than Siri (with an
average rating of 4.17) on the satisfaction rating. These
results are due to Alexa’s ability to set a timer/alarm down
to the second, whereas Siri can only do down to minutes.

8. 88www.metrostarsystems.com/digital
Task 2: Music
Ask Siri and Alexa to perform a music related task, like “Play today’s hits on
Pandora”
16% 100%
Distribution of participants
rating their experience on
a scale of 1-5 (5 being
perfect)
Percentage of users who
were able to complete the
task without any major
difficulties
Distribution of participants
rating their experience on
a scale of 1-5 (5 being
perfect)
Percentage of users who
were able to complete the
task without any major
difficulties
SIRI ALEXA
0
1
2
3
4
5
1 2 3 4 5
Experience Satisfaction
0
2
4
6
8
1 2 3 4 5
Experience Satisfaction

9. 99www.metrostarsystems.com/digital
Task 2
Summary
Alexa was able to understand our participants’ command
to play a certain type of music or music by a specific
artist. Siri was only able to open the music app and had
difficulty playing music, which resulted in low completion
rates. Participants on this task reported an average
satisfaction rate of 4.67 for Alexa, and an average rating
of 2 for Siri.

10. 1010www.metrostarsystems.com/digital
Task 3: Travel & Commute
Ask Siri and Alexa a travel or commute related question like, “What is traffic like right
now?”
100% 50%
Distribution of participants
rating their experience on
a scale of 1-5 (5 being
perfect)
Percentage of users who
were able to complete the
task without any major
difficulties
Distribution of participants
rating their experience on
a scale of 1-5 (5 being
perfect)
Percentage of users who
were able to complete the
task without any major
difficulties
SIRI ALEXA
0
1
2
3
4
5
1 2 3 4 5
Experience Satisfaction
0
0.5
1
1.5
2
2.5
1 2 3 4 5
Experience Satisfaction

11. 1111www.metrostarsystems.com/digital
Task 3
Summary
For this task, Siri performed slightly better than Alexa with
an average rating of 4 (Alexa was 2.28). Alexa requires
users to configure their commute, work, and home
locations, and is only able to provide general traffic
information. With Siri, no configuration is required,
however, many users still had to go through multiple
commands before receiving navigation instructions. Siri
still received a 4 overall because it provided adequate
information. One participant was able to ask Alexa the
“right” question and receive routing instructions.

12. 1212www.metrostarsystems.com/digital
Task 4: Random Question/Conversation
Ask Siri and Alexa anything you want (up to three questions or two minutes,
whichever comes first)
66% 33%
Distribution of participants
rating their experience on
a scale of 1-5 (5 being
perfect)
Percentage of users that
were able to complete the
task without any major
difficulties
Distribution of participants
rating their experience on
a scale of 1-5 (5 being
perfect)
Percentage of users that
were able to complete the
task without any major
difficulties
SIRI ALEXA
0
1
2
3
4
5
1 2 3 4 5
Experience Satisfaction
0
0.5
1
1.5
2
2.5
1 2 3 4 5
Experience Satisfaction

13. 1313www.metrostarsystems.com/digital
Task 4
Summary
Participants asked Siri and Alexa any question they could
think of. Siri performed slightly better than Alexa with an
average satisfaction rating of 3.16 (Alexa’s was 2.67).
Participants noted that Siri was able to provide
informative answers to their questions, while Alexa was
not able to do that. Over 60% of participants thought their
conversation/questions with Siri was successful, while
only 33% of participants felt this way about Alexa.

14. 1414www.metrostarsystems.com/digital
Overall, our participants were slightly
more satisfied with their experience
with Alexa, which received a
satisfaction rating of 3.83 (Siri
received 3.17). When it came to ease-
of-use though, Siri and Alexa received
the same rating (3.83).
OVERALL
IMPRESSIONS

15. 1515www.metrostarsystems.com/digital
How Would You Describe Siri & Alexa?
SIRI
Funny
Helpful
Well-Meaning
Basic
ALEXA
Beta
Newbie
Cool
Competent
Inconsistent
Alexa was the only to receive a negative adjective, which is mostly likely
because Alexa is very good at some tasks (like music). However, Alexa was
not very good (or did not work as expected) at other tasks, like giving
directions or describing traffic conditions.
Easy to Use Cool

16. 1616www.metrostarsystems.com/digital
After aggregating all of the task statistics
and overall satisfaction scores, Siri
received an overall rating of 3.88 and
Alexa, 3.71. These ratings were
consistent with the qualitative data of
the positive adjectives participants used
to describe Siri. Alexa and Siri both
demonstrated strengths, like Alexa’s
music capabilities and Siri’s navigation
capabilities.
Summary

17. 1717www.metrostarsystems.com/digital
About Us
LI LU
User Experience Architect
Li Lu is an User Experience Architect at
MetroStar Systems. She has a passion
for observing and interpreting human
behavior and translating business and
user needs into meaningful and pleasing
user experiences. Her diverse
background include human factors
engineering, user experience design,
and research. She holds a Human
Factors Master's degree from George
Mason University.
From the rough drawings on cave walls to the recap over coffee
about what happened in yesterday's business meeting, humans are
natural storytellers. Every organization, brand and product is built on
a narrative that is told through imagery, copy and multimedia.
Whether it’s the FCC’s promise to provide transparent information to
the American people or AAFMAA’s passion behind providing
premier military insurance, narratives are what connect with your
audience and help you meet business objectives. From copywriting
to graphic design and videography, our technical innovators and
user experience designers create the content for your foundational
narrative.
MetroStar’s Digital Practice

18. 1818www.metrostarsystems.com/digital
Learn more about our Digital
services, which include UI/UX,
Human Factors, and Web & Mobile
Design visit:
www.metrostarsystems.com