2. App Description
The Panoptic Streaming app encompasses the entirety of a user’s
information from all streaming services that they subscribe to. By
using this phone app, individuals with several streaming service
subscriptions are able to keep track of each service’s information in
an organized manner. In addition to being a method of
organization, this app also relieves the user’s stress and mental
load since the app keeps important information on file such as:
passwords, billing information, payment reminders, and payment
method management. Our focus on user experience aims to
relieve the frustrations users have become accustomed to by
creating an app that is intuitive, simple, and pleasing to use.
Overall, our app combines a user’s streaming services into one tidy
package to meet a market need and alleviate the stress associated
with managing all their streaming services’ information on various
platforms.
3. Goals of the Testing
The goals of this test consist of determining whether the user can
easily navigate the system without error or unnecessary
complexities. The goal is that the user will be able to understand
the features of the app on the first run through and be able to
navigate through different parts of the app without confusion. We
will use the System Usability Scale (SUS) to measure how the ease
with which the user can navigate the app and determine whether
our goals for testing were met. This will be discussed in further
detail below.
4. Test Participants
Participant #1
Name: Chari’
Age: 43
Gender: Female
Educational Background:
● Bachelors in Psychology
● Masters in Social Work
Job/Career
Background/Qualifications:
● Licensed Clinical Social
Worker
Technological
Savvy//Experience:
● Moderate
Participant #2
Name: Kyleigh
Age: 24
Gender: Female
Educational Background:
● Pursuing Bachelors in
Earth and Environmental
Science
Job/Career
Background/Qualifications:
● Student
Technological
Savvy//Experience:
● Average
Participant #3
Name: Jason
Age: 50
Gender: Male
Educational Background:
● Bachelors in Education
● Masters in Physical
Education
Job/Career
Background/Qualifications:
● Physical Education
Teacher
Technological
Savvy//Experience:
● Below Average
5. Tasks Being Performed
Users will be asked to complete five tasks through the app. Their ease with
which they complete these five tasks will determine if our goal was met.
The five tasks the users will perform include:
1) Logging into the app
2) Inputting new billing information
3) Setting a payment reminder
4) Setting up auto-pay, and
5) Managing subscriptions (ie. cancelling a subscription).
These five tasks cover the main tasks of the app. If the user can successfully
complete these five tasks then they will be able to navigate through the
whole app.
6. Experimental Design
This experiment was conducted virtually to observe users' behavior
and actions when performing given tasks. To measure how easily
the user can navigate the app, three participants were asked to
perform five tasks on a prototype on their mobile phones. A link to
the prototype was shared with participants on their phones before
the start of the experiment. All participants were told to think aloud
when performing tasks. We recorded their verbal feedback,
physical behavior, and task completion time. We were interested in
how users felt about their experience with the application.
Therefore, after participants completed the five tasks, they were
asked to fill in the System Usability Scale (SUS). Users had ranked
each of the 10 questions above from 1 to 5, based on their level of
agreement. All qualitative and quantitative results were analyzed
to measure the usability of the application and areas of
improvement moving forward in the process.
7. Testing Methods or Processes
Participants
Participants were recruited by
individual requests. Since the
audience of the application is
primarily adults, we reached out to
potential users whose ages ranged
from 24 to 50. Three participants were
recruited and a pre-test
questionnaire was sent to gather
information about the participants'
prior experiences, goals, and
preconceptions regarding the
application, as well as gather
demographic information.
Sessions
Participants were asked to attend a 30-minute
Zoom session with a researcher to observe their
behavior and action on the Panoptic Streaming
app. The sessions took place from November 21 to
23. The participants signed a consent form before
the start of the usability testing. A test script
detailing the procedure and instructions was read
to them after they had signed the consent form.
The researcher took note of the participant's facial
expressions, verbal feedback, and task
completion time. Participants were asked to rank
their experience on the post-test survey after
attempting to complete all of the tasks. The
questions on the post-test survey related to the
ease of use so that we can measure the usability
of the Panoptic Streaming app.
8. Usability Measures & Data Collection
Methods
Usability Measures
System Usability Scale (SUS): The
System Usability Scale measures the
usability of a wide variety of products
and services. Having participants
complete the survey created by the
SUS can give the team some insight
into how easy the Panoptic
Streaming app is to use.
Data Collection Methods
Participants are observed by a team
member and given the list of tasks
that are to be completed. Team
members will take notes of the time
needed to complete the tasks,
participant body language, and
overall user performance. After the
participant has completed all tasks,
they will be asked to complete the
System Usability Scale questionnaire
and the post-test survey developed
by the team.
9. SUS Questionnaire
1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.
Participants are asked to
complete the survey
after interacting with the
Panoptic Streaming app.
They are asked to
indicate on a scale
whether they strongly
agree or disagree with
the statement. This is
how we collect our
quantitative data.
10. SUS Results
Strongly
Disagree
Disagree Neither
Disagree or
Agree
Agree Strongly
Agree
I think that I would like to use this system frequently.
2 1
I found the system unnecessarily complex.
1 2
I thought the system was easy to use.
1 1 1
I think that I would need the support of a technical person to be able to use this
system. 1 1 1
I found the various functions in this system were well integrated.
1 2
I thought there was too much inconsistency in this system.
1 2
I would imagine most people would learn to use this system very quickly.
1 2
I found the system very cumbersome to use.
1 2
I felt very confident using the system.
1 1 1
I needed to learn a lot of things before I could get going with this system.
1 1 1
12. Qualitative Data
Participant 1: Participant found success in completing all tasks. They noted that while they were able to complete them that it
could have been a better experience and some changes could be made to help with some of their concerns. They found the app
to be overly complex with more buttons than they thought were necessary. Additionally, they thought that some categories could
be linked better such as subscriptions, payments, and reminders. They also mentioned wanting the option to turn on notifications
from the screen where they add a new subscription so they don’t need to navigate to a separate section afterwards to add it.
They also mentioned wishing there was a home button because the back buttons over and over were too much if they just
wanted to return to the home screen. They found an issue on the adding user page wherein when they clicked to add information
the ‘X’ in the top right didn’t work to cancel it or exit back to the screen and they had to submit new user information even if they
didn’t fill it out to exit that portion. They found it overall easy to use and really liked the options for push notifications/reminders
and calendar syncing. They had very relaxed body language throughout and didn’t have issues with navigating the app or
providing both quantitative and qualitative data through the SUS Questionnaire and then the verbal comments afterwards.
Participant 2: Participant found success with completing tasks 2, 3, and 5. Participant noted that they enjoyed the payment
method page as it contained all information they would need. Participant vocalized they were confused by the presence of the
billing information and payment method pages and said that they would like it if both pages could be consolidated into one,
participant also noted that the presence of a home screen would be useful for navigation. Participant noted that they found the
adding new service page easy to use. Body language was relaxed throughout, but participant became very noticeably frustrated
during completion of task 4.
Participant 3: Participant was noticeably frustrated throughout most of the test, struggling to complete most tasks in a timely
manner. Participant was vocal about their frustrations stating that they felt that the application was “not made for them”.
Participant performance was poor throughout, finding most success on task 5, participant appeared to be most comfortable with
managing their streaming subscriptions.
13. Executive Summary
● Users found the application unintuitive and confusing at some points,
primarily the process of setting up auto-pay.
● The application was rated higher by users with average to above
average technological experience and lower by users with below
average technological experience.
● Participants reacted positively to features offered by the application
even if they found accessing those features cumbersome.
○ The ability to have payment reminders sync with the users
calendar was viewed especially favorably.
● The application received a 60 on the System Usability Scale.
○ This puts the application at below average for system usability.
● A home button was identified as something that users may find useful.
○ System usability was low, a home button my improve user
experience.
14. Issues & Recommendations
● Some participants cannot navigate to other pages since app is overly complex
○ Implement a persistent home button to increase ease of navigation throughout the
application.
● Participants note there are too many buttons when they try to navigate back to the home
page.
○ Streamline application usage through reduced button options.
● Participants need to take one more step for turning on notifications for a new subscription
since there is no related function on the page now.
○ Add options for notification management.
● Participants found that the ‘X’ in the top right didn’t work to exit back to the screen
when they are on the adding user page.
○ Fix any navigational errors that were found in user testing.
● So far, the average of the time completion on the task 1 is 3.3 seconds,
task 2 is 22.6 seconds, task 3 is 12.3 seconds, task 4 is 25.3 seconds, and
task 5 is 23 seconds. Set up auto-pay seems more difficult to participants.
We will keep an eye on task 4 in the next iteration to see if the new design
reduce the time completion.
