Participant Experience 2014 Mobile Phone App Baseline Usability Study
 (October 27-November 5, 2014)

1. Our vision is to be a valued
strategic business partner by
deepening our understanding of
clients needs and behaviors to
deliver timely and actionable
insights and services
For internal use only
Client Insight
Making Vanguard smarter about clients
Participant Experience 2014 Mobile Phone App Baseline Usability Study
(October 27-November 5, 2014)
Usability Intern: Evan Wiener

2. 2
Client Insight
For internal use only - Confidential
Report contents
1. Background, objectives, and methodology
2. List of tasks
3. Task completion statistics and qualitative feedback
4. Key findings
5. Appendix:
• Demographics forthcoming
• Summary of key recommendations
• Task script wording
Legend:
Good Practice
Insight or User Suggestion
Minor Problem. Some users
hesitated or had some problems
Serious problem. Several Users
had problems and some could
not complete the task
Critical Problem. Most users had
problems and could not complete
the task.
Usability Recommendation
Worked well or good practice
Usability issue
Suggestion for improvement
Note:
Because the feedback is based on qualitative data with a limited sample of users,
it cannot be assumed that these sessions identified all the possible issues or that
the issues can be projected to Vanguard.com's participant user base. However,
neither should the issues identified be given less attention, even when the
proportion of users who experienced a particular problem is small, as valuable
insights can be gained from user actions and comments during a usability study.

3. 3
Client Insight
For internal use only - Confidential
Background and approach
Baseline usability study goals:
• Get a high-level review of the app by seeing how well users complete tasks
that expose them to many areas of the experience
• Isolate usability issues and make recommendations
• Identify areas of opportunity for future enhancements
Approach and methodology
• 12 test sessions – October 27 – November 5
• Focused on mobile phones – no tablets, but some “phablets”
• Participants brought their own smartphone devices to usability lab,
leading to significant content variability
• 2 Android phones from the Samsung Galaxy and Galaxy Note series
• 10 iPhones
• 5 cross-over clients

4. 4
Client Insight
For internal use only - Confidential
What was asked
• When and where people use the app
• Gather what people usually do with the app
• Check personal performance overall
• Performance in a specific time period
• View balance of investments
• Make a change to investments
• Exchange $1,000 into a new fund
• View and change paycheck contribution
• View list of funds in plan
• Asked what criteria someone would use to make
choices for a fun
• Find vested balance
• Get advice if unsure about asset
allocation
• Take a loan
• Rollover in from a previous employer
account
• Taking action upon retirement
• Log on / Log off
• Update security settings
Participants were asked to demonstrate what they commonly do with the
Vanguard app, provide demographic information, and run tasks such as:

5. 5
Client Insight
For internal use only - Confidential
Executive Summary
• Loading times for screens optimized for mobile phone devices were as fast as users
expected.
• Retirement-only accounts were generally able to complete tasks and find what they
were looking for.
• Users appreciated the convenient access to information to learn more about how
their retirement plans are performing.
• Users were lost in the experience due to information architecture issues and
confusion between multiple competing menus shown on the screen.
• Users would benefit from help information in context to their actions, especially for
misunderstood financial industry terms and confusing data presentations.
• Participants had problems finding their vested balance.
• Participants were confused by the options presented on Change Investments
screen and were more likely to call for reassurance.
• Users struggled with inconsistency with UI elements causing confusing
navigation patterns and poor implementations for a touchscreen interface

6. 6
Client Insight
For internal use only - Confidential
My Plans Screen – what’s working
• My Plans screen was generally well-
received and met most user needs.
• Expand/collapse UI was effective to get
people to interact with labels and tap to
view content.
• Meets most user expectations of providing
high-level of information for a small screen
experience.
• Getting to high-level information most
people expected to see was not a great
challenge for retirement-only participants.
• Loading speed of content was satisfactory
for 11 of 12 users. Many factors could have
impacted poor loading times for one test.

7. 7
Client Insight
For internal use only - Confidential
My Plans Screen
• Type size:
4 people complained about type size, even
on mobile optimized pages.
One user thought changing iPhone settings
would effect the app.
Another bought a larger device expecting a
larger screen to improve readability.
• Apply type resize features and consider user
expectations of accessibility features in
mobile apps
• One user perceived this to be an ad, which
discouraged scrolling for more.
• Consider a content strategy for variable
content sections.

8. 8
Client Insight
For internal use only - Confidential
My Plans Screen
• One user voiced confusion about a static
graphic looking like an interactive slider.
Establishing expectations of interactive UI
patterns and allowing static content that
looks to be interactive creates significant
confusion and reluctance to tap.
“Where will this thing take me?”
Consider evaluating content presentation to
ensure conflict with user or business needs isn’t
being introduced.
Consider discussing content strategy with outside
partners we provide user access to.
• Auto-scrolling caused disorientation when 3
users returned to the My Plans screen.
Avoid auto-scrolling when possible.

9. 9
Client Insight
For internal use only - Confidential
My Plans Screen
• 2 users tapped Monthly Balance Change
from Overview section and voiced confusion
about by how the balances and dates were
being displayed.
Reconsider the monthly balance change
label or revise data being displayed to
match the mental model of what that means.
Note: Others voiced similar confusion about
how the information is being presented in
context to time, impact on account balance
and related data (see Personal Performance
custom range).

10. 10
Client Insight
For internal use only - Confidential
My Plans Screen
• Performance-minded, numbers-focused users
were looking for numerical Rate of Return.
Offer a Rate of Return to My Plans screen
Performance section
• Two users weren’t sure if Learn more or
View personal performance links would give
them Performance details.
See Personal Performance screen insights
for findings on graph data and terminology.

11. 11
Client Insight
For internal use only - Confidential
My Plans Screen
• Manage my money section labels were
clear to users
NOTE: Other elements further down the
manage money tasks caused confusion
and/or did not match mental model of user
expectations of managing a retirement
account.
• 6 said they would call when it comes to
rolling money in or out of their plan.

12. 12
Client Insight
For internal use only - Confidential
My Plans Screen
• 10 of 12 users went to History, statements &
forms to find information about something
that recently happened in their account.

13. 13
Client Insight
For internal use only - Confidential
Personal Performance – what’s working
• 11 of 12 users found interactive graph to
be valuable.
• All users clearly understood balance over
time and a cumulative presentation of their
retirement money.
• Users who preferred data in a table would
continue looking for it, so users who
preferred a graph were happy to see that
first.
• Prioritize graphs on small screens when
possible.
• Users appreciated the rate of return inside
the date drop-down and asked if that was
what the number next to the year meant.

14. 14
Client Insight
For internal use only - Confidential
Personal Performance – opportunities
• Some plan names caused drop-down
selector to break outside the UI design.
This caused content to move around the
screen when users scrolled, which is
unusual in a mobile app.
• Horizontal scrolling wasn’t an issue for most
users and 2 users turned the device when
they saw the table.
2 users felt the interaction points were too
narrow and persistent arrows were confused
as active UI.
• Users found a table easier to use to
understand a month of performance data.
• 2 users were surprised to see Net
transactions in a performance data table.
Were not clear of context.

15. 15
Client Insight
For internal use only - Confidential
Personal Performance – opportunities
• 2 users were confused about the pop-over
UI and were surprised it couldn’t be hidden.
Consider adding a show/hide UI design
• Swipe gestures lack obviousness. 3 users
accidentally opened the menu when trying to
interact with the table or graph.
• Consider removing swipe gestures that
aren’t obvious.
• 7 of 12 people were doubtful of what Net
transactions line and term meant in context.
Even those who were correct said "I guess
'net transactions' is what I contributed."
Rx: Reconsider visualization of performance
data, separating data into views which help
users understand what is being shown.

16. 16
Client Insight
For internal use only - Confidential
Personal Performance – opportunities
• When looking for data of just July, all users
who chose to view data that in a graph
struggled with understanding the date input
fields for custom range.
Rx: Users expected to see a date picker UI.

17. 17
Client Insight
For internal use only - Confidential
Investments – What’s working
• 11 of 12 users valued the pie charts and
understood what they were saying.
• When asked, 6 people found change
investment button.
• All users tapped fund name to see details
about that particular fund.
• Horizontal scrolling wasn’t an issue for most
users. 2 turned the device when they saw
the table.

18. 18
Client Insight
For internal use only - Confidential
Investments - opportunities
• 2 said they were discouraged to scroll for
more.
1 said they’d prefer a different order of
information
Rx: Consider moving data visualizations to a
higher priority
• If unsure asset mix is right for them, 3 users
said they would call for advice help.
• One user asked what short-term reserves
meant, another issue with terminology.
Rx: Opportunity for contextual help
• 2 commented on persistent arrows on tables
and confused them for actionable UI.

19. 19
Client Insight
For internal use only - Confidential
Investments cont.
• 6 users initially went to
Manage my money when
asked to change
investments.
Could be due to mental
model of making changes or
the visibility of the Change
your investments button.
• Only 1 participant thought to
go to Investments and found
vested balance in the table.
Most were thinking of
Contributions first.

20. 20
Client Insight
For internal use only - Confidential
Investments cont.
• 4 users would call for advice if unsure about
asset allocation being the right mix.
• 3 users voiced some confusion about seeing
so many pie charts.
• When cross-linking drove users to non-
optimized web experiences, all users said
they would prefer a larger screen to evaluate
this information or call for help when
evaluating their investment choices.
• One user felt they wanted advice to have a
bolder presentation to stand out.

21. 21
Client Insight
For internal use only - Confidential
Contributions
• 11 of 12 users liked the position of data
visualization at top of screen.
• All users who had employer contributions
understood context of their contributions and
employer contributions.
• For all relevant users, they all understood the pre-
tax nature of the contributions.
• All users tried tapping the chart to view pop-over
to see more data, possibly due to patterns
established earlier in the experience.

22. 22
Client Insight
For internal use only - Confidential
Contributions
• 11 of 12 users liked the position of data
visualization at top of screen.
• All users who had employer contributions
understood context of their contributions and
employer contributions.
• For all relevant users, they understood the pre-
tax nature of the money.
• One user with a long history of increasing
contributions said:
“I want to know why contributions are increasing
over time. Is this happening because my
employer contribution rate increased?
Did I get a raise? Did I increase my contribution
rate?”
User expressed desire for contextualized story to
why his numbers were changing.

23. 23
Client Insight
For internal use only - Confidential
Contributions
Change paycheck deduction:
Two went to the main menu and chose Manage my
money.
One user tapped red Paycheck Deduction header
first, then went elsewhere.

24. 24
Client Insight
For internal use only - Confidential
Funds list
• All users found funds list, though some struggled
with multiple nav menus, depending on UI.
• All users stated this is something they would do
on a desktop or laptop at home due to screen
size.
• All users tapped fund name to see fund details.
• Users said it had lots of useful information, but
presentation of information for form factor is too
poor to find this very valuable.
• 2 users wanted data table headers to maintain a
point of reference for evaluation.

25. 25
Client Insight
For internal use only - Confidential
Funds list
• All users found this to be interesting information,
but wanted a more optimized presentation of
information for smaller screens.
• When users arrived at this screen, the introduction
of another form of navigation caused significant
confusion about where they were within the
Vanguard experience.
• Avoid showing users multiple forms of navigation to
reduce confusion.

26. 26
Client Insight
For internal use only - Confidential
Change investments
• 11 of 12 were unsure or incorrect what Rebalance
account would do.
• Consider relabeling actions or providing a better
descriptions to explain the difference between
money people have in their account and future
money they’ll be earning from their employer.
• 8 users said they would call for reassurance about
managing money.
Users voiced concerns about taking these actions
on the app and fear of making the “wrong” choices.

27. 27
Client Insight
For internal use only - Confidential
Transaction history
• Print and download features, specifically with
statements, caused confusion with the mental
model of mobile device expectations.
This confusion could be due to pulling traditional
operating system web browser content into
mobile app/web hybrid experience.
Accessing PDF statements were well-received,
but file formats not matching user mobile phone
mental model caused confusion (Ex: allowing
access to .csv files). Error screens were shown.
Rx: Further research may be needed. Expectations
from “mobile” may change over time. Consider
focusing and refining features retirement plan users
value higher.
Rx: Further research may be needed for print/share
features
• 3 users mentioned an interest in a feature to
share a statement from within the app, possibly
by email.

28. 28
Client Insight
For internal use only - Confidential
Overall experience – what’s working
• All users found the experience informative and appreciated the convenient
access to information about their plan.
• Retirement-only accounts were generally able to complete tasks and find
what they were looking for to meet high-value needs.
• Default settings for security are well-received. Consistency of security
features with familiar patterns seems well-balanced with convenience
expectations in mobile space.
• Loading times for responsive Web screens were said to feel fast. 11 of 12
users said they were happy with speed of app data loads.
• Users were happy to discover new features they weren’t aware of, such as
Manage My Money options, though likelihood of use was mixed.
• Cross-linking was appreciated when information was relevant to surrounding
content.

29. 29
Client Insight
For internal use only - Confidential
Overall experience opportunities
• All users struggled with any screen not designed for the device type.
Establishing an optimized hierarchy & presentation of information helps, but
inconsistent application of those efforts seemed to frustrate users.
“It looks like this is just the website inside the app. I would just go to the
website if this is what the app is.”
“There’s a LOT of information here… Maybe an introduction to what’s in the
app would help, though many probably skip those things. It would be good to
know that sort of thing is there.”
• All users experienced issues where the hybrid Web/native app experience
leaves gaps in UX quality at some point during each session.
• 2 users mentioned concerns about the position of the in-app browser back
button and the log-off UI.

30. 30
Client Insight
For internal use only - Confidential
Overall experience opportunities
• Most users struggled with in-app browser UI
response, confusion about inconsistent interaction
design patterns for the touch experience, and
navigation inconsistencies or gestures causing
unexpected actions.
Ex: Four instances of hitting the back button UI 3-
4 times to get it to register their finger input.
Rx: This could be a touch-target issue and/or fear
of hitting the “Log Off” in close proximity.
• Horizontal swiping some columns in a table
triggered the side menu to open unexpectedly.
• Users with multiple retirement accounts were lost
when drilling down, losing track of which account
they were seeing data on.
Rx: Offer a plan name at the top of each screen to
help frame location.

31. 31
Client Insight
For internal use only - Confidential
Overall experience opportunities
• Inconsistency of navigation UI elements left users
unsure about where they would go next.
For instance, both iPhone and Android crossover
users stated:
“Sometimes I get stuck, because there’s no back
button.”
“l’ll use the hardware button to get back and just
see where I end up.”
Rx: Establish a more consistent UI pattern
between Retail and Retirement experiences and
match user expectations according to mobile OS
human interface guidelines.
Reference Apple Human Interface Guidelines for
iOS mobile apps and do the same for Android.

32. 32
Client Insight
For internal use only - Confidential
Overall experience – Opportunities for crossover clients
• Information architecture for crossover users caused significant confusion
about users understanding where they were within the experience and which
account they were referencing.
• Users were often lost when seeing multiple menus on the app screens and
website menus in the content area.
• All crossover users were expecting to see data presented in the same way
across retail and retirement accounts.

33. Appendix:
Appendix
- Demographics data on:
- Account status
- Age
- Device types
- Location of use
- Feedback on:
- Task completion
- Likelihood of calling to complete a task
- Quality ratings per screen
- Feature requests
- User experience challenges

34. 34
Client Insight
For internal use only - Confidential
Demographics: Account types
0
1
2
3
4
5
6
7
8
9
10
Only
Participant
Experience
Core Voyager Voyager
select
Flagship
Number of people

35. 35
Client Insight
For internal use only - Confidential
Demographics: Age
0
1
2
3
4
5
6
7
8
9
10
20-30 yrs 30-40 yrs 40-50 yrs 50-60 yrs
Number of people

36. 37
Client Insight
For internal use only - Confidential
Where do you use the app?
0
1
2
3
4
5
6
7
8
At home At work Commuting Traveling or
while out
Number of responses
Note: Data doesn’t add up to 12
because the same person could
be in multiple categories