User Experience: Crafting Recommendations

Wisconsin Library Services
Crafting User Experience Research
Recommendations
Maria Dahman
User Experience Architect, UW-Madison
Research Lead, Center for Digital Accessibility and User Experience
maria.dahman@wisc.edu

Agenda
● Introductions
● UX Evangelism
● Research questions: a key for
recommendations
● Case Study
● BREAK (10 min)
● Writing Recommendations

Introductions
● Name
● What kind of work do you do?
● What challenges have you
faced with giving
recommendations?
● What other questions do you
have?

Jesse James Garrett,
The Elements of User Experience
Explaining UX

Explaining UX
Design
● Wireframing
● Mindmapping
● Designing interactions
● Accessibility considerations
Design Research
● Generative Design Research (a.k.a.
“Foundational Research”)
○ Interviews
○ Contextual Interviews
○ Diary Studies
○ Observations
○ Service Design
○ Journey Mapping/Blueprinting
○ Cardsorting
○ Surveys
● Evaluative Design Research (a.k.a.
“Directional Research”)
○ Metrics
○ Accessibility review (tied to usability)
○ Usability review (tied to accessibility)
○ Heuristic review
○ Reverse cardsorting (tree testing)
○ Surveys (System Usability Scale)

Observing user behavior, not asking them to predict behavior.

Defending UX
● “But it’s only a few people!”
● “We have Google Analytics--isn’t that enough?”
● “That’s not statistically significant.”
● “You’re talking about boiling the ocean.”

“But it’s only a
few people!”
N (1-(1- L ) n
)
● Usability specific
● Return on Investment

“We have Google
Analytics--isn’t that
enough?”

Iterative design
and working
with an agile
development
team

“It’s not statistically
significant.”

We are not talking about
medical trials or brain surgery.
We can iterate websites and
applications--one doesn’t want
to iterate brain surgery.
Statistic significance is a huge
burden for most usability
problems, for questions that
are often not that risky.

“You’re talking about
boiling the ocean!”

“We’ve already done
research. We know
our users.”

Agenda
● Introductions
● UX Evangelism
● Research questions: a key
for recommendations
● Case Study
● BREAK (10 min)
● Writing Recommendations

Research Questions
● Sometimes called the
“goals of the research”
● Are not the script
questions for your
interviews
● Do users understand quickly what the
comparison tool is for?
● Are there functions of the old comparison
tool that are helpful and might be missing
from the new prototype?
● Does the user know what his next steps
are?
● What are users’ mental models for how
they organize programs?
● Do users know how to request more
information about a program?
● Are the current wireframes useful to
compare programs? Are too many things
being compared at once?

Talk to stakeholders
(step 1 of 3)

1. In terms of what you do on a
day-to-day basis, what are the goals
of the product or service?
2. Are there ways that it’s not meeting
these goals?
3. Are there questions you want to have
answered about it? If so, what are
they?

Prioritize the goals
(step 2 of 3)

Rewrite the goals as
questions to be
answered (step 3 of 3)

The System Usability Scale--MyUW
https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
I think that I would like to use this system frequently.
I found the system unnecessarily complex.
I thought the system was easy to use.
I think that I would need the support of a technical person to be able to use this
system.
I found the various functions in this system were well integrated.
I thought there was too much inconsistency in this system.

“Is there anything else
you’d like to share
about your experience
with using MyUW?”

“Is there anything else
you’d like to share about
your experience with
using MyUW?”
Goal: Understand pain points
of our users. Assess usability.
Provide benchmarking.
Research question: How do
users describe their
experience using MyUW?
What are the “unknown
unknowns”?
Generative, foundational
research attached to an
evaluative survey.

I think that an evaluation of which apps come preloaded to the dashboard is in order--there are
a lot of things that are useful that I didn't know I had access to because there's no indication that
you can add things to the front page.
There were times when using the search that you wouldn't find what you were looking for, and
there was no clear way to check otherwise. The search has improved though... or maybe I
learned to use it better
I've liked that the myUW homepage hasn't changed much in the four years I've been here.
There was some confusion on my end what widgets/apps could and couldn't be added to the
homepage.
I'd recommend condensing some of the tiles on the MyUW home screen. A lot of programs can
be accessed from multiple applications and condensing them would make the home screen
more streamlined.
The applications on the dashboard page can be confusing. Having options which I do not use is
tiresome because it is just clutter.
I only use a few components. As an online student, my course grid is irrelevant, as is all the
on-campus and advisor information (I've never met 'my advisor' and not sure if I have one
assigned). I only use email, the student center to view bills/tuition, register for classes and the
Lynda.com portion for training/learning videos. The rest is unnecessary fluff for online
professional program student like me.
Some of the apps on the homescreen are very hard to understand and seemingly useless with
no important content. There should be more resources that are useful to students on the main
page, like a UW Health app.
I think overall it works well, but sometimes I don't know what is within the separate tabs and I am
not able to determine it by simply clicking on it.
The applications on the dashboard page can be confusing. Having options which I do not use is
tiresome because it is just clutter. I've liked that the myUW homepage hasn't changed
much in the four years I've been here. There was some confusion on my end what widgets/apps
The financial aid pages are very difficult to work with. Please simplify how students can review
financial aid packages, how payments can be made. - it's very cumbersome and complex. Also,
when I go to view how enrolling in refund, page is always saying error or won't send me directly
to how to enroll or how to update information. I also feel I do not know what I am missing out on
because I have to enter or add the apps I want. Need to have drop down box for all information
I can have access to with tabs explaining each topic so I can use the information I need. For
example, I tried to find out how to sign up for GUTS and found that to be so difficult I missed out
on signing up. Please make everything a student needs much more accessible!!!! Thanks you.
"My use of this system is essentially limited to ""student center"" (which I still get through by trial
and error to register for classes every semester), email (which is reasonably straightforward but,
unfortunately, Outlook, also, it is flooded with spam and irrelevancies); and Canvas, which is
fine. I'm not really sure what any of these other tiles are for."
It'd be nice to have more widgets automatically added to the home screen when you first log in
as a brand new student. For example, grad students should have payroll, benefits, etc. on their
home screen. Many new students don't know what to add because they simply don't know what
is available.
I like the widgets on the homepage. I think adding new ones is a bit confusing and knowing the
full range of options was not intuitive.
i think there are a lot of features that people (and me) don't know exist
I would feel lost when so many items are listed on the home page although it is me who added
them to the home page. I also have not enough knowledge about how I can employ MyUW
thoroughly beside logging into Canvas or managing my Wiscard.
the survey is hard to answer because of the statement structure. i mean, i think i know how to
use the system, but i'm finding there are more things available which i didn't know about... so
me saying I'm an expert when i continue to find there are areas i have not scratched the surface
is a possible response.
It is not very intuitive for grad students who have never used this system to find the things they
need...especially the apps on the myUW page.
163

Recommendations
Answer the question “What do I need, and what is available?”
“I think that an evaluation of which apps come preloaded to the dashboard is in
order--there are a lot of things that are useful that I didn't know I had access to
because there's no indication that you can add things to the front page.”
“It'd be nice to have more widgets automatically added to the home screen when
you first log in as a brand new student. For example, grad students should have
payroll, benefits, etc. on their home screen. Many new students don't know what to
add because they simply don't know what is available.”
Assess technical problems mentioned in the survey--are they on the backlog? Are they
already being addressed?
Address email integration problems
Assess holistic landscape of our users: add content with intention (dimensions of
wellness)
Improve mobile experience

Ideas for a follow up
recommendation?

What
● Create a more obvious way on the homepage for users to know there are more apps they
can add.
● Categorize widgets in the directory in a way that makes sense to users. This could require
tagging and robust metadata for the apps. Some ideas for categories appear in this
spreadsheet.
Why
● Unawareness of what was available to them (“What do I need? What do I have access to?”)
was a consistent theme in user comments in interviews and the 2018 survey.

Where do I start?
● Research questions
● Heuristic evaluation
● Accessibility review

Best practices for
Recommendations:

Recommendations
from generative and
evaluative research
will differ.

“How might we
improve the
admissions process?”

Recommendations for
future research

“I always mess
this up…”

Characteristics of Evaluative Research
● Has a pointed specific
question it is answering
● Has a short life span
● Lives within the agile
teams and is directed by
the agile teams.
● Experiments are small
and self-contained
ADD EXAMPLES OF
EVALUATIVE RESEARCH

Be specific
“Registration was hard.” “The button to register for
the site had poor color
contrast and faded into the
page background.”

Don’t blame the user
“The users didn’t know what
they were searching with
the search bar.”
“The search bar is
positioned in a way that
looks like it is searching the
page and not the whole
site.”

Look for the bigger picture
Problem: recommendations
are focused too tightly on
small issues that indicate a
larger problem
Solution: Keep the smaller
details in the report, but
include the findings about
larger over-arching issues
so they don’t get overlooked

Help identify solutions
Problem: “You are usability
testing. Give us your results
and we’ll design the
solution.”
Solution: Present the results
as a discussion, not a
delivered report that you are
done with. You witnessed
the success and failures of
the designs, and can offer
yourself as a subject matter
expert.

Organize and rank findings
Problems:
1. Teams don’t know what
findings are relevant to them
2. There are anywhere from
5-100 findings
3. The severity of the problem is
unclear
Solutions:
1. Chunk findings by screen
2. List recommendations from
most pressing (from a user
perspective) to least pressing
3. Rank the findings as Low,
Medium, or High severity

Accrue future value from descriptive reports
Problem: We don’t learn
from our past design
mistakes.
Solution: Maintain an
archive of usability findings
that other teams can draw
from.

Summary of Part I
● Be able to explain UX work
● Anticipate criticisms of UX work
● Develop relationships with stakeholders by asking
○ In terms of what you do on a day-to-day basis, what are the goals of
the product or service?
○ Are there ways that it’s not meeting these goals?
○ Are there questions you want to have answered about it? If so, what
are they?
● Develop research questions from these stakeholder conversations

Summary of Part I, continued
● Be in contact with the person who directs and decides the work of the
development team, making this person (and maybe the team) aware of your
research plans
● Start (or maybe even complete) your recommendations report with answering
your research questions
● Consider a heuristic and accessibility review if you don’t know where to start
● In an agile environment, focus on usability for the sprints while doing parallel
generative research
○ Input the tickets directly, if possible
○ Encourage the development team to think of questions they want
answered
○ Encourage them to watch

Summary of Part II
● Be specific
● Don’t blame the user
● Look for the bigger picture
● Help identify solutions
● Organize and rank findings
● Accrue future value from descriptive reports

Evaluative Questions
Should the button label be
“Submit” or “Register and
Continue”?
We ran an A/B test of 500
visitors, and “Register and
Continue” performed 20%
better.

References
● How to Stop UX Research Being a Blocker:
https://medium.com/beakerandflint/how-to-stop-ux-research-being-a-blocker-2
25d91105de8
● Jesse James Garrett, The Elements of User Experience
● Mike Kuniavski, Observing the User Experience
● Making Usability Findings Actionable: 5 Tips for Writing Better Reports
(Nielsen Norman Group)
● Usability.gov
● Digital Services Playbook

Photo credits
roi by popcornarts from the Noun Project

User Experience: Crafting Recommendations

Recommended

Recommended

More Related Content

What's hot

What's hot (14)

Similar to User Experience: Crafting Recommendations

Similar to User Experience: Crafting Recommendations (20)

More from WiLS

More from WiLS (20)

Recently uploaded

Recently uploaded (20)

User Experience: Crafting Recommendations