This is a presentations of our findings from usability testing of OAuth, OpenID, and the Hybrid protocol. This was originally presented at the OpenID Usability Summit hosted at Facebook on Feb. 10th.
2. Test Methodology and Goals
OAuth Standalone
• Gauge ease of use for the feature
• Determine overall likely user acceptance of the feature
OpenID Standalone
• Determine user familiarity with the concept
• Discover user assumptions about how the feature works
• Gauge its overall ease of useOpenID/OAuth Hybrid
OpenID/OAuth Hybrid
• Determine perceived user value of the feature
• Gauge ease of use in a range of options
• Discover user assumptions about how the feature works
3. User Summary
• Users participated in one-on-one sessions
• Equal numbers of male and female users were recruited
• Ages ranged from 14-34
• All had at least one MySpace account; a few users had several MySpaces
• 5 of the 12 respondents had a MySpace URL; 7 did not (or couldn’t remember it)
• There was a strong gender divide between respondents with and without URLs; more
female users had URLs, and more male users did not
• Users ranked themselves either a 3 or a 4 in web-savvy; two users said they were a 5 in
a particular field, and no users thought of themselves as a 1 or 2
• All users were comfortable publicly sharing basic profile information online
Table A: Summary of Test Participants
Has MySpace URL No MySpace URL
Age Range 14-34 (5 total) 18-34 (7 total)
Sex 4 Female/1 Male 2 Female / 5 Male
Web-savvy Medium to High Medium to High
5. Interface Details: OAuth Confirmation Screen
Users who noticed the
redirect tended to believe
they would see an AOL
module on MySpace
The graphic logos at the top of the page were
understood to mean that AOL and MySpace were
sharing information; there was some confusion over
where the information would appear (On AOL? On
MySpace?)
Challenges & Recommendations
• Users thought this was a simple screen; the
visual layout makes it feel approachable
• We did not test a logged out version of the
screen; the logged out version should be tested in
a Pop-up state to determine whether the screen
context helps users understand the flow more
clearly
• The graphical double arrow at the top of the
screen does not accurately depict the exchange
of information that will occur, because the user’s
AOL information is not being shared with
MySpace; this should be clarified
This seemed like an easy
Most users ignored this
• If the OAuth service checkbox is important for the
way to validate their
checkbox; those who checked it
MySpace account; probably users to select, add some education to the screen
assumed it would keep them because it’s visual, it was
(for example, a “What’s this?” link that opens a
logged into the module (cookie)
considered “basic” by most
contextual help overlay describing the purpose of
respondents
the checkbox)
quot;Terms and conditions – I never
read thosequot; (Laughs)
- Byran, 24
6. What We Noticed
• Users were comfortable with the OAuth login page we showed them, which
showed an authenticated user and no log in fields
• Most seemed comfortable with the idea that MySpace might already know who
they are when they click on the “Log In Now” call to action; no one worried out
loud about security in this flow; MySpace in a separate module feels safe and fun
(this is different from the idea that MySpace forms the basis for a 3rd party
account, which as a concept raises more security concerns for users)
• Some users expected log in fields to appear inline (in the AOL module) when
they clicked to sign in, but no one said the separate MySpace redirect would stop
them from logging in
• The graphical double arrow made users believe they were linking their MySpace
account to their AOL account, but there was some confusion about directionality;
most understood they’d be seeing MySpace on their AOL page, and believed
they would be signed into MySpace but viewing it on AOL (though some got it
backwards because they noticed they’d been redirected to the MySpace site)
8. OpenID Testing Overview
Summary
OpenID is completely new to users, and the notion of using a URL to sign in to a website baffles them –
but they love the idea of having an ID that allows them to remember just one set of login credentials across
the web.
• Users see the MySpace account as separate from the 3rd party account
• More frequent MySpace users were enthusiastic about using their MySpace accounts as a “parent” account
across the web, and expected their MySpace information to automatically update the 3rd party account
• Security concerns were paramount and represented the largest barrier to use, even for users who liked the
concept
“I guess that once you register it gives you that – it’s just a quicker way
to sign in.”
- Melany, 27, guessing at the meaning of OpenID
“Isn’t a URL just a website?”
- Melissa, 34, talking through her confusion about the security of OpenID
Key Challenges
• None of the respondents had heard of OpenID, and no one guessed correctly that it was a URL
• Security concerns were high, particularly once users learned that the OpenID is simply their public
MySpace URL
9. Yelp OpenID Login Page
Getting to the Sign In screen from the
Yelp home page was easy for every
respondent. Once on this page, all
users gravitated to the standard Log
In fields and only looked at OpenID
with prompting.
Challenges &
Recommendations
• OpenID was a mystery to
everyone. When pressed, most
respondents guessed that it was
a special code Yelp would give
them when they first registered
with the site. A similar pattern
emerged with Netflix.
• When told the OpenID was a URL, most users recognized the phrase, but only 2 of
12 entered a correct URL structure. Most entered just a unique ID (e.g.
“iamthetom”), and others entered a URL/email address hybrid (e.g.
“url.iamthetom@myspace.com”)
• Users need help! Start by helping them with the URL format.
• Some education about OpenID and how it works is needed here.
10. MySpace OpenID Pop-up Experience
This graphic helped users understand that they
were linking Yelp and MySpace somehow; however,
the bidirectional arrow doesn’t accurately show the
relationship between the two systems. The
directionality depicted in the graphic should be more
literal, as that will help users understand how
OpenID sign in works.
Also, a few users saw the graphics as advertising at
first glance. The visual design should not resemble
MySpace promotional or advertising graphics.
Most users felt more comfortable with the pop-up
version of this screen vs. the Redirect version (even
when they didn’t notice the difference between a
pop-up and a redirect).
Challenges &
Recommendations
• When users got to this screen, they
basically understood that they were
confirming their MySpace identity in
order to use Yelp.
• Seeing the MySpace login fields,
especially the Password field,
greatly increased their comfort level
with OpenID.
• Most users said they would ordinarily just fly through the screen, maybe
These checkboxes seemed redundant
selecting the first checkbox, and not dwell on the details. The visual layout of
to users, and were thus confusing.
The “service” in question is invisible the screen helps create a sense of familiarity and ease, which should be
on the screen (no branding), so users
maintained in future iterations. However, even though users didn’t see the
inevitably made assumptions about
screen as a barrier, clarifying the graphics and providing optional educational
the box that were incorrect – or they
simply ignored it. links (about OpenID and/or OAuth) can add useful context to the screen
11. MySpace OpenID Redirect Experience
Users focused on the center of the page and rarely
noticed that they had been redirected to MySpace;
however, when they did notice the redirect, they
were slightly more confused and uncomfortable with
the flow. However, it was not a barrier, and most
said they would continue anyway.
No one reads the small print. Some respondents
offered comments about it – “I never read that stuff”
– so if it’s there is something very important here,
place it somewhere in the body of the page
(perhaps offering an anchor link down the screen for
additional details).
Challenges & Recommendations
• Logging in with an email and password feels like a normalized activity to users, whereas logging in with a
URL is completely new, and introduces another step in the process. Users must be clear on why that extra
step will be worth the effort.
• Even though the difference was fairly small, users were slightly less comfortable with the redirect screen than
with the pop-up version. Recommend using the pop-up version rather than this redirect.
12. MySpace OpenID Pop-up Experience:
Logged In
The Pop-up version was better liked
than the redirect. It felt more accurate
(I want to sign into Yelp, so keep me
on the Yelp site).
Respondents didn’t realize they were in a
MySpace logged in state, and felt very nervous
about the potential for fraud with this scenario.
Make it more explicit here that the user is
already logged in; asking users to enter a
password might be a welcome measure of
added security that will increase users’ comfort
with loggin in using OpenID.
13. Some User Statements
“I do have a Yelp account – would I have to eliminate the Yelp that I have
and login anew with the OpenID? I can see switching over being a bit
bumpy trying to get all my info straight, but then knowing I have one set
of info that’s being applied to everything would ultimately probably make
it a little easier for me. And it would also solve my problem of having one
password for everything.”
- Royce, 23 (getting to concerns about whether MySpace data would over-write his existing account data if
he signed in using the MySpace OpenID)
“That’s crazy you guys are linking everything together – I think it’s
cool”.”
- Alyson, 22
“I might use the OpenID, because there are so many things you want to
sign up for and…it is such a pain to have to register for everything…if
you could just enter the basic information and it be secure, I would
probably do it… security is just the biggest factor for me.”
- Melany, 27 (frequent MySpace user, expressing a concern we heard from several likely users)
15. Hybrid Testing Overview
Summary
For this flow we used an eCommerce site (Netflix), which by the very fact of being transactional, raised
additional security concerns that we hadn’t seen on the OpenID Stand-Alone flow. This flow raised more
questions for users about the nature and security of their MySpace data.
• Security concerns around the OpenID URL need to be addressed. (See OpenID Stand-Alone for
additional details and recommendations.)
• Context matters here – users couldn’t imagine how their MySpace account information would be
relevant on a 3rd party account until they saw an example of how it might work. Education has to occur
prior to log in, or users simply won’t use the functionality.
• Additionally, education needs to continue throughout the login process, so that users can visualize how
this new form of site registration will work once they complete it.
• Existing functionality forms user expectations about how new functionality will work. In this case, they
imagine the Hybrid to be a more rich version of an email address import – that instead of email
addresses, it will import their MySpace friends (and profile information). Without a Friends feature, this
will have a smaller receptive audience.
“Once you see it and once you get in it, it seems very innovative and very helpful. ”
- Melissa, 34, explaining how we can improve this process
16. MySpace Hybrid Redirect Experience: No
Scoping
Only the Redirect page was tested, but based on
the results of the OpenID test, MySpace should
consider moving this process to a Pop-up overlay.
Most users were comfortable with this identification step, but one user
was confused by the “OR” option and read the choices as buttons she
was meant to select from.
The graphic should accurately depict the directionality of the
data flow.
Some users skipped the password field because it looked like it was
prefilled (perhaps from browser memory). Recommend placing field
labels outside the fields rather than inside them to avoid errors.
This checkbox seemed redundant to some users based on the
context of their activity – after all, wasn’t the point of this flow to
create a Netflix account? This may be more appropriate as part of a
log in flow rather than an account creation flow.
The CAPTCHA was not a problem for any users. All respondents
were accustomed to them and seemed to understand their utility.
Challenges & Recommendations
• At this point in the process, users still don’t
know what they are trying to achieve by
signing up for Netflix with their MySpace
account. Providing them with context and
clarity throughout this process will increase
user comfort levels, and thus should increase
adoption of this new functionality.
17. MySpace OpenID Redirect Experience:
Granular Scoping
This version was the crowd favorite. All users liked this version the
best, because they felt like it gave them more control over their new
3rd party profile.
Seeing the list of options prompted respondents who were
interested in Netflix to wonder aloud if their MySpace Movies
options could prefill some Netflix data for them. For some, this
list set up expectations of a richly engaged Netflix experience
in the next step.
Challenges & Recommendations
• While this screen was the crowd favorite,
most users admitted they would probably
share everything in real life if they didn’t have
any choice in the matter
• More checkboxes start to set up expectations
for the MySpace/3rd party link that the 3rd
party may not be able to fulfill
• The more users looked at the checkboxes,
the more they deemed most of the
information represented here acceptable to
share
• The granular options are superfluous; this is
more information than users need to
complete their registration flow.
18. MySpace Hybrid Summary
• This is the most useful application of the OpenID functionality that we tested.
• Users see its utility once they experiment with it, but will not use it unless they are first convinced they
understand and can trust it.
• Again, context matters. If users can see how their MySpace account data would be relevant to the
new 3rd party account, they are more likely to link the two; otherwise they would skip the OpenID and
just register separately.
• Most users expect that their MySpace account will act as a “parent” account for all 3rd parties that use
the OpenID; for users, this means they have the expectation that updating basic information on
MySpace will also update their linked 3rd party accounts.
• Porting over their MySpace friends (real friends, not Bands/Comedians/Filmmakers) is the killer app
for this functionality – prefilling account information is useful, but bringing their network with them is
the most useful piece for them (and a piece we were not able to test in this round).
“[It’s] like a universal profile.”
- Royce, 23
“Putting it all together just makes it all easier... You can just click on a
link and it’s all there.”
- Jason, 25
“I’d probably use the Netflix one. I’d…create a whole new MySpace…and
make it like really clean…and start over.”
- Kevin, 18