Presentation for Allscripts Developer Partner conference -- Jared Spool's story about the $300m button, a baseline understanding of the difference between interaction and visual design, the importance of feed-back and feed-forward research, and some practical tools to get folks started.
2. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Table of Contents
• The $300M Button
• How Research Fits In
• A Call to Arms
• Putting Research Into Practice
3. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Table of Contents
• The $300M Button
• How Research Fits In
• A Call to Arms
• Putting Research Into Practice
4. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Table of Contents
• Introductions
• The $300M Button
• The Case for Design
• Whaddaya Mean, “Design”?
• How Research Fits In
• A Call to Arms
• Putting Research Into Practice
• Five Mistakes Smart People Make
• Why You?
8. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
- Darian Shirazi, Forbes Magazine
“‘Consumerization of the enterprise’ has just begun…
Publicly-traded enterprise software companies
aren’t investing in design enough and startups are
improving their interfaces at an alarming rate.”
23. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Log InLog InRegisterRegister
The $300M button: First-Time Shoppers
Email Address:
Password:
Forgot Password?
“I’m not here to enter into a relationship, I
just want to buy something.”
24. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Log InLog InRegisterRegister
The $300M button: Repeat Shoppers
Email Address:
Password:
Forgot Password?
“Have I registered here before?
I have no idea….”
25. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
The $300M Fix
Email Address:
Password:
Log InLog In
Forgot Password?
You do not need to create an account to
make purchases on our site. Simply click
Continue to proceed to checkout. To make
your future purchases even faster, you can
create an account during checkout.
or ContinueContinue
32. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Decreased sales
Dissatisfied customers
Poor ratings and reviews
Negative word of mouth
Negative impact on brand
Increased need for documentation and training
Increased support requests and costs
Decreased productivity
Increased errors
Increased support costs
Increased training costs
Decreased job satisfaction
Increased employee turnover
46. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
“I have a need.”
feed-forward
feed-back
Updated from Hugh Dubberly
“It meets my needs.” and
“I can use it.”
49. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Fitting In Research: Lean Startup
“Validated learning… is the principal antidote
to the lethal problem of achieving failure:
successfully executing a plan that leads
nowhere.”
56. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Interviews
…to understand underlying motivations (attitudinal).
... often in person.
... does not test a specific product.
... short. Describes areas of inquiries and sample
questions, but the session is often wide-ranging.
Goal
Location
Context
Protocol
Anatomy
57. Allscripts Developer Conference
1. Predictions often don’t work.
2. Biases and assumptions show and color
our findings.
3. Insights aren’t guaranteed.
4. Synthesizing is critical!
Limitations
Interviews
58. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Interviews
Synthesis focuses on identifying opportunities
based on attitudes / problems, like:
1. experience maps
2. persona development
3. story maps
After the Session
59. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Interviews
Bill
•Value focused
•Heavy researcher
•Focus on convenience
•Uses print
•Habitually replaces cars
“My cars are
practical,
reliable and
dependable, like
me.”
55, married
•now is the time for all good men
to come
•to the aid of their country,
•I want to be lots and lots of
toothpicks
Personas
• define vision
• prioritize features
• speed decision making
• drive a cohesive product
After the Session
61. Allscripts Developer Conference
…to uncover problems and opportunities with a
product (behavioral).
…in person, via teleconference or phone.
…tests a specific product (yours or a competitors).
...Long. May start with a few attitudinal questions, but is
longer and more closely followed.
Usability Research
Goal
Location
Context
Protocol
Anatomy
62. Allscripts Developer Conference
1. Observer effect.
2. Limitations of the artifact.
3. False tasks give false results.
4. Easy to conflate findings and
recommendations.
Limitations
Usability Research
63. Allscripts Developer Conference
Focus on specific changes based on observed
behavior, including:
1.affinity diagramming
2.visual queues with screen shots and photos
and stickies
3.revise and re-test
After the Session
Usability Research
67. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
1. being too engaged
2. being too smart
3. being too responsive to ideas
Five Mistakes Smart People Make
68. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
1. being too engaged
2. being too smart
3. being too responsive to ideas
4. asking people to predict their behavior
Five Mistakes Smart People Make
69. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
1. being too engaged
2. being too smart
3. being too responsive to ideas
4. asking people to predict their behavior
5. confusing success
Five Mistakes Smart People Make
75. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
- Douglas Martin
“Design is inevitable.
The alternative to good design
is bad design, not no design at all.”
80. Allscripts Developer Conference
@audcrane @designmap #ADP_HIT
Appendix
• “The $300 Million Button”, Jared Spool,http://www.uie.com/articles/three_hund_million_button/
• “Web Form Design: Filling in the Blanks”, Luke Wroblewski
• “How Bad UX Killed Jenny”, Jonathan Shariat, https://medium.com/@designuxui/how-bad-ux-killed-jenny-ef915419879e
• “I Wish It Were That Easy”, Kate Daly, https://medium.com/@tikikate/i-wish-it-were-this-easy-aefc02f30749
• Therac-25, http://en.wikibooks.org/wiki/Professionalism/Therac-25
• “What Does Nurse Turnover Rate Mean and What Is the Rate?”, Christine T. Kovner, Carol S. Brewer, Farida Fatehi, Jin
Jun, http://ppn.sagepub.com/content/15/3-4/64
• “The Business Value of User Experience” - D3 Services
My name is Audrey Crane.
I work at DesignMap in San Francisco…
We’re a user experience design agency that provides design services to tech companies…
… (such as ExactTarget, Docker, eBay & HP)
One of the other things we do is work with VCs to run workshops for some of their portfolio companies
To learn how to build design successfully into their organization, culture and products
…especially as the idea of the consumerization of the enterprise gains traction.
…and companies that invest in great product design or with designer cofounders are hugely successful.
VCs want their portfolio companies to be successful, and they believe design is a key to success,
and that’s why Allscripts invited me here.
This is my twitter handle and stuff
So I promised you $300M. This is a story from UI guru Jared Spool that was printed in Luke W’s book “Web Form Design”.
I’m going to start with the punch line:
This is a story about how a $25B online retailer
With a bunch of trained and smart designers
Made $300M by listening better to his users.
Conversions in ecommerce are everything,
and this was a $25B retailer had conversion rates that they knew could be improved.
So they did a heuristic review first… then usability research
There are two major reasons for alternating between heuristic evaluation and user testing as suggested here. First, a heuristic evaluation pass can eliminate a number of usability problems without the need to "waste users," who sometimes can be difficult to find and schedule in large numbers. Second, these two categories of usability assessment methods have been shown to find fairly distinct sets of usability problems; therefore, they supplement each other rather than lead to repetitive findings (Desurvire et al. 1992; Jeffries et al. 1991; Karat et al. 1992).
This was the form he was hired to help with,
or rather this form was part of the checkout process they were hired to help improve.
Nothing wrong with this page on the surface.
It’s so simple, there probably wasn’t even a designer involved.
There are two fields, clearly labeled.
There is a primary call to action and a secondary,
and a “forgot password” link in case people get stuck.
Professional designers conducting a heuristic review,
that is to say identifying a set of heuristics and then applying them,
might move the field titles to be more mobile-friendly, maybe change the buttons,
but nothing major.
The team saw the form as enabling repeat customers to purchase faster.
First-time purchasers wouldn't mind the extra effort of registering because, after all,
they will come back for more and they'll appreciate the expediency in subsequent purchases.
Everybody wins, right?
But when they went out into the field to see how people were getting through the checkout process,
they had a few surprises.
We were wrong about the first-time shoppers.
They did mind registering. They resented having to register when they encountered the page.
Some first-time shoppers couldn't remember if it was their first time
Without even knowing what was involved in registration, all the users that clicked on the button did so with a sense of despair.
Repeat customers weren't any happier.
couldn't remember the email address or password they used.
They remembered the wrong email address.
45% of all customers had multiple registrations in the system, some as many as 10.
160,000 people requested passwords per day.
75% of these people never tried to complete the purchase once requested.
The designers fixed the problem simply.
They took away the Register button.
In its place, they put a Continue button with a simple message:
"You do not need to create an account to make purchases on our site.
Simply click Continue to proceed to checkout.
To make your future purchases even faster, you can create an account during checkout.”
The number of customers purchasing went up by 45%.
The extra purchases resulted in an extra $15 million the first month.
For the first year, the site saw an additional $300,000,000.
Now, Jared’s story has some pretty high stakes, but you guys don’t need me to tell you that the stakes you’re working with are much higher.
The forms you’re working with don’t look like this…
…they look like this.
And the stakes aren’t cash, they’re people.
This is probably the 20th article I forwarded my father on the intersection of healthcare and UX
on medium, called “How bad UX killed Jenny”
And it’s not just the patients
how does that impact the staff?
in June 2013, Forrester estimated that moving from below-average customer experience to above average
would yield more than $3B additional annual revenue for wireless carriers,
more than $1B for hotels,
$262M for insurers,
and $227M for retailers.
In fact, Forrester has documented piles of costs
associated with bad user experience in products
And we all know that bad design is rampant in the health care industry, right?
The Therac-25 machine
Unfortunately, though AECL’s intentions were good, their software design was tragically bad, incorporating a series of horrendous design flaws.
In one instance, during a treatment one machine continuously shut itself down reporting a cryptic ‘H-tilt‘ and ‘no dose‘ error message each time. The operator attempted to deliver the treatment six times before giving up.
From its launch in 1982 till its withdrawal in 1986, six patients received ultimately fatal injuries from Therac-25 treatments.
It was great to hear Dr. Reider talk about usability yesterday.
I’m kind of assuming we’re all bought in to the value of design
but like Cream of Tartar…
we all know it’s important, but what actually is it? And do we have any?
I plan today to talk about research in user experience, but I want to set the context first…
but over the last two days I’ve heard some very different ideas about what “design” is
So just to level set us in the room…
we’re all familiar with this model
you want to hit the mark with a product that’s viable in the market, technically feasible, and usable.
usable means…
Emphasize that interaction and visual design are different!
So to zoom in a bit on how user research fits in to the design process
So how do you know if something is usable?
How do you know if it meets a real need,
and that they can use it?
You have to go talk to those people
and understand their needs
Then you imagine something that would meet those needs
(ford bullshit)
You prototype that (or you make it, but hopefully you prototype it)
And then you show it to more people
to see if it seems to meet their need
and they can use it
Stanley said, “make it invisible to use”. How do you know it’s invisible if you don’t know what they see?
This is research, which I like to break very broadly into two parts:
feed-forward …
… feed-back.
That button thing, that was feed-back.
Hypothesis is an educated guess
We do feed-forward research to educate ourselves
We create a product to instantiate our hypothesis
Then we conduct usability research to test our hypothesis
IF WE DON’T TEST OUR PRODUCT BEFORE WE RELEASE IT, WE ARE TESTING IT IN THE MARKET
In lean startup parlance, this is Think, Make, Check.
Eric Ries talks about a bias towards learning over growth
“Validated learning… is the principal antidote to the lethal problem of achieving failure: successfully executing a plan that leads nowhere.”
Frankly, the best way to advocate for research
is to have someone see it in action
So we did that. Touch works Pro
Just using WebEx
So my goal in this talk
Is to give the people in this room
The tools to conduct two of the most essential types of feed-forward and feed-back research
Having devs and execs see research is critical to getting buy in
If you guys do it, you can take this back to your companies tomorrow
and make a real difference to your products and your cultures
…or contextual inquiry, or field studies…
# of people
People are not good predictors of their own behavior – introspection illusion. "There is a very long history within psychology of people not being very good judges of what they will actually do in a future situation," said the study's senior author, Matthew Lieberman, a UCLA professor of psychology and of psychiatry and bio-behavioral sciences. From Self-Prediction to Self-Defeat: Behavioral Forecasting, Self-Fulfilling Prophecies, and the Effect of Competitive Expectations: “A multiplicity of studies have suggested that people are not very good forecasters at all. For example, work on affective forecasting suggests that judgments of predicted and actual reactions to positive and negative hedonic events often diverge (Gilbert & Wilson, 2000). When predicting emotional states in the future, individuals fail to consider that other events will intercede, thereby lessening the impact of any one event (Wilson, Wheatley, Meyers, Gilbert, & Axsom, 2000). This problem of focalism leads people to predict that their future reactions will be more intense than they actually are (Buehler & McFarland, 2001). Furthermore, people seem to anchor on their current state and feelings when making predictions about their future state and feelings (Loewenstein, 1996; Loewenstein, O’Donoghue, & Rabin, 2000). Not only can individuals not successfully predict how they will feel in the future, but research on behavioral forecasting suggests that people are not very good at accurately predicting their own future behavior. People are only moderately accurate in predicting whether they would engage in certain behaviors and make particular choices in the future (Osberg & Shrauger, 1986; Vallone, Griffin, Lin, & Ross, 1990) and exhibit overconfidence in their self-predictions (Griffin, Dunning, & Ross, 1990; Vallone et al.,1990). Kristina A. DiekmannAnn E. TenbrunselAdam D. Galinsky”
These interviews can veer into details that don’t help, either excavating behaviors (vs. motivations) or doing collaborative design.
It’s easy for moderators to inadvertently let their bias or assumptions slip, and not gain any real insight. Or let them color their perception – you assume you understand but you’re wrong!
There are no guarantees – you’re following your nose to some degree and may change direction or dead end.
It takes some work to synthesize! At least a few days.
Personas are a particularly good “container” for interview findings, that can be used to determine vision and direction, prioritize features, speed decision making, and drive a cohesive product.
…or contextual inquiry, or field studies…
# of people
5. This is what designers do behind the curtain – same issue with feedback from customers who say “make the button bigger”. Will come up later as a best practice.
affinity diagramming (welcome points of contention)
who weighs in? and how? post analysis participation -- Look for observation contentions.
visual queues with screen shots and photos and stickies
When you’re too engaged, it’s hard to be quiet, and if you don’t do anything else, just be quiet. you’re responsive, you’re enthusiastic, you don’t let silence hang in the air.
listen! (count to 5)
don’t finish their sentences
don’t interrupt
let awkward pauses happen, see if they have any more thoughts (count to 5). if you feel too awkward, parrot back what they just said
If you can’t help being engaged, get someone else to test your product
When you’re smart, it’s hard to be neutral
ask questions as incomplete sentences (“So would you say that experience was…?” instead of, “Was that a good experience?”)
use their language (Tivo story)
don’t assume you understand. “How do you check your email?” “I just open the browser and I check it.” “What browser?” “Yahoo!”
When you’re responsive, you may not wait for patterns to emerge.
You may also conflate recommendations with findings.
note assumptions and hypotheses up front — is success proving them right or wrong?
You guys can do this. You have time. Do it on the plane.
Research should be like taking your vitamins, constant, over time.
Not like taking a pain pill, a big deal, after you know something is wrong.
You guys understand the importance of testing.
You wouldn’t release code without going through QA
no matter how great the developer was, right?
You have the power! I could be at a design conference where we all tell each other how important we are, but…
I guarantee if you guys show up with some video of somebody you met at the airport
trying to do something basic with your product and failing
you are going to shake things up
I’ll leave you with this.
Design is happening, whether there is a designer involved or not,
Whether there is research happening or not
Either way,
“Design is inevitable. The alternative to good design
is bad design, not no design at all.”
Questions?
This is another, much more broad model
if you want to get into it, but I won’t here.
Today I’m going to get specifically into two of the most common,
usability studies and in-person interviews.