The Revolution Will be A/B Tested

The Revolution Will Be A/B Tested: the (anti-)aesthetics of data-driven design

You must either make a tool of the creature, or a man of him. You cannot make both. Men are not intended to work with the accuracy of tools, to be precise and perfect in all their actions. If you will have that precision out of them... you must unhumanize them. — John Ruskin, The Stones of Venice They're putting us in identical little boxes No character, just uniformity They're trying to build a computerised community But they'll never make a zombie out of me. — The Kinks, Muswell Hillbilly

• how do we determine that one is better than another? • can be more than two test conditions. point is to limit to a single variable.

A • how do we determine that one is better than another? • can be more than two test conditions. point is to limit to a single variable.

A B • how do we determine that one is better than another? • can be more than two test conditions. point is to limit to a single variable.

A > B • how do we determine that one is better than another? • can be more than two test conditions. point is to limit to a single variable.

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AA AB AC AD AE AF AG AH AI AJ AK AL AM AN AO • Google tested 41 shades of blue-green for links • Presumes a lot: a link looks like a link, it should be blue-green • How do they know that was the right question to ask?

Marissa Mayer, VP of Search Products and User Experience

• Ranked #1 in traffic worldwide • 42% daily reach (~728 M) • 727,000 sites linking in — alexa.com, January, 2010 Mayer is in charge of design decisions for the most visible design artifact on the web

“We let the math and the data govern how things look and feel.” — Marissa Mayer “Data, Not Design, is King in Age of Google” New York Times, May 9, 2009

“Marissa received her bachelor’s degree in symbolic systems and her master’s degree in computer science from Stanford University. For both degrees, she specialized in artificial intelligence.” — Google Corporate Information what’s her background?

Douglas Bowman Visual Design Lead, Google • There are visual designers at Google • Doug Bowman, worked at Wired, started stopdesign • motto: “balancing form and function since 1998”

Douglas Bowman Visual Design Lead, Google Creative Director, Twitter why did he leave?

“I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that.” — Douglas Bowman “Goodbye, Google”, stopdesign blog how do you prove a case?

This is the guy you call when you need to prove it: Jared Spool, founding principal, User Interface Engineering

“Jared spends his time working with the research teams at the company, helps clients understand how to solve their design problems, explains to reporters and industry analysts what the current state of design is all about, and is a top-rated speaker at more than 20 conferences every year.” — User Interface Engineering Author Bios • He’s the “expert witness” on user interface • Third result on google for “Jared” • 9,572 Twitter followers

“Good design, when it’s done well, becomes invisible. It’s only when it’s done poorly that we notice it.” — Jared Spool “Great Designs Should be Experienced and not Seen” hmm... where have i heard this before?

Warde talks about being a “connoisseur” of wine

• connoisseurs use the appropriate glass • the form of the glass determines the way you experience of the wine

“I have asked them. They don’t say ‘it looks nice.’ They don’t mention the looks at all. They only talk about the value they derive from the service altogether.” • client is Netflix, article front-sells a course you can take from UIE • his response to my crystal goblet comment

1. What, and how, did you ask them? raises two questions, first:

The Scientific Method 1. Define the question 2. Form hypothesis 3. Experimentation 4. Analyze data 5. Draw conclusions 6. Publish results 7. Others challenge • Spool doesn’t publish results. He publishes conclusions. • He doesn’t explain methodology (for free)

Research needs interpretation. We interpret through our value systems.

(some) values of web software development: • separate content from presentation (css) • interchangeability (open source) • use default ui components • reuse existing patterns • reduce latency • be agile • values of the developers i work with • these are culturally appropriate for writing code, but not for design

Software Model functional requirements design architecture prototype deploy • about narrowing options from all possible things • necessary because of technical constraints

Design Model research ideate iterate refine • broaden then refine • necessary to explore ideas laterally in order to generate creatively

“I have asked them. They don’t say ‘it looks nice.’ They don’t mention the looks at all. They only talk about the value they derive from the service altogether.” embedded separation of form and content

2. What is the relationship between the “looks” and the “service”?

In this case, Netflix website: grid-based layout, browser default typography, solid color fills, large “content” images

• Simply a wireframe populated with images and colors. • That doesn’t make it invisible, just unremarkable. • e.g. “they don’t mention the looks at all” > self-fulfilling prophecy • another way of saying it’s uninspiring

Vanilla is a flavor. There is no such thing as the absence of form.

As designers, we can’t help but manifest our process in form • Brutalists fell in love with board marks and infrastructure.

two of the most common UX tools are Post-its and markers

Both Mayer and Spool are focused on “use”

User Experience? Both Mayer and Spool are focused on “use”

User Experience? User Interface? Both Mayer and Spool are focused on “use”

User Experience? User Interface? But I hardly know ‘er! Both Mayer and Spool are focused on “use”

experience use use is only part of experience

It is possible to be efficient in action and yet not have a conscious experience... It comes to an end but not to a close or a consummation in consciousness. Obstacles are overcome by shrewd skill, but they do not feed experience. — John Dewey Art as Experience Dewey conceived of experiences as complete narrative loops, with tension and release

Norman’s Three Levels of Design visceral appearance effectiveness behavioral of use reflective self-image • At Zenbe we couldn’t get people to switch email clients, even though they liked ours better • Once they joined, when we tried to nudge them off the old platform, we couldn’t • People become emotionally attached to their daily tools

people were emotional!

“Sad... I don't know what else to say. I'm crying. I hope you will change your mind. Really hope. Thank you.” people were emotional!

• this level of emotion transcends use. it’s indicative of reflective experience. • how to design for experience? some ideas...

“i can understand this decision but i cannot accept it. thank u for the two years, and bye. i dont know what to say, just thank you. i am really sad and upset.” • this level of emotion transcends use. it’s indicative of reflective experience. • how to design for experience? some ideas...

1. Seek to inspire delight rather than eliminate frustration. Design for the whole person, not the “user” inside her.

“Delight lies somewhere between boredom and confusion.” — E.H. Gombrich The Sense of Order

• Most people use wireframes. Apple uses pixel-perfect mockups.

2. Software can be designed through form • Most people use wireframes. Apple uses pixel-perfect mockups.

zenpage modules Posting original compose widgets Welcome, Betty | your profile | your account | upgrade support desk | log out search this flow post comment add a comment... comment files email event map invite options mail compose widgets New Flow All Flows Betty Difaro — 11.14 a updated just now 1 Starred Posts I’ll order some pizza from Patsy’s... any requests? @ttention to me Proposal wrap-up May 21, 2009 Webco Team 12:30 pm - 2 pm Location: Conference Room Annual Report save as ics biz dev Vishal Subramanian —!11.18 a company retreat Just remember I’m a vegetarian : ) Job Candidates marketing Lily Zimmerman — yesterday updated 10 minutes ago 4 Hi Grace et al, here are two alternate versions of the cover design based on your latest comments: twitter qv conversation view 2.0 discussion + activity modules tff_cover_1.png tff_cover_2.png show previous 2 comments Grace Cooper —!8.27 a Wow, these both look great Lily! Hard call... slight preference for #1, but the type is still a little hard to read. Can you increase the contrast on “TFF”? people view zenpages activity view Betty Difaro —!10.48 a These are great! Thanks Lily! I also prefer the one on the left. Pierre Bertrand — 9.14 a updated 20 minutes ago 2 Hey everyone, Here is the latest version of the text. Please take a look and post your comments. Betty, can you set up a planning meeting for Friday? TFF Proposal preview | open original | download people tab “zenpages 2.0” stream Vishal Subramanian —!9.31 a Looks good to me. Grace Cooper-Brown —!10.27 a I found a few mistakes in the Forecast section of the Appendix. Otherwise looks great! Reading Iterative process at Zenbe lets be informed by the form of one interface metaphor in designing the next

3. Agile development and process-based design are natural bedfellows.

“My favorite processes are gradual and meticulous. They force me to build up work slowly, observing the whole as I place a single letter, then a line, then another... there is a relief in creating with a strategy but without a goal, knowing the next step but not the destination.” — Martin Venezky “It is Beautiful... Then Gone”

4. Conduct qualitative research.

4. Conduct qualitative research. Ask questions that broaden, rather than narrow options.

• Contrary to conventional wisdom that “you are not the user.” • If you’re not, become her. Not sympathy, empathy.

5. Live in your interface. Be the user. • Contrary to conventional wisdom that “you are not the user.” • If you’re not, become her. Not sympathy, empathy.

Thanks, mgd program! Jay Harlow jay@zenbe.com @pilgrimmills

The Revolution Will be A/B Tested

by Jay Harlow on Jan 27, 2010

Accessibility

Categories

Upload Details

Usage Rights

Statistics

The Revolution Will be A/B Tested Presentation Transcript