• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Rethinking user research for social design
 

Rethinking user research for social design

on

  • 4,418 views

This is a talk I gave at IA Summit 11 about how the methods we've been using for a generation to evaluate UIs are imperfect for social interfaces.

This is a talk I gave at IA Summit 11 about how the methods we've been using for a generation to evaluate UIs are imperfect for social interfaces.

Statistics

Views

Total Views
4,418
Views on SlideShare
3,947
Embed Views
471

Actions

Likes
17
Downloads
68
Comments
0

12 Embeds 471

http://urfriday.com 312
http://posterous.com 75
http://library.iasummit.org 48
http://guten.posterous.com 8
http://sprmario.posterous.com 7
http://www.slideshare.net 7
http://www.urfriday.com 4
url_unknown 3
http://www.sitemotif.com 3
https://guten.posterous.com 2
http://twitter.com 1
http://sprmario.tumblr.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \nThe story of the session. \nEnding: she and her father would have done this together. \n\nThis tool is a piece of the social web \n\nNothing unusual about this test. \none person, one computer\npretend to do a task \n\n
  • In use at least 20 years \n\nMethod failed Shelly desperately \n\nStarted thinking about this with Buzz\nFTC\n\n
  • \nBuzz testing \n\nAfter I blogged about it... \n\n
  • We’re not sure how to do user research that will tell us what we need to know about relationships online\n Something’s not working, and I think it’s this: \n
  • People simply do operate in real life the way we ask them to behave in usability tests. \n\nWebEx - observe meetings? No \n\nMarriott.com - take into account the others who decide what hotel? No \n\nRetirement planning tool - husbands + household expenses; wives + children about healthcare decisions \n\nEgypt and Libya, using eHarmony and Facebook to pass coded messages: Who knew a social network would *ever* be used that way? \n
  • \n(talk) \n
  • \n(talk) \n
  • \n(talk) \n
  • The methods were probably *always* less than perfect. \n Considering SxD magnifies the imperfections of user research methods\n \n
  • The methods were probably *always* less than perfect. \n Considering SxD magnifies the imperfections of user research methods\n \n
  • The methods were probably *always* less than perfect. \n Considering SxD magnifies the imperfections of user research methods\n \n
  • \nSeems that with ambient technology, everything has changed. \n\nBut it hasn’t. \n\nThere are 5 factors to user research that I’m convinced we should rework, rethink to avoid things like the Buzz and Etsy privacy violations. \n
  • \nAnything one person does to change the behavior of another is social. \n\nName one thing you can do online that isn’t social. \n
  • \n From one person + one computer... \n Large sample sizes won’t tell you everything you need to know. The Google 20,000 didn’t reveal the privacy problem. Why not? \n Looking at scale another way, let’s think about Quora and it’s massively user-generated IA. How do you design for that? How do you test for that? How do you do user research that will tell you how to go? \n
  • Usability testing and user research methods use tasks - but users don’t come to technology with task goals in mind. \n ephemeral, changeable + context, timing, space, what just happened, what happens next\n \n Tasks are probably activities where goals emerge and change depending on context, timing, space, etc. \n
  • This is probably the wrong measure. \n We should look at control by the user and engagement. People want to be the boss of their data. Buzz and Facebook’s Beacon, and now Etsy’s changes take that control away. \n We need to take a hard look at conventional metrics. \n What is effectiveness and efficiency with Twitter and Facebook open all day? What’s the task? \n Measures of success don’t translate to SxD\n time?\n task completion? \n success? \n
  • People use your UI in ways you didn’t intend, but that help them do what they want to do.\n Individual workarounds now surface to everyone in a network. \n Etiquette and social norms adjust over time and scale. \n Design is effectively open sourced to the user community\n
  • Onlineness, ambient technology is about context and\n\nwho is zooming whom \n\nWe must not violate the context of the relationships. Otherwise, problems. \n\nClients learn embarrassing purchases - Etsy\nGirlfriend learns source of engagement ring - Facebook\n
  • Onlineness, ambient technology is about context and\n\nwho is zooming whom \n\nWe must not violate the context of the relationships. Otherwise, problems. \n\nClients learn embarrassing purchases - Etsy\nGirlfriend learns source of engagement ring - Facebook\n
  • Something built for HCI doesn’t cover the expanded view of this one person interacting online and the implications of that. \n\nLab testing & structured field visits are lacking. \n
  • \n
  • Borrow, borrow from, combine. \n\nI interviewed smart people.... \n\nThis is what they told me: The key is combiningmethods. \n
  • \nMet: Twitter road trip\n
  • \nMet: Twitter road trip\n
  • \nMet: Twitter road trip\n
  • \nMet: Twitter road trip\n
  • \nMet: Twitter road trip\n
  • \nMet: Twitter road trip\n
  • \n
  • \n
  • \n
  • \n
  • And so, we’re back in the lab in Waltham. \n\nShelly is crying as she looks out at her future self. \n\nWe asked a business question. \n
  • \nThe business would have been better served by our asking more about Shelly. \n\nWhat’s the right question? \n
  • \n
  • \n
  • With our generation old methods, we really only know how to find things we’re looking for. \n\n\n
  • With our generation old methods, we really only know how to find things we’re looking for. \n\n\n
  • With our generation old methods, we really only know how to find things we’re looking for. \n\n\n
  • We don’t have good ways of learning about things we don’t know about. \n
  • \nTime to rework the old standard methods into new remixes. \n
  • Thank you. \n

Rethinking user research for social design Rethinking user research for social design Presentation Transcript

  • Rethinking User Research forthe Social WebDana Chisnelldana@usabilityworks.net @danachisIA Summit 2011 #rethinking 1
  • Shelly 2
  • The state of theart is ageneration oldTime for a new generation ofuser research methods. 3
  • Usability testing isn’t telling us what weneed to know for SxD Me, Inc. 4
  • We don’t know how to find out about thingswe don’t know about 5
  • People don’t live in the world doing one taskwith one device out of context WebEx - meetings Marriott.com - reserving rooms Me, Inc. - retirement planning eHarmony - coded messages in Libya Facebook & Twitter - organizing demonstrations in Egypt 6
  • 7
  • H+C=I 7
  • 7
  • H <== (computer) ==> H 7
  • 8
  • Methods and measures were probablyalways imperfect. 8
  • Methods and measures were probablyalways imperfect.But they used to be enough. 8
  • Methods and measures were probablyalways imperfect.But they used to be enough.No more. 8
  • Everything isdifferent now.But not different.All this has happened before. 9
  • The nature ofbeing online issocialIt always has been. email calendaring sharing documents --> Name some online transaction you think is not social. http://www.flickr.com/photos/holeymoon/ 10
  • Scale is a gamechangerCaution: large sample sizes 20,000 googlers didn’t expose real issues. Quora moderating overburdened Google Buzz http://www.flickr.com/photos/daguerreotyped/ 11
  • Tasks aren’twhat you thinkActivities = goals that emerge andchange Effectiveness Efficiency ... aren’t meaningful http://www.flickr.com/photos/jeremybrooks/ 12
  • Satisfaction iscorrelated withtask completionInstead: control, engagement But how do you measure control and engagement? What does engagement mean? 13 http://www.flickr.com/photos/litebriteneonstudio/
  • Users How do you design effective tasks in this case? /cc @continuously product reviews that turn into literary projectsdesign your UI Using comments in Facebook for a conversation Hey, where did everybody go? - user-created help page to describein real time a hack to get around a feature they don’t like Coded messages in Egypt on dating sitesWorkarounds = hacksEtiquette and norms http://www.flickr.com/photos/juleshabib/ 14
  • And we have to becareful not to violateeither of those things:Etsy CirclesFacebook engagementring story 15
  • Social is about context. And we have to be careful not to violate either of those things: Etsy Circles Facebook engagement ring story 15
  • Social is about context.And relationships. And we have to be careful not to violate either of those things: Etsy Circles Facebook engagement ring story 15
  • Methods are not robust for understandingcontext and relationships. 16
  • Cultivate polymaths Decision sciences Communication Anthropology sciences Psychologies Sociolinguistics Behavioral economics Organizational behavior Psychobiology Sociology Social neuroscience Social networks 17
  • The most successful userresearch methods for SxDcombine field and testingtechniques 18
  • 19
  • Methods 19
  • MethodsReviewing online profiles, connections in groups ininterviews 19
  • MethodsReviewing online profiles, connections in groups ininterviewsStories of how you met 19
  • MethodsReviewing online profiles, connections in groups ininterviewsStories of how you metMulti-user sessions with people who have strong andloose ties 19
  • MethodsReviewing online profiles, connections in groups ininterviewsStories of how you metMulti-user sessions with people who have strong andloose tiesVideo diaries with retrospective review 19
  • MethodsReviewing online profiles, connections in groups ininterviewsStories of how you metMulti-user sessions with people who have strong andloose tiesVideo diaries with retrospective reviewExperience sampling with SMS 19
  • Takes more timerecruiting, prep, session, analysis 20
  • Takes strong research design rather than templated, commodity usability testing 21
  • Takes deep thinkingabout questions to answer, and what the evidence will be that you have answers 22
  • Takes study of cohesivenesscollaboration, connections, networks, relationships Strong and weak ties Dunbar 23
  • Where are you with planning? what we asked answered a business question 24
  • Who do you ask when you don’t know?How do you make financial decisions was what we wanted to know - which is about context and relationships 25
  • Would have told us Who is part of planning and deciding Who is trusted Why these people are important Why they are trusted How planning happens 26
  • To build a model that supports 27
  • Rethinking userresearch 28
  • Rethinking userresearchWe’re not getting the answers we need. 28
  • Rethinking userresearchWe’re not getting the answers we need.Experimenting is limited because we’repressured to go to market. 28
  • Rethinking userresearchWe’re not getting the answers we need.Experimenting is limited because we’repressured to go to market.We’re looking for things we know about. 28
  • We’re missing things we don’t know about. 29
  • It’s time to rethink user research forthe new (old) social web 30
  • Dana Chisnelldana@usabilityworks.nettwitter: @danachiswww.usabilityworks.com415.519.1148 31