SlideShare a Scribd company logo
1 of 48
Rethinking User Research for
the Social Web


Dana Chisnell
dana@usabilityworks.net   @danachis
IA Summit 2011            #rethinking


                                        1
Shelly
         2
The state of the
art is a
generation old
Time for a new generation of
user research methods.




                               3
Usability testing isn’t telling us what we
need to know for SxD

                                   Me, Inc.




                                              4
We don’t know how to find out about things
we don’t know about




                                             5
People don’t live in the world doing one task
with one device out of context
                        WebEx - meetings

                        Marriott.com - reserving rooms

                        Me, Inc. - retirement planning

                        eHarmony - coded messages in Libya

                        Facebook & Twitter - organizing
                        demonstrations in Egypt




                                                             6
7
H+C=I




        7
7
H <== (computer) ==> H




                         7
8
Methods and measures were probably
always imperfect.




                                     8
Methods and measures were probably
always imperfect.
But they used to be enough.




                                     8
Methods and measures were probably
always imperfect.
But they used to be enough.
No more.




                                     8
Everything is
different now.
But not different.
All this has happened before.




                                9
The nature of
being online is
social
It always has been.

                      email
                      calendaring
                      sharing documents
                      --> Name some online
                      transaction you think is
                      not social.


                                                 http://www.flickr.com/photos/holeymoon/ 10
Scale is a game
changer
Caution: large sample sizes

                     20,000 googlers didn’t
                     expose real issues.

                                              Quora moderating
                                              overburdened

                                              Google Buzz

                                                        http://www.flickr.com/photos/daguerreotyped/ 11
Tasks aren’t
what you think
Activities = goals that emerge and
change



                              Effectiveness
                              Efficiency

                              ... aren’t meaningful
                                                      http://www.flickr.com/photos/jeremybrooks/ 12
Satisfaction is
correlated with
task completion
Instead: control, engagement   But how do you measure
                               control and
                               engagement?

                               What does engagement
                               mean?




                                                                                          13
                                          http://www.flickr.com/photos/litebriteneonstudio/
Users                     How do you design effective tasks in
                          this case?

                          /cc @

continuously              product reviews that turn into literary
                          projects


design your UI            Using comments in Facebook for a
                          conversation
                               Hey, where did everybody go? -
                               user-created help page to describe

in real time                   a hack to get around a feature
                               they don’t like
                          Coded messages in Egypt on dating sites
Workarounds = hacks
Etiquette and norms




                      http://www.flickr.com/photos/juleshabib/      14
And we have to be
careful not to violate
either of those things:

Etsy Circles

Facebook engagement
ring story




                          15
Social is about context.



                           And we have to be
                           careful not to violate
                           either of those things:

                           Etsy Circles

                           Facebook engagement
                           ring story




                                                     15
Social is about context.
And relationships.

                           And we have to be
                           careful not to violate
                           either of those things:

                           Etsy Circles

                           Facebook engagement
                           ring story




                                                     15
Methods are not robust for understanding
context and relationships.




                                           16
Cultivate polymaths
 Decision sciences       Communication
  Anthropology
                         sciences
  Psychologies           Sociolinguistics

  Behavioral economics   Organizational behavior

  Psychobiology          Sociology

  Social neuroscience    Social networks




                                                   17
The most successful user
research methods for SxD
combine field and testing
techniques




                            18
19
Methods




          19
Methods
Reviewing online profiles, connections in groups in
interviews




                                                      19
Methods
Reviewing online profiles, connections in groups in
interviews
Stories of how you met




                                                      19
Methods
Reviewing online profiles, connections in groups in
interviews
Stories of how you met
Multi-user sessions with people who have strong and
loose ties




                                                      19
Methods
Reviewing online profiles, connections in groups in
interviews
Stories of how you met
Multi-user sessions with people who have strong and
loose ties
Video diaries with retrospective review




                                                      19
Methods
Reviewing online profiles, connections in groups in
interviews
Stories of how you met
Multi-user sessions with people who have strong and
loose ties
Video diaries with retrospective review
Experience sampling with SMS




                                                      19
Takes more time
recruiting, prep, session, analysis




                                  20
Takes strong research design
  rather than templated, commodity usability testing




                                                   21
Takes deep thinking
about questions to answer, and what the evidence will be that
                                          you have answers




                                                            22
Takes study of cohesiveness
collaboration, connections, networks, relationships


                                  Strong and weak ties
                                  Dunbar




                                                         23
Where are you with planning?
    what we asked answered a business question

                                             24
Who do you ask when you don’t know?
How do you make financial decisions was what we wanted to
           know - which is about context and relationships
                                                         25
Would have told us
 Who is part of planning and deciding

 Who is trusted

 Why these people are important

 Why they are trusted

 How planning happens




                                        26
To build a model that supports
                             27
Rethinking user
research




                  28
Rethinking user
research
We’re not getting the answers we need.




                                         28
Rethinking user
research
We’re not getting the answers we need.

Experimenting is limited because we’re
pressured to go to market.




                                         28
Rethinking user
research
We’re not getting the answers we need.

Experimenting is limited because we’re
pressured to go to market.

We’re looking for things we know about.




                                          28
We’re missing things we don’t know about.




                                            29
It’s time to
 rethink user
 research for
the new (old)
   social web
                  30
Dana Chisnell
dana@usabilityworks.net
twitter: @danachis
www.usabilityworks.com
415.519.1148
                          31

More Related Content

What's hot

Massively Multiplayer Object Sharing 10328
Massively Multiplayer Object Sharing 10328Massively Multiplayer Object Sharing 10328
Massively Multiplayer Object Sharing 10328Donggi heo
 
Social Usability Workshop - Lift13
Social Usability Workshop - Lift13Social Usability Workshop - Lift13
Social Usability Workshop - Lift13Gianandrea Giacoma
 
Becoming digitally active FINAL
Becoming digitally active FINALBecoming digitally active FINAL
Becoming digitally active FINALEmma Gillaspy
 
Digital Identity and Personal Learning Networks
Digital Identity and Personal Learning NetworksDigital Identity and Personal Learning Networks
Digital Identity and Personal Learning NetworksSue Beckingham
 
Massively multiplayer object sharing (Web 2.0 open 2008)
Massively multiplayer object sharing (Web 2.0 open 2008)Massively multiplayer object sharing (Web 2.0 open 2008)
Massively multiplayer object sharing (Web 2.0 open 2008)Rashmi Sinha
 
Presentation to Rockland County Capacity Building Initiative - Cornell Cooper...
Presentation to Rockland County Capacity Building Initiative - Cornell Cooper...Presentation to Rockland County Capacity Building Initiative - Cornell Cooper...
Presentation to Rockland County Capacity Building Initiative - Cornell Cooper...Howard Greenstein
 
Discover Social Media
Discover Social Media Discover Social Media
Discover Social Media Annika Lidne
 
Bridging The Gap: Virtual Worlds as a Platform for Knowledge Transfer
Bridging The Gap: Virtual Worlds as a Platform for Knowledge TransferBridging The Gap: Virtual Worlds as a Platform for Knowledge Transfer
Bridging The Gap: Virtual Worlds as a Platform for Knowledge TransferJeroen van Bree
 
Designingsocialiasummit 100408180215 Phpapp01
Designingsocialiasummit 100408180215 Phpapp01Designingsocialiasummit 100408180215 Phpapp01
Designingsocialiasummit 100408180215 Phpapp01dougdeye
 
Design for Social Sharing Workshop
Design for Social Sharing WorkshopDesign for Social Sharing Workshop
Design for Social Sharing WorkshopRashmi Sinha
 
The Significance of Social Software
The Significance of Social SoftwareThe Significance of Social Software
The Significance of Social Softwarezephoria
 
Online collaboration and Building Online Community
Online collaboration and Building Online CommunityOnline collaboration and Building Online Community
Online collaboration and Building Online CommunityMark Kithcart
 
Social Software and Social Media Marketing for frivillige organisationer, NGO...
Social Software and Social Media Marketing for frivillige organisationer, NGO...Social Software and Social Media Marketing for frivillige organisationer, NGO...
Social Software and Social Media Marketing for frivillige organisationer, NGO...IBM Danmark
 
Taking Leadership Online: Developing Your Personal Social Media Voice
Taking Leadership Online: Developing Your Personal Social Media VoiceTaking Leadership Online: Developing Your Personal Social Media Voice
Taking Leadership Online: Developing Your Personal Social Media Voice4Good.org
 
Cornell Clubs and Groups Presentation
Cornell Clubs and Groups PresentationCornell Clubs and Groups Presentation
Cornell Clubs and Groups PresentationHoward Greenstein
 
Using Social Media for Professional Development
Using Social Media for Professional DevelopmentUsing Social Media for Professional Development
Using Social Media for Professional DevelopmentVanessa Dennen
 
Social Information Architecture Workshop
Social Information Architecture WorkshopSocial Information Architecture Workshop
Social Information Architecture Workshopgsmith
 

What's hot (20)

Massively Multiplayer Object Sharing 10328
Massively Multiplayer Object Sharing 10328Massively Multiplayer Object Sharing 10328
Massively Multiplayer Object Sharing 10328
 
Social Usability Workshop - Lift13
Social Usability Workshop - Lift13Social Usability Workshop - Lift13
Social Usability Workshop - Lift13
 
Becoming digitally active FINAL
Becoming digitally active FINALBecoming digitally active FINAL
Becoming digitally active FINAL
 
Digital Identity and Personal Learning Networks
Digital Identity and Personal Learning NetworksDigital Identity and Personal Learning Networks
Digital Identity and Personal Learning Networks
 
Massively multiplayer object sharing (Web 2.0 open 2008)
Massively multiplayer object sharing (Web 2.0 open 2008)Massively multiplayer object sharing (Web 2.0 open 2008)
Massively multiplayer object sharing (Web 2.0 open 2008)
 
Presentation to Rockland County Capacity Building Initiative - Cornell Cooper...
Presentation to Rockland County Capacity Building Initiative - Cornell Cooper...Presentation to Rockland County Capacity Building Initiative - Cornell Cooper...
Presentation to Rockland County Capacity Building Initiative - Cornell Cooper...
 
Discover Social Media
Discover Social Media Discover Social Media
Discover Social Media
 
The Wired Nonprofit Excerpt
The Wired Nonprofit ExcerptThe Wired Nonprofit Excerpt
The Wired Nonprofit Excerpt
 
Bridging The Gap: Virtual Worlds as a Platform for Knowledge Transfer
Bridging The Gap: Virtual Worlds as a Platform for Knowledge TransferBridging The Gap: Virtual Worlds as a Platform for Knowledge Transfer
Bridging The Gap: Virtual Worlds as a Platform for Knowledge Transfer
 
Designingsocialiasummit 100408180215 Phpapp01
Designingsocialiasummit 100408180215 Phpapp01Designingsocialiasummit 100408180215 Phpapp01
Designingsocialiasummit 100408180215 Phpapp01
 
Designing Social Iasummit
Designing Social IasummitDesigning Social Iasummit
Designing Social Iasummit
 
Design for Social Sharing Workshop
Design for Social Sharing WorkshopDesign for Social Sharing Workshop
Design for Social Sharing Workshop
 
The Significance of Social Software
The Significance of Social SoftwareThe Significance of Social Software
The Significance of Social Software
 
Online collaboration and Building Online Community
Online collaboration and Building Online CommunityOnline collaboration and Building Online Community
Online collaboration and Building Online Community
 
Designing Social
Designing SocialDesigning Social
Designing Social
 
Social Software and Social Media Marketing for frivillige organisationer, NGO...
Social Software and Social Media Marketing for frivillige organisationer, NGO...Social Software and Social Media Marketing for frivillige organisationer, NGO...
Social Software and Social Media Marketing for frivillige organisationer, NGO...
 
Taking Leadership Online: Developing Your Personal Social Media Voice
Taking Leadership Online: Developing Your Personal Social Media VoiceTaking Leadership Online: Developing Your Personal Social Media Voice
Taking Leadership Online: Developing Your Personal Social Media Voice
 
Cornell Clubs and Groups Presentation
Cornell Clubs and Groups PresentationCornell Clubs and Groups Presentation
Cornell Clubs and Groups Presentation
 
Using Social Media for Professional Development
Using Social Media for Professional DevelopmentUsing Social Media for Professional Development
Using Social Media for Professional Development
 
Social Information Architecture Workshop
Social Information Architecture WorkshopSocial Information Architecture Workshop
Social Information Architecture Workshop
 

Viewers also liked

Upping your game: Five things IAs need to talk about more
Upping your game: Five things IAs need to talk about moreUpping your game: Five things IAs need to talk about more
Upping your game: Five things IAs need to talk about moreLeanna Gingras
 
The Reluctant Design Strategist: The Story of a UX Team of 1.5 Failing Forward
The Reluctant Design Strategist: The Story of a UX Team of 1.5 Failing ForwardThe Reluctant Design Strategist: The Story of a UX Team of 1.5 Failing Forward
The Reluctant Design Strategist: The Story of a UX Team of 1.5 Failing ForwardMegan Ellinger
 
A Practical Guide to Measuring User Experience
A Practical Guide to Measuring User ExperienceA Practical Guide to Measuring User Experience
A Practical Guide to Measuring User ExperienceRichard Dalton
 
Effect Mapping: A Better Way to Get Really Usable Results Out of IT Projects
Effect Mapping: A Better Way to Get Really Usable Results Out of IT ProjectsEffect Mapping: A Better Way to Get Really Usable Results Out of IT Projects
Effect Mapping: A Better Way to Get Really Usable Results Out of IT ProjectsJonas Söderström
 
Discussing Design: The Art of Critique
Discussing Design: The Art of CritiqueDiscussing Design: The Art of Critique
Discussing Design: The Art of CritiqueAdam Connor
 

Viewers also liked (6)

Upping your game: Five things IAs need to talk about more
Upping your game: Five things IAs need to talk about moreUpping your game: Five things IAs need to talk about more
Upping your game: Five things IAs need to talk about more
 
The Reluctant Design Strategist: The Story of a UX Team of 1.5 Failing Forward
The Reluctant Design Strategist: The Story of a UX Team of 1.5 Failing ForwardThe Reluctant Design Strategist: The Story of a UX Team of 1.5 Failing Forward
The Reluctant Design Strategist: The Story of a UX Team of 1.5 Failing Forward
 
A Practical Guide to Measuring User Experience
A Practical Guide to Measuring User ExperienceA Practical Guide to Measuring User Experience
A Practical Guide to Measuring User Experience
 
Effect Mapping: A Better Way to Get Really Usable Results Out of IT Projects
Effect Mapping: A Better Way to Get Really Usable Results Out of IT ProjectsEffect Mapping: A Better Way to Get Really Usable Results Out of IT Projects
Effect Mapping: A Better Way to Get Really Usable Results Out of IT Projects
 
The Stories We Construct
The Stories We ConstructThe Stories We Construct
The Stories We Construct
 
Discussing Design: The Art of Critique
Discussing Design: The Art of CritiqueDiscussing Design: The Art of Critique
Discussing Design: The Art of Critique
 

Similar to Rethinking user research for social design

Rethinking user research for social web
Rethinking user research for social webRethinking user research for social web
Rethinking user research for social webDana Chisnell
 
Rethinking user research for social web
Rethinking user research for social webRethinking user research for social web
Rethinking user research for social webDana Chisnell
 
Social Strategy and Integration for C2GPS
Social Strategy and Integration for C2GPSSocial Strategy and Integration for C2GPS
Social Strategy and Integration for C2GPSSue Spaight
 
Soccnx III - A social revolution... "Can I control that?"
Soccnx III - A social revolution... "Can I control that?"Soccnx III - A social revolution... "Can I control that?"
Soccnx III - A social revolution... "Can I control that?"LetsConnect
 
SOCCNX III: A social revolution can i control that
SOCCNX III: A social revolution can i control thatSOCCNX III: A social revolution can i control that
SOCCNX III: A social revolution can i control thatFemke Goedhart
 
WOMMA: Do You Speak Social?
WOMMA: Do You Speak Social?WOMMA: Do You Speak Social?
WOMMA: Do You Speak Social?Antony Mayfield
 
LiveTech Main Screen Presentation
LiveTech Main Screen PresentationLiveTech Main Screen Presentation
LiveTech Main Screen PresentationDan Bowyer
 
Convergence of social media and SEO
Convergence of social media and SEOConvergence of social media and SEO
Convergence of social media and SEOOptify
 
Science of Social Media Personal Branding Keynote
Science of Social Media Personal Branding KeynoteScience of Social Media Personal Branding Keynote
Science of Social Media Personal Branding KeynoteDevon Smith
 
Benchmark Fall 09 Ext Summary
Benchmark Fall 09 Ext SummaryBenchmark Fall 09 Ext Summary
Benchmark Fall 09 Ext Summarylgarrick
 
The freedom to be you
The freedom to be youThe freedom to be you
The freedom to be youAde Oshineye
 
How to Plan a Successful Association Online Community
How to Plan a Successful Association Online CommunityHow to Plan a Successful Association Online Community
How to Plan a Successful Association Online CommunitySocious
 
Building the Instructional Designer's Relevance in 2012
Building the Instructional Designer's Relevance in 2012Building the Instructional Designer's Relevance in 2012
Building the Instructional Designer's Relevance in 2012Aaron Silvers
 
Tech transfer workshop 20101001 final
Tech transfer workshop 20101001 finalTech transfer workshop 20101001 final
Tech transfer workshop 20101001 finalLuke Harvey-Palmer
 
Facebook: Technologies and Societal Impact
Facebook: Technologies and Societal ImpactFacebook: Technologies and Societal Impact
Facebook: Technologies and Societal ImpactLeon Lei
 
A Notion of Social Taste
A Notion of Social TasteA Notion of Social Taste
A Notion of Social TasteXianhang Zhang
 
Igniting Your Online Community: From Slacktivism to Activism / Boyd Neil, H+...
Igniting Your Online Community:  From Slacktivism to Activism / Boyd Neil, H+...Igniting Your Online Community:  From Slacktivism to Activism / Boyd Neil, H+...
Igniting Your Online Community: From Slacktivism to Activism / Boyd Neil, H+...CanadaHelps / MyCharityConnects
 
Designing Web Interfaces
Designing Web InterfacesDesigning Web Interfaces
Designing Web InterfacesMark Wagner
 

Similar to Rethinking user research for social design (20)

Rethinking user research for social web
Rethinking user research for social webRethinking user research for social web
Rethinking user research for social web
 
Rethinking user research for social web
Rethinking user research for social webRethinking user research for social web
Rethinking user research for social web
 
Social Strategy and Integration for C2GPS
Social Strategy and Integration for C2GPSSocial Strategy and Integration for C2GPS
Social Strategy and Integration for C2GPS
 
Soccnx III - A social revolution... "Can I control that?"
Soccnx III - A social revolution... "Can I control that?"Soccnx III - A social revolution... "Can I control that?"
Soccnx III - A social revolution... "Can I control that?"
 
SOCCNX III: A social revolution can i control that
SOCCNX III: A social revolution can i control thatSOCCNX III: A social revolution can i control that
SOCCNX III: A social revolution can i control that
 
WOMMA: Do You Speak Social?
WOMMA: Do You Speak Social?WOMMA: Do You Speak Social?
WOMMA: Do You Speak Social?
 
LiveTech Main Screen Presentation
LiveTech Main Screen PresentationLiveTech Main Screen Presentation
LiveTech Main Screen Presentation
 
Convergence of social media and SEO
Convergence of social media and SEOConvergence of social media and SEO
Convergence of social media and SEO
 
Science of Social Media Personal Branding Keynote
Science of Social Media Personal Branding KeynoteScience of Social Media Personal Branding Keynote
Science of Social Media Personal Branding Keynote
 
Benchmark Fall 09 Ext Summary
Benchmark Fall 09 Ext SummaryBenchmark Fall 09 Ext Summary
Benchmark Fall 09 Ext Summary
 
Ade Oshineye Google
Ade Oshineye  GoogleAde Oshineye  Google
Ade Oshineye Google
 
The freedom to be you
The freedom to be youThe freedom to be you
The freedom to be you
 
How to Plan a Successful Association Online Community
How to Plan a Successful Association Online CommunityHow to Plan a Successful Association Online Community
How to Plan a Successful Association Online Community
 
Building the Instructional Designer's Relevance in 2012
Building the Instructional Designer's Relevance in 2012Building the Instructional Designer's Relevance in 2012
Building the Instructional Designer's Relevance in 2012
 
Tech transfer workshop 20101001 final
Tech transfer workshop 20101001 finalTech transfer workshop 20101001 final
Tech transfer workshop 20101001 final
 
Facebook: Technologies and Societal Impact
Facebook: Technologies and Societal ImpactFacebook: Technologies and Societal Impact
Facebook: Technologies and Societal Impact
 
A Notion of Social Taste
A Notion of Social TasteA Notion of Social Taste
A Notion of Social Taste
 
Igniting Your Online Community: From Slacktivism to Activism / Boyd Neil, H+...
Igniting Your Online Community:  From Slacktivism to Activism / Boyd Neil, H+...Igniting Your Online Community:  From Slacktivism to Activism / Boyd Neil, H+...
Igniting Your Online Community: From Slacktivism to Activism / Boyd Neil, H+...
 
Designing Web Interfaces
Designing Web InterfacesDesigning Web Interfaces
Designing Web Interfaces
 
Social Usability Workshop @ LIFT13
Social Usability Workshop @ LIFT13Social Usability Workshop @ LIFT13
Social Usability Workshop @ LIFT13
 

More from Dana Chisnell

Next generation of civic designers
Next generation of civic designersNext generation of civic designers
Next generation of civic designersDana Chisnell
 
Making Voting Accessible
Making Voting Accessible Making Voting Accessible
Making Voting Accessible Dana Chisnell
 
Answering voters' questions at county election websites
Answering voters' questions at county election websites Answering voters' questions at county election websites
Answering voters' questions at county election websites Dana Chisnell
 
Fooled by best practice
Fooled by best practiceFooled by best practice
Fooled by best practiceDana Chisnell
 
Don't make me read! Less is more in information for voters
Don't make me read! Less is more in information for votersDon't make me read! Less is more in information for voters
Don't make me read! Less is more in information for votersDana Chisnell
 
I can't find what i'm looking for
I can't find what i'm looking forI can't find what i'm looking for
I can't find what i'm looking forDana Chisnell
 
Usability and security in future voting systems
Usability and security in future voting systemsUsability and security in future voting systems
Usability and security in future voting systemsDana Chisnell
 
Gaining design insight through recruiting research participants
Gaining design insight through recruiting research participants Gaining design insight through recruiting research participants
Gaining design insight through recruiting research participants Dana Chisnell
 
Looking across the voter experience
Looking across the voter experienceLooking across the voter experience
Looking across the voter experienceDana Chisnell
 
Anywhere Ballot iterations
Anywhere Ballot iterationsAnywhere Ballot iterations
Anywhere Ballot iterationsDana Chisnell
 
Iterations paper prototype
Iterations   paper prototypeIterations   paper prototype
Iterations paper prototypeDana Chisnell
 
Usability of County Election Websites
Usability of County Election WebsitesUsability of County Election Websites
Usability of County Election WebsitesDana Chisnell
 
Usability and security in future voting systems
Usability and security in future voting systemsUsability and security in future voting systems
Usability and security in future voting systemsDana Chisnell
 
Making sense of the data
Making sense of the dataMaking sense of the data
Making sense of the dataDana Chisnell
 
The users at the end of the mouse - using the 3As to model personas
The users at the end of the mouse - using the 3As to model personasThe users at the end of the mouse - using the 3As to model personas
The users at the end of the mouse - using the 3As to model personasDana Chisnell
 
Democracy is a design problem
Democracy is a design problemDemocracy is a design problem
Democracy is a design problemDana Chisnell
 
Counter to intent: Voters' mental models of alternative counting methods
Counter to intent: Voters' mental models of alternative counting methodsCounter to intent: Voters' mental models of alternative counting methods
Counter to intent: Voters' mental models of alternative counting methodsDana Chisnell
 
California's top lessons learned in design and language
California's top lessons learned in design and languageCalifornia's top lessons learned in design and language
California's top lessons learned in design and languageDana Chisnell
 
Using voter personas to understand who is coming to your election department ...
Using voter personas to understand who is coming to your election department ...Using voter personas to understand who is coming to your election department ...
Using voter personas to understand who is coming to your election department ...Dana Chisnell
 

More from Dana Chisnell (20)

Next generation of civic designers
Next generation of civic designersNext generation of civic designers
Next generation of civic designers
 
Making Voting Accessible
Making Voting Accessible Making Voting Accessible
Making Voting Accessible
 
Answering voters' questions at county election websites
Answering voters' questions at county election websites Answering voters' questions at county election websites
Answering voters' questions at county election websites
 
Fooled by best practice
Fooled by best practiceFooled by best practice
Fooled by best practice
 
Don't make me read! Less is more in information for voters
Don't make me read! Less is more in information for votersDon't make me read! Less is more in information for voters
Don't make me read! Less is more in information for voters
 
I can't find what i'm looking for
I can't find what i'm looking forI can't find what i'm looking for
I can't find what i'm looking for
 
Usability and security in future voting systems
Usability and security in future voting systemsUsability and security in future voting systems
Usability and security in future voting systems
 
Gaining design insight through recruiting research participants
Gaining design insight through recruiting research participants Gaining design insight through recruiting research participants
Gaining design insight through recruiting research participants
 
Looking across the voter experience
Looking across the voter experienceLooking across the voter experience
Looking across the voter experience
 
Anywhere Ballot iterations
Anywhere Ballot iterationsAnywhere Ballot iterations
Anywhere Ballot iterations
 
Iterations paper prototype
Iterations   paper prototypeIterations   paper prototype
Iterations paper prototype
 
Persona modeler
Persona modelerPersona modeler
Persona modeler
 
Usability of County Election Websites
Usability of County Election WebsitesUsability of County Election Websites
Usability of County Election Websites
 
Usability and security in future voting systems
Usability and security in future voting systemsUsability and security in future voting systems
Usability and security in future voting systems
 
Making sense of the data
Making sense of the dataMaking sense of the data
Making sense of the data
 
The users at the end of the mouse - using the 3As to model personas
The users at the end of the mouse - using the 3As to model personasThe users at the end of the mouse - using the 3As to model personas
The users at the end of the mouse - using the 3As to model personas
 
Democracy is a design problem
Democracy is a design problemDemocracy is a design problem
Democracy is a design problem
 
Counter to intent: Voters' mental models of alternative counting methods
Counter to intent: Voters' mental models of alternative counting methodsCounter to intent: Voters' mental models of alternative counting methods
Counter to intent: Voters' mental models of alternative counting methods
 
California's top lessons learned in design and language
California's top lessons learned in design and languageCalifornia's top lessons learned in design and language
California's top lessons learned in design and language
 
Using voter personas to understand who is coming to your election department ...
Using voter personas to understand who is coming to your election department ...Using voter personas to understand who is coming to your election department ...
Using voter personas to understand who is coming to your election department ...
 

Recently uploaded

Morgenbooster: Storytelling in Identity Design
Morgenbooster: Storytelling in Identity DesignMorgenbooster: Storytelling in Identity Design
Morgenbooster: Storytelling in Identity Design1508 A/S
 
The concept of motion graphics and its applications.
The concept of motion graphics and its applications.The concept of motion graphics and its applications.
The concept of motion graphics and its applications.WeatherOfficialProdu
 
Abortion Clinic in Springs +27791653574 Springs WhatsApp Abortion Clinic Serv...
Abortion Clinic in Springs +27791653574 Springs WhatsApp Abortion Clinic Serv...Abortion Clinic in Springs +27791653574 Springs WhatsApp Abortion Clinic Serv...
Abortion Clinic in Springs +27791653574 Springs WhatsApp Abortion Clinic Serv...mikehavy0
 
GBU INDOOR STADIUM CASE STUDY DESCRIBING ITS STRUCTURE
GBU INDOOR STADIUM CASE STUDY DESCRIBING ITS STRUCTUREGBU INDOOR STADIUM CASE STUDY DESCRIBING ITS STRUCTURE
GBU INDOOR STADIUM CASE STUDY DESCRIBING ITS STRUCTURECHANDRAKIRTISINGH
 
Latest Trends in Home and Building Design
Latest Trends in Home and Building DesignLatest Trends in Home and Building Design
Latest Trends in Home and Building DesignResDraft
 
Bit Dhrumi shah Graphic Designer portfolio
Bit Dhrumi shah Graphic Designer portfolioBit Dhrumi shah Graphic Designer portfolio
Bit Dhrumi shah Graphic Designer portfoliodhrumibshah13
 
如何办理(RUG毕业证书)格罗宁根大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(RUG毕业证书)格罗宁根大学毕业证成绩单本科硕士学位证留信学历认证如何办理(RUG毕业证书)格罗宁根大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(RUG毕业证书)格罗宁根大学毕业证成绩单本科硕士学位证留信学历认证ugzga
 
如何办理(UoB毕业证书)伯明翰大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UoB毕业证书)伯明翰大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UoB毕业证书)伯明翰大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UoB毕业证书)伯明翰大学毕业证成绩单本科硕士学位证留信学历认证ugzga
 
Rise of Digital Textiles__Published in PrintAction
Rise of Digital Textiles__Published in PrintActionRise of Digital Textiles__Published in PrintAction
Rise of Digital Textiles__Published in PrintActionVictoria Gaitskell
 
CADD 141 - Puzzle Cube Project - Product Photos
CADD 141 - Puzzle Cube Project - Product PhotosCADD 141 - Puzzle Cube Project - Product Photos
CADD 141 - Puzzle Cube Project - Product PhotosDuyDo100
 
Spring Summer 26 Colors Trend Book Peclers Paris
Spring Summer 26 Colors Trend Book Peclers ParisSpring Summer 26 Colors Trend Book Peclers Paris
Spring Summer 26 Colors Trend Book Peclers ParisPeclers Paris
 
一模一样英国德比大学毕业证(derby毕业证书)本科学历-国外大学文凭办理
一模一样英国德比大学毕业证(derby毕业证书)本科学历-国外大学文凭办理一模一样英国德比大学毕业证(derby毕业证书)本科学历-国外大学文凭办理
一模一样英国德比大学毕业证(derby毕业证书)本科学历-国外大学文凭办理thubko
 
Evaluating natural frequencies and mode shapes.pptx
Evaluating natural frequencies and mode shapes.pptxEvaluating natural frequencies and mode shapes.pptx
Evaluating natural frequencies and mode shapes.pptxjoshuaclack73
 
如何办理(UAL毕业证书)伦敦艺术大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UAL毕业证书)伦敦艺术大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UAL毕业证书)伦敦艺术大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UAL毕业证书)伦敦艺术大学毕业证成绩单本科硕士学位证留信学历认证ugzga
 
如何办理(UW毕业证书)华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UW毕业证书)华盛顿大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UW毕业证书)华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UW毕业证书)华盛顿大学毕业证成绩单本科硕士学位证留信学历认证ugzga
 
NO1 Best Vashikaran Specialist in Uk Black Magic Specialist in Uk Black Magic...
NO1 Best Vashikaran Specialist in Uk Black Magic Specialist in Uk Black Magic...NO1 Best Vashikaran Specialist in Uk Black Magic Specialist in Uk Black Magic...
NO1 Best Vashikaran Specialist in Uk Black Magic Specialist in Uk Black Magic...Amil baba
 
Digital Marketing Company in Bangalore.pdf
Digital Marketing Company in Bangalore.pdfDigital Marketing Company in Bangalore.pdf
Digital Marketing Company in Bangalore.pdfOnecity
 
如何办理(Columbia College毕业证书)纽约市哥伦比亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(Columbia College毕业证书)纽约市哥伦比亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(Columbia College毕业证书)纽约市哥伦比亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(Columbia College毕业证书)纽约市哥伦比亚大学毕业证成绩单本科硕士学位证留信学历认证ugzga
 
And that's about to change! (Service Design Drinks Berlin May 2024)
And that's about to change! (Service Design Drinks Berlin May 2024)And that's about to change! (Service Design Drinks Berlin May 2024)
And that's about to change! (Service Design Drinks Berlin May 2024)☕ 🥧 🚲 Martin Gude
 
如何办理(UCI毕业证书)加利福尼亚大学尔湾分校毕业证成绩单本科硕士学位证留信学历认证
如何办理(UCI毕业证书)加利福尼亚大学尔湾分校毕业证成绩单本科硕士学位证留信学历认证如何办理(UCI毕业证书)加利福尼亚大学尔湾分校毕业证成绩单本科硕士学位证留信学历认证
如何办理(UCI毕业证书)加利福尼亚大学尔湾分校毕业证成绩单本科硕士学位证留信学历认证ugzga
 

Recently uploaded (20)

Morgenbooster: Storytelling in Identity Design
Morgenbooster: Storytelling in Identity DesignMorgenbooster: Storytelling in Identity Design
Morgenbooster: Storytelling in Identity Design
 
The concept of motion graphics and its applications.
The concept of motion graphics and its applications.The concept of motion graphics and its applications.
The concept of motion graphics and its applications.
 
Abortion Clinic in Springs +27791653574 Springs WhatsApp Abortion Clinic Serv...
Abortion Clinic in Springs +27791653574 Springs WhatsApp Abortion Clinic Serv...Abortion Clinic in Springs +27791653574 Springs WhatsApp Abortion Clinic Serv...
Abortion Clinic in Springs +27791653574 Springs WhatsApp Abortion Clinic Serv...
 
GBU INDOOR STADIUM CASE STUDY DESCRIBING ITS STRUCTURE
GBU INDOOR STADIUM CASE STUDY DESCRIBING ITS STRUCTUREGBU INDOOR STADIUM CASE STUDY DESCRIBING ITS STRUCTURE
GBU INDOOR STADIUM CASE STUDY DESCRIBING ITS STRUCTURE
 
Latest Trends in Home and Building Design
Latest Trends in Home and Building DesignLatest Trends in Home and Building Design
Latest Trends in Home and Building Design
 
Bit Dhrumi shah Graphic Designer portfolio
Bit Dhrumi shah Graphic Designer portfolioBit Dhrumi shah Graphic Designer portfolio
Bit Dhrumi shah Graphic Designer portfolio
 
如何办理(RUG毕业证书)格罗宁根大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(RUG毕业证书)格罗宁根大学毕业证成绩单本科硕士学位证留信学历认证如何办理(RUG毕业证书)格罗宁根大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(RUG毕业证书)格罗宁根大学毕业证成绩单本科硕士学位证留信学历认证
 
如何办理(UoB毕业证书)伯明翰大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UoB毕业证书)伯明翰大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UoB毕业证书)伯明翰大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UoB毕业证书)伯明翰大学毕业证成绩单本科硕士学位证留信学历认证
 
Rise of Digital Textiles__Published in PrintAction
Rise of Digital Textiles__Published in PrintActionRise of Digital Textiles__Published in PrintAction
Rise of Digital Textiles__Published in PrintAction
 
CADD 141 - Puzzle Cube Project - Product Photos
CADD 141 - Puzzle Cube Project - Product PhotosCADD 141 - Puzzle Cube Project - Product Photos
CADD 141 - Puzzle Cube Project - Product Photos
 
Spring Summer 26 Colors Trend Book Peclers Paris
Spring Summer 26 Colors Trend Book Peclers ParisSpring Summer 26 Colors Trend Book Peclers Paris
Spring Summer 26 Colors Trend Book Peclers Paris
 
一模一样英国德比大学毕业证(derby毕业证书)本科学历-国外大学文凭办理
一模一样英国德比大学毕业证(derby毕业证书)本科学历-国外大学文凭办理一模一样英国德比大学毕业证(derby毕业证书)本科学历-国外大学文凭办理
一模一样英国德比大学毕业证(derby毕业证书)本科学历-国外大学文凭办理
 
Evaluating natural frequencies and mode shapes.pptx
Evaluating natural frequencies and mode shapes.pptxEvaluating natural frequencies and mode shapes.pptx
Evaluating natural frequencies and mode shapes.pptx
 
如何办理(UAL毕业证书)伦敦艺术大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UAL毕业证书)伦敦艺术大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UAL毕业证书)伦敦艺术大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UAL毕业证书)伦敦艺术大学毕业证成绩单本科硕士学位证留信学历认证
 
如何办理(UW毕业证书)华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UW毕业证书)华盛顿大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UW毕业证书)华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UW毕业证书)华盛顿大学毕业证成绩单本科硕士学位证留信学历认证
 
NO1 Best Vashikaran Specialist in Uk Black Magic Specialist in Uk Black Magic...
NO1 Best Vashikaran Specialist in Uk Black Magic Specialist in Uk Black Magic...NO1 Best Vashikaran Specialist in Uk Black Magic Specialist in Uk Black Magic...
NO1 Best Vashikaran Specialist in Uk Black Magic Specialist in Uk Black Magic...
 
Digital Marketing Company in Bangalore.pdf
Digital Marketing Company in Bangalore.pdfDigital Marketing Company in Bangalore.pdf
Digital Marketing Company in Bangalore.pdf
 
如何办理(Columbia College毕业证书)纽约市哥伦比亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(Columbia College毕业证书)纽约市哥伦比亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(Columbia College毕业证书)纽约市哥伦比亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(Columbia College毕业证书)纽约市哥伦比亚大学毕业证成绩单本科硕士学位证留信学历认证
 
And that's about to change! (Service Design Drinks Berlin May 2024)
And that's about to change! (Service Design Drinks Berlin May 2024)And that's about to change! (Service Design Drinks Berlin May 2024)
And that's about to change! (Service Design Drinks Berlin May 2024)
 
如何办理(UCI毕业证书)加利福尼亚大学尔湾分校毕业证成绩单本科硕士学位证留信学历认证
如何办理(UCI毕业证书)加利福尼亚大学尔湾分校毕业证成绩单本科硕士学位证留信学历认证如何办理(UCI毕业证书)加利福尼亚大学尔湾分校毕业证成绩单本科硕士学位证留信学历认证
如何办理(UCI毕业证书)加利福尼亚大学尔湾分校毕业证成绩单本科硕士学位证留信学历认证
 

Rethinking user research for social design

  • 1. Rethinking User Research for the Social Web Dana Chisnell dana@usabilityworks.net @danachis IA Summit 2011 #rethinking 1
  • 2. Shelly 2
  • 3. The state of the art is a generation old Time for a new generation of user research methods. 3
  • 4. Usability testing isn’t telling us what we need to know for SxD Me, Inc. 4
  • 5. We don’t know how to find out about things we don’t know about 5
  • 6. People don’t live in the world doing one task with one device out of context WebEx - meetings Marriott.com - reserving rooms Me, Inc. - retirement planning eHarmony - coded messages in Libya Facebook & Twitter - organizing demonstrations in Egypt 6
  • 7. 7
  • 8. H+C=I 7
  • 9. 7
  • 10. H <== (computer) ==> H 7
  • 11. 8
  • 12. Methods and measures were probably always imperfect. 8
  • 13. Methods and measures were probably always imperfect. But they used to be enough. 8
  • 14. Methods and measures were probably always imperfect. But they used to be enough. No more. 8
  • 15. Everything is different now. But not different. All this has happened before. 9
  • 16. The nature of being online is social It always has been. email calendaring sharing documents --> Name some online transaction you think is not social. http://www.flickr.com/photos/holeymoon/ 10
  • 17. Scale is a game changer Caution: large sample sizes 20,000 googlers didn’t expose real issues. Quora moderating overburdened Google Buzz http://www.flickr.com/photos/daguerreotyped/ 11
  • 18. Tasks aren’t what you think Activities = goals that emerge and change Effectiveness Efficiency ... aren’t meaningful http://www.flickr.com/photos/jeremybrooks/ 12
  • 19. Satisfaction is correlated with task completion Instead: control, engagement But how do you measure control and engagement? What does engagement mean? 13 http://www.flickr.com/photos/litebriteneonstudio/
  • 20. Users How do you design effective tasks in this case? /cc @ continuously product reviews that turn into literary projects design your UI Using comments in Facebook for a conversation Hey, where did everybody go? - user-created help page to describe in real time a hack to get around a feature they don’t like Coded messages in Egypt on dating sites Workarounds = hacks Etiquette and norms http://www.flickr.com/photos/juleshabib/ 14
  • 21. And we have to be careful not to violate either of those things: Etsy Circles Facebook engagement ring story 15
  • 22. Social is about context. And we have to be careful not to violate either of those things: Etsy Circles Facebook engagement ring story 15
  • 23. Social is about context. And relationships. And we have to be careful not to violate either of those things: Etsy Circles Facebook engagement ring story 15
  • 24. Methods are not robust for understanding context and relationships. 16
  • 25. Cultivate polymaths Decision sciences Communication Anthropology sciences Psychologies Sociolinguistics Behavioral economics Organizational behavior Psychobiology Sociology Social neuroscience Social networks 17
  • 26. The most successful user research methods for SxD combine field and testing techniques 18
  • 27. 19
  • 28. Methods 19
  • 29. Methods Reviewing online profiles, connections in groups in interviews 19
  • 30. Methods Reviewing online profiles, connections in groups in interviews Stories of how you met 19
  • 31. Methods Reviewing online profiles, connections in groups in interviews Stories of how you met Multi-user sessions with people who have strong and loose ties 19
  • 32. Methods Reviewing online profiles, connections in groups in interviews Stories of how you met Multi-user sessions with people who have strong and loose ties Video diaries with retrospective review 19
  • 33. Methods Reviewing online profiles, connections in groups in interviews Stories of how you met Multi-user sessions with people who have strong and loose ties Video diaries with retrospective review Experience sampling with SMS 19
  • 34. Takes more time recruiting, prep, session, analysis 20
  • 35. Takes strong research design rather than templated, commodity usability testing 21
  • 36. Takes deep thinking about questions to answer, and what the evidence will be that you have answers 22
  • 37. Takes study of cohesiveness collaboration, connections, networks, relationships Strong and weak ties Dunbar 23
  • 38. Where are you with planning? what we asked answered a business question 24
  • 39. Who do you ask when you don’t know? How do you make financial decisions was what we wanted to know - which is about context and relationships 25
  • 40. Would have told us Who is part of planning and deciding Who is trusted Why these people are important Why they are trusted How planning happens 26
  • 41. To build a model that supports 27
  • 43. Rethinking user research We’re not getting the answers we need. 28
  • 44. Rethinking user research We’re not getting the answers we need. Experimenting is limited because we’re pressured to go to market. 28
  • 45. Rethinking user research We’re not getting the answers we need. Experimenting is limited because we’re pressured to go to market. We’re looking for things we know about. 28
  • 46. We’re missing things we don’t know about. 29
  • 47. It’s time to rethink user research for the new (old) social web 30

Editor's Notes

  1. \n
  2. \nThe story of the session. \nEnding: she and her father would have done this together. \n\nThis tool is a piece of the social web \n\nNothing unusual about this test. \none person, one computer\npretend to do a task \n\n
  3. In use at least 20 years \n\nMethod failed Shelly desperately \n\nStarted thinking about this with Buzz\nFTC\n\n
  4. \nBuzz testing \n\nAfter I blogged about it... \n\n
  5. We&amp;#x2019;re not sure how to do user research that will tell us what we need to know about relationships online\n Something&amp;#x2019;s not working, and I think it&amp;#x2019;s this: \n
  6. People simply do operate in real life the way we ask them to behave in usability tests. \n\nWebEx - observe meetings? No \n\nMarriott.com - take into account the others who decide what hotel? No \n\nRetirement planning tool - husbands + household expenses; wives + children about healthcare decisions \n\nEgypt and Libya, using eHarmony and Facebook to pass coded messages: Who knew a social network would *ever* be used that way? \n
  7. \n(talk) \n
  8. \n(talk) \n
  9. \n(talk) \n
  10. The methods were probably *always* less than perfect. \n Considering SxD magnifies the imperfections of user research methods\n \n
  11. The methods were probably *always* less than perfect. \n Considering SxD magnifies the imperfections of user research methods\n \n
  12. The methods were probably *always* less than perfect. \n Considering SxD magnifies the imperfections of user research methods\n \n
  13. \nSeems that with ambient technology, everything has changed. \n\nBut it hasn&amp;#x2019;t. \n\nThere are 5 factors to user research that I&amp;#x2019;m convinced we should rework, rethink to avoid things like the Buzz and Etsy privacy violations. \n
  14. \nAnything one person does to change the behavior of another is social. \n\nName one thing you can do online that isn&amp;#x2019;t social. \n
  15. \n From one person + one computer... \n Large sample sizes won&amp;#x2019;t tell you everything you need to know. The Google 20,000 didn&amp;#x2019;t reveal the privacy problem. Why not? \n Looking at scale another way, let&amp;#x2019;s think about Quora and it&amp;#x2019;s massively user-generated IA. How do you design for that? How do you test for that? How do you do user research that will tell you how to go? \n
  16. Usability testing and user research methods use tasks - but users don&amp;#x2019;t come to technology with task goals in mind. \n ephemeral, changeable + context, timing, space, what just happened, what happens next\n \n Tasks are probably activities where goals emerge and change depending on context, timing, space, etc. \n
  17. This is probably the wrong measure. \n We should look at control by the user and engagement. People want to be the boss of their data. Buzz and Facebook&amp;#x2019;s Beacon, and now Etsy&amp;#x2019;s changes take that control away. \n We need to take a hard look at conventional metrics. \n What is effectiveness and efficiency with Twitter and Facebook open all day? What&amp;#x2019;s the task? \n Measures of success don&amp;#x2019;t translate to SxD\n time?\n task completion? \n success? \n
  18. People use your UI in ways you didn&amp;#x2019;t intend, but that help them do what they want to do.\n Individual workarounds now surface to everyone in a network. \n Etiquette and social norms adjust over time and scale. \n Design is effectively open sourced to the user community\n
  19. Onlineness, ambient technology is about context and\n\nwho is zooming whom \n\nWe must not violate the context of the relationships. Otherwise, problems. \n\nClients learn embarrassing purchases - Etsy\nGirlfriend learns source of engagement ring - Facebook\n
  20. Onlineness, ambient technology is about context and\n\nwho is zooming whom \n\nWe must not violate the context of the relationships. Otherwise, problems. \n\nClients learn embarrassing purchases - Etsy\nGirlfriend learns source of engagement ring - Facebook\n
  21. Something built for HCI doesn&amp;#x2019;t cover the expanded view of this one person interacting online and the implications of that. \n\nLab testing &amp; structured field visits are lacking. \n
  22. \n
  23. Borrow, borrow from, combine. \n\nI interviewed smart people.... \n\nThis is what they told me: The key is combiningmethods. \n
  24. \nMet: Twitter road trip\n
  25. \nMet: Twitter road trip\n
  26. \nMet: Twitter road trip\n
  27. \nMet: Twitter road trip\n
  28. \nMet: Twitter road trip\n
  29. \nMet: Twitter road trip\n
  30. \n
  31. \n
  32. \n
  33. \n
  34. And so, we&amp;#x2019;re back in the lab in Waltham. \n\nShelly is crying as she looks out at her future self. \n\nWe asked a business question. \n
  35. \nThe business would have been better served by our asking more about Shelly. \n\nWhat&amp;#x2019;s the right question? \n
  36. \n
  37. \n
  38. With our generation old methods, we really only know how to find things we&amp;#x2019;re looking for. \n\n\n
  39. With our generation old methods, we really only know how to find things we&amp;#x2019;re looking for. \n\n\n
  40. With our generation old methods, we really only know how to find things we&amp;#x2019;re looking for. \n\n\n
  41. We don&amp;#x2019;t have good ways of learning about things we don&amp;#x2019;t know about. \n
  42. \nTime to rework the old standard methods into new remixes. \n
  43. Thank you. \n