Survey Research in Design
•Download as PPTX, PDF•
10 likes•3,156 views
Presentation to the Master's of Design students at Ontario College of Art and Design, Oct. 14, 2010.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to Survey Research in Design
Similar to Survey Research in Design (20)
More from Sam Ladner
More from Sam Ladner (20)
Recently uploaded
Recently uploaded (20)
Survey Research in Design
- 1. Using Surveys in Design Sam Ladner, PhD October 14, 2010
- 2. Key Themes When to use surveys in design Qualvs quant Sampling From interviews to instruments Survey basics Should we do a survey? Asking good questions The limits of surveys
- 4. Complexity
- 5. Compromise
- 6. ChoiceThe Four Cs ofResearch Design BorjaDe Mozota, Brigitte. 2003. Design Management: Using Design to Build Value. New York: All Worth Press.
- 8. What research design would you choose?
- 9. Quantitative Research in Design
- 10. What do quant researchers worry about? I really spend a lot of time wondering how to measure things. I want to make sure others can repeat my findings. I want to know what causes something else. I wonder how small patterns generalize to big patterns.
- 11. What do qual researchers worry about? I really want my research approach to be flexible and able to change. I want to describe the context in a lot of detail. I want to see the world through the eyes of my respondents. I want to show how social change occurs. I’m interested in how things come to be.
- 13. What is your orientation?
- 14. A famous sampling mistake
- 15. Even with proper sampling…beware! “…predicting behavior on the basis of knowledge of attitude is a very hazardous venture.”
- 16. What exactly IS a “sample”?
- 17. What exactly IS a “sample”? A subset of the population, selected by either “probability” or “non-probability” methods. If you have a “probability sample” you simply know the likelihood of any member of the population being included (not necessarily that it is “random”).
- 18. Assumptions of quantitative sampling We want to generalize to the population. Random events are predictable. We can compare random events to our results. Therefore… Probability sampling is the best approach.
- 19. Assumptions of qualitative sampling Social actors are not predictable like objects. Randomized events are irrelevant to social life. Probability sampling is expensive and inefficient. Therefore… Non-probability sampling is the best approach.
- 20. What kind of sample would you use for a design project?
- 21. From interviews to instruments… “concepts are…categories for the organization of ideas and observations” (Bulmer, 1984: 43)
- 23. What is varying?
- 25. What is a measure? Something that can be counted. Age Height Number of books
- 26. What is an indicator? Something that “indicates” the presence of something else. GDP IQ Number of hospital visits
- 27. What is survey research? Asking structured questions Print or Online surveys Telephone surveys Structured face-to-face interviews
- 28. Surveys versus other qualitative methods
- 29. Is a survey the right research method? Yes No I know a lot about the topic I need to summarize the findings in numerical format I don’t need a lot of detail and nuance I need to generalize the results to the population I need to predict the likelihood of a certain thing happening I want to measure incremental change I don’t really know much about the topic I don’t need to summarize the findings in numerical format I need to communicate a lot of detail and nuance I’m interested in case study information I’m not planning on predicting the likelihood of anything I don’t need to measure any change
- 30. Is it time to do a survey? Clear idea of all possible questions and answers? Formulated all questions and possible answers? Yes Do a survey Yes No Pilot test the questions Yes No Discovered questions and answers through Qual research? Do more Qual research No
- 31. Asking good questions Mutually exclusive and exhaustive No overlap between answers All possible answers No double-barreled questions Wrong: “What do you think about our response times and customer service?” Right: “What is your opinion of our response time?” Focus on opinions or beliefs of participants Wrong: “How usable is our Web site?” Right: “What are users’ opinions of the site’s look and feel?” Close-ended is best Cuts down on analysis time Demographic questions at the end
- 32. The lay of the land: descriptive statistics Summarizes the responses What is the most “typical” response? Mean Mode Median How much variation is there between responses? Standard deviation Inter-quartile range
- 33. Predicting behavior: inferential statistics Infers or predicts future behavior Which respondents are more likely to… Cross-tabulation Which factors most influence Behavior Satisfaction etc
- 34. The tools you need Online Survey Tool Statistical Analysis Software
- 35. The skills you need Knowledge of the topic Ability to write good questions Familiarity with the survey tool Some basic knowledge of statistics A framework for writing the report Important to write the outline of the final report before you start!
- 36. The payoff you’ll get “More men are dissatisfied with the experience.” “Younger users are much more unhappy with customer service that older users.” “Higher-income individuals are the most unlikely to recommend this product to friends and family.” “Men are more likely to prefer a two-step process than women.”
- 37. What’s wrong with quant research? Yeah and they treat people like they’re test tubes or something. It’s the way they do things…it makes it hard for people to see their research as relevant to them. Well their research is just so static. Real life actually changes. They always seem to measure artificial things and say they’re really precise.
- 38. What won’t you learn from your survey?
- 39. What you won’t learn… Why aren’t people using our product? How can we innovate on our current design? Why don’t people finish the checkout process? What are the main characteristics of the people who don’t use our product?
- 40. Copernicus ConsultingDesign research and strategy http://www.copernicusconsulting.net
Editor's Notes
- Hello everyone.Thank you for having me here today. Today we’re going to talk about surveys as a design research method. I know you’ve read Lecompte and Schensul and it gives you an idea about structured questioning in the ethnographic context.I’m going to focus more closely on surveys in the classic sociological tradition for a particular reason. Surveys are a staple of social research. They are an essential tool for understanding how to do social research. In order to understand the differences between surveying and, say, semi-structured interviewing, we need to study the classic survey form. So that’s what we’ll do today.
- So this is a brief overview of what I’ll be talking about. It’s a crash course in research methodology, really. Note I say “methodology” and not “method.” Today we are studying a method, surveys, but also the study of that method, which is methodology. Specifically, I want to make this relevant to design. A lot of social scientists aren’t close enough to the design process to really understand what designers need, so I’ve tried to customize for that. Also, I take a page from Roger Martin’s The Design of Business, and I’ll show you how to build the “analytical” language of indicator systems, something you will need in your business roles.
- So we’ve just learned that there is more to the story. Once you’ve created a product, there is a different set of questions you ask than before you create a product. Turns out, there is a general approach to choosing the right research method for the right phase in the design process. These methods fit along this continuum. Early on in the research process, you want to understand what’s going on. You want to have what Weber called “verstehen.” What is it like to work in an office? Who is in an office? What role do chairs play, symbolically, ergonomically, in the work process? This is what the Aeron chair designers did. They visited a lot of offices. They “hung out.” They began to notice that people would squirm a lot in their existing chairs. Why do they do this? They wondered. It was the fabric that was too hot, they concluded. They theorized they could make a much more comfortable chair, if they got rid of the fabric and let the chair breathe. So that’s what they did. They created the strange looking chair. And as was the norm at Herman Miller, they went to validate the chair in focus groups.Some of you may know that the focus groups widely rejected the designers’ theory. They didn’t like the chair at all. They found it ugly. They thought it was “unfinished.” They did not actually USE the chair in the real context. The focus group likely was not the right way to do a validation. But I want to focus on the last bit there, the optimize section. This is where quantitative measures from surveys are the best. I know the readings suggest that you can do “qualitative” surveys, but the best way to use surveys is quantitative. And later I’ll show you.
- Ok so now that we know how we see reality and how we want to sample. Let’s go back to the research process that Aeron chair designers used. They first went out and observed people on context. Then they started theorizing about the temperature of the chairs. They created a design and validated it using focus groups, but we think they may have used the wrong method. Let’s now talk about creating surveys. Basically, we go through exploratory research in order to create concepts. These are the organizing buckets of all our observations. And they form the basis of our surveys. But a concept differs from a variable, which is what you measure in surveys.
- Basically it comes down to the “whys” of design. If you have a why question, you won’t answer it in a survey because a survey is not an open-ended method!