Design Research at CHI and its Applicability to Design Practice
•Download as PPTX, PDF•
3 likes•796 views
This note describes our analysis of 35 papers from CHI 2011 that aim to improve or support interaction design practice. In our analysis, we characterize how these CHI authors conceptualize design practice and the types of contributions they propose. This work is motivated by the recognition that design methods proposed by HCI researchers often do not fit the needs and constraints of professional design practice. As a complement to the analysis of the CHI papers we also interviewed 13 practitioners about their attitudes towards learning new methods and approaches. We conclude the note by offering some critical reflections about how HCI research can better support actual design practice.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Design Research at CHI and its Applicability to Design Practice
Similar to Design Research at CHI and its Applicability to Design Practice (20)
Recently uploaded
Recently uploaded (20)
Design Research at CHI and its Applicability to Design Practice
- 1. Design Research at CHI and its Applicability to Design Practice David Roedl & Erik Stolterman School of Informatics and Computing Indiana University, Bloomington
- 3. HCI research IxD practice
- 4. HCI research IxD practice theories frameworks methods tools
- 5. HCI research IxD practice Often, methods and approaches do not fit the needs and constraints of practice. e.g. Rogers, Y. (2004) New theoretical approaches for human-computer interaction. Annual review of information science and technology, 38(1), 87– 143.
- 6. analysis of 35 papers from CHI „11 interviews with 13 interaction designers conceptualizations of and contributions to design practice attitudes towards learning about new methods and approaches HCI research IxD practice
- 10. 35 papers oriented toward supporting design practice
- 11. 433 focused on specific systems, technologies, or use domains
- 12. Analysis: how is practice addressed in…
- 13. Analysis: how is practice addressed in… • Conceptualization, i.e. definition of key issues
- 14. Analysis: how is practice addressed in… • Conceptualization, i.e. definition of key issues • Operationalization, i.e. method of inquiry
- 15. Analysis: how is practice addressed in… • Conceptualization, i.e. definition of key issues • Operationalization, i.e. method of inquiry • Generalization, i.e. implications for whom?
- 16. Issue 1 Over-generalization of design situations
- 17. Issue 1 Over-generalization of design situations Few papers distinguish among contexts in which design takes place, e.g. : • commercial product development • corporate innovation • academic design as research
- 18. Issue 2 Over-emphasis on a single design activity without discussion of how each fits into a broader process or how its relative importance might vary across projects and contexts.
- 19. Issue 2 Over-emphasis on a single design activity without discussion of how each fits into a broader process or how its relative importance might vary across projects and contexts. e.g. contextual user research, concept exploration, usability evaluation
- 20. Issue 3 Lack of attention to practical challenges e.g.: • limited time and resources • group decision-making
- 21. A counter-example: Gaver (2011): “This reflects our use of workbooks at the outset of open-ended research through design projects… In a commercial setting, however, such workbooks… could be useful in establishing an ongoing sense of group identity, direction and style”. Gaver, W. (2011). Making spaces: how design workbooks work. In Proc. CHI 2011 (pp. 1551–1560). ACM Press.
- 23. Interview study • 13 professionals, 12 different companies • IxD, UxD, user research • 3-10+ years of experience
- 24. Interview study • What methods are most important? • How do practitioners learn about new methods? • What are practitioners‟ attitudes towards the CHI conference?
- 25. What methods are most important? Collaboration with stakeholders • e.g. workshops for brainstorming and building consensus around problems and goals Fluid adaptation of methods based on specific situation
- 26. What methods are most important? Collaboration with stakeholders • e.g. workshops for brainstorming and building consensus around problems and goals Fluid adaptation of methods based on specific situation
- 27. How do practitioners learn new methods? • Co-workers, social networks, twitter, blogs • Professional conferences (e.g. IxDA, IA Summit, UPA) • Improvisation, trial and error
- 28. Attitudes towards CHI • For academics rather than practitioners • Not enough relevance to day-to-day work • Too “dry” and lacking in good storytelling
- 30. Practitioners are seeking ways to improve their practice. They respond to knowledge that resonates with their day-to-day experience.
- 31. Research implications that are clearly situated in relation to the complex challenges of practice may be more likely to reach practitioners.
- 32. The relationship between research and practice is complex and not always clearly defined. We believe this is an important area for continued investigation and debate.
- 33. Thanks! HCI/d @ Indiana University Marty Siegel, Colin Gray, Omar Sosa Tzec, Nathan Bilancio, Jeff Wain uxdesignpractice.com NSF
- 35. Sources of IxD methods • Design consultancies (e.g. Cooper, IDEO, frog) • Large companies • Individual practitioners • Academic research
- 36. Contribution types 7 Inspiration/Theory 11 Learning/Methodology 17 New Tools & Methods
Editor's Notes
- Good afternoon. I’m David and I’m presenting on behalf of my coauthor Erik Stolterman.Our note is titled Design Research at CHI and its Applicability to Design Practice
- I will start off today by briefly summarizing the motivation for our research.
- This work is motivated by a desire to better understand the relationship between HCI research, as an academic community, and Interaction design, as a professional field of practice.
- We begin with the observation that within HCI, a considerable amount of research is concerned with supporting design practice, by for example, proposing new methods, techniques, and tools that can be applied in a design process.
- However, it has been argued that much of this research output is never fully adopted or put to use by practitioners because it does not fit the needs and constraints of professional design situations.For example, in a 2004 paper, Yvonne Rogers presented a survey study in which she found that many designers were aware of, but did not make use of leading theories in HCI. She argued that the reason is that in many cases the results of research are too abstract, too complex, too difficult to learn and take too much time to use. ------Erik has also argued that a much of this research is based on an overly simplistic understanding of practice, and as a result, the proposed methods and tools do not fit the complexity of real design work.------------ Meeting Notes (4/24/13 14:24) -----too static
- To explore these issues further, we decided to engage in two studies.First we decided to analyze research papers from a recent CHI conference.We focused especially on the way researchers frame their contributions in relation to practice.Second, we conducted interviews with 13 practitioners about their attitudes towards learning new methods and approaches.
- So with that introduction, I will now dive into discussing the paper analysis study and its findings.
- Since CHI is a broad and major conference in the field, we decided to use the proceedings from CHI 2011 as our data.This is just a snapshot of one conference, and of course is not a fair representation of the whole field. However, we saw this as a good starting point in the largerexamination of the production of support for practice.
- In order to identify relevant papers we first read through the entireconference program including each paper’s title and short contributionstatement. We looked in particular for contributions that were orientedtowards improving or supporting design processes in someway.
- In total, we found 35 papers that met this criteria.
- The remaining 433 papers that were not selected were those mostly focused onspecific systems, technologies, or use domains, rather thanon practices of design.---This is of course not say to that these have no implications for design.In fact most of them probably do. but they did not explicitly address design practice as a primary contribution.
- Next, we read and analyzed each of the 35 papers.Our focus in doing so was on how each paper addressed practice in three ways:
- First, Conceptualization, or the definition of key issues,
- Second, operationalization, or the method of inquiry and argument.
- And Third, generalization, or for whom do the implications apply?Through this analysis, we identified 3 potential issues in the way research was being framed.
- First, we found that papers commonly engaged in what we call an over-generalization of design situations.
- By this we simply mean that very few papers made any distinction among the various contexts in which design takes place, such as commercial product development, corporate innovation or design as research in an academic setting. Because of this, it was often unclear if the contributions were intended to benefit professional practice, academic research, or both.
- A second, closely related issue is that many papers focused on a particular type of design activity, without discussing how it fits into a broader design process, or how its relative importance might vary across projects and contexts.----…For example, several authors proposednew methods that were derived from their own projectsdoing research through design, in an academic or corporateinnovation context (e.g. [3,6]). These projects can becharacterized by a focus on broad concept exploration,rather than on creating specific products for commercialuse. Not surprisingly, many of these papers focused onmethods for supporting design exploration (divergence)rather than synthesis (convergence). However these papersdid not attempt to argue or provide any evidence thatexploration is where practice is in most need of support.
- The design activities that received the most study among the papers were contextual user research, concept generation, andusability evaluation.----…For example, several authors proposednew methods that were derived from their own projectsdoing research through design, in an academic or corporateinnovation context (e.g. [3,6]). These projects can becharacterized by a focus on broad concept exploration,rather than on creating specific products for commercialuse. Not surprisingly, many of these papers focused onmethods for supporting design exploration (divergence)rather than synthesis (convergence). However these papersdid not attempt to argue or provide any evidence thatexploration is where practice is in most need of support.
- The third pattern we observed was a lack of consideration for practical challenges in design work, such as working within limited time and resources, and building consensus within a group ororganizational setting. -----For example, Dow et. al [1]conducted an experiment to demonstrate the benefits ofprototyping multiple designs in parallel. In order tocarefully standardize and control their experiment, theauthors removed many of the social dynamics that aretypically present in design practice. Thus, the experimentdid not include the task of establishing group consensus inorder to select among alternatives and converge on a singledesign. The lack of attention to this process may be aconsequence of the fact that in research, the researchersusually have ownership in deciding the final outcome. Theydo not have to engage in persuasion and consensus-buildingin the way that is experienced by practitioners. In contrast,we found two papers [4,5] that focused specifically oncollaboration in the design process. Notably, these wereboth case studies from industry practitioners.
- As way of illustrating, we’d like point to Bill Gaver’spaper on design workbooks, which stands out as a notable counter-example to all 3 issues:In the paper, Gaverreflects on the relative usefulness of the workbook method within various stages of commercial versusacademic design work.And he also acknowledges the influence of production schedules and social dynamics among a design team.We mention this paper because it was one of the only papers in our corpus to frame its contributions in these ways.
- Now, I will move on to discussing our second study which consisted of interviews with interactiondesign professionals. The reason for this study was to find a way to contrast the perspective of designers with the ways that academics are engaged in providingsupport for design.
- We interviewed 13 practitioners from 12 companies, all of whom worked with in the area of interaction design, userexperience design, or user research.The participants worked in a variety of industries and had professional experience ranging from 3 years to over 10 years. ----The practitioners alsohad different educational backgrounds. Of the 13interviewees, 8 had an HCI-oriented education, 3 had atechnical or engineering background, and one had a visualdesign background. Five of the interviewees were female,while the remaining eight were male.
- Each interview consisted of a 30-60 min semi-structured conversation. The open-ended prompts were centered on 3 main topics:What types of methods do practitioners find most important in their work?How do practitioners learn about new methods and approaches, or otherwise keep themselves up to date with knowledge in the field?What are the practitioners’ attitudes towards the CHI conference?I will briefly summarize what we learned about these questions.
- The majority of participants put a strong emphasis on methods for facilitating collaboration among stakeholders outside the design team, such as clients, developers, and internal business stakeholders. Many participants spoke of organizing collaborative workshops, brainstorming or sketching sessions in order to build consensus around the goals and problems to be solved in a given project.
- Another pattern throughout the interviews was that participants frequently talked about adapting and combining methods to suit the needs of the particular situation, ratherusing them in the “right” way or “traditional” way.
- The majority of our participants said that they actively seek information about new design approaches in order to staycurrent with best practices in the field.The most commonly mentioned sources of learning were coworkers, Twitter feeds, blogs, online magazines, and professional conferences like IxDA, IA Summit, UPA, and UX Week.Also, a few of the practitioners emphasized the value of improvising new techniques based on the situation at hand, and in learning to adapt methods based on the experience of trial and error.
- When asked about their perceptions of CHI, nearly all stated that they felt it was a conference for academics and graduate students, and is not appropriate for practitioners. A few participants commented that the content at CHI was “too theoretical” and did not provide enough practical value or relevance to day-to-day work.Several participants also said that they found presentations at CHI to be “dry” and lacking in good “storytelling”, or “personal connection.” ---All of our participants, except for one, were aware of the CHI conference and several said that they had attended at one time in the past.One designer mentioned that CHIpresentations were “esoteric” and “deep into very specifictopics” of which he is not interested. In contrast, anotherparticipant stated that he preferred practitioner conferences,like IxDA Interaction because they were more “big picture,holistic, and touchy-feely”, and helped him think about thewhole field in a new way.
- I will wrap up with a few reflections about what we have learned in these two studies.
- Practitioners are seeking ways to improve their practice. They respond to knowledge that resonates with their day-to-day experience.
- Research implications that are clearly situated in relation to the complex challenges of practice may be more likely to reach practitioners.
- Finally, our studies show that the relationship between research and practice is complex and not always clearly defined. We believe this is an important area for continued investigation and debate. And we hope that with further research we will be able to more elaborately explore and explain this relationship.
- In closing, we would like to thank all ourinterview participants and our collaborators at Indiana University.Thank you all for your attention.
- New approaches, methods, and tools for interaction designare created and developed by many in diverse contexts andfor different purposes. There are for instance some largerdesign consultancies that are constantly engaged in thedevelopment of new methods and techniques as part of theirbusiness model. There are also larger companies that havedeveloped company specific process models and methodsto streamline and control their internal design work. Lesscommon, but still existing are individual practitioners whoformulate and frame their experiences of best practice intomore codified formats. Some of the most advanced attemptsin this group can be found in professional conferences andworkshops (such as IxDA, IA Summit, UPA, UX Week). However, perhaps the most common source for newapproaches and methods is the academic researchcommunity.
- We also tried to characterize the general type of knowledgecontribution offered by each paper. For example, somepapers are fairly abstract in that they propose meta-levelconcepts or theories for understanding interaction design ina new way. Others papers are more concrete in that theyoffer specific recommendations for how to use a tool ortechnique in the design process. As we read the papers, wetook note of these different types of contributions andsorted the papers accordingly.We organized the 35 papers into three categories based oncontribution type. First, at the most abstract level, which welabeled Inspiration/Theory , we found seven papers thatintroduce new theoretical frameworks or provocativeconcepts intended to provide new ways of thinking aboutdesign. Second, we found 11 papers that examined anexisting design method and evaluated, modified, ordeveloped it in some way. We labeled this groupLearning/Methodology. Finally, we found 17 papers thatintroduce new design methods, tools, or approaches that arewell-defined and intended to be applied in a defined andspecific way. We labeled this group as New Tools andMethods .