User Research: Big R meets little R (a Richmond Design Lightning Talk)
•Download as PPTX, PDF•
1 like•5,613 views
These slides were presented as a lightning talk during the August 2018 Richmond Design Meetup. This piece was distilled from a blog post that I authored in April 2018. It can be found at https://makinguxbetter.com/2018/04/25/a-coming-together-big-r-little-r/
Recommended
Recommended
More Related Content
Similar to User Research: Big R meets little R (a Richmond Design Lightning Talk)
Similar to User Research: Big R meets little R (a Richmond Design Lightning Talk) (20)
Recently uploaded
Recently uploaded (20)
User Research: Big R meets little R (a Richmond Design Lightning Talk)
- 1. “User Research” Coming Together Big R meets little r Ashley Cook makinguxbetter@gmail.com August 2018 Photo Credit: User Research @ Periscope from gdsteam on Flickr
- 3. THE WORK big “R” researchers • “scientists” • GOAL: further understanding of how the human brain and body functions within systems of technology • broad generalizable knowledge • rigorous and responsible • performance tied to publication little “r” researchers • “designers” • GOAL: understand user needs to produce wildly innovative solutions to practical real-life problems • very specific context of application • responsive and relatable • performance tied to business KPIs How can we make this work more impactful? How might we make this more credible?
- 4. THE WORK “We need to know what our leadership expects of our research… They want metrics for progress and impact. They clearly want faster results. They want bolder goals… greater innovation…” David Atkins, M.D., M.P.H. Director of HSR&D Acting Deputy Chief Research and Development Officer Veterans Health Administration “I think it only a matter of time before organizations start only hiring certified UX designers.” Jared Spool @jmspool Maker of Awesomeness at @CenterCentre/@UIE
- 5. What can we little “r” researchers do to close the gap? • practice literature review & consultation when you design • partner with Big “R” researchers (find them & get involved) • cite your sources (when you design & blog) & make your work easily citable (include publication dates or a formatted citation) • thoughtfully select your user research methods • be responsible when you share “statistics” (any number!) • strategize for longer term & publishable results
- 6. thoughtfully select your user research methods
- 7. thoughtfully select your user research methods what people do what people say how many & how much why & how to fix Christian Rohrer © 2014: When to Use Which User-Experience Research Methods “Each dimension provides a way to distinguish among studies in terms of the questions they answer and the purposes they are most suited for.” Christian Rohrer, Ph.D. Vice President of Design, Research and Enterprise Services at Capital One
- 8. thoughtfully select your user research methods There are others… “Thank you everyone for fabulous input and references for the map!” Indi Young @indiyoung Co-founder of @AdaptivePath “User research can be done at any point in the design cycle. This list of methods and activities can help you decide which to use when.” Susan Farrell UX Research and Strategy Lead
- 9. strategize for longer term & publishable results Photo Credit: MumsLounge.com Photo Credit: HealthCore.com
- 10. strategize for longer term & publishable results Iteration Zero Sprint 1 Sprint 2 Sprint 3 Development “Track” UX “Track” Plan & Gather Customer Data Design for Sprint 2 Research for Sprint 3 UT Sprint 1 Code Design for Sprint 3 Research for Sprint 4 UT Sprint 2 Code Design for Sprint 4 Research for Sprint 5 Implement high dev-cost low UI-cost features Implement Designs Implement Designs Data Data
- 11. strategize for longer term & publishable results big “R” research community benchmarking studies UX KPI tracking marketing & product requested studies publishable insights
- 12. Photo Credit: Highland Expeditions “You do not climb a mountain like Everest by trying to race ahead on your own, or by competing with your comrades. You do it slowly and carefully, by unselfish teamwork.” - Tenzing Norgay, Nepali-Indian Sherpa mountaineer, one of the first two individuals known to reach the summit of Mount Everest.
- 13. more reading Cook, A. (2018, April 25). A Coming Together: Big “R” & little “r”. Retrieved from https://makinguxbetter.com/2018/04/25/a-coming-together-big-r-little-r/ Rohrer, C. (2014, October 12). When to Use Which User-Experience Research Methods. Retrieved from https://www.nngroup.com/articles/which-ux-research-methods/ Saleem, J. J., Militello, L. G., Russ, A. L., & Wilck, N. R. (2016). The need for better integration between applied research and operations to advance health information technology. Healthcare, 4(2), 80–83. Sanders, E. (2017). Design Research at the Crossroads of Education and Practice. She Ji, 3(1), 3–15. Tullis, T. & William, A. (n.d.). Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. More information available at http://measuringux.com/. VCU Libraries Research Guide. (n.d.). Your First Literature Review. Retrieved from https://guides.library.vcu.edu/lit-review.
Editor's Notes
- Hello, my name is Ashley Cook and I have the pleasure of sharing some ideas about “user research” with you all today.
- Over the past 5 years, I’ve been leading a team of UX researchers supporting the Veterans Health Administration. They are VERY academically minded at the VA and we were working on a daily basis with academic researchers to do UX type work. I had been working with them to do a JM project and was invited to present a poster on it at an academic conference last summer. No one who visited my poster even knew what a journey map was, let alone the power in building and using them. I had to start explaining them as “timeline based visualizations of data about people’s interactions with technology.” Just being there, it became painfully clear to me that a cultural rift existed between me and these people. Throughout the several days at that conference, I repeatedly heard messages about their challenges to share their data and make an impact faster. It was amazing to me and was professionally life changing. I can't stop thinking about the idea that these two populations have the answers to each other's problems. I see this as an opportunity to come together so both groups can make more of a difference in making UX better and that’s what I want to talk about today. I hope that this idea resonates with you and you will walk away with some practical ideas on how to start to bridge the gap.
- So here’s what this looks like in my head: [»] You have folks like those at that conference, the “Big R” researchers. These folks are academics and are very serious about their work in contributing to the academic body of knowledge. [»] Then, you have folks who are more like me, and I assume are more like the folks in this room, the “little r” researchers. We are practical industry folks who are very serious about making an impact. At both ends of the spectrum, we are working towards the same goals, but from very different perspectives. [»] And, we have the strengths that the other side lacks. [»] The trick is that in order to make the most impact, most of the work needs to be done somewhere in the middle.
- We both have to come together to get the work done better. Super smart folks on both sides of the continuum have been talking about this need. [»] This is Dr. Atkins, director of the organization that manages VA’s multi-million dollar research budget. He gave the opening plenary at this conference. The entire theme of his talk was around what UX’ers do as their bread and butter. [»] And, here’s a recent tweet from Jared Spool, who set up a school to create highly and practically educated UX’ers. He’s not the only thought leader in our field talking about how we need to bring more credibility to our work force. I see a lot of conversation happening but not a lot of collaboration happening to come together.
- So, here are some ideas on how we can help start bridging the gap, from the “little r” end: [»] Don’t depend on a hunch when you are designing. Do your homework: literature consultation Join Mendeley Join an organization with access to journals See VCU “how to” reference at the end of this deck [»] Academic researchers cluster around research topics. When you do your homework, keep track of folks who are studying what you are using. Reach out to them. Their contact info is often published with the papers. Make friends and scratch each other’s backs. [»] Do a better job of citing your sources when you use them (in your designs or in blogs). Have you ever read an awesome blog and thought, “damn, I wish they had linked that statistic” or “I sure would like to read more on that topic.”? Stop the madness! [»] We all are now starting to understand that we can’t depend on “best practices” for everything and we need to do “user research”. But, there are lots of different kinds. Be careful that you are picking the right method for the question you are trying to answer. For example, you cannot UT your way out of an information need that requires observation of the user’s environment. [»] For God’s sake, learn the basics of statistics and don’t misrepresent your quantitative data. This is the #1 thing that drives Big R researchers nuts. If you have a cool stat, make sure you are also sharing your error rate and population size. [»] Keep track of your research questions and put the pieces together to publish. [»] Because we’re striking like lightning today, I’m only going to focus on two. But, if you want to talk about any of the others, please grab me afterwards!
- Just because you have a hammer does not mean that everything is a nail. When you first get started doing user research, you may get very comfortable with one or two methods, for example usability testing or user interviews. It is very tempting to feel like your go-to method is appropriate for all of your research questions. [»] But, often times, a different method will be more effective. For example, you can nail a screw into the wall, but it won’t work as well as a screwdriver.
- Thankfully, some of the thought leaders in our field have started tackling this need and have published some models to help with this. [»] I’m going to share one from Christian Rohrer, published via NN/g. Reference at the end of the deck. The basic gist is to think of your research question. For example, what if my team was considering adding a self-help feature to our existing product suite and I needed to answer the question: “Why do people reach out for help?” Christian gives two dimensions for you to consider in what you need to know to answer that question. [»] Behavioral v. Attitudinal Research Attitudinal research is used to understand or measure attitudes and beliefs (e.g., focus groups or interviews focus on how people think). Behavioral research is used to measure behaviors (e.g., usability testing focuses on performance). You often need both for success. [»] Qualitative v. Quantitative Qualitative research gives you rich text-based data that explains how or why something is happening Quantitative research gives you numerical-based data that tells you how many or how much You often need both for success. [»] As you can guess, one method alone is probably not going to be a silver bullet to fully answer the question “Why do people reach out for help?” You probably need to do mixed method or multi-method work (there is a difference, hit Google up for that).
- Ok, so you’re doing “research” and you’re learning all kinds of awesome stuff. You want to share it with people. UX’ers will typically blog first and then write a book if it “sticks”. [»] What if we actually tried to publish alongside the academics? What would that look like? [»] Thinking about it kinda makes my head hurt. There are a lot of potential ways to approach this.
- Here’s a thought as a starting point: I assume that most of you are working in a Scrum Agile environment. [»] You are probably working in some flavor of staggered scrum of scrums. [»] You might do your UX research and design work a sprint or two ahead of the development scrum team. [»] You then might help do UT or other kinds of testing of what they built in the last sprint. [»] Then the cycle repeats ad nauseum until the product is “done” (whatever that looks like).
- So, we’re working along, answering research questions for the dev team. And, its overwhelming. [»] It feels like there is no way that we’ll ever be able to stop supporting them to be able to “think bigger” and produce something that the academics will find credible and valuable. But, we are learning tons of awesome stuff and now, we are doing our homework when we design. We are aware of that Big “R” research community who is studying stuff that we need to make better design decisions. We are actively engaging them to understand where their knowledge gaps are and what they might find valuable. [»] What if, we kept an eye on what they might find valuable and [»] every time we did some sort of UX method, we kept our little golden nuggets of knowledge in these areas and compiled them. What if we also pooled golden nuggets from other scrum teams at our organization? [»] What if we also pooled golden nuggets from other user research activities like bigger studies designed to track UX KPIs, benchmark UX performance of our products prior to release or address marketing or product questions? [»] Before you know it, you have something that can contribute to the academic discussion. Now, let’s partner with our new academic BFF’s to get that into a journal!
- Making user experiences the best they can be feels like Mt. Everest. But, we can get to the top together! Let’s go out and learn from each other to make UX better. Thank you!