UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & designs.
•Download as PPTX, PDF•
0 likes•85 views
As humans, we are biased by design. Our intricate and fascinating brains have developed shortcuts through centuries of human evolution. They reduce an unimaginable load of paralyzing decisions, keep us alive, and help us navigate this complex world. Now, these life saving biases affect how we behave with modern technology. Understanding some of the theories and reasons why these biases exist is the key to unlocking their power. In this workshop we will cover some theories around how the brain works. We will review some of our mental shortcuts, take a look at some common biases, and learn how they affect our users, our research, and our designs. Lastly we will review some advantages of biases, and ways to identify and reduce bias. This workshop is targeted for designers who do their own research, and researchers looking to learn more about removing bias from their studies.
Recommended
Recommended
More Related Content
Similar to UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & designs.
Similar to UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & designs. (20)
More from UXPA International
More from UXPA International (20)
Recently uploaded
Recently uploaded (20)
UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & designs.
- 1. BEHIND THE BIAS: Dissecting human shortcuts for better research & designs Lauren Schaefer @lorielue Seesaw
- 2. What is Bias? How bias works in the brain BIAS IN RESEARCH & TIPS FOR REDUCING IT Recruiting Facilitation Analysis 1 2 3 4 5 OUTLINE
- 3. “I am the sister of two Olympic gymnasts. Yet these two gymnasts are not my sisters. How is that possible?” https://knowledgeone.ca/cognitive-bias-when-our-brain-plays-tricks-on-us/ “
- 8. Our conscious brains can only handle something like 40 bits of information a second, while our unconscious minds can handle 11 million bits of information per second. “ -Nick Morgan, Forbes
- 13. We put too much faith in the accuracy of our intuition because overriding it requires effortful work.
- 23. Neocortex “New Brain” 20% of Behavior Lizard & Mammal Brains “Old Brain” 80% of Behavior
- 24. OLD BRAIN Fast Unconscious Automatic Mundane Error Prone Impulsive Multitasking NEW BRAIN Slow Conscious Effortful Complex Reliable Controlled Focused
- 26. https://knowledgeone.ca/cognitive-bias-when-our-brain-plays-tricks-on-us/ Bias… our perception is affected by… attentive or of attention selective factors of our attention mnemonic factors related to our memory of judgment a distortion of our judgment of reasoning paradoxes in our reasoning linked to the personality socio-cultural factors Some 250 cognitive biases are generally classified into one of the following five categories:
- 27. How likely is it that… How many times have you…
- 28. Mental shortcuts allow us to make decisions quickly… but impact our ability to make rational decisions.
- 29. BIAS IN RESEARCH When it happens and how to help reduce it.
- 30. RECRUITING
- 31. Selection Bias We assume participants of a particular group will be representative of the whole BIASED We spoke to users from our ambassador program. BETTER We started with feedback from our ambassador program, but also included feedback from new users. 💡 Using random sampling can help ensuring each member of the target population has an equal chance of being included.
- 33. Self Selection Bias People with particular traits are more likely sign-up or respond to studies BIASED We ran an unmoderated study with provided participants from their panel BETTER We ran an unmoderated study with panel participants, and balanced it with feedback from our in product recruitment. 💡 Where possible use mixed methods for recruitment to help vary the pool and ensure a broader selection.
- 35. Availability Bias We put more weight on information that comes to mind more quickly. BIASED I talked to two customers this morning who made a great case for adding this new feature! BETTER We interviewed 15 of our current customers and found that 13 of 15 would benefit from this feature. 💡While data can be reached with smaller numbers, it’s recommended to layer rounds of research to further refine understanding and perspectives.
- 37. FACILITATION
- 38. “The information people provide during interviews isn’t always accurate or reliable. Often, research participants don’t know why they do things, what they really need, what they might do in the future, or how a design could be improved. To really understand what people do, you can’t just ask them, you have to observe them.” -Jim Ross
- 39. Serial Position Effect We tend to forget steps, focusing on the beginning and the end BIASED “How would you go about buying a new car?" BETTER “Walk me through some of the steps you take when deciding to buy a new car” “How do you decide when you have found the right car?” 💡 When observation isn’t possible, try to focus on a recall walkthrough of a specific instances and focus on the behaviors.
- 40. Retrospective Bias We may alter or distort memories unknowingly based on current knowledge or experiences. BIASED “What problems did you have when you signed up last year?" BETTER “As a new user I would like to observe as you sign up for this product for the first time. ”
- 41. Hindsight Bias We may perceive past events as more predictable than they were based on current knowledge. BIASED “Did you expect that would happen when you clicked the button?" BETTER “What would you expect to happen next? ”
- 42. Social Desirability Bias We tend to answer in ways that we think will be most socially acceptable BIASED “We think you’ll really love it, what do you think?" BETTER “We are looking for feedback on how well this is or is not working to inform our next steps. On a scale of 1-10…” 💡 Try using multiple methods and triangulation to identify discrepancies in feedback. Remain as neutral as possible in question format, reduce visible social observers.
- 44. Focusing Illusion We tend to overemphasize particular aspects when making future predictions. BIASED “If we added a feature to allow for ____, would that improve your experience with the product?" BETTER “We’ve prototyped a new feature in this flow, we’d like you to try out for us”
- 45. Interpretation Bias When a scale is interpreted differently based on subjective understandings or perspective. BIASED “Do you use this most of the time?" BETTER “Which best describes how frequently you use this: ● Daily ● A few times a week ● A few times a month ● Less than once a month ● Other 💡 Be specific, avoid imprecise words like sometimes, and often, and backup with data where possible.
- 47. Estimation Bias When we have difficulty providing accurate answers regarding frequency and numbers BIASED “Do you spend most of your day in meetings?” BETTER “How much time do you typically spend in meetings each work day?”
- 48. Dunning- Kruger Effect The tendency to under, or over, estimate your knowledge or experience. BIASED “Would you consider yourself novice, intermediate, or expert?” BETTER “Describe some of the tasks you typically complete, and your experience with the product.”
- 49. Experimenter Bias When researchers unintentionally influence participants' responses or behavior through their own behavior BIASED “I think your really going to like the changes we’ve made here [while smiling]” BETTER “Tell me what you think about the changes” 💡 Be careful of unintentionally leading questions, tone of voice, or gestures may influence participants' responses.
- 51. Acquiescence Bias A tendency respond in an agreeable way to please or avoid conflict. BIASED “Do you like working here?” BETTER “Please rate your satisfaction with this employer based on the following scale. Note: All answers are aggregated and anonymized.” 💡 Try to avoid leading questions, watch for situations where a participant may feel their responses could be judged. For example if their information may get back to a manager, or directly benefit the researcher.
- 53. ANALYSIS
- 54. Availability Bias The tendency to make judgements based on the ease with which relevant examples come to mind 💡 Use Objective Measures Objective measures provide quantifiable data that is less susceptible to biases. Ensure examples are representative of the data.
- 55. Confirmation Bias When we have preconceived expectations and intrprete findings that support them. 💡 Think Statistically The more statistical you are, the more you have to consider the logic and rationality of a decision. Be careful to frame the answer multiple ways and take time to reason through it. This requires New Brain thinking which can help to reduce Old Brain defaults.
- 56. Anchoring Bias Our tendency to rely too much on existing reference points when interpreting the data. 💡 Be Critical of the Obvious Think critically and approach the data systematically. It can help to list things out as individual points of feedback to consider. Our old brain tends to focus on readily available information, and when information is limited or singular it is more likely to jump to conclusions.
- 57. False Consensus Effect Our tendency to overestimate the extent to which others share the same opinions 💡 Get second opinions while fostering independent thinking Enlist additional observers to help in subjective interpretation. Each person should write their own inferences for each participant before meeting to discuss the outcomes.
- 58. RECAP A quick review of the tips
- 59. Debiasing Techniques Avoid Multitasking Avoid time sensitive decisions By cynical of the obvious Think statistically
- 60. You can’t eliminate bias, but you can reduce it. ● Use random and diverse sampling techniques ● Standardize research protocols ● Avoid leading questions, and be cautious of phrasing ● Be mindful of your own biases ● Triangulate with multiple methods ● Quantify where possible ● Get second opinion, and diverse perspectives
Editor's Notes
- Congratulations if your figured it out, your logical brain is tuned in and you are thinking critically. For many of this, this feels hard, why? We use learned patterns and stereotypes to quickly and unconsciously solve the problem, but when we can't it causes a discomfort pulls at our attention. We assume that gymnasts are women. However, is you consider they may be men… they become your brothers.
- These assumptions are unconscious. Our unconscious processing is full of programmed defaults, that are constantly learning and used to reduce the load of the incredible amount of decisions and problems we are faced with like decision making. Imagine all of these shoes are your size.. Which pair do you pick? We use a variety of heuristics to narrow down the options in your head based on context and past experiences.
- Think about all of the different types of decisions you’ve made enough to get to this point today, How many were effortful? How many just sort of happened? Deciding what time to get out of bed, choosing which outfit to put on, what you were going to bring with you, what you were going to eat for breakfast, which presentations to attend, etc. Each of those decisions carried different weight and came with different considerations, Photo by Sincerely Media on Unsplash
- Now think of your most routine driving route, and how easy that is.. Or how easy it is to not even think about it. Not even remembering getting from A to B, or that you meant to turn and go to the grocery. Its because of habits, and routines, even muscle memory, all unconscious. These Mental shortcuts are what reduce this load, your Brian relies on evolving heuristic or- “Rules of Thumb”.
- In 17th century Rule of Thumb was a really common reference for a rough estimate, there are lots of versions, such as spacing on a sea chart for rocks, or a measurement of roughly 1 inch from end of thumb to knuckle. -the important takeaway is while it's quick and Practical its not exact, or scientific. This is a perfect metaphor for what cognitive shortcuts are. Ironically most origins of this saying are also rough estimates. Image by Sincerely Media
- How much to we really lean on our unconscious, we like to think we are pretty rational beings? This is almost too big to comprehend..
- 40 in 11M is roughly equal to 3.6 per million To help illustrate that, this painting completely made of dots. (point out corner) 12 dots out of all the dots from the pointilism painting “A Sunday Afternoon on the Island of La Grande Jatte” by George Seurat Reference: https://www.youtube.com/watch?v=aa-m8a-jZ0k based on the TedEd talk by Kim Preshoff
- You might be asking, How can there be that much to process? It's not just heuristics and outcomes, there is also the inputs things like sensory information from our environment. visual processing, lots of movement, auditory information, noises and sounds competing for your attention social information and cues based on the others around you, emotional assessments and stimumi, memory formation, consolidation and retrieval. The inputs, processing, and conclusions are indispensable in daily life. allowing us to process massive amounts of information very quickly and help us make decisions, solve problems, work from similar situations, manage time, assess risks etc. Image by Steven Lewis
- The tricky part is the bias that comes with mental shortcuts we use to make judgements. Think about betting black vs red on roulette. We create patterns out of random events, for example even with radom odds like a coin flip each flip has the same chance, but we will create patterns and predictions about what will happen next based on the last flips. Our brains like to create stories, and reasons for things. We also making quick judgments from limited information Anchoring our judgements Lean on stereotypes Creating causes and reasons without the full picture. accept ambiguity, reducing doubt. Image by Naim Benjelloun
- We go through the world pretty easily with shortcuts, but Some of our heuristics rely on stereotypes to make judgements. And when they fail then, our conscience attention is demanded. Otherwise we just keep on keeping on. https://twitter.com/Psych_Memes/status/801511633352880128
- Our unconscious brains are lazy, and will always look for the path of least resistance.
- Think of it like sipping from a cup of water. You have a limited amount each day and you won’t waste it unless you feel it is really necessary.
- This is why Change is Hard We are used to handling most of our decisions as unconscious habits. When an uninvited roadblock appears in our path It requires conscious thought and effort which is harder.
- Effort can be incentivised through motivation and glucose. Glucose is a key resource used by our brain to get through effortful cognitive tasks.
- Study done on judicial rulings by Israeli judges where they found judges gave more lenient decisions at the start of the day and after meal breaks. They found rulings prisoners were 2x to 6x more likely to be released if they were one of the first 3, versus last 3 before a break.
- 80% of our behavior is unconscious, so as researchers and observers of behavior, we want to dig into this a bit further to help understand when it’s most likely to happen and how we can avoid it.
- Do to that I’m going to lean on the The Triune Model (three part) brain as a set of metaphors to simplify the interplay between different regions. All of the regional references will be in broad strokes for illustration purposes.
- The lizard brain, very primitive like the brainstem. Very primal. It’s goal is to keep you alive. Regulate breathing, heartbeat, etc. Fight, flight or freeze. This is the first half of the “old brain” - called that because evolutionary it’s more primitive and instinctive. Really isn’t bias here, but it can be influenced by other areas of the brain.
- Next is the mammalian brain or emotional brain. This is the second half of “old brain” which includes the Limbic System and has a significant role in biases with emotions, memory, and social interactions. We owe a huge part of our evolutionary success to our ability to relate to others subconsciously Lots of purchasing decisions here (in two equally competing product, the one that fulfills us emotionally will be chosen). This is also where we connect to stories, and need to resolve emotional conflict.
- neocortex or our “Thinking Brain”. This is considered our “New Brain” that is more evolved. And responsible for higher Level function. It’s our Conscious thoughts and planning, Reasoning, logical, rational, thought. Math, reading, imagination. Perception This conscious complex thinking is how we often think of our ourselves as processing information, but in reality it has has been considered as lows a 5% of our overall brain activity. It takes a lot of energy, and effort, and relies on a lot of glucose to function. When it gets tired, the older parts of our brain can overpower it.Emotions run high. Our lizard and monkey brains can easily take over our conscious processing and rational thinking.
- Oversimplified but useful for discussing different major areas and generalized functions. We often default to the lower levels of the old brain and must work to use the upper levels of the brain. Our lower level responses are so fast they are nearly automatic where our upper level cortex can take 4x times longer to respond.
- The New Brain is the the more critical thinker in charge of doubt and accepting or dismissing information. It can handle incompatibility and complex processing. It is effortful and in limited supply, Used in for important processing. The Old Brain is quick and can save our lives: but it is also gullible, and biased. It creates patterns and associations, invents causes and intentions, overgeneralizes and suppresses doubt. It takes almost no effort and is leaned on when the stakes are low.
- There are hundred of biases and they are perfectly normal. You can’t and don’t really want to get rid of mental shortcuts, but we do need to be aware of them, and the situations they are most likely to happen.
- Jumping to conclusions, creating stories to fill gaps in memory, defaulting to cultural norms, faulty estimations
- It can also be triggered, Our new brain isn’t listening and alert in the way the old brain is. This is one of the reason our wording really matters. Asking someone the likelihood of an event it is more likely they will guess using mental shortcuts or heuristics that are plausible but not necessarily probable yet we’ll feel confident in it. You have to cue the new brain to go into logical thinking mode to increase the likelihood of more rational analysis. Image by Adrian from Pixabay
- I’m highlighting some common areas, but as you saw in the large bias codex, this is by no means inclusive That’s why it’s important to keep it top of mind and think critically in our work.
- Biases here are mainly around assuming a smaller sampling represents the same outcome of larger numbers
- When we recruit participants from a specific segment we have to be careful not to apply the findings to a more generalized audience.
- 5-8 participants will generally uncover a majority of usability issues. 10-20 participants is considered more appropriate for qualitative research.
- Sometimes it can be hard not to get excited when we see a finding that feels really intuitive or related to our goals and run with them, but we still have to vet those findings, we can’t assume they align with the broader audience. Kind of like confirmation bias before the study has even started. Can cause stakeholders to think a study isn’t needed.
- Here are some additional variations. To recap there are a lot of ways bias might sneak into your recruiting, think about the samples you are getting and where you are recruiting from and how that will affect your overall insights.
- Response Bias is really anything that gets in the way of getting an accurate answer from a participant, we will just be reviewing some of them around questions that we ask, and how we ask them.
- UX Matters article
- We remember the beginning and the end better than what's in between Keep self report focused on attitudes, and believes and try to avoid process oriented questions. Quick note that biases are often multifaceted, there could be many factors at play at once, but i’m going to try to single out specific areas of consideration.
- Try not to ask questions that rely too much on memory, we tend to fill gaps with stories our old brain has made up that feel most likely to have happened but are not necessarily accurate.
- This bias arises from a desire to present oneself in a positive light or to avoid appearing ignorant or inadequate.
- “Lie Witness News” - Start at 1:25
- Try not to ask questions that require the participant to predict the future or design the interface
- It’s all relative
- Estimates tend to be optimistic and exaggerated, this is another great one that we could have put in the recruiting with screeners.
- Similar but a little more specific
- You can’t get around this to some extent, but try to be neutral and consistent as much as you can.
- https://twitter.com/OxQualiCourses/status/1387729373961367564
- Similar to the Hawthorne effect where participants change their behavior because they know they are being watched. Knowing you answer as a participant is anonymized.
- It’s important to remember that participants aren’t trying to mislead us don’t be frustrated with them or their answers like a kid in trouble. They are doing their best, we have to work to be part of the solution in helping them tap into more logical responses, part of the value of mental shortcuts is that they are automatic and unconscious. It’s up to us as researchers and designers to consider how we are asking for feedback and where bias is most likely to sneak in. We must apply an ongoing effort to ensure that we are approaching our work with as much objectivity and fairness as possible.
- This is similar to primacy and recency where we tend to remember the first or last participant better.
- Similar to Observer Bias where as an observer our preconceived notions influence our assessments.
- The more we can do to get into the New Brain, the more bias we can remove because the New brain is the more logical rational thinker. However, we have a cognitive threshold that is easy to max out, the more our attention is taxed the more likely we will default to mental shortcuts.