Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BE THE NATURALIST! OR: SORRY, YOUR MUM IS NOT A VALID TEST PARTICIPANT

523 views

Published on

SPEAKING TO THE RIGHT USERS AND GETTING MORE THAN FEATURE REQUESTS


"If you want understand how a Lions hunts, don’t go to the Zoo, go to the Savannah."



User Research is one of the cornerstones of UX but the sheer volume of techniques around, combined with jargon and ‘silo’d teams often means the fundamental goal of many approaches like UX and XP always seems beyond reach:

- To bring together (understanding of) the people who use a system with those who actually create it.


Mike Rawling, a ux veteran of products and projects of all sizes and shapes, takes attendees on a safari through the world of user research techniques, combining tried and tested formal UX experiences with the latest practical techniques, hot from the ux trenches you can use as a team. These include methods for user interviews, agile ethnography, user observation and practically testing your ideas with users in a measured way that fits right into the world of digital design and development: without compromising either UX or your XP, Agile, Lean or other development principles.

Those new to UX, new to user research or struggling with getting good feedback will come away from this session with an introduction to using the right types of user research in your Agile/Lean/XP process so they serve as an invaluable source of intelligence for you, your team and stakeholders, whilst most UX practitioners will come away with techniques that can help them solve the conundrum of ensuring rigorous user research in a rapidly changing landscape of disrupted devices, platforms and markets.

Published in: Design
  • Be the first to comment

BE THE NATURALIST! OR: SORRY, YOUR MUM IS NOT A VALID TEST PARTICIPANT

  1. 1. @hedshot YOUR MUM IS NOT A VALID TEST PARTICIPANT, OR *BE* THE NATURALIST… MICHAEL RAWLING @hedshot
  2. 2. @hedshot YOUR MUM IS NOT A VALID TEST PARTICIPANT, OR *BE* THE NATURALIST… MICHAEL RAWLING @hedshot
  3. 3. @hedshot Title position here Your title sit over itOK INTERRUPTIONS TODAY a) A Zombie Apocalypse b) Monkeys c) Nothing… *please silence your mobile phones too...! ☺
  4. 4. @hedshot Title position here Your title sit over itYOUR HOST TODAY… MIKE RAWLING LEAD UX AT UNRULY.CO, LONDON: TEAM-OF-ONE & UX COACH: UX RESEARCH, USER RESEARCH, INTERACTION DESIGNER, UI ENGINEER, UI DESIGNER, TEST PARTICIPANT RECRUITER, USABILITY TESTER, ETHNOGRAPHIC RESEARCHER, MICRO-COPY WRITER, TRAINER, UX COACH, UX MENTOR…. AND SO ON… UX activities dating back to ’98…that’s pre-‘UX’ Consulted, designed, engineered, leader of, coached, trained big & small teams and initiatives in: Toyota & Lexus cars, Konami computer games, Tesco.com, Wiley Publishing, Camelot (UK’s national lottery), ITV & Granada Television, LoveFilm(Amazon Prime)… ...and Unruly…
  5. 5. @hedshot Title position here Your title sit over it @hedshot ABOUT UNRULY
  6. 6. @hedshot Title position here Your title sit over it @hedshot ABOUT UNRULY.CO
  7. 7. @hedshot Title position here Your title sit over it @hedshot WARNING - MAY EXHIBIT UNRULY CULTURE… ...and of course, err, creative cultural appropriation and mixing…;)
  8. 8. @hedshot Title position here Your title sit over itWHO’S HERE?
  9. 9. @hedshot Title position here Your title sit over itTODAY… •  THE RIGHT USER RESEARCH: FRAMING THE CONTEXT •  WORKSHOP ACTIVITY •  COFFEE! •  REVIEW •  OBSERVATIONS TO ACTIONS: EXERCISE •  BEST BOOSTER UX RESEARCHER AWARDS 2016 •  …OUTRO AND Q&A
  10. 10. @hedshot Title position here Your title sit over it“SOME TESTING IS BETTER THAN NONE…?”
  11. 11. @hedshot Title position here Your title sit over itTHIS IS MY MUM
  12. 12. @hedshot Title position here Your title sit over itSHE LIKES… Gardening Her independence Driving around in her Volvo My little niece History – learning Politics - doing Ballet - watching Trying to organise my sisters wedding – should be watching Choosing dress for weddings Freeze dried coffee…*sigh* Me, most of the time ;o)
  13. 13. @hedshot Title position here Your title sit over itTHIS IS MY MUM USING ONE OF UNRULY’S PRODUCTS
  14. 14. @hedshot Title position here Your title sit over itTHIS IS MUM USING ONE OF UNRULY’S PRODUCTS
  15. 15. @hedshot Title position here Your title sit over itTHIS IS ONE OF MY USERS
  16. 16. @hedshot Title position here Your title sit over itAND HERE’S IS HOW SHE FEELS ABOUT GARDENING
  17. 17. @hedshot Title position here Your title sit over itSEEMS TO BE A PROBLEM HERE? Anybody…?
  18. 18. @hedshot Title position here Your title sit over itWE’RE TOO CLOSE!
  19. 19. @hedshot Title position here Your title sit over itNOT YOUR MUM OR…. Any others that would have the same drawbacks?....
  20. 20. @hedshot Title position here Your title sit over itNOT YOUR MUM OR…. The CEO – proud of product…or overly critical. The company secretary – wants to make a good impression and make team happy The graphic designer – proud, subjective/too close Other team members – the creators of a product, pride Any others?....
  21. 21. @hedshot Title position here Your title sit over itOBJECTIVE / SUBJECTIVE
  22. 22. @hedshot Title position here Your title sit over itAT FIRST…
  23. 23. @hedshot Title position here Your title sit over itAT FIRST…
  24. 24. @hedshot Title position here Your title sit over itAT FIRST…
  25. 25. @hedshot Title position here Your title sit over itNOT MAGIC
  26. 26. @hedshot Title position here Your title sit over itBIAS
  27. 27. @hedshot Title position here Your title sit over itDANIEL KAHNEMEN & AMOS TVERSKY
  28. 28. @hedshot Title position here Your title sit over itCOGNITIVE BIAS!
  29. 29. @hedshot Title position here Your title sit over it @hedshot TYPES OF COGNITIVE BIAS… the deviation from what is normally expected can be characterized by: •  Ambiguity effect – the tendency to avoid options for which missing information makes the probability seem "unknown."[8] •  Anchoring or focalism – the tendency to rely too heavily, or "anchor," on one trait or piece of information when making decisions.[9][10] •  Attentional bias – the tendency to pay attention to emotionally dominant stimuli in one's environment and to neglect relevant data when making judgments of a correlation or association.[11][12] •  Availability heuristic – the tendency to overestimate the likelihood of events with greater "availability" in memory, which can be influenced by how recent the memories are or how unusual or emotionally charged they may be.[13] •  Availability cascade – a self-reinforcing process in which a collective belief gains more and more plausibility through its increasing repetition in public discourse (or "repeat something long enough and it will become true").[14] •  Backfire effect – when people react to disconfirming evidence by strengthening their beliefs.[15] •  Bandwagon effect – the tendency to do (or believe) things because many other people do (or believe) the same. Related to groupthink and herd behavior.[16] •  Base rate fallacy or base rate neglect – the tendency to base judgments on specifics, ignoring general statistical information.[17][clarification needed] •  Belief bias – an effect where someone's evaluation of the logical strength of an argument is biased by the believability of the conclusion.[18] •  Bias blind spot – the tendency to see oneself as less biased than other people, or to be able to identify more cognitive biases in others than in oneself.[19] •  Choice-supportive bias – the tendency to remember one's choices as better than they actually were.[20] •  Clustering illusion – the tendency to over-expect small runs, streaks, or clusters in large samples of random data (that is, seeing phantom patterns).[10] •  Confirmation bias – the tendency to search for or interpret information or memories in a way that confirms one's preconceptions.[21] •  Congruence bias – the tendency to test hypotheses exclusively through direct testing, instead of testing possible alternative hypotheses.[10] •  Conjunction fallacy – the tendency to assume that specific conditions are more probable than general ones.[22] •  Conservatism or regressive bias – the tendency to underestimate high values and high likelihoods while overestimating low ones.[23][24][25] •  Conservatism (Bayesian) – the tendency to insufficiently revise one's belief when presented with new evidence.[23][26][27] •  Contrast effect – the enhancement or reduction of a certain perception's weight when compared with a recently observed, contrasting object.[28] •  Curse of knowledge – when knowledge of a topic diminishes one's ability to think about it from a less-informed (but more neutral) perspective.[29] •  Decoy effect – preferences change when there is a third option that is asymmetrically dominated •  Denomination effect – the tendency to spend more money when it is denominated in small amounts (e.g. coins) rather than large amounts (e.g. bills).[30] •  Distinction bias – the tendency to view two options as more dissimilar when evaluating them simultaneously than when evaluating them separately.[31] •  Duration neglect – the neglect of the duration of an episode in determining its value •  Empathy gap – the tendency to underestimate the influence or strength of feelings, in either oneself or others. •  Endowment effect – the fact that people often demand much more to give up an object than they would be willing to pay to acquire it.[32] •  Essentialism – categorizing people and things according to their essential nature, in spite of variations.[33] •  Exaggerated expectation – based on the estimates, real-world evidence turns out to be less extreme than our expectations (conditionally inverse of the conservatism bias).[23][34] •  Experimenter's or expectation bias – the tendency for experimenters to believe, certify, and publish data that agree with their expectations for the outcome of an experiment, and to disbelieve,
  30. 30. @hedshot Title position here Your title sit over it @hedshot TYPES OF COGNITIVE BIAS… •  Hindsight bias – sometimes called the "I-knew-it-all-along" effect, the tendency to see past events as being predictable[41] at the time those events happened. Colloquially referred to as "Hindsight is 20/20". •  Hostile media effect – the tendency to see a media report as being biased, owing to one's own strong partisan views. •  Hot-hand fallacy – The "hot-hand fallacy" (also known as the "hot hand phenomenon" or "hot hand") is the fallacious belief that a person who has experienced success has a greater chance of further success in additional attempts •  Hyperbolic discounting – the tendency for people to have a stronger preference for more immediate payoffs relative to later payoffs, where the tendency increases the closer to the present both payoffs are.[42] Also known as current moment bias, present-bias, and related toDynamic inconsistency. •  Illusion of control – the tendency to overestimate one's degree of influence over other external events.[43] •  Illusion of validity – when consistent but predictively weak data leads to confident predictions •  Illusory correlation – inaccurately perceiving a relationship between two unrelated events.[44][45] •  Impact bias – the tendency to overestimate the length or the intensity of the impact of future feeling states.[46] •  Information bias – the tendency to seek information even when it cannot affect action.[47] •  Insensitivity to sample size – the tendency to under-expect variation in small samples •  Irrational escalation – the phenomenon where people justify increased investment in a decision, based on the cumulative prior investment, despite new evidence suggesting that the decision was probably wrong. •  Just-world hypothesis – the tendency for people to want to believe that the world is fundamentally just, causing them to rationalize an otherwise inexplicable injustice as deserved by the victim(s). •  Less-is-better effect – a preference reversal where a dominated smaller set is preferred to a larger set •  Loss aversion – "the disutility of giving up an object is greater than the utility associated with acquiring it".[48] (see also Sunk cost effects and endowment effect). •  Ludic fallacy – the misuse of games to model real-life situations. •  Mere exposure effect – the tendency to express undue liking for things merely because of familiarity with them.[49] •  Money illusion – the tendency to concentrate on the nominal (face value) of money rather than its value in terms of purchasing power.[50] •  Moral credential effect – the tendency of a track record of non-prejudice to increase subsequent prejudice. •  Negativity bias – the tendency to pay more attention and give more weight to negative than positive experiences or other kinds of information. •  Neglect of probability – the tendency to completely disregard probability when making a decision under uncertainty.[51] •  Normalcy bias – the refusal to plan for, or react to, a disaster which has never happened before. •  Observation selection bias – the effect of suddenly noticing things that were not noticed previously — and as a result wrongly assuming that the frequency has increased. •  Observer-expectancy effect – when a researcher expects a given result and therefore unconsciously manipulates an experiment or misinterprets data in order to find it (see also subject-expectancy effect). •  Omission bias – the tendency to judge harmful actions as worse, or less moral, than equally harmful omissions (inactions).[52] •  Optimism bias – the tendency to be over-optimistic, overestimating favorable and pleasing outcomes (see also wishful thinking, valence effect, positive outcome bias).[53][54] •  Ostrich effect – ignoring an obvious (negative) situation. •  Outcome bias – the tendency to judge a decision by its eventual outcome instead of based on the quality of the decision at the time it was made. •  Overconfidence effect – excessive confidence in one's own answers to questions. For example, for certain types of questions, answers that people rate as "99% certain" turn out to be wrong 40% of the time.[23][55][56][57] •  Pareidolia – a vague and random stimulus (often an image or sound) is perceived as significant, e.g., seeing images of animals or faces in clouds, the man in the moon, and hearing non-existent hidden messages on
  31. 31. @hedshot Title position here Your title sit over it @hedshot TYPES OF COGNITIVE BIAS… •  Restraint bias – the tendency to overestimate one's ability to show restraint in the face of temptation. •  Rhyme as reason effect – rhyming statements are perceived as more truthful. A famous example being used in the O.J Simpson trial with the defense's use of the phrase "If the gloves don't fit, then you must acquit." •  Risk compensation / Peltzman effect – the tendency to take greater risks when perceived safety increases. •  Selective perception – the tendency for expectations to affect perception. •  Semmelweis reflex – the tendency to reject new evidence that contradicts a paradigm.[27] •  Selection bias – the distortion of a statistical analysis, resulting from the method of collecting samples. If the selection bias is not taken into account then certain conclusions drawn may be wrong. •  Social comparison bias – the tendency, when making hiring decisions, to favour potential candidates who don't compete with one's own particular strengths.[59] •  Social desirability bias – the tendency to over-report socially desirable characteristics or behaviours and under-report socially undesirable characteristics or behaviours.[60] •  Status quo bias – the tendency to like things to stay relatively the same (see also loss aversion, endowment effect, and system justification).[61][62] •  Stereotyping – expecting a member of a group to have certain characteristics without having actual information about that individual. •  Subadditivity effect – the tendency to estimate that the likelihood of an event is less than the sum of its (more than two) mutually exclusive components.[63] •  Subjective validation – perception that something is true if a subject's belief demands it to be true. Also assigns perceived connections between coincidences. •  Survivorship bias – concentrating on the people or things that "survived" some process and inadvertently overlooking those that didn't because of their lack of visibility. •  Texas sharpshooter fallacy – pieces of information that have no relationship to one another are called out for their similarities, and that similarity is used for claiming the existence of a pattern. •  Time-saving bias – underestimations of the time that could be saved (or lost) when increasing (or decreasing) from a relatively low speed and overestimations of the time that could be saved (or lost) when increasing (or decreasing) from a relatively high speed. •  Unit bias – the tendency to want to finish a given unit of a task or an item. Strong effects on the consumption of food in particular.[64] •  Well travelled road effect – underestimation of the duration taken to traverse oft-traveled routes and overestimation of the duration taken to traverse less familiar routes. •  Zero-risk bias – preference for reducing a small risk to zero over a greater reduction in a larger risk. •  Zero-sum heuristic – Intuitively judging a situation to be zero-sum (i.e., that gains and losses are correlated). Derives from the zero-sum game in game theory, where wins and losses sum to zero.[65][66] The frequency with which this bias occurs may be related to the social dominance orientation personality factor. Social biases[edit] Most of these biases are labeled as attributional biases. •  Actor-observer bias – the tendency for explanations of other individuals' behaviors to overemphasize the influence of their personality and underemphasize the influence of their situation (see also Fundamental attribution error), and for explanations of one's own behaviors to do the opposite (that is, to overemphasize the influence of our situation and underemphasize the influence of our own personality). •  Defensive attribution hypothesis – defensive attributions are made when individuals witness or learn of a mishap happening to another person. In these situations, attributions of responsibility to the victim or harm-doer for the mishap will depend upon the severity of the outcomes of the mishap and the level of personal and situational similarity between the individual and victim. More responsibility will be attributed to the harm-doer as the outcome becomes more severe, and as personal or situational similarity decreases. •  Dunning–Kruger effect an effect in which incompetent people fail to realise they are incompetent because they lack the skill to distinguish between competence and incompetence[67] •  Egocentric bias – occurs when people claim more responsibility for themselves for the results of a joint action than an outside observer would credit them.
  32. 32. @hedshot Title position here Your title sit over it @hedshot TYPES OF COGNITIVE BIAS… •  Illusion of transparency – people overestimate others' ability to know them, and they also overestimate their ability to know others. •  Illusory superiority – overestimating one's desirable qualities, and underestimating undesirable qualities, relative to other people. (Also known as "Lake Wobegon effect," "better-than-average effect," or "superiority bias").[72] •  Ingroup bias – the tendency for people to give preferential treatment to others they perceive to be members of their own groups. •  Just-world phenomenon – the tendency for people to believe that the world is just and therefore people "get what they deserve." •  Moral luck – the tendency for people to ascribe greater or lesser moral standing based on the outcome of an event rather than the intention •  Naive cynicism – expecting more egocentric bias in others than in oneself •  Outgroup homogeneity bias – individuals see members of their own group as being relatively more varied than members of other groups.[73] •  Projection bias – the tendency to unconsciously assume that others (or one's future selves) share one's current emotional states, thoughts and values.[74] •  Self-serving bias – the tendency to claim more responsibility for successes than failures. It may also manifest itself as a tendency for people to evaluate ambiguous information in a way beneficial to their interests (see also group-serving bias).[75] •  System justification – the tendency to defend and bolster the status quo. Existing social, economic, and political arrangements tend to be preferred, and alternatives disparaged sometimes even at the expense of individual and collective self-interest. (See also status quo bias.) •  Trait ascription bias – the tendency for people to view themselves as relatively variable in terms of personality, behavior, and mood while viewing others as much more predictable. •  Ultimate attribution error – similar to the fundamental attribution error, in this error a person is likely to make an internal attribution to an entire group instead of the individuals within the group. •  Worse-than-average effect – a tendency to believe ourselves to be worse than others at tasks which are difficult[76] Memory errors and biases[edit] Main article: List of memory biases In psychology and cognitive science, a memory bias is a cognitive bias that either enhances or impairs the recall of a memory (either the chances that the memory will be recalled at all, or the amount of time it takes for it to be recalled, or both), or that alters the content of a reported memory. There are many types of memory bias, including: •  Bizarreness effect – bizarre, or uncommon material, is better remembered than common material •  Choice-supportive bias – remembering chosen options as having been better than rejected options[77] •  Change bias – after an investment of effort in producing change, remembering one's past performance as more difficult than it actually was[78] •  Childhood amnesia – the retention of few memories from before the age of four •  Conservatism or Regressive Bias – tendency to remember high values and high likelihoods/probabilities/frequencies lower than they actually were and low ones higher than they actually were. Based on the evidence, memories are not extreme enough[24][25] •  Consistency bias – incorrectly remembering one's past attitudes and behaviour as resembling present attitudes and behaviour.[79] •  Context effect – that cognition and memory are dependent on context, such that out-of-context memories are more difficult to retrieve than in-context memories (e.g., recall time and accuracy for a work-related memory will be lower at home, and vice versa) •  Cross-race effect – the tendency for people of one race to have difficulty identifying members of a race other than their own •  Cryptomnesia – a form of misattribution where a memory is mistaken for imagination, because there is no subjective experience of it being a memory.[78]
  33. 33. @hedshot Title position here Your title sit over it @hedshot TYPES OF COGNITIVE BIAS… •  Illusory correlation – inaccurately remembering a relationship between two events.[23][45] •  Lag effect – see spacing effect •  Leveling and Sharpening – memory distortions introduced by the loss of details in a recollection over time, often concurrent with sharpening or selective recollection of certain details that take on exaggerated significance in relation to the details or aspects of the experience lost through leveling. Both biases may be reinforced over time, and by repeated recollection or re-telling of a memory.[81] •  Levels-of-processing effect – that different methods of encoding information into memory have different levels of effectiveness[82] •  List-length effect – a smaller percentage of items are remembered in a longer list, but as the length of the list increases, the absolute number of items remembered increases as well.[83] •  Misinformation effect – that misinformation affects people's reports of their own memory. •  Misattribution – when information is retained in memory but the source of the memory is forgotten. One of Schacter's (1999) Seven Sins of Memory, Misattribution was divided into Source Confusion, Cryptomnesia and False Recall/False Recognition.[78] •  Modality effect – that memory recall is higher for the last items of a list when the list items were received via speech than when they were received via writing. •  Mood-congruent memory bias – the improved recall of information congruent with one's current mood. •  Next-in-line effect – that a person in a group has diminished recall for the words of others who spoke immediately before or after this person. •  Osborn effect – that being intoxicated with a mind-altering substance makes it harder to retrieve motor patterns from the Basal Ganglion.[84] •  Part-list cueing effect – that being shown some items from a list makes it harder to retrieve the other items[85] •  Peak-end rule – that people seem to perceive not the sum of an experience but the average of how it was at its peak (e.g. pleasant or unpleasant) and how it ended. •  Persistence – the unwanted recurrence of memories of a traumatic event. •  Picture superiority effect – that concepts are much more likely to be remembered experientially if they are presented in picture form than if they are presented in word form.[86] •  Placement bias – tendency of people to remember themselves as better than others at tasks at which they rate themselves above average (also Illusory superiority or Better-than-average effect)[87] and tendency to remember themselves as worse than others at tasks at which they rate themselves below average (also Worse-than-average effect[23][76] •  Positivity effect – that older adults favor positive over negative information in their memories. •  Primacy effect, Recency effect & Serial position effect – that items near the end of a list are the easiest to recall, followed by the items at the beginning of a list; items in the middle are the least likely to be remembered.[88] •  Processing difficulty effect •  Reminiscence bump – the recalling of more personal events from adolescence and early adulthood than personal events from other lifetime periods[89] •  Rosy retrospection – the remembering of the past as having been better than it really was. •  Self-relevance effect – that memories relating to the self are better recalled than similar information relating to others. •  Self-serving bias – perceiving oneself responsible for desirable outcomes but not responsible for undesirable ones. •  Source Confusion – misattributing the source of a memory, e.g. misremembering that one saw an event personally when actually it was seen on television. •  Spacing effect – that information is better recalled if exposure to it is repeated over a longer span of time. •  Stereotypical bias – memory distorted towards stereotypes (e.g. racial or gender), e.g. "black-sounding" names being misremembered as names of criminals.[78] •  Suffix effect – the weakening of the recency effect in the case that an item is appended to the list that the subject is not required to recall[90]
  34. 34. @hedshot Title position here Your title sit over itBIAS GAME …GOTTA NAME THEM ALL
  35. 35. @hedshot Title position here Your title sit over itBIAS EXERCISE – 10 MINS Identify 5+ types of bias observed in your (most recent) project / team Actions Use the wikipedia list of Cognitive Biases or the handouts. Aim Record them
  36. 36. @hedshot Title position here Your title sit over itBIAS EXERCISE – RESULTS…
  37. 37. @hedshot Title position here Your title sit over itRECOGNISE SKEW & AVOID
  38. 38. @hedshot Title position here Your title sit over itSO WHO & WHY ARE YOU TESTING?
  39. 39. @hedshot Title position here Your title sit over itOMG! WHAT ELSE IS WRONG?
  40. 40. @hedshot Title position here Your title sit over it @hedshot OTHER PITFALLS…
  41. 41. @hedshot Title position here Your title sit over itNOT JUST ‘GETTING FEEDBACK’
  42. 42. @hedshot Title position here Your title sit over it…NOT ‘SIGN-OFF’
  43. 43. @hedshot Title position here Your title sit over it…AND NO, NOT UAT EITHER
  44. 44. @hedshot Title position here Your title sit over it…AND NO, NOT UAT EITHER Terms that get abused and confused: -  …but it’s got ‘user’ in the title!?! -  Still no.. -  This is fulfilling maybe 1/9th of user research’s potential -  Not sufficient because it’s only ‘requirements’ orientated, -  a tickbox process -  As often as not done by QA or stakeholders -  Nothing wrong – just not UX “User Acceptance Testing”
  45. 45. @hedshot Title position here Your title sit over itBE OBJECTIVE – THIS IS USEFUL!
  46. 46. @hedshot Title position here Your title sit over itDON’T GIVE UP THOUGH!
  47. 47. @hedshot Title position here Your title sit over itDON’T YOU NEED COMPLETELY CONTROLLED CIRCUMSTANCES?
  48. 48. @hedshot Title position here Your title sit over itNO – NOT NECESSARILY
  49. 49. @hedshot Title position here Your title sit over it @hedshot THE GREAT DEBATE
  50. 50. @hedshot Title position here Your title sit over itACTUALLY NOT EXPENSIVE* *As it used to be…
  51. 51. @hedshot Title position here Your title sit over itLAST DON’T… If you can’t change anything…
  52. 52. @hedshot Title position here Your title sit over itIF YOU CAN’T CHANGE ANYTHING…DON’T TEST! If there’s no scope/time for changes that you find need make after… •  About to make loads of releases – things will change too much! •  Will waste goodwill towards UX •  Will waste budget *Unless you’re Before > After Benchmarking..
  53. 53. @hedshot Title position here Your title sit over it
  54. 54. @hedshot Title position here Your title sit over itIF YOU WANT TO KNOW HOW A LION HUNTS…
  55. 55. @hedshot Title position here Your title sit over itIF YOU WANT TO KNOW HOW A LION HUNTS…
  56. 56. @hedshot Title position here Your title sit over itIF YOU WANT TO KNOW HOW A USER HUNTS… No…
  57. 57. @hedshot Title position here Your title sit over it @hedshot OK, WHAT SHOULD WE *DO*…
  58. 58. @hedshot Title position here Your title sit over itINSPIRATION, HERO, LEGEND…
  59. 59. @hedshot Title position here Your title sit over itFIELD STUDIES…
  60. 60. @hedshot Title position here Your title sit over itREQUIRES TEAM EMPATHY You may not be doing ‘Agile’, where empathy is on strategy… …but these are a good approach for all teams…
  61. 61. @hedshot Title position here Your title sit over it @hedshot USER INTEL UX or product designers present always to marinate the team in your user intelligence throughout
  62. 62. @hedshot Title position here Your title sit over itIT’S A TEAM SPORT
  63. 63. @hedshot Title position here Your title sit over it TEAM LEARN FROM TESTING TOGETHER You and team members design experiments..
  64. 64. @hedshot Title position here Your title sit over itCROSS-DISCIPLINARY PAIRING… https://www.linkedin.com/groups/Pair-design-3315113.S.5968946180594364420 + + + + + +
  65. 65. @hedshot Title position here Your title sit over it ENGAGING ALL IN USER RESEARCH… Team members pairing on design and run experiments together…
  66. 66. @hedshot Title position here Your title sit over itPEOPLE!! RECRUITING IS KEY Right people quote, Jeff Sauro of measuringu.com and MeasuringU LLC: “…it is much more productive to test (products) with a small number of representative users than a 1000 users of the wrong type…”* *sorry Jeff, totally paraphrasing
  67. 67. @hedshot Title position here Your title sit over itPEOPLE!! RECRUITING IS KEY Establish right people via user research: •  Interviews, •  Surveys… •  Persona process •  Data, traffic logs …And Analysis with cross referencing between sources…
  68. 68. @hedshot Title position here Your title sit over itPEOPLE!! RECRUITING IS KEY Can be surprisingly time consuming! Use you time wisely! Look for opportunities!
  69. 69. @hedshot Title position here Your title sit over itPEOPLE!! RECRUITING IS KEY Agencies like Saro, People For Research (UK) or… …I’m sure I can put you in touch with some specialist research recruiting agencies over here
  70. 70. @hedshot Title position here Your title sit over it @hedshot HOW SHOULD WE TEST?
  71. 71. @hedshot Title position here Your title sit over itWHAT TYPE OF TEST?? SO MANY
  72. 72. @hedshot Title position here Your title sit over itTONS ‘A GUNS
  73. 73. @hedshot Title position here Your title sit over itCHOOSE YOUR WEAPON WISELY
  74. 74. @hedshot Title position here Your title sit over it @hedshot A FEW EXAMPLES
  75. 75. @hedshot Title position here Your title sit over itAGILE ETHNOGRAPHY: EXPOSURE HOURS
  76. 76. @hedshot Title position here Your title sit over itAGILE ETHNOGRAPHY: ‘EXPOSURE HOURS’ But don’t…
  77. 77. @hedshot Title position here Your title sit over itTANGENT…CONTACT THEORY
  78. 78. @hedshot Title position here Your title sit over itTANGENT…CONTACT THEORY •  Premise under appropriate conditions interpersonal contact is one of the most effective ways to reduce prejudice between majority and minority group members.
  79. 79. @hedshot Title position here Your title sit over itTANGENT…CONTACT THEORY •  Premise under appropriate conditions interpersonal contact is one of the most effective ways to reduce prejudice between majority and minority group members. If one has the opportunity to communicate with others, they are able to understand and appreciate different points of views involving their way of life.
  80. 80. @hedshot Title position here Your title sit over itTANGENT…CONTACT THEORY •  Premise under appropriate conditions interpersonal contact is one of the most effective ways to reduce prejudice between majority and minority group members. If one has the opportunity to communicate with others, they are able to understand and appreciate different points of views involving their way of life. As a result of new appreciation and understanding, prejudice should diminish...
  81. 81. @hedshot Title position here Your title sit over itTANGENT…CONTACT THEORY •  Also known as Intergroup Contact Theory. •  Premise under appropriate conditions interpersonal contact is one of the most effective ways to reduce prejudice between majority and minority group members. If one has the opportunity to communicate with others, they are able to understand and appreciate different points of views involving their way of life. As a result of new appreciation and understanding, prejudice should diminish... •  More time together, the greater the effect…
  82. 82. @hedshot Title position here Your title sit over it @hedshot OTHER END OF THE SCALE….
  83. 83. @hedshot Title position here Your title sit over itGUERILLA TESTING
  84. 84. @hedshot Title position here Your title sit over itAWAKEN THOSE GUERILLA SKILLS! One – 2 - One exercise! 5 MINS •  Stand up! •  SEARCH Find someone in the room you haven’t spoken to one–to-one to before •  IDENTIFY and record one detailed thing they notice about your appearance or clothing •  Come back to your seats •  …Review…
  85. 85. @hedshot Title position here Your title sit over itGUERILLA TESTING https://www.youtube.com/watch?v=0YL0xoSmyZI
  86. 86. @hedshot Title position here Your title sit over itINTERVIEW
  87. 87. @hedshot Title position here Your title sit over it USER RESEARCH BOARD Other sources: •  User diaries •  Customer service records •  Tech Support issues, patterns •  Feedback link on website •  Remote…
  88. 88. @hedshot Title position here Your title sit over itIPA
  89. 89. @hedshot Title position here Your title sit over itIPA NO…even better! ☺ Interpretative Phenomenological Analysis! …an approach to psychological qualitative research with an idiographic focus, which means that it aims to offer insights into how a given person, in a given context, makes sense of a given phenomenon.
  90. 90. @hedshot Title position here Your title sit over it @hedshot REMOTE USER RESEARCH
  91. 91. @hedshot Title position here Your title sit over it @hedshot USER RESEARCH RESEARCH Micro surveys and team administered feedback tools
  92. 92. @hedshot Title position here Your title sit over it @hedshot REMOTE USER RESEARCH Examples Publishers/ad unit - https://www.usertesting.com/videos/ turcrrx_EP0wwhnQPTXeHA/clips/841068?shared=true Other products: https://www.usertesting.com/videos/LutEz7hRdY5hEB-MHIUJEw/clips/ 841072?shared=true
  93. 93. @hedshot Title position here Your title sit over it @hedshot WHAT ARE YOU TESTING? APPARATUS
  94. 94. @hedshot Title position here Your title sit over itTEST ARTIFACTS Only need to explicitly test only a few things – distinct hypotheses not the whole thing…
  95. 95. @hedshot Title position here Your title sit over itTEST ARTIFACTS Pick something easy, quick that is enough… •  From a Sharpie sketch •  Could be a series of sketches •  Basic Photoshop mockups •  Prototypes at any level •  An actual digital product or website
  96. 96. @hedshot Title position here Your title sit over it @hedshot MAKE IT REAL “Prototyping: where the wheels touch down on the tarmac”
  97. 97. @hedshot Title position here Your title sit over itSUPER LOW RES PROTOTYPING
  98. 98. @hedshot Title position here Your title sit over itTURN UP THE RESOLUTION
  99. 99. @hedshot Title position here Your title sit over it @hedshot MAYBE WE CAN DO THAT?
  100. 100. @hedshot Title position here Your title sit over it @hedshot RIGHT! YOUR TURN…
  101. 101. @hedshot Title position here Your title sit over itHANDS ON TIME!
  102. 102. @hedshot Title position here Your title sit over itPRODUCT UX TESTING PRACTICE 1.  Hypothesis 2.  Method 3.  Apparatus 4.  Recruitment!! 5.  Experiment 6.  Results 7.  Analysis 8.  Conclusions
  103. 103. @hedshot Title position here Your title sit over it THE CHALLENGE: USABILITY TESTING BOOSTERCONF.NO! And get Booster ready for next year! Imagine yourself into the shoes of the people making the BoosterConf 2017 site!!
  104. 104. @hedshot Title position here Your title sit over itCHOOSE A TESTING TECHNIQUE! - 5 MINS Team Choices!!!! •  Create an Impromptu Usability Lab •  Each team needs to have! •  A Test participant: just be yourself! ☺ •  A Moderator: welcomes the test participant and directs the test •  Observers: the silent partner….takes a backseat and takes notes
  105. 105. @hedshot Title position here Your title sit over itSET UP THE CONTEXT PROPERLY
  106. 106. @hedshot Title position here Your title sit over it @hedshot HANDY BOILERPLATE For Moderators to follow: 1.  Welcome! 2.  *put candidate as ease* 3.  Read scene setting boilerplate in natural language: 1.  “we do this for a living, you can’t hurt our feelings!” 2.  “we’re not testing you, we’re testing the system any answer you give is right” 3.  “We’re taking this video because our memories are not so good - we are but human - so we don’t forget everything you said” 4.  “I’m going to show you a prototype - some may appear odd or not functioning: In this usability review, call this out when you see it and say what you think should happen or what you expect to see or do - in fact if you could say aloud what you are thinking and doing throughout”
  107. 107. @hedshot Title position here Your title sit over it @hedshot FOCUSED TASK LIST Write a list of tasks for user to complete – 10 minutes
  108. 108. @hedshot Title position here Your title sit over itWRITE A TEST SCRIPT! - 10 MINS Design some question tasks that cause the user to… •  Yield a measurable metric that measures the problem UX Measures - some examples •  time on task •  clicks taken to success/conversion •  conversions •  comparison of activity types: reading(what does this say) vs. way-finding (where am I?) vs. (inter)acting (what am I meant to do here) Techniques •  Open questions – e.g. “what do you think? what do you see? what is your initial reaction?” •  Set Tasks – e.g. “Starting at the homepage, can you find what one day at the conference costs? Starting at the homepage, can you find what one day at the conference costs?” Don’t Lead your test participants or suggest things Aim To expose valuable insights into weak experiences of the website while watching out for skew
  109. 109. @hedshot Title position here Your title sit over it#2 SET UP YOUR TEST LAB -  Either analog, pen and paper , hand sketches to test with -  Or if you have a laptop download and use Silverback ( http://silverback.s3.amazonaws.com/ silverback2.zip - borrow a webcam…?) -  Website printouts?
  110. 110. @hedshot Title position here Your title sit over it#2 UX SPEED TESTING!! …EACH TEST: 10 MINS/USER Actions •  USERs: rotate between teams •  MODERATORS: ensure test user is relaxed, takes them through the test, ensures they keep on track •  OBSERVERS don’t talk, observe. Record event and user behaiviours Don’t Lead users. It can seem chaotic or tricky - make sure you keep on track but naturally Aim Capture observations and insights ready for the next phase
  111. 111. @hedshot Title position here Your title sit over it REVIEW: HOW’D WE DO? Review •  How far did we get? Round each group… Impressions Did you answer you questions? Did you learn anything new or unexpected? How was your script? How was your technique?
  112. 112. @hedshot Title position here Your title sit over it @hedshot COFFEE TIME! Workshop concludes after the break…meet back here in 15 minutes!
  113. 113. @hedshot Title position here Your title sit over it @hedshot IN THE BREAK: USER SURVEYS! Workshop concludes after the break…meet back here in 15 minutes! IF you have time….. Task Perform tests on any conference attendees you come across! Do watch out for skew/bias Don’t ask leading questions!! Aim Record more data/observations!
  114. 114. @hedshot Title position here Your title sit over it @hedshot COFFEE TIME! Workshop concludes after the break…meet back here in 15 minutes!
  115. 115. @hedshot Title position here Your title sit over it @hedshot PART II: ACTIONABLE INSIGHTS
  116. 116. @hedshot Title position here Your title sit over itANYONE NEW JOINED US??
  117. 117. @hedshot Title position here Your title sit over it @hedshot THE STORY SO FAR Don’t! •  listen to usability intelligence from your mum, your CEO, your designer.. Do! •  Open your mind - watch out for bias •  Field studies: Get out of the office and be the naturalist! •  Choose users carefully! •  Include the whole team! •  Observe carefully •  Choose the right test
  118. 118. @hedshot Title position here Your title sit over it @hedshot SO WE’VE GOT RESULTS What now??
  119. 119. @hedshot Title position here Your title sit over it @hedshot HOW TO MAKE THIS GATHERED RESULTS USEABLE AND USEFUL •  You can’t just do what users ask treat requests as symptoms •  We start analysing data straight away, naturally •  Watch for bias! •  And be prepared to change your conclusions as patterns are identified!
  120. 120. @hedshot Title position here Your title sit over it @hedshot HOW TO MAKE THIS GATHERED RESULTS USEABLE AND USEFUL •  Typing up notes, transcribing audio, watching video back can be time consuming.. •  BUT offers the brain a useful time to analyse and take a step back…
  121. 121. @hedshot Title position here Your title sit over it @hedshot “LEVRS!” We’re searching for LEVERS… ...an action we can take to fix the problem…
  122. 122. @hedshot Title position here Your title sit over it @hedshot WALL OF DATA
  123. 123. @hedshot Title position here Your title sit over it @hedshot A LITTLE MORE WEIRD SCIENCE?
  124. 124. @hedshot Title position here Your title sit over it @hedshot MAKING SENSE Open questions can offer fresh insights and lot of data… ...luckily it’s not a fishing trip!
  125. 125. @hedshot Title position here Your title sit over it @hedshot MAKING SENSE: QUANTIFY! Review questions for results: give a score out of 10… Test Participant #1: Test Participant #2: Test Participant #3: Test Participant #4: Test Participant #5: Test Participant #6: User Journey / Task / Question #1 User Journey / Task / Question #2 User Journey / Task / Question #3 User Journey / Task / Question #4 User Journey / Task / Question #5 AVERAGE / STATISTCAL TREATMENT Handy Quantitative Converter Table
  126. 126. @hedshot Title position here Your title sit over it @hedshot MAKING SENSE Review questions for results: give a score out of 10… Test Participant #1: Test Participant #2: Test Participant #3: Test Participant #4: Test Participant #5: Test Participant #6: User Journey / Task / Question #1 User Journey / Task / Question #2 User Journey / Task / Question #3 User Journey / Task / Question #4 User Journey / Task / Question #5 AVERAGE / STATISTCAL TREATMENT Handy Quantitative Converter Table 1 2 5 2 2
  127. 127. @hedshot Title position here Your title sit over it @hedshot MAKING SENSE Review questions for results: give a score out of 10… Test Participant #1: Test Participant #2: Test Participant #3: Test Participant #4: Test Participant #5: Test Participant #6: User Journey / Task / Question #1 User Journey / Task / Question #2 User Journey / Task / Question #3 User Journey / Task / Question #4 User Journey / Task / Question #5 AVERAGE / STATISTCAL TREATMENT Handy Quantitative Converter Table 1 2 5 2 2 9 8 9 3 7
  128. 128. @hedshot Title position here Your title sit over it @hedshot MAKING SENSE Review questions for results: give a score out of 10… ...maybe add tags (‘codes’) to track common experiences Test Participant #1: Test Participant #2: Test Participant #3: Test Participant #4: Test Participant #5: Test Participant #6: User Journey / Task / Question #1 User Journey / Task / Question #2 User Journey / Task / Question #3 User Journey / Task / Question #4 User Journey / Task / Question #5 AVERAGE / STATISTCAL TREATMENT Handy Quantitative Converter Table Tags / Codes •  [e.g.] •  [Enjoyed] •  [Hated] •  [Difficulty with navigation] •  [Uses keyboard] •  [Uses mouse] •  [etc] •  [make your own]
  129. 129. @hedshot Title position here Your title sit over it @hedshot MAKING SENSE Review questions for results: give a score out of 10…with codes/tags Test Participant #1: Test Participant #2: Test Participant #3: Test Participant #4: Test Participant #5: Test Participant #6: User Journey / Task / Question #1 User Journey / Task / Question #2 User Journey / Task / Question #3 User Journey / Task / Question #4 User Journey / Task / Question #5 AVERAGE / STATISTCAL TREATMENT Handy Quantitative Converter Table 7 – [enjoyed] 6 – [nav] 5 – [button] 2 – [had to think] 2 – [nav] 9 – [enjoyed] 9 – [enjoyed] 8 – [enjoyed] 5 – [disliked flow] 7 – [enjoyed]
  130. 130. @hedshot Title position here Your title sit over it @hedshot NOW YOUR TURN: REVIEW DATA Task very simply, re-read your results, score Do watch out for skew/bias Don’t base it on interpretation Aim Score each task, perhaps [tag] them if you detect interesting phenomena Bonus! As you go, solutions/actions/levers naturally suggest themselves 5-10 minutes to analyse your results…. Test Participant #1: Test Participant #2: Test Participant #3: Test Participant #4: Test Participant #5: Test Participant #6: User Journey / Task / Question #1 User Journey / Task / Question #2 User Journey / Task / Question #3 User Journey / Task / Question #4 User Journey / Task / Question #5 AVERAGE / STATISTCAL TREATMENT Handy Quantitative Converter Table
  131. 131. @hedshot Title position here Your title sit over it @hedshot REVIEW OUR REVIEW… What have we discovered: what Levers has each team identified…
  132. 132. @hedshot Title position here Your title sit over itSO…. *DO* Avoid skew and bias •  Test with actual users - don’t test with your mum/CEO/company staff/ etc •  start out with clear aims •  apply rigorous enough process •  results in clear actions you can be confident in Standardise your method so results – and the process - is repeatable AND
  133. 133. @hedshot Title position here Your title sit over itBOOSTER UX RESEARCH AWARDS! 1.  Category: Best 2016 Observation 2.  Category: Best 2016 Research Team
  134. 134. @hedshot Title position here Your title sit over it @hedshot
  135. 135. @hedshot Title position here Your title sit over it @hedshot Q & A
  136. 136. @hedshot Title position here Your title sit over it @hedshot THANKS FOR LISTENING! MemberVideo Council Whitelisted mike@unrulymedia.com @hedshot
  137. 137. @hedshot Title position here Your title sit over it @hedshot PROPS! Orange bus seniorgif.com Team unicorn Dark parodies Umbrella corporation Memegenerator.net Carter reid Bodies in the crawlspace Hollywood Meme Center destroyer.tumbler.com Someecards SHIELD Star Trek Elvis.ro Gliphy Gifsoup.com The department of homeland security Lolcats everywhere  Random Kittens Ghosthunters The Muppets icanhascheeseburger.com soundbible.com Marvel Anyone else I forgot!! Sony Playstation The NY transit auth. The Wachowskis Jeff Gothelf Jon Innes Thoughtworks Manalo Blahnik Jaguar Disney Bentley Ryan Gosling Clint Eastward Etc…
  138. 138. @hedshot Title position here Your title sit over it @hedshot WELCOME!

×