Downloaded 27 times
Uploaded byIan Collingwood
Ian Collingwood - LeanStartupMachine Rotterdam
The document discusses the importance of combining qualitative and quantitative research methods in user experience (UX) design to gain deeper insights into user behavior. It emphasizes the need for a collaborative approach involving team members and diverse tools for user testing while warning against relying solely on quantitative data. The author advocates for understanding the 'why' behind user actions to enhance product design and user satisfaction.
Ian Collingwood - LeanStartupMachine Rotterdam
- 1. Don’t forget the humans @johnnyforeigner iancollingwood.com ian@uxdna.co.uk
- 2. UX Lean Startup @johnnyforeigner
- 3. UX Lean Startup @johnnyforeigner
- 4. The UCD Cycle ISO 13407 @johnnyforeigner
- 5.
- 6.
- 7. Eric & Stevesay... “Get out of the building!” @johnnyforeigner
- 8. Eric & Stevesay... “Get out of the building!” @johnnyforeigner Image by Jeff Gotthelf
- 9. UX people aregood at getting out of the building @johnnyforeigner
- 10. UX people aregood at getting out of the building @johnnyforeigner
- 11. UX people aregood at getting out of the building Quantitative @johnnyforeigner
- 12. UX people aregood at getting out of the building Quantitative Qualitative @johnnyforeigner
- 13. UX people aregood at getting out of the building Quantitative Qualitative Generative @johnnyforeigner
- 14. UX people aregood at getting out of the building Quantitative Qualitative Generative Evaluative @johnnyforeigner
- 15. UX people aregood at getting out of the building Quantitative Qualitative Depth Interviews Contextual Inquiry Surveys Ethnography Generative Interviews Diary Studies Participatory Design (Co-creation) Mental Models Web Analytics Multi-variant Testing Usability Testing Remote Card Sorting Moderated Card Sorting Evaluative Clicktale / CrazyEgg (et al) Wizard of Oz Testing Treejack (et al) Desirability Toolkit Chalkmark / 5-Second Test (et al) R.I.T. Usertesting.com (et al) @johnnyforeigner
- 16. UX people aregood at getting out of the building Quantitative Qualitative Depth Interviews Contextual Inquiry Surveys Ethnography Generative Interviews Diary Studies Participatory Design (Co-creation) Mental Models Web Analytics Multi-variant Testing Usability Testing Remote Card Sorting Moderated Card Sorting Evaluative Clicktale / CrazyEgg (et al) Wizard of Oz Testing Treejack (et al) Desirability Toolkit Chalkmark / 5-Second Test (et al) R.I.T. Usertesting.com (et al) @johnnyforeigner
- 17. UX people aregood at getting out of the building Quantitative Qualitative Depth Interviews Contextual Inquiry Surveys Ethnography Generative Interviews Diary Studies Participatory Design (Co-creation) Mental Models Web Analytics Multi-variant Testing Usability Testing Remote Card Sorting Moderated Card Sorting Evaluative Clicktale / CrazyEgg (et al) Wizard of Oz Testing Treejack (et al) Desirability Toolkit Chalkmark / 5-Second Test (et al) R.I.T. Usertesting.com (et al) @johnnyforeigner
- 18. UX people aregood at getting out of the building Quantitative Qualitative Depth Interviews Contextual Inquiry Surveys Ethnography Generative Interviews Diary Studies Participatory Design (Co-creation) Mental Models Web Analytics Multi-variant Testing Usability Testing Remote Card Sorting Moderated Card Sorting Evaluative Clicktale / CrazyEgg (et al) Wizard of Oz Testing Treejack (et al) Desirability Toolkit Chalkmark / 5-Second Test (et al) R.I.T. Usertesting.com (et al) @johnnyforeigner
- 19. UX people aregood at getting out of the building Quantitative Qualitative Depth Interviews Contextual Inquiry Surveys Ethnography Generative Interviews Diary Studies Participatory Design (Co-creation) Mental Models Web Analytics Multi-variant Testing Usability Testing Remote Card Sorting Moderated Card Sorting Evaluative Clicktale / CrazyEgg (et al) Wizard of Oz Testing Treejack (et al) Desirability Toolkit Chalkmark / 5-Second Test (et al) R.I.T. Usertesting.com (et al) @johnnyforeigner Adapted from material by Johanna Kollman Janice Fraser, Nate Bolt, Christian Rohrer
- 20. http://trapblog.org/wp-content/uploads/ackbar_mouse_trap2.jpg Do n’t fa ll in to th e trap of re ly ing exclusi ve ly on an al yt ics, A/B te st s an d co nversio ns . Tr ia ng ul ate yo ur data w ith qu al itati ve re se arch . Un de rs ta nd th e “w hy” as wel l as th e “w hat”
- 21. ingredients: two rooms one wifi network two computers some cheap or free software five participants your whole team your product or prototype five colours of mini post it notes blu-tack your eyes & your ears your brain & your heart @johnnyforeigner
- 22. Room 1 Skype and a screensharing tool Your Product or MVP Read Steve K rug’s bo ok : “R oc ke t Su rger y M ade Ea sy” fo r great in tro to pl an ni ng & ru nn ing us er te st ing se ss io ns . Moderator “Participant” (aka Customer) (you) @johnnyforeigner
- 23. Room 2 Journey @johnnyforeigner
- 24. Room 2 Journey @johnnyforeigner
- 25. User 1 User 2 User 3 User 4 User 5 Us e th e sm al le st si ze of Po st-I t no te th at yo u ca n fi nd (38mm x 51mm ). Yo u’ll ne ed a di ff eren t co lo ur no te fo r each pe rs on yo u are te st ing w it h. @johnnyforeigner
- 26. ADD AN IMATIO key: ER N ATED ORD AUTOM G TOOL? TO IM P ROVE T R ACK IN V ISI BI LITY? ISSUE GOOD FEATURE IDEA/ BY CONFUSED REQUEST SOLUTION “I LIK E H OW IT P R EDICTS WHAT I MENU TY P E” LABE LLING Wri te in ALL CAPS Wri te clea rl y & us e “q uo te m ar ks” fo r us er commen ts . Bu t ac ti on s & us er be havi ou r are more va lu able th an op in io ns . @johnnyforeigner
- 27. ADD AN IMATIO key: ER N ATED ORD AUTOM G TOOL? TO IM P ROVE T R ACK IN V ISI BI LITY? ISSUE GOOD FEATURE IDEA/ BY CONFUSED REQUEST SOLUTION “I LIK E H OW IT P R EDICTS WHAT I MENU TY P E” LABE LLING Adding th es e ic on s m ay m ak e yo ur an al ys is ea si er. Th ey are no t es se nt ia l th ough . And yo u ca n m ak e up yo ur ow n, of co urse . @johnnyforeigner
- 28. Room 2 Journey At th e en d of th e fi rs t se ss io n yo ur ro om m ight lo ok lik e th is . @johnnyforeigner
- 29. Room 2 Journey Du ri ng yo ur di sc us si on af te r th e se ss io n, re mov e du pl ic ate no te s an d re w ri te any no te s th at are un clea r. @johnnyforeigner
- 30. Room 2 Journey Du ri ng yo ur di sc us si on af te r th e se ss io n, re mov e du pl ic ate no te s an d re w ri te any no te s th at are un clea r. @johnnyforeigner
- 31. Room 2 Journey A pict ure bu ild s up as yo u se e more us ers. @johnnyforeigner
- 32. Room 2 Journey By th e en d it m ight lo ok lik e th is . @johnnyforeigner
- 33. Room 2 Journey 2 1 Bu t m any no te s of one M any co lo urs mea n th at co lo ur m ay te ll yo u th e proble m is common to somet hi ng in te re st ing us ers, so th is is worth ab ou t th is pa rt ic ul ar us er lo ok ing in to. . What m ak es th em Bu t no t al l proble ms th at di ff eren t? Do es th is adju are common are st yo ur pe rc ep ti on of yo ur ne ce ss ar ily im po rt an t. ta rget us er ? @johnnyforeigner
- 34. Room 2 Journey By th e en d of th e day th e te am sh ou ld ag re e on th e m ai n proble ms an d th ei r pr io ri ty. M ak e a no te of any ot he r in sigh ts ab ou t us er be havi ou r th at emerge d du ri ng th e day. Do n’t lo se th is vi ta l data . @johnnyforeigner
- 35. Thanks @johnnyforeigner ian@uxdna.co.uk iancollingwood.com
Editor's Notes
- #2 About me\nDesigning experiences since about 93...\nWeb since 1996\nAmberlight\nUXdna\n\n\nFirst I want to talk about a few of things that people in the UX field have learned over the years that can help those practising a lean startup approach\n\nThen I’m going to show you a method you can use to get really rich insights into customer behaviour in less than a day and a half and for less than 500 quid. (Or euros).\n
- #3 There’s a lot of crossover between Lean Startup and User Centred Design. \n\nAt Amberlight we made a really nice little business helping companies test their products with users. They’d spend months developing a product and then they’d hand it over to us to test.\n\nAnd what they wanted was feedback on usability, but what we often found was that they problems were not usability they were user need. Nobody needed the product. So we’d try to get them to involve us (and therefore users) earlier. We failed.\n\nWe’d talk to them about defining product need, building a prototype and then testing it with users. We even drew them a diagram...\n\n\n
- #4 UCD has been around so long it even has an ISO standard (can’t get much more mainstream than that)\n\nThen Lean Startup came along and for me at least it was a revelation. Suddenly business people were talking about the kind of things we’d been yelling about for years.\n\n\n
- #5 Lean Startup = UCD applied to business models\n
- #6 \n
- #7 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #8 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #9 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #10 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #11 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #12 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #13 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #14 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #15 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #16 All of these methods are aimed at one thing: \nHelping you to understand your customer and what they need\n\nWhich one you choose will depend on where you are in the product development process and what you’re trying to learn\n\nToday we’re mainly going to look at conducting in-depth interviews, because the techniques you’ll learn will help with almost all the other qualitative techniques.\n
- #17 There are literally hundreds of tools out there now that can tell you something about what users did. With stuff like Website Optimiser, Wufoo, SurveyGizmo, Crazy Egg, Clicktale, Chalkmark it feels like you have a lot to work with.\n\nAnd it’s hard to argue against this - the numbers don’t lie, right? Either people clicked on the button or they didn’t, right? Surely that’s all we need to know? \n\nWell, yes, and no. It’s true, you can find out a lot from these tools, and I use them all the time.\n\nBut many times you’re looking at a heatmap or a mouse trail or a conversion funnel that shows me exactly *what* a user did, but you’re still left wondering... WHY?\n\nIt feels a bit like being an archeologist - you’re sifting through these digital artefacts that represent the activities of some long lost user and you’re trying to understand their significance. Why does everybody miss that button? Why aren’t they filling in the form right? What is it about this page that makes everyone drop out?\n\nBasically - what were they thinking? \n\nAnd that’s the big question that will help you to steer your product to success. And sadly - the stats just won’t get you there. \n\nBecause behind every click, every mouse move, every heatmap is a hand, and attached to that hand is a human, and if you don’t understand that human, you won’t make great products. \n\nThis is what Steve Blank means when he says you need to “get out of the building”. Metrics aren’t getting you out of the building. They can become a trap that keeps you at one remove from your customers. What Steve means is: meet your customers, listen to them, watch them. \n\nBest of all - watch them using your product. \n\nNow, in the past, this meant spending a few weeks and a fair amount of cash on planning and running some formal user testing. This could be several weeks and several thousand pounds. \n\nAnd honestly, that’s often time and money well spent - because there’s nothing like seeing the grimaces, sighs, the frustration and curses of people who are simply unable to understand your product. It really forces you to confront the reality inside the minds of your customers - and apart from anything else, it gives you fantastic ideas and real empathy with these people. That should be your secret weapon - not tech, not ideas, but knowing your customers better than the competition does.\n\nAnd for this reason, regular face-to-face customer research is something you should be doing at least twice a month as a matter of course - not just during customer discovery phase, but forever. \n\nAnd before you freak out about the cost - you’re about to see how to do it for under 500 quid and less than a day and a half per round.\n
- #18 \n
- #19 So you’ve got two rooms. This room is the Testing Room. \n\nYou’ve got a computer in each room, both connected to the same wifi network. \n\nIt helps if both are Macs. \n\nYou’ll be recording what happens on the screen of the computer here, using some cool software called Silverback, which is made by the guys at Clearleft. It’s really simple, and it’ll cost you just 60 bucks, which is a steal.\n\nThe people in the other room will also be watching and for this you’ll be streaming what happens here through to their computer using two neat little bits of software. One is called Soundfly by Abyssoft and the other is called LineIn, by Rogue Amoeba. Both are great little apps which are free, but if you use them, do the decent thing and donate to the guys that wrote them, because they’re both really neat, and they’re starving startups, just like you. \n\nAlso on this computer is your product or prototype. \n\nNow, this could be your Minimum Viable Product, but I suggest you think about this type of testing even before you get to that point. So you might want to think about something even more stripped down - something that has no real functionality, but just represents the interactions that a customer might have with your product. You can call it a “Minimum Testable Prototype”. It’s basically a clickable mockup and you can make it in Keynote or Powerpoint.\n\nNow, we don’t have time to tell you how to write a test plan and how to run a session, so buy yourself a great book by Steve Krug called Rocket Surgery Made Easy. Best 12 quid you’ll spend. \n\nBut you do need to know a bit about finding participants. You should have already thought about your customers and who they are. So now you just need to find some of them and get them to come and test with you. You can do this via friends and family, which will cost you nothing in recruitment fees, but it can take time. So often your best bet is a specialist agency. They’ll take a profile from you and charge you between 30 and 50 quid per person to find the people. It’ll save you time, but do take care to brief them properly or you might get people who aren’t quite what you need.\n\nAlso in the room is you. You will guide the participant through the tasks.\nSo that’s the Testing Room.\n
- #20 The second room is the Viewing Room. \n\nThe rest of your team is here. All of them. \n\nThis method works if you stick to rules, and one of the rules is “Everyone participates”. You’ll see why later.\n\nOn the walls of this room you’re going to stick up printouts of each screen involved in each journey you are testing. Print them out as big as you can and stick them onto sheets that are bigger so you can leave a few inches of margin around them.\n\nIt should look like this.\n
- #21 \nNow you set up the computer so you can see and hear what’s happening in the Testing Room. You might want some external speakers and a bigger screen (even a projector) if there are more than about 3 of you. \n\nYou are all going to be watching what is happening, and making notes.\n\nHere’s the second rule: When testing is happening, the room is silent. This is learning time. There’s time to discuss what you see later - if you talk now you’ll miss something.\n
- #22 You’ll be making notes on mini Post Its, and you’ll be using a different colour for each participant. You’ll see why in a minute.\n\nIn this example, we’ll be using yellow notes for the first participant.\n
- #23 When you make notes, you want to capture just one thing on each Post It you use. \n \nDon’t write down everything - choose things that seem relevant or useful. \n\nWrite down good things and bad things, and capture any ideas you have while watching and any suggestions, frustrations or needs that the participants mention.\n\nIf it’s a quote, write quotation marks.\n\nEach time you write a note, stick it onto the page it relates to, close to the area it concerns. If it’s about the whole page, stick it on the border.\n
- #24 When you make notes, you want to capture just one thing on each Post It you use. \n \nDon’t write down everything - choose things that seem relevant or useful. \n\nWrite down good things and bad things, and capture any ideas you have while watching and any suggestions, frustrations or needs that the participants mention.\n\nIf it’s a quote, write quotation marks.\n\nEach time you write a note, stick it onto the page it relates to, close to the area it concerns. If it’s about the whole page, stick it on the border.\n
- #25 At the end of the first session, the wall should look something like this. (But with more notes)\n\nYou’ll now spend about 15 minutes doing a quick wrap up of what you all saw. Review the post it notes, make sure all are legible and if there are duplicates, remove all the extras.\n\nThis is just a quick chat. You’ll have a full discussion at the end of the day.\n
- #26 Here’s what it might look like after the duplicates are removed.\n\nWhen the second participant arrives for their session, the discussion stops, your moderator pops back into the Testing Room, and you ditch the yellow Post It notes in favour of Orange\n
- #27 And after that session, you repeat your quick discussion, de-duplicate the post its and move to a new colour.\n\nAnd of course, as the days goes on, it starts to build up like this.\n
- #28 \n
- #29 After the final session, you’ll have walls that look a bit like these. \n\nYou’ll notice that some areas have lots of different colours - this means it generated a lot of feedback from many different users (could be good, could be bad). \n\nOther areas have a lot of post its of just one colour. This indicates that only one particular user had problems with that area. \n\nThese two different facts can give you a quick start on where to focus your attention. I don’t have time to talk you through how you might manage this discussion, but I’m happy to chat about it afterwards.\n\nBut the main this is to make sure that when you leave the room that evening, you have some clear, actionable decisions about where you will focus your efforts in the coming days.\n\nYour whole team will have been involved in absorbing what your customers think and feel. Your whole team is engaged in collecting nuggets of data that will help them decide where to go next. \n\nAs I said before, this kind of engagement - backed up with a solid metrics - helps build a culture of customer centred thinking in your startup. I recommend it!\n\nAnd that’s it. \n
- #30 After the final session, you’ll have walls that look a bit like these. \n\nYou’ll notice that some areas have lots of different colours - this means it generated a lot of feedback from many different users (could be good, could be bad). \n\nOther areas have a lot of post its of just one colour. This indicates that only one particular user had problems with that area. \n\nThese two different facts can give you a quick start on where to focus your attention. I don’t have time to talk you through how you might manage this discussion, but I’m happy to chat about it afterwards.\n\nBut the main this is to make sure that when you leave the room that evening, you have some clear, actionable decisions about where you will focus your efforts in the coming days.\n\nYour whole team will have been involved in absorbing what your customers think and feel. Your whole team is engaged in collecting nuggets of data that will help them decide where to go next. \n\nAs I said before, this kind of engagement - backed up with a solid metrics - helps build a culture of customer centred thinking in your startup. I recommend it!\n\nAnd that’s it. \n
- #31 After the final session, you’ll have walls that look a bit like these. \n\nYou’ll notice that some areas have lots of different colours - this means it generated a lot of feedback from many different users (could be good, could be bad). \n\nOther areas have a lot of post its of just one colour. This indicates that only one particular user had problems with that area. \n\nThese two different facts can give you a quick start on where to focus your attention. I don’t have time to talk you through how you might manage this discussion, but I’m happy to chat about it afterwards.\n\nBut the main this is to make sure that when you leave the room that evening, you have some clear, actionable decisions about where you will focus your efforts in the coming days.\n\nYour whole team will have been involved in absorbing what your customers think and feel. Your whole team is engaged in collecting nuggets of data that will help them decide where to go next. \n\nAs I said before, this kind of engagement - backed up with a solid metrics - helps build a culture of customer centred thinking in your startup. I recommend it!\n\nAnd that’s it. \n
- #32 \n