Remote customer feedback

1,577 views
1,458 views

Published on

How to get out of the building without getting out of the building: a practical guide on how to interview and observe
customers remotely

Remote customer feedback

  1. 1. presents
  2. 2. How to get out of the building without getting out of the building@UXdna
  3. 3. What I won’t talk about How to use different remote testing tools Advantages and pitfalls of each type (moderated, unmoderated, automatic, live, etc.) For this you can read Nate Bolt’s amazing book ‘Remote Research’ or watch his presentation here: http:// www.slideshare.net/boltpeters/remote-research-the-book-the-talk@UXdna
  4. 4. This is about A practical guide on how to interview and observe customers remotely@UXdna
  5. 5. PREPARE & ANTICIPATE@UXdna
  6. 6. Example A Spanish company wants to launch an online restaurant listing and booking service in Brazil.@UXdna
  7. 7. The process 2. Who? Who you need to talk to/test with? 1. What? What do you need to learn 3. How? from it? How can you get there, step by step@UXdna
  8. 8. PREPARE & ANTICIPATE@UXdna
  9. 9. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype@UXdna
  10. 10. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype@UXdna
  11. 11. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP@UXdna
  12. 12. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP@UXdna
  13. 13. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP VOICE VOICE VOICE VOICE WEBCAM WEBCAM WEBCAM WEBCAM SCREEN SHARING SCREEN SHARING SCREEN SHARING@UXdna
  14. 14. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP VOICE VOICE VOICE VOICE WEBCAM WEBCAM WEBCAM WEBCAM SCREEN SHARING SCREEN SHARING SCREEN SHARING@UXdna
  15. 15. Example An Axure prototype of a 3 page journey Will need to build and test prototype Will need screen sharing Will need voice Will need to define a scenario to users Will need to explain to users that this is a prototype, not a real site@UXdna
  16. 16. Example An Axure prototype of a 3 page journey Will need to build and test prototype Will need screen sharing Will need voice Will need to define a scenario to users Will need to explain to users that this is a prototype, not a real site@UXdna
  17. 17. 2. Who? RECRUITMENT A. Location B. Behaviours C. Channels D. What to say? What do you need Where are the What will you need to know about them people you will Where can you to say to get their to make them good need to interview/ find them? attention and how candidates for a test with? to do it test/interview?@UXdna
  18. 18. 2. Who? Recruitment A. LOCATION Where are the people you will need to interview/test with? ACCESS TIME ZONE LANGUAGES VIEWERS TO TECHNOLOGYAre the potential Do I speak the local What is the country’s Will I be able to streamparticipants on my time language? internet connection to viewers as well?zone? speed like? Do I need to hireWhat are the times of translation? Skype says 300 kbpsthe day I will be able to but you need at leasttalk to them/test? Do I need to recruit 512 kbps participants who speak English? Speedtest.net@UXdna
  19. 19. Example@UXdna
  20. 20. Example@UXdna
  21. 21. Example PORTUGUESE 5 HOURS BEHIND OK SPEED@UXdna
  22. 22. 2. Who? Recruitment B. BEHAVIOURS What do I need to know about them? TIGHTLY DEFINED SET OF LOOSE SET OF BEHAVIOURS BEHAVIOURS I need to talk to teenagers who I need to talk to have an Android people who go phone and out dining and download at least check for good 3 new games a restaurants online week, every week@UXdna
  23. 23. Example TIGHTLY DEFINED SET OF LOOSE SET OF BEHAVIOURS BEHAVIOURS We needed to know they were people who are likely to be our current hypothesis customers, i.e. anyone with a certain level of income (+ 5 minimum wages), who habitually looked for advice about places to eat online@UXdna
  24. 24. Example TIGHTLY DEFINED SET OF LOOSE SET OF BEHAVIOURS BEHAVIOURS We needed to know they were people who are likely to be our current hypothesis customers, i.e. anyone with a certain level of income (+ 5 minimum wages), who habitually looked for advice about places to eat online@UXdna
  25. 25. 2. Who? Recruitment C. CHANNEL Where can I find them? Easier Difficult List of emails from Publish an ad or Intercept as they Call centre contacts current customers post in a forum use website LIST OF EMAIL ADDRESSES or PHONE NUMBERS@UXdna
  26. 26. Example Easier Difficult List of emails from Publish an ad or Intercept as they Call centre contacts current customers post in a forum use website We published a job position for NOTE: It is important to ‘site tester’ in free job posting get an idea of what sites websites and ‘Craigslist’ style people use in the country you are testing lists@UXdna
  27. 27. Example Easier Difficult List of emails from Publish an ad or Intercept as they Call centre contacts current customers post in a forum use website We published a job position for NOTE: It is important to ‘site tester’ in free job posting get an idea of what sites websites and ‘Craigslist’ style people use in the country you are testing lists@UXdna
  28. 28. 2. Who? Recruitment D. WHAT TO SAY? D1 D2 D3 Create a screener Write an email Collect survey or list of questions through which you results, filter and with which you will send the SURVEY select SURVEY LINK ANSWERS can filter out survey link unsuitable participants@UXdna
  29. 29. 2. Who? Screener questionnaire D. WHAT TO SAY? What makes a good screener? A GOOD SURVEY MAKES IT EASY TO FILTER 1 2 3 4 List 3 List 3 behaviours Based on these Don’t ask ‘would characteristics that make factors, write you’ questions. someone should someone unlikely questions that Ask ‘what was the display to make to be your have discrete last time you...’ them likely to be a customer answers (like ‘yes’ questions customer and ‘no’ or multiple choice)@UXdna
  30. 30. Example 1 2 3 4 List 3 List 3 behaviours Based on these Don’t ask ‘would characteristics that make factors, write you’ questions. someone should someone unlikely questions that Ask ‘what was the display to make to be your have discrete last time you...’ them likely to be a customer answers (like ‘yes’ questions customer and ‘no’ or multiple choice) Must live in SP Only seek Where do you live? What do you recommendations (a-d) usually do... Must have dined in from friends a restaurant about When was the last Would you which they found Always go to the time you visited a generally... reviews online in same restaurant, new restaurant? the last 7 days don’t like trying new places Which, of the sites Must have below, have you consulted at least Only dine out for visited in the last 7 1 review site very special days? occasions@UXdna
  31. 31. Example 1 2 3 4 List 3 List 3 behaviours Based on these Don’t ask ‘would characteristics that make factors, write you’ questions. someone should someone unlikely questions that Ask ‘what was the display to make to be your have discrete last time you...’ them likely to be a customer answers (like ‘yes’ questions customer and ‘no’ or multiple choice) Must live in SP Only seek Where do you live? What do you recommendations (a-d) usually do... Must have dined in from friends a restaurant about When was the last Would you which they found Always go to the time you visited a generally... reviews online in same restaurant, new restaurant? the last 7 days don’t like trying new places Which, of the sites Must have below, have you consulted at least Only dine out for visited in the last 7 1 review site very special days? occasions@UXdna
  32. 32. 2. Who? Screener questionnaireMY SURVEY Only show me answers from people who replied marked the option ‘last 7 days’ to the question “When was the last time you looked online for restaurant reviews?” 456 respondents = 36 fit our criteria@UXdna
  33. 33. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, You are receiving this email because you replied to a job post on [name of website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], ([website link here, if it exists]). We are working on a project about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like to know a little more about you. Could you please answer the questions on the following link (link to survey, very visible)@UXdna
  34. 34. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, This is why you are receiving this email from someone you don’t know. YouYou are interestedemail because yousay, keep onjob post on [name of are receiving this in what I have to replied to a reading. website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], ([website link here, if it exists]). We are working on a project about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like to know a little more about you. Could you please answer the questions on the following link (link to survey, very visible)@UXdna
  35. 35. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, This is why you are receiving this email from someone you don’t know. YouYou are interestedemail because yousay, keep onjob post on [name of are receiving this in what I have to replied to a reading. website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], I([website link here, am exists]). We are working on a project This is who am, this is what I if it working on and these are my credentials about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like to know a little more about you. Could you please answer the questions on the following link (link to survey, very visible)@UXdna
  36. 36. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, This is why you are receiving this email from someone you don’t know. YouYou are interestedemail because yousay, keep onjob post on [name of are receiving this in what I have to replied to a reading. website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], I([website link here, am exists]). We are working on a project This is who am, this is what I if it working on and these are my credentials about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like Weknow a little more including you in the research, but I needquestionsif you to are interested in about you. Could you please answer the to know on the are the right person first following link (link to survey, very visible)@UXdna
  37. 37. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, This is why you are receiving this email from someone you don’t know. YouYou are interestedemail because yousay, keep onjob post on [name of are receiving this in what I have to replied to a reading. website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], I([website link here, am exists]). We are working on a project This is who am, this is what I if it working on and these are my credentials about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like Weknow a little more including you in the research, but I needquestionsif you to are interested in about you. Could you please answer the to know on the are the right person first following link (link to survey, very visible) VERY VISIBLE LINK TO SURVEY@UXdna
  38. 38. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? We will be in touch if you are selected for the interview. You should be available for a phone call on the [day & date] or [day & date] morning. We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You should provide us with a Skype name or a landline number we can call you on. Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account. Thank you already for your interest in participating Best regards [your name here]@UXdna
  39. 39. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? We Next steps and if you are selected for the interview. You should be available, will be in touch confirmation of availability (dates). If you are not available for a phone call on the [day further or [day & date] morning. don’t bother going any & date] We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You should provide us with a Skype name or a landline number we can call you on. Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account. Thank you already for your interest in participating Best regards [your name here]@UXdna
  40. 40. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? We Next steps and if you are selected for the interview. You should be available, will be in touch confirmation of availability (dates). If you are not available for a phone call on the [day further or [day & date] morning. don’t bother going any & date] We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You of what provide us with a have: Skype, or a landline number we Explanation should you will need to Skype name phone,computer with canunrestricted internet connection, etc. call you on. Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account. Thank you already for your interest in participating Best regards [your name here]@UXdna
  41. 41. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? We Next steps and if you are selected for the interview. You should be available, will be in touch confirmation of availability (dates). If you are not available for a phone call on the [day further or [day & date] morning. don’t bother going any & date] We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You of what provide us with a have: Skype, or a landline number we Explanation should you will need to Skype name phone,computer with canunrestricted internet connection, etc. call you on. Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account. How I will pay for your time. If you are not happy, let me know Thank you already for your interest in participating Best regards [your name here]@UXdna
  42. 42. 3. How? Show time! A. Set up B. Pilot C. Test D. Post test What you are Test your whole testing and how What can I improve setup. What can go Show time! you are collecting for next time? wrong? feedback@UXdna
  43. 43. 3. How? A. SET UP What do I need to prepare for the test?@UXdna
  44. 44. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype@UXdna
  45. 45. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype@UXdna
  46. 46. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP@UXdna
  47. 47. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP@UXdna
  48. 48. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP VOICE VOICE VOICE VOICE WEBCAM WEBCAM WEBCAM WEBCAM SCREEN SHARING SCREEN SHARING SCREEN SHARING@UXdna
  49. 49. Example A scenario so you can situate people: “Imagine it is Sunday morning and you need to find a place to have lunch with a friend who just Prototype arrived in town. You want to try something new” Prototype was built with wireframes. It was important to let participants know that they JOURNEY were not going to be looking at a real website, + even though buttons were clickable A PROTOTYPE + SCRIPT OR Themes and questions you would like to TOPIC MAP discuss with participants. Do they understand the order of the search results list? Do they see and understand the ratings ? VOICE WEBCAM Does my set up work? SCREEN SHARING Test all technology works and is compatible RECORDING with each other@UXdna
  50. 50. 3. How?@UXdna
  51. 51. 3. How? PREPARE & ANTICIPATE@UXdna
  52. 52. 3. How? PREPARE & ANTICIPATE What can go wrong, will go wrong@UXdna
  53. 53. 3. How? PREPARE & ANTICIPATE What can go wrong, will go wrong PILOT, PILOT, PILOT! HAVE A FALL BACK PLAN@UXdna
  54. 54. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test@UXdna
  55. 55. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Turn off webcam Internet connection: •Speedtest.net can give you an idea of •Kill other applications gets really slow or drops the connection speed - or at least let •Connections often better at completely you know if participant has dial speed weekends or holidays •Test out screen sharing Screen sharing: •Send ‘how to’ step by step •Alternative screen sharing drags a lot or stops completely screenshots. platform (joinme, WebEx, or participant can’t start it •Check Skype version (4.0 ok, 5.0 GoToMeeting, Co-pilot) better) Can’t hear participant •Test your mic & speakers •Call a landline and still do He can hear you, but you can’t •Test their mic & speakers screen sharing hear him - or the other way around@UXdna
  56. 56. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Turn off webcam Internet connection: •Speedtest.net can give you an idea of •Kill other applications gets really slow or drops the connection speed - or at least let •Connections often better at completely you know if participant has dial speed weekends or holidays •Test out screen sharing Screen sharing: •Send ‘how to’ step by step •Alternative screen sharing drags a lot or stops completely screenshots. platform (joinme, WebEx, or participant can’t start it •Check Skype version (4.0 ok, 5.0 GoToMeeting, Co-pilot) better) Can’t hear participant •Test your mic & speakers •Call a landline and still do He can hear you, but you can’t •Test their mic & speakers screen sharing hear him - or the other way around@UXdna
  57. 57. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Turn off webcam Internet connection: •Speedtest.net can give you an idea of •Kill other applications gets really slow or drops the connection speed - or at least let •Connections often better at completely you know if participant has dial speed weekends or holidays •Test out screen sharing Screen sharing: •Send ‘how to’ step by step •Alternative screen sharing drags a lot or stops completely screenshots. platform (joinme, WebEx, or participant can’t start it •Check Skype version (4.0 ok, 5.0 GoToMeeting, Co-pilot) better) Can’t hear participant •Test your mic & speakers •Call a landline and still do He can hear you, but you can’t •Test their mic & speakers screen sharing hear him - or the other way around@UXdna
  58. 58. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test@UXdna
  59. 59. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Recruit spare participants or ‘floaters’ for each slot allocated for test/ No show interview, who will be waiting in case •If both participant and floater participant doesn’t show up at someone doesn’t show up fail you, check if other floaters show time •They only get paid if they are needed can fill the gap. (you should make this clear when recruiting) •Send them a link to the prototype/ mock prototype (if you don’t want to Prototype/site show real thing before the test) Can participants see the prototype •Screen share it! •Don’t waste time, get your exactly as I planned? •Send link to a friend first spare participant in there Can users access the site? •Ask them to identify something on the screen@UXdna
  60. 60. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Recruit spare participants or ‘floaters’ for each slot allocated for test/ No show interview, who will be waiting in case •If both participant and floater participant doesn’t show up at someone doesn’t show up fail you, check if other floaters show time •They only get paid if they are needed can fill the gap. (you should make this clear when recruiting) •Send them a link to the prototype/ mock prototype (if you don’t want to Prototype/site show real thing before the test) Can participants see the prototype •Screen share it! •Don’t waste time, get your exactly as I planned? •Send link to a friend first spare participant in there Can users access the site? •Ask them to identify something on the screen@UXdna
  61. 61. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test@UXdna
  62. 62. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Try recording the conversation you •Turn off your webcam Recording have during the pilot session •Record voice only Does my recording set up work •WARNING: Silverback is a little odd •Use spare participant with the screen sharing and voice? when webcam + screen sharing! • Language •You shouldn’t get to the actual I can’t understand what the •Have a conversation with the test with someone who participant is saying (mine or participant ahead of time and you will doesn’t fit. participant’s language skills aren’t be able to gauge how you •If you did, and only realise it at as good as I thought, accent is understand each other the time of test, call in a spare really difficult to understand)@UXdna
  63. 63. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Try recording the conversation you •Turn off your webcam Recording have during the pilot session •Record voice only Does my recording set up work •WARNING: Silverback is a little odd •Use spare participant with the screen sharing and voice? when webcam + screen sharing! • Language •You shouldn’t get to the actual I can’t understand what the •Have a conversation with the test with someone who participant is saying (mine or participant ahead of time and you will doesn’t fit. participant’s language skills aren’t be able to gauge how you •If you did, and only realise it at as good as I thought, accent is understand each other the time of test, call in a spare really difficult to understand)@UXdna
  64. 64. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test@UXdna
  65. 65. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •WARNING: first impressions can be deceiving, but talking to the Bad recruiting: participant will give you an idea. You the participant doesn’t sound like a •Spare participant can also ask extra screening customer questions, or check answers in their questionnaire. •It will always be a problem Rapport •The pilot helps! Main drawback of not being in the •If at the time of testing •Smile & be nice room with participant is: How can I participants don’t feel •Webcam, so they know you are not a make them trust me and tell me comfortable for some reason, machine stuff? don’t insist, call in your spare! •Show your surroundings, if you can (don’t ask them to do the same)@UXdna
  66. 66. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •WARNING: first impressions can be deceiving, but talking to the Bad recruiting: participant will give you an idea. You the participant doesn’t sound like a •Spare participant can also ask extra screening customer questions, or check answers in their questionnaire. •It will always be a problem Rapport •The pilot helps! Main drawback of not being in the •If at the time of testing •Smile & be nice room with participant is: How can I participants don’t feel •Webcam, so they know you are not a make them trust me and tell me comfortable for some reason, machine stuff? don’t insist, call in your spare! •Show your surroundings, if you can (don’t ask them to do the same)@UXdna
  67. 67. 3. How? C. TEST How to set it up 1. Establish Skype call with viewers (separate username) SKYPE 2. Turn off their webcam and mic VIEWER SKYPE PRO 3. Add participant to YOU the conversation SKYPE + Screen sharing 4. Ask them to turn on web cam 5. Explain who are the other people in PARTICIPANT the call 6. Begin test!@UXdna
  68. 68. 3. How? Checklist C. TEST You Participant Check participant internet connection Check your own internet connection speed speed Check Skype Pro credit ($$) Check Skype mic & speakers Check screen sharing works and they Check Skype mic know how to do it Check they can see your site or prototype well Add participant to Skype contact list, so you know when they are available Check Paypal email address (might be different from the one participant used Check recording application/device when replying to ad) (Highlight, Silverback, notebook) is working Get their landline & mobile phone number@UXdna
  69. 69. Thank you! sarah@uxdna..c o. uk@UXdna

×