Survey research crash course


Published on

Crash course or refresher for market researchers and their vendors

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Survey research crash course

  1. 1. Primary Research: A Crash Course Part I Laurie Gelb, MPH
  2. 2. The challenge: Provide insight that integrates demographics, attitudes and behavior and turn raw data (numbers or statements) into meaningful, actionable information
  3. 3. What is marketing?• Classically, the four P’s: product, price, promotion, place (distribution) – Now, often characterized as the four C’s: consumer needs and wants, the cost to satisfy these, convenience to buy and communication, which is dialogue rather than monologue• Market research supports activities and decisions made in support of these overarching functions…from manufacturing through consumption and the target’s evaluation of the transaction – So marketing does not stop when a purchase is made or a prescription written – We want decision-makers and influencers to feel good about the decision, and to communicate that feeling to others
  4. 4. What is marketing research?• Any formal or informal attempt to ascertain facts, perceptions and/or behavioral data relating to constituents, customers, peers, prospects, influencers or other people of interest – “Do you want to see a movie tonight?” – “Why don’t you think that I deserve a promotion?” – “Do you want fries with that?” – “What is the major reason for continuing to prescribe Plavix?” – “In the last year, which of the following drugs have you taken to alleviate the symptoms of your arthritis?” – “Can I count on your vote for Senator Niceguy?”
  5. 5. Three data types = two sources of error• What is your age– a fact someone is unlikely to grossly lie about or mischaracterize [the baseline questions on a polygraph]• In the last year, how many days did you stay home from work due to illness or family issues? [“fact” for which recall may be poor or biased]• On a scale of zero to ten, with zero meaning “not at all helpful” and ten meaning “extremely helpful,” how would you rate the GE onboarding program? [“perception” for which answer may be skewed by recency effects, grading pattern, vested interest in outcome, etc.] – A physician may rate a product, rep and/or message to help a rep, hurt a rep, help a product, help a company, help a colleague, etc.
  6. 6. A good project:• Elicits understanding of what our target population thinks it wants, needs, understands, questions and fears – Which is not the same as reality• Respects the respondent’s time and viewpoints – By not being redundant, pedantic, emotional or too long (90% of quantitative studies should take a half hr or less; a TDI or central location interview likewise; a focus or telegroup 1-2 hrs) – By giving the respondent at least one unstructured question in which to state what s/he hasn’t been asked that is also important
  7. 7. A good questionnaire:• Permits and never constrains the respondent’s own language and frame of reference• Presents clearly and unequivocally the stimuli to which the respondent is responding• Flows smoothly from topic to topic without the respondent’s caring or noticing• Could not possibly confuse or annoy the respondent• Does not in itself affect attitudes or behaviors, only reflects them• Has been formulated through reference to previous market research, field reports, sales data, clinical literature, peer-to- peer communication and common sense• Allows us to evaluate the market, not the results of stupid questions
  8. 8. Study objectives checklist• What do we know from secondary research, past primary research, clinical experience?• Are our questions exploratory, well-structured or both?• Do we need an interviewer?• Do we need projectability?• How many purely numeric questions do we have?• Do we need to have multi-phased or iterative questioning?• To what sort of stimulus/stimuli if any, do we need a response?
  9. 9. Typical marketing issues• Which order of indications might optimize uptake?• What barriers to formulary acceptance do we face?• What messages are prescribers actually taking from our and/or competitors’ details? Which of these is/are most convincing as a reason to prescribe?• How are efficacy/safety/convenience/”expensive” defined in this category/class/disorder/symptom cluster?• What is market satisfaction with current tx, on what is it based, and what incremental prescribing could a safer/cheaper/more efficacious/more convenient option generate?• How important are formulation and route of administration, e.g. TID dosing, IV setup, need to take with food, injectables, etc.?• Are patient characteristics actually driving prescribing, or are other factors more prominent, e.g. DTC ad-driven requests, a dominant formulary, sample availability, cost, etc.?
  10. 10. Draw the map before starting the car• Before doing a study, know as much as possible about: – Who will use it and how; any differing agendas – When their big moment is…the forecast is due, the presentation is scheduled…the bigwigs are visiting and what their deliverable is (slides, charts, report) – What you are expected to add to the vendor’s analysis, or simply what you can add on your own initiative• Vendors take orders, professionals are partners
  11. 11. What can we present?• Product descriptions or profiles• Laddered “what if’s”: what if in addition to what you just saw, this agent’s reduction of MS lesions was 30% greater than Avonex’s?• Factor, variable or endpoint lists• Drawings, concepts, diagrams, ads, photos• Maps• Posters, banners, billboards, signs• Direct mail packages• Detail aids• Brochures• Clinical reprints or abstracts
  12. 12. How can we present it?• In graphic format• Interviewer-read• Audiotaped music and/or spoken word• Videotape; steaming video• Graphics that are clickable to reveal more detail, provide feedback or record response (e.g. click on the organ system you’d review first)• Text that can be highlighted, deleted, edited• Role-playing: rep/doc, doc/doc, patient/doc, patient/patient• Free association tasks
  13. 13. Limitations of data collection methods • Mail and fax questionnaires limited as to question types, survey logic; cumbersome for respondents to hand write; difficult to verify data or limit honoraria to qualified respondents • Phone surveys rely on “human element,” often require pre- scheduling, leading to stressful context • Central location or in-office interviews create geographic and sample size limitations, leading to social biases, inconvenience for respondents, travel time/costs for interviewer – However, still the only option if absolute control over materials is needed • Using the Internet, depending on the country and audience, may skew the sample toward early adopters
  14. 14. What issues are most important? Issues Best choice(s) Worst choice(s) Data quality Only CATI and Web impose Phone (human computer-aided quality control, but intermediary) is worst; CATI requires a human: “Can you mail and fax are self- change your answers to total 100%?” administered w/o checks Break-offs or Phone most reliable due to politeness Mail and fax risk incomplete interviews factor, but Web close behind (Cozint breakoffs; papers get studies have <4% break-off rate) lost, discarded Open ends Web captures respondents’ Phone and face to face (unstructured comments as they enter them, mail impose interviewers’ comments) and fax allow similar freedom, but limitations and biases incur fatigue, need to decipher Relaxed environment Nothing beats 24/7 access and Phone and face-to-face choice of location! Web, hands down the most burdensome due to pre-scheduling Social biases Web, mail and fax avoid concerns Phone and face-to-face about saying “the right thing” both risk trying to please or oppose an interviewer
  15. 15. What issues are most important? (cont’d) Issue Best choice(s) Worst choice(s) Respondent’s ability Web, mail and fax are all self-paced; Phone and face-to-face to think fully and however, only the Web does not rely are high pressure, move communicate honestly on maintaining and returning paper along contexts Intrusiveness/bias of Designed properly, Web surveys Mail and fax, since the method itself combine best of both worlds: “point respondent must think and click” ease with self- and write within administration control constraints of form Follow-up with Select findings can be made Most other methods respondent available to respondents immediately rarely offer post-study via e-mail message with URL interaction Compliance Very high with Web; can be high with Declining with phone mail given high honorarium Honorarium fulfillment The shorter the field period, the Phone is worst, followed quicker the checks are mailed: Web by mail
  16. 16. Besides Web surveys, other Net-based options exist• Can e-mail or serve a Word or Excel document (no branching logic, data export may be more difficult), HTML document (still must return to sender), downloadable apps (more expensive; can trigger security, virus concerns)• For group-think without the face-to-face geographic compromises and expense, consider iterative threaded chat• Real-time focus groups can suffer from typing, reading, computer speed biases/frustrations; give the platform a real workout first, and consider your audience• Traditional focus groups can be viewed via the Web; moderators can incorporate your comments in real time via online chat• Posting to a sufferer, caregiver or professional Usenet group, threaded forum or chat room, if its charter permits, can garner interesting feedback; you can also provide a mailto link on your or another site – Remember, however, some AEs must be reported
  17. 17. Whom and how do we sample? • How selective or inclusive need we be in sampling? – Are we interested in all neurologists, Board-certified neuros, only neuros who treat MS or only neuros for whom MS is a large part of their patient load? • Do we need statistical validity (projectability)? – If so, from whom to whom? • Is there any reason to show multiple ads, profiles, designs, etc.? – If so, do we show everything to everyone or is there a reason to have multiple arms (e.g. to test order effects or the impact of better data)? • Basic rules of thumb: N of 30 is minimal, 50 is standard, 60 is better and 75 is essential for multivariate tools, e.g. conjoint
  18. 18. Sampling frame: the first driver of actionability• Does the firm “prepay” respondents to serve on a panel or pay “by the study?” – Panels risk unrepresentative “professional respondents”• Does the vendor recruit by e-mail only if respondents have already consented to e-mail contact? – Are previous respondents favored in sampling? – If an e-mail panel is not fully representative, does the vendor have fax access or other means of offline recruitment (if not using a Web site)? – For patients, is the screening based on specific questions about symptoms, to reconfirm dx, or is it based on a single self-reported diagnosis?
  19. 19. Sampling frame: you can contribute• If at all possible, provide a list from a third party database, stratified by the variable(s) you want proportionally represented, e.g. decile, share of a product, etc. – Make sure user of list has certified that the list remains your property and will not be merged into any permanent database – Do not limit yourself to “heavy prescribers” of something unless there is good reason to do so• If using a phone center’s database to interview or recruit for F2F, to how large a group does the firm have access? – Routinely, “friendlies” are called first unless you specify otherwise – Unless your vendor has her own phone center, one phone center can be doing competing projects; make sure that no survey materials have our name or the name of our products (another reason to blind all screening questions and all materials!) – For central location, this means not only experienced group members but group members who may know each other
  20. 20. Writing the RFP• Specify background and objectives, audience(s) and methods, desired insights, deadlines for proposal and deliverables, deliverables (e.g. topline, report, presentation, data files) – What hypotheses are you testing? Are you looking for confirmation or new information?• Specify report/presentation to be prepared by primary interviewer/moderator, if applicable• Do not do everything for the vendor. Leave room for the vendor to demonstrate his/her own knowledge of the disease state or symptoms, therapies and treatments used, audience(s) to be surveyed, most appropriate means for doing so and questions to be asked• Give vendors a week to respond if at all possible• Provide contact info and make it clear you are open to questions• Encourage vendors to improve upon your RFP, not to copy it
  21. 21. Vendor selection checklist• Is the experience and credentials of primary project personnel clear? – Ask for references if need be• Is the universe of potential participants what the objectives demand? Is it representative of the overall universe?• Are specified honoraria adequate but not excessive? You have the right to ask that they be changed, but that is a red flag• Does the proposal demonstrate knowledge of the objectives and disease state beyond the RFP?• Is the vendor clear about the probability of hitting your deadlines, given the audience and honoraria proposed?• Do you hear phrases like “we’ll see…we might have to increase the honoraria…extend the timeline…lower our sights…change the design? – Is there evidence that the vendor has done something like this before?
  22. 22. Managing the vendor: GIGO• Follow the Golden Rule: return calls and e-mail promptly and honestly; consider a kickoff meeting or call with the vendor and key internal customers to ensure all are on the same page• Do not release information that the vendor does not need to know, but obtain a signed confidentiality agreement• Read the deliverables, and give yourself time to respond fully; don’t rationalize that the vendor will do it right, or you may be disappointed• Work with the vendor in partnership to obtain the desired result; provide the vendor with feedback during and after the project• Do not accept errors in spelling, grammar or logic in any deliverables, and do not sign off on deliverables until you receive a corrected version• Monitor interviews. Ask for tapes of all qualitative work. Listen to the tapes. Listen in to pretests and vendor debriefings. If necessary, listen in to vendor briefings of subcontractors, e.g. phone centers. Ask for copies of interviewer’s guides, pronunciation guides, all field materials.
  23. 23. Managing the vendor: be proactive• Ask for daily dispos while quant work is being fielded• Ask for final incidence and compliance rates• As field proceeds, if incidence is not what was budgeted, we must pay more or relax criteria – This should not happen if criteria were appropriate in the first place• If response rate is low, check the invitation – Is it enticing? Are the topic, length and honorarium clear? – Are you fielding in the middle of a pertinent national meeting? – If phone recruiters are working, ask to monitor their calls• Always get a final respondent list to ascertain potential friendlies, geographic bias (sign a CASRO form first)
  24. 24. Ethics are essential• Although it may be “necessary” to increase honoraria mid-study, it should not be – Don’t let a vendor play the “inch up” game – It’s unfair and actually unethical if you know it’s going to happen – It also skews your data!• CASRO: You can’t target or market or otherwise treat differently a respondent on the basis of his/her answers – You can, however, model (e.g. compare IMS data with self- reported share of preference) – And even compare call reports and other field data
  25. 25. More essential ethics• Don’t talk to one vendor about another, even to tell her from whom you’ve asked for proposals; if you have to mention a price, never disclose from whom• Don’t distribute respondent names, even under CASRO guidelines, to those without a “need to know”• Don’t share contact info for past respondents with any vendor; if the names came from their db originally, they have already them – If they don’t, they may have come from another vendor’s db and are not your property• Don’t share a previous questionnaire or results prepared by vendor A with vendor B without: – Removing vendor A’s name throughout and not sharing it – Including a transmittal memo that stresses it is not to be shared with other employees at vendor B and is to be used only for research purposes
  26. 26. What do you have the right to know?• To whom the vendor subcontracts any aspect of your study, including methodology, phone work, interviewing, data entry, moderating, logistics, analysis, Web survey programming or hosting• The software application(s) and procedures that will be used to analyze data, esp. if a conjoint, DCM or segmentation is involved• The honoraria to be paid, and any changes throughout the study, even if you aren’t asked to pay for them• What the invitation to participate or the script for the recruitment call looks like• The sources of the database(s) from which respondents are being recruited, and the sampling algorithm that is being used
  27. 27. Screening questions: what do you really need?• NEVER telegraph screening criteria like the following: – Are you Board certified in egg drop soup preparation? – Do you treat at least 10 hyperlipidemic patients per month? – How many MS patients do you treat per month: • 0-10 • 11 or more• All questions leading into the main questionnaire should either be terminate points or drive the version of the survey that respondents see—no exceptions! – With rare exceptions, years in practice, age, participation in clinical trials and/or total patient volume should not be screening criteria
  28. 28. Screening questions: the minimum is the max• Re-verify specialty focus or Board certification – Patient volume (total if rare or frequently seen disease, e.g. stage IV breast cancer; per week or per month) for relevant disorder – May need to verify rx of specific drug(s) • Present a clear patient frame and time frame. Always ask about either current regimens (percentage of patients with X on drugs X-Z) or scripts over a time period, e.g. “in the last month, what percentage of your statin prescriptions for newly-diagnosed hyperlipidemia patients were for each of the following?”
  29. 29. Honoraria: potential Achilles heel of projectability• Differences in honoraria among vendors are often large – Are honoraria adequate? Cash or gifts or points? – Major third party data sources pay as little as $100 per year for long, immensely detailed forms – You get what you pay for; lowball potential respondents and response rates may be “OK” but yield skewed results because you interviewed the curious, the bored, the compulsive…instead of the mainstream• “Ballpark” honoraria for a 20-minute Web or phone study should be $>80 for PCPs, >$100 for specialists – Consider topic, complexity, time frame, audience, competing demands and interest level – For convention work, organizers may restrict the monetary value of the gift for on-site work; off-site groups are reimbursed as normal
  30. 30. Recruitment begins your corporate exposure• As the client, whatever the method, understand and approve the vendor’s database source(s), algorithm for sampling, method(s) for inviting persons in that database, honoraria and the means of screening potential respondents – Your firm is the (legal) catalyst for recruitment• Online recruitment options: popup windows, banner ads, sidebars, body text, on your site and/or others’ – With phone or fax follow-ups for small universes• When using third party target lists with e-mail addresses, must have confirmed evidence of opt-in, or reconfirm/obtain it off line – Illegitimate research risks restricted access to panelists, disenrollment, low response rates, formal complaints, etc.
  31. 31. Respecting respondents: make sure it happens• Invitations to participate in surveys should contain: – Survey topic – Length of survey – Honorarium – URL, user ID and password (if applicable) – Toll-free number if respondent needs help• Once respondent is doing survey, s/he should have access to – Live chat support function (Web) – 800 number – E-mail help• At completion of survey, respondent should be reminded of honorarium time frame and given contact information should s/he have questions, need to change mailing address, etc.
  32. 32. What makes a good question?• Only one adjective or adverb• Positive phrasing• Active verbs• Human-speak, not ad, computer or marketing-speak• A scale that is based on the question stem• Brief, precise, concise, unequivocal wording• Time reference: over the last year, month, week, since residency or fellowship…• Object reference: you, your prescriptions, your patients’ prescriptions, your patients, family members of your patients…• Patient reference – Patients diagnosed with…treated for…seen for…evaluated for…hospitalized for…seen in the office…without insurance coverage for prescriptions…
  33. 33. What makes a good scale or set of answers?• 0-10, where 0=“not at all,” “never”… and “10” = “always,” “completely,” “completely effective”• Disagree…somewhat agree…completely agree• Much lower/somewhat lower/the same/somewhat higher/much higher…much more likely• Not at all likely/somewhat likely/very likely• Clear distinction between scale point labels if used: never “strongly” vs. “very” or “extremely” vs. “very”• Do not mix answer items or attributes that are very different structurally, e.g. “is dosed once weekly” vs. “efficacious in green dot disease”• Branch questions to avoid long lists, e.g. first ask about percentage of pts on mono vs. combo tx, then drill down separately into each
  34. 34. What can we measure?• Likelihood of consideration, trial, purchase, further information-seeking• Frequency, interval, duration, strength of need• Means and urgency of information seeking• Interest in various factors, decisions, categories, purchasing• Media usage and role of word of mouth in purchase process• What really closed or lost the sale: not necessarily the most recent or longest exposure; may have been something that didn’t happen• How information is used (did I window shop till I dropped, then pay the first clerk I saw when my feet started to hurt? I listen to my friends’ political views, but do I vote on them?)• Key drivers of perceived value (how price and quality interact), quality, price competitiveness and other desirable attributes for the product class (e.g. which lipid values are looked at in which patients to determine a statin’s efficacy?)
  35. 35. More we can measure• How many X in Y time period (total or frequency)• Total hours or minutes per time period or typical duration• Stage or status of need (Are tires bald? Or are they old so assumed bald? How will the driver judge when it is time to replace them other than if one blows out?)• Yes/no: would you consider/have you heard of/do you own/have/rent… have you ever• If you could improve/change one thing…• To probe a closed end, try “why do you say that?” or a variant first• Have you discussed X with Y? What did you decide?• Aside from X, how do you stay up to date on blue hair disease?• What kind of information about X would be most useful (yes/no on specific types sought, facts, data, pricing, options, support)
  36. 36. Typical survey question types • Check all that apply/ever • Best/worst/most/least (applicable/likely, etc.) • Numeric/percentages/proportions of patient types, therapies by subtype or disorder, allocation of time, etc. • Rankings, ratings or point allocation tasks • Forced totals and choices (where appropriate; don’t precode away your ability to understand the market) • Other specifies, which can be piped into other questions • “Three most potent…” or pseudo-interval scale where “best” and “worst” agents are first identified and others placed between • Disadvantage to advantage scale • Trends and reasons • Benchmarking: “at what reduction in mortality would you begin to consider prescribing a therapy for…” • Customized, open ended probes: “You say that Merck is the most managed care friendly pharmaceutical firm. Why is that?”
  37. 37. Meaningless questions: ever so tempting• What is the first anti-diabetes medication that comes to mind?• Of what other anti-diabetes medications are you aware?• Have you ever heard of Rebif?• For what disorder(s) is Zyrtec indicated?• Without looking at your sample closet, do you think you have enough Nasacort samples? [setting-dependent]• What were your net practice revenues last year?• How do nurse practitioners feel about Claritin-D?• What company uses the most celebrities in its advertising?• What was the last TV commercial directed toward patients that you saw? [painful recall; ask about most recent positive or negative and allow none and no recall]
  38. 38. Questionnaire format starts with the nits• Start blinded product label at “L” or similarly innocuous point (avoid using letter that any major comparators begin with, e.g. do not use “L” in a statin study) and never number items• It’s “percentage,” not “percent of”• Scales are “from 0 to 10,” not “an 11-point scale” and avoid pull- down boxes for rating scales• Keep the answers away from the question itself, except in phone• Ask about percentages, not numbers, except early on• Reassure as to validity of “best estimate,” or “as far as you are aware, “approximately,” never “best guess”• Use “most,” “some” or “other(s)” rather than “you” for sensitive issues
  39. 39. Don’t let format stand in the way of the answers• Using decades for age is passé, irritating and waves the question like a red flag (use 25-34 ranges, or, better yet, just ask for the age!)• Using tens as income ranges does all the same things – Do you really need to know ethnicity, education, income, gender and/or age? Can these be pulled from another file?• Numbering answer list items for no reason, e.g. regimens or products, can bias or simply confuse• Not specifying forced totals or ranges in a question, then displaying an error message, leads to rushed corrections and poor data• Unnecessarily characterizing open ends (“in a few sentences…”) reads like “what I, your first grade teacher, want you to do”• Ordering answer lists by perceived popularity or appropriateness, instead of logically or alphabetically, makes it harder for the respondent to find things, and less likely to care enough to do so
  40. 40. Asking sensitive questions• Present in terms of requests: “Please stop me when I read the range that best describes your household income before taxes.”• Introduce demographics: “These final questions will help us group your answers with others.”• Don’t assume the worst or the best. “How many times did you fall asleep during work today?” is wrong. “During work today, did you sleep at all?” is right. Then if yes, the respondent is asked the quantity question.• Let the respondent see that your specificity is for her benefit, not yours• Don’t ask a question to which the respondent knows the “socially correct” answer; it’s an insult and an invitation to mislead: – Do you usually feed your children nutritious food? – How important is it to vote for the best-qualified candidate?
  41. 41. Know thy frame• The last three…the next two…over the last year… most recent…oldest…most often…• Patients who have been prescribed an NSAID in the last year… osteoarthritis patients not on rx• Three most…least important…most often seen…three statins you associate most often with reduction of LDL in diabetics…the most difficult case of RA you have treated…the last patient of whose outcome you are uncertain…most important reason to stock Mavik samples• NOT: ten reasons to write Mavik, most important reason to write albuterol with a nasal steroid, sources of information about health issues
  42. 42. Compare answers for robust models• Paired comparisons, A vs. B• Rankings• Allocate 10 or 100 points• Not at all/somewhat/very important• Regress potential drivers against preference share or current rx• Ask in terms of a percentage: “what percentage of your statin scripts are changed due to a formulary restriction?”• To what sort of stimulus/stimuli if any, do we need a response?• Always make sure each factor or attribute only reflects one thing: what is wrong with measuring agreement with “I would never buy an Acura, because they are too expensive”
  43. 43. The pitfalls of double-barreled statements• “I would never buy an Acura, because they are too expensive” means what? – Acuras are too expensive? – I would never buy an Acura, period? – I will buy an Acura while still complaining about the price? – I won’t buy an Acura but not because of cost? – I couldn’t afford any Honda--$3K for a used car is my limit? – I take the train? – I have no idea what an Acura is? – I don’t drive?
  44. 44. Care in crafting attribute lists pays off• Attributes should be associated either with the respondent or with something else, e.g. a product• Factors should be controlled by the respondent or not (a patient controls compliance, but not side effects)• List items should be unrelated to each other causally, e.g. we would not ask respondents to rank order factors including “side effects” in a list along with “incidence of tongue mangling”• If any of the above rules is violated, the resultant data are meaningless• Use positive statements, not “trick” double negatives• Use parallel structure, but mix things up a little (don’t present 5 agreement statements in a row that start with “pancreatic cancer.”
  45. 45. The “chase” question• Sometimes a series of branching questions is unnecessary and a waste of time. Ask a direct question in a neutral, non- threatening way. – Have you ever obtained health care information using the Internet? – In the last year, have you used the magazine Good Housekeeping to help you purchase makeup?• Do not cluster chase questions. Ask them sparingly and space them out, except for demographics:• “Are you: Female/Male”• “Is your ethnic background best described as:”
  46. 46. How to insult, patronize and misuse respondents• Phone-speak on screen: “Next we’ll talk about,” “I’d like to now ask,” “Doctor, now please…”• Huge intimidating product grids measuring awareness, usage and/or ratings only prompt trigger-clicking (mindless mouse clicks to get on with the survey) or eyes-closed ratings• Ad-speak: “concept description,” “white space,” “body copy,” “product description,” “story board”• Insults to respondents’ intelligence: “record below,” “type here,” and, worst of all, “click here”; similarly, do not specify “choose one only” if a likelihood or yes/no question• Asking for other specifies which will not be analyzed—breaks flow, wastes precious time and respondents often sense that their effort is futile
  47. 47. More insults and abuse: don’t let it happen• Over-outlining, setting the stage, introducing…• “Time remaining” thermometers or comments: misleading, distracting, scream the context, invite breakoffs, guesses of remaining time, fitting answers into it – Make sure vendor is accurate about length in the invitation and intro!• Listing potential answers in both the question and the answer blanks (the more they have to read, the less time for completion)• Placing demographic questions that are not terminate points or branching criteria at the beginning, thus risking qualified breakoffs• Failure to use automation, e.g. forcing respondent to restate choices: “you may have seen/heard these before” (Why?)
  48. 48. Don’t “lead the witness”• Preceding a specific question with a general intro tempts respondents to skip and misinterpret the real question• Listing scale points or choices from “top down,” e.g. starting with “10” or “completely successful” or “increase” biases responses ; the only exception to this rule is yes/no• Placing performance ratings on the same page/grid as attribute importance ratings increases the likelihood they’ll converge• Using abstract-speak: “Was the level of samples satisfactory?” “On the following scale, how motivating was the message?” “Was the ad believable?” guarantees less than thoughtful answers to these false dichotomies and oversimplified questions• Giving the show away: “Are you aware that not all asthma drugs are equally safe for children?” results in one of four responses: (yes, I actually know; yes, because if I say no, I’m an idiot; no, I’m clueless; no, I’m feeling oppositional)
  49. 49. Open ends: robust feedback for marketing• Don’t allow an open end that challenges professional judgment: “Why do you not prescribe Clarinex for urticaria?” – that looks more negative on screen or in print. Instead, consider: – What are the major reasons for not prescribing Clarinex for urticaria? – You do not currently prescribe Clarinex for urticaria. Why is that? – Under what circumstances might you begin prescribing Allegra for urticaria? – In what time frame, if any, would you anticipate prescribing Clarinex for urticaria? (where a choice is “not in foreseeable future”)• Don’t ask the vendor to fix verbatims’ spelling, grammar or anything else; get the open ends in Excel with the other data even if you get an SPSS file, too – Use some pithy open ends in your summary, but read them all
  50. 50. What makes a good qualitative interviewer ormoderator?• Listen for non-linear probing that follows the respondent, not the discussion guide: – Because… – Under what circumstances would that happen? – Can you give me an example? – Does that happen often?• The interviewer should not “beat a dead horse”– if the respondent has already made a blanket statement that section III will not be applicable, it shouldn’t be asked – This requires that the interviewer be listening, not just talking• With a hesitant or skeptical respondent, the interviewer reinforces the dialogue: – It’s whatever you think is important – Don’t let me put words in your mouth – We want to know how you perceive Begonex
  51. 51. Qualitative interviewing: warning signs• Monitor interviews and debrief the interviewer after each one – What was interesting to you? What wasn’t? What remained unasked and unsaid? – Modify the discussion guide if questions aren’t productive, everyone says the same thing, or the issues don’t apply to your sample – Ask the interviewer for her ideas as well• Be concerned if: – The interviewer adheres word for word to the discussion guide – You hear multiple probes that are expressed in terms of the interviewer’s context or needs: “Yes, but I need a number for that… that’s not what I said…you’re the first one who’s said that…that’s not what I’m looking for” – You want to follow up interesting comments, but the interviewer charges ahead with the next question
  52. 52. Leverage qualitative feedback• Ask some numeric questions, preferably as warmups – But if you need mostly numbers, you need a quant study• Don’t assume that the respondent differentiates drugs on efficacy, side effects or cost. Find out what her rx drivers are. In other words, don’t put the cart before the horse. – Deconstruct: Do you differentiate the statins on the basis of side effects? If so, how? If not, are there any important differences between them? If so, what are they? If not, on what basis do you decide which statin to prescribe for a given patient?• Listen to the interviews. Even from behind a mirror, it’s not entertainment; it’s vital business information, the lack of which could lose your next blockbuster!
  53. 53. Researchers’ value: best practices and insight• Help marketing understand how: – Benefits, costs and value of products and services are determined by target audiences, and how that’s changing – New technology, science and products influence expectations – Your corporate image and brands’ equity can gain and hold value via appropriate marketing• In survey research and Web development: – Follow the highest standards – Work with only vendors who do the same – Look for ways to enhance and convey standards’ benefits