Precision Performances. Uncompromising Results.


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Precision Performances. Uncompromising Results.

  1. 2. B2B Internet Research: the Key to Success Roland Klassen [email_address] Tim Sinke [email_address]
  2. 3. Who are we and what’s our bias?
  3. 4. Acrobat Research <ul><li>CATI and Web – 3 offices – Toronto, Sudbury, Canso, NS </li></ul><ul><li>Web since 1997 </li></ul><ul><li>Majority of Business is CATI </li></ul><ul><li>Background is B2B, traditional focus in IT </li></ul><ul><li>Approach has been to attempt to take an impartial view </li></ul>Questions to Ask
  4. 5. Questions to Ask
  5. 6. Population to be Sampled <ul><li>Identifiable? </li></ul><ul><li>Online? </li></ul><ul><li>Reachable on phone? </li></ul><ul><li>Over-surveyed? </li></ul><ul><li>Already exists in a panel? </li></ul>Questions to Ask
  6. 7. Type of Info to be Collected? <ul><li>Complexity? </li></ul><ul><li>Length? </li></ul><ul><li>Stimuli needed? </li></ul><ul><li>Mixed-mode? </li></ul>Questions to Ask
  7. 8. Geographical coverage? Questions to Ask
  8. 9. Does it need to be Projectable? <ul><li>Limitation based on company size/industry? </li></ul><ul><li>Does it have to be matched to D&B SICs/NAICs/etc? </li></ul>Questions to Ask
  9. 10. Other Questions <ul><li>Response Rate </li></ul><ul><li>Time in field </li></ul><ul><li>Incentives </li></ul><ul><li>Budget Expectations </li></ul><ul><ul><li>Faster and cheaper, right? </li></ul></ul><ul><ul><ul><li>Maybe </li></ul></ul></ul>Questions to Ask
  10. 11. Other Questions... <ul><li>CAN the survey be self-administered? </li></ul><ul><li>SHOULD the survey be self-administered? </li></ul>Questions to Ask
  11. 12. Sample Sources
  12. 13. Internal Lists <ul><li>Customers </li></ul><ul><li>Employees </li></ul><ul><li>Prospects </li></ul>Sample Sources
  13. 14. Drop-in/Intercepts <ul><li>Web-site specific </li></ul><ul><li>For general web sample – dropping – click-through rates are dropping off to nothing </li></ul>Sample Sources
  14. 15. Online Panels <ul><li>Main general panels in NA are SSI, Greenfield, e-Rewards, Harris Interactive </li></ul><ul><ul><li>Get specifics on their panel build and profile processes </li></ul></ul><ul><li>Specialty panels </li></ul>Sample Sources
  15. 16. A word on major online panels <ul><li>Educate yourself </li></ul><ul><li>Review your screening criteria </li></ul><ul><li>Are the numbers available? </li></ul><ul><li>Geographies </li></ul>Sample Sources
  16. 17. Who are you talking to? <ul><li>Went to – typed “online market research pros and cons” got these… </li></ul>Sample Sources
  17. 18. Sample Sources
  18. 19. Sample Sources
  19. 20. A Quick Review of one Player <ul><li>Used SSI as an example </li></ul><ul><ul><li>Not meant to support or criticize </li></ul></ul><ul><li>Following are excerpts from their website: </li></ul>Sample Sources
  20. 21. What it says: <ul><li>SurveySpot panelists are recruited from many sources. These include banner ads on the Web and other Internet methods, and by RDD telephone. All SurveySpot members are recruited exclusively by permission-based techniques. SSI does not use unsolicited e-mail (or spam) in building the panel. </li></ul><ul><li>SurveySpot panel demographics are not based on predictive techniques. They are created from self-reported, respondent-specific information . Researchers are assured that they are reaching the exact target market they seek meaning more cost-effective online research [EMPHASIS ADDED] </li></ul>Sample Sources
  21. 22. Industry Select Offered: <ul><li>Industry </li></ul><ul><ul><li>Agriculture SIC 01-09 </li></ul></ul><ul><ul><li>Mining SIC 10-14 </li></ul></ul><ul><ul><li>Contracting SIC 15-17 </li></ul></ul><ul><ul><li>Manufacturing SIC 20-39 </li></ul></ul><ul><ul><li>Utilities SIC 40-49 </li></ul></ul><ul><ul><li>Wholesale SIC 50-51 </li></ul></ul>Sample Sources <ul><ul><li>Retail SIC 52-59 </li></ul></ul><ul><ul><li>Financial SIC 60-67 </li></ul></ul><ul><ul><li>Services SIC 70-79 </li></ul></ul><ul><ul><li>Professionals SIC 80-89 </li></ul></ul><ul><ul><li>Government SIC 91-97 </li></ul></ul>
  22. 23. What are the issues with this? <ul><li>Respondents don’t know their SIC Code </li></ul><ul><li>How are SIC Codes mapped and assigned? Primary/secondary? To what level of detail? </li></ul><ul><li>Does it matter to you? </li></ul><ul><li>Do they match back to D&B if you’re extrapolating to the industry? </li></ul>Sample Sources
  23. 24. Recruit-to-Web
  24. 25. Background <ul><li>What is R2W? </li></ul><ul><ul><li>CATI recruit – web completion </li></ul></ul><ul><ul><li>User ID/Password protected </li></ul></ul><ul><ul><li>Appropriate follow ups – based on response/completion stage </li></ul></ul><ul><li>Why use R2W? </li></ul><ul><ul><li>Can’t reach everyone through a panel </li></ul></ul><ul><ul><li>You want to be sure of the respondent </li></ul></ul>Recruit-to-Web
  25. 26. Examples <ul><li>Most senior C-Level IT person in Global 500 corporations – US and Europe </li></ul><ul><li>Office equipment DMs mailed physical samples to take a follow-up survey </li></ul><ul><li>Telecom decision makers recruited in cross-section of businesses in specific geographies </li></ul><ul><li>Results – in Survey Stats segment </li></ul>Recruit-to-Web
  26. 27. General Process <ul><li>(CATI) Recruit </li></ul><ul><ul><li>Custom email – immediate or batch </li></ul></ul><ul><ul><li>Follow up – 48-72 hours </li></ul></ul><ul><ul><li>Follow up – 1 week </li></ul></ul><ul><ul><li>Optional after that </li></ul></ul>Recruit-to-Web
  27. 28. Applications <ul><li>Can be effective; however </li></ul><ul><li>Use to </li></ul><ul><ul><li>Support definite online requirement – can’t be done on the phone </li></ul></ul><ul><ul><li>Refusal conversion and/or multi-modal survey </li></ul></ul><ul><li>If “standard” tool – could become more expensive than CATI </li></ul><ul><li>More rigorous in defining and controlling the respondent </li></ul>Recruit-to-Web
  28. 29. Survey Stats
  29. 30. Customer/Internal Lists <ul><li>Response rates – 5 to 75% </li></ul><ul><li>Lower end tends to be lower than low end of CATI CSAT </li></ul><ul><li>What is less intrusive – email (suspected SPAM) or CATI (suspected telemarketing)? </li></ul><ul><ul><li>Subject line/intros/”warm-ups” are key </li></ul></ul>Survey Stats
  30. 31. Web Panel <ul><li>Cost on CPI – response rate “isn’t your problem” – however, feasibility is </li></ul><ul><li>Be careful on over-selling </li></ul>Survey Stats
  31. 32. Factors Influencing Response Rates <ul><li>Survey Length </li></ul><ul><li>Respondent Level </li></ul><ul><li>Incentive </li></ul><ul><li>Reminders </li></ul><ul><li>Company Size </li></ul><ul><li>External stimuli/downloads required – obstacles </li></ul><ul><li>Geography </li></ul>Survey Stats
  32. 33. Examples of Survey Response <ul><li>Historical </li></ul><ul><ul><li>Used to target/achieve 50-60% conversion on R2W studies </li></ul></ul><ul><ul><li>Declining – now aggressive target is 35-50% </li></ul></ul>Survey Stats
  33. 34. C-Level in Global 500 Corporations <ul><li>R2W – High Incidence; most senior contact; 40 minutes, $300 US incentive. Conversion – US – 42.8%; Euro 16.9% </li></ul>Survey Stats
  34. 35. International Office Equipment Study <ul><li>R2W – High Incidence; 30 minutes, incentive $50 US; 40 Euros (had to scale up to 100 Euros). Conversion – US – 58.2%; Euro 20.9% </li></ul>Survey Stats
  35. 36. Production Environment Study <ul><li>Web Panel - Europe </li></ul>Survey Stats
  36. 37. Telecom DM (internet) <ul><li>R2W – Small bus Telecom DMs; 51 minutes $35 incentive. Conversion 35.5% </li></ul>Survey Stats
  37. 38. A Different View <ul><li>From e-Rewards. </li></ul><ul><li>Demographic Profile Variable analysis of response rates, based on: </li></ul><ul><ul><li>Tot emails sent: 5.7 m </li></ul></ul><ul><ul><li>Tot surv resp: 1.3 m </li></ul></ul><ul><ul><li>(NOTE: Not sent on B2B studies specifically) </li></ul></ul>Survey Stats 30.3% Homemaker 30.2% Teacher 27.9% Doctor 27.0% Unemployed 26.4% Clerical 25.7% Artistic/Literary 25.0% Other 24.8% Student 24.5% Retired 24.3% Prof/Tech 23.2% Mid Mgt 23.0% MEAN 23.0% Farming/Primary 21.8% Crafts/Constr 21.6% Bus Owner/Self Empl 20.2% Military 18.8% Sales 17.9% Exec/Upper Mgt Resp Rate Occupation
  38. 39. Quality of Data <ul><li>Lower response rates – higher bias potential? </li></ul><ul><li>Who are the non-responders ? </li></ul><ul><li>Equally important – who are the responders ? </li></ul><ul><ul><li>Who’s answering – quality/validation checks? </li></ul></ul><ul><li>Web response rates declining, but not yet worse than telephone or mail </li></ul>Survey Stats
  39. 40. Incentives
  40. 41. What’s required – how much and how? <ul><li>Panels – costs included in CPI – may change for B2B </li></ul><ul><li>Pressure is increasing </li></ul>Incentives
  41. 42. Amounts to Pay <ul><li>Low-level - $1 per minute </li></ul><ul><li>Mid-level - $1.50-$2.00 per minute </li></ul><ul><li>High-level - $35+ (short); $40+ (mid); $50+ (long) – add $2.00 per minute above 20 minutes in length </li></ul>Incentives
  42. 43. How to Pay <ul><li>Cash (cheque) </li></ul><ul><li>E-Certificate (simpler, but expectation of instant delivery) </li></ul><ul><li>Close survey and review data prior to issuing incentives </li></ul><ul><li>Notify respondents of timeframes </li></ul><ul><li>Create a centralized database and 800 number/central email for queries </li></ul>Incentives
  43. 44. KSF – Summary Checklist
  44. 45. Check points for Success (1) <ul><li>Question research objectives </li></ul><ul><li>Multi-modal? </li></ul><ul><li>Pick sample partner carefully </li></ul><ul><li>Put sample partner “through the wringer” </li></ul><ul><li>Analyze data for comparison and validation </li></ul>KSF – Summary Checklist
  45. 46. Check points for Success (2) <ul><li>Are respondents engaged? Devise tests </li></ul><ul><ul><li>Look for repetition </li></ul></ul><ul><ul><li>Average length </li></ul></ul><ul><ul><li>Volume of DK/RF </li></ul></ul><ul><li>Work in partnership with data collection partner </li></ul><ul><li>Be ready to adjust on the fly </li></ul><ul><li>Always, always, always pilot qualitatively </li></ul><ul><li>Remember, you’re only as good as your sample </li></ul>KSF – Summary Checklist
  46. 47. Thank you. Questions?