• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Mobile surveys - The 5th methodology: a new paradigm or a strategic enhancement?
 

Mobile surveys - The 5th methodology: a new paradigm or a strategic enhancement?

on

  • 1,591 views

Mobile surveys - The 5th methodology: a new paradigm or a strategic enhancement?...

Mobile surveys - The 5th methodology: a new paradigm or a strategic enhancement?

Michael Francesco Alioto - Vice President, Marketing Sciences - Gongos Research

Comparing smartphone-based and online survey data: assessing research-on-research done with Best Buy. Developing effective best practices for designing mobile-based quantitative research. Understanding when to use a mobile/online hybrid approach to surveys.

Statistics

Views

Total Views
1,591
Views on SlideShare
1,591
Embed Views
0

Actions

Likes
1
Downloads
38
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Mobile surveys - The 5th methodology: a new paradigm or a strategic enhancement? Mobile surveys - The 5th methodology: a new paradigm or a strategic enhancement? Presentation Transcript

    • Mobile Surveys | The 5 th Methodology: New Paradigm or Strategic Enhancement ? Market Research in the Mobile World 19-20 July 2011: Atlanta Michael Francesco Alioto Vice President, Marketing Sciences
    • Surveying consumers in their natural environment What Role Will Mobile Play in the Future ?Critical Study Objectives:• Understand/confirm current mobile survey usability— integrative hybrid or next survey platform?• Confirm consumers’ usage of mobile devices in their daily lives—communication and environment• Confirm if surveys can be completed within an integrated platform with similar results• Compare and contrast smartphone and online surveys— scale usage, results, incidence rates, survey length, etc.• Assess potential to analyze data utilizing more advanced analytical techniques (e.g., MaxDiff) on mobile devices
    • Path to understanding research design A Four-Phase ProcessApproaches Toward Analysis: PHASE ONE: PHASE TWO: PHASE THREE: PHASE FOUR: Desk Usage & Survey Survey Advanced Research Lifestyle Design Results Statistical Comparison Comparison Analysis Review literature, Confirm Flash Polls & Smartphone survey conventional Confirm data comparability of vs. online version thoughts, and Dialogues comparability and more complex secondary data; n=1,006 n=496 scope across quantitative establish baseline (495 smartphone/ platforms techniques 511 online) In partnership with
    • PHASE ONE: Current understanding and benchmarking Desk Research: Key Findings Macro Trends Micro Trends • Two schools of thought: strategic enhancement— • Data comparability between online and mobile online supplement vs. platform evolution— survey platforms debatable— supplement vs. 5th Methodology independent platform • Tendency toward hybrid platforms and data • Critical limitations concerning mobile research integration, but results and strategy not conclusive identified: e.g. survey length, exhibits, types of questions, scales, incentives • Suggests major limitations for mobile survey component of hybrid platform—device/technology, • Mobile segments illustrate personal differences: research design elements Millennials, Multicultural • Academic community mostly silent on the issue • Online sample providers just beginning to understand/get comfortable with mobile • 5th Methodology as mainstream future platform— surveys (incidences, costing, viability) will be driven by behavior and mobile device technological improvements
    • PHASE TWO: The mobile device lifestyle integration Usage & Lifestyle: Key Flashpoll Findings Smartphone Lifestyle Usage Smartphone Usage/Integration: • Placing phone calls, texting, sending e-mails, and At home 94% Internet searches are in the top five usage categories—ideal for communication At work 70% • Supports desk research concerning smartphones as “connectivity to the world” In the car 62% • Smartphone usage is well integrated into daily lifestyle (home, work, vehicle) While shopping 61% • Smartphones used extensively while shopping Internet café/ 35% Coffee house • Smartphones used to conduct a wide variety of tasks, many times simultaneously At school 24% • Smartphones used throughout the day/within periods of “constant interruption” Other 19%
    • PHASE THREE: Setting the stage quantitative survey Questionnaire and Design: Key Elements Design elements: Scales types:  Response Rates  Multiple Mention  Incentive Structure  Single Response  Survey Timing  Three-point, fully-anchored  Open-Ended Questions  Five-point, fully-anchored  Respondent Assessment (Worldview)  Five-point, end-anchored  Technology Limitations  Five-point, semantic differential  Six-point, fully-anchored Survey subjects:  Internet Sites Visited, Grids  Store Display, Image Testing Disagree Agree  Mobile Preferences, MaxDiff Completely Completely  Concept Evaluation 1 5  Demographics  Online Survey Assessment  Mobile Survey Assessment  Best Buy Segmentation
    • PHASE THREE: Key Comparisons quantitative survey Design Elements: Key Findings Traditional Research Thoughts Results Significantly lower for No difference between RESPONSE RATES smartphones platforms Higher incentives required INCENTIVES Higher incentives not required for smartphones for smartphones Restricted for smartphones TIMING Not as restricted as thought Difficult for smartphones VERBATIMS Minimal verbatims possible Much different for Different, but thought to be WORLDVIEW smartphone users converging Limited significantly TECHNOLOGY Limited by device
    • PHASE THREE: Comparison of Results quantitative survey Questionnaire Design: Key Findings Image Affinity: 3-Point Scale Respondents were asked how much they liked a store display using a 3-point scale, where: 1 = Dislike it 2 = Feel neutral about it 3 = Like it 25% 40% 41% Like vs. Dislike A A Like it Feel neutral about it 66% Dislike it 49% 49% B,C 9% 11% 10% Online: No Online: Have Smartphone Smartphone Smartphone (C ) (A) (B) n=330 n=174 n=487 Significance testing at the 95% confidence level “Using the scale below, overall, how much do you like this store display?”
    • PHASE THREE: Comparison of Results quantitative survey Questionnaire Design: Key Findings Concept Test: 5-Point Fully Anchored Respondents were asked to rate the concept on a 5-point anchored scale A B,C A 11% A T2B = 39% 24% 24% A A 28% T2B = 64% A T2B = 67% C C 5 Strongly like it 40% 43% A 4 Somewhat like it A B,C 3 Neither like it nor dislike it 47% B,C 2 Somewhat dislike it 31% 24% 1 Strongly dislike it C 7% B B,C 8%B,C 2% A 7% B A A,B 3% 2% Online: No Online: Have Smartphone (C ) Smartphone (A) Smartphone (B) Significance testing at the 95% confidence level n=335 n=176 n=495 “Which of the following statements best describes how much you like or dislike this product/idea?”
    • PHASE THREE: Comparison of Results quantitative survey Questionnaire Design: Key Findings Display Appearance: 5-Point End Anchored Respondents were asked to rate the display table on a 5-point scale, with: 1 = Disagree Completely 5 = Agree Completely 15% 20% 23% 27% 33% 33% Agree Completely 4 31% C 24% 21% 3 2 18% 14% 16% Disagree Completely 8% 8% 7% Online: No Online: Have Smartphone Smartphone Smartphone (C) (A) (B) n=300 n=174 n=487 Significance testing at the 95% confidence level “The table is inviting and I would want to look at the products displayed.”
    • PHASE THREE: Comparison of Results quantitative survey Questionnaire Design: Key Findings Clearly View Image: Respondents were asked to rate the image on Semantic Differential Mean Summary four 5-point differential semantic scales. Online: Have Online: No Smartphone Smartphone Smartphone Sample Size n=312 n=174 n=487 (1) Organized vs. (5) Messy 2.26 2.18 2.30 (1) Appealing vs.(5) Unappealing 2.65* 2.40 2.53 (1) Stands out vs. (5) Blends in 2.72 2.57 2.68 (1) Clear vs. (5) Confusing 2.31 2.28 2.33 * Significantly Higher than 1 other group Significance testing at the 95% confidence level “Below is a list of attributes with differing meanings. For each pair, please select the one attribute that best describes the store display.”
    • PHASE FOUR: Quantitative comparisons statistical analysis MaxDiff Analysis: phone Advanced Analytics: Key Findings feature rankings Respondents completed a MaxDiff exercise where they ranked various mobile phone features, including: brand, price, battery life, size, features (e.g. camera, video) and memory. B,C 4% 6% A 8% A,B 25% 30% A 36% 11% C A,B Memory 10% C Features 19% 8% B,C Size 17% 15% Battery 33% 26% 21% Price B,C C A Brand 8% 12% A 13%A Online, No Online, Have Smartphone (C) Smartphone (A) Smartphone (B) n=495 n=335 n=176 Significance testing at the 95% confidence level “Of the attributes listed below, which one is most important when choosing a mobile phone?”
    • PHASE THREE: Survey comparisons respondent participation Survey Satisfaction: Key Findings Mobile Online 99% 90% 76% 73% 68% 66% 69% 66% Survey Enjoyment Willingness to Participate Willingness to go to Fair Incentive in Future Specified Location/Store to Smartphone/Online Survey Participate Smartphone: n=495 On-Line: n=511 NOTE: Top two boxes on a five-point scale.
    • PHASE THREE: Mobile Survey respondent participation Survey Satisfaction: Key Verbatims LIKES DISLIKES Anywhere at anytime Difficulty typing/Small keyboard • “I dont have to be at home or in front of the computer. I • “Dont like typing on my phone cause the keyboard is too can do this anywhere.” small.” • “Ease of being able to complete the survey from anywhere. • “Excessive typing can be a problem. Id prefer longer, essay Not needing to be tethered to my desktop.” response answers for the computer.” Convenience Pages load slowly/Too long/Drained battery • “I always have my phone on and with me. Not so for the • “Pages load slowly, takes longer than computer.” computer. So, convenience.” • “Too much time, wastes battery life.” • “Its convenient, and I can get them done quickly because I • “Doing anything on the web on my phone takes longer.” always have my phone with me.” Screen too small, survey content too much for screen Easy to do, easy to access • “The screen on my phone is too small to see the questions • “Easy to complete and does not require more effort than a and the answers at the same time.” survey taken on a computer.” • “My blackberry is too small to comfortably take the • “First time taking survey on mobile – it was easy to follow survey.” and easy to get thru, not confusing in the least.” • “The survey was made bigger than my screen and it was Fun, different, new hard to select my choices at times.” • “It was fun and easy to answer the questions. Normally I Too much scrolling take surveys on my laptop, so using a different method • “It takes so long, and I have to scroll left and right, up and such as a mobile phone was interesting.” down to read the question.” • “It was new, interesting, and fun!” Prefer participating from computer • “It’s easier to do most things on my computer.”
    • Critical Findings quantitative survey Design Elements – Scale Types Response rates are similar across platforms. Males trend directionally higher. Smartphone respondents will not require higher or significantly different incentives. There are both technology limitations and different “worldviews” for smartphone users, but this should dissipate in the near future.Smartphone platforms can viably support 5-point fully-anchored scales, but 3-point anchored and 5-point end-anchored scales are recommended. Scale usage does not differ significantly, supporting the use of a hybrid methodology and the integration of mobile and online platforms.Statistical techniques have similar results across survey platforms, supporting the smartphone platform for in-store, real-time data collection.
    • Smartphone platform critical implications Smartphone surveys are a viable option to supplement studies with hard-to-reach segments of the population • Difficulty typing/Small keyboard • “Dont like typing on my phone cause the keyboard is Data and scale usage is comparable across platforms, too small.” supporting the use of hybrid designs • “Excessive typing can be a problem. Id prefer longer, essay response answers for the computer.” • Pages load slowly/Too long/Drained battery • “Pages load slowly, takes longer than computer.” Smartphone users today tend to be more tech-savvy with battery life.” • “Too much time, wastes unique shopping, usage and behavior preferences regardless the platform phone takes longer.” • “Doing anything on of web on my • Screen too small, smaller text, survey content too much for screen • “The screen on my phone is too small to see the questions and the answers at the same time.” Smartphone platform may more readily support specific subgroup segments, i.e. Hispanic, Millennials, “Mywell as global small to comfortably take the • as blackberry is too markets survey.” • “The survey was made bigger than my screen and it was hard to select my choices at times.” • Too much scrollingMobile-enabled surveys are ideal for in-store shopping and exercises,left and right, up • “It takes so long and I have to scroll real-time assessment, and interruptedto read the question.” and down surveying • Prefer participating from computer • “It’s easier and faster to do most things on my computer.”
    • Mobile Research considerations for the futureThe “Strategic Enhancement” of today could well bethe next paradigm—or the 5th methodology—oftomorrow. • How will the marketing research industry prepare for this evolution? • Does it reduce barriers to reaching specialized segments (Millennials, Hispanics, global markets?) • Will this offer greater authenticity to marketing research (rational/emotional surveying). Are smartphone devices more conducive for lifestyle-based interviewing? • Does it support the evolution of the marketing research discipline from the current scientific/data-based structure to a humanistic/decision-making approach?
    • Mobile Surveys | The 5 th Methodology:A New Paradigm or a Strategic Enhancement ? q&a | discussion Michael Francesco Alioto, Ph.D. Vice President Marketing Sciences Gongos Research Greg Heist Vice President Research Innovation Gongos Research
    • Presented at: Market Research in the Mobile World 2nd International Conference | July 19 & 20, 2011 AtlantaOrganized by: Thank you to sponsors:LinkedIn Group: Mobile MRUpcoming Merlien Events: Merlien.org GreenBook Directory: GreenBook.orgMarket Research Blog: GreenBookBlog.org New Qual Blog & Directory: NewQualitative.orgGet Inspired. Stay Informed. Cut Through the Clutter. Sign up for our free email newsletter