• Like
Deliberating the efficiency of desktop vs mobile research   qor,aor and qantas
 

Deliberating the efficiency of desktop vs mobile research qor,aor and qantas

on

  • 190 views

 

Statistics

Views

Total Views
190
Views on SlideShare
189
Embed Views
1

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 1

http://www.slideee.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Deliberating the efficiency of desktop vs mobile research   qor,aor and qantas Deliberating the efficiency of desktop vs mobile research qor,aor and qantas Presentation Transcript

    • March 4-7, 2014 Ramada Hotel, Singapore www.mrmw.net
    • Title Sponsor Gold Sponsor Silver Sponsors Bag Sponsor Workshop Host Sponsor Event App Partner Networking Reception Partner March 4-7, 2014 Ramada Hotel, Singapore www.mrmw.net
    • Association and Media Partners March 4-7, 2014 Ramada Hotel, Singapore www.mrmw.net
    • Brian Fine, Chairman, QOR (Quality Online Research) / CEO, (AOR) Australia Online Research Dr. Con Menictas, Senior Statistician Loyalty Programs, Qantas Deliberating the efficiency of desktop versus mobile research: a pan-Asian perspective
    • www.qor.com.au www.australiaonlineresearch.com2 INTRODUCTION What did we do? • Looked at theory and myths about mobile research and decided to carry out a case study • Covered Asia (Japan, Korea, China) and the West (Australia, UK, USA) • Tested a number of myths: 1. Cannot do a long survey on mobile 2. Results will differ from PC to Mobile 3. The data collected will not be representative and valid
    • www.qor.com.au www.australiaonlineresearch.com3 WHY DID WE DO THIS? Importance of mobile phones to future research • Smartphone penetration growing across the region, particularly in developed markets • 40-70% penetration, with Korea at the highest in the world and Japan one of the lowest • 78% growth across Asia as of September 2012 (GFK Asia report) • Penetration levels as high as 80% among younger demographic segments 2013 Penetration Japan Korea China Australia UK USA Smartphones 41.7% 67.6% 55% 65% 48.4% 56% • Sources: Japan: eMarketer, May 2013; Korea: Strategy Analytics, June 2013; China: China Internet Watch, 2013; Australia: Google, June 2013; US: Pew Research, Jun 2013 and Nielsen, Sep 213; UK: eMarketer, April 2013
    • www.qor.com.au www.australiaonlineresearch.com4 WHAT DID WE NEED TO TEST? • Platforms: • Smartphones (new platform) vs. PC (standard platform) • Smartphone user: • on Smartphone vs. on PC (same people, different platform)
    • www.qor.com.au www.australiaonlineresearch.com5 Questionnaire Design • Mix of question types: • KPIs (Penetration, Satisfaction, Recommendation) • Attitudes • Demographics • Mix of question styles: • Single selection, Multi selection, Drop down box, Grid, Open ended, Rating scales
    • www.qor.com.au www.australiaonlineresearch.com6 METHOD • Sample sizes achieved by country: Japan Korea China Australia UK USA TOTAL Smartphone on Smartphone 543 642 397 493 407 55 2537 Smartphone on PC 155 510 689 498 1213 941 4006 Other on PC 413 356 171 5605 734 607 2786 TOTAL on PC * 568 866 860 1003 1947 1548 6792 TOTAL RESPONDENTS 1118 1508 1259 1498 2355 1604 9342 * General population - Includes smartphone owners responding on PC
    • Screen shots of survey on Smartphone vs. PC Smartphone PC
    • www.qor.com.au www.australiaonlineresearch.com8 32 31 22 33 42 38 65 77 74 54 56 64 60 53 57 30 21 24 0 10 20 30 40 50 60 70 80 90 100 SMoSM SMoPC PC SMoSM SMoPC PC SMoSM SMoPC PC JAPAN KOREA CHINA Enjoyment (% Top 2 Boxes) Enjoyed filling it out It was ok - like other surveys FINDINGS: VIABILITY? Enjoyment • Positive responses (86-98% Top 2 Box ) – particularly China • Comparable across all three cells in each country • Highest scores were among Smartphone users
    • www.qor.com.au www.australiaonlineresearch.com9 Likelihood to do again • Very positive responses (90-96% Top 2 Box ) • Comparable across all three cells in each country • Highest scores were among Smartphone users 63 58 56 69 65 66 71 58 57 33 35 36 25 29 29 23 33 33 0 10 20 30 40 50 60 70 80 90 100 SMoSM SMoPC PC SMoSM SMoPC PC SMoSM SMoPC PC JAPAN KOREA CHINA Likelihood to do again (% Top 2 Boxes) Very likely Somewhat likely
    • www.qor.com.au www.australiaonlineresearch.com10 Wear out at 5 minute intervals – Australia only % Smartphone PC Start of survey 8% 19% Demographics 2% 1% First category 0% 0% Second category 4% 1% Third category 2% 1% Final section 5% 1% TOTAL DROP-OUTS 21% 22% Total sample 157 664 • Overall drop out levels the same across platforms • Slightly higher proportion of drop outs on PC at start of survey and on Smartphone in last 5 minutes (possibly effect of grid question) Drop-outs as % of Total Completes + Incompletes
    • www.qor.com.au www.australiaonlineresearch.com11 ASSESSMENT FOR DIFFERENCE IN RESULTS BY MODE • We carried out extensive statistical analysis to assess differences in the quality of data obtained via SMoSM, SMoPC or PC: • ANOVA - to test for the differences in means • ANOVA PLOTS - to visually demonstrate differences in means • CORRESPONDENCE ANALYSIS - to visually represent spatial associations of differences • MULTINOMIAL LOGIT MODELS - to provide for statistically different predictors (however none were found) • TEXT MINING and TEXT ANALYTICS - to visually demonstrate the way respondents view their experience of each mode
    • www.qor.com.au www.australiaonlineresearch.com12 FINDINGS: SIMILAR DATA? KPI's: Satisfaction and Recommendation • Results are comparable across modes • Most KPIs show no significant differences • The most significant differences are in Korea in ratings of mobile service providers. A couple differences in China on small brands. Satisfaction Japan Korea China Mobile  x  Banks   x Insurance    Recommendation Japan Korea China Mobile  x x Banks    Insurance     = no significant difference between SM and PC X = significant difference for one or more brands
    • www.qor.com.au www.australiaonlineresearch.com13 SIMILAR RESPONDENTS? Attitudes to Technology • Smartphone respondents are generally more engaged with technology • In China there is high agreement across all cells 67 75 75 46 54 71 44 56 69 Japan Korea China “I have a personal interest in IT” (% Agree) SMoMS SMoPC Total PC
    • www.qor.com.au www.australiaonlineresearch.com14 THE ORIGINAL QUESTIONS ANSWERED! 1. Can a longer survey (eg. 15 minutes) be done on a mobile? YES! Enjoyment is high. Response rates and drop out rates do not differ. 2. What kind of surveys can be done on mobile? All question types proved viable; use of software which adjusts to screen size allows for consistency. 3. Is the data valid? YES! Similar results were achieved. 4. Is the data representative of population? YES, as long as weighting by age and gender is applied.
    • www.qor.com.au www.australiaonlineresearch.com15 Benefits of Smartphone Research • Device Independence • Convenience for respondents • More immediate response; Speed of survey results • Accessing younger and hard-to-reach demographics • Increased reach as Smartphone penetration increases • Centralised communication & social medium of the future • Cost effectiveness - fast and non-invasive medium • Smartphone features lend to more capabilities
    • www.qor.com.au www.australiaonlineresearch.com16 Disadvantages • Limitations of smaller screen • Some evidence that higher drop out with more complex questions (e.g. grids, multi-selection) • Need to adapt questions and software to smaller screen • Smaller pool of potential respondents • May need to calibrate results at brand level
    • www.qor.com.au www.australiaonlineresearch.com17 SUMMARY Smartphone research is a viable option: • No need to limit to short 5-minute or SMS surveys! • Need to use comparable software to PC platform • Myths of limitations may be based on historical software limitations • Can reach broader age groups • Can be fun and likely to retain panellists • Provides valid data, across all sections of a 15- minute questionnaire!
    • Brian Fine, Chairman, QOR (Quality Online Research) / CEO, (AOR) Australia Online Research Dr. Con Menictas, Senior Statistician Loyalty Programs, Qantas Deliberating the efficiency of desktop versus mobile research: a pan-Asian perspective
    • Title Sponsor Gold Sponsor Silver Sponsors Bag Sponsor Workshop Host Sponsor Event App Partner Networking Reception Partner March 4-7, 2014 Ramada Hotel, Singapore www.mrmw.net
    • Association and Media Partners March 4-7, 2014 Ramada Hotel, Singapore www.mrmw.net
    • March 4-7, 2014 Ramada Hotel, Singapore www.mrmw.net