Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

10. Voice of America Presentation

1,876 views

Published on

Presentation for Pew Research Center's Methodological Workshop on Public Opinion Research in Sub-Saharan Africa

Published in: Data & Analytics
  • Hi there! Get Your Professional Job-Winning Resume Here - Check our website! http://bit.ly/resumpro
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

10. Voice of America Presentation

  1. 1. Using Mobiles for Survey Research in Africa Lessons Learned Bill Bell Director of Research Voice of America
  2. 2. 2 Voice of America Federally funded; multi-platform content provided in 47 languages Target countries are those lacking free media 29 % of audience is in Africa Biggest audiences in Nigeria, Tanzania, Kenya, Ethiopia Most recent language additions have been in Africa 2
  3. 3. 3 VOA Research Robust program of survey-based research designed to: • Assess audience size, composition, and programming preferences • Understand competitive factors • Track changes in overall media consumption patterns Usually national samples of 1500-2000 All F2F; traditionally P&P but increasingly CAPI 3
  4. 4. 4 What’s Wrong with the Status Quo?
  5. 5. 5 Are we there yet?...
  6. 6. 6 Which way was that sampling point?
  7. 7. 7 Anyone home?
  8. 8. 8 Are Mobile Surveys the Answer? Definition: Administration of survey instrument to randomly selected owners of mobile devices via SMS, IVR, or mobile web 8
  9. 9. 9 How Mobile Surveys Work Vendors obtain operator-provided lists of numbers, which serve as sampling frame Survey invitations sent to randomly selected numbers SMS: Respondents opting in are sent short list of questions as individual text messages with numeric response categories IVR: Questions sent in audio form; numeric responses No cost to respondent; and respondents are incentivized with free air time where possible
  10. 10. 10 Potential Advantages of Mobile Speed: (weeks vs. months) Cost: ($5-10K/country vs. $100K +) Overcome geographic limits due to conflict or logistics No clustering effects Pre-existing sampling frames (lists of mobile numbers) Easy to incentivize participation via air time credits 10
  11. 11. 11 Disadvantages Limits on number/type of question and response categories Cost per question Uncertain quality of sampling frames • Quality depends on accuracy of lists provided by mobile operators and number of operators making information accessible Coverage limits (geographic, demographic) 11
  12. 12. 12 Mobile Ownership in Selected African Countries 87 85 79 59 55 47 45 0 10 20 30 40 50 60 70 80 90 100 Nigeria (2016) Kenya (2017) Cote d'Ivoire (2017) Mali (2017) Tanzania (2015) DRC (2016) Zambia (2016) Source: BBG surveys in years shown 12 Percentage of adults (15+) who personally own a mobile
  13. 13. 13 Demographics of Mobile Ownership Percentageofeachgroupowningamobiledevice 62 68 47 50 60 69 53 54 0 10 20 30 40 50 60 70 80 Tanzania Mali Male Female Urban Rural 13
  14. 14. 14 Our Experiment Goal: Explore whether mobile surveys could substitute for F2F in some markets Methods: Conduct mobile surveys in markets where recent F2F results were available Compare sample composition, media access/use findings; basic audience data 14
  15. 15. HOW DO MOBILE SURVEY RESULTS STACK UP AGAINST TRADITIONAL F2F?
  16. 16. 16 Background and Comparison Mobile surveys run in Tanzania, DRC, Ghana Kenya); only SMS surveys shown here Sample sizes ca. 500 Note questions and response categories not fully comparable due to limitations of SMS format Mobile survey results are compared here to that portion of the F2F samples that owned mobiles and used them for SMS
  17. 17. 17 Methods DRC Ghana Kenya Tanzania N = 555 552 613 617 Languages(s) French English English/Swahili 50/50 Swahili Completion rate* 1.02% 1.99% 1.3% 0.29% No. of poll days 4 1 7 1 F2F n= 620 602 1088 1210 17 *Completion rate = number of complete interviews/number of invitations sent out
  18. 18. 18 Question Form Constraints of SMS format required “compression” of questions and responses, possibly affecting respondents’ comprehension and data quality 18 Do you ever get news about current events in your country or around the world on your mobile phone? 1) Yes 2) No [Reply 1 or 2] Which of these items do you have in your household? 1) Radio 2) TV 3) Computer [Choose all numbers that apply]
  19. 19. 19 Age 15-24 25-34 35+ Mobile Survey F2F TanzaniaKenyaGhanaDRC 34% 28% 38% 24% 37% 37% 35% 29% 36% 5% 26% 69% 30% 38% 32% 17% 28% 57% 39% 29% 32% 11% 24% 65% Avg Age: Mobile F2F 29.7 23.7 26.3 24.9 32.3 32.6 30.9 33.4
  20. 20. 20 Gender Male Female TanzaniaKenyaGhanaDRC 41% 59% 21% 79% 49% 51% 24% 76% 53% 47% 30% 70% 46% 54% 30% 70% Mobile Survey F2F
  21. 21. 21 Home Ownership of Media Technologies Computer Radio TV TanzaniaKenyaGhanaDRC 54% 85% 9% 64% 57% 37% 83% 94% 21% 59% 36% 25% 45% 94% 5% 41% 58% 22% 27% 91% 3% 38% 51% 4% Mobile Survey F2F
  22. 22. 22 Use of Mobile Phone Features View a video Access the Internet Take a picture TanzaniaKenyaGhanaDRC 56% 10% 13% 11% 11% 50% 41% 22% 28% 32% 28% 28% 24% 10% 9% 10% 36% 42% 17% 28% 31% 35% 25% 13% 7% 4% 13% 40% 32% 26% 23% 36% 31% 17% 3% 4% 3% 3% 43% 25% 24% 22% 27% 19% Access FB Send emails Listen to radio N/A N/A Mobile Survey F2F N/A N/A
  23. 23. 23 Radio Station Listenership China Radio GBC1 RFI TanzaniaKenyaGhanaDRC 9.6% 6.5% 23.1% 5.4% 25.7% 20.1% 15.1% 7.2% 34.0% 8.5% 12.6% 22.2% 23.3% 39.7% 46.8% 8.9% 12.3% 44.1% BBC DW VOA Mobile Survey F2F 9.3% 48.8% 14.1% 15.1% 48.3% 19.3%
  24. 24. 24 TV Viewership CCTV France24 TanzaniaKenya GhanaDRC 2.5% 28.3% 18.0% 1.5% 7.2% 47.9% 26.5% 7.0% 2.0% 4.6% 5.2% 6.2% 17.6% 7.2% 21.0% 0.7% 3.2% 2.7% 14.0% 15.0% 26.8% 7.6% 7.8% 13.7% 7.9% 5.5% 6.8% 32.6% 12.0% BBC TV DW TV VOA TV5 Monde Al Jazeera Mobile Survey F2F
  25. 25. 25 Why Such Large Differences? SMS surveys may engage segment of population that is very different, not just in terms of demos but also behavior Changes in question format and response categories may affect results Low response rates on mobile especially problematical, though direction of bias is unclear 25
  26. 26. 26 Moving Forward …  At present mobile surveys don’t permit sufficient statistical rigor to be used for KPI estimates  They may become more useful to the extent that we’re interested in the behavior of highly specific population segments , especially if we’re able to boost participation  The methodology could be particularly useful in instances where just need answers to one or two quick questions, especially if we’re willing to live with less rigorously derived results  Mobiles also offer the opportunity for panel based evaluation cheaply and quickly

×