Struminskaya improved cost-effectiveness-in_mobile_surveys_using_hlr-lookup-173


Published on

A GOR presentation

Published in: Business, Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Struminskaya improved cost-effectiveness-in_mobile_surveys_using_hlr-lookup-173

  1. 1. Improved cost-effectiveness in mobile surveys using HLR-Lookup Bella Struminskaya, Ines Schaurer, Wolfgang Bandilla, Siegfried Gabler, Sabine Häder, Lars Kaczmirek GESIS – Leibniz Institute for the Social Sciences, MannheimPlease use the following citation when referring to this presentation:Struminskaya, B., Schaurer, I., Bandilla, W., Gabler, S., Häder, S., & Kaczmirek, L. (2011): Improved cost-effectiveness inmobile surveys using HLR-Lookup. Paper presented at the GOR 11 Conference, March 14-16, 2011, Düsseldorf.Contact: Bella Struminskaya,
  2. 2. The challenge of telephone sampling• Growth of mobile-only households in Germany – 11% mobile-onlys Czech Republic – 73% in the EU – 25% (Special Eurobarometer 335, 2010)• More likely: men, younger - 20-29 year-olds, single, 1-person-household (Graeske & Kunz 2009) 2
  3. 3. Dual Frame Approach• A sample of mobile and landline numbers• Frames overlap is identified, appropriate weights account for multiple probability of selection• Example: modified RDD samples prefix known, random suffixes, takes varying length of telephone numbers into account• No registers with subscribers information 3
  4. 4. Problems of Mobile Samples• High number of non-existent mobile phone numbers  high survey costs: – higher number of calls – Interviewer „burnout“ & lower productivity – in the US cell phone survey is 2-4 times more expensive than landline (AAPOR 2010)• Unclear status of some numbers  difficulties with Response Rate calculation – Ambiguous Operator Messages: „the subscriber is not accepting calls at this time“, „this service / service attribute is not available“ – Illogical patterns of those messages (Häder & Häder 2009) 4
  5. 5. Possible solution: HLR-Lookup• Service available in the mobile (GSM) networks• Identifies status of a number without calling• Status query from Home Location Register (HLR) ― database in the GSM Network which stores the profiles of users required for administrative issues (e.g. billing, charging)• Costs vary, from 0.6 eurocent per number upwards 5
  6. 6. The Service 6
  7. 7. Test of the HLR: CELLA2-Study• Michael Häder (TU Dresden), Siegfried Gabler, Sabine Häder (GESIS, Mannheim)• Sponsored by German Science Foundation (DFG)• Modified RDD Sample, Dual-frame• Fieldwork: June-August 2010• 30 000 mobile phone numbers attempted to contact• Screened via HLR-Lookup in September 2010 7
  8. 8. Results: Comparison of HLR and CELLA2 (I) 8
  9. 9. Results: Comparison of HLR and CELLA2 (II)• 14 156 (56.3% of all numbers) – had we used the HLR- Lookup, we would have called these numbers• 10 805 (43.8%) – those numbers we would have dropped from the original sample• 10 081 (40.8%) – identified right by the HLR-Lookup• 49 (0.2%) – identified wrong by the HLR-Lookup (possibly due to a not-timely check) 9
  10. 10. Cost Savings: Estimation• For 10 000 mobile numbers identified as non- working: – Average time to identify a non-working number = 1.5 min. – Working hours = 250 – Interviewer costs = € 15 per hour – Costs: € 3750 compared to € 180 for HLR-Lookup (not including supervisor costs, interviewer breaks, „interviewer burn-out“…) – Cost savings = 95% 10
  11. 11. Response Rate Calculation• Ambiguous operator messages: cases of unknown eligibility• 13.6% of numbers with ambiguous operator message can be classified as non-working (non-eligible)  accurate Response Rate 11
  12. 12. Possible drawbacks• Fluctuation of mobile numbers• Real-time lookup is relatively expensive (for small survey organizations)• Advice: – divide the sample in packages for screening – minimize time-lag btw. screening and call 12
  13. 13. Privacy Issues• Data received as a result of a HLR-Lookup concerns only information about numbers, no personal information about subscribers is available.• No automatic calls are made to those numbers, no contacts to the subscriber are made.• Disclaimer policies apply to service providers. 13
  14. 14. Bibliography1. AAPOR Cell Phone Task Force (2010): New Considerations for Survey Researchers When Planning and Conducting RDD Telephone Surveys in the U.S. With Respondents Reached via Cell Phone Numbers. [Online]: tentDisplay.cfm&ContentID=2818.2. AAPOR - The American Association for Public Opinion Research (2011): Standard Definitions: Final Disposition Codes and Outcome Rates for Surveys. 7th edition. AAPOR.3. Graeske, Jennifer and Tanja Kunz (2009): Stichprobenqualität der CELLA-Studie unter besonderer Berücksichtigung der Mobile-onlys. In: Häder, M. and Häder, S. (Eds.): Telefonbefragungen über das Mobilfunknetz: Konzept, Design und Umsetzung einer Strategie zur Datenerhebung. VS Verlag, Wiesbaden. P. 57 – 70.4. Häder, Michael and Sabine Häder (2009): Telefonbefragungen über das Mobilfunknetz: Konzept, Design und Umsetzung einer Strategie zur Datenerhebung. VS Verlag, Wiesbaden.5. Special Eurobarometer 335 / Wave 72.5 (2010): E-Communications Household Survey Report. TNS Opinion and Social. Brussels. [Online]: 10/report_en.pdf. 14