Random digit dialing cell phone surveys and 
  surveillance systems: data quality, data 
       ill        t     d t      ...
Wireless substitution (US), 2005 2008
  Wireless substitution (US), 2005–2008




Source: Blumberg & Luke  Wireless substi...
Demographic differences
                      Demographic differences
    •    Gender: men more likely than 
             ...
Biased health estimates?
        Biased health estimates?
• Potential for biased health estimates due to sample
  Potentia...
Health estimates by phone status
   Health estimates by phone status
                                          Has a landl...
Cigarette smoking among young adults, 
       2003‐2005, NHIS & BRFSS
       2003 2005 NHIS & BRFSS




Source: Delnevo, G...
New challenges: Wireless Mostly?
                                                                      Telephone Status, N...
Reactions?
• Starting in 2009, the Center 
  for Disease Control and 
  f Di         C      l d
  Prevention (CDC) is 
  r...
This special session:
         This special session:
• Analysis of data quality of cell phone
  Analysis of data quality o...
Assessing Data Quality of
Cell Phone Random Digit
                      g
      Dial Surveys
      Frederica R Conrey
    ...
Data Quality

• Q lit of Responses
  Quality f R
• Quality of Sample
2008 New Jersey Adult Tobacco
              Survey
• R d
  Random Di it Di l L d (RR 19%) and
           Digit Dial Land (...
Survival Analysis

• Diff
  Different people get diff
          t     l    t different surveys
                           ...
Quality of Responses
Item Non-Response
               Failure Rates by Phone Mode
                     Mean
                     M             ...
Open Ended Response

                           Total Responses p Open
                                    p      per p
Op...
Quality of Sample
Unit Non-Response

Data
D t quality i th t
       lit is threatened if
                       d
  – Response rates are low...
Survival Analysis

• C ll and l dli respondents may get
  Cell d landline          d t           t
  different surveys
• R...
Survival Model

                      25%
Sample Breaking Off




                      20%
              g




          ...
What does a difference in survey
          survival mean?
• C ll respondents quit sooner th l dli
  Cell       d t      it...
In a population study of tobacco use
              behavior…
Minimal difference
Mi i l diff                  Substantial d...
Weighting Cell Phone Surveys 
Weighting Cell Phone Surveys



          Randal Zuwallack
         Frederica R Conrey
     ...
Thanks

• Cris Delnevo
  Cris Delnevo
• Dan Gundersen
• NCI (R21CA129474), New Jersey Dept of 
   C ( 2 C 29       )      ...
Dual Frame



                          D




A.  Adults in landline households with no cell phone,
A Adults in landline h...
Dual Frame



                         D




Common designs:
Common designs:
  Dual frame w/ no overlap: Landline (A+B) + ...
Weighting Challenges

• Challenge 1: How do we put the dual frames
  Challenge 1: How do we put the dual frames 
  togethe...
Challenge 1

• How do we put the dual frames together?
  How do we put the dual frames together?
• No overlap
  – Estimate...
Cell Survey

• “In addition to your cell phone is there at
   In addition to your cell phone, is there at 
  least one tel...
Landline Survey

• “In addition to your residential landline
   In addition to your residential landline 
  telephone, do ...
Example 1‐‐Colorado

• Combine with BRFSS
  Combine with BRFSS
• Group membership 
  –KKnown for cell
          f     ll
 ...
Example 2

• You are midway through a landline survey and
  You are midway through a landline survey and 
  want to add ce...
Challenge 2

• Differential Nonresponse
  Differential Nonresponse
• Cell‐only overrepresented when conducting 
  cell pho...
Telephone Reliance
                         C
L
                         e
a
                         l
n
                ...
Landline Frame
                               C
L
                               e
a
                               l
n
  ...
Cell Phone Frame
                                      C
L
                                      e
a
                     ...
Dual Frames

 Landline sample             Landline sample




    Cell sample                 Cell sample



Dual Frame sa...
Our Goal

               Rebalance 
               R b l
                 on cell 
                reliance



Rebalance 
...
Measuring Telephone Reliance

• Cell only landline only
  Cell only, landline only
• Classify Dual users
  – “Of ll th t l...
Telephone Reliance
                                                 C
L
                                                 e...
Response Propensity

• Adjust for differential nonresponse by
  Adjust for differential nonresponse by 
  benchmarking aga...
Data sources

                  National cell sample
                  National cell sample   NHIS
                       ...
Applying the model

• Applied to same data—poststatification
  Applied to same data poststatification
• Applied to indepen...
Applying the model

                                  Colorado
                                Cell Sample
               ...
Applying the model

• Assume landline only is 20% (we don’t know)
  Assume landline only is 20% (we don t know)
          ...
City Sample

                   Landline     Cell Sample         Combined 
                    Sample                     ...
Conclusions

• Dual‐frame
  Dual frame
  – There are ways to combine the data, even when 
    we don t have a full picture...
Thank you

Randal.Zuwallack@macrointernational.com
Randal Zuwallack@macrointernational com
Frederica.Conrey@macrointernati...
Examining the bias in landline only 
            g                        y
 surveys: How does the cell phone only 
   pop...
Cell Phone Substitution and RDD surveys
 Cell Phone Substitution and RDD surveys
• RDD surveys (e.g. BRFSS) have tradition...
What is bias due to coverage error in the 
              sampling frame?
                   li f      ?
• Non‐covered popu...
Previous Research
                Previous Research
• Data from Jan 2004‐June 2005 NHIS found2
   – Greater than 1 percent...
Our Study:
                   Our Study:
• Objective:
  Objective: 
  – Assess the presence of bias in landline RDD due to...
Methodology
• Cell Phone sample:
  Ce     o e sa p e:
  – Design weights account for probability of selection
• BRFSS
  – ...
Statistical Analyses
               Statistical Analyses
• Comparisons of BRFSS landline and Cell Only based on design 
  ...
Table. CO BRFSS vs. CO Cell Only, May‐September 2008 (n=5,028)
                               y,   y p              (   , ...
Figure 1. Bias of Smoking Prevalence, CO BRFSS (n=4,527) and CO Cell Only 
 RDD (n=501) May‐September 2008
 RDD (n 501) Ma...
Figure 2. Relative Bias of Smoking Prevalence, CO BRFSS (n=4,527) and CO Cell 
Only RDD (n=501) May‐September 2008
Only RD...
Figure 3. Bias of Ever had an HIV test, CO BRFSS (n=4,527) and CO Cell Only 
 RDD (n=501) May‐September 2008
 RDD (n 501) ...
Figure 4. Relative Bias of Ever had an HIV test, CO BRFSS (n=4,527) and CO 
Cell Only RDD (n=501) May‐September 2008
Cell ...
Figure 5. Bias ‐ Having health insurance, CO BRFSS (n=4,527) and CO Cell Only 
   RDD (n=501) May‐September 2008
   RDD (n...
Figure 6. Relative Bias Having Health Insurance, CO BRFSS (n=4,527) and CO 
Cell Only RDD (n=501) May‐September 2008
Cell ...
Figure 7. Bias Has primary care provider, CO BRFSS (n=4,527) and CO Cell Only 
  RDD (n=501) May‐September 2008
  RDD (n 5...
Figure 8. Relative Bias Has Primary Care Provider, CO BRFSS (n=4,527) and CO 
Cell Only RDD (n=501) May‐September 2008
Cel...
Figure 9. Bias ‐ could not afford health care due to cost, CO BRFSS (n=4,527) 
  and CO Cell Only RDD (n=501) May‐Septembe...
Figure 10. Relative Bias ‐ could not afford health care due to cost, CO 
 BRFSS (n=4,527) and CO Cell Only RDD (n=501) May...
Summary of Findings
             Summary of Findings
• Bias is present not only among those with high wireless 
  substitu...
Implications for study design and 
                 analysis
                     l
• When possible, include an RDD of cel...
Limitations
• Unable to assess bias in some subpopulations
  Unable to assess bias in some subpopulations 
  due to small ...
References
1.   Blumberg SJ & Julian V. Luke. (2009). Wireless Substitution: Early 
     release of estimates from the Nat...
Contact Info
                    Contact Info
      Cristine Delnevo, PhD, MPH delnevo@umdnj.edu
      Cristine Delnevo Ph...
Upcoming SlideShare
Loading in …5
×

Random digit dialing cell phone surveys and surveillance systems: data quality, data weighting strategies, and bias

2,471 views

Published on

Published in: Technology, Business
2 Comments
4 Likes
Statistics
Notes
  • MOBILE-NUMBERTRACKER
    PROTECT YOUR PHONE FROM INSIDE - legal and ethical.. Read on and get informed -
    An amazing resource for the latest in SmartPhone info and the inside truth on SPYWARE and Phone Hacking -- FREE to you,-- (free Cell Phone Spy report as a bonus)!
    GO HERE:
    http://mobile-numbertracker.com/
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • The health estimation is truly amazing. Good job. I hope you won't mind if I embed it at my site at : www.sprintringtones.org/

    Thanks. Hope to see more slides from you.
    Anita Gomez.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
2,471
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
34
Comments
2
Likes
4
Embeds 0
No embeds

No notes for slide

Random digit dialing cell phone surveys and surveillance systems: data quality, data weighting strategies, and bias

  1. 1. Random digit dialing cell phone surveys and  surveillance systems: data quality, data  ill t d t lit d t weighting strategies, and bias Cristine D. Delnevo, PhD, MPH & Daniel A. Gundersen, MA, UMDNJ‐ School of Public Health Randal S. ZuWallack, MS & Frederica R. Conrey, PhD Randal S ZuWallack MS & Frederica R Conrey PhD ICF Macro International Presented at 137th Annual Meeting & Exposition P t d t 137th A l M ti & E iti Philadelphia, PA November 7‐11, 2009 Work supported in part by the National Cancer Institute (R21CA129474 ) and a contract from the New  Jersey Department of Health and Senior Services, through funding from the Cigarette Tax
  2. 2. Wireless substitution (US), 2005 2008 Wireless substitution (US), 2005–2008 Source: Blumberg & Luke  Wireless substitution: Early release of estimates from the National Health Interview Survey, July‐ December 2008. National Center for Health Statistics. May 2009.  
  3. 3. Demographic differences Demographic differences • Gender: men more likely than  Wireless substitution by age (and time) Wireless substitution by age (and time) women to be wireless only b i l l • SES: Adults living in (30.9%) and  near poverty (23.8%) more likely  than higher income adults  h hi h i d l (16.0%) to be wireless only • Region: Wireless substitution  highest in South (21.3%) and  hi h t i S th (21 3%) d Midwest (20.8%) vs. Northeast  (11.4%) or West (17.2 %) • Race/Ethnicity: Wireless  R /Eth i it Wi l substitution highest  among  black (21.4%) and Hispanic  (25.0%) adults vs. Non‐Hispanic  (25 0%) adults vs Non Hispanic white adults (16.6%)   Source: Blumberg & Luke  Wireless substitution: Early release of estimates from the National Health Interview Survey, July‐ December 2008. National Center for Health Statistics. May 2009.  
  4. 4. Biased health estimates? Biased health estimates? • Potential for biased health estimates due to sample Potential for biased health estimates due to sample  under‐coverage remains a real, growing threat to  RDD health surveys • Cell‐phone only also differs with respect to health  behaviors and the validity of some health estimates  based on traditional RDD surveys are increasingly  questionable
  5. 5. Health estimates by phone status Health estimates by phone status Has a landline Wireless-only y No telephone p telephone NHIS July – December 2007 5+ alcoholic drinks in 1 day 17.7 37.3 27.1 Current smoker 18.0 30.6 38.6 Uninsured 13.7 13 7 28.7 28 7 44.1 44 1 Has a usual place for care 87.5 68.0 61.8 Flu vaccination 32.7 16.6 20.9 Ever tested for HIV 34.7 47.6 45.8 Source: Blumberg SJ, Luke JV. Coverage bias in traditional telephone surveys of low‐income and young  adults. Public Opin Q. 2007;71:734–749
  6. 6. Cigarette smoking among young adults,  2003‐2005, NHIS & BRFSS 2003 2005 NHIS & BRFSS Source: Delnevo, Gundersen & Hagman (2008) Declining prevalence of alcohol and smoking estimates  among young adults nationally: artifacts of sample under‐coverage?  Am J of Epidemiology
  7. 7. New challenges: Wireless Mostly? Telephone Status, NHIS July‐ December 2008 • The percentage of adults  living in wireless‐mostly  households has been  increasing • Who are they? Who are they?  (demographic & health  behaviors?) • Will they respond to  landline surveys?  Source: Blumberg & Luke  Wireless substitution: Early release of estimates from the National Health Interview Survey, July‐ December 2008. National Center for Health Statistics. May 2009.  
  8. 8. Reactions? • Starting in 2009, the Center  for Disease Control and  f Di C l d Prevention (CDC) is  requiring states to  incorporate cell phone  incorporate cell phone interviews in their regular  BRFSS sample • Yet there is no widely  y accepted methods of  evaluating data quality or  data weighting, particularly  for state and local area  for state and local area surveys.  • AAPOR Cell Phone Task  Force report Force report
  9. 9. This special session: This special session: • Analysis of data quality of cell phone Analysis of data quality of cell phone  surveys • Demonstrate weighting procedures for  merging cell phone samples with landline  samples • An assessment of bias in landline only An assessment of bias in landline only  surveys, and 
  10. 10. Assessing Data Quality of Cell Phone Random Digit g Dial Surveys Frederica R Conrey Randy Zuwallack
  11. 11. Data Quality • Q lit of Responses Quality f R • Quality of Sample
  12. 12. 2008 New Jersey Adult Tobacco Survey • R d Random Di it Di l L d (RR 19%) and Digit Dial Land (RR=19%) d Cell Phone (RR=16%) • Short version – 49 Questions (min=37; max=70) – 534 Cell completes – 468 Landline completes
  13. 13. Survival Analysis • Diff Different people get diff t l t different surveys t because of skip patterns • Survival analysis – Measures the impact of survey mode on non- response – Controls for differences in survey length
  14. 14. Quality of Responses
  15. 15. Item Non-Response Failure Rates by Phone Mode Mean M Median M di Std Cell 3.3% 2.2% 7.4% Landline 3.4% 2.0% 5.5% Survival predicted by p p y phone mode: Hazard=1.00, p>.95
  16. 16. Open Ended Response Total Responses p Open p per p Open end reports of the O d t f th End by Phone Mode events of 2 recalled Responses commercials / OE DK / OE – 251 respondents were Cell .52 .49 asked at least one Landline .48 .55 open ended question P .56 .33
  17. 17. Quality of Sample
  18. 18. Unit Non-Response Data D t quality i th t lit is threatened if d – Response rates are low AND – The people who DO NOT respond are different from those who DO. If cell phone respondents are less likely to respond, then there is non-response bias.
  19. 19. Survival Analysis • C ll and l dli respondents may get Cell d landline d t t different surveys • Response rates alone don’t tell the story • Survival analysis tells whether cell y respondents are more likely to break off g given the same survey length y g
  20. 20. Survival Model 25% Sample Breaking Off 20% g 15% 10% Cell e Landline 5% 0% 0 20 40 60 80 Survey Questions p<.001 001
  21. 21. What does a difference in survey survival mean? • C ll respondents quit sooner th l dli Cell d t it than landline respondents. • The sample under-represents cell phones • The longer the survey, the worse the g y nonresponse bias • The solution? – Careful weighting – Short surveys
  22. 22. In a population study of tobacco use behavior… Minimal difference Mi i l diff Substantial difference S b t ti l diff between cell and between cell and landline in response landline in response quality rate • No difference emerged in • Sample quality may be item-nonresponse threatened if cell phone • No difference emerged in surveys are too long or richness of open end i h f d weighted incorrectly responses.
  23. 23. Weighting Cell Phone Surveys  Weighting Cell Phone Surveys Randal Zuwallack Frederica R Conrey Frederica R Conrey
  24. 24. Thanks • Cris Delnevo Cris Delnevo • Dan Gundersen • NCI (R21CA129474), New Jersey Dept of  C ( 2 C 29 ) f Health and Senior Services
  25. 25. Dual Frame D A.  Adults in landline households with no cell phone, A Adults in landline households with no cell phone B.  Adults in landline households with a cell phone, and C. Adults in non‐landline households with a cell phone (cell only).
  26. 26. Dual Frame D Common designs: Common designs: Dual frame w/ no overlap: Landline (A+B) + Cell (C) Dual frame w/ overlap: Landline (A+B) + Cell (B+C) Uncommon design: Dual frame w/ no overlap: Landline (A) + Cell (B+C) Dual frame w/ no overlap: Landline (A) + Cell (B+C)
  27. 27. Weighting Challenges • Challenge 1: How do we put the dual frames Challenge 1: How do we put the dual frames  together? • Challenge 2:  Differential Nonresponse
  28. 28. Challenge 1 • How do we put the dual frames together? How do we put the dual frames together? • No overlap – Estimate of cell‐only population size? Estimate of cell only population size? • Internal estimate • External estimate: NHIS (Blumberg et al.) • With overlap – Must determine group membership – Adjust for multiple selection probabilities – Estimate of phone group population sizes?
  29. 29. Cell Survey • “In addition to your cell phone is there at In addition to your cell phone, is there at  least one telephone inside your home that is  currently working and is not a cell phone?  Do  currently working and is not a cell phone? Do not include telephones only used for business  or telephones only used for computers or fax  or telephones only used for computers or fax machines.”  – ‘yes’ = dual user, while those who responded  yes = dual user while those who responded – ‘no’ = cell‐only
  30. 30. Landline Survey • “In addition to your residential landline In addition to your residential landline  telephone, do you also use one or more cell  phone numbers? phone numbers?”  – ‘yes’ = dual user – ‘no’ = landline only. no = landline only
  31. 31. Example 1‐‐Colorado • Combine with BRFSS Combine with BRFSS • Group membership  –KKnown for cell f ll – Unknown for landline • Limited to dual frame w/ no overlap – Used 15% (NHIS state estimates) for merging  landline and cell – Poststratified dual sample to age and sex.
  32. 32. Example 2 • You are midway through a landline survey and You are midway through a landline survey and  want to add cell phones.  You don’t know who  has a cell phone and who doesn t. What are  has a cell phone and who doesn’t What are your options? 1) Add cell only 1) Add cell only 2) Add cell and dual‐users 
  33. 33. Challenge 2 • Differential Nonresponse Differential Nonresponse • Cell‐only overrepresented when conducting  cell phone surveys.  cell phone surveys – Contact rate –CCooperation rate i • Those who rely more on their cell phone will  be easier to reach.
  34. 34. Telephone Reliance C L e a l n l d l i P n h e o n e
  35. 35. Landline Frame C L e a l n l d l i Landline households P n h e o n e Landline sample
  36. 36. Cell Phone Frame C L e a l n l d l i Cell Phone Users P n h e o n e Cell sample Cell sample
  37. 37. Dual Frames Landline sample Landline sample Cell sample Cell sample Dual Frame sample Dual Frame sample Ideal Realistic
  38. 38. Our Goal Rebalance  R b l on cell  reliance Rebalance  on landline  reliance
  39. 39. Measuring Telephone Reliance • Cell only landline only Cell only, landline only • Classify Dual users – “Of ll th t l h “Of all the telephone calls that you receive, are…” ll th t i ” • All or almost all calls received on a cell phone? (cell‐ mostly) • Some received on a cell phone and some on a regular  landline phone? (true‐dual) • Very few or none received on a cell phone? (landline‐ mostly)
  40. 40. Telephone Reliance C L e a l n d Landline Landline True Cell Cell l l i Only Mostly Dual Mostly Only P n (0) (1) (2) (3) (4) h e o n e
  41. 41. Response Propensity • Adjust for differential nonresponse by Adjust for differential nonresponse by  benchmarking against NHIS • Logistic regression model Logistic regression model – Dependent: Survey type •1 b 1 = observe cell user in national cell phone survey ll i ti l ll h • 0 = observe cell user in NHIS – Independent: Cell phone reliance (1 4) age race Independent: Cell phone reliance (1‐4), age, race
  42. 42. Data sources National cell sample National cell sample NHIS n=500 Landline mostly 8% 23% True dual 27% 42% Cell mostly 23% 17% Cell only 42% 18% Cell users 100% 100%
  43. 43. Applying the model • Applied to same data—poststatification Applied to same data poststatification • Applied to independent data –A Assumption: national cell sample measures the  ti ti l ll l th odds ratio for observing a cell‐only respondent in  a cell sample relative to a dual user. a cell sample relative to a dual‐user • S State, local surveys  l l
  44. 44. Applying the model Colorado Cell Sample w/o NR adj w/ NR adj Landline mostly (1) 9% 26% True dual (2) 26% 37% Cell mostly (3) C ll tl (3) 19% 15% Cell only (4) 46% 21% Total cell sample 100% 100%
  45. 45. Applying the model • Assume landline only is 20% (we don’t know) Assume landline only is 20% (we don t know) CO Landline only 20% Landline mostly 21% True dual 30% Cell mostly 12% NHIS  NHIS Cell only 17% state  estimate  Total population p p 100% = 15%
  46. 46. City Sample Landline Cell Sample Combined  Sample Samples w/o NR  w/o NR w/ NR  w/ NR w/o NR  w/o NR w/ NR  w/ NR adj adj adj adj Cell‐only ‐ 43.5 18.9 35.5 13.4 Cell‐mostly 12.4 25.1 19.8 13.8 12.2 True Dual 30.1 23.6 35.5 18.7 25.1 Landline mostly Landline‐mostly 19.1 7.9 25.8 9.9 17.2 Landline‐only 38.4 ‐ ‐ 23.9 32.1
  47. 47. Conclusions • Dual‐frame Dual frame – There are ways to combine the data, even when  we don t have a full picture of group membership. we don’t have a full picture of group membership • Differential nonresponse – Response propensity model rebalances the cell Response propensity model rebalances the cell  sample based on cell reliance.  – Can be applied at state and local levels when no Can be applied at state and local levels when no  benchmarks exist. – Next steps: explore a response propensity model Next steps: explore a response propensity model  for landline.
  48. 48. Thank you Randal.Zuwallack@macrointernational.com Randal Zuwallack@macrointernational com Frederica.Conrey@macrointernational.com
  49. 49. Examining the bias in landline only  g y surveys: How does the cell phone only  population differ from the landline  population differ from the landline population on health indicators, and are  estimates from landline surveys biased? Daniel A. Gundersen, MA, UMDNJ‐SPH Cristine D. Delnevo, PhD, MPH, UMDNJ‐SPH Randy S. ZuWallack, MS, ICF Macro
  50. 50. Cell Phone Substitution and RDD surveys Cell Phone Substitution and RDD surveys • RDD surveys (e.g. BRFSS) have traditionally only sampled  household telephones (i.e. landlines) • Up until early 2000s, rate of cell phone only households  was small • From mid 2000s, rate of substitution has grown  substantially – 6 7% of adults in 2005 to 18 4% in 2008 nationally1 6.7% of adults in 2005 to 18.4% in 2008 nationally • Higher among certain demographic groups1 – Young adults – Hispanics and Blacks – Poor and near poor
  51. 51. What is bias due to coverage error in the  sampling frame? li f ? • Non‐covered population is different from Non covered population is different from  covered population on some variable of  interest – If proportion of non‐covered (    ) is small, bias will  be small – If difference between the covered and  , noncovered                is small, bias will be small
  52. 52. Previous Research Previous Research • Data from Jan 2004‐June 2005 NHIS found2 – Greater than 1 percentage point bias in binge drinking, smoking  prevalence, usual place for medical care, receiving influenza  vaccine • Data from 2007 NHIS found3 Data from 2007 NHIS found – Bias increased slightly for past year binge drinking and receiving  influenza vaccine – These biases were larger among young adults and low income These biases were larger among young adults and low income  persons • Data from 2001‐2005 BRFSS on 18‐24 year olds found4 – Prevalence of binge drinking, heavy drinking, and cigarette  smoking declined during 2003‐2005; coincided with large  increase in wireless substitution among young adults – NHIS and NSDUH did not observe similar declines during this  period
  53. 53. Our Study: Our Study: • Objective: Objective:  – Assess the presence of bias in landline RDD due to  exclusion of cell phone only  on select health  exclusion of cell phone only on select health indicators • Data Source and Instrument: Data Source and Instrument: – Cell phone RDD of adults in Colorado (n=501) • May to September 2008 May to September 2008 • Instrument was shortened version of BRFSS – BRFSS from same data collection period (n=4,527) BRFSS from same data collection period (n 4,527)
  54. 54. Methodology • Cell Phone sample: Ce o e sa p e: – Design weights account for probability of selection • BRFSS – Standard BRFSS design weight accounts for strata,  number of landlines and adults in the household – Postratified b f d by age(7)*sex*race ( )* * • Merged data –D i Design weights scaled to represent share of  i ht l dt t h f population by phone status – Postratified by age(7)*sex*race y g ( )
  55. 55. Statistical Analyses Statistical Analyses • Comparisons of BRFSS landline and Cell Only based on design  weights • Comparison of BRFSS landline and merged data are  postratified to demographic makeup of CO • We assume the merged data to be unbiased (i.e. no coverage  error due to cell only exclusion) error due to cell only exclusion) • All analyses conducted in STATA v.10.1 to account for complex  sampling design
  56. 56. Table. CO BRFSS vs. CO Cell Only, May‐September 2008 (n=5,028) y, y p ( , ) BRFSS landline Cell only y Difference Health Indicator % (95%CI) % (95%CI) % (95%CI) Smoking* * 15.29 (±1.14) 28.14 (±4.46) ‐12.85 (±4.60) ( ) ( ) ( ) Ever had HIV test* 36.64 (±1.85) 52.51 (±5.09) ‐15.87 (±5.41) Having health insurance* 88.36 (±1.11) 72.46 (±4.38) 15.9 (±4.51) Having primary care provider* 85.91 (±1.16) 60.39 (±4.85) 25.52 (±5.00) Not affording care due to cost 12.27 (±1.08) 20.42 (±3.94) Not affording care due to cost* 12 27 (±1 08) 20 42 (±3 94) ‐8.15 (±4.08) ‐8 15 (±4 08) *p<.05; data weighted to correct for sampling design
  57. 57. Figure 1. Bias of Smoking Prevalence, CO BRFSS (n=4,527) and CO Cell Only  RDD (n=501) May‐September 2008 RDD (n 501) May September 2008 10 7.5 75 5 2.5 0 ‐2.5 ‐5 ‐7.5 ‐10
  58. 58. Figure 2. Relative Bias of Smoking Prevalence, CO BRFSS (n=4,527) and CO Cell  Only RDD (n=501) May‐September 2008 Only RDD (n 501) May September 2008 50% 40% 30% 20% 10% 0% ‐10% ‐20% ‐30% ‐40% ‐50% 50%
  59. 59. Figure 3. Bias of Ever had an HIV test, CO BRFSS (n=4,527) and CO Cell Only  RDD (n=501) May‐September 2008 RDD (n 501) May September 2008 10 7.5 5 2.5 0 ‐2.5 ‐5 ‐7.5 ‐10
  60. 60. Figure 4. Relative Bias of Ever had an HIV test, CO BRFSS (n=4,527) and CO  Cell Only RDD (n=501) May‐September 2008 Cell Only RDD (n 501) May September 2008 50% 40% 30% 20% 10% 0% ‐10% ‐20% ‐30% ‐40% ‐50% 50%
  61. 61. Figure 5. Bias ‐ Having health insurance, CO BRFSS (n=4,527) and CO Cell Only  RDD (n=501) May‐September 2008 RDD (n 501) May September 2008 10 7.5 5 2.5 0 ‐2.5 ‐5 ‐7.5 ‐7 5 ‐10
  62. 62. Figure 6. Relative Bias Having Health Insurance, CO BRFSS (n=4,527) and CO  Cell Only RDD (n=501) May‐September 2008 Cell Only RDD (n 501) May September 2008 50% 40% 30% 20% 10% 0% ‐10% ‐20% ‐30% ‐40% ‐50% 50%
  63. 63. Figure 7. Bias Has primary care provider, CO BRFSS (n=4,527) and CO Cell Only  RDD (n=501) May‐September 2008 RDD (n 501) May September 2008 10 7.5 5 2.5 0 ‐2.5 ‐5 75 ‐7.5 ‐10
  64. 64. Figure 8. Relative Bias Has Primary Care Provider, CO BRFSS (n=4,527) and CO  Cell Only RDD (n=501) May‐September 2008 Cell Only RDD (n 501) May September 2008 50% 40% 30% 20% 10% 0% ‐10% ‐20% ‐30% ‐40% ‐50% 50%
  65. 65. Figure 9. Bias ‐ could not afford health care due to cost, CO BRFSS (n=4,527)  and CO Cell Only RDD (n=501) May‐September 2008 and CO Cell Only RDD (n 501) May September 2008 10 7.5 5 2.5 0 ‐2.5 ‐5 ‐7.5 7.5 ‐10
  66. 66. Figure 10. Relative Bias ‐ could not afford health care due to cost, CO  BRFSS (n=4,527) and CO Cell Only RDD (n=501) May‐September 2008 BRFSS (n 4 527) and CO Cell Only RDD (n 501) May September 2008 50% 40% 30% 20% 10% 0% ‐10% ‐20% ‐30% ‐40% ‐50%
  67. 67. Summary of Findings Summary of Findings • Bias is present not only among those with high wireless  substitution rates • Smoking prevalence underestimated among those with  higher wireless substitution rates higher wireless substitution rates • Ever had an HIV test substantially underestimated  among all groups – R l ti bi l Relative bias large among those with high wireless  th ith hi h i l substitution rates (young adults, non‐whites, low SES) • Bias for health care insurance and having primary care  provider is underestimated among non‐whites, but  id i d i d hi b overestimated among other groups – Relative bias is small
  68. 68. Implications for study design and  analysis l • When possible, include an RDD of cell phone only  population (BRFSS now does this) – If you can’t, be aware of the potential for bias and interpret  findings accordingly • If you’re analyzing landline RDD data from past years – Interpret findings with potential bias in mind – Historical trend may observe artificial changes due to coverage  error – Wireless substitution rates differ by geographic region so  problem may be less in certain areas • A bi A bias present today may not be the same historically  or in  d b h hi i ll i the future – Characteristics of the early adopters may not be the same as the  current cell only population today or laggards current cell only population today or laggards
  69. 69. Limitations • Unable to assess bias in some subpopulations Unable to assess bias in some subpopulations  due to small sample size • Study does not account for cell phone mostly Study does not account for cell phone mostly  population
  70. 70. References 1. Blumberg SJ & Julian V. Luke. (2009). Wireless Substitution: Early  release of estimates from the National Health Interview Survey,  l f ti t f th N ti l H lth I t i S July‐December 2008. 2. Blumberg SJ, Luke JV & Marcie L. Cynamon. (2006). Telephone  coverage and health survey estimates: evaluating concern about  coverage and health survey estimates: evaluating concern about wireless substitution. American Journal of Public Health. 96(5):  926‐931. 3. Blumberg SJ & Julian V. Luke. (2009). Reevaluating the need for  concern regarding noncoverage bias in landline surveys. American  Journal of Public Health. 99(10): 1806‐1810. 4. Delnevo CD, Gundersen DA & Brett T. Hagman. (2008). Declining  estimated prevalence of alcohol drinking and smoking among  estimated prevalence of alcohol drinking and smoking among young adults nationally: artifacts of sample undercoverage?  American Journal of Epidemiology. 167(1): 15‐19.
  71. 71. Contact Info Contact Info Cristine Delnevo, PhD, MPH delnevo@umdnj.edu Cristine Delnevo PhD MPH delnevo@umdnj edu Daniel A. Gundersen, MA gunderda@umdnj.edu Randal ZuWallack Randal.Zuwallack@macrointernational.com Riki Conrey Frederica.Conrey@macrointernational.com

×