In Decatur and Beyond, It All Starts with a Conversation
UCONN Final Proposal
1. 1
RESEARCH PROPOSAL
Collecting Labor Market InformationRelatedtoUnemployment Rates
Do the characteristics of people who respond
to household or land line telephone
surveys bias results beyond what errorrate
adjustments can accommodate?
Is it time to introduce an Electronic, Web Based
Mode of Data Collection to the Mix in the
Current Population Survey?
FOR:
PP-5379-Principles and Methods of Survey Research-SECH01-1153
BY:
Sandra Rodriguez
5/1/14
2. 2
PREFACE
In preparingthis research proposal,onereality has become very apparent: “I am in way over my head” in trying to
tacklethis topic as my firstassignmentin my Survey Research studies atUCONN, havingnever taken a statistics
class,struggling to get ‘C’s in math throughout my lifeand knowing that there are scores of research professionals
“on the job” with significantmore understandingof the problem at hand. I set a very lofty goal of tryingto
influencea change in the way we collectthe data that informs unemployment rates in the country. And while
recognizingthere are “reasons”why it is the way itis,this is a mission I amnot willingto giveup. I believe that we
must re-evaluate the methodology for the collection of this information as soon as possible. Land lines are
disappearingand cell phones arelikely not a good replacement for a variety of reasons. Both household and
phone responserates arerapidly declining,and atpresent there is no web-based, electronic means of collecting
data specific to the calculation of unemployment rates collected from the Current Population Survey (CPS.)
Further, with the Baby Boomer generation embarking on their retirement years,the tech savvy,always on-the-go
Gen Y workingage population thatis replacingtheseworkers are even less likely to respond to household and land
linetelephone based surveys. This problem could mushroom faster than you think.
I have juggled my hypothesis around a bitin this process,firststartingwith measuringpreferred modes of survey
delivery to make the casethat itis time to switch to web-based means of collectingthis data. Yet, somehow I feel
this conclusion is too obvious. So instead,I have adjusted my hypothesis to get at the underlyingcharacteristicsof
what “kind of people” respond to what “types of survey modes.” My premise in this proposal is thatthe
observablecharacteristicsof peoplewillingto participatein the CPS survey via telephone and/or in-home
interview are significantly and systematically differentthan the characteristicsof those either not willingto
participatevia telephone or in-home interview or are not accessiblewith those modes. And that this significant
difference is leadingto a potentially unaccounted for non-response bias beyond what an error rate or weighting
can accommodate. Further, with the
In order to assess thecharacteristicsof people who respond, or don’t respond to modes of data collection,
multiplemodes of collection mustbe applied to the same population,with the same survey questions. As I try to
tacklethe problem at hand, I realized I am in “way over my head.” I also realized why the problem of “holdingon
to these traditional modes of collection for the CPS Survey may, to this date, not have been fully addressed by the
experts in the research community and U.S. Department of Labor. It is very complicated.
With that in mind, I hope that this proposal mightat the very leastspur some conversation and consideration of
investment in creatingthe change that is needed. The President’s 2015 Budget includes the 41 million increasefor
the Bureau of Labor Statistics,includingfundingfor improvements to the Job Openings and Labor Turnover Survey
(JOLTS), which provides information aboutlabor market conditions,wages and skillsof jobs created versus
destroyed, and employer perception of opportunities.Included is a $1.6 million supplementto the Current
Population Survey to providebetter data about worker flexibility and entrepreneurship. But I believe the
investment is only slated to address content changes within the survey i.e. addingdata on contingent work and
alternativework arrangements, workplaceflexibility and work-family balanceissues. None of the new investment
is set asidefor assessingor addressingtheways we collectthis information.
Due to my limited knowledge in this area, my proposal would need significantmoreexpertise to address this very
complicated issueand investment of time and resources. I just hope itmay spur some conversation.
3. 3
The ResearchHypothesis - Background
The monthlyunemploymentrate,announcedeachmonthbythe Bureauof Labor Statisticsisa key
indicatorof howwell the U.S.is doing. The rate isthe numberof individualsnotemployed,butwhoare
consideredtobe inthe laborforce,dividedbythe total numberof individualsinthe laborforce. The
informationgatheredtocalculate thisrate comesfromthe monthlyCurrentPopulationSurvey(CPS).
The mode of surveyforthe CPS isinpersoninterviewsandbyphone. Personal visitsare preferredin
the firstmonthof a collectionandinothermonthsthe surveyisconductedbyphone. Approximately
70% of the householdsinanygivenmonthare interviewed bytelephone.1
At the currenttime,none of thisinformationiscollectedelectronically,perhapsbasedonthe
complexityof the informationthatneedstobe gatheredaswell asthe highlevel of impactachange in
the collectionof thisinformationcanhave onglobal markets. The continuousdigital divide,while
closingisalsolikelyafactor.
The ResearchHypothesis –Testable Statements
Primary Research Hypothesis: The observable characteristicsof people whoare accessiblevia
telephoneorin-home interviewand/orare willingtoparticipate inthe CPSsurveyviatelephone
and/orin-home interview;are significantlydifferentthanthe characteristicsof those whoare
not accessible viatelephone orin-homeinterview;and/ornotwillingtoparticipate inthe CPS
Surveyviatelephoneorin-home interview. And,thatsignificant differencemaybe resulting
non-responsebiasforthe unemploymentrate collectpotentially beyondwhatanyerrorrate or
weightingcould accommodate,now orwithinthe nextfive years.
Primary Null Hypothesis: The null hypothesisisthat there isno significantdifference inthe
observable characteristicsof peoplewhoare accessible viatelephoneorin-home interview
and/orare willingtoparticipate inthe CPSsurveyviatelephoneand/orin-home interviewand
those whoare notaccessible viatelephoneorin-home interview;and/ornotwillingto
participate inthe CPSSurveyviatelephoneorin-home interview. And,thatthere isno
potential non-response bias inourunemploymentratesrelatedtothisissue,now orwithinthe
nextfive years.
Secondary Research and Null-Hypothesis
In 1994, the CPSunderwentamajorredesigninordertoimprove the qualityof the dataand to
modernize datacollectionmethods. One of the objectivesof the redesignwastoadopta computer-
assistedinterviewingenvironment.3
Much was learnedfromthatchange that couldbe appliedtoan
electronicmethodof datacollection.
1
Handbook of Methods,chapter1, labor force data derived fromthe
Current PopulationSurvey,Bureau ofLaborStatistics;
http://www.bls.gov/cps/documentation.htm#concepts
4. 4
One such surveythathas introducedthe electronicmethodof datacollectiontoitsalready“mixed
mode”of collectionviapersonal interviewsandtelephonesurveysisthe NationalSurveyof DrugUse
and Health(NSDUH) 2
Secondary Research Hypothesis: Like the NSDUH Survey, the CPS survey needs to introduce a self-
administered, electronic surveying mode to its mixed mode of collection (of personal interviews and
telephone surveys) in order to get a more accurateoutput of the nation’s unemployment rate.
Secondary Null-Hypothesis: The introduction of a self-administered electronic survey mode to the
collection of CPS data will notprovide a more accurateoutput of the nation’s unemployment rate.
Background – Literature Review
DecliningResponse RatesUsing Traditional Survey Modes - Most of today’slaborforce isnot willingto
a) give upa significantamountof theirtime
to complete longandrepetitive surveysand
b) if theyare so willing,are significantlyless
likelytodoso viatelephone. Further,
fewerpeoplehave householdlandline
telephones. OursurveyMethodology
textbook,statesthatlandline coverage in
2008 declinedto80%.3
PewResearchhasa
more timelyassessmentonthistopic
indicatingland-line telephoneresponse
rateshave declinedtoaslowas 9% in
2012.4
One mightthinkcell phonesare a goodwayto combat the decliningresponse rates,yetforvarious
reasons,PewResearchhasalsonotedthatresponse ratesare actuallytypicallylowerforcellphone
surveysthanfor landlinesurveys.5
2 SurveyMethodology, Groves, Fowler, Couper, etc. Pg. 150
3 SurveyMethodology2nd Edition. Robert M. Groves, Floyd F. Fowler, Jr., Mick P. Couper, James M. Lepkowski, Eleanor Singer
and Roger Tourangeau;John WileyandSons, 2009.
4 Pew Research, MA Y 1 5 , 2 0 1 2 Assessing the Representativeness of Public Opinion Surveys
http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/
5 Pew Research, Collecting SurveyData; http://www.pewresearch.org/methodology/u-s-survey-research/collecting-survey-
data/ 10th Paragraph.
5. 5
Complicationsfrom TechnologyEvolution - So off to the internetwe go?Notso easy. Couper,2000
states,“no goodframe has beendevelopedforsamplingthe Internetpopulation.”6
The Pew Research
Centeradds:
“There also is no systematic way to collect a traditional probability sample of the general population using the
internet. There is no national list of email addresses from which people could be sampled, and there is no
standard convention for email addresses, as there is for phone numbers, that would allow random sampling.
Internet surveys of the general public must thus first contact people by another method, such as through the
mail or by phone, and ask them to complete the survey online.”7
In the spiritof addressingthe needtocapitalize onthe internetforcollectinginformation,inthe fall of
2011 the AAPORExecutive Councilappointedataskforce “to examine the conditionsunderwhich
varioussurveydesignsthatdonotuse probabilitysamplesmightstillbe useful formakinginferencesto
a largerpopulation.”8
Further,there isstill the issue of the “digital divides.”Lee Rainie,Directorof the Pew ResearchCenter’s
InternetProject,presentedthe project’slatestfindingsaboutwhohasanddoesn’thave accessto the
internet,broadband,andcell phones.He notedthatsome of the factorsassociatedwithnon-use of
technologyare age,householdincome, educational attainment,communitytype, anddisability.9
Yet,whencollectingunemploymentdata – the complicationsMUSTbe workedthrough. The Pew
InternetProjectReportforMay 2008 stated90% of persons18-25 yearsoldwere online. Sevenyears
laterin2015, these same individuals area25-32, andin 5 years(2020) they will be 30-37 and a
significantportionof the laborforce. Accordingtothe U.S. Bureauof Labor Statistics,by2020, nearly
40%-50% of the U.S. workforce will consistof millennials (the generationof people bornbetween1985-
2000 whowill be 20 – 35 in2020.)10
Pointisprettysoonwe will notbe reachingthe bulkof the labor
force withthe traditional methodscurrentlybeingused. Itmayalreadybe happening. While Idon’t
currentlyhave accessto data on those whowill be atthe higherendof the laborforce age spectrum,I
suspectthere isalsoa significantincreaseinthose usingthe webanddigital devicesdespite the noted
digital divide.
6 Web Surveys: A Review of Issues and Approaches Public OpinionQuarterly, Mick P. Couper, 2000
http://www.uni-frankfurt.de/47929851/Couper_2000.pdf
7 Pew Research, Collecting SurveyData, http://www.pewresearch.org/methodology/u-s-survey-research/collecting-survey-
data/ 10th Paragraph.
8 REPORT OF THE AAPOR TASK FORCE ON NON-PROBABILITY SAMPLING, June 2013.
https://www.aapor.org/AAPORKentico/AAPOR_Main/media/MainSiteFiles/NPS_TF_Report_Final_7_revised_FNL_6_22_13.pdf
9 Pew Research Center, November 5, 2013, The State of Digital Divides.
10 More Firms Bowto GenerationY's Demands - WallStreet Journal, ByLeslie Kwoh; UpdatedAug. 22, 2012
http://www.wsj.com/articles/SB10000872396390443713704577603302382190374 and numerous other media outlets
extracting fromthe Bureauof Labor Statistics – Employment Outlook 2010 – 2020;by Mitra Toosi
http://www.bls.gov/opub/mlr/2012/01/art3full.pdf
6. 6
For the bulkof this researchproposal,the Bureauof LaborStatistics’SurveyMethodology
documentation - Handbookof Methods,chapter1, laborforce data derivedfromthe CurrentPopulation
Surveywasreferredto. 11
Descriptionof the survey questions
The surveyis dividedintothree questiontypes:
1. EligibilityQuestions
2. SurveyMode Accessand Preference Related Questions
3. Observable Characteristics
Name and Eligibility Questions–Survey Sections1 and 2
Question 1 – Name
Thisquestionisasked onlytocross checkfor duplicationbetweenmodes.
Question 2.1 – Residency
Thisquestionisaskedtoensure residencyinthe targetedpopulationarea.
Question 2.2 – Age
Thisquestion isaskedtoensure aqualifiedadultrespondentiscompletingthe survey.We wantto
ensure the respondent’sage doesnotdisqualifythemfrompotentiallybeingpartof the labor force. By
Bureauof Labor Statisticsdefinitions,the laborforce isthe following:"Includedare persons16yearsof
age andolderresidinginthe 50 Statesand the Districtof Columbiawhoare notinmatesof institutions
(forexample,penalandmental facilities,homesforthe aged),andwhoare not on active dutyinthe
ArmedForces."12
Empirically,thisquestionisacertifiedquestionfromthe SurveyMonkeyQuestionBank13
withthe
exceptionof one modificationthatIam hopingisnot substantive andsubjecttobias – the firstanswer
inthe certifiedquestionwas“17 or younger.”I revisedthisone answerintotwoparts,1) youngerthan
16 and 2) 16-17. Those whorespond16-17 are potentiallyeligible forthe laborforce,butas minors,
may require parental orguardianapproval toanswerthe questionnaire. Those answeringwhoare
youngerthan16, are simplynoteligibleforeitherthe surveyorthe laborforce.
11
Handbook ofMethods, chapter 1, labor force data derivedfrom the Current Population Survey, Bureauof Labo r Statistics;
http://www.bls.gov/cps/documentation.htm#concepts
12 BLS GlossaryRetrieved 15 October 2014.
13 The SurveyMonkeyQuestionBank is a source ofthousands of pre-writtenquestions deemedmethodologicallysound and
are “certified” andcreatedbytheir methodologists. https://surveymonkey.com/mp/certified-survey-questions/
7. 7
Questions 2.3 and 2.4 – Employment Status
Question2.4…isthe keyquestionthatthisresearchwill eventuallygetto,once the ideal modesof
collectionare furtheranalyzedandchangesmade if necessary. The questionreferstothe self-
classificationof one’semploymentstatus. Thisisone of the twoquestionsthatI appliedinthe cognitive
testingphase,andwasmodifiedasaresultof learningsfromthatcognitive pretest. The questionand
the wordingof itscorrespondinganswershave noprecedentthatI am aware of.
Question2.3 isa questionIaddedinat the lastminute. SimilartoQuestion2.4,thiswasthe “certified”
SurveyMonkeyQuestionBankquestionthatself-classifiesone’semploymentstatus. Ididnotwant to
“settle”forthisquestion thoughasitdoesn’tansweralot about the relationshipof one’semployment
statusto one’semployment“needs.” Inotherwords,someone maybe workingpart-time bychoice or
because theycan’tlanda full time jobthatthey“need.” Ortheymay be workingmultipleparttime jobs
withno financial needtolanda singularfull time job. Further,manywhohave beenforcedintoearly
retirementwishtheyhadajoband olderworkerswhowere laidoff maygoback and forthbetween
periodsof lookingforajob andacceptingretirement. Hence,Ihad chosentotry to craft the further
qualifyingQuestion2.4.
My guessisthat analyzingthe resultsof bothQuestions2.3and 2.4 couldhave some interesting
findings.
Questions 2.5, 2.6, 2.7 and 2.8
… are deriveddirectlyfromthe CPSSurveytohelpqualifyornegate the response tothe firstquestion –
howpeople self-classifytheirunemploymentstatus. [SOURCE:BLS,Handbookof Methods] For
example,if inQuestion2.5,the individual respondsyesto“lastweek,didyoudoanywork for(either)
pay (orprofit),butanswers“NotEmployed”inQuestion2.3,there wouldbe amismatchof answersand
the respondentsanswersshouldbe discarded.
Survey ModeAccess and Preference Questions – Survey Section 3
Questions 3.1- Mode Access
Thisquestionwasdesigned toidentifywhattechnologytools –bothhardware and software
respondentssimplyhave accessto. Ina larger survey,itwouldmake sense totryto addquestionsthat
helpresearchersunderstandwhyorwhynot,an individual hasaccesstothese technologies.
Questions 3.2 - 3.6
… get directlytothe heartof ‘preferredmode’forsurveyparticipation. Isuspectatleastone of the
authorsof our textbook,MickCouperhas provenquestionsinthisarea,Iam notaware of themat this
time. Couperdoes note,thatcoverage errorrepresentsthe biggestthreattothe representativenessof
sample surveysconductedviathe Internet.14
Ihope toresearchthisfurtherconcernfurtheras well as
any successful questionsinthisareaas we buildoutour researchproposals.
14 Web Surveys:A Review of Issues andApproaches http://www.uni-frankfurt.de/47929851/Couper_2000.pdf
8. 8
Empirically,Question3.3inthissectionwasincludedinthe cognitivetestingphase andthiswasthe key
questionIwascuriousaboutin termsof how well participantsunderstoodthe questionandif itwas
more or lessstraightforward. Instead,Idiscovereditwasactuallythe EmploymentStatusquestion
(Question 2.4),onthe self-classificationof one’semploymentstatusthatwasfraught withissues. There
were twosubstantive changesmade tothisQuestion3.3none the less.
One was to be clearthat the Land Line referredtowas“home based”as one respondent
inquiredif a“work-based”landline wouldbe considered.
The secondchange was made to gathermore information –if someone doesnothave a
“landline”isit a resultof “choice”or “financial constraints.”
Historically,if someone didnothave a“home phone”itwas due to financial constraints,butbecause
cell servicesisnowsoprevalent,manyfinanciallysecureindividualsare notseeinganeedfora home
basedlandline. This altersthe demographicclassificationsassignedtothose whodonot have a home
basedlandline andshouldbe takenintoconsiderationwhenassigningerrorrates.
Question 3.7 – Likelihood of Use
While Question3.1isdesignedtoanswerthe “AccessQuestion,”thisquestiontriestogetat the
likelihoodof use of the variousdigital technologies,andplace of use. The answerstothisquestionwill
furtherhelpthe researcherdeterminebehavioral andlifestylepatternsthatwill lead,ornotleadto
considerationof technological access.
ObservableCharacteristicsQuestions – Survey Section 4
Question 4.1 - # of Children
…is askedtoprovide some additionaldemographictraitsof the individual completingthe survey.The
response isnota qualifierinanyway.It simplyprovidesadditional information. Itisalso a certified
questionfromthe SurveyMonkeyQuestionBank.
Questions 4.2 – Education Level
As mentionedearlier, Lee Rainie,Directorof the Pew ResearchCenter’sInternetProject,notedthat
some of the factorsassociatedwithnon-use of technologyare age,householdincome, educational
attainment,communitytype, anddisability.Question4isintendedtoidentifythe characteristicof
educationlevelof the individualcompletingthe survey. Thismayprovide some correlationbetween
educationlevelandpreferredmode of surveyparticipation. Thisquestionis fromthe SurveyMonkey
CertifiedQuestionBank.
Questions 4.3 – 5-Year Vision
Thisquestionisintendedtoassessthe likelihoodof the respondentremainingin,orbecomingpartof
the labor force inthe next5 years. If there isany attemptat adjustingthe methodsof datacollection
for the CPSsurvey(and otherkeyindicatorsurveys),itisimportanttobe forwardprojectinginthat
processratherthan reflectiveof the pastand currentscenariosonly.
9. 9
Questions 4.4 – Occupation Identifiers
Thischaracteristicrelatedquestionispartof the surveytohelpunderstandif there isanycorrelation
betweenoccupational preferenceswiththe accessorwillingnesstoparticipate insurveysbasedon
surveymode.Forexample,aparticipantwhoselectsComputerandMathematical Occupationsmaybe
more inclinedtopreferanelectronicbasedsurveymode whereassomeone whoselectsFood
Preparationashisor her desiredoccupationmaybe more likelynottohave electronicaccessto
complete sucha survey. Thisquestionisfromthe SurveyMonkeyCertifiedQuestionBank withthe
exceptionof addinginthe words“orfuture”to the questionandaddinginan “Other”category. The
firstchange was made to capture accurate responsesfromstudentsand/orthose incareertransition.
The secondchange was made as a catchall,for respondentswhodonotfeel theiroccupationis
adequatelyrepresentedonthe list.
Open Ended CommentsArea - Survey Section 5
Question 5.5 – Open Ended Comments Area
Notsure howtypical itisto have this,but I have alwaysfoundanopenended,non-directional
commentsareato be useful forthose whohave somethingtosaythat mayor maynot assistinthe
researchprocess.
10. 10
Sample Design
Populationof Study and its Importance
The populationforthisstudyisthe U.S. civiliannon-institutionalizedlaborforce population,aswell as
the institutionalizednonlaborforce populationdefinedasstudentslivingoncampus or individuals
servinginthe military asthese populationsintendtoandare likelytobecome partof the laborforce
withinfive years. Therefore,the core characteristicsof the populationwill be:
16 yearsof age or over(some maybe 15, since the AmericanCommunitySurveyage groupis
15-19, but the 15 year oldswill be screenedout)
Individualsidentifiedasinthe Labor Force:
o Employedindividuals;asdefinedbythe CurrentPopulationSurvey(CPS);see BLS
Descriptionof employed; http://www.bls.gov/cps/cps_htgm.htm
o Unemployedindividuals;asdefinedbythe CurrentPopulationSurvey(CPS);see BLS
Descriptionof unemployed);http://www.bls.gov/cps/cps_htgm.htm
Individuals notinthe laborforce as definedascollege orotherwise campus-basedstudentsand
those currently servinginthe military
Like the largerCPS Surveythatcollectsdatausedto calculate unemploymentrates,the populationwill
alsoinclude self-employedpersons,privatehouseholdworkers,unpaidfamilyworkersandworkerswho
are on leave withoutpayat the time of the survey.[BLSEmploymentMeasures - MonthlyLaborReview,
Feb2006]
The populationwill NOTinclude:
institutionalizedpopulations;otherthancampus-basedstudentsorthose servinginthe
military.
any otherindividualsconsidered“notinthe laborforce.” For example,thoseretired, innursing
homes,orin prison or those whoare neitherworkingnorseekingwork. will notbe considered.
Whyinclude the U.S.civiliannon-institutionalizedlaborforce population?
The labor force isdefinedbythe U.S.Departmentof Labor’sBureauof Labor Statisticsasindividuals
whoare eitheremployed orunemployed. Inordertogauge preferred the observable characteristicsof
individualsdeterminedtobe employedor unemployed asmeasuredbythe CPSSurvey,the population
shouldrepresentthe membersof the laborforce.
Employed– Employed persons,consistof:personswhodidanyworkforpay or profit
duringthe surveyreference week;personswhodidatleast15 hoursof unpaidworkin a
family-operatedenterprise;andpersonswhowere temporarilyabsentfromtheir
regularjobsbecause of illness,vacation,badweather,industrial dispute,orvarious
personal reasons.
11. 11
Unemployed–Personsare classifiedasunemployedif they donothave a job,have
activelylookedforworkinthe prior4 weeks,andare currentlyavailable forwork.
Personswhowere notworkingandwere waitingtobe recalledtoa jobfromwhichthey
had beentemporarilylaidoff are alsoincludedasunemployed.Receivingbenefitsfrom
the UnemploymentInsurance(UI) programhasno bearingonwhetherapersonis
classifiedasunemployed. 15
Further,forthe purposeof measuring employmentrates,theBureau of LaborStatistics
measuresonly thoseindividualswho areage 16 or older.
Whyinclude campus-basedstudentsand those servingin the military?
The populationshouldmirrorthe populationthatiscurrentlyreferredtowhencollecting
unemploymentdatawiththe exceptionof college students. Accordingtothe U.S. Bureauof Labor
Statistics,by2020, 50% of ourLabor Force will be millennials. A significantpercentageof this
populationiscurrentlyincollege orinthe military,andtheyare expectedtosoongraduate orbe
releasedfromactive duty. Responses fromthepercentageof thispopulation stillin High Schoolare
collected already as they are likely living athome.
One thingthe CPS Survey doesnotcalculate isa “forwardprojection”of those expectedto membersof
the labor force inthe near future; inotherwords,individualswhowill eitherbe employedor
unemployedinthe nextfive years. If the methodof collectionisgoingtobe altered,inthiscritical
surveythathas significantimpactonthe worldeconomy, Iamproposingtoadd samplingframesof
college orothereducational studentswhoare livingoncampus withthe surveyperiodduringthe school
year(Sept– May); These individualsare currentlyconsideredin groupquartersandnot part of the labor
force,yettheirresponse isvitally importantinansweringthe questionof preferredsurveymode,as
theyhave a highlikelihoodof enteringintothe laborforce withinthe nextfive years.
In addition,if itisat all possible togainrecordsfromthe VeteransAdministrationof individuaIswho
reside inthe samplingframe areaswhoare currentlyservinginthe military,Iam proposingtoadd
samplingframesof these individualsaswell astheyhave a highlikelihoodof enteringintothe labor
force withinthe nextfive years.
15 Bureau of Labor Statistics, http://www.bls.gov/cps/lfcharacteristics.htm
12. 12
Drawing a Sample from this Population
Sample Design: Multi-StageStratifiedClusteredAreaProbability andMultiplicityNon-Probability
Sampling;Drawninsuccessive stage torepresentall the eligiblepersonsinthe UnitedStates.
The sample size should be similar to that used in the United States Census Bureau’s American Communities
Survey of 36,000 households per year:
The full implementation of the ACS, which began in 2005, sampled approximately 2.9 million housing unit
addresses annually stateside. The PRCS sampled approximately 36,000 housing unit addresses each year in
Puerto Rico. The 2011 ACS sampled approximately 3.3 million housing unit addresses - this corresponds to
an increase in the targeted annual sample size of 3.54 million addresses that began with the June 2011
Research Objective:
IdentifyingPreferredModesof SurveyDeliveryinCollection of Unemployment Data
Mode of Collection:
Telephone RDD - CATI
Sampling Frame:
List assisted random digit sample;
individualrandomly selected if 16+
Mode of Collection:
Cell Phone RDD -CATI
Sampling Frame:
List assisted random digit cell
phone series sample
unduplicate
Mode of Collection:
Household
Sampling Frame:
Households enumerated through
blocks and listed addresses;
individualrandomlyselectedif 16+;
one per household
Mode of Collection:
Web – Online Surveys
Sampling Frame:
Public Workforce System
Customers; Chambers of
Commerce Emails; On Campus
Students
unduplicate
Strata
33.3% Phone
SubStrata
16.7%%
Telephone RDD
16.6% Cell Phone
RDD
_____
33.3% Household
SubStrata:
16.6% Mailed
16.7% Visited
___
33.3% Web
SubStrata
11.1% Public
Workforce
System
11.1% Chamber
members
11.1% On Campus
Students
13. 13
ACS sample. This increase in the targeted annual sample size has continued since then, resulting in a 2013
ACS sample of approximately 3.54 million housing unit addresses.
The full implementation of the ACS and PRCS Group Quarters data collection began in 2006.There are
approximately 200,000 persons in the group quarters sample each calendar year in the ACS.
The ACS sampled between 740,000 and 900,000 housing unit addresses annually in 2000 through 2004.16
For the purpose of thisreport,I am suggestingthe researchshouldbe conductedinpilotformfirst,ina
single municipality. Itshouldbe designedinsuchaway soas to be scaledup as costs allow.
Sample Size
N = 3,540,000 (# of housingunitaddressesusedinthe ACSSurvey)
n =
p% = 50% (persuggestedsamplepercentage–for worstcase scenario – minimumrequiredsample size
islargest
Margin of Error e%= +/- 3% = 1.96% S. E. (acceptinga largermarginof errorthan average as thisisa
newapproachand in orderto recognize thispopulationisconstantlychanging,non-responselevelsare
continuallyincreasingandthere maybe lessconfidence inthe surveyoutcomes.)
n’ = 1.962
(3.8146) 50% (100-50%) / 42
n’ = 3.81462500/16 = 156.25
n’ = 600
We can oversampleto attain a larger samplesize.
RevisedTarget Populationfor the Purposesof this Report – Survey of NewBritain, CT Population
The populationforthisreportisthe U.S. civiliannon-institutionalizedlaborforce populationinNew
Britainas identifiedbythe AmericanCommunitySurvey,2013 ACS PopulationEstimate,Usingthe U.S.
CensusBureau’sAmericanFactFinder,aswell asthe non-institutionalized,non-laborforce population
definedasstudentslivingoncampusor individualsservingin the military. Therefore,the characteristics
of the populationwill be:
Individualsresidinginthe Cityof New Britain,CTwhoare 16 yearsof age or over(some maybe
15, since the AmericanCommunitySurveyage groupis15-19, butthe 15 yearoldswill be
screenedoutastheyare not currentlymeasuredinthe CPSSurveyaspart of the Labor Force).
Individualsidentifiedaspartof the laborforce:
o Employedindividuals;asdefinedbythe CurrentPopulationSurvey(CPS)
16
http://www.census.gov/acs/www/methodology/sample_size_definitions/index.php
14. 14
o Unemployedindividuals;as definedbythe CurrentPopulationSurvey(CPS)
Defined and sourced earlier in this document
Individuals notinthe laborforce as definedascollege orotherwise campus-basedstudentsand
those currentlyservinginthe military
Drawing a Sample from the RevisedPopulation
Sample Design: Multi-Stage, Multi-Mode, StratifiedClusteredAreaProbability andNon-Probability
Sampling:
Sample will be conductedin three stages based on mode of collection:
1. Phone
2. Household
3. Web
Strata and Clusters:
Stage I – Phone Surveys(2strata)
HouseholdTelephone List–assistedRandomDigitDial
Cell Phone –RandomDigitDial takenfrom areadesignatedcell phone numberseries
Duplicationwillbe adjustedbyassociatingapersonwithone landline telephone numberandscreening
out the cell-phoneparticipantif he orshe is identifiedasa duplicate..
Stage II – HouseholdSurvey(1stratum)
HouseholdFace toFace InterviewsSRSof listedhouseholds
Duplicationwillbe adjustedbydeterminingif individual previouslyansweredsurveyviatelephone and
screeningoutif the answerisyes.
Stage III – Web Surveys (3strata)
WebSurvey – of individualsvisitingthe PublicWorkforce System(highpercentagetheywill be
inthe laborforce,andthere will alsobe a portion of militaryVeteransinthispopulation)
Websurvey – deliveredtomembersof the New BritainChamberof Commerce
WebSurveyof potential on-campusstudents –intention isworkwithcampuspersonnel to
obtainstudentemail addresses
Sample Size
Accordingto the AmericanCommunitySurvey,2013 ACSPopulationEstimate,Usingthe U.S.Census
Bureau’sAmericanFactFinder,NewBritainhastotal populationof 73,206.17
When youback out those
17 U. S Census – American Fact Finder http://factfinder.census.gov/faces/nav/jsf/pages/index.xhtml
15. 15
not yet15 (15 yearoldswill be deletedafterwards),the civilianlaborforce inNew Britainis58,993.
Whenyouadd in the non-institutionalizedindividualsinGroupQuarters(thatincludeson-campus
college students) totaling2,404, the total populationof the sample is61,397. See extractfromthe Fact
Findertool below:
Subject Number Percent
SEX AND AGE
Total population 73,206 100.0
Under 5 years 5,043 6.9
5 to 9 years 4,589 6.3
10 to 14 years 4,581 6.3
15 to 19 years 6,026 8.2
20 to 24 years 7,320 10.0
25 to 29 years 6,086 8.3
30 to 34 years 5,363 7.3
35 to 39 years 4,368 6.0
40 to 44 years 4,349 5.9
45 to 49 years 4,609 6.3
50 to 54 years 4,700 6.4
55 to 59 years 4,180 5.7
60 to 64 years 3,260 4.5
65 to 69 years 2,175 3.0
70 to 74 years 1,622 2.2
75 to 79 years 1,469 2.0
80 to 84 years 1,577 2.2
85 years and over 1,889 2.6
In group quarters 3,194 4.4
Institutionalized population 790 1.1
Male 279 0.4
Female 511 0.7
Noninstitutionalized population 2,404 3.3
Male 1,285 1.8
Female 1,119 1.5
U. S Census – American Fact Finder
16. 16
HouseholdSampleSize
N = 61,397
Margin of Error +/- 3% 1.96 standard errorpercentage
To determine the samplesize forthissub-populationof 20,445 we coulduse the formulaforminimum
sample size togeneralizethe population. Foran infinitepopulationthatwouldbe:
Thisis a finite population. Therefore,Iwouldsuggest:
N = 61,397
n =
p% = 50% (persuggestedsamplepercentage–for worstcase scenario – minimumrequiredsample size
islargest
Margin of Error e%= +/- 3% = 1.96% S. E. (average)
n’ =N n’ /N +n’
3.8146 * 2,500/9 (1,067)
61,397*1,067 / 61,397+1067 – 1,048.74 (we will roundupto1,500)
Confidence % -=95%
So witha sample size of 1,500 and eachStage of the studyrepresenting1/3of the total sample frame,I
wouldrecommendthe following:
Stage I – Phone Surveys(2strata) - 500
HouseholdTelephone List – 250
Cell Phone –RandomDigitDial - 250
Duplicationwillbe adjustedbyassociatingapersonwithone numberandscreeningoutthe other.
Stage II – HouseholdSurvey(2strata) - 500
HouseholdFace toFace InterviewsSRSof listedhouseholds
HouseholdMailings
Usingthe RANDformula,250 of the householdswillbe mailedthe survey. Forthe remaining
households,250 householdsshouldbe interviewed.
17. 17
Duplicationwillbe adjustedbydeterminingif individual previouslyansweredsurveyviatelephone and
screeningoutif the answerisyes.
Stage III – Web Surveys(3strata) - 500
WebSurvey – of individualsvisitingthe PublicWorkforce System - 166
Websurvey – deliveredtomembersof the New BritainChamberof Commerce - 167
WebSurveyof potential on-campusstudents –intention isworkwithcampuspersonnel to
obtainstudentemail addresses - 167
ExpectedResults
Overarching Expectations
Basedon whatI have learnedfromthe literature,andwhatIhave beenable togarner in ina veryshort
periodof time,withlimitedexposure tothe labormarketunemploymentrate collectioninformation
priorto thisclass,I am veryhumble inwhatIwouldexpectintermsof reliable outcomes. Surelythe
expertsare “chippingaway”at thisissue withgreaterexpertise andagreaterunderstandingof howto
conduct surveyswithmeaningful outcomes.
All thatbeingsaidI wouldstill expectsome confirmationformyoverall premiseandresearch
hypotheses. AndIwouldexpectthe researchcommunitytochallenge thiswork. Theyshould. The
CurrentPopulationSurvey(CPS) isone of the mostrespected,reliedonsurveysinthe industry. Many
othersurveysuse the resultsfromthissurveyasthe basisfor furtherstudy.
There isa reasonwhythe researchcommunityhasnotcome upwitha true random, probability
approach to collectingdatarelatingtounemploymentstatusthatemployson-line,internetbased
collectionof information.Infactthere are likelymanyreasons –all relatedtothe complicatednature of
accessingcomplete liststhatrepresentthe total population,tothe inherentcoverage errorthatcan
resultforthose withoutdigital accessandtoall the othercombinedsourcesof errorinthismulti-mode
approach.
Yet,if we thinkthe nonresponsecliff we are experiencingissevere,if we holdontotraditional modesof
informationcollectionforsurveysof ALLtypesincludingthe CPS, Ifear“we ain’tseennothingyet –no
response isjustaroundthe corner.” I am determinedtocontinue tomake the case for findingawayto
collectat leasta stratumof thisinformationelectronically.
Specific,NewBritain Study Expectations
Numberof expectedcompletes:
Perour coursework,the average completionrate is30% (Session10– PrinciplesandMethodsof Survey
Research) thoughthe “mode”wasnot identified. Pew Research,Telephone response rateshave
declinedtoaslowas 9% in 2012, and I suspecttheyhave dippedevenlowersince. Thatsame study
18. 18
indicatesboththe wealthyandlowincome householdsyieldsimilarresponseratesinthe 20% - 25%
range. 18
While Iwoulduse the AAPORResponse Rate Calculatortodetermine responserates,Iwouldroughly
estimate completionratesasfollows:
Stage I – Phone Surveys(2strata) – 500 6% (500) = 30
Stage II – HouseholdSurvey(2strata) – 500 20% (500) = 100
Stage III – Web Surveys(3strata) – 500 – I expectthisto be significantlyhigherat40% (500) = 200
Total expectedcompletionrate –330
18 Pew Research, MA Y 1 5 , 2 0 1 2
Assessing the Representativeness of Public Opinion Surveys
http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/
19. 19
ExpectedResults –Final Words
My largergoal of tryingtomake the case for includingarepresentative samplinginthe national CPS
survey,while ontarget,isloftyforsomeone withmylimitedknowledge andshortfallinmathexpertise.
I actuallymetwiththe Asst.Directorof Labor MarketInformationatour CT Dept.of Labor (a UCONN
grad) before completingthisassignmentandinterviewedhimonthe challenge Iamtryingto take on.
He agreesthisisa worthycause,but suggestedthatforthe purpose of thispaperI shouldscale backmy
researchhypothesistoasimplerone tosurveyi.e.justsurveyingafinite population of youngandold
individuals,usingasingle collectionmode andmeasuringcertaincharacteristicsforeachto determineif
the characteristicsof oldand young,andhence theirpreferredmode of surveyswere different.
I was excitedbythisprospect,of asimplerstudy,butdecidedtotake onthe more comprehensive
approach. I recognize Imayhave missedthe markinsome areas,and that to answerthese questions
completely,itwouldtake someone withmore knowledgeandpractical experiencethanme,butI hope I
am headinginthe rightdirection.
The InvestmentisThere, But Shouldbe Directedto Evolve the Survey Mode for Data
Collection
In February2015, the Obama Administrationreleaseditsproposed2016 Budget,andincludedin
that proposedbudgetwasanallocationof $41 millionforthe Bureauof Labor Statistics,that
wouldinclude fundingforimprovementstoitsJobOpeningsandLaborTurnoverSurvey(JOLTS)
survey. The fundingalsosupportsasupplementtothe CurrentPopulationSurvey(CPS). 19
However,Idon’tbelieve thatfundingistakingonthe “elephantinthe room(nopolitical puns
intended),”whichistobetterevaluate andstarttomake changesin the modesof survey
deliveryforthe CPS.
It’s Time to Flipthe Switch
As a workforce professionals,we relyonthe unemploymentrate asan indicatoras to whether
or not we are helpingpeople securefinanciallysustainable employment. Whenthe
unemploymentrate goesdown,itimpliesthereislessworktodo.Yet,I don’tbelieve we are
trulygettingat the root of the true unemploymentquestionwhenwe are not“meetingthe
people where theyare”viaelectronictechnologysurveyoptions. Asmentionedearlier,over
90% of the current and future workforce (millennials) are online. Furthermore than80% of
Fortune 500 companiesrequire onlinejobapplications,andnational chainslike FootLocker
don’tevenallowpotential employeestoapplyinperson. 20
It’stime toflipthe switch,and
consideraddinginanelectroniccomponentof informationgatheringinthe computationof the
nation’sunemploymentrate.
19 Workforce Data QualityCampaign;Budget Proposal Aggressive on Date byRachel Zinn;2/2/15
http://www.workforcedqc.org/news/budget-proposal-aggressive-data-0
20
MASHABLE: The DigitalDivide Is Still LeavingAmericans Behind, byJessica Goodman, August 18, 2013
