M. Plate - Development and Implementation of a Mixed-Mode Multipurpose Survey-Tool for Official Statistics


Published on

9th Workshop on Labour Force Survey Methodology 15-16 maggio

Published in: Education, Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Status Quo: Non-modular architecture: Serves a single intended purpose under well defined and stable environmental conditions.

    Modularity is building a complex product or process … from smaller sub-systems that can be designed independently … yet function together as a whole It is more than just defining modules!

    Modularity in 2 ideas
    1. Interdependence within and independence across modules
    2. Abstraction & Information hiding & Interface
    • Complex system broken down in smaller pieces
    • Look at them separately
    • Interface indicates how the elements interact with the larger system

    Design rules
    • Architecture
    • Interfaces
    • Integration protocols and testing standards

    Advantage of Modularization:
    Modules can be used across domains
    Combining modules provides flexibility to better meet more demands
    Enables separate development processes
    Creates loosely coupled knowledge domains

  • M. Plate - Development and Implementation of a Mixed-Mode Multipurpose Survey-Tool for Official Statistics

    1. 1. www.statistik.at We provide information Development and Implementation of a Mixed-Mode Multipurpose Survey-Tool for Official Statistics Marc Plate Social Statistics Rome 15 May 2014
    2. 2. www.statistik.at slide 2 | 28 May 2014 Development of a new Survey-Tool Status Quo: CATI and CAPI using BLAISE, Mixed-Mode: Mainly alternating modes for Panels 160 CAPI Interviewer 60 CATI Interviewer Up to 200.000 Interviews, in up to 20 Surveys per year New IT-System to reach these goals: • Enable Multipurpose Surveys • Enable different kinds of Mixed-Mode Designs • Enable new mode “CAWI” Planned first use: Household Budget Survey 2014/15 then LFS 2016
    3. 3. www.statistik.at slide 3 | 28 May 2014 Enable Multipurpose Design Vision: Modularize the statistical production. Questionnaires are composed by combining modules Official Statistics have the need to design surveys more flexibel Q1 Q2 Q4 Q5 Q6 Q7 Module 1 Module 2 Module 3 eQuestionnaire … Module-Repository CAWI CATI CAPI PAPI
    4. 4. www.statistik.at slide 4 | 28 May 2014 The Module Repository - an Example Module: Expenditures for Education and Training Submodule : Housing Expenditures connected to formal education Submodule : Expenditures for other (non formal or occupational) education Submodule: Expenditures for formal education Submodule : Expenditures for occupational training Module: Formal Education Submodule: Highest level of education attained Module: Occupational Education Submodule: Type of occupational education Submodule: Place of education Module-Owner Module-Owner
    5. 5. www.statistik.at slide 5 | 28 May 2014 Enable Mixed-Mode Design Vision: Make use of “classic” and “new” ways of data collection (CATI, PAPI, CAPI, CAWI) and supply the possibility of combining modes in a concurrent, sequential or longitudinal mixed-mode design • We need the best balance between standardized ,automated and respondent oriented case-management • We need a combination of modes to achieve highest data quality Centralized System for planning, conducting and monitoring field- phase
    6. 6. www.statistik.at slide 6 | 28 May 2014 Monitoring the Field Phase – an example Activity Performance Indicator Evaluation Criteria Corrective action Timing − median time per page/item − median time for completion per respondent − above a certain level − above or below a certain level − check wording, layout … of the item − consider removing the entry Undesired response patterns − rate of satisfycing patterns per screen/grid − rate of satisfycing patterns per respondent − above a certain level − above a certain level − check wording, layout … of screen/grid − consider removing entry Error/ prompt messaging − rate of error/prompt messages per item − rate of error/prompt messages per respondent − above a certain level − above a certain level − check wording, layout … of item − inspect answers of respondent Response corrections − rate of response corrections per item − above a certain level − check wording, layout … of the item Recruiting − structure of referals - some referals with frequency smaller than expected − check for errors, consider alternative recruiting approach Lozar-Manfreda & Belek 2013
    7. 7. www.statistik.at slide 7 | 28 May 2014 Websurvey – The Vision More and more Respondents raise the wish to answer a survey online Vision: Offer a web-questionnaire so well designed, that it works for the general population without the need of any personal contact. Websurvey On- and Offline Functionality Highest data security Webportal List of Questiontypes Innovative Plausibility Checks Innovative Explanatory notes Paradata Integration of Lookuplists
    8. 8. www.statistik.at slide 8 | 28 May 2014
    9. 9. www.statistik.at slide 9 | 28 May 2014 Multipurpose Design / Modularisation The Challenges Substantial Challenges • Reasonable grouping of questions • How to make links between modules • … Organisational Challenges • Reorganisation of responsibilities into modul-responsible, topic-responsible, questionnaire-responsible Persons. • Strong communications between departments • Module-responsible Person has to design „perfect“ questions • Project Team of one survey is not reasponsible for all questions anymore How can modularity be accomplished without loosing sight of the overall context? What methodological implication arise, when questionnaires are designed part by part?
    10. 10. www.statistik.at slide 10 | 28 May 2014 Mixed-Mode Design / CAWI The Challenges Known methodological issues of Mixed-Mode desing, particularly • Compareability between modes • Break in time series • How to get R to use web when we „only“ know adress and name • CAWI: Does loosing the interviewer mean loosing quality assurance? (Losing control over who is answering, Losing means of collecting feedback on questions) • „New“ Issue: Paradata Need for constant Evaluation of questionnaire for all modes .
    11. 11. www.statistik.at slide 11 | 28 May 2014 Please address queries to: Marc Plate Contact information: Guglgasse 13, 1110 Vienna phone: +43 (1) 71128-7972 Marc.Plate@statistik.gv.at Thank you very much for your attention