SlideShare a Scribd company logo
Fred Oswald Rice University – Department of Psychology Jeff Stanton Syracuse University – School of Information Studies Kauffman Roundtable on Establishment Surveys August 9, 2011 Reducing Response Burden
Reducing Response Burden Goals Reduce time of administration and associated costs Increase response rates, compliance, fewer missing data,  Decrease survey fatigue and increase willingness to re-engage over time Effects Increase information yield per unit of response time/effort Balance multiple stakeholder survey goals based on “lean” and less reliable information sources
Approaches and Constraints  Goal: No practical loss in reliability or validity at the desired aggregate level (unit, sub-unit/cluster) Approaches Reduce instructions/explanatory material Reduce item redundancy Distribute subsets of items strategically across units, using available data or imputation to complete analyses  Automate field completion with NLP and scrapers Constraints Each approach has intrinsic weaknesses Stakeholders wedded to particular items Longitudinal comparability limits changes
Research on Response Burden Example: How to determine efficient item assignments to sub-samples, given available item information, precision goals, and stakeholder concerns? Given different approaches (e.g., multiple imputation) what is the maximum reduction in item content before loss of precision becomes intolerable? How redundant are components of a profile (so certain items ‘drive’ a profile)? What are stakeholder reactions to various imputation/analysis strategies? Our Expertise Methods generalists who examine technology and statistical tools to inform practical goals of survey development and test characteristics Statistical tools: Item response theory, multi-level and cross-classified models, meta-analysis, machine learning Technology tools: Web-based surveys, data scrapers, alternative data collection methods
Selected Papers Non-response issues Converse, P. D., Wolfe, E. W., Huang, X., & Oswald, F. L. (2008). Response rates for mixed-mode surveys using mail and email/web. American Journal of Evaluation, 29, 99-107. Rogelberg, S. G, & Stanton, J. M. (2007). Understanding and dealing with organizational survey nonresponse. Organizational Research Methods, 10, 195-209. Wolfe, E. W., Converse, P. D., & Oswald, F. L. (2008). Item-level non-response rates in an attitudinal survey of teachers delivered via mail and web. Journal of Computer-Mediated Communication, 14, 35-66. Shortening measures Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006). The Mini-IPIP scales: Tiny-yet-effective measures of the Big Five factors of personality. Psychological Assessment, 18, 192-203.  Russell, S.R., Spitzmüller, C., Lin, L.F., Stanton, J.M., Smith, P.C., & Ironson, G. H. (2004). Shorter can also be better: The abridged Job in General Measure. Educational and Psychological Measurement, 64 (5), 878-893. Stanton, J. M., Sinar, E. F., Balzer, W. K., & Smith, P. C. (2002). Issues and strategies for reducing the length of self-report scales. Personnel Psychology, 55 (1), 167-193. Stanton, J. M. (2000). Empirical distributions of correlations as a tool for scale reduction. Behavior Research Methods, Instruments, and Computers, 32, 403-406. Technology issues Rogelberg, S. G., Church, A. H., Waclawski, J., & Stanton, J. M. (2002). Organizational Survey Research: Overview, the Internet/intranet and present practices of concern. In S. G. Rogelberg (Ed.), Handbook of Research Methods in Industrial and Organizational Psychology. Oxford: Blackwell. Stanton, J. M., & Rogelberg, S. G. (2001). Using Internet/Intranet Web Pages to Collect Organizational Research Data. Organizational Research Methods, 4, 199-216. Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel Psychology, 51, 709-725. Parallel forms (item banking, test security) Oswald, F. L., Friede, A. J., Schmitt, N., Kim, B. K., & Ramsay, L. J. (2005). Extending a practical method for developing alternate test forms using independent sets of items. Organizational Research Methods, 8, 149-164.

More Related Content

What's hot

Web Assisted Assessment of Professional Behaviour
Web Assisted Assessment of Professional BehaviourWeb Assisted Assessment of Professional Behaviour
Web Assisted Assessment of Professional Behaviour
dentaladditions
 
Quantitative data analysis - Attitudes Towards Research
Quantitative data analysis - Attitudes Towards ResearchQuantitative data analysis - Attitudes Towards Research
Quantitative data analysis - Attitudes Towards Research
Lee Cox
 
Coursework Evaluation
Coursework   EvaluationCoursework   Evaluation
Coursework Evaluation
Andy Knill
 
Comparative and Non-Comparative
Comparative and Non-ComparativeComparative and Non-Comparative
Comparative and Non-Comparative
u068692
 
Survey research
Survey researchSurvey research
Survey research
shakirhina
 
Basics of Systematic Review and Meta-analysis: Part 1
Basics of Systematic Review and Meta-analysis: Part 1Basics of Systematic Review and Meta-analysis: Part 1
Basics of Systematic Review and Meta-analysis: Part 1
Rizwan S A
 
Research
ResearchResearch
Research
Jahanzeb Jahan
 
Ch05 instrumentation
Ch05 instrumentationCh05 instrumentation
Ch05 instrumentation
Cristina Merilles
 
Research design and methodology infosheet
Research design and methodology infosheetResearch design and methodology infosheet
Research design and methodology infosheet
jeanrummy
 
Data collection
Data collectionData collection
Data collection
Dr NEETHU ASOKAN
 
Action Research Pd
Action Research PdAction Research Pd
Action Research Pd
gueste9e62b
 
Mary ch2 presentation
Mary ch2 presentationMary ch2 presentation
Mary ch2 presentation
jhtrespa
 
Challenges and opportunities in research evaluation: toward a better evaluati...
Challenges and opportunities in research evaluation: toward a better evaluati...Challenges and opportunities in research evaluation: toward a better evaluati...
Challenges and opportunities in research evaluation: toward a better evaluati...
ORCID, Inc
 
Steps in research process
Steps in research processSteps in research process
Steps in research process
Nasir Mughal
 
Questionnaire2002
Questionnaire2002Questionnaire2002
Questionnaire2002
wardah azhar
 
Mixed method research
Mixed method researchMixed method research
Mixed method research
AthiraSankarM
 
Chapter 3 Methodology (Capstone Research)
Chapter 3   Methodology (Capstone Research)Chapter 3   Methodology (Capstone Research)
Chapter 3 Methodology (Capstone Research)
school
 
DB searches vs. snowballing
DB searches vs. snowballingDB searches vs. snowballing
DB searches vs. snowballing
GESSI UPC
 
Research methodology methods and techniques 2004
Research methodology   methods and techniques 2004Research methodology   methods and techniques 2004
Research methodology methods and techniques 2004
Institute of Information Technology, University of Dhaka, Bangladesh
 
Survey research lecture 9
Survey research lecture 9Survey research lecture 9
Survey research lecture 9
University of Balochistan
 

What's hot (20)

Web Assisted Assessment of Professional Behaviour
Web Assisted Assessment of Professional BehaviourWeb Assisted Assessment of Professional Behaviour
Web Assisted Assessment of Professional Behaviour
 
Quantitative data analysis - Attitudes Towards Research
Quantitative data analysis - Attitudes Towards ResearchQuantitative data analysis - Attitudes Towards Research
Quantitative data analysis - Attitudes Towards Research
 
Coursework Evaluation
Coursework   EvaluationCoursework   Evaluation
Coursework Evaluation
 
Comparative and Non-Comparative
Comparative and Non-ComparativeComparative and Non-Comparative
Comparative and Non-Comparative
 
Survey research
Survey researchSurvey research
Survey research
 
Basics of Systematic Review and Meta-analysis: Part 1
Basics of Systematic Review and Meta-analysis: Part 1Basics of Systematic Review and Meta-analysis: Part 1
Basics of Systematic Review and Meta-analysis: Part 1
 
Research
ResearchResearch
Research
 
Ch05 instrumentation
Ch05 instrumentationCh05 instrumentation
Ch05 instrumentation
 
Research design and methodology infosheet
Research design and methodology infosheetResearch design and methodology infosheet
Research design and methodology infosheet
 
Data collection
Data collectionData collection
Data collection
 
Action Research Pd
Action Research PdAction Research Pd
Action Research Pd
 
Mary ch2 presentation
Mary ch2 presentationMary ch2 presentation
Mary ch2 presentation
 
Challenges and opportunities in research evaluation: toward a better evaluati...
Challenges and opportunities in research evaluation: toward a better evaluati...Challenges and opportunities in research evaluation: toward a better evaluati...
Challenges and opportunities in research evaluation: toward a better evaluati...
 
Steps in research process
Steps in research processSteps in research process
Steps in research process
 
Questionnaire2002
Questionnaire2002Questionnaire2002
Questionnaire2002
 
Mixed method research
Mixed method researchMixed method research
Mixed method research
 
Chapter 3 Methodology (Capstone Research)
Chapter 3   Methodology (Capstone Research)Chapter 3   Methodology (Capstone Research)
Chapter 3 Methodology (Capstone Research)
 
DB searches vs. snowballing
DB searches vs. snowballingDB searches vs. snowballing
DB searches vs. snowballing
 
Research methodology methods and techniques 2004
Research methodology   methods and techniques 2004Research methodology   methods and techniques 2004
Research methodology methods and techniques 2004
 
Survey research lecture 9
Survey research lecture 9Survey research lecture 9
Survey research lecture 9
 

Viewers also liked

Mining tweets for security information (rev 2)
Mining tweets for security information (rev 2)Mining tweets for security information (rev 2)
Mining tweets for security information (rev 2)
Syracuse University
 
Carma internet research module visual design issues
Carma internet research module   visual design issuesCarma internet research module   visual design issues
Carma internet research module visual design issues
Syracuse University
 
Siop impact of social media
Siop impact of social mediaSiop impact of social media
Siop impact of social media
Syracuse University
 
What is Data Science
What is Data ScienceWhat is Data Science
What is Data Science
Syracuse University
 
Introduction to Advance Analytics Course
Introduction to Advance Analytics CourseIntroduction to Advance Analytics Course
Introduction to Advance Analytics Course
Syracuse University
 
Getting Started with R
Getting Started with RGetting Started with R
Getting Started with R
Syracuse University
 

Viewers also liked (6)

Mining tweets for security information (rev 2)
Mining tweets for security information (rev 2)Mining tweets for security information (rev 2)
Mining tweets for security information (rev 2)
 
Carma internet research module visual design issues
Carma internet research module   visual design issuesCarma internet research module   visual design issues
Carma internet research module visual design issues
 
Siop impact of social media
Siop impact of social mediaSiop impact of social media
Siop impact of social media
 
What is Data Science
What is Data ScienceWhat is Data Science
What is Data Science
 
Introduction to Advance Analytics Course
Introduction to Advance Analytics CourseIntroduction to Advance Analytics Course
Introduction to Advance Analytics Course
 
Getting Started with R
Getting Started with RGetting Started with R
Getting Started with R
 

Similar to Reducing Response Burden

Topic 5 DQ 2Nov 3-7, 2022Imagine once again that you are an au.docx
Topic 5 DQ 2Nov 3-7, 2022Imagine once again that you are an au.docxTopic 5 DQ 2Nov 3-7, 2022Imagine once again that you are an au.docx
Topic 5 DQ 2Nov 3-7, 2022Imagine once again that you are an au.docx
jolleybendicty
 
Quantitative search and_qualitative_research by mubarak
Quantitative search and_qualitative_research by mubarakQuantitative search and_qualitative_research by mubarak
Quantitative search and_qualitative_research by mubarak
Hafiza Abas
 
quali-lesson-22.ppt
quali-lesson-22.pptquali-lesson-22.ppt
quali-lesson-22.ppt
PhoebeMarcos
 
Research and advocacy by Seetal Daas
Research and advocacy by Seetal DaasResearch and advocacy by Seetal Daas
Research and advocacy by Seetal Daas
Seetal Daas
 
ALL ABOUT RESEARCH Principle of Reseach.pptx
ALL ABOUT RESEARCH Principle of Reseach.pptxALL ABOUT RESEARCH Principle of Reseach.pptx
ALL ABOUT RESEARCH Principle of Reseach.pptx
MariaTheresaCelebre1
 
answer the following questions thoroughly and provide justification.docx
answer the following questions thoroughly and provide justification.docxanswer the following questions thoroughly and provide justification.docx
answer the following questions thoroughly and provide justification.docx
nolanalgernon
 
Research question
Research questionResearch question
Research question
Tarek Tawfik Amin
 
Methods of Gathering Data for Research Purpose and Applications Using IJSER A...
Methods of Gathering Data for Research Purpose and Applications Using IJSER A...Methods of Gathering Data for Research Purpose and Applications Using IJSER A...
Methods of Gathering Data for Research Purpose and Applications Using IJSER A...
IOSR Journals
 
Analyzing qualitative data 4 13-17
Analyzing qualitative data 4 13-17Analyzing qualitative data 4 13-17
Analyzing qualitative data 4 13-17
tjcarter
 
Comparison quantitative and qualitative
Comparison quantitative and qualitativeComparison quantitative and qualitative
Comparison quantitative and qualitative
David Ngilangwa
 
Basic Inculcation on Research Activities among Devotees.pptx
Basic Inculcation  on Research Activities among Devotees.pptxBasic Inculcation  on Research Activities among Devotees.pptx
Basic Inculcation on Research Activities among Devotees.pptx
PrakashPaudel30
 
Data collection.pptx
Data collection.pptxData collection.pptx
Data collection.pptx
RohanRo11
 
1.2 data collection
1.2 data collection1.2 data collection
1.2 data collection
LeenaKP
 
Health elearning rg may 2014
Health elearning rg may 2014Health elearning rg may 2014
Health elearning rg may 2014
Peter Goodyear
 
Essay On Qualitative Research
Essay On Qualitative ResearchEssay On Qualitative Research
Survey and correlational methods of research: Assumptions, Steps and Pros and...
Survey and correlational methods of research: Assumptions, Steps and Pros and...Survey and correlational methods of research: Assumptions, Steps and Pros and...
Survey and correlational methods of research: Assumptions, Steps and Pros and...
Michael J Leo
 
Lo3 Other Elements of Research
Lo3  Other Elements of ResearchLo3  Other Elements of Research
Lo3 Other Elements of Research
Jo Balucanag - Bitonio
 
Intervention forEducationMarkis’ EdwardsJanuary 29, 2018.docx
Intervention forEducationMarkis’ EdwardsJanuary 29, 2018.docxIntervention forEducationMarkis’ EdwardsJanuary 29, 2018.docx
Intervention forEducationMarkis’ EdwardsJanuary 29, 2018.docx
normanibarber20063
 
321423152 e-0016087606-session39134-201012122352 (1)
321423152 e-0016087606-session39134-201012122352 (1)321423152 e-0016087606-session39134-201012122352 (1)
321423152 e-0016087606-session39134-201012122352 (1)
Iin Angriyani
 
statistics 02.ppt
statistics 02.pptstatistics 02.ppt
statistics 02.ppt
Cade143
 

Similar to Reducing Response Burden (20)

Topic 5 DQ 2Nov 3-7, 2022Imagine once again that you are an au.docx
Topic 5 DQ 2Nov 3-7, 2022Imagine once again that you are an au.docxTopic 5 DQ 2Nov 3-7, 2022Imagine once again that you are an au.docx
Topic 5 DQ 2Nov 3-7, 2022Imagine once again that you are an au.docx
 
Quantitative search and_qualitative_research by mubarak
Quantitative search and_qualitative_research by mubarakQuantitative search and_qualitative_research by mubarak
Quantitative search and_qualitative_research by mubarak
 
quali-lesson-22.ppt
quali-lesson-22.pptquali-lesson-22.ppt
quali-lesson-22.ppt
 
Research and advocacy by Seetal Daas
Research and advocacy by Seetal DaasResearch and advocacy by Seetal Daas
Research and advocacy by Seetal Daas
 
ALL ABOUT RESEARCH Principle of Reseach.pptx
ALL ABOUT RESEARCH Principle of Reseach.pptxALL ABOUT RESEARCH Principle of Reseach.pptx
ALL ABOUT RESEARCH Principle of Reseach.pptx
 
answer the following questions thoroughly and provide justification.docx
answer the following questions thoroughly and provide justification.docxanswer the following questions thoroughly and provide justification.docx
answer the following questions thoroughly and provide justification.docx
 
Research question
Research questionResearch question
Research question
 
Methods of Gathering Data for Research Purpose and Applications Using IJSER A...
Methods of Gathering Data for Research Purpose and Applications Using IJSER A...Methods of Gathering Data for Research Purpose and Applications Using IJSER A...
Methods of Gathering Data for Research Purpose and Applications Using IJSER A...
 
Analyzing qualitative data 4 13-17
Analyzing qualitative data 4 13-17Analyzing qualitative data 4 13-17
Analyzing qualitative data 4 13-17
 
Comparison quantitative and qualitative
Comparison quantitative and qualitativeComparison quantitative and qualitative
Comparison quantitative and qualitative
 
Basic Inculcation on Research Activities among Devotees.pptx
Basic Inculcation  on Research Activities among Devotees.pptxBasic Inculcation  on Research Activities among Devotees.pptx
Basic Inculcation on Research Activities among Devotees.pptx
 
Data collection.pptx
Data collection.pptxData collection.pptx
Data collection.pptx
 
1.2 data collection
1.2 data collection1.2 data collection
1.2 data collection
 
Health elearning rg may 2014
Health elearning rg may 2014Health elearning rg may 2014
Health elearning rg may 2014
 
Essay On Qualitative Research
Essay On Qualitative ResearchEssay On Qualitative Research
Essay On Qualitative Research
 
Survey and correlational methods of research: Assumptions, Steps and Pros and...
Survey and correlational methods of research: Assumptions, Steps and Pros and...Survey and correlational methods of research: Assumptions, Steps and Pros and...
Survey and correlational methods of research: Assumptions, Steps and Pros and...
 
Lo3 Other Elements of Research
Lo3  Other Elements of ResearchLo3  Other Elements of Research
Lo3 Other Elements of Research
 
Intervention forEducationMarkis’ EdwardsJanuary 29, 2018.docx
Intervention forEducationMarkis’ EdwardsJanuary 29, 2018.docxIntervention forEducationMarkis’ EdwardsJanuary 29, 2018.docx
Intervention forEducationMarkis’ EdwardsJanuary 29, 2018.docx
 
321423152 e-0016087606-session39134-201012122352 (1)
321423152 e-0016087606-session39134-201012122352 (1)321423152 e-0016087606-session39134-201012122352 (1)
321423152 e-0016087606-session39134-201012122352 (1)
 
statistics 02.ppt
statistics 02.pptstatistics 02.ppt
statistics 02.ppt
 

More from Syracuse University

Discovery informaticsstanton
Discovery informaticsstantonDiscovery informaticsstanton
Discovery informaticsstanton
Syracuse University
 
Basic SEVIS Overview for U.S. University Faculty
Basic SEVIS Overview for U.S. University FacultyBasic SEVIS Overview for U.S. University Faculty
Basic SEVIS Overview for U.S. University Faculty
Syracuse University
 
Why R? A Brief Introduction to the Open Source Statistics Platform
Why R? A Brief Introduction to the Open Source Statistics PlatformWhy R? A Brief Introduction to the Open Source Statistics Platform
Why R? A Brief Introduction to the Open Source Statistics Platform
Syracuse University
 
Chapter9 r studio2
Chapter9 r studio2Chapter9 r studio2
Chapter9 r studio2
Syracuse University
 
Basic Overview of Data Mining
Basic Overview of Data MiningBasic Overview of Data Mining
Basic Overview of Data Mining
Syracuse University
 
Strategic planning
Strategic planningStrategic planning
Strategic planning
Syracuse University
 
Carma internet research module scale development
Carma internet research module   scale developmentCarma internet research module   scale development
Carma internet research module scale development
Syracuse University
 
Carma internet research module getting started with question pro
Carma internet research module   getting started with question proCarma internet research module   getting started with question pro
Carma internet research module getting started with question pro
Syracuse University
 
Basic Graphics with R
Basic Graphics with RBasic Graphics with R
Basic Graphics with R
Syracuse University
 
R-Studio Vs. Rcmdr
R-Studio Vs. RcmdrR-Studio Vs. Rcmdr
R-Studio Vs. Rcmdr
Syracuse University
 
Moving Data to and From R
Moving Data to and From RMoving Data to and From R
Moving Data to and From R
Syracuse University
 
Installing R and R-Studio
Installing R and R-StudioInstalling R and R-Studio
Installing R and R-Studio
Syracuse University
 
PACIS Survey Workshop
PACIS Survey WorkshopPACIS Survey Workshop
PACIS Survey Workshop
Syracuse University
 
Carma internet research module: Future data collection
Carma internet research module: Future data collectionCarma internet research module: Future data collection
Carma internet research module: Future data collection
Syracuse University
 
Carma internet research module: Sampling for internet
Carma internet research module: Sampling for internetCarma internet research module: Sampling for internet
Carma internet research module: Sampling for internet
Syracuse University
 
Carma internet research module: Encouraging responding
Carma internet research module: Encouraging respondingCarma internet research module: Encouraging responding
Carma internet research module: Encouraging responding
Syracuse University
 
Carma internet research module: Survey reduction
Carma internet research module: Survey reductionCarma internet research module: Survey reduction
Carma internet research module: Survey reduction
Syracuse University
 
Carma internet research module: Research design catalog
Carma internet research module: Research design catalogCarma internet research module: Research design catalog
Carma internet research module: Research design catalog
Syracuse University
 
Stanton eScience Presentation
Stanton eScience PresentationStanton eScience Presentation
Stanton eScience Presentation
Syracuse University
 
Carma internet research module detecting bad data
Carma internet research module   detecting bad dataCarma internet research module   detecting bad data
Carma internet research module detecting bad data
Syracuse University
 

More from Syracuse University (20)

Discovery informaticsstanton
Discovery informaticsstantonDiscovery informaticsstanton
Discovery informaticsstanton
 
Basic SEVIS Overview for U.S. University Faculty
Basic SEVIS Overview for U.S. University FacultyBasic SEVIS Overview for U.S. University Faculty
Basic SEVIS Overview for U.S. University Faculty
 
Why R? A Brief Introduction to the Open Source Statistics Platform
Why R? A Brief Introduction to the Open Source Statistics PlatformWhy R? A Brief Introduction to the Open Source Statistics Platform
Why R? A Brief Introduction to the Open Source Statistics Platform
 
Chapter9 r studio2
Chapter9 r studio2Chapter9 r studio2
Chapter9 r studio2
 
Basic Overview of Data Mining
Basic Overview of Data MiningBasic Overview of Data Mining
Basic Overview of Data Mining
 
Strategic planning
Strategic planningStrategic planning
Strategic planning
 
Carma internet research module scale development
Carma internet research module   scale developmentCarma internet research module   scale development
Carma internet research module scale development
 
Carma internet research module getting started with question pro
Carma internet research module   getting started with question proCarma internet research module   getting started with question pro
Carma internet research module getting started with question pro
 
Basic Graphics with R
Basic Graphics with RBasic Graphics with R
Basic Graphics with R
 
R-Studio Vs. Rcmdr
R-Studio Vs. RcmdrR-Studio Vs. Rcmdr
R-Studio Vs. Rcmdr
 
Moving Data to and From R
Moving Data to and From RMoving Data to and From R
Moving Data to and From R
 
Installing R and R-Studio
Installing R and R-StudioInstalling R and R-Studio
Installing R and R-Studio
 
PACIS Survey Workshop
PACIS Survey WorkshopPACIS Survey Workshop
PACIS Survey Workshop
 
Carma internet research module: Future data collection
Carma internet research module: Future data collectionCarma internet research module: Future data collection
Carma internet research module: Future data collection
 
Carma internet research module: Sampling for internet
Carma internet research module: Sampling for internetCarma internet research module: Sampling for internet
Carma internet research module: Sampling for internet
 
Carma internet research module: Encouraging responding
Carma internet research module: Encouraging respondingCarma internet research module: Encouraging responding
Carma internet research module: Encouraging responding
 
Carma internet research module: Survey reduction
Carma internet research module: Survey reductionCarma internet research module: Survey reduction
Carma internet research module: Survey reduction
 
Carma internet research module: Research design catalog
Carma internet research module: Research design catalogCarma internet research module: Research design catalog
Carma internet research module: Research design catalog
 
Stanton eScience Presentation
Stanton eScience PresentationStanton eScience Presentation
Stanton eScience Presentation
 
Carma internet research module detecting bad data
Carma internet research module   detecting bad dataCarma internet research module   detecting bad data
Carma internet research module detecting bad data
 

Recently uploaded

BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
Nguyen Thanh Tu Collection
 
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdfWalmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
TechSoup
 
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdfANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
Priyankaranawat4
 
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdfবাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
eBook.com.bd (প্রয়োজনীয় বাংলা বই)
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
Jean Carlos Nunes Paixão
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
chanes7
 
clinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdfclinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdf
Priyankaranawat4
 
How to Fix the Import Error in the Odoo 17
How to Fix the Import Error in the Odoo 17How to Fix the Import Error in the Odoo 17
How to Fix the Import Error in the Odoo 17
Celine George
 
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
mulvey2
 
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPLAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
RAHUL
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
PECB
 
How to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP ModuleHow to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP Module
Celine George
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
Nicholas Montgomery
 
DRUGS AND ITS classification slide share
DRUGS AND ITS classification slide shareDRUGS AND ITS classification slide share
DRUGS AND ITS classification slide share
taiba qazi
 
Life upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for studentLife upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for student
NgcHiNguyn25
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
Katrina Pritchard
 
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
GeorgeMilliken2
 
Executive Directors Chat Leveraging AI for Diversity, Equity, and Inclusion
Executive Directors Chat  Leveraging AI for Diversity, Equity, and InclusionExecutive Directors Chat  Leveraging AI for Diversity, Equity, and Inclusion
Executive Directors Chat Leveraging AI for Diversity, Equity, and Inclusion
TechSoup
 
How to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 InventoryHow to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 Inventory
Celine George
 
Cognitive Development Adolescence Psychology
Cognitive Development Adolescence PsychologyCognitive Development Adolescence Psychology
Cognitive Development Adolescence Psychology
paigestewart1632
 

Recently uploaded (20)

BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
BÀI TẬP BỔ TRỢ TIẾNG ANH 8 CẢ NĂM - GLOBAL SUCCESS - NĂM HỌC 2023-2024 (CÓ FI...
 
Walmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdfWalmart Business+ and Spark Good for Nonprofits.pdf
Walmart Business+ and Spark Good for Nonprofits.pdf
 
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdfANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
ANATOMY AND BIOMECHANICS OF HIP JOINT.pdf
 
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdfবাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
বাংলাদেশ অর্থনৈতিক সমীক্ষা (Economic Review) ২০২৪ UJS App.pdf
 
A Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdfA Independência da América Espanhola LAPBOOK.pdf
A Independência da América Espanhola LAPBOOK.pdf
 
Digital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments UnitDigital Artifact 1 - 10VCD Environments Unit
Digital Artifact 1 - 10VCD Environments Unit
 
clinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdfclinical examination of hip joint (1).pdf
clinical examination of hip joint (1).pdf
 
How to Fix the Import Error in the Odoo 17
How to Fix the Import Error in the Odoo 17How to Fix the Import Error in the Odoo 17
How to Fix the Import Error in the Odoo 17
 
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptxC1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
C1 Rubenstein AP HuG xxxxxxxxxxxxxx.pptx
 
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UPLAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
LAND USE LAND COVER AND NDVI OF MIRZAPUR DISTRICT, UP
 
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
ISO/IEC 27001, ISO/IEC 42001, and GDPR: Best Practices for Implementation and...
 
How to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP ModuleHow to Add Chatter in the odoo 17 ERP Module
How to Add Chatter in the odoo 17 ERP Module
 
Film vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movieFilm vocab for eal 3 students: Australia the movie
Film vocab for eal 3 students: Australia the movie
 
DRUGS AND ITS classification slide share
DRUGS AND ITS classification slide shareDRUGS AND ITS classification slide share
DRUGS AND ITS classification slide share
 
Life upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for studentLife upper-Intermediate B2 Workbook for student
Life upper-Intermediate B2 Workbook for student
 
BBR 2024 Summer Sessions Interview Training
BBR  2024 Summer Sessions Interview TrainingBBR  2024 Summer Sessions Interview Training
BBR 2024 Summer Sessions Interview Training
 
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
What is Digital Literacy? A guest blog from Andy McLaughlin, University of Ab...
 
Executive Directors Chat Leveraging AI for Diversity, Equity, and Inclusion
Executive Directors Chat  Leveraging AI for Diversity, Equity, and InclusionExecutive Directors Chat  Leveraging AI for Diversity, Equity, and Inclusion
Executive Directors Chat Leveraging AI for Diversity, Equity, and Inclusion
 
How to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 InventoryHow to Setup Warehouse & Location in Odoo 17 Inventory
How to Setup Warehouse & Location in Odoo 17 Inventory
 
Cognitive Development Adolescence Psychology
Cognitive Development Adolescence PsychologyCognitive Development Adolescence Psychology
Cognitive Development Adolescence Psychology
 

Reducing Response Burden

  • 1. Fred Oswald Rice University – Department of Psychology Jeff Stanton Syracuse University – School of Information Studies Kauffman Roundtable on Establishment Surveys August 9, 2011 Reducing Response Burden
  • 2. Reducing Response Burden Goals Reduce time of administration and associated costs Increase response rates, compliance, fewer missing data, Decrease survey fatigue and increase willingness to re-engage over time Effects Increase information yield per unit of response time/effort Balance multiple stakeholder survey goals based on “lean” and less reliable information sources
  • 3. Approaches and Constraints Goal: No practical loss in reliability or validity at the desired aggregate level (unit, sub-unit/cluster) Approaches Reduce instructions/explanatory material Reduce item redundancy Distribute subsets of items strategically across units, using available data or imputation to complete analyses Automate field completion with NLP and scrapers Constraints Each approach has intrinsic weaknesses Stakeholders wedded to particular items Longitudinal comparability limits changes
  • 4. Research on Response Burden Example: How to determine efficient item assignments to sub-samples, given available item information, precision goals, and stakeholder concerns? Given different approaches (e.g., multiple imputation) what is the maximum reduction in item content before loss of precision becomes intolerable? How redundant are components of a profile (so certain items ‘drive’ a profile)? What are stakeholder reactions to various imputation/analysis strategies? Our Expertise Methods generalists who examine technology and statistical tools to inform practical goals of survey development and test characteristics Statistical tools: Item response theory, multi-level and cross-classified models, meta-analysis, machine learning Technology tools: Web-based surveys, data scrapers, alternative data collection methods
  • 5. Selected Papers Non-response issues Converse, P. D., Wolfe, E. W., Huang, X., & Oswald, F. L. (2008). Response rates for mixed-mode surveys using mail and email/web. American Journal of Evaluation, 29, 99-107. Rogelberg, S. G, & Stanton, J. M. (2007). Understanding and dealing with organizational survey nonresponse. Organizational Research Methods, 10, 195-209. Wolfe, E. W., Converse, P. D., & Oswald, F. L. (2008). Item-level non-response rates in an attitudinal survey of teachers delivered via mail and web. Journal of Computer-Mediated Communication, 14, 35-66. Shortening measures Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006). The Mini-IPIP scales: Tiny-yet-effective measures of the Big Five factors of personality. Psychological Assessment, 18, 192-203. Russell, S.R., Spitzmüller, C., Lin, L.F., Stanton, J.M., Smith, P.C., & Ironson, G. H. (2004). Shorter can also be better: The abridged Job in General Measure. Educational and Psychological Measurement, 64 (5), 878-893. Stanton, J. M., Sinar, E. F., Balzer, W. K., & Smith, P. C. (2002). Issues and strategies for reducing the length of self-report scales. Personnel Psychology, 55 (1), 167-193. Stanton, J. M. (2000). Empirical distributions of correlations as a tool for scale reduction. Behavior Research Methods, Instruments, and Computers, 32, 403-406. Technology issues Rogelberg, S. G., Church, A. H., Waclawski, J., & Stanton, J. M. (2002). Organizational Survey Research: Overview, the Internet/intranet and present practices of concern. In S. G. Rogelberg (Ed.), Handbook of Research Methods in Industrial and Organizational Psychology. Oxford: Blackwell. Stanton, J. M., & Rogelberg, S. G. (2001). Using Internet/Intranet Web Pages to Collect Organizational Research Data. Organizational Research Methods, 4, 199-216. Stanton, J. M. (1998). An empirical assessment of data collection using the Internet. Personnel Psychology, 51, 709-725. Parallel forms (item banking, test security) Oswald, F. L., Friede, A. J., Schmitt, N., Kim, B. K., & Ramsay, L. J. (2005). Extending a practical method for developing alternate test forms using independent sets of items. Organizational Research Methods, 8, 149-164.