- Strictly standardized interviewing leads to poor comprehension, while less strictly standardized methods can improve comprehension. - Two studies found that respondents' comprehension was most accurate when interviewers provided clarification not just when requested, but also whenever they suspected it was needed. - Respondents answered survey questions based on fictional scenarios to assess accuracy, and comprehension was nearly perfect for straightforward scenarios but improved with interviewer clarification for complicated scenarios.

1. MISUNDERSTANDING
STANDARDIZED LANGUAGE
IN
RESEARCH INTERVIEWS
AUTHORS: MICHAEL F. SCHOBER, FREDERICK G. CONRAD
AND SCOTT S. FRICKER
PRESENTED BY: FATIMA NAFIE, ELIANA ZAMBLERA
ANNO A.A. 2015/2016
COGNIZIONE E LINGUAGGIO 1

2. PREROGATIVES OF THE ARTICLE
• LEAVING THE INTERPRETATION OF WORDS UP TO PARTICIPANTS IN
STANDARDIZED SURVEY INTERVIEWS, APTITUDE TESTS, AND EXPERIMENT
INSTRUCTIONS CAN LEAD TO UNINTENDED INTERPRETATION;
• MORE COLLABORATIVE INTERVIEWING METHODS CAN PROMOTE UNIFORM
UNDERSTANDING.
COGNIZIONE E LINGUAGGIO 2

3. COGNIZIONE E LINGUAGGIO 3
IN TWO LABORATORY STUDIES, WE’RE GOING TO SEE THAT
COMPREHENSION WAS MORE ACCURATE WHEN INTERVIEWERS
RESPONDED TO REQUESTS FOR CLARIFICATION WITH NON-
STANDARDIZED PARAPHARASED DEFINITIONS, AND MOST ACCURATE
WHEN INTERVIEWERS ALSO PROVIDED CLARIFICATION WHENEVER THEY
SUSPECTED RESPONDENTS NEEDED IT.

4. REDUCING THE BIAS
IN ORDER TO REDUCE THE BIAS AND PROMOTE COMPARABILITY OF RESPONSES,
PSYCHOLOGISTS STANDARDIZE THE MATERIALS THEY USE: THEY EXPOSE ALL
RESPONDENTS AND SUBJECTS TO EXACTLY THE SAME WORDS. THEY ALSO
REQUIRE PARTICIPANTS TO INTERPRET THOSE WORDS FOR THEMSELVES, SO THAT
RESEARCHERS WILL NOT INFLUENCE RESPONSES.
COGNIZIONE E LINGUAGGIO 4

5. PROBLEM
STANDARDIZING WORDING AND LEAVING THE INTERPRETATION UP TO THE
AUDIENCE CAN BE PROBLEMATIC DUE TO THE OVERRELIANCE ON STANDARDIZED
WORDING. THE ARGUMENT IS THAT DIFFERENT LISTENERS, READERS OR USERS
CAN INTERPRET THE SAME WORDS QUITE DIFFERENTLY, AND SO STANDARDIZED
WORDING MAY ACTUALLY DECREASE UNIFORMITY OF INTERPRETATION.
COGNIZIONE E LINGUAGGIO 5

6. FINDING SOLUTIONS
A GOOD SOLUTION COULD BE TO COLLABORATE WITH RESPONDENTS, SAYING
WHATEVER IT TAKES TO ENSURE THAT THE ADDRESSEE HAS UNDERSTOOD THE
WORDS AS INTENDED. PARADOXICALLY, IN ORDER TO STANDARDIZE
INTERPRETATION, SPEAKERS MAY NEED TO FOLLOW NON-STANDARDIZED
‘CONVERSATIONAL’ PRACTICE, CLARIFYING WHAT THEIR WORDS MEAN (WHEN
ADDRESSEES ASK FOR HELP OR SEEM TO NEED IT).
COGNIZIONE E LINGUAGGIO 6

7. • STRICTLY STANDARDIZED INTERVIEWING LEADS TO POOR COMPREHENSION;
• LESS STRICTLY STANDARDIZED METHODS CAN LEAD TO IMPROVED
COMPREHENSION.
COGNIZIONE E LINGUAGGIO 7

8. CASE STUDY 1
IN THIS EXPERIMENT INTERVIEWERS PROVIDED:
• NO CLARIFICATION;
• CLARIFICATION ONLY WHEN RESPONDENTS REQUESTED IT (RESPONDENT-
INITIATED CLARIFICATION);
• CLARIFICATION BOTH WHEN RESPONDENTS REQUESTED IT AND WHENEVER ELSE
THEY BELIEVED RESPONDENTS NEEDED IT (MIXED-INITIATIVE CLARIFICATION).
COGNIZIONE E LINGUAGGIO 8

9. COGNIZIONE E LINGUAGGIO 9
PARTECIPANTS
• 55 PROFESSIONAL INTERVIEWERS;
• 109 RESPONDENTS: THEY HAD RESPONDED TO AN AD IN ‘THE
WASHINGTON POST’. THEY RAPRESENTED A RANGE OF
DEMOGRAPHIC CHARACTERISTICS.

10. COGNIZIONE E LINGUAGGIO 10
METHOD
SOME INTERVIEWERS WERE TRAINED TO READ SCRIPTED DEFINITIONS VERBATIM, AND
OTHERS WERE ALLOWED TO PARAPHRASE THE DEFINITIONS.
RESPONDENTS ANSWERED ON THE BASIS OF FICTIONAL SCENARIOS, SO THAT THEY
COULD ASSESS RESPONSE ACCURACY. THE SCENARIOS WERE DESIGNED IN EITHER A
STRAIGHTFORWARD OR COMPLICATED WAY TO MAP ONTO THE SURVEY QUESTIONS.

11. COGNIZIONE E LINGUAGGIO 11
ALL RESPONDENTS WERE ASKED THE SAME 12 QUESTIONS FROM ONGOING U.S. GOVERNMENT
SURVEYS (4 ABOUT EMPLOYMENT, 4 ABOUT HOUSING, 4 ABOUT PURCHASES). FOR EACH
RESPONDENT, 6 SCENARIOS DESCRIBED SITUATIONS THAT MAPPED ONTO QUESTIONS IN A
STRAIGHTFORWARD WAY, AND 6 SCENARIOS DESCRIBED SITUATIONS THAT MAPPED ONTO
QUESTIONS IN A COMPLICATED WAY. DIFFERENT RESPONDENT SAW DIFFERENT SCENARIOS FOR
DIFFERENT QUESTIONS: THE ORDERING OF THE MAPPINGS WAS COUNTERBALANCED ACROSS
RESPONDENTS.

12. COGNIZIONE E LINGUAGGIO 12

13. • RESPONDENTS COMPREHENDED THE SURVEY QUESTIONS MORE ACCURATELY
WHEN THEY COULD RECEIVE CLARIFICATION THAN WHEN THE INTERPRETATION
WAS LEFT UP TO THEM;
• RESPONDENTS COMPREHENSION FOR STRAIGHTFORWARD SCENARIOS WAS
EXCELLENT;
• RESPONSES WERE MARGINALLY BETTER WHEN INTERVIEWERS USED THEIR OWN
WORDS TO CLARIFY QUESTION MEANING RATHER THAN READING SCRIPTED
DEFINITIONS.
COGNIZIONE E LINGUAGGIO 13

14. CASE STUDY 2
THEY ASKED A SMALL SAMPLE OF INTERVIEWERS FROM THE MAINSTREAM SURVEY
ORGANIZATION USED IN STUDY 1 TO CARRY OUT AS THEY ORDINARY DO.
RESPONDENTS ANSWERED THE SAME QUESTIONS FROM ONGOING SURVEYS USED
IN STUDY ON THE BASIS OF THE SAME FICTIONAL SCENARIOS.
COGNIZIONE E LINGUAGGIO 14

15. • AS IN STUDY 1, RESPONDENTS’ ANSWERS MATCHED THE OFFICIAL DEFINITIONS
VIRTUALLY PERFECTLY FOR STRAIGHTFORWARD MAPPINGS.
• A MORE DETAILED EXAMINATION OF COMPREHENSION ACCURACY SHOWS THAT
INTERVIEWER DEVIATIONS FROM STRICT STANDARDIZATION LED TO
SUBSTANTIAL IMPROVEMENT IN ACCURACY.
COGNIZIONE E LINGUAGGIO 15

16. COMPREHENSION ACCURACY CAN IMPROVE SUBSTANTIALLY WHEN INTERVIEWERS
DEVIATE FROM STRICT STANDARDIZATION.
COGNIZIONE E LINGUAGGIO 16

17. CONCLUSIONS
COGNIZIONE E LINGUAGGIO 17
• WHAT INTERVIEWERS DO AFTER THEY READ THE QUESTIONS IN
STANDARDIZED SURVEYS ABOUT FACTS AND BEHAVIORS CAN
DRASTICALLY AFFECT WHETHER RESPONDENTS UNDERSTAND
QUESTION CONCEPTS AS THE SURVEY DESIGNERS INTEND.
• TRAINING INTERVIEWERS TO INITIATE CLARIFICATION WHENEVER THEY
FEEL IT IS NECESSARY, RATHER THAN RELYING ON RESPONDENTS TO
ASK FOR CLARIFICATION, PROMOTES THE MOST ACCURATE
RESPONDENT COMPREHENSION.

18. COGNIZIONE E LINGUAGGIO 18