pilot study vs pretesting

1. PILOT STUDY AND PRETESTING
BY
DR EBOT WALTER OJONG
SENIOR LECTURER/CHEMICAL PATHOLOGIST

2. INTRODUCTION
‘No amount of intellectual exercise can substitute
for testing an instrument designed to
communicate with ordinary people’ (Backstrom
and Hursch, 1963)
Surveys must be thoroughly pretested before
they are launched (Hulland et al., 2018)

3. INTRODUCTION
To ensure that the meaning of the question items
are clear to the respondents, and that the
respondents will clearly understand the
questions the way they are designed and
intended, it is vey important that a survey
questionnaire must be pretested (Sekaran,
2003)

4. MEANING OF PILOT STUDY
• A mini version or small scale version of the research study aimed at
trying or testing the research process and protocols in order to help in
assessing the feasibility of the main/proposed research project
• Could be a mini version of the entire process of the main research
project or of certain aspects e.g. data collection, analysis tools etc.
• Examples piloting of observations, interviews, questionnaires or other
tools
• In experimental studies it could be piloting of the experiments/tests

5. OBJECTIVES OF PILOT STUDIES
• The general goal of a pilot study is to save some time, effort
and money, which can be lost if a major research study fails
because of unforeseen attributes
• Test adequacy of research instruments
• Assess feasibility of a full-scale study
• Assess whether the research protocol is realistic and
workable
• Reveal logistic issues
• Collect preliminary data

6. OBJECTIVES OF PILOT STUDIES
• Ensure whether the sampling frame and technique are
effective
• Determine sample size (The pilot study sample is
inappropriate to be used as the main study sample)
• Convince funding bodies that the major study is feasible and
worth funding

7. OBJECTIVES OF PILOT STUDIES
• ‘To see if the beast will fly’ (Teijingen and Hundley, 2002)
• ‘Reassessment without tears’ (Blaxter, Hughes and Tight,
1996:121)
•‘Do not take the risk. Pilot test first’ (Van Teijilingen and
Hundley, 2001:2)

8. IMPORTANCE OF PILOT STUDIES
• Helps in establishing the feasibility of the research project
• Feasibility of data collection and analysis tools/processes
• Authentication and improvement of research processes and
tools. Validity and reliability of research tools and processes
which add rigour to the research
• Precautions against possible large scale project failures

9. IMPORTANCE OF PILOT STUDIES
• Data/results generated from pilot study could be made
part of the main research.
• Establishing the need/importance of full/large scale
studies.
• Useful for junior researchers and student researchers.
• Helps in obtaining funding for large scale studies.

10. HOW ARE PILOT STUDIES CARRIED OUT?
QUANTITATIVE STUDIES
• Surveys should be carried out on small samples during the
pilot process which are subsets of the larger
sample/population for the main study.
• Generally these mini versions of surveys will establish
validity, reliability and feasibility of data collection, analysis
and access processes
• Highly important/essential

11. HOW ARE PILOT STUDIES CARRIED OUT?
QUALITATIVE STUDIES
• Pilot studies are generally useful for for improvement of
data collection tools and processes (interviews,
observation protocols) in terms of language, content and
data obtained.
• This helps in refining and authenticating the data obtained
in the main study
• Important/useful

12. HOW ARE PILOT STUDIES CARRIED OUT?
• MIXED METHODS STUDIES
• As addressed above
• Important/useful

13. PILOT STUDY OR PRETESTING
•Although pre-testing is often understood as
a pilot study, they both serve distinctive
purposes.
•They are similar but different from each
other

14. PILOT STUDY OR PRETESTING
•A pilot study is a mini-version of a full-scale study or a
trial run done in preparation of the complete study to
be undertaken in order to ensure that a full-fledged
study will be carried out successfully; a feasibility
study.
•It is a try-out not only of research techniques and
methods but also of questionnaires and interviews

15. SAMPLE SIZE FOR PILOT STUDY
•A sample between 25 and 100 individuals (Cooperand
Schindler, 2011)
• A range from 10 to 30 individuals are enough for a pilot
test (Hill, 1998)
•A sample size should be 10 per cent of the sample
projected for the main study (Connelly, 2008)

16. WHAT IS PRE-TESTING PROCESS?
•Objectives of pre-testing
To ensure whether
a) the wording of the questions is correct
b) the sequence of questions is correct
c) the respondents have clearly understood all the
questions
d) Additional questions are needed or some questions
should be eliminated
e) the instructions are clear and adequate

17. WHAT IS PRE-TESTING PROCESS?
•Fundamental issues of pretesting a questionnaire
a) The specific items to be pretested
b) The method to be used to conduct the pretest
c) The person/s who should conduct the pretest
d) The participants who should be the subjects of the
pretest
e) The sample size needed for the pretest

18. THE SPECIFIC ITEM TO BE TESTED
Items about the questionnaire itself
•Format of the questions
•Layout
•Length
•Number of lines in case respondents need to write a
response, and the sequencing of questions
Items about specific questions
•Whether the respondent interpreted the question
differently or whether the respondents did not understand
the statement very well

19. THE METHOD TO BE USED TO CONDUCT
THE PRETEST
Cognitive (personal) interviews
Debriefing method
Protocol method
Telephone interview
Mail self-reports

20. THE METHOD TO BE USED TO CONDUCT
THE PRETEST
Cognitive (personal) interviews
Debriefing method
 The respondent is asked to answer the questionnaire
completely while the interviewer observes any
indication gestures of confusion from the respondent
 After completing the questionnaire, the interviewer
will ask the respondent for any possible problems
which he/she may have encountered while filling out
the questionnaire

21. THE METHOD TO BE USED TO CONDUCT
THE PRETEST
Cognitive (personal) interviews
Protocol method
 The respondent is asked to express his or her
thoughts once it occurs while answering the
questionnaire
A recorder will be helpful to make carefulnotes about
the verbalizations of the respondents thoughts

22. THE PERSON TO CONDUCT THE PRETEST
Only the best interviewers should conduct pretesting as
they have the keen ability to observe any uneasiness,
confusion and resistance among the respondents
The selection of interviewers should also involve different
levels of competence

23. THE PARTICIPANTS DURING THE PRETEST
Individuals who are similar to the target population of the
study
Individuals who are typical or representatives of the actual
population to be studied
Individuals at a site different from the actual study site

24. THE SAMPLE SIZE FOR THE PRETEST
If the questionnaire is long and complex, it would
need a bigger sample for pretesting
If the target population of the study is
unsophisticated, a bigger sample for pretesting size is
also necessary

25. SUMMARY
PILOT STUDY: To try out the research questions,
methodology, research instrumentor anu aspect of the
research
PRE-TESTING: To ensure that the instrument is clearly
understood by the respondents