3. Is the method of selection of the sample clear?
4. Could the method of selection affect the result?
5. Are subjects likely to be motivated to give biased responses?
METHODOLOGY: INSTRUMENTS• 1. Is evidence for validity and reliability clearly presented and adequate?• Qualitative – matter of degree than absolute state.• Quantitative – – Stability – Equivalence – Internal consistency
2. Is there a clear description of how the instrument was administered?• What instructions and guidelines were given?• What was the nature of the text? • What was the degree of difficulty?• Was there interview?• Focus-group discussion?• Who conducted the interviews/FGDs?
3. Is it likely that subjects would fake their responses?
METHODOLOGY: PROCEDURES• 1. Are there clear weaknesses in the design of the study?
2. Are the procedures forcollecting information describedfully?a. kinds of written questionsb. kinds of semi-structuredinterview questions.
3. Is it likely that the researcher is biased?• Use help of colleagues to triangulate problems.
RESULTS• 1. Are the findings presented clearly?• 2. Is there appropriate use of tables, charts, and figures?
3. Is the number of subjectstaken into consideration when presenting the results?
4. Is there sufficient descriptive information to interpret the results?
5. Do illustrative quotes andspecific instances accompany the results?
DISCUSSIONS AND CONCLUSIONS• 1. Is the discussion based on the research problem and results, or is there a tendency to discuss unrelated material or ideas?
2. Is there an adequateinterpretation of the findings?
3. Is the interpretation separate from the results? 4. Are the results discussed in relation to previous studies?
• 5. Are limitations due to methodology included in the discussion?• 6. Are the conclusions clearly stated and based on the results and the discussion?• 7. Are the conclusions reasonable?
• 8. What is the external validity of the study?• 9. What factors would affect the external validity? – Does the sampling capture variation in a diverse enough manner? (Profiling) – Is the context fully described so that the reader can judge transferability to other settings? – How well have the concepts been extracted from the data?