The document discusses instrument development in quantitative research. It covers the main steps in quantitative research including developing and validating research instruments. It provides examples of quantitative research methods such as experiments, surveys, and statistical analysis. It also discusses important considerations for developing valid and reliable questionnaires such as question format, question wording, pilot testing, and response rates. The key aspects of instrument development discussed are validity, reliability, question design, and sampling methodology.
This purpose of this workshop is to facilitate novice participants through the typical steps recommended for the design of a research survey (or questionnaire). The focus in on the design and development of surveys; it is not about data analysis.
School of Social Sciences1Course title Research Methods (C3.docxjeffsrosalyn
School of Social Sciences
1
Course title: Research Methods (C39RE)
Professor Rania Kamla [email protected]
Dr Bing Xu [email protected]
Dr Esinath Ndiweni [email protected]
Course Introduction
Research Methods develops research skills which are essential for your dissertation and for a career in accounting, finance or research. It is designed to help you to prepare for the dissertation by allowing you the opportunity to learn about approaches to research and how to use them.
To research is to methodically search for new knowledge and/or practical solutions in the form of answers to specific questions. Developing skills in providing robust, convincing answers to different questions is critical to success in many professional, as well as, academic settings. This course will provide you the opportunity to understand how to gather relevant data/evidence, how to analyse and interpret evidence, how to make sense of complex situations, how to draw conclusions or make recommendations and how to communicate your finding.
A series of lectures will provide you with a sound grounding in social science research methods and guide you through the assigned readings. This will be complemented with tutorials/workshops to develop a range of practical research skills. The assessments provide you with an opportunity to apply the knowledge and provide a sound foundation for the honours year, in particular the dissertation.
Aims and Objectives
· To provide the knowledge and skills required to conduct a sizeable piece of independent research
· To generate an understanding of the social science research process
· To develop an appreciation of the nature of accountancy and finance research
· To enable students to practice their skills and test their understanding of aspects of the research process
Learning Outcomes
· Understand differences between qualitative and quantitative research
· Select and develop a research topic in accountancy or finance
· Identify, access and evaluate literature relevant to the research topic
· Select an appropriate research methodology
· Obtain relevant data
Personal Abilities
· Work independently
· Analyse and interpret research results
· Communicate and present ideas effectively by written and verbal means
Teaching Overview
This course comprises of lectures and tutorials, detailed in the table below. The lectures will last for up to 2 hours and the tutorials are an hour long.
Week
Topic
Tutorial
1
Purpose of Research & An introduction to the dissertation
2
What to Research and how to select YOUR research Question
3
Literature Review - using other peoples’ research – reviewing literature, plagiarism and referencing.
Finding a research question
4
How to research – using library resources
Setting your dissertation aims & objectives
5
Content Analysis /Interviews most popular qualitative methods
Planning your sources of evidence
6
Reading Week Coursework Preparation
None
7
Descriptive statistics.
The Masters of Business and Technology Program offer a range of study skills webinars for students. This is the exam prep webinar. For any copyright issues or to request a takedown please contact a.chambers@unsw.edu.au. Note this work is not at present under a CCSA licence.
Palestine last event orientationfvgnh .pptxRaedMohamed3
An EFL lesson about the current events in Palestine. It is intended to be for intermediate students who wish to increase their listening skills through a short lesson in power point.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
CLASS 11 CBSE B.St Project AIDS TO TRADE - INSURANCE
Kesahan & kebolehpercayaan pembentukan instrumen, kesahan dan kebolehpercayaan instrumen
1. Instrumentasi:
Pembentukan Instrumen,
Kesahan dan
Kebolehpercayaan
Instrumen
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
2. Main Steps in Quantitative
Research:
1. Theory
2. Hypothesis
3. Research design
4. Plan/Develop/Create measures of concepts
5. Select research site(s)
6. Select research subjects/respondents
7. Administer research instruments/ collect data
8. Process data
9. Analyse data
10. Write up findings and conclusions
Powerpoint Templates
3. Examples of Quantitative Research
Methods:
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
Experiments
Social surveys
Cross-sectional
Comparative (cross-national)
Longitudinal
Content Analysis
Statistical Analysis
Official Statistics
Demography
Epidemiology
Field stimulations
Structured Interviews and Observation
4. Terms “survey” and “questionnaire” are
often used interchangeably
“Survey” refers to a descriptive
research technique or methodology
(also includes interviews, and focus
groups)
“Questionnaire” refers to a paper and
pencil instrument through which
information is obtained
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
“Survey”/Tinjauan
5. • Useful for surveying groups of any size
• Typically designed to determine
opinions, attitudes or present practices
• Limitation: rely on self report
information v. observed behavior
• Challenges to validity become a
concern
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
6. Validity: the extent to which the test measures
what it significances/meaning/senses to
measure
Internal Validity: Relates to instrument
(questionnaire) quality. Are the appropriate
questions asked clearly and logically?
External validity: can the results be
generalized?
Measurement error: difference between the
characteristics of your sample and the
characteristics of the population
Non-response bias: answers of respondents
do not mirror the characteristics of those
who refused participation.
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
Validity
7. Create a questionnaire that is clear
and logical
Content Relevance And Content
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
Representativeness
Content Relevance: are the questions relevant to
the purpose of the questionnaire? Unimportant
content should be eliminated.
Content Representativeness: Are the questions an
adequate representation of the universe of
possible questions?
Solicit the input of experts in scale development
and the content of the questionnaire for an
objective evaluation
8. QUESTIONNAIRE PLANNING
• Determine the Objective
– Is the development of a new questionnaire
• Necessary?
• Feasible ?
(practicable/viable/achievable/possible..etc)
– What do you want to ask and of whom?
– Consider how the data will be analyzed. What
information is needed?
• Select mode of administration
– Web Based?
– Paper and pencil?
– Combination?
Powerpoint Templates
9. QUESTIONNAIRE PLANNING
• Determine the Sampling Methodology
– To whom will you send the questionnaire? Who can supply
the information?
– Will everyone be surveyed?
• ALL current students? ALL alumni?
• If yes, this is a census of the finite population
– The selection of a representative sample of the population
may be more effective based on time and cost of
administration
• Decision depends on the size of the finite population
Powerpoint Templates
10. CONSTRUCTING the QUESTIONNAIRE
• Consider what objective each question measures
• Three to five questions per objective is recommended
• How many questions?
– Find a balance: long enough to obtain the necessary
information but short enough so respondents won’t lose
interest.
– Questionnaires that are too long won’t be completed
– Goal: 25-30 minutes MAX
Powerpoint Templates
11. DESIGNING THE
QUESTIONNAIRE
• Question Formats: Open Ended
Questions
– Allow respondents to express feelings and expand on ideas.
– Question should be phrased to avoid a one word response.
• “What aspects of the Sociology program best prepared
you for your current position?”
– Open ended questions at the beginning of the questionnaire
may be discouraging to respondent – include one or two at
the end
– Analysis of responses can be challenging
Powerpoint Templates
12. QUESTIONNAIRE
DEVELOPMENT
• Question Formats: Closed Questions
– Ranking
• Forces the respondent to rank order responses based on a
value judgment.
• Rank the following items with regard to how you spend time
at Bucknell from 1 (spend the most time) to 5 (spend the
least time).
_____ Studying
_____ Socializing
_____ Volunteering
_____ Working at a part time job
_____ Exercising
Powerpoint Templates
13. QUESTIONNAIRE
DEVELOPMENT
• Question Formats: Closed Questions
– Checklist
• A number of possible answers are provided and the
respondent is asked to choose one or choose all that apply.
• How did you finance your Bucknell University education?
Choose all that apply.
_____ Parental contribution
_____ Student Loan
_____ Work Study
_____ Off campus employment
_____ Other (please specify): _____________
Powerpoint Templates
14. QUESTIONNAIRE
DEVELOPMENT
• Question Formats: Closed Questions
– Scaled items
• Very common
• Indicate strength of agreement or disagreement with a
statement.
• Numbers are assigned to each response on the continuum
• The English curriculum at Bucknell University adequately
prepared me for my current position.
Strongly
1 2 3 / No
4 5
Disagree
Disagree
Neutral Opinion
Agree
Strongly
Agree
Powerpoint Templates
15. QUESTIONNAIRE
DEVELOPMENT
• Question Formats: Closed Questions
– Suggestions for writing scaled items
• Use positively worded or negatively worded items
consistently
• Use an odd number of responses along the continuum to
allow for a neutral response
• Anchors may be different based on focus of the item
(Strongly Agree – Strongly Disagree; Frequently – Never;
Very Good – Very Poor)
• Keep items short
Powerpoint Templates
16. QUESTIONNAIRE
DEVELOPMENT
• Question Formats: Categorical
– Typical demographic questions:
• Yes/No/?, Male/Female, F/S/J/S/G
– Answers fall into certain categories – respondents pick the one
that applies to them
– Analyzed using frequency, proportion, percentages
– 2 considerations:
• Questions on a continuum should stay on a continuum if they
will be directly analyzed
– Example: Age – allow respondent to write in their age
rather than check off a box for their age range
• Categories must be collectively exhaustive & mutually
exclusive (next slide)
Powerpoint Templates
17. QUESTIONNAIRE
DEVELOPMENT
• Problematic:
• Number of hours spent
studying per week:
Powerpoint Templates
– 18 – 21
– 21 – 24
– 24 – 27
• Categories are not
mutually exclusive
• Corrected:
• Number of hours spent
studying per week:
– 18 – 21
– 22 – 25
– 26 – 29
18. IMPORTANT
CONSIDERATIONS
• APPEARANCE AND DESIGN
– Have clear directions that are age appropriate & easy to
follow
– Format, size, reproduction should facilitate completion
• No less than 12 pt
• Times, Arial, Courier
Powerpoint Templates
• PILOT STUDY
– Give the questionnaire to a few people to try it out
– Have them evaluate readability and clarity of questions
– Estimate time required to complete the questionnaire
– Suggestions for revisions
19. SENDING IT OUT
• INITIAL CONTACT / COVER LETTER
– Short & to the point
– Identify person by name
– Convince respondent that participation is
important
– Ensure confidentiality
– Deadline for return of questionnaire
– SASE for return (mailed questionnaires only)
Powerpoint Templates
20. FINAL THOUGHTS
• FOLLOW UP PROCEDURES
– Send to all participants to preserve confidentiality (“If you
haven’t already returned the questionnaire…..”)
– May include a second copy if available
– Usually done in 2 week intervals
• RESPONSE RATE FOR MAILED QUESTIONNAIRES
– Typically low - 30% is considered good
– Higher response rate with web-based questionnaires
– Good follow-up increases response rate
Powerpoint Templates
21. SUMMARY OF STEPS
1. Determine the Objective (consider analyses)
2. Determine the Mode of Administration
3. Determine the Sampling Methodology
4. Construct the Questionnaire
5. Institutional Approval
6. Conduct the Pilot Study
7. Write the Initial Communication
8. Send the Questionnaire
9. Follow up
10. Analyze the Results
Powerpoint Templates
22. Research design and methods
Powerpoint Templates
Develop your own
research design
and methods of
your study
23. DEVELOPING QUALITATIVE
INSTRUMENT
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
24. Main Steps in Qualitative
Research:
1. General research question
2. Select relevant site(s) and subjects
3. Collection of relevant data
4. Interpretation of data
5. Conceptual and theoretical work
6. Tighter specification of the research question
7. Collection of further data
8. Conceptual and theoretical work
9. Write up findings
Powerpoint Templates
25. Examples of Qualitative Research
In-depth Interviews
Focus Groups
Ethnography/Field Research
Historical-Comparative Research
Discourse Analysis
Narrative Analysis
Media Analysis
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
26. RELIABILITY & VALIDITY OF
INSTRUMENT - QUANTITATIVE
INSTRUMENT
Powerpoint Templates Mengilham harapan, mencipta masa depan
Inspiring Futrures, Inspiring Possibilities
54. Core Qualitative Methods
• Semi- or Un-structured, Open-Ended:
– In-depth Interviews
(in the field, face-to-face)
– Participant Observation
(field/site visits)
– Archival Research
(document review and analysis)
Powerpoint Templates
55. Guiding Principles
• Qualitative research designs consider
ways to foster:
– Reflexivity (an ongoing process of reflecting on
the researcher’s subjective experience, ways to
broaden and enhance this source of knowing, &
examining how it informs research)
– Iteration (a spiraling process: sequential and
repetitive steps in examining preliminary findings
for the purposes of guiding additional data
collection and analysis)
– Intersubjectivity (a process of reaching a
shared/ “objective” agreement about how to assign
meaning to a social experience - with insiders and
outsiders)
Powerpoint Templates
56. The Iterative Process of Qualitative
Research: A Model
Powerpoint Templates
DDaattaa CCoolllleeccttiioonn
RReefflleeccttiioonn
AAnnaallyyssiiss
57. Qualitative Research Techniques
• Instrumentation:
– Key Informants (question development and
piloting of instrument)
– Unstructured to Semi-structured
– Probing
– Data Processing and Analytic Tools
Powerpoint Templates
58. Qualitative Research Techniques/
Considerations
Powerpoint Templates
• Sampling
– Single v. Multiple Cases (not an individual)
– Expert and Key Informants (identification and
recruitment of sample)
– Roles of the Researcher (identification and
recruitment of sample)
59. Qualitative Research Techniques
• Data Collection
– Participants as Data Collectors
– Field Notes (personal reflections, observations,
emerging concepts/theories)
– Debriefing (a participant, a participating
researcher, a non-participating researcher)
Powerpoint Templates
60. Qualitative Research Techniques
• Analysis
– Key Informant Feedback
– Codebooks (specifies definitions and
relationships of concepts and terms)
– Memos (emerging patterns, concepts;
documentation of analytic pathways)
– Case Analysis Meeting (a meeting of a
research team for the purposes of reflecting on
analytic process, tools, and findings)
– Matrices or Diagrams (to identify and examine
time sequencing, the structure of relationships,
conditions of Powerpoint cross case Templates
events)
61. Qualitative Research Techniques
• Analysis
– Key Informant Feedback
– Codebooks (specifies definitions and
relationships of concepts and terms)
– Memos (emerging patterns, concepts;
documentation of analytic pathways)
– Case Analysis Meeting (a meeting of a
research team for the purposes of reflecting on
analytic process, tools, and findings)
– Matrices or Diagrams (to identify and examine
time sequencing, the structure of relationships,
conditions of Powerpoint cross case Templates
events)
62. Three Qualitative Methods:
Ethnography
• Focus: study human behaviour in the
cultural context in which it is embedded
• Ethnography is the work of describing a
culture – the way of life of a cultural
group
– Associated with Cultural Anthropology
• Example: David Counts and Dorothy
Ayer Counts: “An Ethnography of RVing
Seniors”
Powerpoint Templates
63. Three Qualitative Methods:
Grounded Theory
• Focus: develop a theory to explain underlying
social processes of a cultural group
• Useful in areas where little is known or when a
new perspective is needed
– Used for exploratory, descriptive studies
• Because the theory emerges from the data, it
is said to be grounded in the data
– Foundation in Symbolic Interactionism
• Example: Kerry Daly and the social
construction of fatherhood
Powerpoint Templates
64. Three Qualitative Methods:
Phenomenology
• Focus: reveal the meaning of the lived experience from
the perspective of participants
• Describe the essences of lived experience
– Essences: elements related to the true meaning of
something that gives common understanding to the
phenomenon under study
– Conveyed with descriptive language
– Drawn from Philosophy; used across disciplines
• Example: J.E. Solchany: A phenomenological study of
women’s preadoptive experiences
Powerpoint Templates
65. Data Collection Methods
in Qualitative Studies
• Three data collection strategies
introduced:
1. Participant observation
2. In-depth interviews
3. Focus group interviews
• Qualitative researchers may combine
more than one method
Powerpoint Templates
66. Data Collection Methods:
1. Participant Observation
• Intensive, usually long term, examination
of a social group, an organization, etc.
• Researcher becomes a participant in the
lives of group members
– Observes their behaviour and learns meaning
systems (which are tied to language)
• Most closely associated with Ethnography,
as developed in Classical Anthropology
• Now done in a variety of disciplines
Powerpoint Templates
67. 1. Participant Observation (cont’d)
• Many classic participant observations studies
– E.g., Asylums (Erving Goffman), Tally’s
Corner (Elliot Liebow), Street Corner
Society (William F. Whyte), etc.
• Today most ethnographers take an overt role
– I.e., their identity as a researcher is known
to the people being studied
• Covert participation (i.e., identity concealed
from participants) is fraught with ethical issues
– e.g., Humphrey’s “Tearoom Trade”
Powerpoint Templates
68. Steps Involved in
Participant Observation Research
A. Gaining entry into the group
B. Developing and maintaining rapport
C. Developing a method for taking field
notes
D. Integrating data collection and data
analysis
Powerpoint Templates
69. Steps in Participant Observation:
A. Gaining Entry into the Group
• Take into consideration the type of group
– formal organizations require formal entry;
involves letter writing, permission requests,
etc.
– Informal groups – different strategy needed
• Access may be gained through a gatekeeper
(an individual with special status)
• Want to involve key informants (those who are
most knowledgeable about the group)
Powerpoint Templates
70. Steps in Participant Observation:
B. Developing/Maintaining Rapport
• Researcher must work hard to develop
and maintain good relationships in the
field
– E.g., be sure not to become associated
with one faction in a group or organization
• Researcher could be blamed for
problems that arise in the setting
Powerpoint Templates
71. Steps in Participant Observation:
C. Strategies for Taking Field Notes
Field notes – integral to participant observation
• Include descriptions and interpretations of individuals,
interactions, and events
– Distinguish descriptions from interpretations
• Record time and location of observations, as well as
key information (weather, events happening and their
significance)
• Keep theoretical memos – which are the tentative
interpretations emerging and being assessed through
further data collection
Powerpoint Templates
72. Field Notes (cont’d)
• May not be possible or advisable to
take notes while in the field
– Important that they be done as soon after
field observation as possible
• Note-taking is time-consuming because
it is integral to guiding the data
collection and continuing the analysis
– E.g., field notes for When Prophecy Failed
were well over 1,000 typed pages
Powerpoint Templates
73. Steps in Participant Observation:
D. Integrating Data Collection and
• Organizing field noteAsn inatloy dsiifsferent types of files
facilitates data analysis
• Master field file – complete journal of field notes;
number pages and include entry dates
• Background, history file – subfile organizing
background material
• Key character files – subfiles on key players in
the group or organization
• Analytic files – subfiles for different types of
observations or relationships
Powerpoint Templates
74. Data Collection Methods:
2. In-depth Interviews
• Some studies cannot employ the participant
observation method
– E.g., Desroches’s study of bank robbers
• In-depth interviews allow participants to describe
their experiences and the meaning of events
taking place in their lives
– Verbatim quotes capture the language and
meaning expressed by participants
• Interviews are flexible and allow for probing
– Interview method is quite diverse, adaptive
Powerpoint Templates
75. 2. In-depth Interviews (cont’d)
• Spradley – three key elements for the interview
method to be successful
1. Explicit purpose – researcher and informant are
aware that the discussion has a purpose
2. Ethnographic explanations – researcher tries
out explanations on the participants to see if
they make sense
– Encourage the informants to use colloquial
language, and teach the researcher its
meaning
Powerpoint Templates
76. 2. In-depth Interviews (cont’d)
3. Ethnographic questions include:
i. Descriptive questions – ask participants to
describe their experiences (e.g., their ideas,
circumstances, viewpoints, dilemmas, etc)
ii. Structural questions – ask participants how they
organize their world (e.g., activities)
iii. Contrast questions – ask participants what is
meant by specific terminology
• Prus & Grills stress the value of the interview method
in a multi-method approach
Powerpoint Templates
77. Data Collection Methods:
3. Focus Group Interviews
• Interview format, but in a group setting
– 6-12 participants with common experience
• Dates back to the 1940s – used to assess
effectiveness of morale-boosting radio shows
– 1970s onward – used by market researchers
– 1980s onward – used by academics
• Transcript of discussion is the data
– Plus accompanying notes
– Use content analysis or grounded theory
approach to analyze the data
Powerpoint Templates
78. 3. Focus Group Interviews (cont’d)
Powerpoint Templates
• Strengths:
– Open-ended question
• Spontaneously deal with issues as they arise
– Cost-effective method of collecting data
– Less time-consuming
• Weaknesses:
– One or two participants may dominate
– Not done in a natural setting, so little “observation” to
help understand the experience of the participants
79. Implications for Handling Threats
to Validity and Reliability
• In quantitative research, threats to
validity are addressed by prior design
features (such as randomization and
controls)
Powerpoint Templates
80. Implications for Handling Threats
to Validity and Reliability
• In qualitative research, such prior elimination of
threats to validity is less possible because:
– qualitative research is more inductive, and
– it focuses primarily on understanding particulars
rather than generalizing to universals.
• Qualitative researchers view “threats” as an
opportunity for learning
- e.g. researcher effects and bias are part of the story
that is told; they Powerpoint are not “Templates
controlled for”
81. Validity
• In general, validity concerns the degree
to which an account is accurate or
truthful
• In qualitative research, validity concerns
the degree to which a finding is judged to
have been interpreted in a correct way
Powerpoint Templates
82. Assessing the Validity of
Qualitative Research
Can another research read your field (and other types
of) notes (i.e., the explication of your logic) and come
to the same understandings of a given phenomenon?
• Concern about validity (as well as reliability) is the
primary reason thick description is an essential
component of the qualitative research enterprise
• Resources:
– Handout: Different Types of Notes
– Example: ACY Site Visit Toolkit
Powerpoint Templates
83. Major Types of Validity in
Qualitative Research
• Descriptive Validity
• Interpretive Validity
• Theoretical Validity
• External Validity (i.e., generalizability)
Powerpoint Templates
84. Descriptive Validity
• Concerned with the factual accuracy of an
account (that is, making sure one is not
making up or distorting the things one hears
and sees)
• All subsequent types of validity are
dependent on the existence of this
fundamental aspect of validity
Powerpoint Templates
85. Descriptive Validity
Behavior must be attended to, and with
some exactness, because it is through
the flow of behavior – or, more
precisely, social action – that cultural
forms find articulation.
Powerpoint Templates
Geertz 1973:17
86. Interpretive Validity
• Interpretive accounts are grounded in
the language of the people studied and
rely, as much as possible, on their own
words and concepts
• At issue, then, is the accuracy of the
concepts as applied to the perspective
of the individuals included in the
account
Powerpoint Templates
87. Interpretive Validity: Design
Consideration
• While the relevant consensus about the terms used in
description rests in the research community, the
relevant consensus for the terms used in
interpretation rests, to a substantial extent, in the
community studied
• An important design element, for increasing
interpretive validity, therefore, is to employee, at
some level/to some degree, a participatory research
approach (e.g., through member checks, peer to peer
research model, etc.)
Powerpoint Templates
88. Theoretical Validity
• Theoretical understanding goes beyond concrete
description and interpretation; it’s value is derived
based on its ability to explain succinctly the most
amount of data
• A theory articulates/formulates a model of
relationships as they are postulated to exist between
salient variables or concepts
• Theoretical validity is thus concerned, not only with
the validity of the concepts, but also their postulated
relationships to one another, and thus its “goodness
of fit” as an explanation
Powerpoint Templates
89. Major Threats to Validity
• Type I error: believing a principle to be
true when it is not (i.e., mistakenly
rejecting the null hypothesis)
• Type II error: rejecting a principle
when, in fact, it is true
• Type III error: asking the wrong
Powerpoint Templates
question
90. Triangulation: An Important
Theoretical Validity Check
• Case example: Parable of the blind men
and the elephant
Powerpoint Templates
91. Triangulation: An Important
Theoretical Validity Check
The most fertile search for validity comes from a
combined series of different measures, each with its
own idiosyncratic weaknesses, each pointed to a
single hypothesis. When a hypothesis can survive
the confrontation of a series of complementary
methods of testing, it contains a degree of validity
unattainable by one tested within the more
constricted framework of a single method.
Powerpoint Templates
Webb et al. 1966:174
92. External Validity in Qualitative
Research
• There is broad agreement that generalizability (in the
sense of producing laws that apply universally) is not
a useful standard or goal for qualitative research
• This is not to say, however, that studies conducted to
examine a particular phenomenon in a unique setting
cannot contribute to the development of a body of
knowledge accumulating about that particular
phenomenon of interest
• Consensus appears to be emerging that for
qualitative researchers generalizability is best thought
of as a matter of the “fit” between the situation
studied and others to which one might be interested
in applying the concepts and conclusions of that
study.
Powerpoint Templates
93. Enhancing External Validity
• Thick descriptions are crucial.
• Such descriptions of both the site in which the studies
are conducted and of the site to which one wishes to
generalize (or apply one’s findings) are critical in
allowing one to search for the similarities and
differences between the situations.
• Analysis of these similarities and differences makes it
possible to make a reasoned judgment about the
extent to which we can use the findings from one
study as a working hypothesis about what might
occur in another situation.
Powerpoint Templates
94. Multi-site Studies: Another Way to
Enhance Generalizability
• A finding emerging repeatedly in the study of
numerous sites would appear to be more likely to be
a good working hypothesis about some as yet
unstudied site than a finding emerging from just one
or two sites.
• A finding emerging from the study of several very
heterogeneous sites would be more robust and, thus,
more likely to be useful in understanding various
other sites than one emerging from the study of
several very similar sites.
• Heterogeneity may be obtained by creating a
sampling frame that maximizes the variation inherent
in the sample, specifically in terms of potentially
theoretically important Powerpoint dimensions
Templates
95. Reliability
• Reliability concerns the ability of different
researchers to make the same
observations of a given phenomenon if
and when the observation is conducted
using the same method(s) and
procedure(s)
Powerpoint Templates
96. Enhancing the Reliability of
Qualitative Research
• Researchers can enhance the reliability of their
qualitative research by:
– Standardizing data collection techniques and
protocols
– Again, documenting, documenting, documenting
(e.g., time day and place observations made)
– Inter-rater reliability (a consideration during the
analysis phase of the research process)
Powerpoint Templates
Editor's Notes
Validity, in a broad sense, pertains to the relationship between an account and something external to it – that is, the phenomenon/a that the account is about, whether that phenomenon/a is construed as objective reality, the constructions of actors, or a variety of other possible interpretations.
“As observers and interpreters of the world, we are inextricably part of it; we cannot step outside our own experience to obtain some observer-independent account of what we experience, Thus, it is always possible for there to be different, equally valid accounts from different perspectives.
Validity, in a broad sense, pertains to the relationship between an account and something external to it – that is, the phenomenon/a that the account is about, whether that phenomenon/a is construed as objective reality, the constructions of actors, or a variety of other possible interpretations.
“As observers and interpreters of the world, we are inextricably part of it; we cannot step outside our own experience to obtain some observer-independent account of what we experience, Thus, it is always possible for there to be different, equally valid accounts from different perspectives.
Validity, in a broad sense, pertains to the relationship between an account and something external to it – that is, the phenomenon/a that the account is about, whether that phenomenon/a is construed as objective reality, the constructions of actors, or a variety of other possible interpretations.
“As observers and interpreters of the world, we are inextricably part of it; we cannot step outside our own experience to obtain some observer-independent account of what we experience, Thus, it is always possible for there to be different, equally valid accounts from different perspectives.
Validity, in a broad sense, pertains to the relationship between an account and something external to it – that is, the phenomenon/a that the account is about, whether that phenomenon/a is construed as objective reality, the constructions of actors, or a variety of other possible interpretations.
“As observers and interpreters of the world, we are inextricably part of it; we cannot step outside our own experience to obtain some observer-independent account of what we experience, Thus, it is always possible for there to be different, equally valid accounts from different perspectives.
Primary understanding follows from capturing accurately “acts” as opposed to “actions”; not trying to ascribe meaning to what is being observed (yet)
Just trying to capture validly activities seen as physical or behavioral events rather than in terms of the meanings that these have of the actor or others involved in the activity (or for the researcher for that matter)
This is not to say that what the researcher is observing has not in some way been “bracketed”
Describing (i.e., reporting what one sees, hears, touched, smelled, etc.) is a fundamental aspect of the scientific endeavor; the truthfulness of these accounts: primary descriptive validity.
Secondary descriptive validity: the validity of inferences derived from the data
Defining types of validity in terms of procedures: an approach generally labeled instrumentalist or positivist
Understanding is a more fundamental concept for qualitative research than validity
According to Kirk and Miller, “Asking the wrong question actually is the source of most validity errors. Devices to guard against asking the wrong question are critically important to the researcher. Diversity of method is a strong candidate for such a device” (1986:30).
Defining types of validity in terms of procedures: an approach generally labeled instrumentalist or positivist
Understanding is a more fundamental concept for qualitative research than validity
Validity, in a broad sense, pertains to the relationship between an account and something external to it – that is, the phenomenon/a that the account is about, whether that phenomenon/a is construed as objective reality, the constructions of actors, or a variety of other possible interpretations.
“As observers and interpreters of the world, we are inextricably part of it; we cannot step outside our own experience to obtain some observer-independent account of what we experience, Thus, it is always possible for there to be different, equally valid accounts from different perspectives.
Validity, in a broad sense, pertains to the relationship between an account and something external to it – that is, the phenomenon/a that the account is about, whether that phenomenon/a is construed as objective reality, the constructions of actors, or a variety of other possible interpretations.
“As observers and interpreters of the world, we are inextricably part of it; we cannot step outside our own experience to obtain some observer-independent account of what we experience, Thus, it is always possible for there to be different, equally valid accounts from different perspectives.