2. Generating Topics
Translate ideas into valid and reliable
ways of measuring them
Collect evidence
Unique Topics
Innovative but Difficult
3. Reliability and Validity
The collecting of data (measurement)
and doing research always raises the
issues of reliability and validity. The
issue of reliability is essentially the
same for both measurement and
research design. Reliability attempts to
answer our concerns about the
consistency of the information collected,
while validity focuses on accuracy.
4. Reliability and Validity -- Relationship
The relationship between reliability and validity
can be confusing because measurements and
research can be reliable without being valid, but
they cannot be valid unless they are reliable.
For a study to be valid it must consistently (reliability) do
what it purports to do (validity).
For a measurement to be judged reliable it should
produce a consistent score.
For the research study to be considered reliable each
time it is replicated it too should produce similar results.
5. Definition; Reliability
Reliability is the consistency of your
measurement, or the degree to which an
instrument measures the same way each
time it is used under the same condition with
the same subjects.
It is important to remember that reliability
is not measured, it is estimated.
6. Ways to Estimate Reliability
Test/retest is a conservative method to
estimate reliability. The three main components
to this method are as follows:
implement your measurement instrument at two
separate times for each subject
compute the correlation between the two separate
measurements
assume there is no change in the underlying
condition (or trait you are trying to measure)
between implementation
7. Estimating Reliability
Internal consistency estimates reliability by
grouping questions in a questionnaire that
measure the same concept.
One common way of computing correlation values
among the questions on your instruments is by
using Cronbach's Alpha.
Cronbach's alpha splits all the questions on your instrument
every possible way and computes correlation values for
them all (SPSS will do this).
Like a correlation coefficient, the closer the value is to one,
the higher the reliability estimate of your instrument.
8. Definition; Validity
Cook and Campbell (1979) define
validity as the "best available
approximation to the truth or falsity of
a given inference, proposition or
conclusion."
Basically, were we right?
9. Thought to Ponder
It is my belief that validity is more
important than reliability because if an
instrument does not accurately measure
what it is supposed to, there is no reason
to use it even if it measures consistently.
10. Measurement Concepts
Measurement is the process of assigning numbers to
represent the amount of a variable (a characteristic,
attribute, trait present in a person, object, situation
under study). Measurement results that contain little
error are said to be reliable.
Sources of measurement error include
the instrument (eg, improper calibration)
the environment (eg, noise level)
the researcher (eg, fatigue, mood)
data processing (eg, data entry error)
11. How to find a topic?
Curiosity and Experience
Too big or Narrow
Assignments, Theses, and Grants
RFPs, RFAs, Work related assignments
Other Research Findings
Scholarly articles, secondary sources, replication,
‘filling in the hole’
Serendipity (by accident)
A finding that you were not expecting
12. What is a Research Grant?
Research Grants and contracts are written
agreements with external sponsors. They contain
one or more of the following provisions:
A research protocol or other statement of work
A designated principal investigator(s)
A designated period of performance
A budget
Obligation to account for costs & return unspent $
Disposition of intellectual property rights
13. Searching for Research
Internet
Academic versus Nonacademic
Library Databases
Ask New Questions
Once have articles, use those references
Popular newspapers/magazines
14. Let’s do some group work
What is/are the best way(s) to ‘find’ research
ideas? Where did you find your survey idea?
Explain inductive and deductive reasoning. Is
either more prominent in survey research?
How is a literature review helpful in conducting
survey research?
List ethical issues related to survey research.
Discuss one issue related to your survey research
idea.
15. Literature Reviews
Evaluate Previous Research
Create a database (or collection)
Attention to Methodology
Sampling
Questions/Hypotheses
Variables
Measurement
Analyses
Conclusions
Limitations
16. Once you assess
Do you want to replicate?
Same measures or modifications
Making changes limits the comparisons
Are there themes? Or links?
Organize the literature (Chapter 10)
17. Writing a Literature Review
Your literature review should reflect the
important thinking in the area that will
impact your work, and should provide a
context for the background and importance of
the question. You should identify existing
knowledge and the gaps in the knowledge,
and indicate methodologies that have been
used in other similar research questions. The
literature review is often included as part of
your research proposal.
18. Two levels of Review
Conducting a literature review
Your research investigation of the literature
Writing a literature review
The review you write for your own project
19. Literature Review Sources
Looking for resources, start here:
http://libraries.uky.edu/
http://web.pdx.edu/~dbls/HowtoWriteLiteratureReview.htm
20. Theory and Reasoning
Theory (Nardi, 2006): a set of statements
logically linked to explain some
phenomena in the world around us
21. Deductive Reasoning
Deductive reasoning works from the more general
to the more specific.
It is informally called a "top-down" approach.
We might begin with thinking up a theory about our
topic of interest. We then narrow that down into
more specific hypotheses. We narrow down even
further when we collect observations to address the
hypotheses. This leads us to be able to test the
hypotheses with specific data; a confirmation (or
not) of our original theories.
22. Inductive Reasoning
Inductive reasoning moves from specific
observations to broader generalizations and
theories.
Sometimes called a "bottom up" approach
In inductive reasoning, we begin with specific
observations and measures, begin to detect
patterns and regularities, formulate some tentative
hypotheses that we can explore, and finally end up
developing some general conclusions or theories.
23. Deductive Versus Inductive
"Deductive reasoning" refers to the process of concluding that
something must be true because it is a special case of a general
principle that is known to be true.
If you know the general principle that the sum of the angles
in any triangle is always 180 degrees, and you have a
particular triangle in mind, you can then conclude that the
sum of the angles in your triangle is 180 degrees.
"Inductive reasoning" is the process of reasoning that a general
principle is true because the special cases you've seen are true.
If all the people you've ever met from a particular town have
been very strange, you might then say "all the residents of
this town are strange".
24. The Ethics of Research
These are designed by your governing
institution, granting agencies, organizations,
and yourself
One such Ethics Standards are presented by
AERA
http://www.aera.net/AboutAERA/AERARulesPolicies/C
odeofEthics/tabid/10200/Default.aspx
Another is specific to Survey Research
http://www.casro.org/?page=TheCASROCode
25. IRB
Institutions that conduct research set
up IRB; Institutional Review Board
University of Kentucky
Office of Research Integrity
http://www.rgs.uky.edu/ori/
