This document summarizes key concepts from chapters 4 and 5 of a research methods textbook. It discusses reliability and validity, describing reliability as the consistency of measurements and validity as how accurately a study measures its intended concept. It also outlines different types of reliability like test-retest and internal consistency, as well as types of validity like content, criterion, and construct validity. Finally, it reviews observational research methods like naturalistic observation, participant observation, and field experiments.
4. Reliability
The consistency or repeatability of the
measurements
Validity
The degree of accuracy to which a study reflects
or assesses the specific concept that the
researcher is attempting to measure
4
6. Stability and Equivalence
• Test-retest reliability
• Alternate-form reliability
Version Version Reliability
• Internal-consistency reliability
1 1
Equivalence
Version Version
2 2
time 6
7. Internal Consistency
How related are individual items within the
measure?
• Spearman-Brown
• Kuder and Richardson Formula 20 (KR20)
• Cronbach’s alpha
7
8. Replication in Research
• What do we mean by ‘replication’?
• Three factors to consider:
– When
– How is the replication is conducted?
– By whom
8
9. Replication in Research
• What do we mean by ‘replication’?
• Three factors to consider:
– When
– How is the replication is conducted?
– By whom
Early replications are more valuable than later
ones 9
10. Replication in Research
• What do we mean by ‘replication’?
• Three factors to consider:
– When
– How is the replication is conducted?
– By whom
The difference between precise and varied
replication 10
11. Replication in Research
• What do we mean by ‘replication’?
• Three factors to consider:
– When
– How is the replication is conducted?
– By whom
Independence of replicators
11
12. Validity
• Content-related
– How well a measure represents all dimensions
• Criterion-related
– Concurrent vs. predictive validity
• Construct-related
– Convergent vs. discriminant validity
12
14. Observational Methods
• Categories based on intervention by
researcher
– Observation without Intervention
– Observation with Intervention
• Categories based on methods for recording
behavior
– comprehensive record
– selected behaviors
15. Observation without Intervention
• Naturalistic Observation
– observation in natural (real-world) setting without
attempt to intervene or change situation
– use when ethical considerations prevent
experimental manipulation
• Goals
– describe “normal” behavior, examine relationships
among naturally occurring variables
– establish external validity of lab findings
16. Observation with Intervention
• Most psychological research involves
intervention
• Three methods in natural settings
– participant observation
– structured observation
– field experiment
17. Observation with Intervention
• Participant observation
– observer is active participant in the natural setting
he or she observes
• undisguised: people know they’re being observed
(e.g., Kitchen Stories, 2003 Norwegian film)
• disguised: people don’t know they’re being observed
18. Observation with Intervention
• Problems with participant observation
– Reactivity
• when people change their usual behavior because
they’re being observed
• disguised participant observation controls reactivity
– Observers lose objectivity or become too involved
in situation
– Observers influence behavior of people they’re
observing (e.g., Hawthorne Effect)
19. Observation with Intervention
• Structured observation
– set up (structure) specific situation in order to
observe behavior
– used when behavior is difficult to observe as it
naturally occurs
– researchers use confederates to structure
situations
– problems: when observers don’t follow same
procedures across observations
20. Observation with Intervention
• Example of structured observation
– Simons and Levin (1998): “change blindness”
http://www.youtube.com/watch?v=FWSxSQsspiQ
21. Observation with Intervention
• Field Experiment
– manipulate independent variable in natural
setting and observe behavior (dependent variable)
• two or more conditions to compare (IV)
• often use confederates to create conditions
• strive for control in natural setting
22. On Being Sane in Insane Places
Original Article:
http://cooley.libarts.wsu.edu/soc3611/Documents/Being
_Sane_in_Insane_Places.pdf
Video:
http://www.youtube.com/watch?v=Kq-7uvVOoyk
In Fiction:
Simpson’s Episode S03E01 – “Stark Raving Dad“
Dr. David Rosenhan
22
23. Maximising Serendipity
• With
– Common sense
– Hard work
– The right info
– And a bit of luck
Raising the flag on Iwo Jima - 1945
23
24. Errors in Observational Research
• Non-interactional artifacts
– Interpreter biases
– Observer biases
• Interactional artifacts
24