Review of "Survey Research Methods & Design in Psychology"
Upcoming SlideShare
Loading in...5
×
 

Review of "Survey Research Methods & Design in Psychology"

on

  • 18,246 views

Reviews the 150 hour, third year psychology unit which examined survey research methods, with an emphasis on the second-half of the unit on MLR, ANOVA, power, and effect size.

Reviews the 150 hour, third year psychology unit which examined survey research methods, with an emphasis on the second-half of the unit on MLR, ANOVA, power, and effect size.

Statistics

Views

Total Views
18,246
Views on SlideShare
17,993
Embed Views
253

Actions

Likes
9
Downloads
799
Comments
0

11 Embeds 253

http://ucspace.canberra.edu.au 175
http://www.slideshare.net 65
http://www.e-presentations.us 5
https://sanjacinto.blackboard.com 1
http://health.medicbd.com 1
http://translate.googleusercontent.com 1
https://knowledge.udmercy.edu 1
https://bblearn.uidaho.edu 1
http://pinterest.com 1
https://www.edu-apps.org 1
http://www.pinterest.com 1
More...

Accessibility

Upload Details

Uploaded via as OpenOffice

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • 7126/6667 Survey Research & Design in Psychology Semester 1, 2011, University of Canberra, ACT, Australia James T. Neill Home page: http://ucspace.canberra.edu.au/display/7126 Lecture page: http://ucspace.canberra.edu.au/display/7126/Lecture+-+Review Notes page: http://en.wikiversity.org/wiki/Survey_research_methods_and_design_in_psychology Image name: Recycle symbol Taiwan.svg Image source: http://commons.wikimedia.org/wiki/File:Recycle_symbol_Taiwan.svg Image author:Bryan Derksen http://commons.wikimedia.org/wiki/User:Bryan_Derksen License : Public domain Description: Reviews this semester-long (150-hour), third year undergraduate psychology research unit which focused on survey research methods and survey design. This lecture emphasises the second-half of the unit's content on MLR, ANOVA, significance testing, power, and effect size, as well as providing advice about the lab report and final exam assessment items.
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain Image source: http://commons.wikimedia.org/wiki/File:Autoroute_icone.svg License: CC-BY-A 2.5 Author: http://commons.wikimedia.org/wiki/User:Doodledoo
  • http://en.wikiversity.org/wiki/Survey_research_and_design_in_psychology/Overview
  • Image source: James Neill, Creative Commons Attribution-Share Alike 2.5 Australia, http://creativecommons.org/licenses/by-sa/2.5/au/
  • Image source: Unknwn
  • Image source: James Neill, Creative Commons Attribution-Share Alike 2.5 Australia, http://creativecommons.org/licenses/by-sa/2.5/au/
  • Image source: James Neill, Creative Commons Attribution-Share Alike 2.5 Australia, http://creativecommons.org/licenses/by-sa/2.5/au/
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Compare models across groups % variance explained No. of factors Item loadings
  • Reversing a scale e.g., IM = mean(item1,item2,item3) EM = mean (item4,item5,item6) M = IM + (8 – EM) 1 2 3 4 5 6 7 7 6 5 4 3 2 1
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: James Neill, Creative Commons Attribution-Share Alike 2.5 Australia, http://creativecommons.org/licenses/by-sa/2.5/au/
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain
  • Image source: http://commons.wikimedia.org/wiki/File:Information_icon4.svg License: Public domain

Review of "Survey Research Methods & Design in Psychology" Review of "Survey Research Methods & Design in Psychology" Presentation Transcript

  • Lecture 12 Survey Research & Design in Psychology James Neill, 2011 Survey Research & Design in Psychology: Review & Summary
  • Overview
    • Review
      • Unit aims and outcomes
      • Research process
      • Survey design
      • MLR, ANOVA, ES & Power
      • What type of analysis? (decision tree)
    • Assessment
      • Lab report
      • Final exam
    • Evaluation & feedback
  • Unit aims & outcomes
  • Aims & outcomes
    • Knowledge & skills for conducting ethical , well-designed , survey-based research in psychology.
    How confident are you that could conduct a good quality survey-based research study? For 4 th year Honours? In the work-place?
  • Aims & outcomes
    • Theory & practice of survey-based research, incl.:
      • Research questions / hypotheses
      • Survey design
      • Sampling
      • Interpreting & communicating results
  • Aims & outcomes
    • Use of SPSS for:
      • Data entry
      • Correlations
      • Factor analysis
      • Qualitative analysis
      • Reliability
      • MLR
      • Advanced ANOVA
  • Research process
  • Research process 1. Establish need for info/ research 2. Problem definition/ Hypotheses 3. Research design 4. Sampling/ Data collection 5. Data analysis 6. Reporting
  • Funnel model
  • Survey design
  • Survey design
    • Operationalisation of fuzzy concepts
    • Question types (objective/subjective, open/closed)
    • Response formats (e.g., dichotomous, frequency, Likert, multiple response, idiographic)
    • Levels of measurement (LOM)
    • Reliability & validity
    • Sampling (probability/non-probability)
    • Modes of administration (self-report (f2f, mail, web), interview (f2f/phone))
  • Items should measure different aspects of the latent (underlying) construct Latent Construct Measured Construct
  • Latent Construct Poor items will create ‘brown sludge’ (noisy (unreliable) measure) Measured Construct
  • Describing data
  • Describing data
    • Data screening (out of range, invalid etc.)
    • Discrete data:
      • Frequencies & %s
    • Continuous data: 4 moments of a normal distribution
      • Central tendency
      • Dispersion
      • Skewness
      • Kurtosis
  • Visualisation of data
    • Aids interpretation of descriptives and tests of differences or relationships.
    • Univariate :
      • histogram, bar graph, error-bar graph
    • Bivariate :
      • scatterplot, clustered bar graph
    • Multivariate :
      • Venn diagrams, multiple line graph, 3-d scatterplot
  • Software for data visualisation (graphing)
    • Statistical packages
      • e.g., SPSS Graphs or via Analyses
    • Spreadsheet packages
      • e.g., MS Excel
    • Word-processors
      • e.g., MS Word – Insert – Object – Micrograph Graph Chart
  • Correlations
  • Correlations
    • Strength & direction of bivariate linear relations
    • Building block for FA & MLR
    • Non-parametric correlations (e.g., Point bi-serial, Phi/Cramer's V )
    • Scatterplots – watch out for:
      • Outliers
      • Non-linearity
      • Limited range
    • Caution with causal interpretation
  • Factor analysis
  • Factor analysis
    • Purpose
      • Data reduction
      • Developing reliable & valid measures of fuzzy construct
    • Assumptions
      • Linear relations
      • Sample size (min. 5 cases: 1 variable)
  • Factor analysis
    • Extraction
      • PC vs. PAF
    • Rotation method
      • Varimax vs. Oblimin
    • Number of factors
      • Kaiser’s criterion
      • Scree plot
      • Theoretical structure
      • Interpretation of factor loadings
  • Factor analysis
    • Decide how many factors
    • Iteratively eliminate items
      • Communality > .5?
      • Primary loadings > .5?
      • Cross-loadings < .3?
      • Diff. btw primary & cross-loadings > .2?
      • Sufficient items per factor
      • Face validity
    • Correlations between factors
    • Compare model across groups
  • Reliabilities & composite scores
    • Internal reliability : For each factor, calculate Cronbach’ s  :
      • > .8 = very good
      • > .7 = good
      • > .6 = OK
    • Composite scores
      • Unit-weighting
      • Regression-weighting
  • Qualitative analysis
  • Qualitative analysis
    • Purposes
      • Pilot study, exploratory research
      • Theory testing; Data reduction (synthesise meaning)
      • Validity-testing
    • Methods
      • Quantitative (Content analysis, Multiple response analysis, Graph (e.g., bar graph)
      • Qualitative (Thematic analysis)
  • Inferential statistical decision making tree
  • Statistical decision tree
    • Establish a research question and/or hypothesis
      • Differences or relationships?
      • No. of IVs and DVs
      • Identify levels of measurement
    • Use a statistical decision tree to identify an appropriate analysis
    • See Inferential statistics decision-making tree
  • Multiple linear regression
  • Multiple linear regression
    • L inear regression formula
    Y hat = ax + b Y = ax + b + e
    • Proportion of variance in a DV explained by one or more IVs
      • R , R 2 , Adjusted R 2
      • F (significance of R )
      • Change in R 2 (Hierarchical)
  • Multiple linear regression
    • Assumptions :
      • LOM
        • Continuous DV
        • Dichotomous or continuous IVs
      • Normality (helps satisfy other assumptions)
      • Linearity
      • Homoscedasticity
      • Multicollinearity
      • Multivariate outliers
  • Multiple linear regression
    • Methods
      • Standard / Direct
      • Hierarchical
      • Forward
      • Backward
      • Stepwise
    • Overall hypothesis :
      • (Null) That the IVs do not explain variance in the DV (i.e., that R is 0)
    • + one hypothesis per predictor :
      • (Null) That each IV is not a significant predictor of variance in the DV (i.e., that t for each predictor is less than the critical value)
    Multiple linear regression
    • Consider/interpret :
      • Direction of each predictor
      • Size of each predictor (β - standardised)
      • When IVs are correlated, interpret zero-order vs. partial correlations
      • sr 2 = % variance in DV explained by each IV
    • Can use Venn or path diagrams to depict relationships between variables
    Multiple linear regression
  • ANOVA
  • ANOVA
    • Extension of t -test
    • Like MLR , ANOVA involves:
      • One continuous DV (although ANOVA can handle multiple DVs)
      • One or more IVs
    • Unlike MLR , in ANOVA:
      • Interactions are automatically tested
      • IVs must be categorical
      • Significant results may indicate need for followup or post-hoc tests
  • Types of ANOVA
    • 1-way ANOVA
    • 1-way repeated measures ANOVA
    • 2-way factorial ANOVA
    • Mixed design ANOVA (Split-plot ANOVA)
    • ANCOVA
    • MANOVA
  • ANOVA
    • Assumptions
      • Cell size > 20 (Ideal)
      • Normally distributed DVs ( for each level of the IVs )
      • Homogeneity of variance (b/w subjects)
      • Sphericity (w/in subjects)
    • Follow-up tests:
      • Post-hoc tests
      • Planned comparisons
    • Effect sizes : η 2 , η p 2 and Cohen’s d
  • Power, effect size, significance testing, publication bias & academic integrity
  • Significance testing Significance testing has dominated psychology, but is problematic, mainly because:
      • Results are dichotomous (sig. or not), which doesn't help us to understand the size of effect.
      • Sig. test results are influenced by power esp. if particularly high or low.
  • Power & effect sizes Power and effect sizes have been neglected. Therefore:
      • Calculate the power of studies (prospectively &/or retrospectively)
      • Report ESs & CIs to complement inferential statistics
        • r, r 2 , R , R 2
        • η 2 , η p 2 , d
  • Publication bias & academic integrity Publication bias (low power; favouritism of sig. findings; funnel plots) Academic integrity - “Integrity is doing the right thing, especially when no one is watching”.
  • Lab report
  • Lab report - Tips
    • Check the lab report guidelines and the marking criteria
    • Use example HD lab report and sample write-ups as guides
    • Demonstrate your knowledge via independent thinking/work
    • Tell a story
  • Lab report - Structure
    • Cover Sheet
    • Title Page
    • Abstract
    • Body
    • References
    • Appendices
  • Lab report - Introduction
    • Explain the rationale – what is the problem to be solved?
    • No waffle – cut to the chase – only review literature or argument relevant to the RQs / hypotheses
    • State clear RQs and/or hypotheses – One per test/analysis/effect
  • Lab report - Method
    • Well-organised (like a recipe)
    • Present relevant details efficiently (avoid getting bogged down in extraneous detail)
  • Lab report - Method
    • Sections:
      • Participants
      • Materials or Instrumentation
      • Procedure
    • Replicable? A naïve reader must be able to replicate the study
  • Lab report – Results
    • Data screening
    • Consider LOM assumptions
    • Caution in use of overall scores
    Overall Score not valid Overall Score valid 1 3 2 1 3 2
  • Lab report – Results (EFA)
    • EFA :
      • PC/PAF?
      • Varimax/Oblimin?
      • 2-6 factors?
      • 5-30 items removed?
      • 50%-60% of variance?
    • Table of factor loadings and communalities
    • Correlations between factors
    • Internal consistency for each factor
  • Lab report – Results (MLR)
    • Example MLR : Hierarchical:
      • DV = Campus Satisfaction
      • Step 1
        • IV1 = Gender (M / F)
      • Step 2
        • IV1 = Planning TM (Continuous)
        • IV2 = Time wasting TM (Continuous)
      • R 2 , Adjusted R 2 , Change in R 2
    • Table of correlations and regression coefficients
  • Lab report – Results (ANOVA)
    • ANOVA : e.g., 2 x (3) Mixed ANOVA
      • Between-subjects IV: Enrolment Status (FT / PT)
      • Within-subjects DV: Satisfaction (Educational / Social / Campus)
    • Table of cell and marginal descriptives ( M , SD , Sk, Kurt) + Graph
    • Effect sizes ( η p 2 , d )
  • Lab report – Results (Qualitative)
    • Example Qualitative analysis : Least or most satisfactory aspects of UC
      • Research question?
      • Qualitative or quantitative approach?
      • Table of main themes (names, description, example quotes, frequency/%)
      • Bar graph?
      • Thick description?
      • Sub-samples?
  • Lab report - Discussion
    • Provide insight about the results
    • Draw conclusions about the RQs & hypotheses in light of the results.
    • Discuss key strengths & limitations of the study. ( Balanced criticism )
    • Draw out implications and recommendations
  • Lab report - Discussion
    • Offer specific, practical recommendations e.g.,
      • Theory : What are the implications for the theory/rationale upon which the study was based?
      • Methods : How could the research design (e.g., instrumentation) be improved?
      • Practice : Implications for students and universities e.g., for improving satisfaction?
  • Lab report – Appendices
    • Optional : Include appendices where relevant and referred to in the body text. Appendices may not be consulted by a reader, so if its important make sure key content is covered in the text .
    • Use for content which would break the flow , but which is relevant to understanding the study e.g., the EFA correlation matrix.
  • Lab report – Appendices
    • APA style not necessary.
    • Use headings (e.g., Appendix A, B, C etc.) and possibly titles e.g.,
    Appendix A: Bivariate correlations amongst the university student satisfaction items
  • Lab report - Submission
    • Insert & complete the Cover Sheet
    • Submit ONE DOCUMENT containing the coversheet, lab report and appendices
    • NO EXTRA ATTACHMENTS
    • Upload via the Moodle drop-box
    • You can re-submit up until the due date (after that, resubmission will incur late penalties)
  • Lab report - Marking
    • Lab report marks should be returned before the exam .
    • Keep an eye on your student email and Moodle Announcements .
  • Final exam
  • Final exam
    • Computer-based , supervised
    • Style, structure and content will be similar to the quizzes
    • ~ 120 multi-choice questions in 180 mins (plenty of time).
    • Practice exams are available
  • Final exam
    • Permitted materials
      • Non-programmable calculator
      • Non-annotated foreign language dictionary
    • Marks and feedback
      • Upon submitting, you will receive your exam mark and feedback.
  • Evaluation & feedback
  • Evaluation & feedback: USS Unit Satisfaction Survey
    • Available now on OSIS – but you may wish to wait until after assessment results
    • UC takes these results very seriously – ~40% response rate
  • Possible issues & topics Please contribute your honest feedback:
    • Lectures?
    • Tutorials?
    • Textbooks?
    • Assessment?
    • Website(s)?
    • Software – SPSS?
    • Workload?
  • Open Office Impress
    • This presentation was made using Open Office Impress.
    • Free and open source software.
    • http://www.openoffice.org/product/impress.html