Inquiry based approaching to teaching and assessing Psychological Research Methods Richard Rowe Department of Psychology University of Sheffield

Inquiry Based Learning Learning through self-directed exploration Open-ended scenarios Different approaches may provide valid solutions Students choose the methods employed

Learning through self-directed exploration

Open-ended scenarios

Different approaches may provide valid solutions

Students choose the methods employed

Transferable inquiry skills Information literacy Collaboration Interpersonal skills Reflection Independent learning

Information literacy

Collaboration

Interpersonal skills

Reflection

Independent learning

Research Methods and IBL Statistical analysis is not inherently inquiry based Psychological research IS an inquiry task Psychology students not well motivated to learn research methods Become interested during Level 3 project When using statistics as an inquiry tool

Statistical analysis is not inherently inquiry based

Psychological research IS an inquiry task

Psychology students not well motivated to learn research methods

Become interested during Level 3 project

When using statistics as an inquiry tool

The quantitative research process Topic choice Literature search Hypothesis generation Analysis Data Collection Experimental design Reporting Interpreting Write-up

Existing exam question An experimenter wants to know whether anxiety is related to running time in high level athletes…

An experimenter wants to know whether anxiety is related to running time in high level athletes…

The quantitative research process Topic choice Literature search Hypothesis generation Analysis Data Collection Experimental design Reporting Interpreting Write-up

Inquiry based revisions Combine with didactic lectures Want to cover the whole research process at Levels 1 and 2 Introduce Information literacy Hypothesis formation and experimental design Collaboration Develop Analysis and interpretation in context of research literature Transferable skills: interpersonal, communication, reflection, independence

Combine with didactic lectures

Want to cover the whole research process at Levels 1 and 2

Introduce

Information literacy

Hypothesis formation and experimental design

Collaboration

Develop

Analysis and interpretation in context of research literature

Transferable skills:

interpersonal, communication, reflection, independence

Resources available Level One 4 postgraduate led tutorials 2 coursework assignments Level Two 2 lab classes 2 coursework assignments (130 students)

Level One

4 postgraduate led tutorials

2 coursework assignments

Level Two

2 lab classes

2 coursework assignments

(130 students)

Level One Information literacy activity Topic choice Literature search Hypothesis generation Analysis Data Collection Experimental design Reporting Interpreting Write-up

Collaborative review Run in Groups of 4-5 Write single collaborative review Use of different research methods in… Topic chosen by PG tutors Face-to-face tutorial Introduce topic Introduce literature search (Web of Knowledge)

Run in Groups of 4-5

Write single collaborative review

Use of different research methods in…

Topic chosen by PG tutors

Face-to-face tutorial

Introduce topic

Introduce literature search (Web of Knowledge)

Collaborative review Search Web of Knowledge for relevant papers 1 week to find 1 each Discuss suitability with tutor via on-line discussion board 1 week to write up integrated joint collaborative review

Search Web of Knowledge for relevant papers

1 week to find 1 each

Discuss suitability with tutor via on-line discussion board

1 week to write up integrated joint collaborative review

On-line tutorial

Submission Single integrated literature review All receive the same mark Individual reflection on process of literature search and teamwork Accounted for 8% and 2% of course mark

Single integrated literature review

All receive the same mark

Individual reflection on process of literature search and teamwork

Accounted for 8% and 2% of course mark

Evaluation Introduced students to independent study gently First experience of a literature review First time asked to write without a reading list Engages students with research literature Engages students with different research methods

Introduced students to independent study gently

First experience of a literature review

First time asked to write without a reading list

Engages students with research literature

Engages students with different research methods

I feel more confident using the Web of Knowledge database 89% Agree

I feel I have improved my ability to evaluate the published literature 77% Agree

Selected qualitative student feedback “ Group work just tends to be left to one person who has to take responsibility for it.” “ The other members of my group had not put as much effort in as they should, especially one member that did a 2-second job of copying an articles abstract and writing a useless concluding section. ”

“ Group work just tends to be left to one person who has to take responsibility for it.”

“ The other members of my group had not put as much effort in as they should, especially one member that did a 2-second job of copying an articles abstract and writing a useless concluding section. ”

I enjoyed working collaboratively with my group members 75% Agree

Feedback from tutors “ Lots of questions were asked about the topic and how to reference, showing an improvement in communication between the group members… ”

“ Lots of questions were asked about the topic and how to reference, showing an improvement in communication between the group members… ”

Feedback from tutors “ I found this tutorial rewarding because it was nice to talk about my area of research and to see that this was interesting to other people” “ While I made sure to give the information clearly and concisely, there was the risk of an information overload!”

“ I found this tutorial rewarding because it was nice to talk about my area of research and to see that this was interesting to other people”

“ While I made sure to give the information clearly and concisely, there was the risk of an information overload!”

Discussion Expansion of research methods focus Value of reflection Involvement of postgraduate tutors Approach to collaboration

Expansion of research methods focus

Value of reflection

Involvement of postgraduate tutors

Approach to collaboration

Level One Design and Analysis Topic choice Literature search Hypothesis generation Analysis Data Collection Experimental design Reporting Interpreting Write-up Literature search activity

Research design tutorial Tutors present choice of research questions Collaboratively design study to test one Fill out summary sheet at end of tutorial Hypothesis , IV levels, Design choice, DV choice, confounds

Tutors present choice of research questions

Collaboratively design study to test one

Fill out summary sheet at end of tutorial

Hypothesis , IV levels, Design choice, DV choice, confounds

Analysis Workshop tutorial Collaboratively analyse data from experimental design Data not collected – time constraint Data simulated to have properties expected from collected data More variables than specified to allow all Level One Tests

Collaboratively analyse data from experimental design

Data not collected – time constraint

Data simulated to have properties expected from collected data

More variables than specified to allow all Level One Tests

Analysis Workshop tutorial Analyze and write-up data in one hour session (groups of 4): Choose statistical tests, run tests, interpret, write-up 2 Laptop computers provided (running SPSS and Word), notes and books allowed, Tutor as a resource Submit report at end of tutorial All get same mark: 10% of course

Analyze and write-up data in one hour session (groups of 4):

Choose statistical tests, run tests, interpret, write-up

2 Laptop computers provided (running SPSS and Word), notes and books allowed, Tutor as a resource

Submit report at end of tutorial

All get same mark: 10% of course

Analysis worksheet Describes variables in data set 5 questions address: T-test (related) T-test (unrelated) Pearson correlation Chi-square

Describes variables in data set

5 questions address:

T-test (related)

T-test (unrelated)

Pearson correlation

Chi-square

Example data description age (measured in years), sex (0=f 1=m) driving aggression (dragg) pre intervention scored on a questionnaire, range 0-20, higher scores means more aggressive driving aggression post intervention (dragg2). The same driving aggression scale conducted at a later date. In the interim all drivers had taken part in an anger management course. Ever had an accident (accid: 0=no 1=yes)

age (measured in years), sex (0=f 1=m)

driving aggression (dragg) pre intervention scored on a questionnaire, range 0-20, higher scores means more aggressive

driving aggression post intervention (dragg2). The same driving aggression scale conducted at a later date. In the interim all drivers had taken part in an anger management course.

Ever had an accident (accid: 0=no 1=yes)

Example questions 1. Is age related to pre-intervention driving aggression? 2. Are male drivers more aggressive than female drivers (pre-intervention)? 3. Are male drivers more likely to have been involved in an accident? 4. Did the anger management program reduce driving aggression? 5. Choose one further analysis to run based on your dataset.

1. Is age related to pre-intervention driving aggression?

2. Are male drivers more aggressive than female drivers (pre-intervention)?

3. Are male drivers more likely to have been involved in an accident?

4. Did the anger management program reduce driving aggression?

5. Choose one further analysis to run based on your dataset.

Conceptual evaluation Test interpretation and writing Also test choice of statistical test Use of SPSS Without risk of unfair collusion More like real research Collaboration, tutor as “expert” resource Group obscures individual assessment 80% course assessed via exam Too many questions in assignment!

Test interpretation and writing

Also test

choice of statistical test

Use of SPSS

Without risk of unfair collusion

More like real research

Collaboration, tutor as “expert” resource

Group obscures individual assessment

80% course assessed via exam

Too many questions in assignment!

Collaboration and tutor role Shared responsibility for daunting task Tutor helps to ensure success Mimics availability of real-world expert resource Unsure how much help to give Tutorial is a learning environment Assessment provides motivation to engage Fewer scripts to mark Easier to provide detailed feedback

Shared responsibility for daunting task

Tutor helps to ensure success

Mimics availability of real-world expert resource

Unsure how much help to give

Tutorial is a learning environment

Assessment provides motivation to engage

Fewer scripts to mark

Easier to provide detailed feedback

I feel more able to formulate research questions 69% Agree

Tutor feedback Design tutorial “ I have learned that by speaking in an enthusiastic way about an area of research, it is possible to inspire students to take a deeper level of interest.” Analysis workshop “ The atmosphere was quite different to normal and I realised I had to be very considerate of the instructions I issued.” “ The students seemed familiar enough with the material being tested, but had trouble applying their knowledge to a practical problem.”

Design tutorial

“ I have learned that by speaking in an enthusiastic way about an area of research, it is possible to inspire students to take a deeper level of interest.”

Analysis workshop

“ The atmosphere was quite different to normal and I realised I had to be very considerate of the instructions I issued.”

“ The students seemed familiar enough with the material being tested, but had trouble applying their knowledge to a practical problem.”

Tutor feedback “ The students completed the task within the set guidelines and seemed pleased with their work-although they may just have been relieved it was over!”

“ The students completed the task within the set guidelines and seemed pleased with their work-although they may just have been relieved it was over!”

The group projects helped me develop my communication skills 71% Agree

The group projects helped me develop my negotiation skills 63% Agree

I feel more confident working collaboratively with others 51% Agree

I feel more confident studying independently 60% Agree

I feel the IBL activities encouraged me to take an active role in class 74% Agree

Discussion Introducing students to research process More interactive teaching Drawbacks to IBL approach Issues in collaboration Tutor uncertainty about contribution Logistic demands

Introducing students to research process

More interactive teaching

Drawbacks to IBL approach

Issues in collaboration

Tutor uncertainty about contribution

Logistic demands

Level 2: Putting it all together Topic choice Literature search Hypothesis generation Analysis Data Collection Experimental design Reporting Interpreting Write-up

Level 2 Semester One Research proposal Analysis and report simulated data Semester Two Full experimental design and report

Semester One

Research proposal

Analysis and report simulated data

Semester Two

Full experimental design and report

Semester 1 Research proposal Work in groups of 4 during lab class 2 computers per group Choose research topic between group Search for papers on topic Design experiment to follow-up Constrained to design suitable for 1-way ANOVA Summarize proposal during class

Work in groups of 4 during lab class

2 computers per group

Choose research topic between group

Search for papers on topic

Design experiment to follow-up

Constrained to design suitable for 1-way ANOVA

Summarize proposal during class

Data generation and analysis Dataset simulated to mimic that which might have been collected Personalised versions for each student Analysis in own time 1 way ANOVA (all significant) Post hoc test (all 2 groups same, 1 different)

Dataset simulated to mimic that which might have been collected

Personalised versions for each student

Analysis in own time

1 way ANOVA (all significant)

Post hoc test (all 2 groups same, 1 different)

Semester 1 Submission Individual research proposal Comprises literature review Details hypothesis and methods Individual results write-up Write-up analysis of simulated data 10% of module mark

Individual research proposal

Comprises literature review

Details hypothesis and methods

Individual results write-up

Write-up analysis of simulated data

10% of module mark

Semester 2 activity Similar topic choice and literature search Design suitable for multiple regression required Data simulation Write up as if a real experimental study Individual analysis and write-up 20% of module mark

Similar topic choice and literature search

Design suitable for multiple regression required

Data simulation

Write up as if a real experimental study

Individual analysis and write-up

20% of module mark

Evaluation Historically research methods has lots of maths Challenge to fill the void Conceptual understanding of statistical tests? Full engagement with entire research process Inquiry based methods well suited Also develops transferable skills Further test of efficacy will be project students in 2008-2009

Historically research methods has lots of maths

Challenge to fill the void

Conceptual understanding of statistical tests?

Full engagement with entire research process

Inquiry based methods well suited

Also develops transferable skills

Further test of efficacy will be project students in 2008-2009

Acknowledgements Psychology Department Paul Norman Myles Jones CILASS Pamela McKinney Jamie Wood

Psychology Department

Paul Norman

Myles Jones

CILASS

Pamela McKinney

Jamie Wood

Inquiry based approaching to teaching and assessing Psychological Research Methods Richard Rowe Department of Psychology University of Sheffield