This paper evaluates the impact of a teaching methodology aimed at enhancing student learning and self-assessment skills in a large-class flipped learning environment involving a cohort of First Year students in Economics. We develop an analytic framework to investigate four distinct features of the learning process, comprising: (i) students’ ability to self-assess their performance, (ii) the association between learning gains generated by Peer-instruction and student self-assessment statements, (iii) the correlation between learning gains and student attainment, and (iv) students’ perceptions of their learning experience.

1. Peer-Instruction Unveiled:
Measuring Self-Assessment Skills
and Learning Gains in a Large
Flipped Learning Environment
Fabio R. Aricò
Duncan Watson
APT – July 2015

2. ACKNOWLEDGEMENTS
Chris Thomson – UEA alumnus and Research Assistant
UEA-HEFCE Widening Participation Teaching Fellowship
HEA – Teaching Development Grant Scheme
2

3. OUTLINE
1. Peer-Instruction in a Large Flipped Learning Environment
2. Measuring Self-Assessment
 attainment and confidence levels
 objective and subjective measures of confidence
3. The effectiveness of the Peer-Instruction pedagogy
4. Student appraisal of the Peer-Instruction pedagogy
5. Summary of empirical findings.
3

4. ETHICAL REMARK
You will be presented with data collected during teaching sessions.
Students involved have given informed consent for me to analyse
their responses and present the results of this analysis.
I can assist with ethical queries as well, please ask me.
4

5. 1. Peer-Instruction in
A Large Class Flipped
Learning Environment
5

6. TEACHING PROTOCOL – the module
Introductory
Macroeconomics  Level 1 – compulsory year-long module - 170 students
Lectures  traditional frontal-teaching (10 per sem.)
Seminars  small group, pre-assigned problem sets (4 per sem.)
Workshops  large group, problem-solving sessions (4 per sem.)
Support Sessions  non-compulsory drop-in sessions (4 per sem.)
6

7. FLIPPED CLASS and PEER-INSTRUCTION
• Flipped classroom & Peer-Instruction  pre-reading + student interaction
 Mazur (1997)
 Henderson and Dancy (2009)
 well-developed research in Physics and STEM
• Learning analytics for Peer-Instruction
 Learning gains: Mazour Group - Bates & Galloway (2012)
 Student satisfaction: Hernandez Nanclares & Cerezo Menendez (2014)
• There is no literature on the links with self-assessment skills
Open field, with many unanswered questions
 e.g. role of demographics, language, previous background
 Pedagogically: self-assessment blends with flipping and Peer-instruction.
7

8. WORKSHOPS – learning environment
8
Round 1
- formative question
- 4 choices
- no information
- no answer
Self-Assessment
- confidence question
- 4 level Likert-scale
- information shared
Peer-Instruction
- students talk
- compare answers
- explain each other
Round 2
- formative question
- Identical to R1
- information shared
- correct answer
Are students correctly self-assessing?
Is Peer-Instruction working?

9. 2. Self-Assessment Skills
PART A – attainment and confidence
9

10. WORKSHOPS – data coding
For each session (7 in a year in 2013-14):
• Code 1st response: 1 = correct
0 = incorrect
• Code confidence in response: 1 = strongly/agree
0 = strongly/disagree
• Compute average score and average confidence per student.
attainment self-assessment
measure measure
10

11. WORKSHOPS – data coding
For each student (in each session):
If student average score > session average score  high-attainment
otherwise low-attainment
If student average conf. > session average conf.  high-confidence
otherwise low-confidence
11

12. WORKSHOPS – data analysis
Cross-tabulate results:
12

13. WORKSHOPS - results
What is the relationship between attainment and confidence?
13

14. WORKSHOP - results
What is the relationship between attainment and confidence?
14

15. 2. Self-Assessment Skills
PART B – confidence and entropy
15

16. WORKSHOPS – data coding and modelling
For each 1st response question (4-10) across all sessions (7 in a year in 2013-14):
• Compute entropy index across responses (A,B,C,D)  objective measure
of confidence
• Compute average confidence per question  subjective measure
of confidence
Model: confidence = f ( entropy ) + session-dummies
16

17. WORKSHOPS – results
confidence = f ( entropy ) + session-dummies
17

18. 3. Peer-Instruction and
Learning Gains
18

19. WORKSHOPS – learning gains
For each 1st and 2nd response question (4-10) across all sessions
(7 in a year in 2013-14):
Learning Gain = % correct R2  % correct R1
effectiveness
of
Peer-Instruction
Model: Learning Gain = f ( % correct R1, confidence ) + session-dummies
19

20. WORKSHOPS – results
LearGains = f ( % R1, Conf ) + session-dummies
20

21. 4. Student appraisal of
Peer-Instruction
21

22. WHAT DO STUDENT THINK?
• The literature on Peer-Instruction is far too focused on whether
students ‘enjoy’ their experience (student satisfaction)
 typical of academic practice literature.
• I tried to give more focus on the perception of learning:
 1st lecture: introduced the concept of Peer-Instruction
asked the students to share what they think of it.
 each workshop: asked students to share their view on the session
and whether they felt they learnt from each other.
 informal end-of-module feedback: what was the most effective component
of the blended learning environment mix within the module.
22

23. 23
1st lecture: “‘Peer-instruction’ sessions (students teaching
each other) are more effective than lectures
(teacher teaching students)”
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
strongly agree
agree
disagree
strongly disagree
No. Respondents = 82

24. 24
0%
10%
20%
30%
40%
50%
60%
70%
80%
Wk 4 Wk 6 Wk 8 Wk14 Wk16 Wk18 Wk20
strongly agree agree disagree strongly disagree
No. 89 38 72 69 52 39 40
Workshop feedback statement: “I have learnt more
Economics by discussing answers with my classmates”

25. 25
Comparing student opinion about Peer-Instruction as
an effective pedagogy before and after exposure
0%
10%
20%
30%
40%
50%
60%
70%
1st Lecture (N=82) Avg Workshop (N=57)
strongly agree
agree
disagree
strongly disagree

26. 6%
20%
51%
4%
6%
13%
Lectures
Seminars
Workshops
Support
VLE
NA
26
End-of-module Feedback: What is the component of the Macro
module which had the strongest impact on your learning?

27. 5. Summary of
Empirical Findings
27

28. SUMMARY of FINDINGS
• Positive association between attainment and confidence in performance
 this formative assessment design elicits good self-assessment outcomes
• Negative association between entropy and confidence levels
 objective and subjective measures of confidence align
• Negative association between learning gains and % correct R1
 Peer-Instruction supports low-performers – ‘catching up effect’
• No association between learning gains and confidence
 confidence levels do not influence the effectiveness of Peer-Instruction
• Students seem to recognise the power of Peer-Instruction
 consistent opinions across different sources of feedback.
28

29. CURRENT and FURTHER RESEARCH
• Corroborate and strengthen empirical analysis and methodology.
 homogenise methodologies used for self-assessment and learning gains
 further robustness checks on empirical findings.
• “Assessing Self-Assessment”
 compare and contrasts 2 different learning environments
 assess the relationship between attainment and confidence.
• Learning analytics at student-level
 investigate the role of demographics
 investigate the impact of formative assessment on summative assessment.
29