A Critique of the Proposed National Education Policy Reform
Asp2013
1.
To connect to the internet:
If you have a laptop or tablet and a gmail account
visit: http://sjsuriot.appspot.com/
Sign in with your gmail account
We will be using this application during part of the
workshop
Don’t have a laptop or tablet? No problem! You can
still fully participate.
Welcome!
3.
Courses using Interactive-Engagement techniques have
been shown to improve instruction. (Hake 1998; Crouch &
Mazur 2001; Prather et al. 2009; and many others)
Many classroom observation protocols are developed for
researchers/evaluators to use, and not for instructors.
(Examples: Active-Learning Inventory Tool, Amburgh et al. 2007; Reformed
Teaching Observation Protocol, Sawada et al. 2002)
A poor ‘score’ on a protocol can cause a negative overall
evaluation experience for the instructor.
The UC Davis Physics Education Research group wanted
to make an observing tool that:
Was easy for instructors to use
Gave instructors lots of data on their classroom
Let instructors make their own judgments on how to change
Motivation
4.
Real-time Instructor
Observing Tool
UC Davis team developed
RIOT
Categories created for easy
coding
Instructors see a visual
representation of data
collected in their classroom
No video
RIOT data can be discussed in
an professional development
environment, consultation or
private reflection
Clarifying Instructions
Explaining Physics
Listening to Question
Closed Dialogue w/students
Open Dialogue w/ students
Open Dialogue with Ideas
Passively Observing Students
Actively Observing Students
Checking Homework
Out of Room
Fixing Apparatus
Not Interacting/Reading
Chatting with Students
*West, E. A., Paul, C. A., Potter, W. H., Webb, D. Variation of
instructor-student interactions in an introductory interactive physics
course Phys. Rev. ST Phys. Educ. Res., Vol. 9 (March 2013)
5.
Computerized Classroom Observing Protocol
Observer watches class
Clicks icons pertaining to interaction taking place
Report is auto-generated at the end
Designed to be used by and for instructors to inform teaching
Interaction categories are (relatively) easy to interpret
Initially developed for Physics TAs at UC Davis
Also used as research tool*
Little training required
After today you should be ready to go!
(More training required if using RIOT for research purposes.)
Real-time Instructor
Observing Tool
*West, E. A., Paul, C. A., Potter, W. H., Webb, D. Variation of instructor-student interactions in an introductory interactive
physics course Phys. Rev. ST Phys. Educ. Res., Vol. 9 (March 2013)
6.
Outline
Intro
Using RIOT:
How to make observations
How to download and view results
Interpreting RIOT reports:
Small groups: Look at example data from courses
Whole class discussion: group findings and
applications
Identifying Interactions:
Practice coding
9. Clarifying Instructions
Explaining Physics
Listening to Question
Closed Dialogue w/students
Open Dialogue w/ students
Open Dialogue with Ideas
Passively Observing Students
Actively Observing Students
Checking Homework
Out of Room
Fixing Apparatus
Not Interacting/Reading
Chatting with Students
Time in minutes
TA is Interacting w/ Whole Class
TA is interacting w/ Group 1
TA is interacting w/ Group 2
TA is not Interacting
Sum of all Group rows
TA is interacting with individual
R.I.O.T. OUTPUT EXPLAINED BY ROW
TA is interacting w/ Whole Class during time
when students are in small groups
11.
CLASP
COLLABORATIVE LEARNING THROUGH ACTIVE SENSE-MAKING IN PHYSICS
2 pieces of
CLASP
curriculum
Time spent
in class per
week:
Interactivity: Number of
Students:
Instructors:
Lecture 1x 80 minutes
(25 min Quiz)
(sometimes)
Peer-Instruction
~150
students (
x2 per
course)
Usually faculty,
sometimes
lecturer or
advanced grad
Discussion-
Lab
2 x 140
minutes
Series of
interactive
activities spliced
with whole class
discussions
25-30
students
(x11 per
course)
The vast
majority are
grad students.
12. Sodium
7A DLM 17 3/12/2008
Silicon
7A DLM 17 3/11/2008
Clarifying Instructions
Explaining Physics
Listening to Question
Closed Dialogue w/students
Open Dialogue w/ students
Open Dialogue with Ideas
Passively Observing Students
Actively Observing Students
Checking Homework
Out of Room
Fixing Apparatus
Not Interacting/Reading
Chatting with Students
17.
Dr. Tom Fleming – University of Arizona
intro astronomy class
http://www.youtube.com/watch?v=LIa5hjVAAD8
&list=PL19CE26ACB414B35B
Lecture Clip
19.
Benefits of RIOT
Coarse measurement allows for lots of observations
Output gives you an illustrative view of classroom (you
can learn a lot of things about the instructor in
seconds, our eyes respond to patterns)
Excellent TA training tool
Not as invasive as video tape
instructors more likely to allow (invite!) observations
(objective & anonymous)
Easier to act naturally in front of
Students not video taped (IRB exempt)
Can be modified to measure MANY things
Instantly turns qualitative data into quantitative data for
statistical analysis
20.
Weaknesses of RIOT
Not a replacement for video tape
Coarse observation
Info on quality lost (in current form)
Only gives you info regarding what TA is doing (student
info is lost)
You can’t go back and re-analyze interactions
(Next step to see if Active Observing is a true indicator for
student achievement is to see what happens before and after
active observing)
21.
Future uses for RIOT
Professional development tool (faculty love data!)
Organizer for field notes (new feature!)
Small Group analysis (identifying common patterns in
student discourse)
Look for issues with the curriculum (for example: places
where instructors explain a lot)
As a ‘tag’ for videotape data
PRIME grant: Student Participation Observing Tool
(SPOT)
30.
Instructors spend between 1% and 75% of their whole
class discussion time explaining to their students.
WHOLE CLASS TIME
30
31.
RIOT Findings
There is a large variation in the range of instructor-
student interactions between instructors
32.
THE AMOUNT OF TIME
SPENT IN WHOLE CLASS
DISCUSSION VARIES IN
EACH OF THE THREE
SEGMENTS OF THE COURSE.
THE REGULARITY OF
CERTAIN
INTERACTIONS, LIKE
DIALOGUING, VARIES IN
EACH OF THE THREE
SEGMENTS OF THE COURSE
34.
RIOT Findings
There is a large variation in the range of instructor-
student interactions between instructors
The curriculum affects how instructors spend their
time in class.
By changing the materials, developers can affect the
amount of time instructors spend interacting in
certain ways
38.
RIOT Findings
There is a large variation in the range of instructor-
student interactions between instructors
The curriculum affects how instructors spend their time
in class.
By changing the materials, developers can affect the amount
of time instructors spend interacting in certain ways
There is evidence that suggests that certain instructor
interactions are correlated with student achievement
(active observing in particular)
How many of you use: clickers, think pair share, peer instruction, or some student polling technique during class? How many of you use small group activities? How many of you teach labs or discussions? How many of you have discussions will individual students during class? This observing protocol will give you detailed information on what types of interactions you are having with your students, and what percentage of class time you are devoting to each one.
First version of the RIOT ran on expensive software. Andrew here re-wrote the program to work on google app engine, and it’s now free for everyone to use. Well, it’s free for you and ALMOST free for us. Andrew will now show you how it works. Imagine that you are walking into the classroom of a colleague, and you
Tell participants they can follow along
Did not observe the lecturers at all, data is from DLs
Handout
Handout
Do any of you do educational research?
Switch this with next slideMake a web app, I pad
What is striking about this is that there don’t seem to be ‘categories’ of TAs. They really exhibit a spectrum of behaviors
Arrange better (maybe two slides)titleThe point I’m trying to get across in this data, is that TAs do all kinds of things in this classroom that was developed to be an interactive engagement classroom.Determine what is motivating instructors to lecture during the interactive engagement pieces and ignore their students, and we need professional development for that.
The point I’m trying to get across in this data, is that TAs do all kinds of things in this classroom that was developed to be an interactive engagement classroom.\
