This document discusses managing a paradigm shift from a traditional assessment system to a more formative approach at Casa Thomas Jefferson language school in Brazil. The traditional system involved high-stakes summative exams every 10 weeks that focused on grammar and caused student anxiety. A pilot program implemented more frequent, lower-stakes assessments throughout each module to provide ongoing feedback. While most teachers and students supported the changes, some resisted due to beliefs about how assessment and learning should work. The document emphasizes examining underlying mental models to better manage resistance to change.

1. Managing a paradigm shift in
assessment – a matter of
mindset
Isabela Villas Boas, Casa Thomas Jefferson, Brasilia, Brazil

2. Grade oriented
Private school ranking
Public school annual
evaluation
Culture of teaching to the
test
Fierce competition to get
into a good university –
national examination

3. Casa Thomas Jefferson at a glance
Founded in 1963
16,000 + students
Six branches + school contracts + in-
company
Around 200 teachers
Children - adults
Basic - post-advanced
Course Supervision – standardization
Balanced assessment system, but
with a greater emphasis on
summative tests

5. The “Flex” courses
Around 3,000 Ss
Only adults
Four ten-week modules a
year
Typical adult students

6. Traditional assessment system
After ten weeks of instruction:
One day for written test: reading comprehension,
grammar, vocabulary; about 60% of the test
focused on grammar structures and functions
One day for oral test: done in pairs; students stand
outside the classroom and wait for their turn
(following the proficiency test model)

7. How it all started - E-teacher course
Assessment
Summative
and
Formative
Practices in
Language
Learning
and
Teaching

8. Traditional assessment system
Written test + oral test at the
end of ten-week module
Ten weeks of instruction:
accumulation of content
Focus on grammar and written
exercises
Student anxiety
More selected-response
items on the test
Lack of alignment
Negative washback
effect

9. Summative
Assessment
Formative
Assessment
Traditional
Assessment
Alternative
Assessment
Aims of the new assessment system
Very little feedback
Actionable
feedback
Poor alignment Better alignment

10. New assessment system
1 2 3 4 5 6
Short assessments - 10 to 20 minutes
2 oral assessments, necessarily
At least 1 reading, 1 listening, and 1 writing
Mostly performance assessment of grammar
100
points
Last day of class: Final assessment day
Ss take the assessments they missed during the module or that
they would like to retake (to improve outcomes)

13. Traditional tests administered at
the end to compare scores;
comparison of final grades with
typical student performance
Constant student
feedback
Constant teacher
feedback
Formal survey
with students at
the end

14.  Students had similar scores on their
assessments and on the summative
test
 Students had similar or higher final
grades than the final grades in previous
modules
 Students felt more at ease with the
pilot system; less pressure

15. Building a shared vision - E-teacher course
Assessment
Summative
and
Formative
Practices in
Language
Learning and
Teaching

18. Phase-in stage

19. Implementation
Workshops on
assessment
Ongoing
feedback on
the outcomes
of the new
system
Class
observations
Revision of
assessment
tools
Feedback from
teachers and
students at
the end of the
module

21. New system
Old system
Students

22. More communication in class
Content is not accumulated
No need to memorize rules
Need to study more frequently = more effective learning
Less stressful
More accurate assessment of the student because it is
ongoing
The oral assessments are less stressful
No last-minute studying only
Lighter
“Forces” students to come to class
Makes me feel more comfortable in class
Students

23. Fewer assessments
A final test is more effective in measuring knowledge
Have the six assessments + final test
Too easy

24. Fully Satisfactorily Partiallly No
Understand reasons for
change
89% 6% 0% 0%
Support change 61% 17% 6% 0%
Teachers

25. No accumulation of content to assess.
Allows for remedial work and teacher self-assessment.
Student progress is more visible.
Immediate and personalized feedback to students.
Encourages more teacher reflection.
Students are assessed at their best.
Less stressful for students.
Possibility to re-teach and re-assess.
Teachers

26. Too many assessments.
Students keep asking if there will be an
assessment the next class.
A little more time-consuming for teachers.
Difficulty with the oral assessments.

27. Two pilot
groups
Two modules of each course
Four modules of each course
All levels: all teachers

28. The assessment system is appropriate for
the students’ profile.
Totally Most of
the time
Partially No No response

30. Change in staff

31. Few but very loud complaints
Too many assessments; too much time spent on assessments. Why
not have four or three at the most?
Assessments are too easy. Students get higher grades than they
used to; they won’t be ready for the next level.
Sometimes a whole class is spent on the oral assessment.
Grammar is not being assessed.

32. Our mental models –
theories about the
way the world works
– influence our
actions.
Examine mental
models: Unexamined
mental models limit
people’s ability to
change.
Schools that Learn, Peter Senge

33. Expecting an assessment will
capture excellence,
mediocrity, and failure as an
educational measure
reinforces a mindset that
some students will always fail.
That mindset has no place in
our schools.
How Can Schools Develop Assessments That Matter?
By Jill Berkowicz and Ann Myers, March 22, 2016
Education Week

34. Too many
assessments; too
much time spent on
assessments. Why
not have three or
four at the most?
It is okay to spend
two full class days
on oral and written
tests, but it doesn’t
seem okay to spend
this same amount
of time on smaller
doses of
assessment.
Belief Mental Model

35. It’s too easy. Students
get higher grades
than they used to;
they won’t be ready
for the next level.
Assessment as a
student ranking
tool;
Testing drives
learning;
Quality = difficulty.
Belief Mental Model

36. Sometimes a whole
class is spent on the
oral assessment.
It is okay to spend a
whole class on fill-
in-the blanks
grammar exercises,
but it’s not okay to
spend a whole class
practicing speaking
and then being
assessed on it.
Belief Mental Model

37. Grammar is not being
assessed.
Grammar should be
assessed by way of
discrete grammar
items.
Belief Mental Model

38. Example of assessment: Speaking
Students will be able to
talk about past
experiences related to
music and expand the
conversation by
providing details on the
experience.

39. Ask your partner some questions about musical experience. Begin with a “have you
ever” question and then ask some follow-up questions. Below are some ideas, but you
can create your own questions about experiences related to music.
Have you ever been to a rock/jazz/pop concert?
Have you ever played in a band?
Have you ever traveled just to go to a concert?
_____________________________________________________________?
Instructions

40. Students practice as much as they
need until they feel ready;
Teacher can:
- Walk around the classroom
and assess pairs
- Ask pairs to present to the
class when they are ready
- Pairs record dialogues.
- Teacher assesses students by
way of “real-time, almost
surreptitious recording of
student verbal and nonverbal
behavior”(Brown, 2004. p.
267).

42. Assessment Course for Teachers

43. Assessment
Consultants

45. Develop a growth mindset in teachers and
students
Challenge mental models
Build a shared vision
Create a learning community
Managing change

46. CHANGE ONE TEACHER AND ONE
STUDENT AT A TIME