Assessment 2018 handout

www.ph-noe.ac.at
Assessing foreign language learners:
The contribution of
formative assessment
to teaching and learning
Not everything that can be counted counts,
and not everything that counts can be counted.
Albert Einstein
(c) Dr. Claudia Mewald1

www.ph-noe.ac.at
What are we doing when we assess?
2
What is assessment?
(c) Dr. Claudia Mewald

www.ph-noe.ac.at
• Roles of assessment
• Testing & assessment terminology
• Teaching & learning design
• Decision making processes or events
• Classroom-based assessment
3
Content

www.ph-noe.ac.at
Assessment  methods that educators use to
document, measure and evaluate the academic
readiness, learning progress and skill acquisition of
learners
Don’t take time to assess, unless you are going to take
action with what you discover. Wormeli 2006
Feedback  hold a mirror up to learners and help them
see what they did, comparing it to what they were
supposed to do
4
Assessment for learning

www.ph-noe.ac.at
What does our understanding of classroom assessment and
feedback mean for our work in the classroom?
Is classroom assessment more formative or summative in nature?
Depending on its focus, its role in informing teaching and learning
and in determining grades will be dramatically different.
5
Assessment of learning

www.ph-noe.ac.at6
Curriculum
What should
the learners
know and be
able to do?
What should
the learners
learn?
What should
the learners
be taught?
Instruction
How should
the learners
be taught?
What do the
learners know
and what are
they able to
do?
What have
the learners
learned?
Assessment
of learning

www.ph-noe.ac.at7
Coursebook
What is in
the next
unit?
Instruction
How should
the learners
be taught?
What
knowledge
and skill does
the material
require?
Assessment
of learning
© Claudia Mewald (c) Dr. Claudia Mewald

www.ph-noe.ac.at8
Curriculum
What should
learners
know and be
able to do?
What should the
learners learn?
How can we
observe what the
learners have
learned?
Assessment
GOALS &
PERFORMANCE
DESCRIPTORS

www.ph-noe.ac.at9
Curriculum
What should
learners know
and be able to
do?
What should
the learners
learn?
How can we
observe what
the learners
have learned?
Assessment
How should
the learners
be taught?
How can we
tell them what
they know and
what are they
able to do?
How should
the learners
learn?
Instruction
Assessment
for learning
GOALS
TOPICS
PERFORMANCE
DESCRIPTORS
TASKS FEEDBACK
ACTIVITIES

www.ph-noe.ac.at
• Assessment
(formal, informal, formative,
summative, alternative)
• Testing
(norm-referenced, standards-based,
criterion-referenced, achievement,
diagnostic, proficiency)
• Performance
• Competence
• Measurement
• Evaluation
• Reliability
(rater-related, test-related, student-
related)
• Validity
(content-related, construct-related,
criterion-related)
10
Assessment Terminology

www.ph-noe.ac.at
 makes use of a wide range of methodological
techniques
 to document, measure, and/or evaluate
the academic readiness, learning progress,
and skill acquisition of learners of all ages
 in an ongoing process
11
Assessment
participation
test
performance

www.ph-noe.ac.at
 a subset of assessment
 physical existence: prepared,
administered
 at predefined times in a course
 to measure and evaluate
learner performance
 instruments of evaluation
 usually graded
12
Testing/Tests

www.ph-noe.ac.at
A grade is an inadequate report of an inadequate
judgment by a biased and variable judge of the extent
to which a student has attained an undefined level of
mastery of an unknown proportion of an indefinite
material.
Source: P. Dressel (1983). "Grades: One More Tilt at the Windmill." In A. W. Chickering (Ed.), Bulletin. Memphis: Memphis State
Univ., Center for Study for Higher Education, p. 12.
13
Dressel quote

www.ph-noe.ac.at
 the action or process of performing a
language task or language function
 learners listen, speak, read, or write
 a sample of language or some kind of
response to input
 learner output that can be used to infer general
learner competence
14
Performance

www.ph-noe.ac.at
 a set of abilities, knowledge & skills,
commitment, motivation and willpower that
enable a person to act effectively
 in a wide variety of situations
 each level/stage of development has its own
requirements so that competence can occur
at any time in the learning process
15
Competence

www.ph-noe.ac.at16
COMPETENCE
abilities
drive
goals
motivation
determination
DOING
LEARNING OUTCOME
volitional
motivational
16

www.ph-noe.ac.at17
Motivation
Independence Self-direction Autonomy
DispositionCompetences are dispositions for performance  not observable
and therefore assumptions or a construct
LearningCompetences can be learnt and are distinct from inborn abilities
(talents)  competence development can be influenced by
pedagogical measures
ContextualisationCompetence development happens in the context of particular
tasks and requirements  competences are functional and targeted
at situations  performance requirements are also context-specific
Knowledge, Abilities & Skills
COMPETENCE
develops from
knowledge,
self-regulative, social,
and communicative
abilities and skills
G
O
A
L
G
O
A
L
G
O
A
L
G
O
A
L
G
O
A
L
G
O
A
L
G
O
A
L
G
O
A
L
G
O
A
L
G
O
A
L
17

www.ph-noe.ac.at
 the process of quantifying observed
performances
 to provide exact descriptions of performances
to compare them or to hold
them against a norm
 to assign numbers
 to rank order or grade
 to describe and provide
feedback
18
Measurement
What can we
measure?
We cannot measure competence,
we can only measure performance

www.ph-noe.ac.at
 the systematic determination of the worth and
significance of a performance
 uses criteria governed by a set of standards
 assists in decision-making (planning)
 helps to ascertain the degree of achievement or
value in regard to the aim and objectives
 enables reflection and
assists in the identification
of future action/change
19
Evaluation

www.ph-noe.ac.at
 the consistency of a measure
 if it produces similar results under consistent
conditions
20
Reliability

www.ph-noe.ac.at
 the extent to which a test measures what it is
supposed to measure
 the degree to which evidence and theory
support the interpretations of scores (construct)
 the extent to which a test actually
samples the subject matter (content)
 the extent to which criteria match
teaching and learning goals and results are
supported by other concurrent performances
21
Validity

www.ph-noe.ac.at
designed to measure performances against a fixed set
of criteria
criteria are often described as can-do descriptors or
standards descriptors
standards describe what learners should be able to do
at a specific stage of their education and frequently
associated with criterion-referenced (standardised)
tests
22
Criterion-orientated assessment

www.ph-noe.ac.at
The Common European Framework of
Reference & Companion Volume
ISBN:3-468-49469-6
(Langenscheidt) € 19,95
English
http://www.culture2.coe.int/
portfolio/documents_intro/
common_framework.html
https://rm.coe.int/common-european-
framework-of-reference-for-
languages-learning-
teaching/168074a4e2
23 (c) Dr. Claudia Mewald

www.ph-noe.ac.at24
24

www.ph-noe.ac.at25 (c) Dr. Claudia Mewald

www.ph-noe.ac.at26
http://www.englishprofile.org/resources/information-booklet

www.ph-noe.ac.at
CEFR, COMPANION VOLUME & COLLATED
DESCRIPTORS
27 (c) Dr. Claudia Mewald

www.ph-noe.ac.at
The CEFR “describes in a comprehensive way
what language learners have to learn to do in
order to use a language for communication and
what knowledge and skills they have to develop
so as to be able to act effectively.”
Council of Europe, 2001:1
(emphasis added)
28
Hi, I’m Luke.
I’m Jenny.

www.ph-noe.ac.at
The Framework also defines levels of proficiency
which allow learners’ progress to be measured
at each stage of learning and on a life-long basis.
Council of Europe, 2001:1
(emphasis added)
29

www.ph-noe.ac.at30
A
Basic User
B
Independent User
C
Proficient
User
C1 C2B1 B2A1 A2
A2+ B1+ B2+Pre-A1
N
E
W

www.ph-noe.ac.at
The focus on achievement is essential: only
positive formulations of levels of proficiency
are used as objectives.
31
Can introduce
him/herself and
others and can ask
and answer questions
about personal
details …
Global Scale, Council of Europe 2001:24
I can use simple
phrases and
sentences to
describe where I live
and people I know.
Self-assessment grid, Council of Europe 2001:26
Can interact in a
simple way but
communication is
totally dependent on
repetition….
Qualitative aspects of spoken language use, Council of Europe 2001:29

www.ph-noe.ac.at
In order to provide teachers and curriculum
planners with orientation points the CEFR
summarises the set of proposed Common
Reference Levels in single holistic paragraphs in a
global scale.
Council of Europe 2001:24
Common Reference Levels

www.ph-noe.ac.at
 communication activities (reception,
interaction, production)
 strategies (hinge between competences and
communicative activities)
 communicative language competences
(linguistic, pragmatic, sociolinguistic)
Illustrative descriptors
coherent
flexible

www.ph-noe.ac.at
• functional and interactional perspective on the nature
of language
• language learning always needs to be connected to the
social context it is used in
• language is seen as a medium of interaction and
communication between people who want to achieve
specific goals and purposes
• especially in situations in which the learner has to
fulfill a particular role with language skills which can
be predicted or determined for the relevant context
34
Competence Based Language Teaching

www.ph-noe.ac.at
• mosaic approach to language learning
• assumes that language can be divided into appropriate
parts and subparts
• communicative competence is constructed from these
subparts put together in the correct order
• CBLT is in some respects similar to Communicative
Language Teaching
Richards & Rodgers, 2001, p.143
35
Competence Based Language Teaching

www.ph-noe.ac.at
• learning design in CBLT differs from the traditional
approach to developing a syllabus
• CBLT is designed not around the notion of subject
knowledge but around the notion of competency
• focus on how the students can use the language
instead of their knowledge about the language
Richards & Rodgers, 2001, p.144
36
Learning design in CBLT

www.ph-noe.ac.at
• systematically designed to achieve a certain
competence
• real-world tasks which may be related to any domain
of life, especially to survival-oriented situations in a
new environment
Richards & Rodgers, 2001:144
37
Learning activities in CBLT

www.ph-noe.ac.at
• criterion-based assessment is essential for CBLT
• specific language skills learnt during the course are
assessed
• the competencies tested consist of a description of the
essential skills, knowledge, attitudes, and behaviours
required for effective performance of real-world tasks
or activities
• performance-criteria form the basis for the
assessment Richards & Rodgers, 2001, p.144
38
Assessment in CBLT

www.ph-noe.ac.at
• teacher observation
• teacher/peer-feedback
• output/products
• homework
• diagnostic tests
39
Formative assessment
assessments become
formative when the
information is used to adapt
teaching and learning to meet
the learner’s needs
Brown & Wiliam 1998b
define expected learning outcomes
= target goals
define steps on the way to target
goals = graded goals
define performance
descriptors
& assessment criteria

www.ph-noe.ac.at
design goals with
target
performance in
mind
describe evidence
for target
performance
(can-do
statements)
develop criteria to
assess evidence
plan teaching and
learning
feedback on teaching and
learning through formative
or summative assessment
making use of criteria
adjust teaching
and learning
according to
feedback based on
assessment
40
The design cycle
from goals to
teaching, learning &
assessment

www.ph-noe.ac.at
LOA
Jones, N. & Saville, N. (2017) Learning Oriented Assessment. A systemic
approach. Cambridge: CUP

Education
Assessment
Personal
world
Social
world
Adapted from Jones & Saville 2017: 4
Cognition
individual’s
development
Subjects
curricular
objectives
Constructs
of language
ability
Skills
social and
professional
Four worlds of
learning

Education
Assessment
Personal
world
Social
world
interactionfeedback
observation
TASKS
Learning
Subjects
curricular
objectives
Cognition
individual’s
development
Constructs
of language
ability
Skills
social and
professional

Education
Assessment
Personal
world
Social
world
interactionfeedback
observation
TASKS
Learning
Subjects
curricular
objectives
Cognition
individual’s
development
Constructs
of language
ability
Skills
social and
professional
Learning factors
Intrinsic motivation,
autonomy,
confidence, interest,
experience, etc.
Learning factors
Extrinsic motivation,
meaningful incentives,
opportunities, needs,
etc.
Learning factors
Formal structure,
guidance, learning
how to learn, etc.
Learning factors
Orientation,
evidence of progress,
diagnostics, formal
accreditation, etc.
Roles
Individual
learners
Roles
Head teachers,
teachers,
educationalists,
parents,
politicians, etc.
Roles
Friends,
employers,
services,
politicians,
etc.
Roles
Experts in measurement,
applied linguists,
cognitive scientists, etc.

Pre-A1
A1
A2
B1
B2
C1
C2
Quantitative (measurement)
dimension
Qualitative
(teaching and
learning)
dimension
Skills profile

High level
objectives/goals:
curriculum
(communicative
competence)
Pre-A1
A1
A2
B1
B2
C1
C2
Content:
themes, topics,
functions,
tasks
Learning
oriented
syllabus
Assessment
activity
Task
Record
(informal)
Language
activity
Observation
Interpretation/
translation
Decision-
making
Feedback,
modified
objectives
Check prior
knowledge
Record of
achievement
(report)
Frame of
Reference
(CEFR)
Structured
record
(formal)
External exam
Summative
monitoring
Interpretation
performance

www.ph-noe.ac.at
Begin with global goals
What should the learners be able to do in the future?
Example:
The pupil is able to start, maintain and end a
conversation
(to be able to communicate with other speakers of
English in the private and public domain in the future).
Goals & objectives

www.ph-noe.ac.at
Identify target goals for
the period of instruction
Which content should the
learners be helped to
understand?
Which performances should
they be helped to carry out?
Which attitudes should they
be helped to adopt?
→ content goals
(declarative/cognitive)
→ performance goals
(procedural)
→ affective goals
(social, attitudinal,
motivational)
Goals & objectives

www.ph-noe.ac.at
The learner is able to start, maintain, and end a
conversation
Goals & objectives
CONTENT GOALS PERFORMANCE
GOALS
AFFECTIVE GOALS

www.ph-noe.ac.at
The learner is able to start, maintain, and end a
conversation
Goals & objectives
CONTENT GOALS PERFORMANCE
GOALS
AFFECTIVE GOALS
Knowledge of
relevant chunks
and phrases for –
greeting,
asking for one’s
well being, saying
good bye –
personal domain
Ability to use
body language, facial
expression, voice
tonality etc. in
different situations
(e.g. well-being,
happiness, stress,
nervousness,
boredom...)
Ability and
willingness to
collaborate,
sympathise, take
on different
roles, give
constructive
feedback etc.

www.ph-noe.ac.at52
Graded goals
Target learning goal3.0
4.0 More complex learning goal
Simpler learning goal2.0
With help, partial success1.0
Even with help, no success0

www.ph-noe.ac.at
Target goal
The pupil is able to use taught chunks and phrases for
greeting, asking for one’s well being (giving positive and
negative answers) and saying good bye.
More complex goal
.... use taught chunks and phrases for greeting, asking for
one’s well being (giving varied answers), and saying good
bye using different levels of formality (e.g. with friends,
adults, foreigners ...).
Simpler goal
.... greet, ask for one’s well being (giving standard answers,
e.g. Fine, thanks. I’m OK), and say good bye in role plays.
Graded goals

www.ph-noe.ac.at54
I can greet my friends
and people I know well
and ask for their well-
being, if I can use the
phrase bank or if my
peers or teachers help
me.
Simpler goal
being.
I know one or two
phrases to do this.
Target goal
being.
I know several different
phrases to do this.
More complex goal
I use formal or informal
language when I greet
people or ask for their
well-being
appropriately.
3.0
2.0
4.0
1.0

www.ph-noe.ac.at56
Mewald 2014b

www.ph-noe.ac.at57
57
Mewald 2014b

www.ph-noe.ac.at58
Based on: Stiggins, R., Arter, J., Chappuis, J., & Chappuis, S. (2006). Classroom assessment for student learning —
Doing it right, using it well. Portland: Educational Testing Service.
Ratings into grades
Min. 50% of all ratings are 4.0, the rest is
3.0
Min. 75% of all ratings are 3.0 or 4.0, the
rest is 2.0
Min. 40% of all ratings are 3.0 or 4.0, the
rest is 2.0
Min. 50% of all ratings are 2.0 or above, the
rest is 1.0
More than 50% of all ratings are 2.0 or
below
one rating or more re 0.0
Grade
Sehr gut; A/top grade
Gut; B
Befriedigend; C
Genügend; D
Nicht genügend;
E/failure

www.ph-noe.ac.at59
59
A
B
C
D
E

www.ph-noe.ac.at
• a learner’s evaluation of his/her own success
• a learner’s evaluation of a peer’s success
• based on criteria
• known and understood by learners
• implementation checked by peers and teachers
61
Self- and peer assessment

www.ph-noe.ac.at62
Mewald 2014b
Beginning
Clear
Information
Words & Phrases
End

www.ph-noe.ac.at63
Mewald 2014b
Headline
Information
Main points
Summary

www.ph-noe.ac.at65
65

www.ph-noe.ac.at66
66

www.ph-noe.ac.at
Commented marking
 provides the learners with a focus for progression instead of a
reward or punishment (e.g. a correction or a grade)
 comments could be made in the text or in a table
 comments should make explicit how the learner can improve a
text

www.ph-noe.ac.at68
Coding system 1

www.ph-noe.ac.at69
Coding system 2

www.ph-noe.ac.at(c) Dr. Claudia Mewald70

www.ph-noe.ac.at
Available Designs
Designing
The Redesigned
Resources for Meaning: Available
Designs of Meaning (Models)
The work performed on or with
Available Designs in the semiotic
process (Modelling)
The resources that are produced
and transformed through Designing
(Creative Production)
Adapted from Cope & Kalantzis, 2000:23
72
Designs of Meaning 1

www.ph-noe.ac.at
learners are given good examples that exemplify what it is they
are being asked to do
understand what they need to
do in order to meet the
assessment criteria
learners assess exemplar work
using the assessment criteria/
rating scale
learners model what is being
asked for and how it relates
to the process of assessment
 learners create new designs
Designs of Meaning 2

www.ph-noe.ac.at74
Poster rubrics
Task:
Create age adequate
and comprehensible
rubric for learners at
primary school

www.ph-noe.ac.at
 learners set themselves targets based on their self-assessment
 learning goals are recorded and revisited
 comparison with teacher targets
consensus on targets
self-directed learning
Self-assessment targets
Schimek & Höltzer, 2013

www.ph-noe.ac.at76
http://www.coe.int/t/dg4/education/elp/
The European Language Portfolio (ELP) was developed by the
Language Policy Division of the Council of Europe
 to support the development of learner autonomy, plurilingualism
and intercultural awareness and competence;
 to allow users to record their language learning achievements
and their experience of learning and using languages.

www.ph-noe.ac.at77
77

www.ph-noe.ac.at78
Dossier
 opportunity to select
materials to document and
illustrate achievements or
experiences
Components
78

www.ph-noe.ac.at79 79
Source: http://www.ncca.ie/uploadedfiles/Curriculum/inclusion/primary_elp.pdf
Biography

Passport

Dossier

www.ph-noe.ac.at82 82(c) Dr. Claudia Mewald

www.ph-noe.ac.at
Auerbach, E. R. (1986) Competency-based ESL: One step forward or two steps back? TESOL Quarterly 20(3): 411 – 415.
Brown, D. H., & Abeywickrama, P. (2010) Language Assessment. Principles and Classroom Practices. New York: Pearson.
Cope, B. & Kalantzis, M. (2000) Multiliteracies. Literacy Learning and the Design of Social Futures. London: Routledge
Dressel , P. (1983). Grades: One More Tilt at the Windmill. In A. W. Chickering (Ed.), Bulletin. Memphis: Memphis State Univ.,
Center for Study for Higher Education, p. 12.
Jäckl, C. & Manhart, M. (2017) Die Grundkompetenzen Fremdsprache 2. Schulstufe - GK2. Mit exemplarischen
Umsetzungsbeispielen für den Englischunterricht in der Grundstufe 1. Graz: ÖSZ
Jones, N. & Saville, N. (2017) Learning Oriented Assessment. A systemic approach. Cambridge: CUP.
Marzano, R. (2009) Designing & Teaching Learning Goals & Objectives. Bloomington: FSC.
Mewald, C. (2014) Practice what you preach – test what you teach. Testing English as a foreign language in
heterogeneous groups. R&E-Source. Open Online Journal for Research and Education. PH NOE: Baden. Vol 1. Pp. 24-35
Available at: http://journal.ph-noe.ac.at/index.php/resource/article/view/14/13
Mewald, C. (2014) Assessment. In: BIFIE (Hrsg.) Bildungsstandards Englisch 8. Unterrichtsvideos und Begleitmaterialien. BIFIE:
Salzburg
Richards, J. C., & Rodgers, T. S. (2001) Approaches and Methods in Language Teaching (Second Edition). Cambridge: CUP.
Schenck, A. (1978) Oregon Competency Based Education Program. Northwest Regional Educational Laboratory.
Schimek, F., & Höltzer, R. (2013). Europäisches Sprachenortfolio für die Mitteleuropäische Region. Primarstufe (für
SchülerInnen von 6 bis 10 Jahren). Wien: Stadtschulrat für Wien.
Stiggins, R., Arter, J., Chappuis, J., & Chappuis, S. (2006) Classroom assessment for student learning—Doing it right, using it
well. Portland: Educational Testing Service.
Wormeli, R. (2006) Fair Isn’t Always Equal. Portland: Stenhouse Publishers.
Wiggins, G., Mc Tighe, J. (2005) Understanding by Design. Alexandria, VA: Pearson.
84
References

ACHIEVE-
MENT
TEST
Learning
designs
WHAT
= target goal(s)
(simpler,
more complex
goals)
LEARNERS:
readiness,
interests,
learning
profiles, prior
knowledge…
HOW
= process
approach
strategies
tasks & activities
classroom
management
PRODUCT
learning
outcomes &
ASSESSMENT
Curriculum
what can be differentiated and what cannot ….
D
I
A
G
N
O
S
I
S

www.ph-noe.ac.at86
86
Example for learner profile

www.ph-noe.ac.at
A 1: YLE Flyers
A 2: KET
B 1: PET
http://www.cambridgeenglish.org/prepare-and-practise/find-free-resources/
87
Proficiency testing vs. diagnostic
testing

www.ph-noe.ac.at
Assessment stations
Book, film/video or game review
Concept map
Design it
Do it
Field report
Lab report (book)
Learning log (diary)
Media profile
Assessment Methods A-Z
back

www.ph-noe.ac.at
Observation
Oral presentation
Paper and pencil quiz
Performance
Portfolio
Poster (presentation)
Question banks
Assessment Methods A-Z
back

Assessment 2018 handout

More Related Content

What's hot

Similar to Assessment 2018 handout

More from claudiamewald

Recently uploaded

Assessment 2018 handout