Dimova: National oral language assessment in Denmark: Where are we now and where are we going?

Centre for Internationalisation and Parallel Language Use

National oral language
assessment in Denmark:
Where are we now and
where are we going?
Slobodanka Dimova
Centre for Internationalisation and Parallel
Language Use (CIP)


Introduction
 Overview of Danish educational system

 Oral English language exams in 9th grade -Folkeskolens Afgangsprøve (FSA)
 A-level and B-level oral English exams in upper
secondary school (gymnasium)
 Synergies and tensions between different aspects
of exam development, administration, and use

 Future developments and research


Overview of the Danish education system

(http://eng.uvm.dk/Education/Overview-of-the-Danish-Education-System)

FSA


Foreign language examination system
New national structure to promote evaluation and assessment in
compulsory education

•Since 2006, creation of a new national structure to establish the
importance of evaluation and assessment in compulsory education
•Three national bodies with a role in the evaluation and assessment
of compulsory education: the Education Agency; the Quality and
Supervision Agency; and the Danish Evaluation Institute (EVA)
•The Quality and Supervision Agency is responsible for the
financial, institutional and pedagogical supervision of both
compulsory and upper secondary education. Major tasks include
developing and running the national tests and final
examinations in compulsory education, implementation of
international assessments, development of materials for schools in
the Evaluation portal.


FSA ORAL ENGLISH EXAM (9th grade)
The FSA oral English exam consists of two tasks:
1. Presentation on a selected topic (~5 min.)
2. Conversation/interview on specific topics (~8 min.)
Total examination time including evaluation
(~20min.)
Individual exam (though pupils can prepare in pairs or
groups)
Two examiners (teacher and external examiner)
Mandatory for all pupils


Before the exam day
• The teacher, together with pupils, develops a bibliography
with reading and video material organized around at least
three main themes. Each theme consists of several different
topics.
• The pupil chooses a topic under one of the themes.

• The pupil selects sources related to the chosen topic and
prepares the presentation. This is done with the teacher as
supervisor.
• The pupil prepares and submits an outline of his/her
presentation before the exam day.
• The teacher approves the student's outline with a signature.
The pupil also signs the outline.


Examples of themes and topics
Themes

Topics

Life in South Africa before and
after the Apartheid

Life in the townships
AIDS in SA
Apartheid
Racism
Sports in SA
Story/movie/poem related to the
topic

Gun violence in the USA

The American Constitution
The Ku Klux Klan
Gang life
Eminem’s lyrics

Fit or fat?

Fitness
Children and obesity
Eating disorders
What’s wrong with McDonald’s?
What’s healthy living?


An example of presentation outline

Pupil’s name and class:

Theme: Young People Today

My topic: Obesity

1. Why did I choose this topic?
a) My family background. b) Films I’ve seen. c) A personal interest

2. Different kinds of eating disorders
Obesity due to binge-eating; Anorexia, Bulimia

3. Who becomes obsessive – and why?
A case story

4. Whose problem is it – and what should be done about the problem?
A personal problem
”Society is to blame”- or?
School Lunch
TV-programs

Sources:
Morgan Spurlock’s film “Super Size Me”,
Jamie Oliver’s Diary : http://www.jamieoliver.com/diary/
British School Dinners:
http://www.channel4.com/life/microsites/J/jamies_school_dinners/fom/index.html ns

Student’s signature:

Teacher’s signature:


During the exam
Task 1: Pupils give a 5-min presentation on the selected topic.
• Internal-examiner (teacher) can ask questions related to
the topic
• External-examiner (censor) can ask only clarifying
questions
• Examiners can use the pupil’s presentation outline as the
basis for questions during the exam
Task 2: Pupils draw one card from a number of cards containing
a question and pictures related to the themes covered in the
bibliography.
• Pupils should take an initiative in a conversation with the
examiners


An example of Task 2 card


Grading criteria
One overall grade based on
four areas:
1.Communicative skills
2.Language use
3.Preparation (acquisition)
4.Culture and society
Teacher and censor each
assign a grade, but
final/reported grade is
decided through discussion.


After the exam
The external-examiner (censor) prepares a report
discussing:
•
•
•
•
•
•
•
•
•

Text selection
Themes and topics
Pupils’ outlines
Sources used for preparation
Questions in Task 2
Common mistakes
Examination process
Grading
Complaints

The report is published and used for improvement of
future examination procedures.


Synergies
• Dynamic collaboration among pupils, teachers,
and external-examiners, involvement of all
stakeholders in the process
• Direct relation to material covered in the
classroom, i.e. integrated teaching, learning,
and assessment
• Clear expectations about the examination tasks,
i.e. examination procedure transparency
and lower negative individual impact


Tensions
• The exam is in line with the Common Goals outlined by
the curriculum. However,
• The Common Goals (Fælles Mål) are broad and lack
specific description of the assesed construct.
____________
• FSA is nationally mandated and grades are used to a
certain extent for school and teacher comparison.
However,
• The local administration of the exam brings a large
degree of variability (e.g., topics, task 2, grading,
mentorship).


A-LEVEL ORAL ENGLISH EXAM (stx)
The A-level oral English exam consists of two
parts:
1.Presentation based on text analysis
2.Interview about the analysis
Preparation time (60min.)
Examination time including evaluation
(~30min.)
Two examiners (teacher and external examiner)
Only randomly selected pupils take it


Before the exam day
• Pupils prepare throughout all three years in
gymnasium by
• reading texts related to different topics
• analyzing texts
• learning different terminology used in text analysis
• Teachers prepare at least three new texts related to
the topics covered in class and send them to external
examiners (censors)
• Censors check the teacher selected texts for the
fulfillment of the requried criteria and approve them
•

Pupils are selected to take the exam by random draw


During the exam: Task (text analysis)
• Pupils are asked to prepare a presentation based on
a literary or non-literary text. In the presenation they
should,
• Analyze the text
• Interpret the text
• Relate the text to a topic discussed during the
semester
• The text specifications are as follows:
• 2-4 pages
• New (not discussed in class)
• 1300 letters (if a short story or book chapter)
• 30 lines from a poem
• 30 lines from a play


Task example
Theme: Urban Life Past and Present
Text: An excerpt from the novel Saturday by Ian McEwan,
Jonathan Cape 2005, p. 74-81.

The novel is set in London, 2003, and describes a day in
the life of Henry Perowne, a succesful neurosurgeon. In
this excerpt he is on his way to his weekly game of
squash. The streets are filled with anti-war protestors.
The occasion is the impending war against Iraq.
…the excerpt follows…


The emphasis is on:
1. Coherent (fluent) and mostly correct English
2. Presentation, analysis, understanding, interpretation and
discussion in relation to the knowledge gained in working with
the topic studied
3. Communicating one’s own views and arguments


Synergies
• Some collaboration among pupils, teachers, and
external-examiners, involvement of all
stakeholders in the process
• Direct relation to the material covered in the
classroom, i.e. some integration of teaching,
learning and assessment.
• Clear expectations about the examination tasks,
i.e. examination procedure transparency
and lower negative individual impact.
• Integration of two language skills reading
and speaking.


Tensions
• The exam is nationally mandated and grades are
used to a certain extent for school and teacher
comparison. However,
• The local administration of the exam brings a large
degree of variability.
______________
• Emphasis is on literary text analysis. However,
• Grades are used for admission in EMI programs
at university level.


Where are we now?
• Traditional oral exams—have existed a long time with
small revisions
• Practicality:
• FSA and A-level/B-level oral exams are administered
locally
• No additional resources needed except for censors
• Authenticity:
• Materials in prompts are authentic
• Authentic tasks?


Where are we now?
• Reliability:
• inter- and intra- grader reliability?
• administration reliability?
• pupil reliability?
• Validity:
• content validity
• construct underdefined, especially for FSA
• Possible construct-irrelevant variation?
• Consequences:
• stx scores use for admission in EMI
• FSA scores used for employment
• Washback:
• too much emphasis on literary texts
• 9th grade FSA focused on topic and presentation
development


Where are we going?


Where are we going?
OPTION 1
• Keep traditional testing methods, but restrict
result interpretations to local contexts

• No generalization
• No school comparison
• Local use
• Advisory purposes


Where are we going?
OPTION 2
• Improve definition of the construct
• Standardize tasks

• Standardize examination procedures
• Monitor intra- and inter-rater reliability
• Conduct on-going validation investigations
• Re-think the relationship between tasks and
expectations for real-life language use
• Follow the consequences of exam changes


The right way…
…depends on many factors.
• If FSA and A-level exams become higher stakes
exams, quality measures have to be
implemented
• Tradition could still be maintained
• Balancing practicality and other exam qualities
will still greatly depend on the available
resources (financial, material, professional)


Thank you!
plq379@hum.ku.dk

Dimova: National oral language assessment in Denmark: Where are we now and where are we going?

More Related Content

What's hot

Viewers also liked

Similar to Dimova: National oral language assessment in Denmark: Where are we now and where are we going?

More from IirisNii

Recently uploaded

Dimova: National oral language assessment in Denmark: Where are we now and where are we going?