A presentation on the use of informal diagnostic assessment of the writing of international students during orientation in identifying students who would benefit from additional support and guidance.

1. ATLAANZ 2012
Assessing and advising
international students on
academic writing during
orientation.
Martin McMorrow, Massey University

2. OVERVIEW
1.Rationale
2. Method
3. Outcomes

3. RATIONALE
• Early identification of students who may benefit
from support
• Raise student awareness of their writing
ability and challenges that lie ahead
• Evidence of impact of our services
• Evidence to support advocacy of institutional,
curricular and pedagogical change
• Promote our services

4. METHOD OF
ASSESSMENT & ADVICE
• Students spend up to 20 mins writing a
paragraph on why international students
choose to study at universities in NZ
• The writing task is the last in a sequence of
orientation activities for new international
students
• Before writing, the students brainstorm
ideas in groups – and can use these ideas or
refer to suggestions on the screen as they
write – They’re also given the first sentence
of the paragraph to get them going

5. METHOD OF
ASSESSMENT & ADVICE
• Paragraphs written by the students are
sorted into four bands (from most to least
effective) based on:
 Coherence
 Accuracy
 Range
• For analysis purposes, these four bands are
merged into two groups:
• Group A = effective
• Group B = ineffective

6. METHOD OF
ASSESSMENT & ADVICE
BAND 2: The paragraph communicates an adequate, though
rather simple, argument. It has a simple structure, with basic
connections between sentences (and, but, firstly, secondly). At
least half of the sentences are correct – i.e. without basic
grammar or vocabulary errors. The range of sentence types
and vocabulary is limited with only one or two examples of
non-basic vocabulary or syntax.
BAND 1: The paragraph develops a coherent and convincing
argument. It is well-structured, with an opening, a set of three
or more linked sentences, and a concluding comment. Most
sentences are correct, with no basic grammar or vocabulary
errors. There is evidence of a reasonable range of sentence
types and some non-basic vocabulary.
Criteria for Group A (effective paragraphs)

7. METHOD OF
ASSESSMENT & ADVICE
BAND 4: The paragraph does not communicate a convincing
argument as a whole, or at the level of individual sentences,
which are mostly unclear. The overall structure is minimal
and/or not coherent. Most sentences contain more than one
basic grammar or vocabulary error. The range of vocabulary
and sentence types is limited to basic forms.
BAND 3: The paragraph as a whole does not communicate a
convincing argument at tertiary level. At points it is hard to
follow. It has a simple structure and connections between
sentences are rather awkward and/or inconsistent. Most
sentences contain at least one basic grammar or vocabulary
error. The range of sentence types and vocabulary is limited to
basic forms.
Criteria for Group B (ineffective paragraphs)

8. There are various reasons why students choose to study
in NZ. To begin with, New Zealand is a beautiful country
to live, travel and study. People here enjoy all of the
greens everywhere, which is the top reason for many of
my friends coming New Zealand. Plus, New Zealand has
prestigious education system. I heard the four top
universities in New Zealand are all in the top 400
universities of the world, which is a fabulous thing. Last
but not the least, there are people from different
background travelling and living in New Zealand, which
makes our socieity with variority and cheallanges. We
can make friends from many nations around the world.
This can broaden our horizons.
METHOD OF
ASSESSMENT & ADVICE
Example paragraph 1

9. There are several advantages of studying in uni in NZ.
Firstly, learning multi-cultures is one of good ways that if
you go to overseas countries for studying. For example,
Asian students have not much knowledge about Europe,
midle-east and American. Moreover, The development of
English skill is fairly one of the goal that studying in uni
in NZ. As we well know, NZ is a English speaking
country, so it is a good way to everyone for learning
English.
METHOD OF
ASSESSMENT & ADVICE
Example paragraph 2

10. METHOD OF
ASSESSMENT & ADVICE
• Students are sent personalised feedback by
email ASAP after Orientation
• If students have written ineffective
paragraphs, I check their paper enrolments
and highlight in the email any particular
challenges they might be up against in
Semester 1.
• Here is an example of an email:

11. METHOD OF
ASSESSMENT & ADVICE
Hi_____________,
My name is Martin McMorrow and I work in the Centre for Teaching
and Learning at Massey. We are upstairs in the library.
Actually, I’m writing to give you some feedback on the writing
exercise you did during my session in the International Orientation,
and to check how things are going for you with your studies so far.
I recommend that you contact the Centre at (email & phone) to make
an appointment or just come in to see us on Level 3 of the library.
I’ll also include some advice about resources etc. We can help
students to develop their writing – and we also have consultants who
can help with maths, finance, economics, accountancy, statistics and
biochemistry. All of this is free to Massey students. It will be really
useful for you in xxx.xxx, xxx.xxx and xxx.xxx this semester.
regards, Martin

12. METHOD OF
ASSESSMENT & ADVICE
Contents of this email
1. Feedback on your writing
2. What makes writing good?
3. What makes writing bad?
4. How can you make your writing better?
5. Kiwi Quiz
1. Feedback on your writing
This is the paragraph which you wrote during International Orientation:
(student paragraph transcribed here)
What you wrote contained several grammar mistakes. It could be that this is
because of the situation when you were writing. If so, no worries.
But it looks like you’ll need to improve your written English in order to succeed in
your studies.
You can start to do this by making an appointment to see me soon!

13. OUTCOMES
• Group A students were more successful on writing
intensive business papers in the first year
0%
10%
20%
30%
40%
50%
60%
70%
Group A (18 sts) Group B (22 sts)
Successful attempts at
writing-intensive 1st year
business papers
18 / 28 15 / 31

14. OUTCOMES
• Group A students used the Centre for Teaching and
Learning (CTL) more
• However, nearly all international students who came
to the CTL for writing-intensive papers
passed, regardless of level
0%
20%
40%
60%
80%
100%
Group A (18 sts) Group B (22 sts)
Successful attempts at
writing-intensive 1st
year business papers
Successful attempts by
students who'd had
consultations at the CTL
18 / 28
9 / 10
5 / 6
15 / 31

15. OUTCOMES
• Students who entered with limited writing ability and
were unsuccessful in first year writing-intensive
papers continue to struggle with writing into their
second year
• For instance, in semester 1 (2012), among this cohort
of second-year international students, only five out of
nine retakes (of first year papers) were successful
• Many of the students who most need it appear
reluctant to engage with writing development
support. Very few take elective writing papers, attend
consultations at the CTL or respond to my cheery
emails!
• I’ve used these figures to advocate for a greater
emphasis on writing development and ‘strategic’
paper choices for international students

16. CONCLUSIONS
• Informal writing assessment and advice can
be incorporated ‘seamlessly’ into Orientation
• Even such a rough and ready assessment
as this has some predictive value
• Impact on awareness-raising and advocacy
appears to have been limited so far – some
seeds fall on stony ground!
• International students who attend one-to-
one consultations at the CTL do
considerably better than their peers on
writing intensive papers.