Paper Presented at Tehran UniversityPresentation Transcript
In the Name of the Most High The First Post-Graduate Conference Azar 30, 1385 (December 21, 2006) The Effect of Portfolio Assessment on Reading Comprehension Performance of Iranian High school sophomores By: Fatemeh Nikui Nezhad M.A. in TEFL
A Message of Thanks
Many Thanks to all great professors at Tehran University
And all post graduate students
Using Portfolios in the Classroom ( Presentation Outline)
- Defining the portfolio
-characteristics of a model portfolio
Traditional vs. Portfolio Assessment
Values & Challenges
- Research question
- Background description of the course and students
- Instrumentation and materials
- a framework for portfolio assessment
- Procedures and portfolio contents
The data analysis
Pros and Cons
Empirical studies on Portfolio Assessment
- Conclusion and Implications
Definition of Portfolios
Portfolio: “a narrative that tells a coherent story of your learning experiences in the program, and highlights thoughtful reflection on, and analysis of, those experiences. It is not simply an accumulation of pieces and products. It is an unfolding of your understanding about learning” (Darling, 2001, p.111).
Five characteristics of a model portfolio
Comprehensiveness : comprehensive data collection and analysis.
Predetermined and systematic : planned prior to implementation (purpose, contents, data collection schedule, and student grading criteria).
Informative : The information is meaningful to all stakeholders in the process (i.e., teachers, students, staff, and parents).
Tailored : tailored to the purpose for which it will be used, classroom goals and objectives, and individual student assessment needs.
Authentic : student information based on assessment tasks that reflect authentic activities (Moya and O’Malley, 1994, p. 15).
Measure student’s ability over time
Done by teacher and student; student aware of criteria
Embedded in instruction
Involves student in own assessment
Captures many facets of language learning performance
Allows for expression of teacher’s knowledge of student as learner
Student learns how to take responsibility
Barnhardt et al. (1998, p.10)
Measures student’s ability at one time
Done by the teacher alone; students often unaware of criteria
Conducted outside instruction
Assign student a grade
Does not capture the range of student’s language ability
Does not include the teacher’s knowledge of student as a learner
Does not give student responsibility
Portfolio Assessment Traditional Assessment
Possible Values of Using Portfolios
Strengths of portfolio assessment:
1 ) evaluate both product and process;
2) integration of learning and assessment;
3) evaluation not limited to a single score;
4) provide more information about a student's progress;
5) encourage students to take charge of their own learning;
6) students feel that they are a part of the assessment process;
7) help develop skills for lifelong learning;
8) information gained is meaningful and substantial;
9) provide a continuous example of a student's work in a context that is relevant and understandable ( Gilman & Richard, 1995, p.20).
Portfolio has the capability to reform curriculum and instruction (Stecher, 1998; Herman & Winters, 1994).
1) time-consuming, messy, and complicated;
2) have logistical problems such as handling and storage;
3) require a level of self-discipline that not all students have;
4) most evaluation is subjective;
5)use for large-scale assessment could interfere with their use for instruction;
6) reliability and validity are questionable, at least as defined by traditional measurement . (Gilman & Andrew ,1995, p. 1)
‘ portfolio assessment is an expert system’, and
designing a portfolio assessment demands
highly professional teachers in terms of
pedagogical and assessment skills
(Hamp-Lyons, 1996, p.155).
Reasons for Change ( Why Portfolio)
CLT has undergone many changes in how second language teaching is conducted and conceived (Richard and Rodgers, 2001).
The significance of the communicative competence and process learning. (Moya & O'Malley, 1994)
The incompatibility of process learning and product assessment (Shaaban, 2001)
Portfolio assessment. …allows for self-directed work, self-correction, greater autonomy and greater time frames. Students can work outside the time constraints of the school timetable. Students are free to select topics in which they have personal interests, thus portfolios have the potential for encouraging greater motivation (Moore, 1994, p.632).
Portfolios as powerful tools can be used so as to facilitate the learning process, enhance self-directed learning, encourage learner autonomy, and raise learners’ awareness about learning strategies, (Moore, 1994; Barnhardt et al., 1998; Yang, 2003; Banfi, 2003).
There are many gaps in the research on the innovative assessment methods such as portfolios (Johnston, 2004)
Why focus on Reading
L2 reading has been the focus of extensive research.
Reader as an interactive processor constructs meaning. (Grabe & Stoller, 2002)
The awareness and the use of cognitive and metacognitive strategies are closely related to the deficiency of reading process (Lau & Chan, 2003).
1. Does portfolio assessment affect Iranian sophomore high school students’ performance on reading comprehension?
two intact groups of female sophomore high school students majoring in mathematics aged 15-16.
30 students as the experimental group and the other 30 as the control group .
Instrumentation and materials A general proficiency test (Nelson 150 B); Portfolio assessment (introduction &weight); Students' Reading Log; Teachers’ Marking Sheet.
Portfolio Development Guide
planning the assessment purpose,
matching the purposes with instructional objectives,
establishing criteria for the assessment, developing checklists based on these criteria,
integrating portfolio into classroom activities, introducing the idea of portfolio assessment to the students,
its justifications, necessary components, the how of assessment,
monitoring the portfolio during the semester,
and finally evaluating the final portfolio.
1) five dated reading passages
2) Reading Log for each passage used as self-assessment tool
3) Portfolio Assessment Marking Sheet in which the teachers and raters graded students’ portfolio pieces
Student’s Reading Log کارنامه خواندن دانش آموز سعی می کنم متن را واضح و با تلفظ صحیح بخوانم وضبط کنم . متن را خلاصه می کنم . سوالاتی در مورد متن می نویسم . ایده اصلی متن را می نویسم . بعد از هر خواندن پیشرفت خود را ارزیابی می کنم و نظر خود را در مورد متن و میزان یادگیری بیان می کنم . خواندن R1 Yes=1 No=0 R2 Yes=1 No=0 R3 Yes=1 No=0 R4 Yes=1 No=0 R5 Yes=1 No=0 نمره دانش آموز نمره معلم
Portfolio Assessment Marking Sheet per lesson 10.Student reads with correct pronunciation and intonation 9.Student reads with a suitable speed Audio Tape in Portfolio (Total required number of audio tapes: 5 ) Yes (1) No (0) 8. Portfolio is clear and neat. 7.Student writes summary of readings 6.Student makes questions after reading 5.Student writes main ideas 4.Student Initiates own reading 3.Student shows creativity in tasks 2.Portfolio contains student comments and feedback after each task 1. Portfolio includes all the required items. Reading (Total required readings: 5 ) Yes (1) No (0)
Procedures of the Study
2. Portfolio Assessment 3. Evaluating Portfolios Traditional vs. Reading Portfolio Assessment
Table1:T-Test for the control and portfolio group’s performance on Nelson test at the pretest stage P<.05 N=30 df=58 t-critical=2.000 2.15 30.56 30 Control .86* 2.02 31.3 30 Portfolio t-observed SD Mean N Groups
Table 2 The t-value and descriptive statistics for the posttests of the two groups P<. 05 N=30 df=58 t-critical=2.000 2.17 16.66 30 Control 2.66* .62 17.76 30 Experimental (portfolio) Reading t- observed SD Mean N Groups Test
Benefits of portfolio creation: Learners’ own words (positive)
I’m so happy with this assessment method.
It helps me a lot to answer reading comprehension exercises.
It was a good experience for me and so helpful.
I really liked this kind of assessment since I read more passages and learnt more.
I enjoyed taping my reading aloud.
It improved my reading ability a lot.
I read the passages faster and comprehend them more .
Portfolio creation: Learners’ own words (negative)
It’s Time-consuming and Complicated
Sometimes it gets boring and tiresome.
It proves useful if implemented for a long time rather than one semester.
Empirical studies on Portfolio Assessment
Portfolio assessment used by Rafferty (1994) in a teacher education course “Teaching Reading in the Content Areas”;
portfolio application by Staehler (1994) in adult basic education and English as a second language;
development and implementation of portfolio management strategies by Newman et al. (1995) for his students;
investigating the studying and reading processes of limited-English-Speaking Hmong university using think-aloud protocols, reading journals, and study skills portfolios by Starks-Martin (1996);
using portfolio to implement an innovative ESL workplace program (ESLWP) by Schwarzer et al. (2002);
Empirical studies on Portfolio Assessment (continued)
Barnhardt et al. (1998) research, a 3- year portfolio assessment project in elementary through higher education foreign language classrooms;
integrating portfolios into a learning strategy-based EFL course by Yang (2003);
and compilation of second language portfolios as part of pre-service French teachers’ professional development in a university in Western Canada conducted by Christiansen and Laplante (2004).
integration of portfolios and technology (e-mail) into a writing class among Iranian undergraduates of English at Allameh Tabatabaii University by Marefat (2004).
Concluding Remarks Portfolios engage students. Help teach reflection and self-evaluation skills. Help teachers document learning in areas not covered in traditional assessment. Portfolio assessment increases learners’ involvement in the assessment and monitoring of their own learning. Benefits of portfolio assessment clearly outweigh the negative aspects.
used in educational settings as a method for judging students’ capabilities as an integral component of instruction.
indicates to the students as well as their teacher how well they are making progress
makes the students aware of their strengths and weakness .