BEYOND TEST : ALTERNATIVES IN
           ASSESSMENT


Source:
Brown, D. (2004). Language Assessment: Principles and Classroom
   Practices. New York: Pearson Longman.

Yamith J. Fandiño
La Salle University
Bogotá, Colombia
Introduction
• In Chapter 1, an important distinction was made between testing
  and assessing.

• Tests are formal procedures, usually administered within strict time
  limitations, to sample the performance of a test-taker in a specified
  domain.

• Assessment connotes a much broader concept in that most of the
  time when teachers are teaching, they are also assessing.
  Assessment includes all occasions from informal impromptu
  observations and comments up to and including tests.
Alternative assessment
Early in the decade of the 1990s, in a culture of rebellion against the notion
   that all people and all skills could be measured by traditional tests, a novel
   concept emerged that began to be labeled "alternative" assessment.

That concept was to assemble additional measures of students—portfolios,
   journals, observations, self-assessments, peer-assessments, and the like
   —in an effort to triangulate data about students.

Brown and Hudson (1998) noted that to speak of alternative assessments is
   counterproductive because the term implies something new and different
   that may be "exempt from the requirements of respon-sible test
   construction" (p. 657). So they proposed to refer to "alternatives" in
   assess-ment instead. Their term is a perfect fit within a model that
   considers tests as a subset of assessment.
The characteristics of alternatives in
                  assessment
1. They require students to perform, create, produce or do
   something
2. They use real-word context or simulations
3. They are noinstrusive in that they extend the day to day
   classroom activities
4. They allow students to be asssesed on what they normally
   do in class
5. They use tasks that represent meaningful instructional
   activities
6. They focus on processes as well as products
7. They tap into higher-level thinking and problem solving skills
Dilemma in standardized and alternatives
              in assessment
• Formal standardized tests are almost by definition highly practical,
  reli-able instruments. They are designed to minimize time and
  money on the part of test designer and test-taker, and to be
  painstakingly accurate in their scoring.

• Alternatives such as portfolios or conferencing with students on
  drafts of written work, or observations of learners over time all
  require considerable time and effort on the part of the teacher and
  the student.

• But the alternative techniques also offer markedly greater
  washback, are superior formative measures, and, because of their
  authenticity, usually carry greater face validity.
Relationship of practicality /
                  reliability to washback
 High


Practicality
    and
 reliability




  Washback




   low
Performance-based assessment
Performance-based assessment implies productive, observable skills,
   such as speaking and writing, of content-valid tasks. Such
   performance usually, but not always, brings with it an air of
   authenticity—real-world tasks that students have had time to
   develop. It often implies an integration of language skills, perhaps
   all four skills in the case of project work.

The characteristics of performance assesment :
1. Students make a constructed response
2. They engage in higher- order thinking , with open –ended tasks
3. Tasks are meaningful , engaging, and authenthic
4. Tasks call for the integration of language skills
5. Both process and product are assesed
6. Depth of a student’s mastery is emphasized over breadth
Procedures for performance-
           based assessment

Performance-based assessment procedures need to be treated with
   the same rigor as traditional tests. This implies that teachers should

• state the overall goal of the performance,
• specify the objectives (criteria) of the performance in detail,
• prepare students for performance in stepwise progressions,
• use a reliable evaluation form, checklist, or rating sheet,
• treat performances as opportunities for giving feedback and
  provide that feedback systematically, and
• if possible, utilize self- and peer-assessments judiciously.
Portofolios

A portopolio is a purposeful collection of students work that
  demonstrates students’ efforts, progress, and achievements in
  given areas (Genesee and Upshur, 1996).

Portfolios include materials such as
• essays and compositions in draft and final forms;
• reports, project outlines;
• audio and/or video recordings of presentations, demonstrations,
   etc.;
• journals, diaries, and other personal reflections;
• tests, test scores, and written homework exercises;
• self- and peer-assessments--comments, evaluations, and checklists.
Attributes of portofolios
• Gottlieb (1995) suggested a developmental scheme for
  considering the nature and purpose of portfolios, using the
  acronym CRADLE to designate six possible attributes of a
  portfolio:

• Collecting: an expression of students' lives and identities.
• Reflecting: thinking about experiences and activities.
• Assessing: evaluating quality and development over time.
• Documenting: demonstrating student achievement.
• Linking: connecting student and teacher, parent,
  community, and peer
• Evaluating: generating responsible outcomes.
Steps and guidelines
•   State objectives clearly
•   Give guidelines on what materials to include
•   Communicate assesment criteria to students
•   Designate time within the curriculum for portfolio development.
•   Establish periodic schedules for review and conferencing.
•   Designate an accessible place to keep portfolios.
•   Provide positive washback when giving final assessments.

It is inappropriate to reduce the personalized and creative process of
    compiling a portfolio to a number or letter grade. Instead, teachers
    should offer a qualitative evaluation such a final appraisal of the
    work, with questions for self-assessment of a project, and a
    narrative evaluation of perceived strengths and weakness.
Journals
•    A journal is a log of one’s thought , feelings,
    reactions, assessments, ideas, or progress,
    toward goals, usually written with little attention
    to structure , form, o correctness.

• Journals obviously serve important pedagogical
  purposes : practice in the mechanics of writing ,
  using writing as a thinking process,
  individualization , and communications with the
  teacher .
Steps for journals
1. Sensitively introduce students to the concept of journal writing.
2. State the objective(s) of the journal: Language-learning logs,
    Grammar journals, Responses to readings, strategies-based
    learning logs, Self-assessment reflections, etc.
3. Give guidelines on what kinds of topics to include.
4. Carefully specify the criteria for assessing or grading journals.
    Effort as exhibited in the thoroughness of students' entries will
    no doubt be important. Also, the extent to which entries reflect
    the processing of course content might be considered.
5. Provide optimal feedback in your responses: cheerleading
    feedback, instructional feedback, or reality-check feedback.
6. Designate appropriate time frames and schedules for review.
7. Provide formative, washback-giving final comments.
Conferences and interviews
• Conferences are not limited to drafts of written work. It
  must assume that the teacher plays the role of a
  facilitator and guide , not of an administrator of a formal
  assesment.

• A number of generic question that may be usefull to pose
   in conference are
1. What did you like about this work?
2. What do you think you did well?
3. How does it show improvement from previous work? Can
   you show me the improvement?
4. What did you do when you did not know a word that you
   want to write/say? (Genesee and Upshur, 1996).
Guidelines for conferences and
           interviews
1.     Offer an initial atmosphere of warmth and
   anxiety-lowering (warm-up).
2.     Begin with relatively simple questions.
3.     Continue with level-check and probe questions,
   but adapt to the interviewee as needed.
4.     Frame questions simply and directly.
5.     Focus on only one factor for each question. Do
   not combine several objec-tives in the same question.
6.     Be prepared to repeat or reframe questions that
   are not understood.
7.     Wind down with friendly and reassuring dosing
   comments.
Observations
•   Observation is a systematic, planned procedure for real-time, almost
    furtive recording of student verbal and nonverbal behavior. One of
    the objectives of such observation is to assess students without their
    awareness (and possible consequent anxiety) of the observation so
    that the naturalness of their linguistic performance is maximized.

•   Potential observation foci
-   sentence-level oral production skills.
-   pronunciation of target sounds, intonation, etc.
-   grammatical features (verb tenses, question formation, etc.
-   discourse-level skills (conversation rules, turn-taking, and other
    macroskills)
-   interaction with classmates (cooperation, frequency of oral
    production)
-   frequency of student-initiated responses (whole class, group work)
Steps for observations
•   Determine the specific objectives of the observations
•   Decide how many students will be observed at one time
•   Set up the logistics for making unnoticed observations
•   Design a system for recording observed performances
•   Do not overestimate the number of different elements you can
    observe at one time
•   Plan how many observations you will make
•   Determine specifically how you will use the results
Alternatives in observation
Checklists are a viable alternative for recording observation results.

•   The observer identifies an activity or episode and checks appropriate
    boxes along a grid. This grid refers to variables such as whole-class, group,
    and individual participation, linguistic competence (form, function,
    discourse, sociolinguistic), etc. Each variable has subcategories for better
    analysis.

Rating scales have also been suggested for recording observations.

•   One type of rating scale asks teachers to indicate the frequency of
    occurrence of target performance on a separate frequency scale (always
    = 5; never = 1).
•   Another is a holistic assessment scale that requires an overall assessment
    within a number of categories (for example, vocabulary usage,
    grammatical correctness, fluency).
Self and peer assesment
•   Self –assesment derives its theoritical justification from a number of well
    established principles of second language acquisition. The principle of
    autonomy is vital. It consists of the ability to set one's own goals both
    within and beyond the structure of a classroom curriculum, to pursue
    them without the presence of an external push, and to independently
    monitor that pursuit. Developing intrinsic motivation that comes from a
    self-propelled desire to excel is at the top of the list of successful
    acquisition of any set of skills.

•   Peer-assesment appeals to similar principles , the most obvious of which is
    cooperative learning. Many people go through a whole regimen of
    education from kindergaten up through a graduate degree and never
    come to appreciate the value of collaboration in learning.

•   Peer assesment is simply one arm of a plethora of tasks and procedures
    within the domain of learner-centered and collaboration education.
Types of self and peer
              assessments
1.   Assessment of a specific performance
2.   Indirect assesment of general competence
3.    Metacognitive assesment for setting goals
4.   Socioaffective assesment
5.   Student generated test
Guidelines for self and peer
             assessments
1. Tell students the purpose of the assessment.
2. Define the task(s) clearly.
3. Encourage impartial evaluation of performance or ability
4. Ensure beneficial washback through follow-up tasks.
Self- and peer-assessment tasks
Listening Tasks
• listening to TV or radio broadcasts and checking comprehension with a partner
• listening to an academic lecture and checking yourself on a "quiz" of the content

Speaking Tasks
• using peer checklists and questionnaires
• rating someone's oral presentation (holistically)

•   Reading Tasks
•   reading passages with self-check comprehension questions following
•   taking vocabulary quizzes

•   Writing Tasks
•   revising written work on your own or with a peer (peer editing)
•   proofreading

Beyond tests alternatives in assessment

  • 1.
    BEYOND TEST :ALTERNATIVES IN ASSESSMENT Source: Brown, D. (2004). Language Assessment: Principles and Classroom Practices. New York: Pearson Longman. Yamith J. Fandiño La Salle University Bogotá, Colombia
  • 2.
    Introduction • In Chapter1, an important distinction was made between testing and assessing. • Tests are formal procedures, usually administered within strict time limitations, to sample the performance of a test-taker in a specified domain. • Assessment connotes a much broader concept in that most of the time when teachers are teaching, they are also assessing. Assessment includes all occasions from informal impromptu observations and comments up to and including tests.
  • 3.
    Alternative assessment Early inthe decade of the 1990s, in a culture of rebellion against the notion that all people and all skills could be measured by traditional tests, a novel concept emerged that began to be labeled "alternative" assessment. That concept was to assemble additional measures of students—portfolios, journals, observations, self-assessments, peer-assessments, and the like —in an effort to triangulate data about students. Brown and Hudson (1998) noted that to speak of alternative assessments is counterproductive because the term implies something new and different that may be "exempt from the requirements of respon-sible test construction" (p. 657). So they proposed to refer to "alternatives" in assess-ment instead. Their term is a perfect fit within a model that considers tests as a subset of assessment.
  • 4.
    The characteristics ofalternatives in assessment 1. They require students to perform, create, produce or do something 2. They use real-word context or simulations 3. They are noinstrusive in that they extend the day to day classroom activities 4. They allow students to be asssesed on what they normally do in class 5. They use tasks that represent meaningful instructional activities 6. They focus on processes as well as products 7. They tap into higher-level thinking and problem solving skills
  • 5.
    Dilemma in standardizedand alternatives in assessment • Formal standardized tests are almost by definition highly practical, reli-able instruments. They are designed to minimize time and money on the part of test designer and test-taker, and to be painstakingly accurate in their scoring. • Alternatives such as portfolios or conferencing with students on drafts of written work, or observations of learners over time all require considerable time and effort on the part of the teacher and the student. • But the alternative techniques also offer markedly greater washback, are superior formative measures, and, because of their authenticity, usually carry greater face validity.
  • 6.
    Relationship of practicality/ reliability to washback High Practicality and reliability Washback low
  • 7.
    Performance-based assessment Performance-based assessmentimplies productive, observable skills, such as speaking and writing, of content-valid tasks. Such performance usually, but not always, brings with it an air of authenticity—real-world tasks that students have had time to develop. It often implies an integration of language skills, perhaps all four skills in the case of project work. The characteristics of performance assesment : 1. Students make a constructed response 2. They engage in higher- order thinking , with open –ended tasks 3. Tasks are meaningful , engaging, and authenthic 4. Tasks call for the integration of language skills 5. Both process and product are assesed 6. Depth of a student’s mastery is emphasized over breadth
  • 8.
    Procedures for performance- based assessment Performance-based assessment procedures need to be treated with the same rigor as traditional tests. This implies that teachers should • state the overall goal of the performance, • specify the objectives (criteria) of the performance in detail, • prepare students for performance in stepwise progressions, • use a reliable evaluation form, checklist, or rating sheet, • treat performances as opportunities for giving feedback and provide that feedback systematically, and • if possible, utilize self- and peer-assessments judiciously.
  • 9.
    Portofolios A portopolio isa purposeful collection of students work that demonstrates students’ efforts, progress, and achievements in given areas (Genesee and Upshur, 1996). Portfolios include materials such as • essays and compositions in draft and final forms; • reports, project outlines; • audio and/or video recordings of presentations, demonstrations, etc.; • journals, diaries, and other personal reflections; • tests, test scores, and written homework exercises; • self- and peer-assessments--comments, evaluations, and checklists.
  • 10.
    Attributes of portofolios •Gottlieb (1995) suggested a developmental scheme for considering the nature and purpose of portfolios, using the acronym CRADLE to designate six possible attributes of a portfolio: • Collecting: an expression of students' lives and identities. • Reflecting: thinking about experiences and activities. • Assessing: evaluating quality and development over time. • Documenting: demonstrating student achievement. • Linking: connecting student and teacher, parent, community, and peer • Evaluating: generating responsible outcomes.
  • 11.
    Steps and guidelines • State objectives clearly • Give guidelines on what materials to include • Communicate assesment criteria to students • Designate time within the curriculum for portfolio development. • Establish periodic schedules for review and conferencing. • Designate an accessible place to keep portfolios. • Provide positive washback when giving final assessments. It is inappropriate to reduce the personalized and creative process of compiling a portfolio to a number or letter grade. Instead, teachers should offer a qualitative evaluation such a final appraisal of the work, with questions for self-assessment of a project, and a narrative evaluation of perceived strengths and weakness.
  • 12.
    Journals • A journal is a log of one’s thought , feelings, reactions, assessments, ideas, or progress, toward goals, usually written with little attention to structure , form, o correctness. • Journals obviously serve important pedagogical purposes : practice in the mechanics of writing , using writing as a thinking process, individualization , and communications with the teacher .
  • 13.
    Steps for journals 1.Sensitively introduce students to the concept of journal writing. 2. State the objective(s) of the journal: Language-learning logs, Grammar journals, Responses to readings, strategies-based learning logs, Self-assessment reflections, etc. 3. Give guidelines on what kinds of topics to include. 4. Carefully specify the criteria for assessing or grading journals. Effort as exhibited in the thoroughness of students' entries will no doubt be important. Also, the extent to which entries reflect the processing of course content might be considered. 5. Provide optimal feedback in your responses: cheerleading feedback, instructional feedback, or reality-check feedback. 6. Designate appropriate time frames and schedules for review. 7. Provide formative, washback-giving final comments.
  • 14.
    Conferences and interviews •Conferences are not limited to drafts of written work. It must assume that the teacher plays the role of a facilitator and guide , not of an administrator of a formal assesment. • A number of generic question that may be usefull to pose in conference are 1. What did you like about this work? 2. What do you think you did well? 3. How does it show improvement from previous work? Can you show me the improvement? 4. What did you do when you did not know a word that you want to write/say? (Genesee and Upshur, 1996).
  • 15.
    Guidelines for conferencesand interviews 1. Offer an initial atmosphere of warmth and anxiety-lowering (warm-up). 2. Begin with relatively simple questions. 3. Continue with level-check and probe questions, but adapt to the interviewee as needed. 4. Frame questions simply and directly. 5. Focus on only one factor for each question. Do not combine several objec-tives in the same question. 6. Be prepared to repeat or reframe questions that are not understood. 7. Wind down with friendly and reassuring dosing comments.
  • 16.
    Observations • Observation is a systematic, planned procedure for real-time, almost furtive recording of student verbal and nonverbal behavior. One of the objectives of such observation is to assess students without their awareness (and possible consequent anxiety) of the observation so that the naturalness of their linguistic performance is maximized. • Potential observation foci - sentence-level oral production skills. - pronunciation of target sounds, intonation, etc. - grammatical features (verb tenses, question formation, etc. - discourse-level skills (conversation rules, turn-taking, and other macroskills) - interaction with classmates (cooperation, frequency of oral production) - frequency of student-initiated responses (whole class, group work)
  • 17.
    Steps for observations • Determine the specific objectives of the observations • Decide how many students will be observed at one time • Set up the logistics for making unnoticed observations • Design a system for recording observed performances • Do not overestimate the number of different elements you can observe at one time • Plan how many observations you will make • Determine specifically how you will use the results
  • 18.
    Alternatives in observation Checklistsare a viable alternative for recording observation results. • The observer identifies an activity or episode and checks appropriate boxes along a grid. This grid refers to variables such as whole-class, group, and individual participation, linguistic competence (form, function, discourse, sociolinguistic), etc. Each variable has subcategories for better analysis. Rating scales have also been suggested for recording observations. • One type of rating scale asks teachers to indicate the frequency of occurrence of target performance on a separate frequency scale (always = 5; never = 1). • Another is a holistic assessment scale that requires an overall assessment within a number of categories (for example, vocabulary usage, grammatical correctness, fluency).
  • 19.
    Self and peerassesment • Self –assesment derives its theoritical justification from a number of well established principles of second language acquisition. The principle of autonomy is vital. It consists of the ability to set one's own goals both within and beyond the structure of a classroom curriculum, to pursue them without the presence of an external push, and to independently monitor that pursuit. Developing intrinsic motivation that comes from a self-propelled desire to excel is at the top of the list of successful acquisition of any set of skills. • Peer-assesment appeals to similar principles , the most obvious of which is cooperative learning. Many people go through a whole regimen of education from kindergaten up through a graduate degree and never come to appreciate the value of collaboration in learning. • Peer assesment is simply one arm of a plethora of tasks and procedures within the domain of learner-centered and collaboration education.
  • 20.
    Types of selfand peer assessments 1. Assessment of a specific performance 2. Indirect assesment of general competence 3. Metacognitive assesment for setting goals 4. Socioaffective assesment 5. Student generated test
  • 21.
    Guidelines for selfand peer assessments 1. Tell students the purpose of the assessment. 2. Define the task(s) clearly. 3. Encourage impartial evaluation of performance or ability 4. Ensure beneficial washback through follow-up tasks.
  • 22.
    Self- and peer-assessmenttasks Listening Tasks • listening to TV or radio broadcasts and checking comprehension with a partner • listening to an academic lecture and checking yourself on a "quiz" of the content Speaking Tasks • using peer checklists and questionnaires • rating someone's oral presentation (holistically) • Reading Tasks • reading passages with self-check comprehension questions following • taking vocabulary quizzes • Writing Tasks • revising written work on your own or with a peer (peer editing) • proofreading