OBJECTIVES 1. Distinguish performance-based assessment with the traditional paper and pencil tests. 2. Construct tasks that are performance based. Design a rubric to assess a performance based task
TERMS Authentic assessment Direct assessment Alternative assessment Performance testing Performance assessment Changes are taking place in assessment
METHOD Assessment should measure what is really important in the curriculum. Assessment should look more like instructional activities than like tests. Educational assessment should approximate the learning tasks of interest, so that, when students practice for the assessment, some useful learning takes place.
WHAT IS PERFORMANCE ASSESSMENT? Testing that requires a student to create an answer or a product that demonstrates his/her knowledge or skills (Rudner & Boston, 1991).
FEATURES OF PERFORMANCE ASSESSMENT Intended to assess what it is that students know and can do with the emphasis on doing. Have a high degree of realism about them. Involve: (a) activities for which there is no correct answer, (b) assessing groups rather than individuals, (c) testing that would continue over an extended period of time, (d) self-evaluation of performances. Likely use open-ended tasks aimed at assessing higher level cognitive skills.
PUSH ON PERFORMANCE ASSESSMENT Bring testing methods more in line with instruction. Assessment should approximate closely what it is students should know and be able to do.
EMPHASIS OF PERFORMANCE ASSESSMENT Should assess higher level cognitive skills rather than narrow and lower level discreet skills. Direct measures of skills of interest.
PROBLEMS OF IMPLEMENTATION Experience in constructing and using performance tests is lacking among many educational measurement specialist. Performance tests take more time to construct, administer, and score than objective tests. Standardization, reliability, and validity will be difficult to apply
GUIDELINES TO ESTABLISH VALIDITY OFPERFORMANCE TESTS: Consequences: Do teachers teach differently? What do students learn? Fairness: There is no guarantee Transfer and generalizability: Extent of small tasks generalized to larger tasks. Cognitive complexity: no guarantee that high level cognitive skills are tapped. Content quality: limited sampling of content is possible
GUIDELINES TO ESTABLISH VALIDITY OFPERFORMANCE TESTS: Content coverage: number of tasks chosen is small. Meaningfulness: evidence that the assessment is meaningful for students Cost and efficiency: time consuming and costly to construct, administer, and score thn objective forms of assessment.
TERMSPerformance-and-product The emphasis is on the students’ ability to perform tasks by producing their own work with their knowledge and skills.Alternative assessment Method that differs from conventional paper-and-pencil tests, most particularly objective tests.Authentic assessment Direct examination of student’s ability to use knowledge to perform a task that is like what is encountered in real life or in the real world.
CHARACTERISTICS OF PERFORMANCE-BASEDASSESSMENT Students perform, create, construct, produce, or do something. Deep understanding and/or reasoning skills are needed and assessed. Involves sustained work, often days and weeks. Calls on students to explain, justify, and defend. Performance is directly observable. Involves engaging in ideas of importance and substance. Relies on trained assessor’s judgments for scoring Multiple criteria and standards are prespecified and public There is no single correct answer. If authentic, the performance is grounded in real world contexts and constraints.
LEARNING TARGETS Skills Communication and presentation skills Ex: Speaking 1. Speaking clearly, expressively, and audibly a. Using voice expressively b. Speaking articulately and pronouncing words correctly c. Using appropriate vocal volume 2. Presenting ideas with appropriate introduction, development, and conclusion 1. Presenting ideas in an effective order 2. Providing a clear focus on the central idea 3. Providing signal words, internal summaries, and transitions
3. Developing ideas using appropriate support materials a) Being clear and using reasoning processes b) Clarifying, illustrating, exemplifying, and documenting ideas4. Using nonverbal cues a. Using eye contact b. Using appropriate facial expressions, gestures, and body movement5. Selecting language to a special purpose a. Using language and conventions appropriate for the audience
Psychomotor skills Fine motor: cutting papers with scissors, drawing a line tracing, penmanship, coloring drawing, connecting dots Gross motor: Walking, jumping, balancing, throwing, skipping, kicking Complex: Perform a swing golf, operate a computer, drive a car, operate a microscope Visual: Copying, finding letters, finding embedded figures, identifying shapes, discrimination Verbal and auditory: identify and discriminate sounds, imitate sounds, pronounce carefully, blend vowels
Products Write promotional materials Report on a foreign country Playing a new song
VARIATION OF AUTHENTICITYRelatively authentic Somewhat authentic AuthenticIndicate which parts of Design a garden Create a gardena garden design areaccurateWrite a paper on Write a proposal to Write a proposal tozoning change fictitious present to city council zoning laws to change zoning lawsExplain what would Show how to perform Play a basketballyou teach to students basketball skills in game.learning basketball practice
CONSTRUCTING PERFORMANCE BASEDTASKS1. Identify the performance task in which students will be engaged2. Develop descriptions of the task and the context in which the performance is to be conducted.3. Write the specific question, prompt, or problem that the student will receive.• Structure: Individual or group?• Content: Specific or integrated?• Complexity: Restricted or extended?
COMPLEXITY OF TASK Restricted-type task Narrowly defined and require brief responses Task is structured and specific Ex: Construct a bar graph from data provided Demonstrate a shorter conversation in French about what is on a menu Read an article from the newspaper and answer questions Flip a coin ten times. Predict what the next ten flips of the coin will be, and explain why. Listen to the evening news on television and explain if you believe the stories are biased. Construct a circle, square, and triangle from provided materials that have the same circumference.
Extended-type task Complex, elaborate, and time-consuming. Often include collaborative work with small group of students. Requires the use of a variety of information Examples: Design a playhouse and estimate cost of materials and labor Plan a trip to another country: Include the budget and itinerary, and justify why you want to visit certain places Conduct a historical reenactment (e. g. impeachment trial of ERAP) Diagnose and repair a car problem Design an advertising campaign for a new or existing product
IDENTIFYING PERFORMANCE TASKDESCRIPTION Prepare a task description Listing of specifications to ensure that essential if criteria are met Includes the ff.: Content and skill targets to be assessed Description of student activities Group or individual Help allowed Resources needed Teacher role Administrative process Scoring procedures
PERFORMANCE-BASED TASK QUESTIONPROMPT Task prompts and questions will be based on the task descriptions. Clearly identifies the outcomes, outlines what the students are encourage dot do, explains criteria for judgment.
CHARACTERISTICS OF TASKS1. Should integrate the most essential aspects of the content being assessed with the most essential skills.2. Should be authentic Realistic Require judgment and innovation Ask the student to do the subject Replicates or stimulates Assess the students ability to efficiently and effectively use a repertoire of knowledge and skill to negotiate a complex task Allows opportunities to rehearse, practice, consult resources, and get feedback and refine performances and products.
3. Structure the task to assess multiple learning targets4. Structure the task so that you can help students succeed.5. Think through what students will do to be sure that the task is feasible6. The task should allow for multiple solutions7. The task should be clear8. The task should be challenging and stimulating to students9. Include explicitly stated scoring criteria as part of the task10. Include constraints in completing the task
PERFORMANCE CRITERIA What you look for in student responses to evaluate their progress toward meeting the learning target. Dimensions of traits in performance that are used to illustrate understanding, reasoning, and proficiency. Start with identifying the most important dimensions of the performance What distinguishes an adequate to an inadequate demonstration of the target?
QUESTIONS TO ASK: What are the attributes of good writing, or good scientific thinking, or good collaborative group process, of effective oral presentation? More generally, by what qualities or features will I know whether students have produced an excellent response to my assessment task? What do I expect to see if this task is done excellently, acceptably, or poorly?
Do I have samples or models of student work, from my class or other sources, that exemplify some of the criteria I might use in judging this task? What criteria for this or similar task exist in my state curriculum framework, my state assessment program, my district curriculum guides, my school assessment program? What dimensions might I adapt from work done by natural curriculum councils, by other teachers?
EXAMPLE OF CRITERIA Learning target: Students will be able to write a persuasive paper to encourage the reader to accept a specific course of action or point of view. Criteria: Appropriateness of language for the audience Plausibility and relevance of supporting arguments. Level of detail presented Evidence of creative, innovative thinking Clarity of expression Organization of ideas
RATING SCALES Indicate the degree to which a particular dimension is present. Three kinds: Numerical, qualitative, combined qualitative/quantitative
Numerical Scale Numbers of a continuum to indicate different level of proficiency in terms of frequencyor qualityExample:Complete Understanding 5 4 3 2 1 No understandingNo organization 5 4 3 2 1 Clear organizationEmergent reader 5 4 3 2 1 Fluent reader
Qualitative scale Uses verbal descriptions to indicate student performance. Provides a way to check the whether each dimension was evidenced. Type A: Indicate different gradations of the dimension Type B: Checklist
Example of Type A: Minimal, partial, complete Never, seldom, occasionally, frequently, always Consistent, sporadically, rarely None, some, complete Novice, intermediate, advance, superior Inadequate, needs improvement, good excellent Excellent, proficient, needs improvement Absent, developing, adequate, fully developed Limited, partial, thorough Emerging, developing, achieving Not there yet, shows growth, proficient Excellent, good, fair, poor
Holistic scale The category of the scale contains several criteria, yielding a single score that gives an overall impression or ratingExample level 4: Sophisticated understanding of text indicated with constructed meaning level 3: Solid understanding of text indicated with some constructed meaning level 2: Partial understanding of text indicated with tenuous constructed meaning level 1: superficial understanding of text with little or no constructed meaning
Analytic Scale One in which each criterion receives a separate score. ExampleCriteria Outstanding Competent Marginal 5 4 3 2 1Creative ideasLogical organizationRelevance of detailVariety in words andsentencesVivid images
RUBRICS When scoring criteria are combined with a rating scale, a complete scoring guideline is produced or rubric. A scoring guide that uses criteria to differentiate between levels of student proficiency.
Rubrics should answer the following questions: By what criteria should performance be judged? Where should we look and what should we look for to judge performance success? What does the range in the performance quality look like? How do we determine validity, reliability, and fairly what scores should be given and what that score means? How should the different levels of quality be described and distinguished from one another?
GUIDELINES IN CREATING A RUBRIC1. Be sure the criteria focus on important aspects of the performance2. Match the type of rating with the purpose of the assessment3. The descriptions of the criteria should be directly observable4. The criteria should be written so that students, parents, and others understand them.5. The characteristics and traits used in the scale should be clearly and specifically defined.6. Take appropriate steps to minimize scoring frame
PORTFOLIO ASSESSMENT: EXPLORATION Have you ever done a portfolio? Tell me about this experience. Did you enjoy it? What elements did you include in your portfolio? Are the materials placed in the portfolio required?
WHAT ARE PORTFOLIOS? Purposeful, systematic process of collecting and evaluating student products to document progress toward the attainment of learning targets or show evidence that a learning target has been achieved. Includes student participation in the selection and student self-reflection. “A collection of artifacts accompanied by a reflective narrative that not only helps the learner to understand and extend learning, but invites the reader of the portfolio to gain insight about learning and the learner (Porter & Cleland, 1995)
CHARACTERISTICS OF PORTFOLIOASSESSMENT Clearly defined purpose and learning targets Systematic and organized collection of student products Preestablished guidelines for what will be included Student selection of some works that will be included Student self-reflection and self-evaluation Progress documented with specific products and/or evaluations Portfolio conferences between students and teachers
A PORTFOLIO IS: Purposeful Systematic and well-organized Prestablished guidelines are set-up Students are engaged in the selection of some materials Clear and well-specified scoring criteria
PURPOSE OF PORTFOLIO Showcase portfolio: Selection of best works. Student chooses work, profile are accomplishments and individual profile emerges. Documentation portfolio: Like a scrapbook of information and examples. Inlcudes observations, tests, checklists, and rating scales. Evaluation portfolio: More standardized. Assess student learning with self-reflection. Examples are selected by teachers and predetermined.
ADVANTAGES OF PORTFOLIO Students are actively involved in self-evaluation and self-reflection Involves collaborative assessment Ongoing process where students demonstrate performance, evaluate , revise , and produce quality work. Focus on self-improvement rather than comparison with others Students become more engaged in learning because both instruction and assessment shift from teacher controlled to mix of internal and external control. Products help teachers diagnose learning difficulties clarify reasons for evaluation Flexible
DISADVATNTAGES Scoring difficulties may lead to low reliability Teacher training needed Time-consuming to develop criteria, score and meet students Students may not make good selections of which of which material to include Sampling of student products may lead to weak generalization Parents find the portfolio difficult to underdstand
STEPS IN PLANNING AND IMPLEMENTINGPORTFOLIO ASSESSMENT1. Determine the the purpose2. Identify physical structure3. Determine sources of content4. Determine sources of content5. Determine student reflective guidelines and scoring criteria6. Review with students7. Portfolio content supplied by teacher and/or student8. Student self-evaluation of contents9. Teacher evaluation of content and student self- evaluation10. Student-teacher conference11. Portfolios returned to students for school
PURPOSE Based on specific learning targets Ideal for assessing product, skill, and reasoning targetsUses: Showcase portfolio-to illustrate what students are capable of doing Evaluation of portfolio-standardization of what to include For parents-what will make sense to parents“Provide specific attention to purpose and corresponding implications when implementing a portfolio.”
PHYSICAL STRUCTURE What will it look like? How large will the portfolios be? Where are they stored so that students can easily access them? Will it be in folders or scrap books? How will the works be arranged in the portfolio? What materials are needed to separate the works in the portfolio?
SOURCES OF CONTENT Work samples Student and teacher evaluationsGuidelines: Select categories that will allow you to meet the pupose of the portfolio. Show improvement in the portfolio Provide feedback on the students on the procedures they are putting together Provide indicator system
SELF-REFLECTIVE GUIDELINES AND SCORING Establish guidelines for student self-reflection and the scoring criteria Scoring guidelines are explained to the students before they begin instruction
IMPLEMENTING PORTFOLIO ASSESSMENT Review with students: Explain to students what is involved in doing a portfolio. Begin with learning targets Show examples Give opportunities to ask questions Provide just enough structure so that they can get started without telling them exactly what to do. Selection of content will depend on the age and previos experience of students Students and teachers decide together what to include with nonrestrictive guidelines
SOME ORGANIZATION Include table of contents Brief description of activities Date produced Date submitted Date evaluated
STUDENT SELF-EVALUATIONS Reflective and self-evaluation activities need to be taught. Some guide questions for students: Can you tell me what you did? What did you like best abut this sample of your writing? What will you do next? Self-reflective questions: What did you learn from writing this piece? What would you have done differently if you had more time? What are your greatest strengths and weaknesses in this sample? What would you do differently if you did this over?
PEER EVALUATIONS Analysis and constructive, supportive criticism of strategies, styles, and other concrete aspects of the product. Can include comments or a review by parentsTeacher evaluations: Checklist of content Portfolio structure evaluation: selection of samples, thoroughness, appearance, self-reflection, and organization. Evaluation of individual entries: use rubrics Evaluation of entire content: use rubrics
STUDENT-TEACHER CONFERENCES Conference is conducted with sudents before returning the portfolio Scheduled throughout the school year; some have it monthly Clarify purposes and procedure with students, answer questions and establish trust Give guidelines to prepare for each conference Allow the students to do most of the talking Have students compare your reflections with theirs Weaknesses and areas for improvement need to be communicated –show them what is possible for progress
STUDENT-TEACHER CONFERENCES At the end of the conference there is an action plan for the future Limit the conference to no more than 10 minutes Students are encouraged to take notes Focus on one or two major areas of each conference-helps to have a thoughtful discussion
Effect of Previous Practices: rank students on achievement by graduationNew Expectation: Assure competence in Math, Reading, Writing, etc. Implications? Assessment and grading procedures had the effect of helping some students succeed now must serve to help all students succeed.
MISTAKEN BELIEFS ABOUT HOW TO USE ASSESSMENTTO SUPPORT SCHOOL IMPROVEMENT: 1. High-stakes tests are good for all students because they motivate learning 2. If I threaten to fail you, it will cause you to try harder 3. If a little intimidation doesn’t work, use a lot of intimidation
4. The way to maximize learning is to maximize anxiety5. It is the adults who use assessment results to make the most important instructional decision.
PROFOUND MISTAKE Teachers and leaders don’t need tounderstand sound assessment practices – thetesting people will take care of us.COUNTER BELIEF They do need to understand soundassessment practices.
ASSESSMENT LEGACY1. Assessment has been far more a matter of compliance than of teaching and learning2. Disregard of the information needs students and teachers who make the most frequent and highest impact decisions3. Assessment that drive as many students to give up in hopelessness as they spur to more learning4. And we fail to provide practitioners with the assessment understandings needed to help