Assessment inCurriculum DesignRahma Kaffa,Younes Taia, Zakaria JamaatiCurriculum Development CourseProf. Mohamed El Fatihi
OutlineI. Assessment Rational of incorporating in CD, proceduresII. Types of Assessment.a. Placement assessmentb. Observation of learningc. Achievement assessment (short-term and long-term)d. Diagnostic Assessmente. Proficiency assessmentIII. Approaches to Assessment:a. Validityb. Reliabilityc. Practicality
Assessment• is the process of collecting information byreviewing the products of student work,interviewing, observing, or testing.
Rationale for AssessmentGuide and encourage effectiveapproaches to learning; validly andreliably measure expected learningoutcomes, in particular the higher-orderlearning that characterizes highereducation; and define and protectacademic standards.
General principles underpin Assessment Procedures:• Assessment should be an aid to successfullearning and should encourage students to applytheir knowledge and skills in an analytical andcritical manner;• Modes of assessment should be specifically linkedto the learning outcomes of a unit and courseincluding outcomes for the development ofgraduate attributes;• Assessment requirements should be based on pre-determined and clearly articulated criteria thatdescribe standards of knowledge, skills,competencies and/or capabilities.
General principles underpin Assessment Procedures:• Students should receive feedback on their work ina timely manner that assists them to monitor theirprogress towards the achievement of specifiedlearning outcomes and to improve the quality oftheir work;• Assessment should be inclusive and equitable forall students;• Assessment should be valid and reliable;• Students and staff should act in accordance withthe Academic Regulations, the Academic HonestyPolicy and the Assessment Policy and AssessmentProcedures.
How to create assessmentGuiding Questions for CreatingAssessments When creating anyassessment, you should use the samebackwards planning mindset that framesall of your instructional planning. To helpyou backwards plan your assessments,consider the following questions:
How to create assessment1. What are your learning goals orstandards?2. What evidence would you need fromstudents to demonstrate mastery of thestandards/goals?3. What method will you use to assess?4. What are the criteria for success? Whatare the characteristics of a high qualityresponse?
Placement assessmentThe learners are assessed at thebeginning of a course to see what level ofthe course they should be in. The aim ofthis testing is to ensure that the course isnot going to be too easy or too difficult forthe learner.• It often has to be done just before a coursebegins.• The results have to be available quickly.
Possible placement testsA grammar-based test could ask the learners tocomplete sentences, choose appropriate itemsfrom multiple-choices, locate the items referred toby reference words in a text, or complete a clozetest or dictation.Tests which focus on vocabulary. For example, aplacement test which consists of a computerizedyes/no test where the learners see individualwords without context from various frequencylevels and have to indicate whether they know theword or not. About one-third of the words arenonsense words which are used to correct forover-estimation by the learner.
Tests which focus on language in terms ofpronunciation use include interviews, roleplays, listening tests with message-focusedquestions, reading passages with message-focused questions and composition writing.Note:Teachers should therefore not feel that they have toinclude mainly language use tests in a placement test, andshould feel comfortable with making use of tests focusing onknowledge of language items.Possible placement tests
Observation of learningWhile the course is running, theactivities that the learners do are carefullymonitored to see if each particular activityis likely to achieve its learning goal. Thisinvolves technique analysis andclassroom observation.
• Monitoring learners‟ progress in a course canoccur at the level of the learning activity.• This monitoring does not assess the learnersbut is directed towards the tasks that they do.• The purpose of the monitoring is to see if it isnecessary to make changes to the learningactivities in order to encourage learning.Observation of learning
Aspect that should be Considered in Observationof learning• There are four questions that should beasked when observing learning activities(Nation, 2001: 60–74).1. What is the learning goal of the activity?2. What are the learning conditions thatwould lead to the achievement of thisgoal?
Aspect that should be Considered inObservation of learning3. What are the observable signs that theselearning conditions are occurring?4. What are the design features of theactivity that set up the learningconditions or that need to be changed toset up the learning conditions?
Example:In a spoken fluency development activity. The learning goal of the activity is toDevelop Spoken fluency• The learning conditions are (assumed)- The learners are focused on the meaning of thetask.- The task involves very limited languagedemands, i.e. all the language needed to do thetask is familiar to the learners.
Example: The observable signs:- involvement in communicating with apartner.- reasonably high speed of speaking with asmall number of hesitations.- some signs of comprehension by the listener.
The design features of the activity.include opportunity for preparation, a chanceto repeat the task several times to differentlisteners, a familiar topic and an involvedlistener, and time pressure.Example:
Achievement AssessmentShort-term Achievement Assessment.• Short-term achievement assessment monitors theproduct of activities, or small sets of activities applied inthe classroom.• The goal is to check whether learners are progressingthroughout the course time.• Provides feedback for both Ts and Ss.• They can have motivational purposes:– Oblige learners to do work they are required to do in due time.– Develop feeling of achievement for Ss as they succeed on tests
Achievement AssessmentShort-term Achievement Assessment.• In courses where there fewer classes, this type ofassessment is not feasible.• Short-term objectives are effectively assessed if clear„performance objectives‟ are set ahead for some learninggoals.• A Performance Objective: (brown, 1995) It is a specific,accountable and measurable description of the desiredaccomplishment we want the S to attain by the end ofthe assessment period.– 1. Subject. 2. The performance.– 3. The conditions 4. The measure.– 5. The criterion.
Achievement AssessmentShort-term Achievement Assessment.• The subject: who should achieve the objective of thelesson.• The performance: what the learner should be able toperform.• The conditions: under what conditions the performanceof the learner should occur.• The measure: how the performance will be tested.• The criterion: what level of the performance must bereached.
Achievement AssessmentShort-term Achievement Assessment.• E.g.:The learner should be able to quickly read andcomprehend a 550 word text containing only well-knownvocabulary and constructions at a speed greater than 250words per minute and with at least 70 per centcomprehension as measured by ten four back to the textand while they answer the questions.
Achievement AssessmentShort-term Achievement Assessment.• Using performance objectives is recommended to clarifygoals for both T and learners, and to monitor progress.• Recently, most curriculum designers do write POs forsome of the goals of the course.• Generally, STAA should provide clear record of theprogress that can be easily interpreted:– Should be administered in a user friendly format. (motivation)– Should not be time consuming.– Should be done at regular point of the class.– Activities should be familiar to the Ss
Achievement AssessmentLong-term Achievement Assessment.• It measures both the achievement of learners during aperiod of a course and the effectiveness of the course.• The length of the course determines the number of testslearners will sit for:– A final achievement test.– Tests part of the way through a course.• The latter may be done to pick up learners that arestruggling with the course.• They may be also used to assess the first part of material
Achievement AssessmentLong-term Achievement Assessment.• They have three characteristics:– 1. They are based on material taught in the course.– 2. Learners do know ahead the test‟s type of questions andmaterial will be covered.– 3. They are criterion referenced (standard) # Norm referenced.• Achievement tests may also be considered Mastery testswhere the learners have to reach a criterion of mastery topass to the next level. If they fail they may be repeatedopportunities to study for the test and sit for it.
Achievement AssessmentLong-term Achievement Assessment.• Rationale behind masterytests:– If something is to belearned, it should belearned.– Every learners is capable ofachieving mastery,regardless of time needed.– T responsibility is to Ss andto learning.– The mastery of area isrewarding and motivatingfor both T and Ss.• Opposing view:– T‟s responsibility is to thesubject matter, not to thestudent.– One role of courses is toinform employers andinstitution‟s admissionresponsible thatcapabilities are different.– Learners are responsiblefor their own learning andmanaging it.
Achievement AssessmentLong-term Achievement Assessment.• It is advisable that questions in achievement tests shouldbe slightly different to the activities done in class.• Language learning is supposed to be able to transfer to avariety of uses.• General: Achievement tests are:– A very important component of curriculum design..– Provide feedback for Ts, Ss, curriculum designers.– Have a “washback” effect on teaching and learning (motivatesand guides learning)
Diagnostic Assessment• It is implemented to find out the weaknesses and gaps inlearners‟ knowledge so an action can be done aboutthem and not to waste time on teaching what alreadystudents know well.• DA is a very good instrument of needs analysis bothbefore a course starts and during the course.• The results of DA determines what should a coursecover. Thus a good DA is accurate and easy and easy tointerpret in terms of actions to be taken.
Diagnostic Assessment• Some other types of assessment can provide us withdiagnostic information, such as achievement tests,proficiency test, achievement tests, or placement tests.• There some designed specifically for diagnosis:– The vocabulary level test (Schmitt et al., 2002)The test is designed to determine what should be the focus of thecourse: whether high frequency vocabulary or low frequencyvocabulary as teaching each category requires different teachingstrategies.
Diagnostic Assessment• The findings of diagnostic assessment can thus have aimportant effect on the design of a course.• DA can be also in different forms:– Analysis of language use ( written compositions, reading tasks,spoken performances).– Observation of learner performance (the process of writing,note taking, etc.)– Learner self-assessment where learners work with check lists toindicate their perceived areas of strength and weakness.It is very difficult to determine learners subjective concernsfrom objective judgments
Proficiency assessment• A unique feature about language proficiency test is itdraws its items from the language itself not from anycourse.• The purpose of proficiency assessment is that to reflectto what extent a learner knows of the language or aspecific part of it.• The well know proficiency tests are:– Test of English as Foreign Language (TOEFL)– International English Language Testing System (IELTS)
Proficiency assessment• These tests sometimes awaits a learner at the end of acourse.• They may be working as a criterion-referenced test todetermine the admission of a learners into an Englishmedium university:– TOEFL iBT (internet-based) is out of 120.– IELTS test uses a 9-point scale.• In some cases these tests may act as norm-referenced toshow learners their level of proficiency in relation toothers
Proficiency assessment• TOEFL as a criterion referenced test can have animportant washback effect on a language course thatprecedes it.• It can have the effect of making the content of the courseresembles the content and test items formats in the test.• Courses that predict what will be in the test and do copythe format of the test in their learning activities doconsider proficiency test as achievement tests for thecourse, except that the course is drawing items from oldtests.
Proficiency assessment• For these courses the presence of a proficiency test at theend of the course affect different teachers in differentways. Thus, the quality of channels communicationbetween test designers and test designers and the Tsand learners is paramount.• There are constraints of proficiency test at the end of thecourse:– 1. courses become commercial business for publishers whichtheir books may mislead sometimes the test takers.– 2. making a course look like a test makes dependent to the test.– 3. Remove the test, for some international universities who offerintensive English programs, or their own proficiency tests
Proficiency assessment• Proficiency tests are a good source for evaluation datafor a program as they represent an independentmeasure of the relevance, and adequacy of a languagecourse.
Approaches to assessment• All assessment needs to be checked to see that it is doingits job properly and that is not causing unnecessarywork.• Most investigate procedures including the tools for needanalysis, course evaluation procedures, and tests andother measures for assessment can be examined byconsidering THREE criteria: Reliability, Validity andPracticality.• We will discuss these three criteria in relation to tests.
ReliabilityReliability is synonymous with the consistency of a test,a survey, observation or other measuring device.Let us consider this example:Imagine stepping on your bathroom scale and weighing140 pounds only to find that your weight on the same scalechange to 180 pounds an hour later and 100 pounds anhour after that. So, base on the inconsistency of this scale,any research relying on it would certainly be unreliable,and its results would not be accurate.
• So, Reliability is the degree to which an assessment toolproduces stable and consistent results.• The reliability of a test refers to its degree of consistencyand repeatability.• Statistically, reliability is measured by having thelearners sit the test twice and seeing if the learners getnear enough to the same result, or more commonly, bysplitting the scores on the individual test items into twoequal groups and seeing if the learners get the samescore on both groups.
Characteristics of a reliabletest• A reliable test is always given under the same conditions,which include the amount of time allowed for the test,and the instructions are always presented in the sameway.• It is consistently marked, because if the marking isunreliable, then the results are unreliable. Reliablemarking is consistent, and consistency is helped byhaving some kind of answer key or well-thought-outscoring procedure. Markers need to be trained.
• It has many questions or points of assessment, then ifsomething is wrong with one or two of the points, thiswill not greatly influence the result.• Its questions and instructions are clear andunambiguous.
Validity• A valid test measures what is supposed to measure. Avalid achievement test measures what has been learnedon the course. A valid listening test measures skill atlistening.• The most practical ways for a teacher or curriculumdesigner to check the validity of a test are to look at itsface validity and content validity.
• Face validity:Face validity simply means that if the test is called a readingtest, does it look like a reading test? If it is called a vocabularytest, does it look like a vocabulary test?Face validity is important because it reflects how the learnersand perhaps their parents, and other teachers will react to thetest. For example, a vocabulary test which has a low facevalidity, it presents words in isolation without a context, it doesnot ask for the learner to give a meaning, and it does not requirehim or her to use the vocabulary, this could affect the learners’reaction to the test and their acceptance of its results.It is an advantage if tests look like the kind of test they aresupposed to be.
• Content validity:It is little like face validity, except that the decision-making about validity is not made by looking at the test’s“face”, but by analyzing the test and comparing it to whatit is supposed to test. For example, To find the contentvalidity of an achievement test, we would have to look atthe part of the course that was being tested and list theitems and skills taught. Then we would have to look at thetest and list the items and skills tested, if these two listsmatched each other quite closely, or if the test involved arepresentative sample of the course list, we could say thetest has content validity.
• In other words, validity exists when a test isused for the purpose for which it wasdesigned.
Practicality• Not only must a test be reliable and valid, it must also bepractical. Practicality is examined by looking at the costinvolved in administering and scoring the test, the timetaken to administer and sit the test, the time taken tomark the test, the number of people needed toadminister and mark the test, and the ease ininterpreting and applying the results of the test.• Tests can be made more practical by being carefullyformatted for easy marking, by being not too long, andby using objectively scored items such as true/false ormultiple choice.
• The requirements of practicality, reliability and validitydo not always agree with each other, for example shorttests are practical but not very reliable or valid.Sometimes it is necessary to sacrifice some practicality toget a valid test (For example an individual oral interviewfor 200 people), or to sacrifice validity to make the testmore practical (multi-choice vocabulary tests). Wherepossible, reliability and validity should be preferred overpracticality, but usually compromise is necessary.
• Assessment is a major source of information for theevaluation of a course and thus its gradual improvement.Assessment also contributes significantly to the teacher’sand learners’ sense of achievement in a course and thusis important for motivation.• Curriculum design should include the planning of a well-thought-out program of assessment of various kinds.
References• Macalister, J. (2010). Language Curriculum Design. NYC:Routledge.• Retrieved from website on 20 May, 2013:http://www.dartmouth.edu/~hrs/profldev/performance_management/performance_objective.html