1. UNIVERSIDAD CATOLICA DE LA SSMA. CONCEPCION THEORETICAL FRAMEWORK FACULTAD DE EDUCACION DEPARTAMENTO DE LENGUASEVALUACION DE LA COMPETENCIA COMUNICATIVA EN INGLES KARLA ESSMANN TORRES ANALIA NOVA GONZALEZ DOCENTE ROXANNA CORREA PEREZ FECHA CONCEPCION, 2013
2. Theoretical Framework This essay attempts to provide an overall view of an issue presented in the field oflanguage teaching and learning, which is assessing on Second Language Acquisition(SLA). This has been considered a complex topic in the area of teaching and learning alanguage (Fernandez, 1996), which could be either a Foreign Language (FL) or a SecondLanguage (SL). Since SLA is a complex matter, to asses it, it might be equally difficult aswell, and an optimum instrument is needed in order to obtain real and trustworthy results. Regarding this subject, in the following essay it will be analyzed the perspectives ofthe authors who present principles or qualities that must be considered when designing atest on FL or SL. These authors are: Brown D. (2003), Coombe,Ch. et al. (2007) andBachman, L. F. & Palmer A. S. (1996). These authors convey and agree that concerningassessment there are different principles or qualities that should be taking into accountwhen designing and developing a test in order to make it effective. Notwithstanding, theydiffer in the number of principles and in some cases also their names. This essay focuses onsome of those differences regarding the available theory of the authors previouslymentioned. First, it is going to be reviewed Coombe et al. (2007) perspective, and secondlyBrown’s (2003) view regarding principles of assessment on SLA. To continue, the authors’perspectives will be compared in order to highlight some differences on their viewpoints.The following authors to be analyzed in this essay will be Bachman, L. F. & Palmer A. S.(1996). After this, the authors will be compared to the perspective of Coombe et al. (2007).Finally some conclusion will be stated at the end of the essay.
3. According to Coombe et al. (2007), there are several principles that are connected togood test design described as “The cornerstones of testing”. These principles are:usefulness, reliability, practicality, washback, authenticity, transparency, and security. The author defines the principle of usefulness as the most important one, since a testhas a specific purpose, a particular group to be tested, and a specific target language. The principle of validity aims to measure “what you teach and how teach it”. Thisprinciple is subdivided in: a) Content validity, which means that a test has to assess contentusing a format familiar to the test- takers. b) Construct validity, which refers to themethodology of language learning and the type of assessment. c) Face validity, whichmeans that a test looks as if measures what it is expected to measure. The principle of reliability stands for trustworthiness in terms of scoring. There arethree factors regarding reliability which are: a) test factors, b) administrative factors, and c)affective factors. The principle of practicality includes the cost of development, maintenance,adequate time, resources, ease of marking, availability of suitable/trained graders, andadministrative logistics. The principle of washback refers to the effect of testing on either teaching orlearning. In other words this effect is connected to feedback which can be negative orpositive. The principle of authenticity refers to real- world situations and contexts whenfacing a task. Good process of assessment uses tasks which reflect real contexts where thestudents can actually use the target language. The principle of transparency is connected to the clear information when assessingand testing. This information includes the outcomes to be tested, the formats, item and
4. sections, time to complete the test, and graduating criteria. Transparency involves thestudents in the process of assessment, and leaves aside the myths surrounding the test. Security includes both reliability and validity in the testing process, giving studentsthe adequate information about tests; consequently, there is security in the results whentesting and assessing students’ performance. Brown (2003) is another author who has described another list of principlesregarding language assessment. The first principle on his list is called Practicality, this concept is referred tocharacteristics such as how expensive a test is, the appropriateness of the time, the facilitiesin terms of administration and also if it has a specific and time efficient scoring. To continue a second principle is defined by Brown (2003), which is reliability. Atest is reliable if it is consistent and dependable; this means that if a test is applied in twodifferent groups of students it must have similar results in both groups. However; there arefactors affecting the students score, one of them is named, a) Student-related reliabilitywhich has to do with Student’ physical or physiological state, for example if it is ill, sad,tired, etc. Another factor affecting the score is, b) Rater reliability, oriented to thesubjectivity and human errors that can affect the score. A third concept appears, which is, c)Test administration reliability that refers to the conditions of the test administration, forexample a noisy place, the amount of light, temperature variations, etc. These elements caninterfere with the real score. Finally, a fourth concept is found, which is, d) Test reliability,that has to do with the aspects of the test itself, if the test is too long and the students areconditioned by a clock ticking, their results wont be the ones they could obtain without thispressure.
5. The following principle described is called Validity. When the coherence betweenassessment result and assessment purpose, match with the assessment intention, a test isvalid. Inside validity are found branches such as, a) Content related evidence, this aspect ofvalidity is identify in a test when could be clearly defined the achievement which is beingmeasured. A second kind of validity is found which is, b) Criterion related evidence, andthis is reached when the criterion of a test has been accomplished. Then a third concept ofvalidity appears called, c) Construct related evidence, which is the theoretical constructwhen assessing any skill. To follow, another branch of validity comes into view, d)Consequential validity; this concept is related to the impact of assessing on accuracy inmeasuring intended criteria, the impact on the preparation of the test- takers, the effect onthe learners, and the social consequences of a test. Finally, the last branch supportingvalidity is e) Face validity which means that the assessment looks fair, and really measureswhat it is supposed to measure from the point of view of the test- takers. Regarding authenticity the author states that there are some factors that determinethis principle such as the natural way of the language in a task, contextualized items ratherthan isolated, meaningful and interesting topics for the students, and finally real worldsituations tasks. The principle of washback is connected to the effect that tests have on instructionregarding how students prepare for a test; moreover, the appropriate feedback regardingprogress and achievements, and finally its effects on the students. Regarding the authors previously mentioned (Coombe et al, 2007 and Brown, 2003)it can be stated that even though both authors describe a list of principles regardinglanguage assessment; they differ in the number of principles on each list. Since Coombe etal(2007) explain a list of eight principles, whereas Brown (2003) explains a list of just five
6. principles. There is also a difference regarding one principle which is described by thedifferent authors, which is validity, while Coombe et al(2007) describe just three branchesof it, Brown, 2003 describes five branches in relation to this concept. Concerning thisdifference it might be seen that Brown(2003) considers the concept of validity as the mostremarkable one, whereas Coombe et al (2007) considers as the most important principleusefulness, since each test has an intention, a purpose and a specific groups of test takers.The authors also describe this principle at the beginning of “the cornerstone of testing”whereas Brown (2003) starts by saying that each principle is equally important. According to Bachman and Palmer (1996) when developing a language tests themost important consideration is the usefulness (purpose). They present a model of testusefulness which contains six qualities. The first one is reliability which is the consistency of measurement and theconsistency of scores from one set of tests and test tasks to another. The second one is construct validity which consists of the interpretation made onthe bases of test score. Teachers have to demonstrate the validity of the interpretation of agiven test score, since evidence is needed in order to demonstrate that it reflects the area(s)of language ability that is going to be measured The third one is authenticity which consists of the degree of correspondencebetween the language used in specific domains and the characteristics of a given languagetest or task. The fourth one is interactiveness which consists of the extent of the type ofinvolvement on the test taker’s individual characteristics in accomplishing a test or task; themost important characteristic in this case is the test taker’s language ability (languageknowledge, metacognitive strategies).
7. The fifth one is impact, this concept attempts to look forward at what will be theresults for the future. The impact of atest in society and in the educational system operatesin two levels: micro level (individuals) and macro level (educational system and society). Finally, the last quality is practicality which is the relationship between theresources, development, and the use of a test. Regarding Coombe et al (2007) and Bachman and Palmer(1996) it can be stated thatthe first difference between them is the number of criteria when describing the list of eitherprinciples or qualities regarding assessment on SLA. While Coombe et al(2007) describeseight principles, Bachman and Palmer(1996) describe six qualities. Concerning theterminology of their principles/qualities it can be stated that Coombe et al(2007) do notconsider the principles/qualities of impact and interactiveness, whereas Bachman andPalmer(1996) do. Another difference between the descriptions of principles/qualities is thatCoombe et al(2007) describe washback as one single principle, whereas Bachman andPalmer(1996) describe washback as a branch of the quality of impact. Regardingprinciples/qualities another difference can be noticed, concerning validity; while Coombe etal(2007) considers validity as a single principle with three branches (content, construct, andface validity) Bachman and Palmer(1996) considers as one principle/quality constructvalidity, leaving aside the other two branches of this concept. Finally, another differencebetween them is the focus of their criteria; while Coombe et al (2007) might focus mainlyon the test itself Bachman and Palmer (1996) focus on the impact of the test in society. As it was said at the beginning of this theoretical framework, the purpose of it is tohave a view of assessing on second language acquisition (SLA). Three perspectives wereseen, those were: Brown D. (2003), Coombe, Ch. et al. (2007) and Bachman, L. F. &Palmer A. S. (1996), in order to understand the qualities and principles which are
8. considered at the moment of assessing second language acquisition, it can be said that is acomplex issue because there are several factors that determine the effectiveness andtrustworthiness of a test. Having compared the three authors it can be concluded that there exist elements atthe moment of assessing on second language acquisition which are called either principlesor qualities according to the authors previously mentioned. It can be concluded also, thatthere are some principles/qualities that are mentioned in the three chapters, for examplereliability, authenticity and practicality. Validity is also mentioned in all of the authors;notwithstanding, Brown (2003) might be more specific and detailed describing thisprinciple. This framework is based mostly on Coombe,Ch. et al., 2007, because thisperspective was more convincing and understandable for us, because the principles weredescribed in detail. To sum up, assessing on second language acquisition is a remarkablematter for future English teachers, and the reading and analysis of the authors previouslymentioned will help us to know how to design an optimum test at the moment of assessinglearners of FL or SL.
9. ReferencesBachman, L. F. & Palmer A. S. (1996)Language Testing in Practice. Oxford Universitypress.Brown, D. (2003) Language Assessment, Principles and Classroom Practices.Longman.Coombe, Ch., Folse, K. &Hubley, N. (2007) A practical guide to assessing English language learners. University of Michigan Press.Fernández, P. (1996) Modelos sobre la adquisición del lenguaje.Didactica 8. 105- 106. Servicio de publicaciones UCM: Madrid.