EFFECTIVENESS OF RATIONAL CLOZE – TESTAS INTEGRATIVE ASSESSMENT IN HISTORYAT LAGUNA STATE POLYTECHNIC UNIVERSITY SAN PABLO CITY CAMPUS ACADEMIC YEAR, 2012-2013 A Research Paper Presented to the Faculty of the College of Teacher Education Laguna State Polytechnic University San Pablo City Campus San Pablo City In Partial of Fulfillment Of the Requirements in Methods of Research for the Degree Bachelor of Secondary Education Major in Social Studies MARILYN ATIENZA MENDOZA October 2012 Chapter I
The Problem and Its BackgroundIntroduction The cloze procedure is considered by many as an integrative method ofassessment, in contrast with discrete point methods, The Cloze procedure deals withseveral linguistic components at once, focusing more on language use, and typicallyrequiring the examinees to read and comprehend a substantial amount of discourse. Muchresearch has been devoted to the validity and the reliability of Cloze tests. Cloze tests arereported to have moderate to high correlations with standardized tests and their subtestssuch as listening comprehension and reading comprehension. The report on highcorrelations between Cloze tests and International Journal of Human and Social Sciencestests of listening comprehension, writing, reading comprehension, and oral interview. Thefindings of these researchers reveal a relation between the scores on Cloze and globallanguage ability tests. As for C-test, since its inception in 1981, its principles have been applied to morethan 20 languages. C-tests have been used in numerous contexts and for various purposes.C-tests, then, figure prominently in the fields of language testing and assessment. C-tests, like the classic Cloze test, are an operationalization principle of reduced test.Although there are weaknesses in the reliability of cloze procedure, it is widely used inEnglish teaching. As Legenza and Elijah (2001) point out, many studies have beenconducted on cloze procedure concerning the construction of cloze tests, the relationshipbetween cloze and reading comprehension, and the usefulness of cloze as a teaching
technique. Cloze procedure has been used in English teaching in accordance to Historysubject for decades since 1953. It was firstly used as an instrument for assessing thereadability of written materials for school children in the United States (Brown, 2002).Then it was used in teaching for different purposes. Cloze procedure is not only valuable in providing information as to how readablethe particular texts are for the students who are going to have to use them, but alsoinvaluable as a means of comparing different text. Cloze procedure is also used fortesting purposes. It seems that a wide range of skills like vocabulary, grammar, structure,and reading skills are involved in the process of completing a cloze procedure. Manyresearchers indicate that cloze procedure is a good test of overall English languageproficiency and in History. For example, Ahluwalia (2005) claims that cloze procedure isan integrative, global measure of language competence. She explains that cloze testsmeasure the grammar of expectancy underlying the skills of thinking, understanding,speaking, reading, understanding and writing. For Cohen (2001 cited in Ahluwalia, 2005)cloze procedure measures global language competence consisting of linguisticknowledge, textual knowledge, and knowledge of the world. As it calls on examinee touse knowledge such as vocabulary, grammar, sentence construction, text structure,cohesion and the reader‘s prior background knowledge. Askes (2002) regards cloze procedure as one of the integrative tests (global tests)that integrate language components into a total language event, which requires anintegrated performance from the learner in a meaningful context. Thus for Steinman(2002), a cloze can replace the sections on test structure, written expression, vocabulary,
and reading comprehension not only in English subject as well as different areas oflearning. The reason behind giving a Cloze test and obtaining a test score is interpretingthat score as an indicator of what a test taker knows or what he/she can do with thatknowledge. Furthermore, our interpretation of that test score forms the basis for decisionmaking. As such, when using a test score, we make an implicit link between testperformance and a domain of language knowledge the test taker has or something the testtaker can do with language in some language use domain beyond the test itself. In otherwords, when we use test scores, we are essentially reasoning from evidence, using thetest score as the evidence for inferences or interpretations and decisions we want to make.Yet, we cannot simply draw on test score to make inferences and decisions withoutefficient justification. If we want to use a test score for a particular purpose, we mustjustify it through a rationale and supporting evidence. As Bachman (2005) puts it, "Weneed to demonstrate, with logical argumentation and empirical evidence, that theintended interpretations and uses are valid." Validity in testing and assessment hastraditionally been understood to mean "discovering whether a test measures accuratelywhat it is intended to measure", or uncovering the "appropriateness of a given test or anyof its component parts as a measure of what it is purposed to measure".Background of the Study Cheng (1999) claims that the term ‗cloze procedure‘ was first developed byWilson Taylor in 1953. He explains that the term cloze derived from the Gestaltpsychology concept of ‗closure‘. It describes a tendency that humans have to complete a
familiar but not-quite-finished pattern. Ellis (2004) further explain that it refers to thetendency of individuals to complete a pattern once they have grasped its overallsignificance. Rye (1982) explains that Cloze procedure is essentially a cognitive task. Thereader has to reason and construct suggestions to fill the gap on the basis of the evidencederived from the context the completion of meaning, based on understanding andreasoning is a cognitive task. Brown (2002) claims that it is not difficult to get people totake a cloze test because of the compulsive human need to fill gaps. Clarke (2001) was the first to study cloze procedure for its effectiveness as aninstrument for determining the readability of materials in the reader‘s native language in1953. After that initial study, it was investigated for its appropriateness as a measure ofreadability of L1 and L2 materials. In the 1960s, studies focused on cloze tests as ameasurement of reading comprehension in L1 and L2. During the 1970s, cloze testsbegan to be used as a measurement of overall L2 proficiency (Ahluwalia, 1992:82).Today, cloze tests are widely used in some places (such as China) and as part of somelarge-scale language. What is a cloze test? A standard cloze test is a passage with blanksof standard length replacing certain deleted words which students are required tocomplete by filling in the correct words or their equivalents. During traditional clozetesting, every fifth word is removed from a 250-500 word reading passage, and isreplaced by a standard-length blank space (Helfeldt et al, 1986:216). Usually, no word isomitted either in the first or the last sentence of the passage. Students are required tosupply either the original word of the author or an appropriate equivalent in the blankspace. Many studies show that the reliability and the validity of cloze tests are affected byfactors like the rate of deletion, nature of the text and scoring systems, etc. ‗Deletion rate‘
refers to the frequency of deleting words. As Steinman (2002:293-294) explains, there aretwo options in designing a cloze test according to its deletion rate: a random cloze or arational cloze. A random cloze deletes every nth word consistently, so that all classes andtypes of words have an equal chance of being deleted. A rational cloze is the one inwhich a specific type of word is deleted according to a linguistic principle, such as nouns,verbs, adjectives, etc. Alderson (2000) cited by Yamashita (2003:269) clearly differentiates betweenthese two types of format by calling the rational cloze ‗gap-filling tests‘ and confining theterm ‗cloze‘ only to the random cloze. Ahluwalia (1992:83-84) states that differentdeletion rates would result in either increasing the difficulty of the text or inunpredictable differences in the difficulty. Another factor that would affect the reliabilityand the validity of the cloze is the nature of the text. The nature of the text such as itsfamiliarity level and difficulty level would improve or hamper cloze performance(Ahluwalia, 1992:86-88). However, Alderson (1980, 1983 cited in Ahluwalia, 1992:86) and Yamashita(2003:286-287) argue that cloze test performance is not directly related to the difficultylevel of the text; it involves other factors such as scoring procedures, and contentfamiliarity for the readers. Different scoring methods include: exact word method,multiple-choice scoring method and contextually acceptable word method. 16‗Exactword method‘ requires the examinee to provide the original word deleted from the textwhile ‗contextually acceptable word method‘ allows for the words that fit the gap eitherto be synonyms of the deleted.
Statement of the Problem This study aims to know the effectiveness of cloze – test as integrativeassessment in history at Laguna State Polytechnic University, San Pablo City Campus,Academic Year, 2012-2013. 1. What is the profile of the respondents in terms of: 1.1 Age; 1.2 Gender; 1.3 Grade point in history subject? 2. What is the level of perception of the following assessment – related variables as perceive by the respondents: 2.1 Controllability; 2.2 Reliability and validity; and 2.3 Usability? 3. What is the mean score of the respondents in Cloze – test? 4. Is there a significant relationship between the person – related factors and their perception of the following variables: 4.1 Controllability; 4.2 Reliability and validity; and 4.3 Usability? 5. Is there a significant relationship between the level of perception of the respondents and their mean score in Cloze – test?
6. Is the grade point average in History is significantly related to the result of Cloze – test?Scope and Limitations This study focused on the effectiveness of Cloze test in measuring performancerelated to History subject. It is limited only to twenty - nine (29) respondents of the third year students whoare enrolled in Academic Year 2011-2012 at Laguna State Polytechnic University – SanPablo City Campus, they were randomly selected. In this study, the researcher used instruments such as survey questionnaireintended for students and 25 items for measuring their performance using Cloze type oftest which served as the main instrument of the study.Significance of the Study The study will be significant to the following: Teachers and instructors especially for those specializing History, for it willbenefit them in a way of identifying the factors underlying proper assessment; Students, for it will help them to improve and be competent in studying Historyand provoke their interest. Parents, to furnish and give aid for their children an interest to learn thecomplexities of motivating a child in History;
And finally, for researchers, for it will help them gain satisfaction for knowingsome insights about the given problem and develop research skills to be more productivein the field that they had chosen to belong in the future.Theoretical framework Cloze technique, developed by Taylor (1952), is firstly designed as a tool formeasuring readability of texts, and now it has been used widely as an exercise or testmethod in foreign language teaching and testing. ―It works on the principle of closure andanticipation, in that the reader is required to reconstruct systematically mutilated passage‖(Hofman, J. E. & Habib-Allah, M., 1982:276). Actually, as early as 1971, John Olleradvocated that cloze testing was an excellent integrative test. Joseph Boyle and PeterFalvey (1994) claims that cloze test is one variant of integrative tests, too. And theybelieve that ―for cloze test, it is pragmatic expectancy grammar that constitutes the traitthat reflects linguistic competence‖. In addition, many researches from 1970s to 1980s oncloze test supply people with an objective viewpoint that cloze test is a reliable and validtest method to investigate the difficulty level of the text and the test-takers‘ capability inreading comprehension. It can be used as a part of standard integrative languageproficiency testing and a subsidiary tool for language teaching. Cloze is originally called ―cloze procedure‖, whose theoretical foundation is Gestaltpsychology. Those psychologists believe that when people observe the shape of an object,they will unconsciously fill in the broken part. Such is the case in reading process. Asentence or a passage, deleted some words, is just like a broken picture. People willcomplete it unconsciously according to their comprehensive language knowledge. The
Gestalt theory also claims that the more familiar with the picture the easier for people torecognize it. Expounded in language learning, the case is that the higher languageproficiency the higher ability to complete the broken passage. Besides the Gestalt theory, researchers try to use Redundancy Information theoryand Expectancy Grammar theory to explain the cloze test. In Longman Dictionary ofLanguage Teaching and Applied Linguistics, redundancy is defined as ―the degree towhich a message contains more information than is needed for it to be understood.Languages have built-in redundancy, which means that utterances harbor moreinformation than is necessary for comprehension. And 50% of normal language is said tobe redundant‖. Li Xiaoju (2001) also approves that in cloze test, people use theredundancy information of natural language to complete the broken language. Actually,this theory is crucial for Chinese learners when learning English. The process ofaccumulating language knowledge is the process of recognizing the redundancyinformation in English language. However, the information is necessary in that it canmake the meaning of the text much more accurate, although it is regarded as more than isneeded. For Chinese students, the recognition of the redundancy information is helpfulfor them to improve their sensitivity to the English language environment and theirability to understand the inner relationship in language itself and the consistency of itsusage, which enhances the students‘ language proficiency. Therefore, the cloze testremoves some redundancy information from the completed text to examine whether test-takers obtain the capability to comprehend the broken passage and to fill in the blanks ornot.
Expectancy Grammar was firstly advocated by Oller in 1976. Later Carter (1998)believes that it refers to language user‘s comprehensive language ability to use syntax,semantic and discourse knowledge. In reading process, the reader can use the ability topredict in what way the sentence will end and to what trend the text will develop. Clozetest investigates the very ability acquired by language learners through long-term learningand perception, and proficient mastering of language structure as well as basic useprinciple of the target language. In a broken passage, according to the context and theirlanguage ability, students can predict the syntactic and semantic function of the deletedword, along with the relationship between the word and the other part within thesentence.Conceptual Framework The conceptual framework of this study is presented to give view of the work tobe done. The inputs or the independent variables are the profile of the respondents mainly3rd year high school students and their perception regarding the effectiveness of a testwhich composed of Controllability, Reliability and Validity and Usability. The process is the strategy to be used such as generating data through the use ofquestionnaire. The possible output or the dependent variable is the result or theirperformance in History by means of using Cloze Test. It is assumed to be affected by theindependent variable.
Paradigm Independent Variables Dependent Variable I. Profile of the respondents Age Gender Cloze – Test Grade in history II. Assessment – related factors Controllability Reliability & Validity Usability Figure1. Shows the relationship between the independent variables (profile of therespondents and the assessment – related factors) and the dependent variable or the Cloze– test in History.Hypotheses 1. There is no significant relationship between the person – related factors and their perception to the assessment related variables. 2. There is no significant relationship between the level of perception of the respondents and their mean score in Cloze – test 3. There is no significant relationship between grade point average in History and their result of Cloze – test
Definition of Terms For a better understanding of this study, the following terms areoperationally defined.Cloze Test. Refers to integrative assessment in measuring higher cognitive task, dealswith several linguistic components at once, focusing more on language use, and typicallyrequiring the examinees to read and comprehend a substantial amount of discourse.Age. Refers to how old or young a particular person was.Gender. It refers to a particular person whether s/he is a male or a female.Grade Point Average. Refers to the recent average grade of 3rd year high school studentsduring 2012 – 2013.Controllability. It refers to the level of manipulation of effectiveness of Cloze type oftest.Validity. Refers to what purports to measure in a given context.Reliability. Refers to the consistency of Cloze test in assessing History subject.
Chapter II Review of Related Literature and Studies The chapters presents a selection of literature and studies that has abearing on present study. Most of the literature gathered comprises Cloze Testprocedure.Related Literature Cloze test has been changing and developing. According to different deletion andfill-in-blank methods, it can be mainly classified to fixed-ratio cloze, rational cloze,multiple choice cloze, C-test and banked cloze. Fixed-ratio cloze was proposed by Oller in 2005. It is constructed by deleting wordsaccording to a fixed pattern, either at even numbers (4, 6, 8, 10, etc.) or at odd numbers(5, 7, 9, 11, etc.). ―This procedure is intended to sample regularly various variants ofwords, some of which are governed by local grammatical constraints and wide disciplinevocabularies in sorted areas of History, Mathematics, Statistics, others of which aregoverned by long-range textual constraints‖. The advantages of this cloze variant areconvenience and high consistency, while there are also a lot of problems such as some ofthe blanks being too easy or too difficult, the broken passage sometimes arousing test-takers‘ dislike and resulting in a dilemma for testers to choose subjective or objectivescoring method, and being lack of control which possibly leads to examine the languagepoints the designer doesn‘t want to test actually. Rational cloze can also be called opencloze or gap-filling. The test developer control over the types of the words deleted such
as functional words, verbs or nouns, and thus the language traits measured. This selecteddeletion changes the random of cloze testing into objectivity and practicality. Just like infixed-ratio cloze, test-takers have to fill in the blanks in rational cloze according to theircomprehension capability. ―Rational cloze research and practice rests on the assumptionthat different cloze items can be explicitly chosen to measure different language traits‖(Chapelle & Roberta, 1990:122). The understanding advantage of rational cloze is thatthe test designer can select particular item to determine test-point. However, it isconfronted with the problem that subjective or objective scoring method, either. In order to solve the scoring problem of fixed-ratio and rational cloze, testdevelopers design multiple-choice cloze. This cloze test format requires test-takers toselect the correct answer from the provided options. It offers four or five options to eachblank, while only one is the correct answer and the others are distracters. There are noaccepted answers which get rid of scoring controversies. Generally speaking, highreliability is the specialty of multiple-choice cloze. However, the inappropriate options orexplanation of new words will give test-takers some hints, which possibly reduce thedifficulty level and the validity of test. Therefore, multiple-choice cloze demands testdevelopers‘ enough attention and obeying some strict rules. A cloze test (also cloze deletion test) is an exercise, test, or assessment consistingof a portion of text with certain words removed (cloze text), where the participant isasked to replace the missing words. Cloze tests require the ability to understand contextand vocabulary in order to identify the correct words or type of words that belong in the
deleted passages of a text. This exercise is commonly administered for the assessment ofnative and second language learning and instruction. The word cloze is derived from closure in Gestalt theory . The exercise was firstdescribed by W.L. Taylor in 1953. Words may be deleted from the text in question either mechanically (every nthword) or selectively, depending on exactly what aspect it is intended to test for. Themethodology is the subject of an extensive academic literature; nonetheless, teacherscommonly devise ad hoc tests. A Cloze test removes certain words from a sample of your text and asks users tofill in the missing words. Your test participants must rely on the context as well as theirprior knowledge of the subject to identify the deleted words. It‘s based on the Gestalttheory of closure—where the brain tries to fill in missing pieces—and applies it to writtentext.It looks something like this:If you want to __________ out whether your site __________ understand your content,you __________ test it with them. It looks a lot like a Mad Lib, doesn‘t it? Instead of coming up with a sentence thatsounds funny or strange or interesting, participants must guess the exact word the authorused. While Cloze tests are uncommon in the user experience field, educators have used
them for decades to assess whether a text is appropriate for their students, particularly inEnglish-as-an-additional-language instruction.Here‘s how to do it:Take a sample of text—about 125-250 words or so.Remove every fifth word, replacing it with a blank space.Ask participants to fill in each space with the word they think was removed.Score the answers by counting the number of correct answers and dividing that by thetotal number of blanks. A score of 60% or better indicates the text is appropriate for the audience.Participants who score 40-60%, will have some difficulty understanding the original text.It‘s not a deal breaker, but it does mean that the audience may need some additional helpto understand your content. A score of less than 40% means that the text will frustratereaders and should be rewritten. It might sound farfetched, but give this method a try before you dismiss it. In agovernment study on healthcare information readability, an expert panel categorizedhealth articles as either easy or difficult. We ran a Cloze test using those articles withparticipants—who had low to average literacy skills—and found that the results reflectedthe expert panel‘s findings. The average score for the ―easy‖ version was 60, indicatingthe article was written at an appropriate level for these readers. The average score for the―difficult‖ version was 39: too hard for this audience.
Cloze tests are simple to create, administer, and score. They give you a good ideaas to whether the content is right for the intended audience. If you use Cloze tests—eitheron their own or with more traditional usability testing methods—know that it takes a lotof cognitive effort to figure out those missing words. Aim for at least 25 blanks to getgood feedback on your text; more than 50 can be very tiring.When to test Test your content at any point in your site development process. As long as youhave content to test, you can test it. Need to convince your boss to budget for contenttesting? Run it through a readability formula. Got content but no wireframes or visualdesign? Run a Cloze test to evaluate content appropriateness. Understands the contentkey to a task or workflow? Display it in context during usability testing.What to test You can‘t test every sentence on your site, nor do you need to. Focus on tasks thatare critical to your users and your business. For example, does your help desk get callsabout things the site should communicate? Test the content to find out if and where thesite falls short.So get to it While usability testing watches what users do, not what they say they do, contenttesting determines what users understand, not what they say they understand.
Whatever your budget, timeline, and access to users, there‘s a method to testwhether your content is appropriate for the people reading it. So test! And then, eitherrest assured that your content works, or get cracking on that rewrite.A language teacher may give the following passage to students:“ Today, I went to the ________ and bought some milk and eggs. I knew it was going to rain, but I forgot to take my ________, and ended up getting wet on the way ________. ” Students would then be required to fill in the blanks with words that would bestcomplete the passage. Context in language and content terms is essential in most, if notall, cloze tests. The first blank is preceded by "the"; therefore, a noun, an adjective oran adverb must follow. However, a conjunction follows the blank; the sentence would notbe grammatically correct if anything other than a noun were in the blank. The words"milk and eggs" are important for deciding which noun to put in the blank; "supermarket"is a possible answer; depending on the student, however, the first blank could eitherbe store, supermarket, shop or market while umbrella or raincoat fit the second. The definition of success in a given cloze test varies, depending on the broadergoals behind the exercise. Assessment may depend on whether the exercise is objective(i.e. students are given a list of words to use in a cloze) or subjective (i.e. students are tofill in a cloze with words that would make a given sentence grammatically correct).
“ I saw a man lay his jacket on a puddle for a woman crossing the street. I thought that was very ______. ” Given the above passage, students answers may then vary depending on theirvocabulary skills and their personal opinions. However, the placement of the blank at theend of the sentence restricts the possible words that may complete the sentence;following an adverb and finishing the sentence, the word is most likely an adjective.Romantic, chivalrous or gallant may, for example, occupy the blank, as wellas foolish or cheesy. Using those answers, a teacher may ask students to reflect on theopinions drawn from the given cloze. In addition to use in testing, cloze deletion can be used in learning, particularlylanguage learning, but also learning facts. This may be done manually – for example, bycovering sections of a text with paper, or highlighting sections of text with a highlighter,than covering the line with a colored ruler in the complementary color (say, red ruler forgreen highlighter), so the highlighted text disappears; this is popular in Japan, forinstance. Cloze deletion can also be used as part of spaced repetition software, andthe Super Memo application features semi-automated creation of cloze tests, particularlyas part of its incremental reading feature.Related Studies The Cloze test is used primarily to determine a student‘s level of reading ability.A student is first tested for reading level; in this case the Slosson oral has previously beenadministered. After a student‘s reading level has been determined, a reading selection is
taken from a book of like grade level. As is standard, every fifth word is removed andreplaced by a blank space, in this assignment there were 20 blanks in the selection. In thisassignment, as was suggested, a paragraph of text preceded the actual test material.Evaluation of the test determines a student‘s reading level; an independent reading levelindicates that the student could read the selected book on their own, instructional levelmeans that the student can read the book with assistance and frustration level means thatthe book is difficult for the student to read and is likely above the student‘s reading gradelevel. For the purposes of this assignment, the following scale was assigned forevaluation: the independent reading level is based as 50% or more of the blanks filled incorrectly, instructional level is 30% - 50% and frustration level is rated as 30% or less ofthe blanks correctly filled in. On Friday, October 6, 1995, I administered a Cloze test to a student namedNatalie at Stivers Middle School in Dayton, who I had administered a Slosson oral test tothe previous week. On the Slosson oral, Natalie scored in the 8th grade reading level(8.95 to be exact). Taking her Slosson score, I set out in search of a text book involvingAfrican American studies, an area she is very interested in. After much searching, Isecured the book African American History at the Wright State ERC and submitted it toFry Graph analysis (attachment 1). Taking three 100 word samples randomly from thebook I computed the approximate reading grade level of the book. Sample 1 consisted of7.3 sentences and 149 syllables, sample 2 had 9.2 sentences and 155 syllables and sample3 had 8.5 sentences and 158 syllables. The average of the samples was 8.33 sentences and154 syllables. I plotted this point on the graph, and fortunately it fell within the 8th gradereading level. I then constructed a Cloze test to be administered.
The test was administered to Natalie during the 4th period in the same conference roomMrs. King had selected for me to give Natalie the Slosson oral test the previous week. Iexplained the test to Natalie and gave her the test to fill out (attachment 2). While Nataliewas taking the test, I either sat down at a nearby table or walked to a nearbywindow. Natalie finished the test in about eight minutes, and I quickly compared it to mymaster copy (attachment 3). I then sat down with Natalie and asked her why she hadchosen each of her selections, the correct and incorrect ones alike. I recorded herresponses, and thanked her for helping me and she returned to class.Natalie correctly filled in 11 of the 20 blanks (55%) and placed in the independentreading level. Below is a chart of error, indicating the correct response, the responsegiven, the reason the answer was given, if there was a change in grammar and if therewas a change in the meaning of the statement. Following the chart will be a more detailedaccount of why each incorrect answer was given. I asked Natalie why she chose each of her selections, and asked her if otherwords would make sense in the context of the sentence. I did this for all 20 selections;including the ones she got right as well as the ones she got wrong. For example onselection 1, which she filled in correctly, I asked her if ‗is‘ would have worked. Shepromptly told me that everything was in the past, so ‗is‘ would be wrong. In her first mistake on the test, Natalie substituted ‗food‘ for the word‗literature‘. I asked Natalie if the word ‗literature‘ would fit better, and Natalie told methat most likely books would be together with art. On the second mistake, Natalie usedthe phrase ‗brought on‘ instead of ‗stimulated‘. I asked her about using ‗stimulated‘ and I
was promptly asked what the word meant. The third mistake made was the substitution ofthe phrase ‗in many‘ instead of ‗of‘. When I asked Natalie about this, she read thesentence back with ‗of‘ in its place and said it didn‘t sound right. On this I really had toagree. The fourth mistake made was the usage of ‗racism‘ instead of ‗oppression‘.When I asked Natalie about this I was met with the same response I got for ‗stimulated‘:what does it mean? Natalie‘s next mistake came when she used ‗art‘ instead of ‗society‘in the selection. Again I asked her about using society in the sentence, and Natalie said itseemed right to her since art had been used before in the paragraph. Natalie‘s sixth error was in putting ‗something‘ in the selection instead of‗things‘. When I asked her about the possibility of using ‗things‘, she simply told me thatnobody used that word that way. Again, I would have to agree with her since I have heardfew people ever say ‗things new‘. The seventh mistake was the use of ‗movement‘instead of ‗Renaissance‘. As I had done several times before, I asked her about her choiceand asked if ‗Renaissance‘ would have worked. Natalie‘s response was that it wouldn‘twork because a renaissance is something really old. I could understand this reasoning,since the only time students really hear this term is in early European history. The eighth mistake made on the Cloze test was the use of the word ‗racist‘instead of ‗this‘. I really didn‘t understand this substitution, and when I asked about itNatalie said that she couldn‘t think of anything else, that she had used ‗racist‘ before inthe selection and ‗racist‘ was the only thing she could associate with New York in thecontext of the sentence. The ninth, and final, mistake Natalie made was the use of the
phrase ‗period for‘ instead of ‗by‘. By this time Natalie seemed tired of all this andsimply said she couldn‘t think of any other word to use. She wanted to return to her class,so I thanked her for putting up with me and my tests and let her return to the room. From my perspective, most of Natalie‘s errors on the Cloze test could beattributed to two things: she was unfamiliar with some of the vocabulary used and thetext in some places was written in a way in which few students (or adults for that matter)speak. So, what can be done about it? If this had been an actual textbook assignment, onething that could be done is to provide the students with a list of vocabulary words they donot understand completely.
Chapter III Research Methodology This chapter describes how the study was conducted and the researcher used in gatheringthe necessary data for the completion of the study. This includes the research design, thepopulation and sampling, sampling procedure, data gathering procedure, research instrument andthe statistical treatment of data.Research Design This study used the quasi - experimental type of research in determining theeffectiveness of Cloze Test in measuring History as a subject in secondary level since theabsence of a control group. The possible solutions obtained through a questionnaire aswell as interviews and observation made by the researcher. This study used of the thirdyear students in Laguna State Polytechnic University as its respondents, the main sourceof data. They were selected by random order among the total number of junior level. Theinstrument used was a collaboration of information based on Crystal,David (2004). Contextual Constraints in Cloze Test. Cambridge: Cambridge UniversityPress. ISBN 0-521-53032-6. Some of its modifications were undertaken to adapt on theresearch problem and scope of the study.Population and Sampling This study was conducted at Laguna State Polytechnic University, San Pablo CityCampus Academic Year 2012-2013. The researcher gets a representative for 3rd year highschool department to suit the twenty - nine (29) respondents needed for the study. A purposive sampling as a sampling technique was utilized.
Sampling Procedure To determine the sample size needed for the study, the researcher used randomsampling technique. This sampling technique is unbiased since the respondent has thesame chance of being chosen and the selection of respondent has no influence of theselection of other respondents.Research Instrument The main instrument used in the study is a questionnaire which was used to knowperson related factors such as age, gender, and grade point in history during secondsemester A.Y. 2012- 2013. And an integrative assessment in History comprises of twenty– five (25) items using Rational Cloze Test procedure.Statistical Treatment of Data The data were gathered from the respondents treated and interpreted using thestatistical tools. The following statistical analysis was coded in the study:Survey Questionnaire Age Legend: 14y/o below – 1 15y/o – 2 16y/o and above – 3
Gender Legend: Male – 1 Female – 2 Current Grade Point in History Legend: 93 and above – 1 90 – 92 – 2 87 – 89 – 3 84 – 86 – 4 81 – 83 – 5 80 and below – 6 Cloze test Legend: correct – 1 incorrect - 0 Percentage distribution was utilized to describe the person related variables interms of, age, gender, and grade point in history. For the assessment – related factors. The researcher utilized the mean andstandard deviation using a Likert scale, 5 being the highest, denotes ‗strongly agree‘ and1 being the lowest, denotes ‗strongly disagree‘.
To find that if there is a relationship existing between the independent anddependent variables, Pearson Product Moment of Correlation Coefficient was used to tellhow well two sets of continuous data correlate to each other at 0.05 level of significance.
Chapter IV Presentation, Analysis, and Interpretation of Data This chapter presents the findings of the study in illustrative tables and analysis aswell as the interpretation based from the treatment of the data. Table 1 Distribution of Respondents According to Age Age Frequency Percent 14 y/o below 8 28 15y/o 20 69 16y/o and above 1 3 TOTAL 29 100Table 1 shows the distribution of respondents according to their age, the majority of therespondents are 15y/o having twenty (20) or 69% of the total respondents.This implies though the inability to randomize the selection because the research has lack offeatures of the true experiment and based on the experimental values the respondents areequipped subject for experimental condition since they undergoneHistory as subject. As Datta (2002) mentioned Cloze tests require the ability to understandcontext and vocabulary in order to identify the correct words or type of words that belong in thedeleted passages of a text. This exercise is commonly administered for the assessment of nativeand second language complex learning and instruction. Table 2 Distribution of Respondents According to Gender Gender Frequency Percent Male 12 41.38 Female 17 56.62 TOTAL 29 100Table 2 clearly shows that female outnumbered the male having a frequency of 17 or 56.62%and the male has 12 frequency or 41.38%.This implies that more female are subjected in the experimental condition. Table 3 Distribution of Respondents According to Grade Point in History Grade point Frequency Percent 93 and above 0 0 90 – 92 6 21 87 – 89 10 34
84 – 86 8 28 81 – 83 5 17 80 and below 0 0 TOTAL 29 100Table 3 shows the grade point average in History of the respondents. The highest grade pointaverage that the respondents attained ranged from 87 – 89 having a frequency of ten (10) or34% second ranging from 84 – 86 having a frequency of 8 (eight) or 28% followed by 90 – 92having six (6) or 21% subsequent to that is ranged from 81 – 83 and the grade point average of93 and above and 80 below has zero frequency. Table 4 Respondents’ Perception on Controllability as a Factor of Integrative Assessment Standard Indicators Mean Remarks Deviation 1. The Cloze test procedure can easily adjust Strongly 4.55 0.56 the instructional planning. Agree 2. It can limit the learners the extension of Strongly 4.72 0.58 different vocabularies. Agree 3. Can easily manipulate by the teachers and Strongly 3.72 0.83 learners as well. Agree 4. Placement of the blank at the end of the Strongly sentence in the passage restricts the possible 4.59 0.56 Agree words that may complete the sentence. 5. The instructors can readily assume the 4.0 0.74 Agree fairness in the interpretation results. Strongly TOTAL 4.32 0.65 AgreeLegend:4.21 – 5.0 : Strongly Agree3.41 – 4.20 : Agree2.61 – 3.40 : Fair1.81 – 2.60 : Disagree1.0 – 1.80 : Strongly DisagreeHowever, Alderson (1980, 1983 cited in Ahluwalia, 1992:86) and Yamashita (2003:286-287)argue that cloze test performance is not directly related to the difficulty level of the text; itinvolves other factors such as scoring procedures, and content familiarity for the readers. Table 5 Respondents’ Perception on Reliability and Validity as a Factor of Integrative Assessment Standard Indicators Mean Remarks Deviation 1. Cloze test have adequacy and Strongly 4.55 0.56 representativeness of learning outcomes. Agree
2. The passage measures what it purports to 4.14 0.59 Agree measure. 3. Cloze test must first consider the errors of Strongly 4.31 0.70 measurement. Agree 4. It can easily interpret the test results to Strongly 4.76 0.73 improve teaching and learning. Agree 5. It has a value of testing as a tool to improve Strongly 4.90 0.55 instruction and learning performance. Agree Strongly TOTAL 4.53 0.63 AgreeLegend:4.21 – 5.0 : Strongly Agree3.41 – 4.20 : Agree2.61 – 3.40 : Fair1.81 – 2.60 : Disagree1.0 – 1.80 : Strongly Disagree . Much research has been devoted to the validity and the reliability of Cloze tests.Cloze tests are reported to have moderate to high correlations with standardized tests andtheir subtests such as listening comprehension and reading comprehension. The report onhigh correlations between Cloze tests and International Journal of Human and SocialSciences tests of listening comprehension, writing, reading comprehension, and oralinterview. The findings of these researchers reveal a relation between the scores on Clozeand global language ability tests (Greene, 2001). Table 6 Respondents’ Perception on Usability as a Factor of Integrative Assessment Standard Indicators Mean Remarks Deviation 1. It manifests strong retention constructed – 3.48 1.04 Agree response type. 2. Creates critical reasoning for learners on Strongly 4.62 0.89 how to arrive on precise conclusions. Agree 3. Learners can employ to assess vocabulary, Strongly 4.48 0.56 understanding facts, seeing, relationships, Agree
drawing inferences, detecting author‘s literary style, and approximating the patterns of language structure. 4. The test provides communicative skills in Strongly understanding such areas of learning such as 4.55 0.97 Agree in the history subject. 5. Contributes logical argumentation and empirical evidence for learners such as Strongly 4.46 0.62 resourceful in resolving authentic problems Agree in studying history. Strongly TOTAL 4.32 0.82 AgreeLegend:4.21 – 5.0 : Strongly Agree3.41 – 4.20 : Agree2.61 – 3.40 : Fair1.81 – 2.60 : Disagree1.0 – 1.80 : Strongly Disagree A sentence or a passage, deleted some words, is just like a broken picture. Peoplewill complete it unconsciously according to their comprehensive knowledge. The Gestalttheory also claims that the more familiar with the picture the easier for people torecognize it. Expounded in language learning, the case is that the higher languageproficiency the higher ability to complete the broken passage and promotes higher orderthinking skills but also more on life – relating tendencies which could be applicable tofurther level of a child. Table 7 Distribution of Result in Rational Cloze Test Item no. Frequency Percent Mean Interpretation 1 13 52 0.45 Satisfactory 2 13 52 0.45 Satisfactory 3 8 32 0.27 Fair 4 9 36 0.31 Fair 5 13 52 0.45 Satisfactory 6 12 48 0.44 Fair 7 2 8 0.06 Needs improvement
8 12 48 0.17 Fair 9 5 20 0.51 Needs improvement 10 15 60 0.51 Satisfactory 11 15 60 0.37 Satisfactory 12 11 44 0.06 Fair 13 2 8 0.34 Needs improvement 14 10 40 0.04 Fair 15 12 48 0.27 Fair 16 8 32 0.41 Needs improvement 17 12 48 0.17 Fair 18 0 0 0 Needs improvement 19 5 20 0.51 Needs improvement 20 0 0 0 Needs improvement 21 8 32 0.27 Fair 22 11 44 0.06 Fair 23 6 24 0.22 Needs improvement 24 6 24 0.22 Needs improvement 25 13 52 0.45 Satisfactory TOTAL 221 100 0.28 FairLegend:24 – 25 : Outstanding20 – 23 : Very Satisfactory13 – 19 : Satisfactory8 – 12 :Fair0 – 7 : Needs ImprovementThis table shows the result of the cloze test of 3rd year high school, as we observed item number10 and 11 has a high frequency which interpreted as ‘Satisfactory’ based on the scale valuesabove having a 60% of the total correct answer. As we look generally there are more resultsranged from 8 - 12 thus the mean score of the respondents based on the experiment which isinterpreted as ‘Fair’.This implies that the respondents performed the test even-handedly since the totalinterpretation is said to be ‘Fair’ with regards to this Legenza (2001) reasoned behind giving aCloze test and obtaining a test score is interpreting that score as an indicator of what a testtaker knows or what he/she can do with that knowledge. Furthermore, our interpretation ofthat test score forms the basis for decision making. As such, when using a test score, we makean implicit link between test performance and a domain of knowledge the test taker has orsomething the test taker can do with schema in some area use domain beyond the test itself. Table 8 Correlation between the Person – related factors and Assessment Variables Variables r - value p - value Interpretation Age .209 .0320 Not Significant Gender .227 .0411 Not Significant
Grade Point Ave. .124 .0309 Not SignificantLegend:p >.05 : not significantp <.05 : significantThe advantages of this cloze variant are convenience and high consistency, while there are alsoa lot of problems such as some of the blanks being too easy or too difficult, the broken passagesometimes arousing test-takers’ dislike and resulting in a dilemma for testers to choosesubjective or objective scoring method, and being lack of control which possibly leads toexamine the language points the designer doesn’t want to test actually. Rational cloze can alsobe called open cloze or gap-filling. Table 9 Correlation between Assessment – related variables and Cloze Test Variables r - value p - value Interpretation Controllability -.789 .0776 Not Significant Reliability and .076 .0467 Not Significant Validity Usability .098 .0202 Not SignificantLegend:p >.05 : not significantp <.05 : significantThis procedure is intended to sample regularly various variants of words, some of whichare governed by local grammatical constraints and wide discipline vocabularies in sortedareas of History, Mathematics, Statistics, others of which are governed by long-rangetextual constraints (Datta, 2002). Table 9 Correlation between Grade Point Average in History and Cloze Test Variable r - value p - value Interpretation Grade Point Ave. -.565 .0676 Not SignificantLegend:p >.05 : not significantp <.05 : significant
Askes (2002) regards cloze procedure as one of the integrative tests (global tests)that integrate language components into a total language event, which requires anintegrated performance from the learner in a meaningful context. Thus for Steinman(2002), a cloze can replace the sections on test structure, written expression, vocabulary,and reading comprehension not only in English subject as well as different areas oflearning.The reason behind giving a Cloze test and obtaining a test score is interpreting that scoreas an indicator of what a test taker knows or what he/she can do with that knowledge.Furthermore, our interpretation of that test score forms the basis for decision making. Assuch, when using a test score, we make an implicit link between test performance and adomain of language knowledge the test taker has or something the test taker can do withlanguage in some language use domain beyond the test itself. In other words, when weuse test scores, we are essentially reasoning from evidence, using the test score as theevidence for inferences or interpretations and decisions we want to make. Yet, we cannotsimply draw on test score to make inferences and decisions without efficient justification.If we want to use a test score for a particular purpose, we must justify it through arationale and supporting evidence.
Chapter V Summary, Conclusions, And Recommendations This chapter presents the summary of findings, conclusions, andrecommendations related to the effectiveness of Cloze test in assessing History subject.
Appendix A Laguna State Polytechnic University San Pablo City Campus San Pablo City LETTER TO THE DIRECTOR OF LABORATORY HIGH SCHOOLDate:Nelia T. Salvador, Ed.D.Laguna State Polytechnic UniversitySan Pablo City Campus, San Pablo CityGreetings of the day! I am fourth year student of Laguna State Polytechnic University San Pablo CityCampus, taking up Bachelor of Secondary Education, major in Social Studies. I ampresently conducting a study entitled ―Effectiveness of Cloze Test in History at LagunaState Polytechnic University San Pablo City Campus A.Y 2012-2013. I would like to ask permission from your good office to evaluate teacher‘sperformance by administering a researcher-made test to twenty - nine (29) third year highschool students of LSPU-SPCC. It is in this juncture that I express my warmest gratitude. Should my request begranted. Thank you very much.Truly yours,MARILYN A. MENDOZA Researcher
Appendix B Republic of the Philippines LAGUNA STATE POLYTECHNIC UNIVERSITY San Pablo City Campus San Pablo City COLLEGE OF TEACHER EDUCATION LETTER TO THE RESPONDENTDate:Dear Respondents,This is in reference to my study of the requirements for the course Methods of Researchentitled ―Effectiveness of Cloze Test in History at Laguna State Polytechnic University‖.The data that will be gathered from you will be extremely useful for the successfulrealization of my research.It is in this juncture that I express my warmest gratitude. And will be treated with strictestconfidentiality. Thank you very much.Very truly yours,Marilyn A. MendozaResearcher
Appendix C Republic of the Philippines LAGUNA STATE POLYTECHNIC UNIVERSITY San Pablo Campus Del Remedio, San Pablo City COLLEGE OF TEACHER EDUCATION BACHELOR OF SECONDARY EDUCATION Survey QuestionnaireI. Profile of the RespondentsDirection: Please fill in the blanks with your personal informationName (optional):____________Age:__________Gender: male ( ) female ( )Grade point in History subject 1st semester 2012 – 2013:____________II. Perception of the respondents ABOUT EFFECTIVENESS OF CLOZE – TEST AS INTEGRATIVE ASSESSMENT IN HISTORY.Direction: Please indicate your responses on the following statements by putting a check(√) using the scale below as follows: 5 – Strongly Agree 4 – Agree 3 – Fair 2 – Disagree 1 – Strongly Disagree
A. Controllability 1 2 3 4 5 indicators 1. The Cloze test procedure can easily adjust the instructional planning. 2. It can limit the learners the extension of different vocabularies. 3. Can easily manipulate by the teachers and learners as well. 4. Placement of the blank at the end of the sentence in the passage restricts the possible words that may complete the sentence. 5. The instructors can readily assume the fairness in the interpretation results.B. Reliability & Validity 1 2 3 4 5 indicators 1. Cloze test have adequacy and representativeness of learning outcomes. 2. The passage measures what it purports to measure. 3. Cloze test must first consider the errors of measurement. 4. It can easily interpret the test results to improve teaching and learning. 5. It has a value of testing as a tool to improve instruction and learning performance.C. Usability 1 2 3 4 5 indicators 1. It manifests strong retention constructed – response type. 2. Creates critical reasoning for learners on how to arrive on precise conclusions. 3. Learners can employ to assess vocabulary, understanding facts, seeing, relationships, drawing inferences, detecting author‘s literary style, and approximating the patterns of language structure. 4. The test provides communicative skills in understanding such areas of learning such as in the history subject. 5. Contributes logical argumentation and empirical evidence for learners such as resourceful in resolving authentic problems in studying history.
III. Cloze – test (adopted from Crystal David’s educational research: Cambridge 2004) Directions: Read the passage and hint or predict the word that was removed andreplace it with the word that best fits the intended meaning in a passage. (Time allotmentis 25minutes). The cat has a 1.__________ as fascinating and mysterious as the creatureitself. The true beginnings of the domestic cat are unknown, but the cat may have firstappeared around3000 B.C. in a 2.__________ called Nubia, which bordered Egypt. By2500 B.C., the cat was domesticated in Egypt. The cats first 3.__________ in Egypt wasMau. The Mau‘s 4.__________ in Egypt grew rapidly; she was eventually consideredguardian of the temple and was worshipped as a goddess. Besides being worshipped asgoddesses, cats also had a practical 5.__________: they kept 6.__________ fromoverrunning the Egyptian grain store-houses. The Greeks were probably the first 7.__________ to recognize cats for theirmouse- catching talents. When International Journal of Human and Social Sciences 5:92010556Egyptians refused to sell or trade any of their cats, the Greeks 8.__________several of the Egyptian cats and sold the 9.__________ of these stolen cats to Romans.The cat became the 10.__________ of liberty in ancient Rome. By the end of theeleventh 11.__________ cats were popular among sailors because of their rat-catchingskills. Sailors admired cats because they 12__________ disease-infested rats which livedon ships. Many sailors believed that cats possessed special powers that could13.__________ them at sea. Although the cat was held in high regard and fancied during 14.__________times, the cat didnt fare will in Europe in the Middle Ages. Cats were associated withevil, witchcraft, and black 15.__________. Many people believed that 16.__________regularly transformed themselves into cats. Men and women were killed for helping a17.__________ or injured cat. During the witch-hunts in Europe many innocent peoplewere accused of witchcraft simply because they owned cats. Black cats were especiallyfeared. Some legends and 18.__________ about cats exist today, like that about the nine
lives of cats. Another legend that survived from Europes Middle Ages into the presentstates that a black cat crossing ones path brings bad 19.__________. Today the elegant, graceful cat has become a popular house 20.__________throughout the 21.__________. The cat is one of the smartest of tame animals, but theyare independent and harder to train. Cats are valued for their gentle, affectionate natures.They have 22.__________ memories; they 23.__________ who treats them well and whotreats them badly. A cats loyalty is earned; a cat wont stay where it is 24.__________.They respond to loving owners with loyalty, affection, and respect. Cats are noted fortheir keen senses: their sharp hearing, sense of smell, and ability to 25.__________ innear darkness. Perhaps Leonardo Da Vinci summed it up best when he referred to the catas ―Nature‘s Masterpiece.‖
Appendix DThe Cloze test answer key: 1. history 2. country 3. name 4. status 5. function 6. mice 7. Europeans 8. stole 9. kittens 10. symbol 11. century 12. destroyed 13. protect 14. ancient 15. magic 16. witches 17. sick 18. superstitions 19. luck 20. pet 21. world 22. good 23. remember 24. mistreated 25. see
BIBLIOGRAPHYAhluwalia, N. (1992). Major Issues in the Cloze Procedure. Indian Journal of Applied Linguistics. (Volume No is not available):81-96.Askes, H. (1991). Some Techniques in Communicative Second Language Testing. Journal for Language Teaching. Vol.25.1:36-63.Bai, J. (2004). An Analysis of Methods for Facilitating the Completion of Cloze (Chinese). English Coaching Paper (for Senior Grade Three).Vol.2004/01/26.Carrell, P. L. & Eisterhold, J. C. (1988). Schema Theory and ESL Reading Pedagogy. In Carrell, P. L., Devine, J. & Eskey, D. (eds.) Interactive Approaches to Second Language Reading. Cambridge: Cambridge University Press. 73-89. 92CET public website. (2004). College English Test. [Online]. Available: http://www. cet.edu.cn [Accessed on 2004, August 19].Chan, D. & Mok, K. (2001). Educational Reforms and Coping Strategies under the Tidal Wave of Marketisation: a Comparative Study of Hong Kong and the Mainland. Comparative Education. Vol.37.1: 21-41.Cheng, K. (1999). Young Adults in a Changing Socialist Society: Post-compulsory Education in China. Comparative Education. Vol.30.1: 63-73.Clarke, M. A. & Silberstein, S. (2001). Toward a Realization of Psycholinguistic Principles in the ESL Reading Class. In Long, M. H. & Richards, J. C. (eds.) Methodology in TESOL. New York: Newbury House. 233-247.Datta, S. & Macdonald-Ross, M. (2002). Reading Skills and Reading Habits: a Study of New Open University Undergraduate Reservees. Open Learning. Vol.7.1: 69-88.Dreyer, C. & Nel, C. (2003). Teaching Reading Strategies and Reading Comprehen- sion within a Technology-enhanced Learning Environment. System. Vol.31. 3: 349-366. 93Du, H. (2001). The globalisation of the English language: Reflections on the Teaching of English in China. International Education Journal Educational Research Conference 2001 Special Issue. Vol.2.4: 126-133.Ellis, R. (1994). The Study of Second Language Acquisition. Oxford: Oxford University Press.Greene, B. B. (2001). Testing Reading Comprehension of Theoretical Discourse with Cloze. Journal of Research in Reading. Vol.24.1: 82-98.
Groebel, L. (1981). Reading: The Students’ Approach as Compared to Their Teachers’ Recommended Method. English Language Teaching Journal. Vol. XXXV.3: 282- 287.Helfeldt, J. P., Henk, W. A. & Fotos, A. (1986). A Test of Alternative Cloze Test Formats at the Six-Grade Level. Journal of Educational Research. Vol.79.4: 216-221.Legenza, A. & Elijiah, D. (2001). The cloze procedure: Some New Applications. The Journal of Educational Research. Vol.72.6: 351-355.