Afl research proposal


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Afl research proposal

  1. 1. ASSESSMENT FOR LEARNING 1Assessment for Learning: What Assessment Methods Do Teachers of Mathematics Use and Why Yulia Shadyrya University of Washington Bothell
  2. 2. ASSESSMENT FOR LEARNING 2Research Topic In her book What’s Math Got to Do With It?: Helping Children Learn to Love TheirLeast Favorite Subject, Jo Boaler describes the gloomy state of mathematics in the United States.Our country’s ranking in math, when compared to thirty-nine other countries around the world, isas low as twenty-eighth (and plummets down to fortieth if we take into account the amount ofspending on education). The number of students graduating with a major in mathematics at four-year colleges has decreased by nineteen percent over the last ten years and almost half of allmaster’s degrees and more than a third of all bachelor’s degrees in engineering, mathematics,and information science are attained by nonresident aliens (2008). Among the main reasons forsuch depressing data, the author points out, is the fact that American schools use standardizedhigh stakes tests and other forms of summative assessment, which is a form of assessment oflearning. Wikipedia’s definition of the assessment of learning is: assessment that is accompaniedby a number or letter grade, assessment that compares one student’s achievement with standards,and assessment that occurs at the end of the learning unit. Boaler suggests that Americanteachers need to learn to practice assessment for learning. The concept of the assessment forlearning ―is based upon the principle that student should have a full and clear sense of what theyare learning, of where they are in the path toward mastery, and of what they have to do tobecome successful‖ (p. 98). In related literature the term assessment for learning is often used interchangeably withthe term formative assessment, which might lead readers to believe that they are the same thing.However, according to Richard Stiggins, the founder of the Assessment Training Institute ofPearson Education, Inc. and the author of multiple publications on assessment, assessment forlearning is not limited to formative assessment. In addition to frequent testing and revising their
  3. 3. ASSESSMENT FOR LEARNING 3teaching styles based on testing information, teachers must involve students in the process ofassessment. Stiggins recommends a list of actions to teachers, who want to use the assessment toadvance student learning. These actions include: explaining achievement targets to students,educating themselves on accurate uses of assessment and on using it to strengthen students’confidence as learners, translating assessment into frequent feedback, adjusting instruction basedon assessment, engaging students in self-assessment, and teaching students to communicate withthem and with family members regarding their learning progress (2002). Dylan William, thedeputy director of the Institute of Education in London, and education researcher of many years,who specifically focuses on formative assessment research wrote about very specific techniquesof assessment for learning: rich questioning, feedback, and learner’s role in assessment (1999).Rich questioning refers to questioning technique that requires students to analyze problem,instead of just guessing an answer which teacher will like. Mathematical problems that confusestudents are usually avoided by teachers. However ―tricky‖ questions are a valuable part of richquestioning because they provoke deep thinking, and help teachers in assessing their pupils’ trueunderstanding of math. Other techniques of rich questioning include: allowing for longconversations to grow from one student’s question, giving ample time for students to think abouta question, asking students to come up with questions, and framing questions as statements toprovoke discussions. Feedback given to students can take different forms. Based on clearresearch, William suggests letting students know what they need to do to improve, instead ofpraising them or comparing to others, and design feedback so it instills in students belief thatthey can improve their mathematical skills with practice. He also insists that students have animportant role in assessment for learning. Thus, sharing assessment criteria with students andletting them monitor their own progress is essential.
  4. 4. ASSESSMENT FOR LEARNING 4 In addition to the obvious benefits of assessment for learning, in my opinion a majoradvantage of assessment for learning lies in relating classroom setting to real life. In real life werarely get grades for our actions. Constructive criticism and help is more relevant and useful thangrades. To illustrate my point, I will share an example from my own work experience. Once Iworked at a firm that did not give proper training or any feedback of its employees’ performance.I was satisfied with my pay and position at that time, and did not wonder why the companypromoted several other workers before me. Luckily, one of the managers was observing my workfor several months. She had a lot of work experience and was a top performer at that company.Only through conscious observation was she able to notice and truly assess my performance.After I spent some time training with her, I was able to correct a few mistakes, and master skillsnecessary for me to be successful. It turned out that due to poor training and the absence offeedback, many new employees were underperforming, and, as a result either quit or got fired.Some employees, who were able to overcome poor reviews, became stuck at the same positions,and only a few employees moved up the corporate ladder. Needless to say, that the companymorale as well as productivity suffered because of such a poor feedback system. Today, manystudents in American schools are stuck in a situation that is not unlike the one described above.Unlike adults, who have more experience in life, students often do not try to figure out what isthe real reason behind their poor grades, and merely accept the label of being ―stupid in math.‖Literature Review The importance of assessment for learning (improved formative assessment) was shownby Paul J. Black and Dylan William in 1998. Their study is referenced by most authors whowrote about the subject of assessment for learning. Black and William examined more than 160journals, 580 articles or chapters on assessment and together with the experts from all over the
  5. 5. ASSESSMENT FOR LEARNING 5world, concluded that a lot of evidence exists to the fact that formative assessment improveslearning standards, that there is room for improvement, and that there is information available onhow to improve formative assessment. The authors illustrate their argument by stating thatimproved formative assessment could help an average student to move to the top 35% of theirclass and an average ranking country, like the United States, could rise to be in the top 5%.Moreover, the study also shows that improved formative assessment tends to help lowerachieving students more so than average or high achieving students, which has tremendousimplications for closing the achievement gap present in the United States. One of the two biggestareas where assessment needs to be improved, that the authors list is the teaching practice, wherecurrently assessment is proven ineffective and even damaging to students. The study suggeststhat in order to make a positive difference, formative assessment needs to be improved byfocusing on the following: self-esteem of pupils, which is improved by moving away fromcompetitive classroom culture and helping individual students with specific problems; self-assessment by students, which is only possible and greatly beneficial when students have a clearpicture of where their learning goals are; and the evolution of effective teaching, which involvesrevising whole teaching plan instead of just one of its aspects, which is assessment. The authors emphasize that the strategy for improving formative assessment needs tofocus on the ―inside of the black box,‖ in other words, in the classroom. Neither more traditionalresearch nor large implementation programs will help with the implementation of assessment forlearning. Rather ―each teacher must find his or her own ways of incorporating the lessons andideas set out above into his or her own patterns of classroom work and into the cultural normsand expectations of a particular school community‖ (1998, p. 12,). Specific steps includelearning from development, which means having plenty of real life examples for teachers on how
  6. 6. ASSESSMENT FOR LEARNING 6to implement assessment for learning, such examples must be specific to subject areas; andpractitioner research, one example of which is investigating ways teachers are dealing withbalancing formative and summative assessment in their classrooms. Moreover, in 1999 Williampublishes three studies that describe specific strategies that can be used by teachers in theirclassrooms. These studies include rich questioning, feedback, and the learner’s role inassessment (for more description see introduction). Beyond these recommendations, the authorprovides evidence gathered from research. For example, one of the articles references anexperiment done by Ruth Butler, where 200 fifth- and sixth-graders were given four kinds ofassessment: comments, grades, praise, and no feedback. This study showed that the interest aswell as performance of students increased after comments, and stayed the same after grades,praise, or no feedback (1987). More current literature suggests that the problem of inadequate assessment still exists inthe United States. Richard Stiggins conveys the urgency of assessment reform in the U.S.A.,which would balance assessment of learning with assessment for learning (2002). He describesthe attitude of our nation toward assessment as: ―obsessed with the belief that the path to schoolimprovement is paved with better, more frequent and more intense standardized testing‖ (p. 759).As a result, he states, there are no resources left to support teachers in ongoing classroomassessment. Only a small portion of states require teachers to be skilled in assessment, andalmost no states require that of school administration. Stiggins argues that instead of constantlyincreasing challenge in attempt to motivate students, the country should balance and improveassessments of and for learning. He lists relevant position statements from such groups asAmerican Foundation of Teachers, National Education Association, National Council onMeasurement in Education, American Association of School Administrators, and others.
  7. 7. ASSESSMENT FOR LEARNING 7Popham, W. explains that No child Left Behind (NCLB) and its measurement system ofadequate yearly progress (AYP) pressures teachers to teach to the test, and consequently get inthe way of attempts to instill assessment for learning. The author insists that states needs todevelop instructionally sensitive tests, which measure teaching as well as learning (2006). Itmust be noted that although the two sources listed above agree with the urgency of the situation,they both are focused on the strategies on large scale, which is contrary to the recommendationsof Black and William. Finally, as described in introduction, the subject of mathematics needs specialconsideration when making changes in assessment strategies. Jo Boaler describes that mostmathematics students, who are being evaluated in traditional ways, cannot even describe whatthey are learning beyond the names of the chapters in textbook or problem numbers. Those whosuffer the most from the standardized testing are usually children from low income families andstudents who are English language learners. Moreover, students who are low achievers inschools are not being helped, but are instead merely labeled as ―below average,‖ which furtherdiscourages them from learning mathematics (2008). My work experience taught me that havinga mentor skilled in assessment for learning is invaluable. Teachers, who have the opportunity toobserve their students directly and often, are at a perfect position to become such mentors andempower students to take learning in their own hands.Research Questions Here are some aspects connected to assessment that I would like to investigate in the processof my research: 1) What are the current techniques of assessing students’ knowledge ofmathematics? Are the elements of assessment for learning present? 2) What factors do
  8. 8. ASSESSMENT FOR LEARNING 8mathematics teachers in a particular school consider when developing assessment plans? According to the section above on literature review, American students are suffering from theoverwhelming presence of judgmental feedback in the form of grades, and would instead benefitfrom the presence of meaningful assessment—assessment that could help them take charge oftheir learning. Clearly, mathematics is a subject in need of special attention. There is a lot ofinformation about how to improve formative assessment.. Here are some ways my researchmight contribute to theoretical development of the concept of the assessment for learning: simply raising such important questions might make teachers re-think their assessmenttechniques or even their personal teaching philosophies data gathered in the process can be used in further research of what elements of theformative assessment American teachers currently practice (if at all), and to compare it to similardata collected in the past in order to measure the progress(or lack thereof) of the formativeassessment in American classrooms or in a particular school information about different types of assessment can be used in future practitionerresearch projects. One such project could evaluate the effect of different types of assessment onstudent learning and self-esteem in a particular school, which in turn might influence schoolpolicies to support teachers with assessment for learning increase in the assessment for learning in a particular school and whole America, willlead to overall increase in student learning and to decreasing the achievement gap between lowand high achieving students Considering the recommendation of Black and William, it is clear that practitioner research iscrucial in implementing assessment for learning in schools. Therefore these research questions
  9. 9. ASSESSMENT FOR LEARNING 9are aimed at investigating assessment situations in one particular school by either a teacher orsomeone else, who is able to work with teachers regularly. The combined analysis of bothquestions should help mathematics teachers of this particular school in their daily practice,because it will allow them to see what the overall assessment strategy of the mathematicsdepartment is as a whole, and it will allow them to see what some of the most importantconstrains are as perceived by teachers and the administration. It will also help each teacher totake an objective look at his or her own assessment approaches and to judge whether theseassessment approaches assist student learning. Separate teachers, as well as the wholemathematics department, will be able to revise assessment strategies if necessary, whether it willbe by adding more elements of the assessment for learning, or by reconstructing the wholeapproach based on the most important constrains. Having information about constrains can alsohelp with releasing potentially damaging findings, according to Robinson and Lai (2006). Research Setting Design This research project is designed to be conducted by a group of student-teachers in theschool of their placement. Each student-teacher will primarily collect information on theircooperating teacher, but in general research, planning, and analysis will be done by these studentteachers as a group. Prior to initiating research, student-teachers will have a discussion meetingwith teachers in order to explain the general purpose of this research. Student-teachers will bevery explicit about what they will do. A question might arise as to whether it is appropriate to tellteachers that this research focuses on assessment for learning, as it might influence the wayteachers normally evaluate their student. However, in my opinion, one of the great advantages ofpractitioner research is that it is able to improve teaching practices in its process. Even if teachers
  10. 10. ASSESSMENT FOR LEARNING 10will start to include elements of assessment for learning, objective descriptions of theirassessment styles with these alterations will still be beneficial, because it is not as important forus to know past actions of teachers as it is to promote assessment for learning techniques.Another concern is that teachers might feel like they are being judged by researchers if theyreveal their intention of finding out whether teachers’ assessment techniques are assessment forlearning or not. Explaining that part of research is intended to discover constrains which shapeassessment strategies, might help in relieving this tension. The student-teacher position fits very well with this type of research, because a lot ofinitial responsibility of student-teachers is to observe their cooperating teachers. In addition,student-teachers will be working in close contact with cooperating teachers and with all thestudents, which will give student-teachers better understanding of the set of assessment andrelated constrains teachers face. On the other hand, because student-teachers will be new to thesetting, they might have less bias in their assessment of the situation. Putting this research in thehands of student-teachers is also practical: every student-teacher will have only one cooperatingteacher to follow, which is very helpful with keeping the load of work under control. Consent forms will need to be signed by parents or guardians of all student participants.Final and intermediate results will be open to all participating teachers. Any explanations andcorrections will be welcome. In case the research will be published, pseudonyms will be used forschools and all participants. Preparations for this research will take about two months, includingwriting the research proposal, talking to teachers, and collecting consent forms. Therefore,according to 40-20-40 rule described by Robinson and Lai(2006), the optimal time frame fordata collection should be one month, and for results – two months.
  11. 11. ASSESSMENT FOR LEARNING 11Methods for Collecting Information1) What are current techniques of assessing students’ knowledge of mathematics? Are theelements of assessment for learning present? These questions can be thought of as exploratory in nature, because student-teachers willbe investigating different assessment techniques used by mathematics teachers. However, theultimate goal of raising these questions is to encourage teachers to think of assessment assomething that should improve student learning, and, while collecting information, researcherswill look for specific indicators of assessment for learning. For this purpose we will think ofthese questions as both exploratory and checking. The process of information collection will startas exploratory, and as clear patterns of assessment develop, observation notes, questions, andother inquiries will become checking. In order to investigate these questions, student-teachers will conduct interviews with allmathematics teachers, and two or three students from each teacher’s class. Student-teachers willalso conduct observations of each participating classroom once a week throughout the quarter,and look at examples of assignments (homework, group work, tests, etc). Classroomobservations will be done two class hours a week, alternating days and classes to assure broadcoverage of collected information. One week student-teachers will swap cooperating teachers inorder to increase validity. During classroom observations student-teachers will take notes--writing down every instance teacher is giving any type of verbal feedback to students or that ofstudents giving feedback to each other whether prompted or not. Copies of various assignmentswill be made for analysis. Interviews will be conducted after classroom observations are done and assignmentsstudied, in order to ask teachers and students for any clarifications or elaborations. Interviews
  12. 12. ASSESSMENT FOR LEARNING 12will start with a general question: Please describe way in which you assess your students’understanding of mathematical concepts? If necessary, more specific questions will follow: Whatinstruments do you use to assess your student’s knowledge (tests, dialogue, projects, etc)? Howdo you give them feedback on each of these instruments (grade, comments, grade and comments,praise, credit, etc)? How often do you assess each of your students (every day, week, month,etc)? Do you use such techniques as self-assessment and peer-assessment by students? Studentswill be asked some questions that are similar, for example: How often do you receive feedbackfrom your teacher and what kind of feedback is it (grade, comment, etc)? Some questions will bequite different, for example: Did your teacher clearly communicate the learning objectives forthis week(unit)? Does your teacher explicitly tell you where you are and what you need to do toachieve class objectives? Does your teacher encourage you to discuss your learning progresswith your teacher and your family? In order to increase validity, student-teachers will switchtheir observed teachers once in two weeks, and audio- or videotape all interviews, and willtranscribe and analyze them together.2) What factors do mathematics teachers in a particular school need to take into account whendeveloping assessment plans? This question will be asked at interviews with teachers and will be also posed toadministrators. All answers will be listed, and a follow-up questionnaire will ask teachers andadministrators to assign significance value to each factor: very important, important, not veryimportant, or unimportant.Methods for Analyzing Information During analysis I will be looking for these elements of assessment for learning: presence
  13. 13. ASSESSMENT FOR LEARNING 13of rich questioning as described by Dylan William(1999); teacher commenting on students’progress without giving a grade or praise; presence and quality of self and peer assessment bystudents; quality of classroom discussions (Does teacher often answer her own questions? Arestudents allowed to discuss answers among themselves?); frequency of tests; general openness ofthe teacher to discussing students’ learning progress with the student and his or her family. Transcribed interviews with teachers will be scanned for any information on assessmenttools and classroom observation notes will be analyzed for any evidence of assessment, such as:teacher taking notes on where her students are, teacher giving evaluative comments to herstudents, students evaluating each other with and without prompt from a teacher, and any othersuch instances. This information will be organized in portfolios for each separate teacher, in aform of a time line one quarter in length. In my opinion measuring assessments against time isrelevant, because a crucial part of assessment for learning is giving students timely feedback.Each instrument of assessment will be listed on timeline with a detailed description of suchthings as what the main purpose of the assessment was (reporting to the state, teacher’s insightto students understanding, letting student know where she stands, etc), feedback to student, andso on. Transcribed interviews with students will be analyzed for two things: whether studentsare have a feel about where they are in the process of learning based on their teacher’sassessment and whether or not discussion of their learning process with their teacher and familiesis encouraged by their teacher. Depending on how clear the data will be, the data might bequantified and added to teacher’s portfolios in form of a simple table or if additional informationis discovered, descriptions of how students perceive their assessment. Together with the
  14. 14. ASSESSMENT FOR LEARNING 14information from the teacher, a picture will be formed illustrating what a particular teacher’sassessment techniques are, and which techniques can be classified as assessment for learning. One list of constraints on assessment strategies will be developed for all mathematicsteachers and administration separately. All constraints will be marked as: very important,important, not very important, unimportant. This information should help to see what are themost important constrains considered in this particular school’s math department. Constraintsperceived as important by teachers might be compared to those perceived important byadministration.Reflection The most important realization I have made while working on this research proposal ishow important it is for a teacher to think like a researcher. While I knew that a good teacher mustbe reflective, all this meant to me was that a teacher must look critically at his or her own workand must be open to change. It was difficult for me to imagine how I would measure theeffectiveness of my teaching and to know what criteria I would use to gauge myself. While Iknew that it is important for a teacher to work in collaboration with other teachers, I could notimagine the process of collaboration to be anything more than discussing student relatedproblems. Working on this project gave me an idea of how practitioner research might establish aframework for self-reflection as well as for collaboration with other teachers. When working on developing research questions, I realized that knowing what you wantto research is only one part of research preparation. The way research questions are formeddefines what the research project will have a potential to accomplish. In my case, I was veryinterested in researching something relating to assessment for learning, since I was convinced
  15. 15. ASSESSMENT FOR LEARNING 15that this approach can really influence student learning. However, when deciding on whatexactly can and should be researched, I realized how many factors needed to be considered. First,I had to do preliminary research of relevant literature and somehow classify all availableinformation deciding which to consider when developing research questions. It was veryimportant for me to imagine a situation that was possible, and for this proposal to be forpractitioner research. Therefore, I picked a realistic setting: student-teaching placement; anddesigned research considering such details as practicality, ethics, and validity. I asked myselfsuch questions as: How much time can I dedicate to this research? How can I discuss thisresearch with a cooperating teachers without offending him or her? What can I do in my positionof student-teacher? Forming research questions with all these questions in mind was,surprisingly, the most difficult part of this proposal. Searching for relevant information, I came across some great and not-so-great projects. Irealized that a valid topic, well formed research questions, and a solid research design can lead toa study that many teacher researches will find useful. Being in the position of student teachermeans having access to great data and knowing what research topics are significant. Learningabout and practicing research means being able to design a study that will bring useful results.However, in such setting researchers must be very careful about preparing, conducting, andanalyzing their research. They need to remember that they are new in the school; and beforestarting any such activity it is necessary to study school’s environment. I learned that even if allparticipants are eager to help, it is important to know that information provided by them is onlytheir perception. In my research project, I tried to establish a process of triangulation, where Ilook at one issue from the eyes of teachers, administration, students, and researchers. Finally,preparing this proposal, I imagined how much different information will be gathered in the
  16. 16. ASSESSMENT FOR LEARNING 16process, and how important it is to collect it using appropriate research techniques, which couldmake the difference between groundbreaking revelations and ―oops, never mind‖ kind ofresearchReferencesBlack, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. The Phi Delta Kappan, 80(2), 139-148. Retrieved from, J. (2008). What’s math got to do with it?: Helping children learn to love their least favorite subject –and why it’s important for America. London: Penguin Group.Butler, R. (1987). Task-involving and ego-involving properties of evaluation: Effects of different feedback conditions on motivational perceptions, interest, and performance. Journal of Educational Psychology, 79(4), 474-482. doi:10.1037/0022-0663.79.4.474.Doganay, A., & Bal, A. P. (2010). The measurement of students achievement in teaching primary school fifth year mathematics classes. Educational Sciences: Theory and Practice, 10(1), 199-215. Retrieved from ehost-livePopham, W. (2006). Assessment for Learning: An Endangered Species? Educational Leadership, 63(5), 82-3. Retrieved from
  17. 17. ASSESSMENT FOR LEARNING 17 tml?_DARGS=/hww/results/results_common.jhtml.33Stiggins, R. J. (2002). Assessment crisis: The absence of assessment FOR learning. The Phi Delta Kappan, 83(10), 758-765. Retrieved from, D. (1999) Formative assessment in mathematics. Part 1: Rich questioning. Equals: Mathematics and special educational needs, 5(2). Retrieved from, D. (1999) Formative assessment in mathematics. Part 2: Feedback. Equals: Mathematics and special educational needs, 5(3). Retrieved from epart2_8.pdfWilliam, D. (1999) Formative assessment in mathematics. Part 3: The learner’s role. Equals: Mathematics and special educational needs, 6(1). Retrieved from